Hostname: page-component-76fb5796d-vfjqv Total loading time: 0 Render date: 2024-04-25T17:26:40.527Z Has data issue: false hasContentIssue false
  • English
  • Français

Can Micro-Gestural Inflections Be Used to Improve the Soniculatory Effectiveness of Parameter Mapping Sonifications?

Published online by Cambridge University Press:  26 February 2014

David Worrall
David Worrall*
Affiliation:
Emerging Audio Research Group, International Audio Laboratories, Fraunhofer-Institut für Integrierte Schaltungen, Am Wolfsmantel 33, 91058 Erlangen, Germany
*
E-mail: david.worrall@iis.fraunhofer.de
Get access Rights & Permissions [Opens in a new window]

Abstract

Parameter mapping sonification is the most widely used technique for representing multi-dimensional data in sound. However, it is known to be unreliable when used for detecting information in some types of data. This is generally thought to be the result of the co-dependency of the psychoacoustic dimensions used in the mapping.

Positing its perceptual basis in a theory of embodied cognition, the most common approach to overcoming this limitation involves techniques that afford the interactive exploration of the data using gross body gestures. In some circumstances, such exploration is not possible and, even when it is, it may be neither necessary nor sufficient.

This article explores some other possible reasons for the unreliability of parameter mapping sonification and, drawing from the experience of expressive musical performance, suggests that the problem lies not in the parametric approach per se, nor in the lack of interactivity, but in the extent to which the parameters employed contribute to coherent gestalts. A method for how this might be achieved that relies on the use of micro-gestural information is proposed. While this is speculative, the use of such gestural inflections is well known in music performance, is supported by findings in neuroscience and lends itself to empirical testing.

Type
Articles
Information
Organised Sound , Volume 19 , Special Issue 1: Sonification , April 2014 , pp. 52 - 59
Copyright
Copyright © Cambridge University Press 2014 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Adorno, T.W. 1991. On the Fetish Character in Music and the Regression of Listening. In The Culture Industry, ed. J.M. Bernstein, London: Routledge.Google Scholar
Beyer, C. 2011. Edmund Husserl. In E.N. Zalta (ed.) The Stanford Encyclopedia of Philosophy. http://plato.stanford.edu/archives/win2011/entries/husserl.Google Scholar
Cary, T.O. 1992. Illustrated Compendium of Musical Technology. London: Faber and Faber.Google Scholar
Diniz, N., Demey, M., Leman, M. 2010. An Interactive Framework for Multilevel Sonification. In Proceedings of ISon 2010, 3rd Interactive Sonification Workshop, KTH, Stockholm, Sweden, 7 April.Google Scholar
Doornbusch, P. 2002. Composers’ Views on Mapping in Algorithmic Composition. Organised Sound 7(2): 145–156.CrossRefGoogle Scholar
Dreyfus, H. 1992. What Computers Still Can't Do. Cambridge, MA: The MIT Press.Google Scholar
Flowers, J.H. 2005. Thirteen Years of Reflection on Auditory Graphing: Promises, Pitfalls, and Potential New Directions. In Proceedings of the First Symposium on Auditory Graphs, Limerick, Ireland, 10 July.Google Scholar
Flowers, J.H., Buhman, D.C., Turnage, K.D. 1997. Cross-Modal Equivalence of Visual and Auditory Scatterplots for Exploring Bivariate Data Samples. Human Factors 39: 341–351.Google Scholar
Frysinger, S.P. 2005. A Brief History of Auditory Data Representation to the 1980s. In Proceedings of the First Symposium on Auditory Graphs, Limerick, Ireland, 10 July.Google Scholar
Fyk, J. 1997. Intonational Protention in the Performance of Melodic Octaves on the Violin. In M. Leman (ed.) Music, Gestalt, and Computing: Studies in Cognitive and Systematic Musicology. Berlin: Springer.Google Scholar
Goehr, L. 1994. The Imaginary Museum of Musical Works. Oxford: Oxford University Press.CrossRefGoogle Scholar
Godøy, R.I. 2006. Gestural-Sonorous Objects: Embodied Extensions of Schaeffer's Conceptual Apparatus. Organised Sound 11(2): 149–157.Google Scholar
Godøy, R.I. 2010. Images of Sonic Objects. Organised Sound 15(1): 54–62.Google Scholar
Godøy, R.I., Leman, M. (eds.) 2010. Musical Gestures: Sound, Movement, and Meaning. New York: Routledge.Google Scholar
Godøy, R.I., Jensenius, A.R., Voldsund, A., Glette, K., Høvin, M., Nymoen, K., Skogstad, S., Tørresen, J. 2012. Classifying Music-Related Actions. In Proceeding of the 12th International Conference on Music Perception and Cognition and the 8th Triennial Conference of the European Society for the Cognitive Sciences of Music, Thessaloniki, Greece, 23–28 July.Google Scholar
Grond, F., Berger, J. 2011. Parameter Mapping Sonification. In T. Hermann, A. Hunt and J.G. Neuhoff (eds.) The Sonification Handbook. Berlin: Logos.Google Scholar
Großhauser, T., Großekathöfer, U., Herman, T. 2010. New Sensors and Pattern Recognition Techniques for String Instruments. In Proceedings of the 2010 Conference on New Interfaces for Musical Expression (NIME 2010), Sydney, Australia.Google Scholar
Hermann, T., Ritter, H. 2005. Model-Based Sonification Revisited: Authors’ Comments on Hermann and Ritter, ICAD 2002. In ACM Transactions on Applied Perception 2(4): 559–63.Google Scholar
Hermann, T., Krause, J., Ritter, H. 2002. Real-Time Control of Sonification Models with an Audio-Haptic Interface. In Proceedings of the International Conference on Auditory Display (ICAD 2002), International Community for Auditory Display, 82–86.Google Scholar
Hunt, A., Hermann, T. 2011. Interactive Sonification. In T. Hermann, A. Hunt and J.G. Neuhoff (eds.) The Sonification Handbook. Berlin: Logos.Google Scholar
Hunt, A, Wanderley, M.M. 2002. Mapping Performer Parameters to Synthesis Engines. Organised Sound 7(2): 97–108.CrossRefGoogle Scholar
Iazzetta, F. 2000. Meaning in Musical Gesture. In M.M. Wanderley and M. Battier (eds.) Trends in Gestural Control of Music. Paris: IRCAM.Google Scholar
Kane, B. 2007. L'Objet Sonore Maintenant: Pierre Schaeffer, Sound Objects and the Phenomenological Reduction. Organised Sound 12(1): 15–24.Google Scholar
Kima, J.H., Demeyb, M., Moelantsb, D., Leman, M. 2010. Performance Micro-Gestures Related to Musical Expressiveness. In Proceedings of the 11th International conference on Music Perception and Cognition, Seattle, WA, USA, 23–27 August.Google Scholar
Kramer, G. 1994. Some Organizing Principles for Representing Data with Sound. In: G. Kramer (ed.), Auditory Display: Sonification, Audification, and Auditory Interfaces, Santa Fe Institute Studies in the Sciences of Complexity Proceedings 18. Reading, MA: Addison-Wesley.Google Scholar
Loui, P., Wessel, D.L., Hudson Kam, C.L. 2010. Humans Rapidly Learn Grammatical Structures in a New Scale. Music Perception 27(5) (June): 377–388.CrossRefGoogle Scholar
Mathews, M.V. 1969. The Technology of Computer Music. Cambridge, MA: The MIT Press.Google Scholar
McCorduck, P. 2004. Machines Who Think, 2nd ed. Natick, MA: A. K. Peters.CrossRefGoogle Scholar
Miller, G.A. 1956. The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capacity for Processing Information. Psychological Review 63: 81–97.Google Scholar
Moore, F.R. 1990. Elements of Computer Music. Englewood Cliffs, NJ: Prentice-Hall.Google Scholar
Nichols, C. 2002. The vBow: A Virtual Violin Bow Controller for Mapping Gesture to Synthesis with Haptic Feedback. Organised Sound 7(2): 215–220.CrossRefGoogle Scholar
Paine, G. 2009. Towards Unified Design Guidelines for New Interfaces for Musical Expression. Organised Sound 14(2): 143–156.Google Scholar
Palombini, C.V. de L. 1993. Pierre Schaeffer.s typo-morphology of sonic objects. Doctoral thesis, Durham University. Durham E-Theses Online: http://etheses.dur.ac.uk/1191. Accessed on 1 May 2013.Google Scholar
Polansky, L., Childs, E. 2002. Manifestation and Sonification: The Science and Art of Sonification, Tufte's Visualization, and the ‘Slippery Slope’ to Algorithmic Composition. An Informal Response to Ed Childs’ Short Paper on Tufte and Sonification; with Additional Commentary by Childs. http://eamusic.dartmouth.edu/~larry/sonification.html. Accessed on 16 October 2012.Google Scholar
Scaletti, C. 1994. Sound Synthesis Algorithms for Auditory Data Representation. In: G. Kramer (ed.) Auditory Display: Sonification, Audification, and Auditory Interfaces. Santa Fe Institute Studies in the Sciences of Complexity Proceedings 18. Reading, MA: Addison-Wesley.Google Scholar
Schaeffer, P. 1966. Traité des objets musicaux: essai interdisciplines. Paris: Seuil.Google Scholar
Smalley, D. 1986. Spectro-Morphology and Structuring Processes. In S. Emmerson (ed.) The Language of Electroacoustic Music. London: Macmillan.Google Scholar
Smalley, D. 1992. The Listening Imagination: Listening in the Electroacoustic Era. In J. Payner, T. Howell, R. Orton and P. Seymour (eds.) Companion to Contemporary Musical Thought. London and New York: Routledge.Google Scholar
Smalley, D. 1997. Spectromorphology: Explaining Sound-Shapes. Organised Sound 2(2): 107–126.Google Scholar
Smalley, D. 2007. Space-Form and the Acousmatic Image. Organised Sound 12(1): 107–126.Google Scholar
Thoresen, L., Hedman, A. 2007. Spectromorphological Analysis of Sound Objects: An Adaptation of Pierre Schaeffer's Typomorphology. Organised Sound 12(2): 129–141.Google Scholar
Todd, P.M, Loy, G. (eds.) 1991. Music and Connectionism. Cambridge, MA: The MIT Press.Google Scholar
Tuuri, K., Eerola, T. 2012. Formulating a Revised Taxonomy for Modes of Listening. Journal of New Music Research 41(2): 137–152.Google Scholar
Tuuri, K., Mustonen, M., Pirhonen, A. 2007. Same Sound – Different Meanings: A Novel Scheme for Modes of Listening. In Proceedings of Audio Mostly, Ilmenau, Germany: Fraunhofer Institute for Digital Media Technology IDMT, 13–18.Google Scholar
Worrall, D. 2009a. An Introduction to Data Sonification. In R.T. Dean (ed.), The Oxford Handbook of Computer Music and Digital Sound Culture. Oxford: Oxford University Press.Google Scholar
Worrall, D. 2009b. Sonification: Concepts, Instruments and Techniques. PhD thesis, University of Canberra. Available at http://erl.canberra.edu.au/public/adt-AUC20090818.142345.Google Scholar
Worrall, D. 2010. Parameter Mapping Sonic Articulation and the Perceiving Body. In Proceeding of the 16th International Conference on Auditory Display, Washington, DC, USA, 9–15 June.Google Scholar
Worrall, D., Bylstra, M., Barrass, S., Dean, R. 2007. SoniPy: The Design of an Extendable Software Framework for Sonification Research and Auditory Display. In Proceedings of the 13th International Conference on Auditory Display, Montreal, Canada, 26–29 June.Google Scholar
Xenakis, I. 1971. Formalized Music: Thought and Mathematics in Music. Bloomington: Indiana University Press.Google Scholar