Particularly in the eighteenth century, the conviction that there was an essential relationship between musical contents and the facial expressions and gestures of performers, so that “a large part of music” lay in the art of gesture (Johann Mattheson), belonged to the topoi of an enlightened musical aesthetic. In the tradition of antique rhetoric, facial expressions and gestures were understood to be an important element in an expressive musical “sound discourse.” Namely, the expositions of Quintilian and Cicero on the orator’s action served as foundation for later views concerning the significance of both those phenomena for musical performance; Cicero postulated, for example, that by its very nature every stirring of the spirit or disposition had a characteristic correspondence in the orator’s facial expressions, intonation, and gestures—“Omnis enim motus animi suum quendam a natura habet vultum et sonum et gestum“—so that the actor’s entire body, his play with facial expressions, and his voice had the sound of an instrument’s strings, struck by corresponding emotion (“corpusque totum hominis et eius omnis vultus omnesque voces, ut nervi in fidibus, ita sonant, ut a motu animi quoque sunt pulsae”).
In this project, we aim to understand the role played by facial expressions and gestures in the perception of musical expressions. There are two aspects to this question. Musical performance is perceived via several different sensory channels. Besides the auditory system, vision is of importance, too. On the one hand, we are therefore interested in how music emotional content and expressivity is communicated by auditory and visual information. On the other hand, we study the specific effect of facial expressions on perceived emotional expressivity. To do so, we asked professional singers to interpret pieces from their repertoire either with or without expressive face, and we recorded their performance. Those interpretations were then evaluated by musical laymen and experts. Stimuli were presented as either audio (without visuals), or visual (without audio), or visual plus audio (bimodal stimuli). We demonstrated that expressive facial expressions have a specific effect on perceived expressivity and emotion expressions, but was affected by presentation mode, that is, the effect was most prominent for visual and visual-auditory perception (Grimm, Fünderich, & Lange, 2017). This visual dominance was modulated by expertise, showing that experts have learned to decode the visual information of musical performance more efficiently.
Lange, E.B., Fünderich, J. & Grimm, H. (2022) Multisensory integration of musical emotion perception in singing. Psychological Research (2022). doi.org: 10.1007/s00426-021-01637-9
Grimm, H., Fünderich, J., & Lange, E.B. (August, 2017). How is the perceived musical expressivity of singers affected by their mimic and gestural interpretation? 25th Anniversary Edition of the European Society for the Cognitive Sciences of Music (ESCOM), Ghent, Belgium.