Abstract
Recent instruments measuring continuous self-reported emotion responses to music have tended to use dimensional rating scale models of emotion such as valence (happy to sad). However, numerous retrospective studies of emotion in music use checklist style responses, usually in the form of emotion words, (such as happy, angry, sad…) or facial expressions. A response interface based on six simple sketch style emotion faces aligned into a clock-like distribution was developed with the aim of allowing participants to quickly and easily rate emotions in music continuously as the music unfolded. We tested the interface using six extracts of music, one targeting each of the six faces: ‘Excited’ (at 1 o’clock), ‘Happy’ (3), ‘Calm’ (5), ‘Sad’ (7), ‘Scared’ (9) and ‘Angry’ (11). 30 participants rated the emotion expressed by these excerpts on our ‘emotion-face-clock’. By demonstrating how continuous category selections (votes) changed over time, we were able to show that (1) more than one emotion-face could be expressed by music at the same time and (2) the emotion face that best portrayed the emotion the music conveyed could change over time, and (3) the change could be attributed to changes in musical structure. Implications for research on orientation time and mixed emotions are discussed.
Keywords
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Yang, Y.H., Lin, Y.C., Su, Y.F., Chen, H.H.: A regression approach to music emotion recognition. IEEE Transactions on Audio, Speech, and Language Processing 16(2), 448–457 (2008)
Schmidt, E.M., Turnbull, D., Kim, Y.E.: Feature selection for content-based, time-varying musical emotion regression. In: MIR 2010 Proceedings of the International Conference on Multimedia Information Retrieval. ACM, New York (2010)
Korhonen, M.D., Clausi, D.A., Jernigan, M.E.: Modeling emotional content of music using system identification. IEEE Transactions on Systems Man and Cybernetics Part B-Cybernetics 36(3), 588–599 (2006)
Schubert, E.: Continuous self-report methods. In: Juslin, P.N., Sloboda, J.A. (eds.) Handbook of Music and Emotion: Theory, Research, Applications, pp. 223–253. OUP, Oxford (2010)
Madsen, C.K., Frederickson, W.E.: The experience of musical tension: A replication of Nielsen’s research using the continuous response digital interface. Journal of Music Therapy 30(1), 46–63 (1993)
Nielsen, F.V.: Musical tension and related concepts. In: Sebeok, T.A., Umiker-Sebeok, J. (eds.) The Semiotic Web 1986. An International Year-Book. Mouton de Gruyter, Berlin (1987)
Russell, J.A.: Affective space is bipolar. Journal of Personality and Social Psychology 37(3), 345–356 (1979)
Russell, J.A.: A circumplex model of affect. Journal of Social Psychology 39, 1161–1178 (1980)
Krumhansl, C.L.: An exploratory study of musical emotions and psychophysiology. Canadian Journal of Experimental Psychology 51(4), 336–352 (1997)
Cowie, R., Douglas-Cowie, E., Savvidou, S., McMahon, E., Sawey, M., Schröder, M.: FEELTRACE: An instrument for recording perceived emotion in real time. In: Cowie, R., Douglas-Cowie, E., Schroede, M. (eds.) Speech and Emotion: Proceedings of the ISCA workshop, pp. 19–24. Newcastle, Co. Down (2000)
Nagel, F., Kopiez, R., Grewe, O., Altenmüller, E.: EMuJoy: Software for continuous measurement of perceived emotions in music. Behavior Research Methods 39(2), 283–290 (2007)
Schubert, E.: Measuring emotion continuously: Validity and reliability of the two-dimensional emotion-space. Australian Journal of Psychology 51(3), 154–165 (1999)
Schimmack, U., Rainer, R.: Experiencing activation: Energetic arousal and tense arousal are not mixtures of valence and activation. Emotion 2(4), 412–417 (2002)
Schimmack, U., Grob, A.: Dimensional models of core affect: A quantitative comparison by means of structural equation modeling. European Journal of Personality 14(4), 325–345 (2000)
Wundt, W.: Grundzüge der physiologischen Psychologie. Engelmann, Leipzig (1905)
Plutchik, R.: The emotions: Facts, theories and a new model. Random House, New York (1962)
Russell, J.A., Mehrabian, A.: Evidence for a 3-factor theory of emotions. Journal of Research in Personality 11(3), 273–294 (1977)
Barrett, L.F., Wager, T.D.: The Structure of Emotion: Evidence From Neuroimaging Studies. Current Directions in Psychological Science 15(2), 79–83 (2006)
Barrett, L.F.: Discrete emotions or dimensions? The role of valence focus and arousal focus. Cognition & Emotion 12(4), 579–599 (1998)
Lewis, M., Haviland, J.M. (eds.): Handbook of emotions (1993)
Izard, C.E.: The psychology of emotions. Plenum Press, New York (1991)
Izard, C.E.: Organizational and motivational functions of discrete emotions. In: Lewis, M., Haviland, J.M. (eds.) Handbook of Emotions, pp. 631–641. The Guilford Press, New York (1993)
Namba, S., Kuwano, S., Hatoh, T., Kato, M.: Assessment of musical performance by using the method of continuous judgment by selected description. Music Perception 8(3), 251–275 (1991)
Juslin, P.N., Laukka, P.: Communication of emotions in vocal expression and music performance: Different channels, same code? Psychological Bulletin 129(5), 770–814 (2003)
Laukka, P., Gabrielsson, A., Juslin, P.N.: Impact of intended emotion intensity on cue utilization and decoding accuracy in vocal expression of emotion. International Journal of Psychology 35(3-4), 288 (2000)
Juslin, P.N.: Communicating emotion in music performance: A review and a theoretical framework. In: Juslin, P.N., Sloboda, J.A. (eds.) Music and Emotion: Theory and Research, pp. 309–337. Editors. Oxford University Press, London (2001)
Schubert, E., Ferguson, S., Farrar, N., McPherson, G.E.: Sonification of Emotion I: Film Music. In: The 17th International Conference on Auditory Display (ICAD-2011), International Community for Auditory Display (ICAD), Budapest (2011)
Hevner, K.: Expression in music: a discussion of experimental studies and theories. Psychological Review 42, 187–204 (1935)
Hevner, K.: The affective character of the major and minor modes in music. American Journal of Psychology 47, 103–118 (1935)
Hevner, K.: Experimental studies of the elements of expression in music. American Journal of Psychology 48, 246–268 (1936)
Hevner, K.: The affective value of pitch and tempo in music. American Journal of Psychology 49, 621–630 (1937)
Rigg, M.G.: The mood effects of music: A comparison of data from four investigators. The Journal of Psychology 58(2), 427–438 (1964)
Han, B., Rho, S., Dannenberg, R.B., Hwang, E.: SMERS: Music emotion recognition using support vector regression. In: Proceedings of the 10th International Society for Music Information Retrieval Conference (ISMIR 2009), Kobe International Conference Center, Kobe, Japan, October 26-30 (2009)
Schlosberg, H.: The description of facial expressions in terms of two dimensions. Journal of Experimental Psychology 44, 229–237 (1952)
Russell, J.A.: Reading emotion from and into faces: Resurrecting a dimensional-contextual perspective. In: Russell, J.A., Fernández-Dols, J.M. (eds.) The Psychology of Facial Expression, pp. 295–320. Cambridge University Press, Cambridge (1997)
Dimberg, U., Thunberg, M.: Rapid facial reactions to emotional facial expressions. Scandinavian Journal of Psychology 39(1), 39–45 (1998)
Britton, J.C., Taylor, S.F., Sudheimer, K.D., Liberzon, I.: Facial expressions and complex IAPS pictures: common and differential networks. Neuroimage 31(2), 906–919 (2006)
Waller, B.M., Cray Jr, J.J., Burrows, A.M.: Selection for universal facial emotion. Emotion 8(3), 435–439 (2008)
Ekman, P.: Facial expression and emotion. American Psychologist 48(4), 384–392 (1993)
Lang, P.J.: Behavioral treatment and bio-behavioral assessment: Computer applications. In: Sidowski, J.B., Johnson, J.H., Williams, T.A. (eds.) Technology in Mental Health Care Delivery Systems, pp. 119–137. Ablex, Norwood (1980)
Bradley, M.M., Lang, P.J.: Measuring emotion - the self-assessment mannequin and the semantic differential. Journal of Behavior Therapy and Experimental Psychiatry 25(1), 49–59 (1994)
Ekman, P., Rosenberg, E.L. (eds.): What the face reveals: Basic and applied studies of spontaneous expression using the Facial Action Coding System (FACS). Series in affective science. Oxford University Press, London (1997)
Eerola, T., Vuoskoski, J.K.: A comparison of the discrete and dimensional models of emotion in music. Psychology of Music 39(1), 18–49 (2011)
Kratus, J.: A developmental study of children’s interpretation of emotion in music. Psychology of Music 21, 3–19 (1993)
Schubert, E., McPherson, G.E.: The perception of emotion in music. In: McPherson, G.E. (ed.) The Child as Musician: A Handbook of Musical Development, pp. 193–212. Oxford University Press, Oxford (2006)
Ekman, R., Friesen, W.V., Ellsworth, R.: Emotion in the human face: Guidelines jbr research and an integration of findings. Pergamon Press, New York (1972)
Zentner, M., Grandjean, D., Scherer, K.R.: Emotions evoked by the sound of music: characterization, classification, and measurement. Emotion 8(4), 494–521 (2008)
Schubert, E.: Update of the Hevner adjective checklist. Perceptual and Motor Skills 96(3), 1117–1122 (2003)
Kostov, V., Yanagisawa, H., Johansson, M., Fukuda, S.: Method for Face-Emotion Retrieval Using A Cartoon Emotional Expression Approach. JSME International Journal Series C 44(2), 515–526 (2001)
Schubert, E.: Continuous measurement of self-report emotional response to music. In: Juslin, P.N., Sloboda, J.A. (eds.) Music and Emotion: Theory and Research, pp. 393–414. Oxford University Press, Oxford (2001)
Schubert, E.: Reliability issues regarding the beginning, middle and end of continuous emotion ratings to music. Psychology of Music 41(3), 350–371 (2013)
Bachorik, J.P., Bangert, M., Loui, P., Larke, K., Berger, J., Rowe, R., Schlaug, G.: Emotion in motion: Investigating the time-course of emotional judgments of musical stimuli. Music Perception 26(4), 355–364 (2009)
Schubert, E., Dunsmuir, W.: Regression modelling continuous data in music psychology. In: Yi, S.W. (ed.) Music, Mind, and Science, pp. 298–352. Seoul National University, Seoul (1999)
Juslin, P.N., Friberg, A., Bresin, R.: Toward a computational model of expression in music performance: The GERM model. Musicae Scientiae. Special Issue: pp. 63-122 (2001)
Lucas, B.J., Schubert, E., Halpern, A.R.: Perception of emotion in sounded and imagined music. Music Perception 27(5), 399–412 (2010)
Upham, F.: Quantifying the temporal dynamics of music listening: A critical investigation of analysis techniques for collections of continuous responses to music. McGill University (2011)
Schubert, E., Vincs, K., Stevens, C.J.: Identifying regions of good agreement among responders in engagement with a piece of live dance. Empirical Studies of the Arts 13(1), 1–20 (2013)
Hunter, P.G., Schellenberg, E.G., Schimmack, U.: Mixed affective responses to music with conflicting cues. Cognition & Emotion 22(2), 327–352 (2008)
Hunter, P.G., Schellenberg, E.G., Schimmack, U.: Feelings and perceptions of happiness and sadness induced by music: Similarities, differences, and mixed emotions. Psychology of Aesthetics, Creativity, and the Arts 4(1), 47–56 (2010)
Trohidis, K., Tsoumakas, G., Kalliris, G., Vlahavas, I.: Multilabel classification of music into emotions. In: Proceedings of the 9th International Conference on Music Information Retrieval (ISMIR 2008), Philadelphia, PA (2008)
Levy, M., Sandler, M.: A semantic space for music derived from social tags. In: Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR 2007), Vienna, Austria (2007)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Schubert, E., Ferguson, S., Farrar, N., Taylor, D., McPherson, G.E. (2013). The Six Emotion-Face Clock as a Tool for Continuously Rating Discrete Emotional Responses to Music. In: Aramaki, M., Barthet, M., Kronland-Martinet, R., Ystad, S. (eds) From Sounds to Music and Emotions. CMMR 2012. Lecture Notes in Computer Science, vol 7900. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-41248-6_1
Download citation
DOI: https://doi.org/10.1007/978-3-642-41248-6_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-41247-9
Online ISBN: 978-3-642-41248-6
eBook Packages: Computer ScienceComputer Science (R0)