Skip to main content
SpringerLink
Log in

Reading the Author and Speaker: Towards a Holistic and Deep Approach on Automatic Assessment of What is in One’s Words

  • Conference paper
  • First Online:
Computational Linguistics and Intelligent Text Processing (CICLing 2017)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10762))

  • 1091 Accesses

Abstract

Computational text analysis is continuously becoming richer in ways the author of a text is ‘read’ in terms of the states and traits of the person behind the words such as writer’s age, gender, personality, emotion or sentiment to name but a few. Similarly, in the analysis of spoken language, one finds a broadening palette of such characteristics of speakers automatically analysed in recent Computational Paralinguistics research. It seems wise to assess these characteristics in one pass to understand their interrelationship rather than going one by one in isolation. As an example, it may help to estimate one’s personality knowing the age, gender, and cultural background of the person. Thus, a holistic approach is advocated that aims at automatically assessing the ‘larger’ picture of a person that wrote or spoke words of analysis. Here, a short motivation and inspirations ‘en route’ to holistic author and word-based speaker profiling are given.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    More information can be accessed from http://www.compare.openaudio.eu.

References

  1. Abbott, R., Walker, M., Anand, P., Fox Tree, J.E., Bowmani, R., King, J.: How can you say such things? recognizing disagreement in informal political argument. In: Proceedings Workshop on Languages in Social Media, pp. 2–11. ACL, Portland, Oregon (2011)

    Google Scholar 

  2. Balota, D.A., Burgess, G.C., Cortese, M.J., Adams, D.R.: The word-frequency mirror effect in young, old, and early-stage alzheimer’s disease: evidence for two processes in episodic recognition performance. J. Mem. Lang. 46(1), 199–226 (2002)

    Article  Google Scholar 

  3. Balota, D.A., Duchek, J.M.: Semantic priming effects, lexical repetition effects, and contextual disambiguation effects in healthy aged individuals and individuals with senile dementia of the alzheimer type. Brain Lang. 40(2), 181–201 (1991)

    Article  Google Scholar 

  4. Beňuš, Š.: Social aspects of entrainment in spoken interaction. Cogn. Comput. 6(4), 802–813 (2014)

    Article  Google Scholar 

  5. Biel, J.I., Tsiminaki, V., Dines, J., Gatica-Perez, D.: Hi youtube!: personality impressions and verbal content in social video. In: Proceedings ICMI, pp. 119–126. ACM, Sydney, Australia (2013)

    Google Scholar 

  6. Cambria, E., Schuller, B., Xia, Y., Havasi, C.: New avenues in opinion mining and sentiment analysis. IEEE Intell. Syst. Mag. 28(2), 15–21 (2013)

    Article  Google Scholar 

  7. Celiktutan, O., Eyben, F., Sariyanidi, E., Gunes, H., Schuller, B.: MAPTRAITS 2014: the first audio/visual mapping personality traits challenge. In: Proceedings Personality Mapping Challenge & Workshop (MAPTRAITS), Satellite of ICMI, pp. 3–9. ACM, Istanbul, Turkey (2014)

    Google Scholar 

  8. Chakravarthula, S.N., Xiao, B., Imel, Z.E., Atkins, D.C., Georgiou, P.G.: Assessing empathy using static and dynamic behavior models based on therapist’s language in addiction counseling. In: Proceedings Interspeech, pp. 668–672. ISCA, Dresden, Germany (2015)

    Google Scholar 

  9. Chaski, C.E.: Empirical evaluations of language-based author identification techniques. Forensic Linguist. 8, 1–65 (2001)

    Google Scholar 

  10. Coppersmith, G., Dredze, M., Harman, C., Hollingshead, K.: From adhd to sad: analyzing the language of mental health on twitter through self-reported diagnoses. In: Proceedings CLPsych at NAACL HLT, p. 1. ACL, Denver, Colorado (2015)

    Google Scholar 

  11. Davidov, D., Tsur, O., Rappoport, A.: Semi-supervised recognition of sarcastic sentences in twitter and amazon. In: Proceedings 14th Conference on Computational Natural Language Learning, pp. 107–116. ACL, Uppsala, Sweden (2010)

    Google Scholar 

  12. De Vel, O., Anderson, A., Corney, M., Mohay, G.: Mining e-mail content for author identification forensics. ACM SIGMOD Rec. 30(4), 55–64 (2001)

    Article  Google Scholar 

  13. Deng, J., Xu, X., Zhang, Z., Frühholz, S., Schuller, B.: Universum autoencoder-based domain adaptation for speech emotion recognition. IEEE Signal Process. Lett. 24, 5 (2017)

    Article  Google Scholar 

  14. Desmet, B., Pauwels, K., Hoste, V.: Online suicide risk detection using automatic text classification. In: Proceedings International Summit on Suicide Research. IASR/AFSP, New York City, 1 p. (2015)

    Google Scholar 

  15. Dhall, A., Goecke, R., Joshi, J., Wagner, M., Gedeon, T.: Emotion recognition in the wild challenge 2013. In: Proceedings ICMI, pp. 509–516. ACM, Sydney, Australia (2013)

    Google Scholar 

  16. González-Ibánez, R., Muresan, S., Wacholder, N.: Identifying sarcasm in twitter: a closer look. In: Proceedings ACL, vol. 2, pp. 581–586. ACL, Portland, Oregon (2011)

    Google Scholar 

  17. Holtgraves, T., Fogle, K., Marsh, L.: Pragmatic language production deficits in Parkinson’s disease. In: Advances in Parkinson’s Disease, vol. 2, pp. 31–36 (2013)

    Article  Google Scholar 

  18. Jarrold, W.L., Peintner, B., Yeh, E., Krasnow, R., Javitz, H.S., Swan, G.E.: Language analytics for assessing brain health: cognitive impairment, depression and pre-symptomatic alzheimer’s disease. In: Yao, Y., Sun, R., Poggio, T., Liu, J., Zhong, N., Huang, J. (eds.) BI 2010. LNCS (LNAI), vol. 6334, pp. 299–307. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15314-3_28

    Chapter  Google Scholar 

  19. Justo, R., Corcoran, T., Lukin, S.M., Walker, M., Torres, M.I.: Extracting relevant knowledge for the detection of sarcasm and nastiness in the social web. Knowl. Based Syst. 69, 124–133 (2014)

    Article  Google Scholar 

  20. Li, Y., Tao, J., Schuller, B., Shan, S., Jiang, D., Jia, J.: MEC 2016: the multimodal emotion recognition challenge of CCPR 2016. In: Tan, T., Li, X., Chen, X., Zhou, J., Yang, J., Cheng, H. (eds.) CCPR 2016. CCIS, vol. 663, pp. 667–678. Springer, Singapore (2016). https://doi.org/10.1007/978-981-10-3005-5_55

    Chapter  Google Scholar 

  21. Lin, J.: Automatic author profiling of online chat logs. Ph.D. thesis, Naval Postgraduate School, Monterey, California (2007)

    Google Scholar 

  22. Mairesse, F., Walker, M.A., Mehl, M.R., Moore, R.K.: Using linguistic cues for the automatic recognition of personality in conversation and text. J. Artif. Intell. Res. 30, 457–500 (2007)

    Article  Google Scholar 

  23. Mihalcea, R., Strapparava, C.: Making computers laugh: investigations in automatic humor recognition. In: Proceedings Conference on Human Language Technology and Empirical Methods in Natural Language Processing, pp. 531–538. ACL, Vancouver, Canada (2005)

    Google Scholar 

  24. Mihalcea, R., Strapparava, C.: The lie detector: explorations in the automatic recognition of deceptive language. In: Proceedings ACL-IJCNLP Conference Short Papers, pp. 309–312. ACL, Singapore (2009)

    Google Scholar 

  25. Neuman, Y., Cohen, Y., Assaf, D., Kedma, G.: Proactive screening for depression through metaphorical and automatic text analysis. Artif. Intell. Med. 56(1), 19–25 (2012)

    Article  Google Scholar 

  26. Patra, B.G., Banerjee, S., Das, D., Saikh, T., Bandyopadhyay, S.: Automatic author profiling based on linguistic and stylistic features. Notebook for PAN at CLEF 1179 (2013)

    Google Scholar 

  27. Pokorny, F.B., Marschik, P.B., Einspieler, C., Schuller, B.W.: Does she speak RTT? Towards an earlier identification of RETT syndrome through intelligent pre-linguistic vocalisation analysis. In: Proceedings Interspeech, pp. 1953–1957 (2016)

    Google Scholar 

  28. Poria, S., Gelbukh, A., Agarwal, B., Cambria, E., Howard, N.: Common sense knowledge based personality recognition from text. In: Castro, F., Gelbukh, A., González, M. (eds.) MICAI 2013. LNCS (LNAI), vol. 8266, pp. 484–496. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-45111-9_42

    Chapter  Google Scholar 

  29. Ranganath, R., Jurafsky, D., McFarland, D.: It’s not you, it’s me: detecting flirting and its misperception in speed-dates. In: Proceedings Conference on Empirical Methods in Natural Language Processing, vol. 1, pp. 334–342. ACL, Singapore (2009)

    Google Scholar 

  30. Ranganath, R., Jurafsky, D., McFarland, D.A.: Detecting friendly, flirtatious, awkward, and assertive speech in speed-dates. Comput. Speech Lang. 27(1), 89–115 (2013)

    Article  Google Scholar 

  31. Rangel, F., Rosso, P.: Use of language and author profiling: identification of gender and age. In: Natural Language Processing and Cognitive Science, vol. 177 (2013)

    Google Scholar 

  32. Rangel, F., Rosso, P.: On the impact of emotions on author profiling. Inf. Process. Manag. 52(1), 73–92 (2016)

    Article  Google Scholar 

  33. Rangel, F., Rosso, P., Koppel, M.M., Stamatatos, E., Inches, G.: Overview of the author profiling task at pan 2013. In: Proceedings CLEF Conference on Multilingual and Multimodal Information Access Evaluation, pp. 352–365. CELCT, Amsterdam, The Netherlands (2013)

    Google Scholar 

  34. Rangel, F., Rosso, P., Potthast, M., Stein, B., Daelemans, W.: Overview of the 3rd author profiling task at pan 2015. In: Proceedings CLEF, 18 p. CLEF Association, Toulouse, France (2015)

    Google Scholar 

  35. Razavi, A.H., Inkpen, D., Uritsky, S., Matwin, S.: Offensive language detection using multi-level classification. In: Farzindar, A., Kešelj, V. (eds.) AI 2010. LNCS (LNAI), vol. 6085, pp. 16–27. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-13059-5_5

    Chapter  Google Scholar 

  36. Regneri, M., King, D.: Automatically evaluating atypical language in narratives by children with autistic spectrum disorder. In: Sharp, B., Delmonte, R. (eds.) Natural Language Processing and Cognitive Science: Proceedings 2014, pp. 173–186. Walter de Gruyter GmbH & Co KG, Berlin (2015)

    Google Scholar 

  37. Reyes, A., Rosso, P., Buscaldi, D.: From humor recognition to irony detection: the figurative language of social media. Data Knowl. Eng. 74, 1–12 (2012)

    Article  Google Scholar 

  38. Rude, S., Gortner, E.M., Pennebaker, J.: Language use of depressed and depression-vulnerable college students. Cogn. Emot. 18(8), 1121–1133 (2004)

    Article  Google Scholar 

  39. Schuller, B.: Recognizing affect from linguistic information in 3D continuous space. IEEE Trans. Affect. Comput. 2(4), 192–205 (2012)

    Article  Google Scholar 

  40. Schuller, B.: The computational paralinguistics challenge. IEEE Signal Process. Mag. 29(4), 97–101 (2012)

    Article  Google Scholar 

  41. Schuller, B., Batliner, A.: Computational Paralinguistics: Emotion, Affect and Personality in Speech and Language Processing. Wiley, U.S. (2013)

    Book  Google Scholar 

  42. Schuller, B., Batliner, A., Steidl, S., Seppi, D.: Recognising realistic emotions and affect in speech: state of the art and lessons learnt from the first challenge. Speech Commun. 53(9/10), 1062–1087 (2011)

    Article  Google Scholar 

  43. Schuller, B., Mousa, A.E.D., Vasileios, V.: Sentiment analysis and opinion mining: on optimal parameters and performances. WIREs Data Min. Knowl. Discov. 5, 255–263 (2015)

    Article  Google Scholar 

  44. Schuller, B., Müller, R., Hörnler, B., Höthker, A., Konosu, H., Rigoll, G.: Audiovisual recognition of spontaneous interest within conversations. In: Proceedings ICMI, pp. 30–37. ACM, Nagoya, Japan (2007)

    Google Scholar 

  45. Schuller, B., Rigoll, G., Lang, M.: Speech emotion recognition combining acoustic features and linguistic information in a hybrid support vector machine-belief network architecture. In: Proceedings ICASSP, vol. 1, pp. 577–580. IEEE, Montreal, Canada (2004)

    Google Scholar 

  46. Schuller, B., etal.: The INTERSPEECH 2017 computational paralinguistics challenge: addressee, cold & snoring. In: Proceedings Interspeech, 5 p. ISCA, Stockholm, Sweden (2017)

    Google Scholar 

  47. Schuller, B., et al.: Paralinguistics in speech and language—state-of-the-art and the challenge. Comput. Speech Lang. 27(1), 4–39 (2013)

    Article  Google Scholar 

  48. Schuller, B., et al.: The INTERSPEECH 2014 computational paralinguistics challenge: cognitive & physical load. In: Proceedings Interspeech. ISCA, Singapore (2014)

    Google Scholar 

  49. Schuller, B., et al.: The INTERSPEECH 2015 computational paralinguistics challenge: degree of nativeness, parkinson’s & eating condition. In: Proceedings Interspeech, pp. 478–482. ISCA, Dresden, Germany (2015)

    Google Scholar 

  50. Schuller, B., et al.: The INTERSPEECH 2016 computational paralinguistics challenge: deception, sincerity & native language. In: Proceedings Interspeech, pp. 2001–2005. ISCA, San Francisco, California (2016)

    Google Scholar 

  51. Schuller, B., et al.: A survey on perceived speaker traits: personality, likability, pathology, and the first challenge. Comput. Speech Lang. 29(1), 100–131 (2015)

    Article  Google Scholar 

  52. Schuller, B., et al.: Medium-term speaker states—a review on intoxication, sleepiness and the first challenge. Comput. Speech Lang. 28(2), 346–374 (2014)

    Article  Google Scholar 

  53. Schuller, B., et al.: The INTERSPEECH 2013 computational paralinguistics challenge: social signals, conflict, emotion, autism. In: Proceedings Interspeech, pp. 148–152. ISCA, Lyon, France (2013)

    Google Scholar 

  54. Schuller, B., Zhang, Y., Eyben, F., Weninger, F.: Intelligent systems’ holistic evolving analysis of real-life universal speaker characteristics. In: Proceedings 5th International Workshop on Emotion Social Signals. Sentiment & Linked Open Data (ES\(^3\)LOD 2014), Satellite of LREC, pp. 14–20. ELRA, Reykjavik, Iceland (2014)

    Google Scholar 

  55. Schwartz, H.A., et al.: Personality, gender, and age in the language of social media: the open-vocabulary approach. PloS ONE 8(9), e73791 (2013)

    Article  Google Scholar 

  56. Smith, J.M.: Breastfeeding and language outcomes: a review of the literature. J. Commun. Disord. 57, 29–40 (2015)

    Article  Google Scholar 

  57. Spertus, E.: Smokey: automatic recognition of hostile messages. In: Proceedings AAAI/IAAI, pp. 1058–1065. AAAI Press, Providence, Rhode Island (1997)

    Google Scholar 

  58. Squires, G.D., Chadwick, J.: Linguistic profiling a continuing tradition of discrimination in the home insurance industry? Urban Aff. Rev. 41(3), 400–415 (2006)

    Article  Google Scholar 

  59. Strapparava, C., Mihalcea, R.: Semeval-2007 task 14: affective text. In: Proceedings 4th International Workshop on Semantic Evaluations, pp. 70–74. ACL, Prague, Czech Republic (2007)

    Google Scholar 

  60. Strapparava, C., Mihalcea, R.: Learning to identify emotions in text. In: Proceedings ACM Symposium on Applied Computing, pp. 1556–1560. ACM, Fortaleza, Brazil (2008)

    Google Scholar 

  61. Strapparava, C., Mihalcea, R.: A Computational Analysis of the Language of Drug Addiction, vol. 2, pp. 136–142 (2017)

    Google Scholar 

  62. Valstar, M., Gratch, J., Schuller, B., Ringeval, F., Cowie, R., Pantic, M.: Summary for AVEC 2016: depression, mood, and emotion recognition workshop and challenge. In: Proceedings ACM International Conference on Multimedia, pp. 1483–1484. ACM, Amsterdam, The Netherlands (2016)

    Google Scholar 

  63. Van Halteren, H.: Linguistic profiling for author recognition and verification. In: Proceedings ACL, p. 199. ACL, Barcelona, Spain (2004)

    Google Scholar 

  64. Vijayaraghavan, P., Vosoughi, S., Roy, D.: Automatic detection and categorization of election-related tweets. In: Proceedings 10th International AAAI Conference on Web and Social Media. AAAI, Cologne, Germany (2016)

    Google Scholar 

  65. Wang, W.Y., Biadsy, F., Rosenberg, A., Hirschberg, J.: Automatic detection of speaker state: lexical, prosodic, and phonetic approaches to level-of-interest and intoxication classification. Comput. Speech Lang. 27(1), 168–189 (2013)

    Article  Google Scholar 

  66. Weninger, F., Wöllmer, M., Schuller, B.: Automatic assessment of singer traits in popular music: gender, age, height and race. In: Proceedings ISMIR, pp. 37–42. ISMIR, Miami, FL, USA (2011)

    Google Scholar 

  67. Xu, D., Gilkerson, J., Richards, J., Yapanel, U., Gray, S.: Child vocalization composition as discriminant information for automatic autism detection. In: Proceedings EMBC, pp. 2518–2522. IEEE, Minneapolis (2009)

    Google Scholar 

  68. Xu, D., Richards, J.A., Gilkerson, J., Yapanel, U., Gray, S., Hansen, J.: Automatic childhood autism detection by vocalization decomposition with phone-like units. In: Proceedings 2nd Workshop on Child, Computer and Interaction, Satellite of ICMI-MLMI, p. 5. ACM, Cambridge, MA, USA (2009)

    Google Scholar 

  69. Zhang, Y., Zhou, Y., Shen, J., Schuller, B.: Semi-autonomous data enrichment based on cross-task labelling of missing targets for holistic speech analysis. In: Proceedings ICASSP, pp. 6090–6094. IEEE, Shanghai, P.R. China (2016)

    Google Scholar 

Download references

Acknowledgement

The author would like to thank his colleague Anton Batliner for discussion regarding the taxonomies as discussed herein. He further acknowledges funding from the European Research Council within the European Union’s 7th Framework Programme under grant agreement no. 338164 (Starting Grant Intelligent systems’ Holistic Evolving Analysis of Real-life Universal speaker characteristics (iHEARu)), and the European Union’s Horizon 2020 Framework Programme under grant agreement no. 645378 (Research Innovation Action Artificial Retrieval of Information Assistants – Virtual Agents with Linguistic Understanding, Social skills, and Personalised Aspects (ARIA-VALUSPA)). The responsobility lies with the author.

Author information

Authors and Affiliations

  1. Department of Computing, Imperial College London, SW7 2AZ, London, UK

    Björn W. Schuller

  2. University of Passau, Chair of Complex and Intelligent Systems, 94032, Passau, Germany

    Björn W. Schuller

Authors
  1. Björn W. Schuller
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Björn W. Schuller .

Editor information

Editors and Affiliations

  1. CIC, Instituto Politécnico Nacional, Mexico City, Mexico

    Alexander Gelbukh

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Schuller, B.W. (2018). Reading the Author and Speaker: Towards a Holistic and Deep Approach on Automatic Assessment of What is in One’s Words. In: Gelbukh, A. (eds) Computational Linguistics and Intelligent Text Processing. CICLing 2017. Lecture Notes in Computer Science(), vol 10762. Springer, Cham. https://doi.org/10.1007/978-3-319-77116-8_20

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-77116-8_20

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-77115-1

  • Online ISBN: 978-3-319-77116-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation