Skip to main content
SpringerLink
Log in

Cross Modal Evaluation of High Quality Emotional Speech Synthesis with the Virtual Human Toolkit

  • Conference paper
  • First Online:
Intelligent Virtual Agents (IVA 2016)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 10011))

Included in the following conference series:

Abstract

Emotional expression is a key requirement for intelligent virtual agents. In order for an agent to produce dynamic spoken content speech synthesis is required. However, despite substantial work with pre-recorded prompts, very little work has explored the combined effect of high quality emotional speech synthesis and facial expression. In this paper we offer a baseline evaluation of the naturalness and emotional range available by combining the freely available SmartBody component of the Virtual Human Toolkit (VHTK) with CereVoice text to speech (TTS) system. Results echo previous work using pre-recorded prompts, the visual modality is dominant and the modalities do not interact. This allows the speech synthesis to add gradual changes to the perceived emotion both in terms of valence and activation. The naturalness reported is good, 3.54 on a 5 point MOS scale.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    The mapping is currently only available for US accented voices but further accents will become available. Previous studies have shown phoneme-based lip animation is superior to viseme-based approaches [11]. Phone sequences including stress is available from the CereVoice system API and could be incorporated into later releases of VHTK.

References

  1. The semaine project. http://www.semaine-project.eu/

  2. Anagnostopoulos, C.N., Iliou, T., Giannoukos, I.: Features and classifiers for emotion recognition from speech: a survey from 2000 to 2011. Artif. Intell. Rev. 43(2), 155–177 (2015)

    Article  Google Scholar 

  3. Aylett, M.P., Pidcock, C.J.: The cerevoice characterful speech synthesiser SDK. In: Pelachaud, C., Martin, J.-C., André, E., Chollet, G., Karpouzis, K., Pelé, D. (eds.) IVA 2007. LNCS (LNAI), vol. 4722, pp. 413–414. Springer, Heidelberg (2007). doi:10.1007/978-3-540-74997-4_65

    Chapter  Google Scholar 

  4. Aylett, M.P., Pidcock, C.J.: UK patent GB2447263A: Adding and controlling emotion in synthesised speech (2012)

    Google Scholar 

  5. Aylett, M.P., Potard, B., Pidcock, C.J.: Expressive speech synthesis: synthesising ambiguity. In: SSW8, pp. 133–138, Barcelona, Spain, August 2013

    Google Scholar 

  6. Buchholz, S., Latorre, J.: Crowdsourcing preference tests, and how to detect cheating. In: Proceedings of Interspeech, pp. 3053–3056 (2011)

    Google Scholar 

  7. Cowie, R., Douglas-Cowie, E., Savvidou, S., McMahon, E., Sawey, M., Schröder, M.: FEELTRACE: an instrument for recording perceived emotion in real time. In: ITRW on speech and emotion, pp. 19–24 (2000)

    Google Scholar 

  8. Gobl, C., Chasaide, A.N., et al.: The role of voice quality in communicating emotion, mood and attitude. Speech Commun. 40(1), 189–212 (2003)

    Article  MATH  Google Scholar 

  9. Hartholt, A., Traum, D., Marsella, S.C., Shapiro, A., Stratou, G., Leuski, A., Morency, L.-P., Gratch, J.: All together now. In: Aylett, R., Krenn, B., Pelachaud, C., Shimodaira, H. (eds.) IVA 2013. LNCS (LNAI), vol. 8108, pp. 368–381. Springer, Heidelberg (2013). doi:10.1007/978-3-642-40415-3_33

    Chapter  Google Scholar 

  10. Hofer, G.O., Richmond, K., Clark, R.A.: Informed blending of databases for emotional speech synthesis. In: Proceedings of Interspeech (2005)

    Google Scholar 

  11. Mattheyses, W., Latacz, L., Verhelst, W.: Comprehensive many-to-many phoneme-to-viseme mapping and its application for concatenative visual speech synthesis. Speech Commun. 55(7), 857–876 (2013)

    Article  Google Scholar 

  12. Plutchik, R.: The Psychology and Biology of Emotion. Harper Collins College Publishers, New York (1994)

    Google Scholar 

  13. Schlosberg, H.: A scale for the judgement of facial expressions. J. Exp. Psychol. 29(6), 497–510 (1941)

    Article  Google Scholar 

  14. Schröder, M.: Emotional speech synthesis: a review. In: Proceedings Eurospeech, vol. 01, pp. 561–564 (2001)

    Google Scholar 

  15. Schröder, M.: Dimensional emotion representation as a basis for speech synthesis with non-extreme emotions. In: André, E., Dybkjær, L., Minker, W., Heisterkamp, P. (eds.) ADS 2004. LNCS (LNAI), vol. 3068, pp. 209–220. Springer, Heidelberg (2004). doi:10.1007/978-3-540-24842-2_21

    Chapter  Google Scholar 

  16. Schröder, M., Grice, M.: Expressing vocal effort in concatenative synthesis. In: Proceedings of 15th International Conference of Phonetic Sciences, pp. 2589–2592 (2003)

    Google Scholar 

  17. Taylor, P.A., Black, A., Caley, R.: The architecture of the festival speech synthesis system. In: SSW3. pp. 147–151. Jenolan Caves, Australia (1998)

    Google Scholar 

  18. Valbret, H., Moulines, E., Tubach, J.P.: Voice transformation using psola technique. In: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP-1992, vol. 1, pp. 145–148. IEEE (1992)

    Google Scholar 

Download references

Acknowledgements

This research was funded by the Royal Society through a Royal Society Industrial Fellowship and and by the European Union’s Horizon 2020 research and innovation programme under grant agreement No 645378 (ARIA-VALUSPA).

Author information

Authors and Affiliations

  1. CereProc Ltd., Edinburgh, UK

    Blaise Potard, Matthew P. Aylett & David A. Baude

  2. University of Edinburgh, Edinburgh, UK

    Matthew P. Aylett

Authors
  1. Blaise Potard
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Matthew P. Aylett
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. David A. Baude
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Blaise Potard .

Editor information

Editors and Affiliations

  1. University of Southern California, Los Angeles, California, USA

    David Traum

  2. University of Southern California, Los Angeles, California, USA

    William Swartout

  3. US Army Research Laboratory, Los Angeles, California, USA

    Peter Khooshabeh

  4. Universität Bielefeld, Bielefeld, Nordrhein-Westfalen, Germany

    Stefan Kopp

  5. University of Southern California, Los Angeles, California, USA

    Stefan Scherer

  6. University of Southern California, Los Angeles, California, USA

    Anton Leuski

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing AG

About this paper

Cite this paper

Potard, B., Aylett, M.P., Baude, D.A. (2016). Cross Modal Evaluation of High Quality Emotional Speech Synthesis with the Virtual Human Toolkit. In: Traum, D., Swartout, W., Khooshabeh, P., Kopp, S., Scherer, S., Leuski, A. (eds) Intelligent Virtual Agents. IVA 2016. Lecture Notes in Computer Science(), vol 10011. Springer, Cham. https://doi.org/10.1007/978-3-319-47665-0_17

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-47665-0_17

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-47664-3

  • Online ISBN: 978-3-319-47665-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation