Recognizing Time Pressure and Cognitive Load on the Basis of Speech: An Experimental Study

  • Christian Müller
  • Barbara Großmann-Hutter
  • Anthony Jameson
  • Ralf Rummer
  • Frank Wittig⋆
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 2109)


In an experimental environment, we simulated the situation of a user who gives speech input to a system while walking through an airport. The time pressure on the subjects and the requirement to navigate while speaking were manipulated orthogonally. Each of the 32 subjects generated 80 utterances, which were coded semi-automatically with respect to a wide range of features, such as filled pauses. The experiment yielded new results concerning the effects of time pressure and cognitive load on speech. To see whether a system can automatically identify these conditions on the basis of speech input, we had this task performed for each subject by a Bayesian network that had been learned on the basis of the experimental data for the other subjects. The results shed light on the conditions that determine the accuracy of such recognition.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    André Berthold. Repräsentation und Verarbeitung sprachlicher Indikatoren für cognitive Ressourcenbeschränkungen [Representation and processing of linguistic indicators of cognitive resource limitations]. Master’s thesis, Department of Computer Science, Saarland University, Germany, 1998.Google Scholar
  2. 2.
    André Berthold and Anthony Jameson. Interpreting symptoms of cognitive load in speech input. In Judy Kay, editor, UM99, User Modeling: Proceedings of the Seventh International Conference, pages 235–244. Springer Wien New York, Vienna, 1999.Google Scholar
  3. 3.
    Wray Buntine. A guide to the literature on learning probabilistic networks from data. IEEE Transactions on Knowledge and Data Engineering, 8:195–210, 1996.CrossRefGoogle Scholar
  4. 4.
    Russell Greiner, Adam J. Grove, and Dale Schuurmans. Learning Bayesian nets that perform well. In Dan Geiger and Prakash P. Shenoy, editors, Uncertainty in Artificial Intelligence: Proceedings of the Thirteenth Conference, pages 198–207. Morgan Kaufmann, San Francisco, 1997.Google Scholar
  5. 5.
    Anthony Jameson, Barbara Großmann-Hutter, Leonie March, Ralf Rummer, Thorsten Bohnenberger, and Frank Wittig. When actions have consequences: Empirically based decision making for intelligent user interfaces. Knowledge-Based Systems, 14:75–92, 2001.CrossRefGoogle Scholar
  6. 6.
    Anthony Jameson and Frank Wittig. Leveraging data about users in general in the learning of individual user models. In Bernhard Nebel, editor, Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence. Morgan Kaufmann, San Francisco, CA, 2001.Google Scholar
  7. 7.
    Sharon Oviatt. Multimodal interactive maps: Designing for human performance. Human Computer Interaction, 12:93–129, 1997.CrossRefGoogle Scholar
  8. 8.
    Judea Pearl. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, San Mateo, CA, 1988.Google Scholar
  9. 9.
    Stuart J. Russell and Peter Norvig. Artificial Intelligence: A Modern Approach. Prentice Hall, Englewood Cliffs, NJ, 1995.zbMATHGoogle Scholar
  10. 10.
    Ralph Schäfer and Thomas Weyrath. Assessing temporally variable user properties with dynamic Bayesian networks. In Anthony Jameson, Cécile Paris, and Carlo Tasso, editors, User Modeling: Proceedings of the Sixth International Conference, UM97, pages 377–388. Springer Wien New York, Vienna, 1997.Google Scholar
  11. 11.
    Frank Wittig and Anthony Jameson. Exploiting qualitative knowledge in the learning of conditional probabilities of Bayesian networks. In Craig Boutilier and Moisés Goldszmidt, editors, Uncertainty in Artificial Intelligence: Proceedings of the Sixteenth Conference, pages 644–652. Morgan Kaufmann, San Francisco, 2000.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2001

Authors and Affiliations

  • Christian Müller
    • 1
  • Barbara Großmann-Hutter
    • 2
  • Anthony Jameson
    • 2
  • Ralf Rummer
    • 3
  • Frank Wittig⋆
    • 2
  1. 1.Department of Computational LinguisticsSaarland UniversitySaarland
  2. 2.Department of Computer ScienceSaarland UniversitySaarland
  3. 3.Department of PsychologySaarland UniversitySaarland

Personalised recommendations