Interpreting Symptoms of Cognitive Load in Speech Input

  • André Berthold
  • Anthony Jameson
Part of the CISM International Centre for Mechanical Sciences book series (CISM, volume 407)


Users of computing devices are increasingly likely to be subject to situationally determined distractions that produce exceptionally high cognitive load. The question arises of how a system can automatically interpret symptoms of such cognitive load in the user’s behavior. This paper examines this question with respect to systems that process speech input. First, we synthesize results of previous experimental studies of the ways in which a speaker’s cognitive load is reflected in features of speech. Then we present a conceptualization of these relationships in terms of Bayesian networks. For two examples of such symptoms—sentence fragments and articulation rate—we present results concerning the distribution of the symptoms in realistic assistance dialogs. Finally, using artificial data generated in accordance with the preceding analyses, we examine the ability of a Bayesian network to assess a user’s cognitive load on the basis of limited observations involving these two symptoms.


Bayesian Network Cognitive Load User Modeling Concurrent Task Dynamic Bayesian Network 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Arnold, S. R. (1997). Wo das Perlhuhn trudelt. Die Zeit 33–34. 6 July 1997.Google Scholar
  2. Baber, C., Mellor, B., Graham, R., Noyes, J. M., and Tunley, C. (1996). Workload and the use of automatic speech recognition: The effects of time and resource demands. Speech Communication 20:37–53.CrossRefGoogle Scholar
  3. Berthold, A. (1998). Repräsentation und Verarbeitung sprachlicher Indikatoren für kognitive Ressourcenbeschränkungen [Representation and processing of linguistic indicators of cognitive resource limitations]. Master’s thesis, Department of Computer Science, University of Saarbrücken, Germany.Google Scholar
  4. Goldman-Eisler, F. (1968). Psycholinguistics: Experiments in Spontaneous Speech. London: Academic Press.Google Scholar
  5. Henrion, M., Pradhan, M., Favero, B. D., Huang, K., Provan, G., and O’Rourke, P. (1996). Why is diagnosis using belief networks insensitive to imprecision in probabilities? In Horvitz, E., and Jensen, F., eds., Proceedings of the Twelfth Conference on Uncertainty in Artificial Intelligence. San Francisco: Morgan Kaufmann. 307–314.Google Scholar
  6. Jameson, A., Schäfer, R., Weis, T., Berthold, A., and Weyrath, T. (1999). Making systems sensitive to the user’s time and working memory constraints. In Maybury, M. T., ed., IUI99: International Conference on Intelligent User Interfaces. New York: ACM. 79–86.Google Scholar
  7. Jameson, A. (1996). Numerical uncertainty management in user and student modeling: An overview of systems and issues. User Modeling and User-Adapted Interaction 5:193–251.CrossRefGoogle Scholar
  8. Jameson, A. (1998). Adapting to the user’s time and working memory limitations: New directions of research. In Timm, U. J., and Rössel, M., eds., ABIS-98, Adaptivität und Benutzermodellierung in interaktiven Softwaresystemen. Erlangen, Germany: FORWISS.Google Scholar
  9. Pearl, J. (1988). Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. San Mateo, CA: Morgan Kaufmann.Google Scholar
  10. Roßnagel, C. (1995a). Kognitive Belastung und Hörerorientierung beim monologischen Instruieren [Cognitive load and listener-orientation in instruction monologs]. Zeitschrift für Experimentelle Psychologie 42:94–110.Google Scholar
  11. Roßnagel, C. (1995b). Übung und Hörerorientierung beim monologischen Instruieren: Zur Differenzierung einer Grundannahme [Practice and listener-orientation in the delivery of instruction monologs: Differentiation of a basic assumption]. Sprache & Kognition 14:16–26.Google Scholar
  12. Russell, S. J., and Norvig, P. (1995). Artificial Intelligence: A Modern Approach. Englewood Cliffs, NJ: Prentice-Hall.MATHGoogle Scholar
  13. Schäfer, R., and Weyrath, T. (1997). Assessing temporally variable user properties with dynamic Bayesian networks. In Jameson, A., Paris, C., and Tasso, C., eds., User Modeling: Proceedings of the Sixth International Conference, UM97. Vienna, New York: Springer Wien New York. 377–388.Google Scholar

Copyright information

© Springer Science+Business Media New York 1999

Authors and Affiliations

  • André Berthold
    • 1
  • Anthony Jameson
    • 1
  1. 1.Department of Computer ScienceUniversity of SaarbrückenGermany

Personalised recommendations