Advertisement

Applying the Wizard of Oz technique to the study of multimodal systems

  • Daniel Salber
  • Joëlle Coutaz
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 753)

Abstract

The Wizard of Oz (WOz) technique is an experimental evaluation mechanism. It allows the observation of a user operating an apparently fully functioning system whose missing services are supplemented by a hidden wizard. From our analysis of existing WOz systems, we observe that this technique has primarily been used to study natural language interfaces. With recent advances in interactive media, multimodal user interfaces are becoming popular but our current understanding on how to design such systems is still primitive. In the absence of generalizable theories and models, the WOz technique is an appropriate approach to the identification of sound design solutions. We show how the WOz technique can be extended to the analysis of multimodal interfaces and we formulate a set of requirements for a generic multimodal WOz platform. The Neimo system is presented as an illustration of our early experience in the development of such platforms.

Keywords

Evaluation Expert Client Application Multimodal Interface Multimodal Interaction Multimodal System 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    R. Amalberti and C. Valot: “Le Magicien d'Oz”, CERMA, journée du PRC Rhône-Alpes, 1993.Google Scholar
  2. 2.
    G. Ambone, B. Noz, D. Salber: “Projet Neimo, Spécifications Externes”, internal technical report, équipe IHM, LGI-IMAG, 1992.Google Scholar
  3. 3.
    S. Balbo, J. Coutaz and D. Salber: “Towards Automatic Evaluation of Multimodal User Interfaces”, Workshop on Intelligent User Interfaces '93.Google Scholar
  4. 4.
    P.J. Barnard, “Cognitive Resources and the Learning of Human-Computer Dialogs”, in Interfacing Thought, Cognitive Aspects of Human-Computer Interaction, J.M. Carroll Ed., MIT Press Publ., 1987, pp.112–158Google Scholar
  5. 5.
    S.K. Card, T.P. Moran and A. Newell: “The Psychology of Human-Computer Interaction”, Lawrence Erlbaum Associates, 1983.Google Scholar
  6. 6.
    N. Dahlbäck and A. Jönsson: “Empirical studies of discourse representations for natural language interfaces”, Fourth Conference of the European Chapter of the ACL, proceedings 291-8 1989.Google Scholar
  7. 7.
    N. Dahlbäck, A. Jönsson and L. Ahrenberg: “Wizard of Oz studies — why and how”, Third Conference on Applied Natural Language Processing, Trento, Italy, 31 march–3 April 1992.Google Scholar
  8. 8.
    D. Diaper: “The Wizard's Apprentice: A Program to Help Analyse Natural Language Dialogues”, People and Computers V, proceedings of the 5th Conference of the British Computer Society, 1989.Google Scholar
  9. 9.
    P. Falzon: “Ergonomie Cognitive du Dialogue”, Presses Universitaires Grenobloises, 1989.Google Scholar
  10. 10.
    J.-M. Francony, E. Kuijpers and Y. Polity: “Towards a methodology for Wizard of Oz experiments”, Third Conference on Applied Natural Language Processing, Trento, Italy, 31 march–3 April 1992.Google Scholar
  11. 11.
    N. Fraser, N. Gilbert and C. McDermid: “The Value of Simulation Data”, Third Conference on Applied Natural Language Processing, Trento, Italy, 31 march–3 April 1992.Google Scholar
  12. 12.
    R.S. Fish, R.E. Kraut, R.W. Root: “Evaluating Video as a Technology for Informal Communication”, in the CHI'92 Conference Proceedings, ACM Press Publ., 1992, pp. 37–48Google Scholar
  13. 13.
    A. Gourdol, L. Nigay and D. Salber: “Multimodal Systems: Aspects of Event Fusion and a Taxonomy”, to be published in IFIP Conference '92 Proceedings.Google Scholar
  14. 14.
    R. Guindon and K. Schuldberg: “Grammatical and Ungrammatical Structures in User-Adviser Dialogues; Evidence For Sufficiency of Restricted Languages in Natural Language Interfaces to Advisory Systems”, proceedings of the 25th Annual Meeting of the Association for Computational Linguistics.Google Scholar
  15. 15.
    M.L. Hammontree, J.J. Hendrickson, B.W. Hensley, “Integrated Data Capture and Analysis Tools for Research and Testing on Graphical User Interfaces”, in the CHI'92 Conference Proceedings, ACM Press Publ., 1992, pp. 431–432Google Scholar
  16. 16.
    A.G. Hauptmann: “Speech and Gesture for Graphic Image Manipulation”, CHI '89 Proceedings, pp. 241–245.Google Scholar
  17. 17.
    A. Jönsson and N. Dahlbäck: “Talking to A Computer is Not Like Talking To Your Best Friend”, Proceedings of the Scandinavian Conference on Artificial Intelligence '88, pp. 53–68.Google Scholar
  18. 18.
    D. Kieras, P.G. Polson: “An Approach to the Formal Analysis of User Complexity”, International Journal of Man-Machine Studies, 22, 1985, pp. 365–394Google Scholar
  19. 19.
    C. Lewis, P. Polson, C. Wharton, J. Rieman: “Testing a Walkthrough Methodology for Theory-Based Design of Walk-Up-and-Use Interfaces”, CHI '90 Proceedings, pp. 235–241.Google Scholar
  20. 20.
    J. Löwgren and T.Nordqvist: “A Knowledge-Based Tool for User Interface Evaluation and its Integration in a UIMS”, Human-Computer Interaction—INTERACT '90, pp. 395–400.Google Scholar
  21. 21.
    D. Maulsby, S. Greenberg, R. Mander: “Prototyping an Intelligent Agent through Wizard of Oz”, InterCHI'93 Proceedings, pp. 277–284.Google Scholar
  22. 22.
    Y. Polity, J.-M. Francony, R. Palermiti, P. Falzon, S. Kazma: “Recueil de dialogues homme-machine en langue naturelle écrite”, Les Cahiers du CRISS, n∘ 17, 1990.Google Scholar
  23. 23.
    M. Richards & K. Underwood: “How Should People and Computers Speak to Each Other”, Proceedings of Interact '84 — First IFIP Conference on Human-Computer Interaction, pp. 268–273.Google Scholar
  24. 24.
    A. Sochi & D. Hix, “A study of Computer-Supported User interface Evaluation Using Maximal Repeating Pattern Analysis”, in Proceedings of the CHI'91 Conference, ACM Press, pp. 301–305, 1991.Google Scholar
  25. 25.
    M. Turk and A. Pentland: ”Eigenfaces for recognition”, Journal of Cognitive Neuroscience, Vol. 3, No. 1, pp. 71–86, 1991.Google Scholar
  26. 26.
    S. Whittaker and P. Stenton: “User Studies and the Design of Natural Language Systems”, Fourth Conference of the European Chapter of the ACL, proceedings 291-8 1989.Google Scholar
  27. 27.
    QuickTime Reference Guide, QuickTime CD, Apple Computer Inc.Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 1993

Authors and Affiliations

  • Daniel Salber
    • 1
  • Joëlle Coutaz
    • 1
  1. 1.IMAGLaboratoire de Génie InformatiqueGrenoble CedexFrance

Personalised recommendations