Head Pose Tracking and Focus of Attention Recognition Algorithms in Meeting Rooms

  • Sileye O. Ba
  • Jean-Marc Odobez
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4122)


The paper presents an evaluation of both head pose and visual focus of attention (VFOA) estimation algorithms in a meeting room environment. Head orientation is estimated using a Rao-Blackwellized mixed state particle filter to achieve joint head localization and pose estimation. The output of this tracker is exploited in an Hidden Markov Model (HMM) to estimate people’s VFOA. Contrarily to previous studies on the topic, in our set-up, the potential VFOA of people is not restricted to other meeting participants only, but includes environmental targets (table, slide screen), which renders the task more difficult due to more ambiguity between VFOA target directions. By relying on a corpus of 8 meetings of 8 minutes on average featuring 4 persons involved in the discussion of statements projected on a slide screen, and for which head orientation ground truth was obtained using magnetic sensor devices, we thoroughly assess the performance of the above algorithms, demonstrating the validity of our approaches and pointing out to further research directions.


Ground Truth Hide Markov Model Gaussian Mixture Model Head Rotation Head Orientation 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Ba, S.O., Odobez, J.M.: A rao-blackwellized mixed state particle filter for head pose tracking. In: ACM-ICMI Workshop on Multi-modal Multi-party Meeting Processing (MMMP), Trento Italy, pp. 9–16. ACM, New York (2005)Google Scholar
  2. 2.
    Brown, L., Tian, Y.: A study of coarse head pose estimation. In: IEEE Workshop on Motion and Video Computing (Dec. 2002)Google Scholar
  3. 3.
    Cootes, T., Kittipanya-ngam, P.: Comparing variations on the active appearance model algorithm. In: BMVC (2002)Google Scholar
  4. 4.
    Doucet, A., Godsill, S., Andrieu, C.: On sequential monte carlo sampling methods for bayesian filtering. In: Statistics and Computing (2000)Google Scholar
  5. 5.
    Lu, L., Zhang, Z., Shum, H., Liu, Z., Chen, H.: Model and exemplar-based robust head pose tracking under occlusion and varying expression. In: CVPR (Dec. 2001)Google Scholar
  6. 6.
    McGrath, J.: Groups: Interaction and performance. Prentice-Hall, Englewood Cliffs (1984)Google Scholar
  7. 7.
    Odobez, J.-M.: Focus of attention coding guidelines. Technical Report 2, IDIAP-COM (Jan. 2006)Google Scholar
  8. 8.
    Otsuka, K., Takemae, Y., Yamato, J., Murase, H.: A probabilistic inference of multiparty-conversation structure based on markov-switching models of gaze patterns, head directions, and utterances. In: Proc. of International Conference on Multimodal Interface (ICMI’05), Trento, Italy, pp. 191–198 (Oct. 2005)Google Scholar
  9. 9.
    Parker, K.: Speaking turns in small group interaction: a context sensitive event sequence model. Journal of Personality and Social Psychology (1988)Google Scholar
  10. 10.
    Rae, R., Ritter, H.: Recognition of human head orientation based on artificial neural networks. IEEE Trans. on Neural Network (March 1998)Google Scholar
  11. 11.
    Stiefelhagen, R., Yang, J., Waibel, A.: Modeling focus of attention for meeting indexing based on multiple cues. IEEE Transactions on Neural Networks 13(4) (2002)Google Scholar
  12. 12.
    Toyama, K., Blake, A.: Probabilistic tracking in metric space. In: ICCV (Dec. 2001)Google Scholar
  13. 13.
    Waibel, A., Bett, M., Metze, F., Ries, K., Schaaf, T., Schultz, T., Soltau, H., Yu, H., Zechner, K.: Advances in automatic meeting record creation and access. In: Proc. ICASSP (May 2001)Google Scholar
  14. 14.
    Wang, P., Ji, Q.: Multi-view face tracking with factorial and switching hmm. In: Workshops on Application of Computer Vision (WACV/MOTION’05), Breckenridge, Colorado (2005)Google Scholar
  15. 15.
    Wu, Y., Toyama, K.: Wide range illumination insensitive head orientation estimation. In: IEEE Conf. on Automatic Face and Gesture Recognition (April 2001)Google Scholar
  16. 16.
    Yang, J., Lu, W., Weibel, A.: Skin color modeling and adaptation. In: ACCV (Oct. 1998)Google Scholar
  17. 17.
    Zhao, L., Pingali, G., Carlbom, I.: Real-time head orientation estimation using neural networks. In: Proc. of ICIP (Sept. 2002)Google Scholar

Copyright information

© Springer Berlin Heidelberg 2007

Authors and Affiliations

  • Sileye O. Ba
    • 1
  • Jean-Marc Odobez
    • 1
  1. 1.IDIAP Research Institute, MartignySwitzerland

Personalised recommendations