Developing a Face Monitoring Robot for a Desk Worker

  • Ryosuke Kondo
  • Yutaka Deguchi
  • Einoshin SuzukiEmail author
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8850)


We have developed an autonomous mobile robot which monitors the face of a desk worker. The robot uses three kinds of information observed with its Kinect to search for the desk worker and adjusts its position for monitoring. The monitoring is based on incremental clustering of the faces. Our experiments revealed that not only Animation Units (AUs) features, which represent deviations from the neutral face, but also the pitch angle of the face normalized in a new way are necessary for a valid clustering under specific conditions. Our robot lost sight of a desk worker only once in experiments for 8 persons for about 50 minutes. The resulting clusters correspond to “yawning”, “smiling”, and “reading” for a half of the desk workers with high NMI (normalized mutual information), which is an evaluation measure often used in clustering.


Human monitoring robot Clustering Face tracking 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aarts, E., Encarnacao, J.: Into Ambient Intelligence. In: True Vision - The Emergence of Ambient Intelligence, pp. 1–16. Springer-Verlag (2006)Google Scholar
  2. 2.
    Aggarwal, C.C., Han, J., Wang, J., Yu, P.S.: A Framework for Clustering Evolving Data Streams. In: Proc. VLDB 2003, pp. 81–92 (2003)Google Scholar
  3. 3.
    Ahlberg, J.: Candide-3 An Updated Parameterised Face. Technical Report LiTH-ISY-R-2326, Dept. Elec. Eng., Linköping University Sweden (2001)Google Scholar
  4. 4.
    Coradeschi, S., et al.: GiraffPlus: Combining Social Interaction and Long Term Monitoring for Promoting Independent Living. In: Proc. HSI, pp. 6–15 (2013)Google Scholar
  5. 5.
    Deguchi, Y., Suzuki, E.: Skeleton Clustering by Autonomous Mobile Robots for Subtle Fall Risk Discovery. In: Andreasen, T., Christiansen, H., Cubero, J.-C., Raś, Z.W. (eds.) ISMIS 2014. LNCS, vol. 8502, pp. 500–505. Springer, Heidelberg (2014)CrossRefGoogle Scholar
  6. 6.
    Deguchi, Y., Takayama, D., Takano, S., Scuturici, V.-M., Petit, J.-M., Suzuki, E.: Multiple-Robot Monitoring System Based on a Service-Oriented DBMS. In: Proc. Seventh ACM International Conference on Pervasive Technologies Related to Assistive Environments (PETRA 2014) (2014)Google Scholar
  7. 7.
    Erna, A., Yu, L., Zhao, K., Chen, W., Suzuki, E.: Facial Expression Data Constructed with Kinect and Their Clustering Stability. In: Ślȩzak, D., Schaefer, G., Vuong, S.T., Kim, Y.-S. (eds.) AMT 2014. LNCS, vol. 8610, pp. 421–431. Springer, Heidelberg (2014)CrossRefGoogle Scholar
  8. 8.
    Fischinger, D., et al.: HOBBIT - The Mutual Care Robot. In: Proc. ASROB (2013)Google Scholar
  9. 9.
    Gripay, Y., Laforest, F., Petit, J.-M.: A Simple (yet Powerful) Algebra for Pervasive Environments. In: Proc. EDBT, pp. 359–370 (2010)Google Scholar
  10. 10.
    Hossny, M., et al.: Low Cost Multimodal Facial Recognition via Kinect Sensors. CISR, Deakin University, Australia, Technical report (2013)Google Scholar
  11. 11.
    Kranen, P., Assent, I., Baldauf, C., Seidl, T.: The ClusTree: Indexing Micro-Clusters for Anytime Stream Mining. Knowledge and Information Systems 29(2), 249–272 (2011)CrossRefGoogle Scholar
  12. 12.
    Lyons, M.J., Akamatsu, S., Kamachi, M., Gyoba, J.: Coding Facial Expressions with Gabor Wavelets. In: Proc. Third International Conference on Face and Gesture Recognition (FG), pp. 200–205 (1998)Google Scholar
  13. 13.
    Rubenstein, L.Z.: Falls in Older People: Epidemiology, Risk Factors and Strategies for Prevention. Age and Ageing, ii37–ii41 (2006)Google Scholar
  14. 14.
    Ruvolo, P., Eaton, E.: ELLA: An Efficient Lifelong Learning Algorithm. In: Proc. ICML 2013, vol. 1, pp. 507–515 (2013)Google Scholar
  15. 15.
    Sebe, N., Lew, M., Sun, Y., Cohen, I., Geners, T., Huang, T.: Authentic Facial Expression Analysis. Image and Vision Computing 25, 1856–1863 (2007)CrossRefGoogle Scholar
  16. 16.
    Seidl, T., Assent, I., Kranen, P., Krieger, R., Herrmann, J.: Indexing Density Models for Incremental Learning and Anytime Classification on Data Streams. In: Proc. EDBT 2009, pp. 311–322 (2009)Google Scholar
  17. 17.
    Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., Kipman, A., Blake, A.: Real-Time Human Pose Recognition in Parts from Single Depth Images. In: Proc. CVPR 2011, pp. 1297–1304 (2011)Google Scholar
  18. 18.
    Suzuki, E., Deguchi, Y., Takayama, D., Takano, S., Scuturici, V.-M., Petit, J.-M.: Towards Facilitating the Development of Monitoring Systems with Low-Cost Autonomous Mobile Robots. In: Kawtrakul, A., Laurent, D., Spyratos, N., Tanaka, Y. (eds.) ISIP 2013. CCIS, vol. 421, pp. 57–70. Springer, Heidelberg (2014)CrossRefGoogle Scholar
  19. 19.
    Valstar, M.F., Jiang, B., Mehu, M., Pantic, M., Schrer, K.: The First Facial Expression Recognition and Analysis Challenge. In: Proc. International Conference on Face and Gesture Recognition (FG), pp. 921–926 (2011)Google Scholar
  20. 20.
    Verma, A., Sharma, L.K.: A Comprehensive Survey on Human Facial Expression Detection. Int’l J. Image Processing 7(2), 171–182 (2013)Google Scholar
  21. 21.
    Yong, C., Sudirman, R., Chew, K.: Facial Expression Monitoring System Using PCA-Bayes Classifier. In: Future Computer Sciences and Application (ICFCSA), pp. 187–191 (2011)Google Scholar
  22. 22.
    Zhang, T., Ramakrishnan, R., Livny, M.: BIRCH: A New Data Clustering Algorithm and its Applications. Data Mining and Knowledge Discovery 1(2), 141–182 (1997)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2014

Authors and Affiliations

  • Ryosuke Kondo
    • 1
  • Yutaka Deguchi
    • 1
  • Einoshin Suzuki
    • 1
    Email author
  1. 1.Department of Informatics, ISEEKyushu UniversityFukuokaJapan

Personalised recommendations