Vision Based Referee Sign Language Recognition System for the RoboCup MSL League

  • Paulo Trigueiros
  • Fernando Ribeiro
  • Luís Paulo Reis
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8371)


In RoboCup Middle Size league (MSL) the main referee uses assisting technology, controlled by a second referee, to support him, in particular for conveying referee decisions for robot players with the help of a wireless communication system. In this paper a vision-based system is introduced, able to interpret dynamic and static gestures of the referee, thus eliminating the need for a second one. The referee’s gestures are interpreted by the system and sent directly to the Referee Box, which sends the proper commands to the robots. The system is divided into four modules: a real time hand tracking and feature extraction, a SVM (Support Vector Machine) for static hand posture identification, an HMM (Hidden Markov Model) for dynamic unistroke hand gesture recognition, and a FSM (Finite State Machine) to control the various system states transitions. The experimental results showed that the system works very reliably, being able to recognize the combination of gestures and hand postures in real-time. For the hand posture recognition, with the SVM model trained with the selected features, an accuracy of 98,2% was achieved. Also, the system has many advantages over the current implemented one, like avoiding the necessity of a second referee, working on noisy environments, working on wireless jammed situations. This system is easy to implement and train and may be an inexpensive solution.


Hand gesture recognition Support Vector Machine (SVM) Hidden Markov Model (HMM) Finite State Machine (FSM) RoboCup MSL 


  1. 1.
    Kitazumi, Y., Ishii, K.: Survey of cooperative algorithm in robocup middle size league. In: World Automation Congress, WAC 2010, pp. 1–6 (2010)Google Scholar
  2. 2.
    M. T. Committe, Middle Size Robot League Rules and Regulations for 2013 (2013),
  3. 3.
    Trigueiros, P., Ribeiro, F., Reis, L.P.: A Comparative Study of different image features for hand gesture machine learning. In: 5th International Conference on Agents and Artificial Intelligence, Barcelona, Spain (2013)Google Scholar
  4. 4.
    Ben-Hur, A., Weston, J.: A User’s Guide to Support Vector Machines. In: Data Mining Techniques for the Life Sciences, vol. 609, pp. 223–239. Humana Press (2008)Google Scholar
  5. 5.
    Faria, B.M., Lau, N., Reis, L.P.: Classification of Facial Expressions Using Data Mining and machine Learning Algorithms. In: 4a Conferência Ibérica de Sistemas e Tecnologias de Informação, Póvoa de Varim, Portugal, pp. 197–206 (2009)Google Scholar
  6. 6.
    Faria, B.M., Reis, L.P., Lau, N., Castillo, G.: Machine Learning Algorithms applied to the Classification of Robotic Soccer Formations ans Opponent Teams. presented at the IEEE Conference on Cybernetics and Intelligent Systems (CIS), Singapore (2010)Google Scholar
  7. 7.
    Ke, W., Li, W., Ruifeng, L., Lijun, Z.: Real-Time Hand Gesture Recognition for Service Robot, pp. 976–979 (2010)Google Scholar
  8. 8.
    Maldonado-Báscon, S., Lafuente-Arroyo, S., Gil-Jiménez, P., Gómez-Moreno, H.: Road-Sign detection and Recognition Based on Support Vector Machines. IEEE Transactions on Intelligent Transportation Systems, 264–278 (June 2007)Google Scholar
  9. 9.
    Oshita, M., Matsunaga, T.: Automatic learning of gesture recognition model using SOM and SVM. In: Bebis, G., et al. (eds.) ISVC 2010, Part I. LNCS, vol. 6453, pp. 751–759. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  10. 10.
    Almeida, R., Reis, L.P., Jorge, A.M.: Analysis and Forecast of Team Formation in the Simulated Robotic Soccer Domain. In: Lopes, L.S., Lau, N., Mariano, P., Rocha, L.M. (eds.) EPIA 2009. LNCS (LNAI), vol. 5816, pp. 239–250. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  11. 11.
    Trigueiros, P., Ribeiro, F., Reis, L.P.: A comparison of machine learning algorithms applied to hand gesture recognition. In: 7th Iberian Conference on Information Systems and Technologies, Madrid, Spain, pp. 41–46 (2012)Google Scholar
  12. 12.
    Rabiner, L.R.: A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE 77, 257–286 (1989)CrossRefGoogle Scholar
  13. 13.
    Oka, K., Sato, Y., Koike, H.: Real-time fingertip tracking and gesture recognition. IEEE Computer Graphics and Applications 22, 64–71 (2002)CrossRefGoogle Scholar
  14. 14.
    Perrin, S., Cassinelli, A., Ishikawa, M.: Gesture recognition using laser-based tracking system. In: Sixth IEEE International Conference on Automatic Face and Gesture Recognition, Seoul, South Korea, pp. 541–546 (2004)Google Scholar
  15. 15.
    Binh, N.D., Shuichi, E., Ejima, T.: Real-Time Hand Tracking and Gesture Recognition System. In: Proceedings of International Conference on Graphics, Vision and Image, Cairo, Egypt, pp. 362–368 (2005)Google Scholar
  16. 16.
    Chen, F.-S., Fu, C.-M., Huang, C.-L.: Hand gesture recognition using a real-time tracking method and hidden Markov models. Image and Vision Computing 21, 745–758 (2003)CrossRefGoogle Scholar
  17. 17.
    Kelly, D., McDonald, J., Markham, C.: Recognition of Spatiotemporal Gestures in Sign Language Using Gesture Threshold HMMs. In: Wang, L., Zhao, G., Cheng, L., Pietikäinen, M. (eds.) Machine Learning for Vision-Based Motion Analysis, pp. 307–348. Springer, London (2011)CrossRefGoogle Scholar
  18. 18.
    Zafrulla, Z., Brashear, H., Starner, T., Hamilton, H., Presti, P.: American sign language recognition with the kinect. presented at the 13th International Conference on Multimodal Interfaces, Alicante, Spain (2011)Google Scholar
  19. 19.
    Cooper, H., Bowden, R.: Large lexicon detection of sign language. In: Lew, M., Sebe, N., Huang, T.S., Bakker, E.M. (eds.) HCI 2007. LNCS, vol. 4796, pp. 88–97. Springer, Heidelberg (2007)Google Scholar
  20. 20.
    Ribeiro, F., et al.: Robot Orientation with Histograms on MSL. In: Röfer, T., Mayer, N.M., Savage, J., Saranlı, U. (eds.) RoboCup 2011. LNCS, vol. 7416, pp. 507–514. Springer, Heidelberg (2012)Google Scholar
  21. 21.
    Tara, R.Y., Santosa, P.I., Adji, T.B.: Sign Language Recognition in Robot Teleoperation using Centroid Distance Fourier Descriptors. International Journal of Computer Applications 48 (June 2012)Google Scholar
  22. 22.
    Theodoridis, S., Koutroumbas, K.: An Introduction to Pattern Recognition: A Matlab Approach: Academic Press (2010)Google Scholar
  23. 23.
    Theodoridis, S., Koutroumbas, K.: Pattern Recognition, 4th edn. Elsevier (2009)Google Scholar
  24. 24.
    Wu, Y., Huang, T.S.: Vision-Based Gesture Recognition: A Review. In: Braffort, A., Gibet, S., Teil, D., Gherbi, R., Richardson, J. (eds.) GW 1999. LNCS (LNAI), vol. 1739, pp. 103–115. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  25. 25.
    Fink, G.A.: Markov Models for Pattern recognition - From Theory to Applications. Springer (2008)Google Scholar
  26. 26.
    Camastra, F., Vinciarelli, A.: Machine Learning for Audio, Image and Video Analysis. Springer (2008)Google Scholar
  27. 27.
    Alpaydin, E.: Introduction to Machine Learning. MIT Press (2004)Google Scholar
  28. 28.
    Reis, L.P., Lau, N.: COACH UNILANG - A Standard Language for Coaching a (Robo) Soccer Team. In: Birk, A., Coradeschi, S., Tadokoro, S. (eds.) RoboCup 2001. LNCS (LNAI), vol. 2377, pp. 183–192. Springer, Heidelberg (2002)Google Scholar
  29. 29.
    Backus, J.W., Bauer, F.L., Green, J., Katz, C., McCarthy, J., Perlis, A.J., et al.: Revised Report on the Algorithmic Language ALGOL 60. Communications of the ACM, 1–17 (1960, January 1963),
  30. 30.
    Bradski, G., Kaehler, A.: Learning OpenCV: Computer Vision with the OpenCV Library, 1st edn. O’Reilly Media (2008)Google Scholar
  31. 31.
    OpenNI, The standard framework for 3D sensing (2013),
  32. 32.
    Miner, R.: RapidMiner: Report the Future (December 2011),
  33. 33.
    Buckland, M.: Programming Game AI by Example. Wordware Publishing, Inc. (2005)Google Scholar
  34. 34.
    Millington, I., Funge, J.: Artificial Intelligence for Games, 2nd edn. Elsevier (2009)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2014

Authors and Affiliations

  • Paulo Trigueiros
    • 1
  • Fernando Ribeiro
    • 1
  • Luís Paulo Reis
    • 2
  1. 1.Departamento de ElectrónicaIndustrial da Universidade do MinhoGuimarãesPortugal
  2. 2.DSIEEUM – Escola de Engenharia da Universidade do MinhoGuimarãesPortugal

Personalised recommendations