Abstract
In this paper a novel method for the automatic command of three degrees of freedom of a robot using mouth gestures is presented. The method uses a set of different pixel–based segmentation algorithms and morphological restrictions in order to extract the mouth area from the frames. A fuzzy inference system is then used in order to produce a small subset of discriminante features for gesture classification. A state machine was designed in order to stabilize the robot command task by using a temporal sliding mean on the detected gestures. Experimental results show that the method is both robust and reliable when operated by different people, and fast enough to keep the detection’s rate in real–time.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Sackier, J.M., Wang, Y.: Robotically assisted laparoscopic surgery from concept to development. Surgical Endoscopy 8(1), 63–66 (1994)
Murioz, V.F., Thorbeck, C.V., DeGabriel, J.G., Lozano, J.F., Sanchez-Badajoz, E., Garcia-Cerezoand, A., Toscano, R., Jimenez-Garrido, A.: A medical robotic assistant for minimally invasive surgery. In: IEEE Int. Conf. Robotics and Automation, San Francisco, CA, USA, pp. 2901–2906. IEEE Computer Society Press, Los Alamitos (2000)
Hong, X., Yao, H., Wan, Y., Chen, R.: A pca based visual dct feature extraction method for lip-reading. In: IIH-MSP, pp. 321–326 (2006)
Arsic, I., Vilagut, R., Thiran, J.P.: Automatic extraction of geometric lip features with application to multi-modal speaker identification. In: ICME. IEEE International Conference on Multimedia and Expo, Toronto, ON, Canada, pp. 161–164. IEEE Computer Society Press, Los Alamitos (2006)
Nishikawa, A., Hosoi, T., Koara, K., Negoro, D., Hikita, A., Asano, S., Kakutani, H., Miyazaki, F., Sekimoto, M., Yasui, M., Miyake, Y., Takiguchi, S., Monden, M.: Face mouse: A novel human-machine interface for controlling the position of a laparoscope. IEEE Trans. on Robotics and Automation 19(5), 825–841 (2003)
Hsu, R., Abdel-Mottaleb, M., Jain, A.: Face detection in color images. IEEE Trans. on Pattern Analysis and Machine Intelligence 24(5), 696–706 (2002)
Eveno, N., Caplier, A., Coulon, P.: A new color transformation for lips segmentation. In: IEEE Fourth Workshop on Multimedia Signal Processing, Cannes, France, pp. 3–8. IEEE Computer Society Press, Los Alamitos (2001)
Liévin, M., Delmas, P., Coulon, P., Luthon, F., Fristot, V.: Automatic lip tracking: Bayesian segmentation and active contours in a cooperative scheme. In: ICMCS (1999)
Wang, S., Lau, W., Leung, S., Liew, A.: Lip segmentation with the presence of beards. In: IEEE International Conference on Acoustics, Speech and Signal Processing, vol. 3, pp. 529–532. IEEE Computer Society Press, Los Alamitos (2004)
Wang, S., Lau, W., Leung, S.: Automatic lip contour extraction from color images. Pattern Recongnition 37(12), 2375–2387 (2004)
Hasanuzzaman, M., Zhang, T., Ampornaramveth, V., Ueno, H.: Gesture-based human-robot interaction using a knowledge-based software platform. Industrial Robot: An International Journal 33(1), 37–49 (2006)
Zelinsky, A., Heinzmann, J.: Human-robot interaction using facial gesture recognition. In: Proceedings of the International Workshop on Robot and Human Communication, pp. 256–261 (November 1996)
Heinzmann, J.: Real-time human face tracking and gesture recognition. Master’s thesis, Universität Karlsruhe, Fakultät für Informatik (1996)
Lewis, L., Powers, D.: Lip feature extraction using red exclusion. In: Trent, W. (ed.) Pan-Sydney Workshop on Visual Information Processing (2001)
Gómez, J., Prieto, F., Redarce, T.: Lips movement segmentation and features extraction in real time. In: CISSE 2006. International Joint Conferences on Computer, Information and Systems Sciences and Engineering (2006)
Eckert, M.: Compensación de movimiento avanzada para codificación de vídeo. PhD thesis, Universidad Politécnica de Madrid (2003)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gómez, J.B., Hernández, J.E., Prieto, F., Redarce, T. (2007). Real–Time Robot Manipulation Using Mouth Gestures in Facial Video Sequences. In: Mele, F., Ramella, G., Santillo, S., Ventriglia, F. (eds) Advances in Brain, Vision, and Artificial Intelligence. BVAI 2007. Lecture Notes in Computer Science, vol 4729. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-75555-5_21
Download citation
DOI: https://doi.org/10.1007/978-3-540-75555-5_21
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-75554-8
Online ISBN: 978-3-540-75555-5
eBook Packages: Computer ScienceComputer Science (R0)