Lip Tracking Using Deformable Models and Geometric Approaches

  • Sumita NainanEmail author
  • Vaishali Kulkarni
Conference paper
Part of the Smart Innovation, Systems and Technologies book series (SIST, volume 106)


Multimodal biometrics addresses the issue of recognizing and validating the identity of a person; however, the issue is for a single modality to be robust enough. Voice, being a simple biometric feature to acquire, and the accompanying movement of the lips being distinct for every person can stand up to this challenge. Tracking Lip Movement in real time can be an important biometric trait for Person Recognition and Authentication. A biometric system should be robust and secure especially when they must be deployed in vital domains. In this paper, we have used three different methods to draw the contours around the lips to detect the lip edges to establish lip movement. In the first method, dynamic lip edge patterns were drawn and simultaneously saved in a database created for each person. Secondly, Lip contours were created using Active Snakes models while in the third technique, the motion extraction of lip was implemented using edge detection algorithm. In the work presented here, we have segmented the lip region from the facial images, and have implemented and compared three different approaches to contour the lip region. However, no single model fits every application due to the various face poses and unpredicted lip movements. The target application should be the deciding factor for the consideration of lip movement as the biometric modality along with the voice biometric trait.


ASR Lip movement Active contours Segmentation Edge detection 



The consent to use the image for this work has been obtained as the image used is of the author and there is no objection in publishing this work.


  1. 1.
    Nainan, S., Kulkarni, V.: A comparison of performance evaluation of ASR for noisy and enhanced signal using GMM. In: International Conference on Computing and Security Trends. IEEE, Pune (2016)Google Scholar
  2. 2.
    Wang, Liu: A new spectral image assessment based on energy of structural distortion.In: International Conference on Image Analysis and Signal Processing. IEEE (2009)Google Scholar
  3. 3.
    Matsui, T., Furui, S.: Concatenated phoneme models for text-variable speaker recognition. In: Proceedings International Conference on Acoustics, Speech and Signal Processing, ICSLP, pp. 391–394 (1993)Google Scholar
  4. 4.
    Chetty, G., Wagner, M.: Automated lip feature extraction for liveness verification in audio-video authentication. HCC Laboratory University of Canberra, Australia (2004)Google Scholar
  5. 5.
    Shen, X., Wei, W.: An algorithm of lips secondary positioning and feature extraction based on YCbCr color space. In: International Conference on advances in Mechanical Engineering and Industrial Informatics AMEII (2015)Google Scholar
  6. 6.
    Shiraishi, J., Saitoh, T.: Optical Flow based Lip Reading using Non Rectangular ROI and Head Motion Reduction. IEEE (2015). ISBN: 971-1-4799-6026-2Google Scholar
  7. 7.
    John Hubert, P., Sheeba, M.S.: Lip and head gesture recognition based PC interface using image processing. Biomed. J. Pharm. J. 8(1), 77–82 (2015)Google Scholar
  8. 8.
    Gurumurthy, S., Tripathy, B.K.: Design and implementation of face recognition system in Matlab using the features of lips. IJISA 4(8–4) (2012)Google Scholar
  9. 9.
    Mehrotra, H., Agrawal, G.: Automatic lip contour tracking and visual character recognition for computerized lip reading. Int. J. Electr. Comput. Energ. Electron. Commun. Eng. 3(4) (2009)Google Scholar
  10. 10.
    Wang, L., Wang, X.: Lip detection and tracking using variance based Harr-like features and Kalman filter. In: Fifth International Conference on Frontier of computer Science and Technology. IEEE (2010). ISBN: 978-0-7695-4139-6/10Google Scholar
  11. 11.
    Craig, B., Harte, N.: Region of Interest Extraction using Colour Based Methods on the CUAVE Database ISSC, Dublin (2009)Google Scholar
  12. 12.
    Ooi, W.C., Jeon, C.: Effective lip localization and tracking and achieving multimodal speech recognition. In: International Conference on Multisensor Fusion and Integration for Intelligent Systems, Seoul Korea (2008)Google Scholar
  13. 13.
    Morade, S., Patnaik, S.: Automatic lip tracking and extraction of lip geometric features for lip reading. Int. J. Mach. Learn. Comput. 3(2) (2013)Google Scholar
  14. 14.
    Arthur, C.: Image processing using active contours model. Thesis (2012)Google Scholar
  15. 15.
    Sanderson, C., Lovell, B.C.: Multi-Region Probabilistic Histograms for Robust and Scalable Identity Inference. Lecture notes in Computer Science (LNCS), vol. 5558, pp. 199–208 (2009)Google Scholar

Copyright information

© Springer Nature Singapore Pte Ltd. 2019

Authors and Affiliations

  1. 1.Department of Electronics and Telecommunication EngineeringMPSTME (SVKM’s NMIMS)MumbaiIndia

Personalised recommendations