Facial Landmark Localization and Feature Extraction for Therapeutic Face Exercise Classification
In this work, we examine landmark localization and feature extraction approaches for the unexplored topic of therapeutic facial exercise recognition. Our goal is to automatically discriminate nine therapeutic exercises that have been determined in cooperation with speech therapists. We use colour, 2.5D and 3D image data that was recorded using Microsoft’s Kinect. Our features comprise statistical descriptors of the face surface curvature as well as characteristic profiles that are derived from face landmarks. For the nine facial exercises, we yield an average recognition accuracy of about \(91\,\%\) in conjunction with manually labeled landmarks. Additionally, we introduce a combined method for automatic landmark localization and compare the results to landmark positions obtained from Active Appearance Model fitting as well as manual labeling. The combined localization method exhibits increased robustness in comparison to AAMs.
KeywordsFacial expressions Curvature analysis Point signatures Line profiles Therapeutic exercises
We would like to thank the m&i Fachklinik Bad Liebenstein (in particular Prof. Dr. med. Gustav Pfeiffer, Eva Schillikowski) and Logopädische Praxis Irina Stangenberger, who supported our work by giving valuable insights into rehabilitation and speech-language therapy requirements and praxis. This work is partially funded by the TMBWK ProExzellenz initiative, Graduate School on Image Processing and Image Interpretation.
- 1.Lanz, C., Denzler, J., Gross, H.M.: Facial movement dysfunctions: conceptual design of a therapy-accompanying training system. In: Wichert, R., Klausing, H. (eds.) Ambient Assisted Living - Advanced Technologies and Societal Change. Springer, Heidelberg (2013)Google Scholar
- 5.Gebhard, A., Paulus, D., Suchy, B., Wolf, S.: A system for diagnosis support of patients with facialis paresis. Kl 3/2000, 40–42 (2000)Google Scholar
- 7.Wang, J., Yin, L., Wei, X., Sun, Y.: 3d facial expression recognition based on primitive surface feature distribution. In: International Conference on Computer Vision and Pattern Recognition, vol. 2, pp. 1399–1406 (2006)Google Scholar
- 9.Chua, C.S., Han, F., Ho, Y.K.: 3d human face recognition using point signature. In: Proceedings of the 4th International Automatic Face and Gesture Recognition Conference, pp. 233–238 (2000)Google Scholar
- 12.Salomon, D.: Data Compression: The Complete Reference. Springer, New York (2004)Google Scholar
- 13.Martin, C., Werner, U., Gross, H.M.: A real-time facial expression recognition system based on active appearance models using gray images and edge images. In: International Conference on Automatic Face and Gesture Recognition (2008)Google Scholar
- 16.Zhu, X., Ramanan, D.: Face detection, pose estimation and landmark localization in the wild. In: International Conference for Computer Vision and Pattern Recognition, pp. 2879–2886 (2012)Google Scholar
- 18.Khoshelham, K.: Accuracy analysis of kinect depth data. In: ISPRS Workshop Laser Scanning, vol. 38 (2011)Google Scholar
- 19.Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2, 27:1–27:27 (2011). http://www.csie.ntu.edu.tw/~cjlin/libsvm
- 20.Hsu, C., Chang, C., Lin, C.: A practical guide to support vector classification. TR available at http://www.csie.ntu.edu.tw/~cjlin/papers/guide/guide.pdf (2009)