Abstract
In this paper, we address the modality integration issue on the example of a smart room environment aiming at enabling person identification by combining acoustic features and 2D face images. First we introduce the monomodal audio and video identification techniques and then we present the use of combined input speech and face images for person identification. The various sensory modalities, speech and faces, are processed both individually and jointly. It’s shown that the multimodal approach results in improved performance in the identification of the participants.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Casas, J., Stiefelhagen, R., et al.: Multi-camera/multi-microphone system design for continuous room monitoring. CHIL-WP4-D4.1-V2.1-2004-07-08-CO, CHIL Consortium Deliverable D4.1, July (2004)
Nadeu, C., Macho, D., Hernando, J.: Time and frequency filtering of filter-bank energies for robust HMM speech recognition. Speech Communication 34, 93–114 (2001)
Reynolds, D.A.: Robust text-independent speaker identification using Gaussian mixture speaker models. IEEE Transactions on Speech and Audio Processing 3(1), 72–83 (1995)
Kirby, M., Sirovich, L.: Application of the Karhunen-Loeve procedure for the characterization of human faces. IEEE Trans. PAMI 12(1), 103–108 (1990)
Bolle, R.M., Connell, J.H., Pankanti, S., Ratha, N.K., Senior, A.W.: Guide to Biometrics. Springer, New York (2004)
Brunelli, R., Falavigna, D.: Person Identification Using Multiple Cues. IEEE on PAMI 17(10), 955–966 (1995)
Davis, S.B., Mermelstein, P.: Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences. IEEE Transactions on Acoustic, Speech and Signal Processing 28, 357–366 (1980)
Furui, S.: Speaker independent isolated word recognition using dynamic features of speech spectrum. IEEE Transactions ASSP 34, 52–59 (1986)
Messer, K., Matas, J., Kittler, J.V., Luettin, J., Maitre, G.: XM2VTSDB: The extended M2VTS Database. AVBPA99 (1999)
Fox, N.A., Gross, R., Chazal, P., Cohn, J.F., Reilly, R.B.: Person Identification Using Automatic Integration of Speech, Lip and Face Experts. In: Proc. of the ACM SIGMM 2003 Multimedia Biometrics Methods and Applications Workshop, WBMA’03, Berkeley, CA, pp. 25–32. ACM Press, New York (2003)
Indovina, M., Uludag, U., Snelik, R., Mink, A., Jain, A.: Multimodal Biometric Authentication Methods: A COTS Approach. In: Proceedings of the MMUA 2003, Workshop on Multimodal User Authentication, Santa Barbara, CA, pp. 99–106 (2003)
Lucey, S., Chen, T.: Improved Audio-visual Speaker Recognition via the Use of a Hybrid Combination Strategy. In: The 4th International Conference on Audio- and Video- Based Biometric Person Authentication, Guilford, U.K. (2003)
Wang, Y., Wangm, Y., Tan, T.: Combining Fingerprint and Voiceprint Biometrics for Identity Verification: an Experimental Comparison. In: Proceedings of the ICBA, Hong Kong, China, pp. 663–670 (2004)
D. Mostefa, et al.: CLEAR Evaluation Plan v1.1 (2006), http://www.clear-evaluation.org/downloads/chil-clear-v1.1-2006-02-21.pdf
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer Berlin Heidelberg
About this paper
Cite this paper
Luque, J. et al. (2007). Audio, Video and Multimodal Person Identification in a Smart Room. In: Stiefelhagen, R., Garofolo, J. (eds) Multimodal Technologies for Perception of Humans. CLEAR 2006. Lecture Notes in Computer Science, vol 4122. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-69568-4_23
Download citation
DOI: https://doi.org/10.1007/978-3-540-69568-4_23
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-69567-7
Online ISBN: 978-3-540-69568-4
eBook Packages: Computer ScienceComputer Science (R0)