Abstract
This paper introduces improved fission rule depending on SNNR (Signal Plus Noise to Noise Ratio) and fuzzy value for simultaneous multi-modality, and suggests the Fusion User Interface (hereinafter, FUI) including a synchronization between audio-gesture modalities, based on the embedded KSSL (Korean Standard Sign Language) recognizer using the WPS (Wearable Personal Station for the next generation PC) and Voice-XML. Our approach fuses and recognizes 62 sentential and 152 word language models that are represented by speech and KSSL, then translates recognition results that is fissioned according to a weight decision rule into synthetic speech and visual illustration (graphical display by HMD-Head Mounted Display) in real-time. The experimental results, average recognition rates of the FUI for 62 sentential and 152 word language models were 94.33% and 96.85% in clean environments (e.g. office space), and 92.29% and 92.91% were shown in noisy environments.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Multimodal Interaction Activity.: Extending the Web to support multiple modes of interaction, http://www.w3.org/2002/mmi/
Fuchs, M., Hejda, P., Slavík, P.: Architecture of Multi-modal Dialogue System. In: Sojka, P., Kopeček, I., Pala, K. (eds.) TSD 2000. LNCS (LNAI), vol. 1902, pp. 433–438. Springer, Heidelberg (2000)
Wooldridge, M.J., Jennings, N.R.: Intelligent agents: Theory and practice, Know. Eng. Review 10(2), 115–152 (1995)
Fagin, R., Halpern, J.Y., Moses, Y., Vardi, M.Y.: Reasoning about Knowledge. MIT Press, Cambridge (1995)
McGlashan, S., et al.: Voice Extensible Markup Language (VoiceXML) Version 2.0. W3C Recommendation (1992), http://www.w3.org
Kim, J.-H., Kim, D.-G., Shin, J.-H., Lee, S.-W., Hong, K.-S.: Hand Gesture Recognition System Using Fuzzy Algorithm and RDBMS for Post PC. In: Wang, L., Jin, Y. (eds.) FSKD 2005. LNCS (LNAI), vol. 3614, pp. 170–175. Springer, Heidelberg (2005)
i.MX21 Processor Data-sheet, http://www.freescale.com/
Duda, R.O., et al.: Pattern Classification, 2nd edn. Wiley, New York (2001)
Paulus, D., Hornegger, J.: Applied Pattern Recognition, 2nd edn. Vieweg, Wiesbaden (1998)
Schuermann, J.: Pattern Classification, A Unified View of Statistical and Neural Approaches. Wiley & Sons, Chichester (1996)
Kim, S.-G.: Korean Standard Sign Language Tutor, 1st edn. Osung, Seoul (2000)
Chen, C.H.: Fuzzy Logic and Neural Network Handbook, 1st edn. McGraw-Hill, New York (1992)
Vasantha Kandasamy, W.B.: Smaranda Fuzzy Algebra. American Research Press, Rehoboth (2003)
NIOSH working group.: STRESS...AT WORK NIOSH, Publication No. 99-101,U.S. National Institutes of Occupational Health (2006)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer Berlin Heidelberg
About this paper
Cite this paper
Kim, JH., Hong, KS. (2007). An Improved Fusion Design of Audio-Gesture for Multi-modal HCI Based on Web and WPS. In: Lee, YH., Kim, HN., Kim, J., Park, Y., Yang, L.T., Kim, S.W. (eds) Embedded Software and Systems. ICESS 2007. Lecture Notes in Computer Science, vol 4523. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-72685-2_29
Download citation
DOI: https://doi.org/10.1007/978-3-540-72685-2_29
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-72684-5
Online ISBN: 978-3-540-72685-2
eBook Packages: Computer ScienceComputer Science (R0)