Influence of Handshape Information on Automatic Sign Language Recognition
Research on automatic sign language recognition (ASLR) has mostly been conducted from a machine learning perspective. We propose to implement results from human sign recognition studies in ASLR. In a previous study it was found that handshape is important for human sign recognition. The current paper describes the implementation of this conclusion: using handshape in ASLR. Handshape information in three different representations is added to an existing ASLR system. The results show that recognition improves, except for one representation. This refutes the idea that extra (handshape) information will always improve recognition. Results also vary per sign: some sign classifiers improve greatly, others are unaffected, and rare cases even show decreased performance. Adapting classifiers to specific sign types could be the key for future ASLR.
Keywordssign language automatic sign language recognition handshape representation
Unable to display preview. Download preview PDF.
- 3.Derpanis, K.G., Wildes, R.P., Tsotsos, J.K.: Hand gesture recognition within a linguistics-based framework. In: Pajdla, T., Matas, J(G.) (eds.) ECCV 2004. LNCS, vol. 3021, pp. 282–296. Springer, Heidelberg (2004)Google Scholar
- 4.Vogler, C., Metaxas, D.: Handshapes and movements: Multiple-channel american sign language recognition. In: Camurri, A., Volpe, G. (eds.) GW 2003. LNCS (LNAI), vol. 2915, pp. 247–258. Springer, Heidelberg (2004)Google Scholar
- 5.ten Holt, G.A., van Doorn, A.J., de Ridder, H., Reinders, M.J.T., Hendriks, E.A.: Signs in which handshape and hand orientation are either not visible or are only partially visible: What is the consequence for lexical recognition? Sign Language Studies 10(1) (2009)Google Scholar
- 7.ten Holt, G.A., Arendsen, J., de Ridder, H., van Doorn, A.J., Reinders, M.J., Hendriks, E.A.: Sign language perception research for improving automatic sign and gesture recognition. In: SPIE Human Vision and Electronic Imaging XIV, vol. 7240. SPIE, Bellingham (2009)Google Scholar
- 11.Caridakis, G., Diamanti, O., Karpouzis, K., Maragos, P.: Automatix sign language recognition: vision based feature extraction and probabilistic recognition scheme from multiple cues. In: Proceedings of ACM PETRA (2008)Google Scholar