Improvement and Estimation of Intensity of Facial Expression Recognition for Human-Computer Interaction

Abstract

With the ubiquity of new information technology and media, more effective and friendly methods for human computer interaction (HCI) are being developed. The first step for any intelligent HCI system is face detection and one of the most friendly HCI systems is facial expression recognition. Although Facial Expression Recognition for HCI introduces the frontiers of vision-based interfaces for intelligent human computer interaction, very little has been explored for capturing one or more expressions from mixed expressions which are a mixture of two closely related expressions. This paper presents the idea of improving the recognition accuracy of one or more of the six prototypic expressions namely happiness, surprise, fear, disgust, sadness and anger from the mixture of two facial expressions. For this purpose a motion gradient based optical flow for muscle movement is computed between frames of a given video sequence. The computed optical flow is further used to generate feature vector as the signature of six basic prototypic expressions. Decision Tree generated rule base is used for clustering the feature vectors obtained in the video sequence and the result of clustering is used for recognition of expressions. Manhattan distance metric is used which captures the relative intensity of expressions for a given face present in a frame. Based on the score of intensity of each expression, degree of presence of each of the six basic facial expressions has been determined. With the introduction of Component Based Analysis which is basically computing the feature vectors on the proposed regions of interest on a face, considerable improvement has been noticed regarding recognition of one or more expressions. The results have been validated against human judgement.