Visual Non-verbal Social Cues Data Modeling

Abstract

Although many methods have been developed in social signal processing (SSP) field during the last decade, several issues related to data management and scalability are still emerging. As the existing visual non-verbal behavior analysis (VNBA) systems are task-oriented, they do not have comprehensive data models, and they are biased towards particular data acquisition procedures, social cues and analysis methods. In this paper, we propose a data model for the visual non-verbal cues. The proposed model is privacy-preserving in the sense that it grants decoupling social cues extraction phase from analysis one. Furthermore, this decoupling allows to evaluate and perform different combinations of extraction and analysis methods. Apart from the decoupling, our model can facilitate heterogeneous data fusion from different modalities since it facilitates the retrieval of any combination of different modalities and provides deep insight into the relationships among the VNBA systems components.