A Generic, Multimodal Framework for Sensorial Feedback on Android Systems

Abstract

The success of software applications, in a worldwide setup offering simple development and distribution models, is often determined by the quality and ease of use of provided interfaces. In this paper, we present a framework for multimodal signal analysis operating in conjunction with any other Android application to estimate the cognitive load imposed by its interface. The framework integrates seamlessly with such applications, even existing ones, by acting as middleware between the sensors layer and the application logic. The interaction between interface elements is mainly targeted in this work: a game presenting an increasingly complex interface was designed and tracking modules for touch events and eye movements were implemented. We show that the framework is able to capture and present raw data together with underlying models estimated by least squares approximation. We then discuss the implications of such a framework for the evaluation of efficient application interfaces.