Towards Usability Evaluation of Multimodal Assistive Technologies Using RGB-D Sensors

  • José Alberto Fuentes
  • Miguel Oliver
  • Francisco Montero
  • Antonio Fernández-Caballero
  • Miguel Angel Fernández
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 7931)


To date there are many solutions in the field of assistive technologies addressing different kinds of disabilities. Each solution has opted for very specific (and incompatible) hardware and software technologies. Recently, new devices initially destined to electronic entertainment are appearing. They have joined in a single sensor various types of technologies typical for assistance. In this paper, we show and evaluate how RGB-D sensors are capable of replacing traditional heterogeneous technologies and a single device covers several products in the field of multimodal human-computer interaction and assistive technologies. Furthermore, a prototype of a software equivalent to a traditional assistive technology product is shown.


Assistive technology RGB-D sensor Human-computer interaction Multimodal interface 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Arumugam, D., Purushothaman, S.: Emotion classification using facial expression. International Journal of Advanced Computer Science and Applications 2(7), 92–98 (2011)Google Scholar
  2. 2.
    Brooke, J.: SUS: a “quick and dirty” usability scale. Usability Evaluation in Industry. Taylor & Francis (1986)Google Scholar
  3. 3.
    Carneiro, D., Castillo, J.C., Novais, P., Fernández-Caballero, A.: Multimodal behavioural analysis for non-invasive stress detection. Expert Systems with Applications 39(18), 13376–13389 (2012)CrossRefGoogle Scholar
  4. 4.
    Castillo, J.C., Carneiro, D., Serrano-Cuerda, J., Novais, P., Fernández-Caballero, A., Neves, J.: A multi-modal approach for activity classification and fall detection. International Journal of Systems Science (in press, 2013)Google Scholar
  5. 5.
    Chang, Y.J., Chen, S.F., Chuang, A.F.: A gesture recognition system to transition autonomously through vocational tasks for individuals with cognitive impairments. Research in Developmental Disabilities 32(6), 2064–2068 (2011)CrossRefGoogle Scholar
  6. 6.
    Costa, A., Castillo, J.C., Novais, P., Fernández-Caballero, A., Simoes, R.: Sensor-driven agenda for intelligent home care of the elderly. Expert Systems with Applications 39(15), 12192–12204 (2012)CrossRefGoogle Scholar
  7. 7.
    Doukas, C., Metsis, V., Becker, E., Le, Z., Makedon, F., Maglogiannis, I.: Digital cities of the future: extending home assistive technologies for the elderly and the disabled. Telematics and Informatics 28(3), 176–190 (2011)CrossRefGoogle Scholar
  8. 8.
    enPathia - Eneso - Tecnología para personas con discapacidad,
  9. 9.
    Fernández-Caballero, A., Castillo, J.C., Rodríguez-Sánchez, J.M.: Human activity monitoring by local and global finite state machines. Expert Systems with Applications 39(8), 6982–6993 (2012)CrossRefGoogle Scholar
  10. 10.
    Harada, S., Wobbrock, J.O., Landay, J.A.: Voicedraw: a hands-free voice-driven drawing application for people with motor impairments. In: Ninth Annual ACM Conference on Assistive Technologies 2007, pp. 27–34 (2007)Google Scholar
  11. 11.
    Henry, P., Krainin, M., Herbst, E., Ren, X., Fox, D.: RGB-D mapping: using Kinect-style depth cameras for dense 3D modeling of indoor environments. The International Journal of Robotics Research 31(5), 647–663 (2012)CrossRefGoogle Scholar
  12. 12.
    Hernández-López, J.J., Quintanilla-Olvera, A.L., López-Ramírez, J.L., Rangel-Butanda, F.J., Ibarra-Manzano, M.A., Almanza-Ojeda, D.L.: Detecting objects using color and depth segmentation with Kinect sensor. Procedia Technology 3, 196–204 (2012)CrossRefGoogle Scholar
  13. 13.
    ISO/IEC 25062: Software engineering – Software product Quality Requirements and Evaluation (SQuaRE) – Common Industry Format (CIF) for usability test reports (2006)Google Scholar
  14. 14.
    López-Jaquero, V., Montero, F., Molina, J.P., González, P., Fernández-Caballero, A.: A seamless development process of adaptive user interfaces explicitly based on usability properties. In: Feige, U., Roth, J. (eds.) EHCI-DSVIS 2004. LNCS, vol. 3425, pp. 289–291. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  15. 15.
    López-Jaquero, V., Montero, F., Molina, J.P., Fernández-Caballero, A., González, P.: Model-based design of adaptive user interfaces through connectors. In: Jorge, J.A., Jardim Nunes, N., Falcão e Cunha, J. (eds.) DSV-IS 2003. LNCS, vol. 2844, pp. 245–257. Springer, Heidelberg (2003)CrossRefGoogle Scholar
  16. 16.
    Microsoft Corporation: Microsoft Kinect for Windows SDK - V1.0 Release notes (2012),
  17. 17.
    Nielsen, J.: Why You Only Need to Test with 5 Users (2000),
  18. 18.
    Roccetti, M., Marfia, G., Semeraro, A.: Playing into the wild: a gesture-based interface for gaming in public spaces. Journal of Visual Communication and Image Representation 23(3), 426–440 (2012)CrossRefGoogle Scholar
  19. 19.
    Schwarz, L.A., Mkhitaryan, A., Mateus, D., Navab, N.: Human skeleton tracking from depth data using geodesic distances and optical flow. Image and Vision Computing 30(3), 217–226 (2012)CrossRefGoogle Scholar
  20. 20.
    Sirohey, S., Rosenfeld, A., Duric, Z.: A method of detecting and tracking irises and eyelids in video. Pattern Recognition 35(6), 1389–1401 (2002)zbMATHCrossRefGoogle Scholar
  21. 21.
    Sony Computer Entertainment Inc.: User-driven three-dimensional interactive gaming environment (2011),
  22. 22.
    Zhang, Q., Song, X., Shao, X., Shibasaki, R., Zhao, H.: Unsupervised skeleton extraction and motion capture from 3D deformable matching. Neurocomputing 100, 170–182 (2012)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • José Alberto Fuentes
    • 1
  • Miguel Oliver
    • 1
  • Francisco Montero
    • 1
    • 2
  • Antonio Fernández-Caballero
    • 1
    • 2
  • Miguel Angel Fernández
    • 1
    • 2
  1. 1.Instituto de Investigación en Informática de Albacete (I3A)AlbaceteSpain
  2. 2.Departamento de Sistemas InformáticosUniversidad de Castilla-La ManchaAlbaceteSpain

Personalised recommendations