Multimodal Intelligent Wheelchair Interface

  • Filipe CoelhoEmail author
  • Luís Paulo Reis
  • Brígida Mónica Faria
  • Alexandra Oliveira
  • Victor Carvalho
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 1160)


Intelligent wheelchairs allow individuals to move more freely, safely, and facilitate users’ interaction with the wheelchair. This paper presents the results focused on the study and analysis of the state of the art related to topics as interaction, interfaces, intelligent wheelchairs and the analysis of the Intellweelsproject. The main goal is to create and implement a multimodal adaptive interface to be used as the control and interaction module of an intelligent wheelchair. Moreover, it will be important to have in mind the usability, by facilitating the control of a complex system, interactivity, by allowing the control using diverse kinds of input devices, and expansibility, by integrating easily with several intelligent external systems. This project features a complex input/output system with linked parameters simplified by a node system to create the input/output actions with automatic input recording and intuitive output association as well as a powerful, device-agnostic design, providing an easy way to extend the inputs, outputs and event the user interface. Results reveal a positive users’ feedback and a responsive way when using the multimodal interface in simulated environment.


Adaptability Intelligent wheelchair Interaction Multimodal interfaces 



This work is supported by project IntellWheels2.0, funded by Portugal2020 (POCI-01-0247-FEDER-039898). This research was partially supported by LIACC (FCT/UID/CEC/0027/2020).


  1. 1.
    Churchill, E.F., Bowser, A., Preece, J.: The future of HCI education: a flexible, global, living curriculum. Interactions 23(2), 70–73 (2016)CrossRefGoogle Scholar
  2. 2.
    Faria, B., Dias, D., Reis, L.: Multimodal interaction robotic devices in a simulated environment. In: 11th Iberian Conference on Information Systems and Technologies (2016)Google Scholar
  3. 3.
    Faria, B., Dias, D., Reis, L., Moreira, A.: Multimodal interaction and serious game for assistive robotic devices in a simulated environment. In: Cunha, B., Lima, J., Silva, M., Leitao, P. (eds.) 2016 IEEE International Conference on Autonomous Robot Systems and Competitions, ICARSC, pp. 253–258 (2016)Google Scholar
  4. 4.
    Faria, B., Ferreira, L., Reis, L., Lau, N., Petry, M.: Intelligent wheelchair manual control methods a usability study by cerebral palsy patients. In: Correia, L., Reis, L., Cascalho, J. (eds.) EPIA 2013. Lecture Notes in AI, vol. 8154, pp. 271–282 (2013)Google Scholar
  5. 5.
    Faria, B., Reis, L., Lau, N.: Cerebral palsy EEG signals classification: facial expressions and thoughts for driving an intelligent wheelchair. In: Vreeken, J., Ling, C., Zaki, M., Siebes, A., Yu, J., Goethals, B., Webb, G., Wu, X. (eds.) 12TH IEEE International Conference on Data Mining Workshops, ICDM Workshops, pp. 33–40 (2012)Google Scholar
  6. 6.
    Faria, B., Reis, L., Lau, N.: Manual, automatic and shared methods for controlling an intelligent wheelchair adaptation to cerebral palsy users. In: Calado, J., Reis, L., Rocha, R. (eds.) 13th International Conference on Autonomous Robot Systems (2013)Google Scholar
  7. 7.
    Faria, B., Reis, L., Lau, N.: A survey on intelligent wheelchair prototypes and simulators. In: Rocha, A., Correia, A., Tan, F., Stroetmann, K. (eds.) New Perspectives in Information Systems and Technologies, volume 1, Advance in Intelligent Systems and Computing, vol. 275, pp. 545–557 (2014)Google Scholar
  8. 8.
    Faria, B., Reis, L., Lau, N.: User modeling and command language adapted for driving an intelligent wheelchair. In: Lau, N., Moreira, A., Ventura, R., Faria, B. (eds.) 2014 IEEE International Conference on Autonomous Robot Systems and Competitions, pp. 158–163 (2014)Google Scholar
  9. 9.
    Faria, B., Reis, L., Lau, N.: A methodology for creating an adapted command language for driving an intelligent wheelchair. J. Intell. Robot. Syst. 80(3–4), 609–623 (2015)CrossRefGoogle Scholar
  10. 10.
    Faria, B., Reis, L., Lau, N.: Adapted control methods for cerebral palsy users of an intelligent wheelchair. J. Intell. Robot. Syst. 77(2), 299–312 (2015)CrossRefGoogle Scholar
  11. 11.
    Faria, B., Reis, L., Lau, N., Moreira, A., Petry, M., Ferreira, L.: Intelligent wheelchair driving: bridging the gap between virtual and real intelligent wheelchairs. In: Pereira, F., Machado, P., Costa, E., Cardoso, A. (eds.) Progress in AI. Lecture Notes in Artificial Intelligence, vol. 9273, pp. 445–456 (2015)Google Scholar
  12. 12.
    Faria, B., Reis, L., Lau, N., Soares, J., Vasconcelos, S.: Patient classification and automatic configuration of an intelligent wheelchair. In: Fred, F.J. (ed.) Communication in Computer and Information Science, vol. 358, pp. 268–282 (2013)Google Scholar
  13. 13.
    Faria, B., Vasconcelos, S., Reis, L., Lau, N.: A methodology for creating intelligent wheelchair users’ profiles. In: Filipe, J., Fred, A. (eds.) ICAART: 4th International Conference on Agents and Artificial Intelligence, volume 1. ICAART (1), vol. 1, pp. 171–179 (2012)Google Scholar
  14. 14.
    Faria, B., Vasconcelos, S., Reis, L., Lau, N.: Evaluation of distinct input methods of an intelligent wheelchair in simulated and real environments: a performance and usability study. Assist. Technol. 25(2), 88–98 (2013)CrossRefGoogle Scholar
  15. 15.
    Hasan, M.S., Yu, H.: Innovative developments in HCI and future trends. Int. J. Autom. Computing 14(1), 10–20 (2017). Scholar
  16. 16.
    Johnson, J., Card, S.K.: Foreword. In: Designing with the Mind in Mind, pp. ix–xi, January 2014.
  17. 17.
    Kammer, D., McNutt, G., Senese, B., Bray, J.: Chapter 9 - designing an audio application. In: Bluetooth Application Developer’s Guide, pp. 379 – 417. Syngress, Burlington (2002)Google Scholar
  18. 18.
    Karpov, A.A., Yusupov, R.M.: Multimodal interfaces of human-computer interaction. Herald Russ. Acad. Sci. 88(1), 67–74 (2018)CrossRefGoogle Scholar
  19. 19.
    Linietsky, J., Manzur, A.: Free and open source 2D and 3D game engine.
  20. 20.
    Park, K., Lee, S.W.: Model-based approach for engineering adaptive user interface requirements. In: Liu, L., Aoyama, M. (eds.) Requirements Engineering in the Big Data Era, pp. 18–32. Springer, Heidelberg (2015)CrossRefGoogle Scholar
  21. 21.
    Petry, M., Moreira, A., Faria, B., Reis, L.: IntellWheels: intelligent wheelchair with user-centered design. In: 2013 IEEE 15th International Conference on e-Health Networking, Applications and Services, Healthcom 2013, pp. 414–418. Healthcom (2013)Google Scholar
  22. 22.
    Pineau, J., West, R., Atrash, A., Villemure, J., Routhier, F.: On the feasibility of using a standardized test for evaluating a speech-controlled smart wheelchair. Int. J. Intell. Control Syst. 16(2), 124–131 (2011)Google Scholar
  23. 23.
    Reeves, L., Lai, J., Larson, J.A., Oviatt, S., Balaji, T., Buisine, S., Collings, P., Cohen, P., Kraal, B., Martin, J.C., McTear, M., Stanney, K.M., Su, H., Wang, Q.Y.: Guidelines for multimodal user interface design. Commun. ACM 47(1), 57–59 (2004)CrossRefGoogle Scholar
  24. 24.
    Reis, L., Faria, B., Vasconcelos, S., Lau, N.: Multimodal interface for an intelligent wheelchair. In: Ferrier, J., Gusikhin, O., Madani, K., Sasiadek, J. (eds.) Informatics in Control, Automation and Robotics. Lecture Notes in Electrical Engineering, vol. 325, pp. 1–34 (2015)Google Scholar
  25. 25.
    Vanhooydonck, D., Demeester, E., Huntemann, A., Philips, J., Vanacker, G., Brussel, H.V., Nuttin, M.: Adaptable navigational assistance for intelligent wheelchairs by means of an implicit personalized user model. Robot. Auton. Syst. 58(8), 963–977 (2010). Scholar
  26. 26.
    Zhang, P.: Chapter 13 - Human -machine interfaces. In: Advanced Industrial Control Technology, pp. 527–555. William Andrew Publishing, January 2010Google Scholar

Copyright information

© The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Filipe Coelho
    • 1
    • 2
    Email author
  • Luís Paulo Reis
    • 1
  • Brígida Mónica Faria
    • 1
    • 3
  • Alexandra Oliveira
    • 1
    • 3
  • Victor Carvalho
    • 2
  1. 1.Laboratório de Inteligência Artificial e Ciência de Computadores (LIACC), Faculdade de Engenharia da Universidade do PortoPortoPortugal
  2. 2.OptimizerPortoPortugal
  3. 3.Escola Superior de Saúde do Instituto Politécnico do Porto (ESS-IPP)PortoPortugal

Personalised recommendations