Neurophysiological Closed-Loop Control for Competitive Multi-brain Robot Interaction

  • Bryan Hernandez-CuevasEmail author
  • Elijah Sawyers
  • Landon Bentley
  • Chris Crawford
  • Marvin Andujar
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 962)


This paper discusses a general architecture for multi-party electroencephalography (EEG)-based robot interactions. We explored a system with multiple passive Brain-Computer Interfaces (BCIs) which influence the mechanical behavior of competing mobile robots. Although EEG-based robot control has been previously examined, previous investigations mainly focused on medical applications. Consequently, there is limited work for hybrid control systems that support multi-party, social BCI. Research on multi-user environments, such as gaming, have been conducted to discover challenges for non-medical BCI-based control systems. The presented work aims to provide an architectural model that uses passive BCIs in a social setting including mobile robots. Such structure is comprised of robotic devices able to act intelligently using vision sensors, while receiving and processing EEG data from multiple users. This paper describes the combination of vision sensors, neurophysiological sensors, and modern web technologies to expand knowledge regarding the design of social BCI applications that leverage physical systems.


BCI EEG Competitive Architecture Robotics Multi-party Drones Intelligent 


  1. 1.
    Amai, W., Fahrenholtz, J., Leger, C.: Hands-free operation of a small mobile robot. Auton. Robot. 11(1), 69–76 (2001). Scholar
  2. 2.
    Ariely, D., Berns, G.S.: Neuromarketing: the hope and hype of neuroimaging in business. Nat. Rev. Neurosci. 11(4), 284–292 (2010)CrossRefGoogle Scholar
  3. 3.
    Bradski, G., Kaehler, A.: OpenCV. Dr. Dobbs journal of software tools 3 (2000)Google Scholar
  4. 4.
    Brooks, R.A.: A robust layered control system for a mobile robot. IEEE J. Robot. Autom. (1986). Scholar
  5. 5.
    Carlson, T., del R. Millan, J.: Brain-controlled wheelchairs: a robotic architecture. IEEE Robot. Autom. Mag. 20(1), 65–73 (2013)CrossRefGoogle Scholar
  6. 6.
    Chanel, G., Rebetez, C., Btrancourt, M., Pun, T.: Emotion assessment from physiological signals for adaptation of game difficulty. IEEE Trans. Syst. Man Cybern. Part A: Syst. Hum. 41(6), 1052–1063 (2011)CrossRefGoogle Scholar
  7. 7.
    Cutrell, E., Tan, D.: BCI for passive input in HCI. In: Proceedings of CHI, vol. 8, pp. 1–3. Citeseer (2008)Google Scholar
  8. 8.
    Escolano, C., Antelis, J.M., Minguez, J.: A telepresence mobile robot controlled with a noninvasive brain-computer interface. IEEE Trans. Syst. Man Cybern. Part B: Cybern. 42(3), 793–804 (2012). Scholar
  9. 9.
    Galn, F., Nuttin, M., Lew, E., Ferrez, P.W., Vanacker, G., Philips, J., del R. Milln, J.: A brain-actuated wheelchair: asynchronous and non-invasive braincomputer interfaces for continuous control of robots. Clin. Neurophysiol. 119(9), 2159–2169 (2008)CrossRefGoogle Scholar
  10. 10.
    Gandhi, V., Prasad, G., Coyle, D., Behera, L., McGinnity, T.M.: EEG-based mobile robot control through an adaptive brain-robot interface. IEEE Trans. Syst., Man, Cybern.: Syst. 44(9), 1278–1285 (2014)., Scholar
  11. 11.
    Grimes, D., Tan, D.S., Hudson, S.E., Shenoy, P., Rao, R.P.: Feasibility and pragmatics of classifying working memory load with an electroencephalograph. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 835–844. ACM (2008)Google Scholar
  12. 12.
    Hjelm, S.I., Browall, C.: Brainball-using brain activity for cool competition. In: Proceedings of NordiCHI, vol. 7 (2000)Google Scholar
  13. 13.
    Iturrate, I., Antelis, J.M., Kubler, A., Minguez, J.: A noninvasive brain-actuated wheelchair based on a p300 neurophysiological protocol and automated navigation. IEEE Trans. Rob. 25(3), 614–627 (2009)CrossRefGoogle Scholar
  14. 14.
    Lin, C.T., Chang, C.J., Lin, B.S., Hung, S.H., Chao, C.F., Wang, I.J.: A real-time wireless braincomputer interface system for drowsiness detection. IEEE Trans. Biomed. Circuits Syst. 4(4), 214–222 (2010)CrossRefGoogle Scholar
  15. 15.
    Lotte, F., Congedo, M., Le´cuyer, A., Lamarche, F., Arnaldi, B.: A review of classification algorithms for eeg-based brain–computer interfaces. J. Neural Eng. 4(2), R1 (2007)CrossRefGoogle Scholar
  16. 16.
    Monajjemi, M.: bebop_autonomy.
  17. 17.
    Müller-putz, G.R., Pereira, J., Ofner, P., Schwarz, A., Dias, C.L., Kobler, R.J., Hehenberger, L., Pinegger, A., Sburlea, A.I.: Towards non-invasive brain-computer interface for hand/arm control in users with spinal cord injury. In: 2018 6th International Conference on Brain-Computer Interface (BCI), pp. 1–4 (2018)Google Scholar
  18. 18.
    Müller-Putz, G.R., Pfurtscheller, G.: Control of an electrical prosthesis with an SSVEP-based BCI. IEEE Trans. Biomed. Eng. 55(1), 361–364 (2008). Scholar
  19. 19.
    Müller-Putz, G.R., Scherer, R., Pfurtscheller, G., Rupp, R.: Eeg-based neuroprosthesis control: a step towards clinical practice. Neurosci. Lett. 382(1), 169–174 (2005)CrossRefGoogle Scholar
  20. 20.
    Nacke, L.E., Kalyn, M., Lough, C., Mandryk, R.L.: Biofeedback game design: using direct and indirect physiological control to enhance game interaction. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 103–112. ACM (2011)Google Scholar
  21. 21.
    Nijholt, A.: Multi-modal and multi-brain-computer interfaces: a review. In: 2015 10th International Conference on Information, Communications and Signal Processing, ICICS 2015 (2016).
  22. 22.
    Nijholt, A., Allison, B.Z., Jacob, R.J.K.: Brain-computer interaction: can multimodality help? In: Proceeding ICMI ’11 Proceedings of the 13th International Conference on Multimodal Interfaces, pp. 35–39 (2011).
  23. 23.
    Nijholt, A., Gürkök, H.: Multi-brain games: cooperation and competition. In: International Conference on Universal Access in Human-Computer Interaction, pp. 652–661. Springer (2013)Google Scholar
  24. 24.
    Nijholt, A., Gürkök, H.: Multi-brain games: cooperation and competition. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) 8009 LNCS (PART 1), pp. 652–661 (2013).
  25. 25.
    Nijholt, A., Reuderink, B., Bos, D.O.: Turning shortcomings into challenges: Brain-computer interfaces for games. Lecture Notes of the Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering, vol. 9, LNICST (2), pp. 153–168 (2009).,
  26. 26.
    Parrot, S.: Parrot bebop 2 (2016). Retrieved from
  27. 27.
    Quigley, M., Conley, K., Gerkey, B., Faust, J., Foote, T., Leibs, J., Wheeler, R., Ng, A.Y.: Ros: an open-source robot operating system. In: ICRA Workshop on Open Source Software, vol. 3, p. 5. Kobe, Japan (2009)Google Scholar
  28. 28.
    Szafir, D., Mutlu, B.: Pay attention!: designing adaptive agents that monitor and improve user engagement. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 11–20. ACM (2012)Google Scholar
  29. 29.
    Tanaka, K., Matsunaga, K., Wang, H.O.: Electroencephalogram-based control of an electric wheelchair. IEEE Trans. Rob. 21(4), 762–766 (2005). Scholar
  30. 30.
    Zander, T.O., Kothe, C.: Towards passive brain–computer interfaces: applying brain–computer interface technology to human–machine systems in general. J. Neural Eng. 8(2), 025005 (2011)CrossRefGoogle Scholar
  31. 31.
    Zander, T.O., Kothe, C., Jatzev, S., Gaertner, M.: Enhancing human-computer interaction with input from active and passive brain-computer interfaces. In: Brain-Computer Interfaces, pp. 181–199. Springer (2010)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Bryan Hernandez-Cuevas
    • 1
    Email author
  • Elijah Sawyers
    • 1
  • Landon Bentley
    • 1
  • Chris Crawford
    • 1
  • Marvin Andujar
    • 2
  1. 1.The University of AlabamaTuscaloosaUSA
  2. 2.University of South FloridaTampaUSA

Personalised recommendations