A Wearable Device for Brain–Machine Interaction with Augmented Reality Head-Mounted Display

  • Mattia SalvaroEmail author
  • Simone Benatti
  • Victor Kartsch
  • Marco Guermandi
  • Luca Benini
Conference paper
Part of the EAI/Springer Innovations in Communication and Computing book series (EAISICC)


Brain–computer interfaces (BCIs) have started to enter the consumer market with appealing head-mounted devices, primarily aiming at entertainment applications. However, the accuracy and information rate of those devices prevent their employment in fields where reliability and real-time constraints are stronger, such as robotic control and closed-loop human–machine interaction (HMI). In this paper, we present an entirely wearable, hands-free, embedded BCI system based on steady state visual evoked potentials (SSVEPs), integrating augmented reality (AR) for stimuli presentation, with a custom low-power board for EEG signal acquisition and real-time processing. The system has been tested on five subjects with four target stimuli, achieving high overall accuracy (80%) and an average transfer rate of 0.42 bits per second.


EEG SSVEP Brain–computer interface Augmented reality 



This work was supported by the EU H2020 project “OPRECOMP.OPEN TRANSPRECISION COMPUTING” (grant no. 732631)


  1. 1.
  2. 2.
    Latré, B., et al.: A survey on wireless body area networks. Wirel. Netw. 17(1), 1–18 (2011)CrossRefGoogle Scholar
  3. 3.
    Nick, T.A., Berman, L.M., Barnehama, A.Z.: Personalized neuroscience: user modeling of cognitive function and brain activity in the cloud. In: Proceedings of the 10th EAI International Conference on Body Area Networks, ICST (Institute for Computer Sciences, Social-Informatics and Telecommunications Engineering), pp. 78–84 (2015)Google Scholar
  4. 4.
    Benatti, S., Milosevic, B., Tomasini, M., Farella, E., Schoenle, P., Bunjaku, P., Rovere, G., Fateh, S., Huang, Q., Benini, L.: Multiple biopotentials acquisition system for wearable applications. In: SmartMedDev BIODEVICES, pp. 260–268 (2015)Google Scholar
  5. 5.
    Kartsch, V., Benatti, S., Rossi, D., Benini, L.: A wearable EEG-based drowsiness detection system with blink duration and alpha waves analysis. In: 2017 8th International IEEE/EMBS Conference on, IEEE, Neural Engineering (NER), pp. 251–254 (2017)Google Scholar
  6. 6.
    Kartsch, V.J., Benatti, S., Schiavone, P.D., Rossi, D., Benini, L.: A sensor fusion approach for drowsiness detection in wearable ultra-low-power systems. Inform. Fusion 43, 66–76 (2018)CrossRefGoogle Scholar
  7. 7.
    Farwell, L.A., Donchin, E.: Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalogr. Clin. Neurophysiol. 70(6), 510–523 (1988)CrossRefGoogle Scholar
  8. 8.
    Blankertz, B., Tangermann, M., Vidaurre, C., Fazli, S., Sannelli, C., Haufe, S., Maeder, C., Ramsey, L.E., Sturm, I., Curio, G., et al.: The Berlin brain–computer interface: non-medical uses of BCI technology. Front. Neurosci. 4, 198 (2010)CrossRefGoogle Scholar
  9. 9.
    Mindmaze, Mindmaze Mask, (2018)
  10. 10.
    Neurosky, Neurosky MindWave, (2018)
  11. 11.
  12. 12.
    OpenBCI, (2018)
  13. 13.
  14. 14.
    Sur, S., Sinha, V.: Event-related potential: an overview. Ind. Psychiatry J. 18(1), 70 (2009)CrossRefGoogle Scholar
  15. 15.
    Nicolas-Alonso, L.F., Gomez-Gil, J.: Brain computer interfaces, a review. Sensors 12(2), 1211–1279 (2012)CrossRefGoogle Scholar
  16. 16.
    Salvaro, M., Kartsch, V., Benatti, S., Milano, M., Benini, L.: Towards a novel HMI paradigm based on mixed EEG and indoor localization platforms, in: 2017 New Generation of CAS (NGCAS), pp. 217–220. IEEE, Piscataway (2017)Google Scholar
  17. 17.
    Guger, C., Holzner, C., Grönegress, C., Edlinger, G., Slater, M.: Control of a smart home with a brain-computer interface, Citeseer (2008)Google Scholar
  18. 18.
    Kim, Y., Kaongoen, N., Jo, S.: Hybrid-BCI smart glasses for controlling electrical devices. In: 2015 54th Annual Conference of the Society of Instrument and Control Engineers of Japan (SICE), pp. 1162–1166. IEEE, Piscataway (2015)Google Scholar
  19. 19.
    Saboor, A., Rezeika, A., Stawicki, P., Gembler, F., Benda, M., Grunenberg, T., Volosyak, I.: SSVEP-based BCI in a smart home scenario. In: International Work-Conference on Artificial Neural Networks, pp. 474–485. Springer, Berlin (2017)Google Scholar
  20. 20.
  21. 21.
    Yang, P., Fan, C., Wang, M., Li, L.: A comparative study of average, linked mastoid, and REST references for ERP components acquired during fMRI. Front. Neurosci. 11, 247 (2017)CrossRefGoogle Scholar
  22. 22.
    Vuforia, Vuforia SDK for digital eyewear, (2018)
  23. 23.
    Lin, Z., Zhang, C., Wu, W., Gao, X.: Frequency recognition based on canonical correlation analysis for SSVEP-based BCIs. IEEE Trans. Biomed. Eng. 53(12), 2610–2614 (2006)CrossRefGoogle Scholar
  24. 24.
    Golub, G.H.: Matrix decompositions and statistical calculations. In: Statistical Computation, pp. 365–397. Elsevier, Amsterdam (1969)Google Scholar
  25. 25.
  26. 26.
    Zhu, D., Bieger, J., Molina, G.G., Aarts, R.M.: A survey of stimulation methods used in SSVEP-based BCIs. Comput. Intell. Neurosci. 2010, 1 (2010)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  1. 1.DEIUniversity of BolognaBolognaItaly
  2. 2.Energy Efficient Embedded Systems Lab (EEES), DEIUniversity of BolognaBolognaItaly
  3. 3.Integrated System Laboratory (IIS), ETH ZurichZürichSwitzerland

Personalised recommendations