Abstract
Motor-Imagery based Brain-Computer Interfaces (BCIs) can provide alternative communication pathways to neurologically impaired patients. The combination of BCIs and Virtual Reality (VR) can provide induced illusions of movement to patients with low-level of motor control during motor rehabilitation tasks. Unfortunately, current BCI systems lack reliability and good performance levels in comparison with other types of computer interfaces. To date, there is little evidence on how BCI-based motor training needs to be designed for transferring rehabilitation improvements to real life. Based on our previous work, we showcase the development and assessment of NeuRow, a novel multiplatform immersive VR environment that makes use of multimodal stimulation through vision, sound and vibrotactile feedback and delivered through a VR Head Mounted Display. In addition, we integrated the Adaptive Performance Engine (APE), a statistical approach to optimize user control in a self-paced BCI-VR paradigm. In this paper, we describe the development and pilot assessment of NeuRow as well as its integration and assessment with APE.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Hanakawa, T.: Organizing motor imageries. Neurosci. Res. 1, 56–63 (2015)
Eaves, D.L., Haythornthwaite, L., Vogt, S.: Motor imagery during action observation modulates automatic imitation effects in rhythmical actions. Front. Hum. Neurosci. 8, 28 (2014)
Neuper, C., Scherer, R., Wriessnegger, S., Pfurtscheller, G.: Motor imagery and action observation: modulation of sensorimotor brain rhythms during mental control of a brain–computer interface. Clin. Neurophysiol. 120, 239–247 (2009)
Mulder, T.: Motor imagery and action observation: cognitive tools for rehabilitation. J. Neural Transm. 114, 1265–1278 (2007)
Lledo, P.-M., Alonso, M., Grubb, M.S.: Adult neurogenesis and functional plasticity in neuronal circuits. Nat. Rev. Neurosci. 7, 179–193 (2006)
Rossini, P.M., Calautti, C., Pauri, F., Baron, J.-C.: Post-stroke plastic reorganisation in the adult brain. Lancet Neurol. 2, 493–502 (2003)
Dickstein, R., et al.: Effects of integrated motor imagery practice on gait of individuals with chronic stroke: a half-crossover randomized study. Arch. Phys. Med. Rehabil. 94, 2119–2125 (2013)
Wolpaw, J.R., Birbaumer, N., McFarland, D.J., Pfurtscheller, G., Vaughan, T.M.: Brain-Computer Interfaces for communication and control. Clin. Neurophysiol. Off. J. Int. Fed. Clin. Neurophysiol. 113, 767–791 (2002)
Dobkin, B.H.: Brain-computer interface technology as a tool to augment plasticity and outcomes for neurological rehabilitation. J. Physiol. 579, 637–642 (2007)
Lotte, F., et al.: Combining BCI with virtual reality: towards new applications and improved BCI. In: Allison, B., Dunne, S., Leeb, R., Del, R., Millán, J., Nijholt, A. (eds.) Towards Practical Brain-Computer Interface. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-29746-5_10
Vourvopoulos, A., Niforatos, E., Hlinka, M., Škola, F., Liarokapis, F.: Investigating the effect of user profile during training for BCI-based games. In: 2017 9th International Conference on Virtual Worlds and Games for Serious Applications (VS-Games), pp. 117–124 (2017)
Friedman, D.: Brain-computer interfacing and virtual reality. In: Nakatsu, R., Rauterberg, M., Ciancarini, P. (eds.) Handbook of Digital Games and Entertainment Technologies, pp. 1–22. Springer, Singapore (2015). https://doi.org/10.1007/978-981-4560-52-8_2-1
Slater, M.: Place illusion and plausibility can lead to realistic behaviour in immersive virtual environments. Philos. Trans. R. Soc. Lond. B Biol. Sci. 364, 3549–3557 (2009)
Bermudez i Badia, S., Cameirao, M.S.: The Neurorehabilitation Training Toolkit (NTT): a novel worldwide accessible motor training approach for at-home rehabilitation after stroke. Stroke Res. Treat. 2012 (2012). https://doi.org/10.1155/2012/802157
Lange, B., et al.: Designing informed game-based rehabilitation tasks leveraging advances in virtual reality. Disabil. Rehabil. 34, 1863–1870 (2012)
Trompetto, C., et al.: Pathophysiology of spasticity: implications for neurorehabilitation. BioMed Res. Int. 2014 (2014). https://doi.org/10.1155/2014/354906
Vourvopoulos, A., Bermúdez i Badia, S.: Usability and cost-effectiveness in brain-computer interaction: is it user throughput or technology related? In: Proceedings of the 7th Augmented Human International Conference, Geneva, Switzerland. ACM (2016)
Allison, B.Z., Neuper, C.: Could anyone use a BCI? In: Tan, D.S., Nijholt, A. (eds.) Brain-Computer Interfaces, pp. 35–54. Springer, London (2010). https://doi.org/10.1007/978-1-84996-272-8_3
Vidaurre, C., Blankertz, B.: Towards a cure for BCI illiteracy. Brain Topogr. 23, 194–198 (2009)
Gomez-Rodriguez, M., Peters, J., Hill, J., Schölkopf, B., Gharabaghi, A., Grosse-Wentrup, M.: Closing the sensorimotor loop: haptic feedback facilitates decoding of motor imagery. J. Neural Eng. 8, 036005 (2011)
Hinterberger, T., et al.: A multimodal brain-based feedback and communication system. Exp. Brain Res. 154, 521–526 (2004)
Leeb, R., Gwak, K., Kim, D.-S., del R Millán, J.: Freeing the visual channel by exploiting vibrotactile BCI feedback. In: Conference Proceedings: Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society Annual Conference 2013, pp. 3093–3096 (2013)
Cincotti, F., et al.: Vibrotactile feedback for brain-computer interface operation. Comput. Intell. Neurosci. 2007, e48937 (2007)
Gwak, K., Leeb, R., Millan, J.D.R., Kim, D.-S.: Quantification and reduction of visual load during BCI operation. In: 2014 IEEE International Conference on Systems, Man and Cybernetics (SMC), pp. 2795–2800 (2014)
Jeunet, C., Vi, C., Spelmezan, D., N’Kaoua, B., Lotte, F., Subramanian, S.: Continuous tactile feedback for motor-imagery based brain-computer interaction in a multitasking context. In: Abascal, J., Barbosa, S., Fetter, M., Gross, T., Palanque, P., Winckler, M. (eds.) INTERACT 2015. LNCS, vol. 9296, pp. 488–505. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-22701-6_36
Yao, L., Meng, J., Zhang, D., Sheng, X., Zhu, X.: Combining motor imagery with selective sensation toward a hybrid-modality BCI. IEEE Trans. Biomed. Eng. 61, 2304–2312 (2014)
Leonardis, D., Frisoli, A., Solazzi, M., Bergamasco, M.: Illusory perception of arm movement induced by visuo-proprioceptive sensory stimulation and controlled by motor imagery. In: 2012 IEEE Haptics Symposium (HAPTICS), pp. 421–424 (2012)
Vourvopoulos, A., Bermúdez i Badia, S.: Motor priming in virtual reality can augment motor-imagery training efficacy in restorative brain-computer interaction: a within-subject analysis. J. NeuroEng. Rehabil. 13, 69 (2016)
Lotte, F.: On the need for alternative feedback training approaches for BCI. Presented at the Berlin Brain-Computer Interface Workshop September (2012)
Lotte, F., Larrue, F., Mühl, C.: Flaws in current human training protocols for spontaneous Brain-Computer Interfaces: lessons learned from instructional design. Front. Hum. Neurosci. 7, 568 (2013)
Shute, V.J.: Focus on formative feedback. Rev. Educ. Res. 78, 153–189 (2008)
Hattie, J., Timperley, H.: The power of feedback. Rev. Educ. Res. 77, 81–112 (2007)
Vourvopoulos, A., Ferreira, A., Bermúdez i Badia, S.: NeuRow: an immersive VR environment for motor-imagery training with the use of Brain-Computer Interfaces and vibrotactile feedback. Presented at the PhyCS 2016 - 3rd International Conference on Physiological Computing Systems Lisbon, July 2016
Ferreira, A., Vourvopoulos, A., Bermúdez i Badia, S.: Optimizing performance of non-expert users in brain-computer interaction by means of an adaptive performance engine. In: Guo, Y., Friston, K., Aldo, F., Hill, S., Peng, H. (eds.) BIH 2015. LNCS (LNAI), vol. 9250, pp. 202–211. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-23344-4_20
Klem, G.H., Lüders, H.O., Jasper, H.H., Elger, C.: The ten-twenty electrode system of the International Federation. The International Federation of Clinical Neurophysiology. Electroencephalogr. Clin. Neurophysiol. Suppl. 52, 3–6 (1999)
Renard, Y., et al.: OpenViBE: an open-source software platform to design, test, and use Brain-Computer Interfaces in real and virtual environments. Presence Teleoperators Virtual Environ. 19, 35–53 (2010)
Vourvopoulos, A., Faria, A.L., Cameirao, M.S., Bermudez i Badia, S.: RehabNet: a distributed architecture for motor and cognitive neuro-rehabilitation. In: 2013 IEEE 15th International Conference on e-Health Networking, Applications Services (Healthcom), pp. 454–459 (2013)
Taylor II, R.M., Hudson, T.C., Seeger, A., Weber, H., Juliano, J., Helser, A.T.: VRPN: a device-independent, network-transparent VR peripheral system. In: Proceedings of the ACM Symposium on Virtual Reality Software and Technology, pp. 55–61. ACM, New York (2001)
Pfurtscheller, G., et al.: Graz-BCI: state of the art and clinical applications. IEEE Trans. Neural. Syst. Rehabil. Eng. Publ. IEEE Eng. Med. Biol. Soc. 11, 177–180 (2003)
Roberts, R., Callow, N., Hardy, L., Markland, D., Bringer, J.: Movement imagery ability: development and assessment of a revised version of the vividness of movement imagery questionnaire. J. Sport Exerc. Psychol. 30, 200–221 (2008)
Adams, E., Ip, B.: From casual to core: a statistical mechanism for studying gamer dedication. http://www.gamasutra.com/view/feature/131397/from_casual_to_core_a_statistical_.php
Hart, S.G., Staveland, L.E.: Development of NASA-TLX (task load index): results of empirical and theoretical research. In: Hancock, P.A., Meshkati, N. (eds.) Advances in Psychology, pp. 139–183. North-Holland, Amsterdam (1988)
IJsselsteijn, W., Poels, K., de Kort, Y.A.: The Game Experience Questionnaire: development of a self-report measure to assess player experiences of digital games. TU Eindh,. Eindhoven, Netherlands (2008)
Brooke, J.: SUS-A quick and dirty usability scale. Usability Eval. Ind. 189, 194 (1996)
Delorme, A., Makeig, S.: EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J. Neurosci. Methods 134, 9–21 (2004)
Pope, A.T., Bogart, E.H., Bartolome, D.S.: Biocybernetic system evaluates indices of operator engagement in automated task. Biol. Psychol. 40, 187–195 (1995)
Berka, C., et al.: EEG correlates of task engagement and mental workload in vigilance, learning, and memory tasks. Aviat. Space Environ. Med. 78, B231–B244 (2007)
Boostani, R., Moradi, M.H.: A new approach in the BCI research based on fractal dimension as feature and Adaboost as classifier. J. Neural Eng. 1, 212–217 (2004)
Garcia, G.N., Ebrahimi, T., Vesin, J.: Support vector EEG classification in the Fourier and time-frequency correlation domains. In: First International IEEE EMBS Conference on Neural Engineering. Conference Proceedings, pp. 591–594 (2003)
Obermaier, B., Guger, C., Neuper, C., Pfurtscheller, G.: Hidden Markov models for online classification of single trial EEG data. Pattern Recogn. Lett. 22, 1299–1309 (2001)
Solhjoo, S., Moradi, M.: Mental task recognition: a comparison between some of classification methods. In: BIOSIGNAL 2004 International EURASIP Conference, pp. 24–26 (2004)
Bangor, A., Kortum, P., Miller, J.: Determining what individual SUS scores mean: adding an adjective rating scale. J Usability Stud. 4, 114–123 (2009)
Gregg, M., Hall, C., Butler, A.: The MIQ-RS: a suitable option for examining movement imagery ability. Evid. Based Complement. Altern. Med. ECAM. 7, 249–257 (2010)
Sigrist, R., Rauter, G., Riener, R., Wolf, P.: Augmented visual, auditory, haptic, and multimodal feedback in motor learning: a review. Psychon. Bull. Rev. 20, 21–53 (2013)
Neuper, C., Scherer, R., Reiner, M., Pfurtscheller, G.: Imagery of motor actions: differential effects of kinesthetic and visual-motor mode of imagery in single-trial EEG. Brain Res. Cogn. Brain Res. 25, 668–677 (2005)
Acknowledgements
This work was supported by the European Commission through the RehabNet project - Neuroscience Based Interactive Systems for Motor Rehabilitation - EC (303891 RehabNet FP7-PEOPLE-2011-CIG), by the MACBIOIDI project financed by the EC through the INTERREG program (MAC/1.1.b/098), by the Fundação para a Ciência e Tecnologia (Portuguese Foundation for Science and Technology) through SFRH/BD/97117/2013, and LARSyS (Laboratório de Robótica e Sistemas em Engenharia e Ciência) through UID/EEA/50009/2013.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Vourvopoulos, A., Ferreira, A., Bermudez i Badia, S. (2019). Development and Assessment of a Self-paced BCI-VR Paradigm Using Multimodal Stimulation and Adaptive Performance. In: Holzinger, A., Pope, A., Plácido da Silva, H. (eds) Physiological Computing Systems. PhyCS PhyCS PhyCS 2016 2017 2018. Lecture Notes in Computer Science(), vol 10057. Springer, Cham. https://doi.org/10.1007/978-3-030-27950-9_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-27950-9_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-27949-3
Online ISBN: 978-3-030-27950-9
eBook Packages: Computer ScienceComputer Science (R0)