Skip to main content
SpringerLink
Log in

Action Games, Motor Imagery, and Control Strategies: Toward a Multi-button Controller

  • Reference work entry
  • First Online:
Handbook of Digital Games and Entertainment Technologies

Abstract

Motor imagery is a skill that can be learned to maximize the accuracy of sensorimotor rhythm (SMR)-based brain-computer game interaction (BCGI). Strategies for learning to intentionally modulate sensorimotor cortex activity have been developed, using computer games as a training paradigm and gameplay characteristics to motivate and challenge players. These range from one-dimensional movement of a game object to single-button or multi-button BCGI controllers. This chapter overviews SMR-based BCGI focusing on a number of studies to illustrate the key concepts, principles, and methodologies. Examples drawn from the action genre, the most popular BCI game genre, with progressive difficulty and challenges, are presented, including a classic ball-basket game, a spaceship game involving asteroid avoidance, and a platform-based combat-fighter game. A focus is on elucidating the prospects and challenges for BCGI. Preliminary results from a proof-of-concept study of a BCGI multi-button controller referred to as the “CircleTime” controller are presented. The CircleTime controller offers the user the option of selecting between six separate buttons using just two motor imagery tasks. Results involving five able-bodied and seven physically impaired users are presented to provide evidence that the games are accessible even without motor control and the typical levels of control accuracy given the length of time played. The CircleTime controller is tested within combat-fighter game which requires higher cognitive processes to determine commands and select actions as well as completion of short-term and longer-term time-critical actions. The chapter covers basic SMR-based BCI signal processing and performance assessment, progressive learning across games, and camouflaging prolonged training.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 699.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 949.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Recommended Reading

  • A. Bashashati, M. Fatourechi, R.K. Ward, G.E. Birch, A survey of signal processing algorithms in brain-computer interfaces based on electrical brain signals. J. Neural Eng. [Online] 4(2), R32–R57 (2007). doi:10.1088/1741-2560/4/2/R03. Accessed 2 Nov 2012

    Google Scholar 

  • R. Beveridge, D. Marshall, S. Wilson, and D. Coyle, “Classification Effects on Motion-Onset Visual Evoked Potentials using Commercially Available Video Games,” in 20th International Computer Games Conference, 2015, pp. 28–37

    Google Scholar 

  • N. Birbaumer, N. Ghanayim, T. Hinterberger, I. Iversen et al., A spelling device for the paralysed. Nature. [Online] 398, 297–298 (1999). doi:10.1038/18581

    Google Scholar 

  • T.M. Blakely, J.D. Olson, K.J. Miller, R.P.N. Rao et al., Neural correlates of learning in an electrocorticographic motor-imagery brain-computer interface. Brain Comput. Interfaces. [Online] 1(3–4), 147–157 (2014). doi:10.1080/2326263X.2014.954183. Accessed 5 Jan 2015

    Google Scholar 

  • B. Blankertz, G. Dornhege, M. Krauledat, M. Schroeder et al., The Berlin Brain-Computer Interface presents the novel mental typewriter Hex-o-Spell. [Online] 2–3 (2006). doi:10.1.1.66.7603

    Google Scholar 

  • B. Blankertz, R. Tomioka, S. Lemm, M. Kawanabe, and K. Muller, “Optimizing Spatial filters for Robust EEG Single-Trial Analysis,” IEEE Signal Process. Mag., 25(1), pp. 41–56, (2008)

    Google Scholar 

  • L. Bonnet, F. Lotte, A. Lécuyer, Two brains, one game: design and evaluation of a multiuser BCI Video Game based on motor imagery, IEEE transactions on computational intelligence and ai in games. 5(2), 185–198 (2013)

    Google Scholar 

  • Bordoloi, S., Sharmah, U. & Hazarika, S.M. (2012) Motor imagery based BCI for a maze game. 4th International Conference on Intelligent Human Computer Interaction (IHCI). [Online] 1–6. Available from: doi:10.1109/IHCI.2012.6481848

    Google Scholar 

  • D.P. Bos, B. Reuderink, B. van de Laar, H. Gürkök, C. Mühl, M. Poel, A. Nijholt, D. Heylen, Brain-computer interfacing and games, in Brain-Computer Interfaces, ed. by A. Nijholt, D.S. Tan [Online]. (Springer, London, 2009), p. 149–178. doi:10.1007/978-1-84996-272-8

    Google Scholar 

  • D.P. Bos, M. Obbink, A. Nijholt, G. Hakvoort, M.C. (2010) Towards multiplayer BCI games, in BioS-Play (2010), pp. 1–4

    Google Scholar 

  • T.J. Bradberry, R.J. Gentili, J.L. Contreras-Vidal, Reconstructing three-dimensional hand movements from noninvasive electroencephalographic signals. J. Neurosci. [Online] 30(9), 3432–3437 (2010). doi:10.1523/JNEUROSCI.6107-09.2010. Accessed 7 Nov 2012

    Google Scholar 

  • G.E. Chatrian, M.C. Petersen, J.A. Lazarte, The blocking of the rolandic wicket rhythm and some central changes related to movement. Electroencephalogr. Clin. Neurophysiol. [Online] 11497–11510 (1959). doi:10.1016/0013-4694(59)90048-3

    Google Scholar 

  • D. Coyle, Neural network based auto association and time-series prediction for biosignal processing in brain-computer interfaces. IEEE Comput. Intell. Mag. (November), 4(4), 47–59 (2009)

    Google Scholar 

  • D. Coyle, Real-time spaceship game control. [Online]. (2010). Available from: www.youtube.com/watch?v=CSZG_oXf0lg

  • D. Coyle, Spaceship game control showing all difficulty levels (speed x 3). [Online]. (2011). Available from: http://www.youtube.com/watch?v=j7uOinkVQUY&feature=plcp

  • D. Coyle, Brainwave Controlled Combat-Fighter Game. [Online] (2012). Available from: https://www.youtube.com/watch?v=IiV_Gn3-oo0

  • D. Coyle, G. Prasad, T.M. McGinnity, A time-frequency approach to feature extraction for a brain-computer interface with a comparative analysis of performance measures. EURASIP J. Adv. Signal Process. [Online] 2005(19), 3141–3151 (2005a). doi:10.1155/ASP.2005.3141

    Google Scholar 

  • D. Coyle, G. Prasad, T.M. McGinnity, A time-series prediction approach for feature extraction in a brain-computer interface. IEEE Trans. Neural Syst. Rehabil. Eng. [Online] 13(4), 461–467 (2005b). doi:10.1109/TNSRE.2005.857690

    Google Scholar 

  • D. Coyle, T.M. Mcginnity, G. Prasad, A multi-class brain-computer interface with SOFNN-based prediction preprocessing. 44(0) (2007)

    Google Scholar 

  • D. Coyle, G. Prasad, T.M. McGinnity, Faster self-organizing fuzzy neural network training and a hyperparameter analysis for a brain-computer interface. IEEE Trans. Syst. Man Cybern. Part B Cybern. Publ. IEEE Syst. Man Cybern. Soc. [Online] 39(6), 1458–1471 (2009). doi:10.1109/TSMCB.2009.2018469

    Google Scholar 

  • D. Coyle, J. Garcia, A.R. Satti, T.M. Mcginnity, EEG-based continuous control of a game using a 3 channel motor imagery BCI, in IEEE Symposium Series on Computational Intelligence, (2011a), pp. 88–94

    Google Scholar 

  • D. Coyle, A. Satti, J. Stow, K. Mccreadie et al., Operating a brain computer interface: able bodied vs. physically impaired performance, in Proceedings of the Recent Advances in Assistive Technology & Engineering Conference, 2011

    Google Scholar 

  • G. Dornhege, B. Blankertz, G. Curio, K.R. Müller, Boosting bit rates in noninvasive EEG single-trial classifications by feature combination and multiclass paradigms. IEEE Trans. Biomed. Eng. [Online] 51993–51002 (2004). doi:10.1109/TBME.2004.827088

    Google Scholar 

  • C. Enzinger, S. Ropele, F. Fazekas, M. Loitfelder et al., Brain motor system function in a patient with complete spinal cord injury following extensive brain-computer interface training. Exp. Brain Res. [Online] 190215–190223 (2008). doi:10.1007/s00221-008-1465-y

    Google Scholar 

  • T. Geng, J.Q. Gan, Motor prediction in brain-computer interfaces for controlling mobile robots, in Conference Proceedings: … Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Conference. [Online] 634–637 (2008). doi:10.1109/IEMBS.2008.4649232

    Google Scholar 

  • I.I. Goncharova, D.J. McFarland, T.M. Vaughan, J.R. Wolpaw, EMG contamination of EEG: spectral and topographical characteristics. Clin. Neurophysiol. [Online] 114(9), 1580–1593 (2003). doi:10.1016/S1388-2457(03)00093-2

    Google Scholar 

  • P. Herman, G. Prasad, T.M. McGinnity, D. Coyle, Comparative analysis of spectral approaches to feature extraction for EEG-based motor imagery classification. IEEE Trans. Neural Syst. Rehabil. Eng. Publ. IEEE Eng. Med. Biol. Soc. [Online] 16(4), 317–326 (2008). doi:10.1109/TNSRE.2008.926694

    Google Scholar 

  • J. Kennedy, R. Eberhart, Particle swarm optimization, in Proceedings of ICNN’95 – International Conference on Neural Networks. [Online] 4 (1995). doi:10.1109/ICNN.1995.488968

    Google Scholar 

  • A. Kreilinger, V. Kaiser, C. Breitwieser, J. Williamson et al., Switching between manual control and brain-computer interface using long term and short term quality measures. Front. Neurosci. [Online] 5 (January), 147 (2011). doi:10.3389/fnins.2011.00147. Accessed 07 Dec 2012

    Google Scholar 

  • D.J. Krusienski, M. Grosse-Wentrup, F. Galán, D. Coyle et al., Critical issues in state-of-the-art brain-computer interface signal processing. J. Neural Eng. [Online] 8(2), 025002 (2011). doi:10.1088/1741-2560/8/2/025002. Accessed 02 Nov 2012

    Google Scholar 

  • A. Kübler, F. Nijboer, J. Mellinger, T.M. Vaughan et al., Patients with ALS can use sensorimotor rhythms to operate a brain-computer interface. Neurology. [Online] 64(10), 1775–1777 (2005). doi:10.1212/01.WNL.0000158616.43002.6D. Accessed 01 Aug 2013

    Google Scholar 

  • E.C. Lalor, S.P. Kelly, C. Finucane, R. Burke et al., Steady-state VEP-based brain-computer interface control in an immersive 3D gaming environment. Eurasip J. Appl.Signal Process. [Online] 3156–3164 (2005). doi:10.1155/ASP.2005.3156

    Google Scholar 

  • A. Lecuyer, F. Lotte, R. B. Reilly, R. Leeb, M. Hirose, and M. Slater, “Brain-Computer Interfaces, Virtual Reality, and Videogames,” Computer (Long. Beach. Calif)., 41(10), pp. 66–72, (2008)

    Google Scholar 

  • R. Leeb, M. Lancelle, V. Kaiser, D.W. Fellner et al., Thinking Penguin: multimodal brain–computer interface control of a VR game. IEEE Trans. Comput. Intell. AI Games. [Online] 5 2), 117–128 (2013). doi:10.1109/TCIAIG.2013.2242072

    Google Scholar 

  • F. Lotte, M. Congedo, A. Lécuyer, F. Lamarche et al., A review of classification algorithms for EEG-based brain-computer interfaces. J. Neural Eng. [Online] 4(2), R1–R13 (2007). doi:10.1088/1741-2560/4/2/R01. Accessed 26 Oct 2012

    Google Scholar 

  • D. Marshall, D. Coyle, S. Wilson, M. Callaghan, Games, gameplay, and BCI: the state of the art. IEEE Trans. Comput. Intell. AI Games. [Online] 5(2), 82–99 (2013). doi:10.1109/TCIAIG.2013.2263555. Accessed 04 July 2013

    Google Scholar 

  • D. Marshall, R. Beveridge, S. Wilson, and D. Coyle, “Interacting with Multiple Game Genres using Motion Onset Visual Evoked Potentials,” in 20th International Computer Games Conference, 2015, pp. 18–27

    Google Scholar 

  • S.G. Mason, A. Bashashati, M. Fatourechi, K.F. Navarro et al., A comprehensive survey of brain interface technology designs. Ann. Biomed. Eng. [Online] 35(2), 137–169 (2007). doi:10.1007/s10439-006-9170-0. Accessed 02 Nov 2012

    Google Scholar 

  • D.J. McFarland, W.a Sarnacki, J.R. Wolpaw, Electroencephalographic (EEG) control of three-dimensional movement. J. Neural Eng. [Online] 7(3), 036007 (2010). doi:10.1088/1741-2560/7/3/036007. Accessed 29 Oct 2012

    Google Scholar 

  • K.J. Miller, E.C. Leuthardt, G. Schalk, R.P.N. Rao et al., Spectral changes in cortical surface potentials during motor movement. J. Neurosci. [Online] 27(9), 2424–2432 (2007). doi:10.1523/JNEUROSCI.3886-06.2007. Accessed 25 Jan 2013

    Google Scholar 

  • K.-R. Müller, C.W. Anderson, G.E. Birch, Linear and nonlinear methods for brain-computer interfaces. IEEE Trans. Neural Syst. Rehabil. Eng. Publ. IEEE Eng. Med. Biol. Soc. [Online]. 11, 165–169 (2003). doi:10.1109/TNSRE.2003.814484

    Google Scholar 

  • B. Obermaier, C. Neuper, C. Guger, G. Pfurtscheller, Information transfer rate in a five-classes brain-computer interface. IEEE Trans. Neural Syst. Rehabil. Eng. Publ. IEEE Eng. Med. Biol. Soc. [Online] 9283–9288 (2001). doi:10.1109/7333.948456

    Google Scholar 

  • G. Pfurtscheller, D. Flotzinger, C. Neuper, Differentiation between finger, toe and tongue movement in man based on 40 Hz EEG. Electroencephalogr. Clin. Neurophysiol. [Online] 90(6), 456–460 (1994). doi:10.1016/0013-4694(94)90137-6. Accessed 07 Apr 2014

    Google Scholar 

  • G. Pfurtscheller, C. Neuper, D. Flotzinger, M. Pregenzer, EEG-based discrimination between imagination of right and left hand movement. Electroencephalogr. Clin. Neurophysiol. [Online] 103642–103651 (1997). doi:10.1016/S0013-4694(97)00080-1

    Google Scholar 

  • G. Pfurtscheller, C. Neuper, A. Schlögl, K. Lugger, Separability of EEG signals recorded during right and left motor imagery using adaptive autoregressive parameters. IEEE Trans. Rehabil. Eng. [Online] 6(3), 316–325 (1998). Available from: http://www.ncbi.nlm.nih.gov/pubmed/9749909

  • G. Pfurtscheller, C. Guger, G. Müller, G. Krausz et al., Brain oscillations control hand orthosis in a tetraplegic. Neurosci. Lett. [Online] 292(3), 211–214 (2000). Available from: http://www.ncbi.nlm.nih.gov/pubmed/11018314

  • G. Pfurtscheller, C. Brunner, A. Schlögl, F.H. Lopes da Silva, Mu rhythm (de)synchronization and EEG single-trial classification of different motor imagery tasks. NeuroImage. [Online] 31(1), 153–159 (2006). doi:10.1016/j.neuroimage.2005.12.003. Accessed 26 Oct 2012

    Google Scholar 

  • G. Pfurtscheller, B.Z. Allison, C. Brunner, G. Bauernfeind et al., The hybrid BCI. Front. Neurosci. [Online] 4(April), 30 (2010). doi:10.3389/fnpro.2010.00003. Accessed 08 Nov 2013

    Google Scholar 

  • H. Ramoser, J. Muller-Gerking, G. Pfurtscheller, Optimal spatial filtering of single trial EEG during imagined hand movement. IEEE Trans. Rehabil. Eng. [Online] 8(4), 441–446 (2000). doi:10.1109/86.895946. Accessed 06 Feb 2013

    Google Scholar 

  • R.G. Robinson, L.B. Starr, J.R. Lipsey, K. Rao et al., A two-year longitudinal study of post-stroke mood disorders: dynamic changes in associated variables over the first six months of follow-up. Stroke J. Cerebral Circ. [Online] 15(3), 510–517 (1984). doi:10.1161/01.STR.15.3.510

    Google Scholar 

  • A.S. Royer, A.J. Doud, M.L. Rose, B. He, EEG control of a virtual helicopter in 3-dimensional space using intelligent control strategies. IEEE Trans. Neural Syst. Rehabil. Eng. Publ. IEEE Eng. Med. Biol. Soc. [Online] 18(6), 581–589 (2010). doi:10.1109/TNSRE.2010.2077654

    Google Scholar 

  • A. Satti, D. Coyle, G. Prasad, Optimal frequency band selection with particle swarm optimization for a brain computer interface, Workshop/Summer School on Evolutionary Computing Lecture Series by Pioneers. 44(0), 72–75 (2008)

    Google Scholar 

  • A. Satti, D. Coyle, G. Prasad, Continuous EEG classification for a self-paced BCI, in 2009 4th International IEEE/EMBS Conference on Neural Engineering. [Online] 315–318 (2009a). doi:10.1109/NER.2009.5109296

    Google Scholar 

  • A.R. Satti, D. Coyle, G. Prasad, Spatio-spectral & temporal parameter searching using class correlation analysis and particle swarm optimization for a brain computer interface, in 2009 I.E. International Conference on Systems, Man and Cybernetics. [Online] 1731–1735 (2009b). doi:10.1109/ICSMC.2009.5346679

    Google Scholar 

  • G. Schalk, Brain-computer symbiosis. J. Neural Eng. [Online] 5(1), P1–P15 (2008). doi:10.1088/1741-2560/5/1/P01. Accessed 11 Nov 2012

    Google Scholar 

  • A. Schlögl, C. Neuper, G. Pfurtscheller, Estimating the mutual information of an EEG-based brain-computer interface. Biomed. Tech. Biomed. Eng. [Online] Biomed. Tech. Eng., 47(1–2) pp. 3–8, 2002. doi:10.1515/bmte.2002.47.1-2.3

    Google Scholar 

  • A. Schlögl, D. Flotzinger, G. Pfurtscheller, Adaptive autoregressive modeling used for single-trial EEG classification – Verwendung eines Adaptiven Autoregressiven Modells für die Klassifikation von Einzeltrial-EEG-Daten. Biomed. Tech. Biomed. Eng. [Online] 42(6), 162–167 (1997). doi:10.1515/bmte.1997.42.6.162. Accessed 11 Feb 2013

    Google Scholar 

  • A. Schlögl, F. Lee, H. Bischof, G. Pfurtscheller, Characterization of four-class motor imagery EEG data for the BCI-competition 2005. J. Neural Eng. [Online] 2(4), L14–L22 (2005). doi:10.1088/1741-2560/2/4/L02. Accessed 02 Nov 2012

    Google Scholar 

  • C.E. Shannon, A mathematical theory of communication. ACM SIGMOBILE Mobile Comput. Commun. Rev. [Online]. 5, 3 (2001). doi:10.1145/584091.584093

    Google Scholar 

  • S. Silvoni, A. Ramos-Murguialday, M. Cavinato, C. Volpato et al., Brain-computer interface in stroke: a review of progress. Clin. EEG Neurosci. [Online] 42(4), 245–252 (2011). doi:10.1177/155005941104200410. Accessed 03 Feb 2013

    Google Scholar 

  • J. Stow, D. Coyle, A. Carroll, A. Satti et al., Achievable brain computer communication through short intensive motor imagery training despite long term spinal cord injury. Irish Institute of Clinical Neuroscience workshop (abstract) (2012)

    Google Scholar 

  • R. Tomioka, S. Lemm, Filters for robust EEG. (January 2008), 41–56 (n.d.)

    Google Scholar 

  • B. Blankertz, R. Tomioka, S. Lemm, M. Kawanabe, and K. Muller, “Optimizing Spatial filters for Robust EEG Single-Trial Analysis,” IEEE Signal Process. Mag., 25(1), pp. 41–56, (2008)

    Google Scholar 

  • F. Velasco-Álvarez, R. Ron-Angevin, L. da Silva-Sauer, S. Sancha-Ros, Audio-cued motor imagery-based brain-computer interface: navigation through virtual and real environments. Neurocomputing. [Online] 12189–12198 (2013). doi:10.1016/j.neucom.2012.11.038

    Google Scholar 

  • C. Vidaurre, A. Schlöogl, R. Cabeza, R. Scherer et al., A fully on-line adaptive BCI. IEEE Trans. Biomed. Eng. [Online] 531214–531219 (2006). doi:10.1109/TBME.2006.873542

    Google Scholar 

  • Y. Wang, R. Wang, X. Gao, B. Hong et al., A practical VEP-based brain-computer interface. IEEE Trans. Neural Syst. Rehabil. Eng. [Online]. 234–239 (2006). doi:10.1109/TNSRE.2006.875576

    Google Scholar 

  • J.R. Wolpaw, H. Ramoser, D.J. McFarland, G. Pfurtscheller, EEG-based communication: improved accuracy by response verification. IEEE Trans. Rehabil. Eng. Publ. IEEE Eng. Med. Biol. Soc. [Online] 6(3), 326–333 (1998). http://www.ncbi.nlm.nih.gov/pubmed/9749910

  • J.R. Wolpaw, N. Birbaumer, W.J. Heetderks, D.J. McFarland et al., Brain-computer interface technology: a review of the first international meeting. IEEE Trans. Rehabil. Eng. [Online] 8(2), 164–173 (2000). Available from: http://www.ncbi.nlm.nih.gov/pubmed/10896178. Accessed 18 Feb 2013

  • J.R. Wolpaw, N. Birbaumer, D.J. McFarland, G. Pfurtscheller et al., Brain-computer interfaces for communication and control. Clin. Neurophysiol. Off. J. Int. Fed. Clin. Neurophysiol. 113(6), 767–791 (2002)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Intelligent Systems Research Centre, Ulster University, Derry, BT48 7JL, Northern Ireland, UK

    Damien Coyle, Karl. A. McCreadie, Chen Li & Jhonatan Garcia

  2. National Rehabilitation Hospital, Rochestown Avenue, Dun Laoghaire, Republic of Ireland

    Jacqueline Stow, Jacinta McElligott & Aine Carroll

Authors
  1. Damien Coyle
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jacqueline Stow
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Karl. A. McCreadie
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chen Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jhonatan Garcia
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jacinta McElligott
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Aine Carroll
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Damien Coyle .

Editor information

Editors and Affiliations

  1. Design School, Kyoto University, Kyoto, Japan

    Ryohei Nakatsu

  2. Industrial Design, Eindhoven University of Technology, Eindhoven, The Netherlands

    Matthias Rauterberg

  3. Dipto. Scienze dell'Informazione, Università di Bologna , Bologna, Italy

    Paolo Ciancarini

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer Science+Business Media Singapore

About this entry

Cite this entry

Coyle, D. et al. (2017). Action Games, Motor Imagery, and Control Strategies: Toward a Multi-button Controller. In: Nakatsu, R., Rauterberg, M., Ciancarini, P. (eds) Handbook of Digital Games and Entertainment Technologies. Springer, Singapore. https://doi.org/10.1007/978-981-4560-50-4_1

Download citation

Publish with us

Policies and ethics

Search

Navigation