Abstract
Brain-Machine Interfaces utilize distinct brain patterns as control commands. However, many BMIs suffer from low performance issue even with the state-of-the-art classification algorithms in hand. Herein, we propose a novel BMI interface using semantically congruent audiovisual stimuli involving contextual motions to assist BMI performance enhancement. We designed two motion classes of “up” and “down” using two paradigms: visual only and congruent audiovisual pair. We first compared the level of spectral discernibility between the two given commands within each paradigm using frontal, temporal, and occipital channels. We then applied these paradigms onto a EEG-controlled drone system. Although the power spectral density did not show any statistically significant differences, the subjects’ drone controlling performance increased by 16% with the audiovisual interface compared to the visual only interface. Thus, this semantically congruent audiovisual BMI interface using contextual motion stimuli may be used as a supportive tool for enhancing BMI performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Lebedev, M.A., Nicolelis, M.A.: Brain-machine interfaces: past, present and future. Trends Neurosci. 29(9), 536–546 (2006)
Wolpaw, J., Birbaumer, N., McFarland, D.J., Pfurtscheller, G., Vaughan, T.M.: Brain-computer interfaces for communication and controls. Clin. Neurophysiol. 113, 767–791 (2002)
Debener, S., et al.: How about taking a low-cost, small, and wireless EEG for a walk? Psychophysiology 49(11), 1617–1621 (2012)
Lotte, F., et al.: A review of classification algorithms for EEG-based brain-computer interfaces. J. Neural Eng. 4(2), R1–R13 (2007)
Johannesen, J.K., et al.: Machine learning identification of EEG features predicting working memory performance in schizophrenia and healthy adults. Neuropsychiatr. Electrophysiol. 2, 3 (2016)
Samuel, O.W., et al.: Motor imagery classification of upper limb movements based on spectral domain features of EEG patterns. In: 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE (2017)
Curran, E.: Learning to control brain activity: a review of the production and control of EEG components for driving brain–computer interface (BCI) systems. Brain Cogn. 51(3), 326–336 (2003)
Czigler, I., Balázs, L.: Event-related potentials and audiovisual stimuli: multimodal interactions. NeuroReport 12(2), 223–226 (2001)
Hein, G., et al.: Object familiarity and semantic congruency modulate responses in cortical audiovisual integration areas. J. Neurosci. 27(30), 7881–7887 (2007)
Calvert, G.A.: Crossmodal processing in the human brain: insights from functional neuroimaging studies. Cereb. Cortex 11(12), 1110–1123 (2001)
Meredith, M.A., Stein, B.E.: Interactions among converging sensory inputs in the superior colliculus. Science 221(4608), 389–391 (1983)
Doehrmann, O., Naumer, M.J.: Semantics and the multisensory brain: how meaning modulates processes of audio-visual integration. Brain Res. 1242, 136–150 (2008)
Molholm, S., et al.: Multisensory auditory–visual interactions during early sensory processing in humans: a high-density electrical mapping study. Cogn. Brain. Res. 14(1), 115–128 (2002)
Talsma, D., Doty, T.J., Woldorff, M.G.: Selective attention and audiovisual integration: is attending to both modalities a prerequisite for early integration? Cereb. Cortex 17(3), 679–690 (2006)
van Driel, J., et al.: Interregional alpha-band synchrony supports temporal cross-modal integration. Neuroimage 101, 404–415 (2014)
Peirce, J.W.: PsychoPy—psychophysics software in Python. J. Neurosci. Methods 162(1), 8–13 (2007)
Delorme, A., Makeig, S.: EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J. Neurosci. Methods 134(1), 9–21 (2004)
Chaumon, M., Bishop, D.V., Busch, N.A.: A practical guide to the selection of independent components of the electroencephalogram for artifact correction. J. Neurosci. Methods 250, 47–63 (2015)
Stein, B.E., Stanford, T.R.: Multisensory integration: current issues from the perspective of the single neuron. Nat. Rev. Neurosci. 9(4), 255–266 (2008)
McGurk, H., MacDonald, J.: Hearing lips and seeing voices. Nature 264(5588), 746–748 (1976)
Shimojo, S., Shams, L.: Sensory modalities are not separate modalities: plasticity and interactions. Curr. Opin. Neurobiol. 11(4), 505–509 (2001)
Chen, Y.C., Yeh, S.L., Spence, C.: Crossmodal constraints on human perceptual awareness: auditory semantic modulation of binocular rivalry. Front. Psychol. 2, 212 (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Kim, S., Kim, J. (2019). A Novel Semantically Congruent Audiovisual Interface for Assisting Brain-Machine Interface (BMI) Performance Enhancement. In: Stephanidis, C. (eds) HCI International 2019 - Posters. HCII 2019. Communications in Computer and Information Science, vol 1032. Springer, Cham. https://doi.org/10.1007/978-3-030-23522-2_21
Download citation
DOI: https://doi.org/10.1007/978-3-030-23522-2_21
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-23521-5
Online ISBN: 978-3-030-23522-2
eBook Packages: Computer ScienceComputer Science (R0)