1 Introduction

A brain-computer interface (BCI) is a system that translates brain activity patterns of a user into messages or commands for an interactive application [1]. BCIs based on voluntary modulation of Sensorimotor Rhythm (SMR) have increased the interest of many researchers, due to evidence that real or imagery motor intention of upper and lower limbs induces changes the rhythmic activity recorded on the sensorimotor cortex [2]. Pfurtscheller et al. [3] indicate that an internal or external paced event results not only in generation of event-related potential (ERP), but also in a change in the ongoing EEG/MEG in form of an event-related desynchronization (ERD) or event-related synchronization (ERS). The ERP on the one side and the ERD/ERS on the other side are different responses of neuronal structures in the brain. Whereas the former is phase-locked, the latter is not phase-locked to the event. The most important difference between both phenomena is that the ERD/ERS is highly frequency band specific, whereby either the same or different locations on the scalp can display ERD and ERS simultaneously. Also, it was found that ERD/ERS are detectable from EEG in a majority of stroke patients while performing Kinesthetic Motor Imagery (KMI) and Visual Motor Imagery (VMI) [4,5,6]. A KMI can be described as the ability to imagine performing a movement without executing it, by imagining haptic sensations felt during the real movement (i.e., tactile, proprioceptive, and kinesthetic). In contrast, a VMI mainly relies on the visualization of the execution of that movement. VMI and KMI share common neural networks particularly in the primary motor cortex, motor cortex, supplementary motor areas, somatosensory cortex and cerebellum, but also involve different cortical structures, due to the intuitive nature of KMI tasks. More precisely, KMI produces a greater activation of the primary motor cortex and of the supplementary motor areas. The resulting synaptic plasticity phenomenon makes the use of KMI-based BCIs a promising instrument of acquisition and refinement of motor skills [7]. In fact, various studies have demonstrated that similar cortical locations are activated during the same execution of real or imagery movements [3, 8, 9], whose changes can be captured through electroencephalography (EEG) signals to be translated into commands for end applications, such as robotic exoskeletons. BCI is a useful technology in helping people who have suffered a nervous system injury by providing them with an alternative pathway of communication, mobility, and rehabilitation. Therefore, stroke patients suffering severe limb weakness, but still able to imagine movements of the paretic limb, can use a BCI to trigger a contingent feedback upon the detection of MI-related EEG signals, enhancing neuroplasticity and motor recovery [10].

The objective of this work is to propose a system based on spatial feature extraction to discriminate shoulder flexion/extension motor imagery (SMI) tasks to assist shoulder movements through a 4-DoF (degrees of freedom) upper-limb robotic exoskeleton applied to post-stroke patients that cannot voluntarily start a movement [11]. Both Riemannian geometry and Common Spatial Pattern (CSP) were used on six healthy subjects to obtain the best alternative to discriminate SMI tasks.

This work presents four sections ordered as follows: Sect. 2 describes the experimental protocol, the proposed system for SMI discrimination, and the methodology to evaluate the performance of our proposal. Section 3 presents the achieved results, and their discussion. Finally, Sect. 4 presents the conclusions and future works.

2 Materials and Methods

2.1 Experimental Protocol

An experimental protocol was conducted on six healthy participants (all males with age from 23 to 57 years) to evaluate the performance of the proposed system to discriminate SMI tasks. Four stages were carried out for the experiments: (a) completing the informed consent form; (b) protocol familiarization; (c) electrode preparation; (d) protocol execution.

During the experiments, the subjects were seated on a chair in front of a PC screen. They provided their written consent prior to the data collection. Later, each volunteer was familiarized with the protocol through the execution of kinesthetic motor imageries [7] related to shoulder flexion/extension. The electrode preparation was done using conductor gel to reduce the skin-electrode impedance, preserving it below 10 kΩ. Subsequently, the experiment was executed in two-phases: (1) Collecting an EEG dataset while the subject was executing the SMI tasks for the training phase of our BCI; (2) Data collection to validate offline the BCI after training phase. To record the database for both training and validation stages, a sequence of four visual cues (black screen, red circle, yellow circle, and green circle) was used with period of 5 s. Red, yellow and green cues indicate the rest state, ready (similar to rest state, but waiting for the next cue without mental activity) and SMI tasks, respectively. All volunteers were asked to avoid any voluntary during the red, yellow and green cues, but slight and unavoidable movements execution, such as eye blinks, were allowed on the black screen. This cue sequence was displayed on a PC screen 12 times per session, through a developed Graphical User Interface (GUI) to guide the subjects during the experiment. For each participant, six sessions were completed for both training and validation sets, giving 3 min for resting after each session.

2.2 Proposed System

Figure 1 shows the block diagram of the proposed system to discriminate the SMI by EEG signals acquired through the Nexus-10 Mark II equipment from MindMedia manufacturer to be used as control commands towards an upper-limb robotic exoskeleton [11], which was built to rehabilitate people with severe motor disabilities. Our recognition system is composed of the following three stages: raw EEG pre-processing by a 2nd order Butterworth band-pass filter with a frequency range from 8 to 30 Hz; feature extraction (here two approaches based on Riemannian covariance matrices and CSP were analyzed); and classification by LDA. Both training and testing feature datasets were normalized using the mean and the standard deviation.

Fig. 1.
figure 1

Block diagram of the proposed system for the BCI based on SMI task.

Full size image

Data Acquisition

To capture the brain response through SMI tasks, three pairs of bipolar electrodes (C3-FC1, C4-FC2, Cz-Fz) were located on the motor primary and supplementary motor areas, according the 10–20 international system. The ground electrode was attached between the eyebrows. The EEG signal was recorded with Nexus-10 Mark II using BioTrace+ application software from MindMedia manufacturer, in a frequency range from 0.1 to 100 Hz, and sampling rate at 256 Hz. Moreover, an Arduino board was used to synchronize both cues (visual and sound stimuli) and EEG acquisition system.

2.3 Features Extraction

Common Spatial Pattern

Common Spatial Pattern (CSP) analysis was applied to analyze the multichannel data based on recordings from two classes (rest and imagery) from the band-pass filtered EEG data. This technique aims at using different spatial filters, such that the variance of the spatially filtered signals is maximized for one class and minimized for the other class. CSP yields a data-driven supervised decomposition of the signal parameterized by a matrix \( {\mathbf{W}} \in {\mathbb{R}}^{C x C} \) (C being the number of channels) that projects the signal \( \varvec{x}\left( \varvec{t} \right) \in {\mathbb{R}}^{C } \) that belongs to trial t in the original sensor space to \( \varvec{x}_{{\varvec{CSP}}} \left( \varvec{t} \right) \in {\mathbb{R}}^{C} \), which lives in the surrogate sensor space, as follows:

$$ \varvec{x}_{{\varvec{CSP}}} \left( \varvec{t} \right) = \varvec{W}^{\rm T} \varvec{x}\left( \varvec{t} \right) $$
(1)

where each column vector \( \varvec{w}_{\varvec{j}} \in {\mathbb{R}}^{C} \) (j = 1,…, C) of W, a spatial filter or simply a filter [12]. CSP maximizes the variance of the spatially filtered signal under one task while minimizing it for the other task, since the variance of a bandpass filtered signal is equal to band power, such as shown in [13]. This is done by a simultaneous diagonalization of the estimated covariance matrices \( \sum\nolimits_{1} { = X_{1} X_{1}^{\text{T}} } \) and \( \sum\nolimits_{2} { = X_{2} X_{2}^{\text{T}} } \) of the data for the two classes:

$$ \varvec{W}^{\text{T}} {\varvec{\Sigma}}_{1} \varvec{W} = \varvec{ \varLambda }_{1} , $$
(2)
$$ \varvec{W}^{\text{T}} {\varvec{\Sigma}}_{2} \varvec{W} =\varvec{\varLambda}_{2} , $$
(3)
$$ {\text{s.t.}}\,\,\varvec{\varLambda}_{1} +\varvec{\varLambda}_{2} = \varvec{I} $$
(4)

where \( \varvec{\varLambda}_{1} \) and \( \varvec{\varLambda}_{2 } \) are diagonal matrices, and each λ on the diagonal corresponds to an eigenvector \( \varvec{w}^{\text{T}} \). This way, the eigenvectors are the same for both decompositions and the same eigenvector, i.e. a spatial filter, corresponds to a large eigenvalue for one class and to a small eigenvalue for the other class. Since eigenvectors with large eigenvalues correspond to a large variance of the data, spatial filters with extreme eigenvalues maximize the difference in the variances for the two classes. Technically, this can be achieved by solving the generalized eigenvalue problem:

$$ {\varvec{\Sigma}}_{1} \varvec{w} = \lambda \,{\varvec{\Sigma}}_{2} \varvec{w} $$
(5)

Choosing D filters corresponding to extreme eigenvalues (either close to 1 or close to 0), the filtered data \( \varvec{x}_{{\varvec{CSP}}} \left( \varvec{t} \right) = \varvec{W}_{D}^{\text{T}} \varvec{x}\left( \varvec{t} \right) \) will have smaller dimensionality D < N and the two classes will be maximally separated by their variance.

For our study, CSP was individually selected for each subject using the band-pass filtered signals from red (rest state) and green (imagery state) cues. The number of CSP components was automatically selected, and ranged between 2 and 3. Details about this technique can be found in [14].

Riemannian Geometry

Such as described by Yger et al. [15], this approach enables the direct manipulation of EEG signal covariance matrices and subspaces, with an appropriate and dedicated geometry. Also, they show the full Riemannian BCI pipeline, through signal preprocessing, representing them as covariance matrices, and classifying them using covariance matrix-based classifiers, which exploit the Riemannian geometry. These approaches have been used for EEG-based BCI, in particular for feature representation and learning, as well as classifier design and calibration time reduction [16,17,18]. In our study, we used Riemannian geometry for feature extraction using tangent space representation from covariance matrices of the EEG trials by mean of the standard covariance method and the average of Riemann, where the tangent space projection maps a set of covariance matrices to its tangent space, according to [16]. The tangent space projection can be seen as a kernel operation described by Barachant et al. [17]. After projection, each matrix is represented as a vector of size \( \frac{{N \left( {N + 1} \right)}}{2} \), where N is the dimension of the covariance matrices. The tangent space projection is a local approximation of the manifold, whose approximation will be bigger if the matrices in the set are scattered in the manifold, and lower if they are grouped in a small region of the manifold. Details about the Riemannian geometry can be found in [19].

2.4 Data Description

To evaluate our recognition system using Riemannian or CSP for spatial feature extraction, a total of 12 sessions for the training and validation sets were used. For every subject, the first 6 sessions were used for training, and the remaining 6 sessions were used for validation. Each session is formed by two classes (rest state and SMI execution), with each class performing 12 times for a period of 5 s. Then, labeled EEG segments of 1 s in length, overlapped of 0.5 s, were used throughout segments of 4 s, taking into account 0.5 s after each cue beginning.

2.5 Evaluation

Metrics such as Accuracy (ACC), Kappa, True Positive Rate (TPR) and False Positive Rate (FPR) were used to validate the performance of our system using one of the two spatial feature extraction methods (Riemannian geometry and CSP). This comparison looks for the best method of feature extraction, in order to use it in our on-line BCI for patients to trigger, by MI, the upper limb exoskeleton.

3 Results and Discussion

Table 1 presents a comparison about the performance of our system with both Riemannian geometry and CSP in addition to the features extracted during the SMI tasks, using three bipolar channels (C3-FC1, C4-FC2, Cz-Fz), and classifying them with LDA. The best performance was achieved with Subject S01 (ACC > 89.68%), but ACC was >68% for all subjects, with mean ACC > 75.12%, using Riemannian covariance matrices and tangent space on the band-pass filtered EEG data. Meanwhile, CSP reached mean ACC around 66% for all subjects. It is worth mentioning that Subject S03 began to feel mental fatigue in session 3 of the validation stage. Table 2 shows the performance achieved that subject for different training and validation sets. The best result for this subject was achieved for the case where the first 3 sessions were used (ACC > 70%), because after the volunteer felt mental exhaustion, he showed signs of sleep during the acquisition of the remaining sessions, not executing the protocol in the desired way. Thus, it can be appreciated as the subject involvement can influence the performance of the recognition system. Both methods show inter-subject variability in the EEG with respect to the brain signal characteristics mentioned in previous works [10].

Table 1. Performance of the proposed system during shoulder flexion/extension MI tasks.
Full size table
Table 2. Performance for subject S03, for each session of the training phase and validation phase.
Full size table

4 Discussion

Bipolar channels have been widely used in several BCI studies, particularly in a set of reduced channels for upper-limb MI-based BCI [20,21,22]. In [23] the authors analyzed the largest weights of mu rhythms at C3 and C4 locations, finding no evident changes of this rhythm during one week. This conclusion is reasonable because the distribution of the mu rhythm is physiologically determined by the structure of the subject’s sensorimotor cortex, which normally does not vary in a period of time. Although subjects may have varied intensities of ERD/ERS during on-line sessions, the selected electrodes have obtained always a promising result. A practical BCI system requires a stable electrode layout, but the variability of brain activity makes it difficult to satisfy this requirement. Other studies have presented different methodologies to recognize upper limb MI tasks in healthy subjects. For instance, in [24] the authors conducted an MI experiment of left and right hands with 52 healthy subjects. They used a 64-channel montage based on the international 10–10 system to record EEG signals at 512 Hz of sampling rate. For the training phase, each trial (from 0.5 to 2 s, after the cue onset) was bandpass filtered with a frequency range from 8 to 3 Hz and extracted temporally. They used both CSP and LDA for spatial feature extraction and classification, respectively. To calculate the performance over the MI data, they conducted a cross validation, achieving mean ACC of 67.46 ± 13.17% over 50 subjects. In [25] the authors showed a method based on the deep convolutional neural network (CNN) to perform feature extraction and EEG MI classification for two subjects. Similarly, they used a bandpass filter over the raw EEG from 8 to 30 Hz, with the EEG data recorded on 28 locations. For this purpose, the authors built a 5-layer CNN model based on the spatio-temporal characteristics of EEG to recognize the MI tasks linked to left- and right-hand movements, obtaining an average ACC of 86.41 ± 0.77%. In [22] the authors used three bipolar channels, obtaining ACC of 77%, 84%, and 78% for three subjects, respectively.

5 Conclusions

This study showed the feasibility of using the Riemannian geometry to discriminate SMI tasks. The results obtained are encouraging for conveying control commands towards an upper-limb robotic exoskeleton by means of an online MI-based BCI. Our proposed system may be used in future works to increase the effectiveness of an upper- limb rehabilitation process for people with severe motor disabilities, providing them an alternative way to trigger, through EEG signals, the 4-DOF upper-limb robotic exoskeleton available in our lab. Also, the proposed system will be integrated to our robotic exoskeleton to be part of the motor rehabilitation system.