Keywords

1 Introduction

A Brain-Computer Interface (BCI) is an emergent technology that provides a non-muscular communication channel for people with motor disabilities [1], therefore, this technology provides a novel way to recover functionality of limbs impaired by diseases and injuries of the central nervous system. The key in a BCI is the recording and processing of the brain activity [2]. On the first hand, the non-invasive electroencephalogram (EEG) technique is the most used method to record the brain activity. On the second hand, machine learning algorithms are employed to classify patterns or changes contained in the recorded EEG signals which are associated to a mental task performed by the BCI user.

The recognition of the mental task is very difficult because the EEG signals are highly noisy, moreover they are non-stationary and present strong variability across participants and time. In order to overcome these situations, BCI technology uses linear discriminant analysis (LDA), artificial neural networks (ANN) and support vector machines (SVM) which are classical supervised learning models commonly used in these systems [3]. This has resulted in several applications that have demonstrated the functionality and usability of BCI technology. However, the performance in the recognition of mental tasks from EEG signals (or classification accuracy) is prone to errors, thus novel strategies in the classification need be developed, evaluated and incorporated in BCI technologies.

As a novel alternative to recognize mental tasks for EEG-based BCI, this work proposes the use of self-organized maps (SOM) [4] to recognize motor imagery tasks from EEG signals recorded in a real BCI experiment. SOM is a (supervised or not supervised) learning method that transforms high dimensional data to a lower representation in a way that similar inputs are associated to a region that is easily separated from the regions of other similar inputs [4]. This classification model has been extensively used with biomedical and topographical data, however, its application in the recognition of mental tasks for BCI technologies is still very limited [5]. This work consists of an experimental study devised to obtain EEG signals from a participant performing motor imagery tasks. The goal is to implement and evaluate the classification performance of supervised SOM, SOM+k-means and k-means in the recognition of mental motor tasks from EEG signals recorded in a cue-based BCI experiment. Recorded signals were used to evaluate the three-class classification of rest versus \(left \, motor \, imagery\) versus \(right \, motor \, imagery\). The results showed that the proposed supervised SOM provided the higher classification accuracies (mean of 77% and maximum of 85%) which were above the chance level.

The rest of this work is organized as follows: Sect. 2 describes the experiments performed to obtain EEG signals, the theoretical basis and the implementation of the SOM algorithm and the evaluation process; Sect. 3 presents and discusses the results; Sect. 4 presents the conclusions.

2 Methods and Materials

This section describes (i) the experiments carried out to obtain EEG signals from a volunteer who performed motor imagery of the upper limbs, (ii) the data analysis and the feature extraction process, (iii) the classification model based on SOM that was used to recognize motor imagery tasks from EEG signals, and (iv) the evaluation procedure and the metric used to asses performance.

2.1 EEG Data Description

Participants. The experiment was conducted with a male student from our faculty. He had no known neurological or motor disorder, moreover he had no prior experience with EEG or BCI experiments. The participant was duly informed about the aim of the study and freely signed a consent form.

Recording system. EEG signals were recorded using the Emotiv EPOC System. This is a low-cost and wireless technology useful for EEG recording and BCI experiments. This system records the brain electrical activity from 14 scalp locations at AF3, F7, F3, FC5, T7, P7, O1, AF4, F8, F4, FC6, T8, P8 and O2. These electrodes are distributed according to the international 10/20 system. The EEG signals were recorded at a sampling frequency of 128 Hz with two reference electrodes (CMS, which is located on the left side; DRL which is located on the right side). A band-pass filtering from 0.5 to 60 Hz was applied during the recording. Prior to the initiation of the experiment, a saline solution was applied to each electrode to maintain the impedance below \(5\,\text {k}\Omega \). This was cheeked with the software application provided by the Emotiv manufacturer.

Description of the experiment. The experiment consisted of a synchronous motor imagery task performed with the left hand or the right hand. The participant had to perform many repetitions or trials composed of three consecutive periods: rest, motor imagery and relax. Each of these three periods has a duration of 3 s, therefore the duration of a trial is 9 s. Figure 1 illustrates the temporal sequence of a trial. In rest period, the participant had to maintain a natural body position without performing or imaging any body movement. In the motor imagery period, the participant had to imagine opening and closing either the left or the right hand at a natural and comfortable pace. The participant was requested to avoid blinking or performing any movement during the rest and the motor imagery periods. Finally, in the relax period, the participant could move and blink.

Fig. 1.
figure 1

Temporal sequence of a trial. The trial has a duration of 9 s and consists of three periods of 3 s each: rest (maintain a natural body position without performing any movement), motor imagery (imaging opening and closing either the left or right hand) and relax (blink and/or move any body part).

Full size image

The experiment was executed in blocks of 20 trials, therefore, a block lasted 3 min. In total, 10 blocks were recorded, that is, 200 trials were recorded which required 30 min. Note that this resulted in 200 periods of rest, 100 periods of \(left \, motor \, imagery\) and 100 periods of \(right \, motor \, imagery\). For the execution of the experiment, the participant was seated if front a computer screen that displayed visual stimuli indicating to perform either, the rest, the motor imagery or the relax period. The presentation of these visual stimuli, the recording of the EEG signals from the Emotiv system and the recording of synchronization signals that marked the rest, motor imagery and relax periods were controlled by a Raspberry Pi microcomputer with an application developed in Python.

Preprocessing. After the experimental session, a four-order Butterworth-type bandpass from 2 to 40 Hz and a common average reference (CAR) filters were applied to the recorded EEG signals. Then, the filtered EEG signals were trimmed in 200 trials of 9 s containing the rest, motor imagery and relax periods, corresponding to the time intervals [0, 3) s, [3, 6) s and [6, 9) s, respectively.

2.2 Feature Extraction and Selection

It has been established that the execution, imagination, observation or attempt to perform movements produces changes in the spectral power of the EEG in the motor-related \(\alpha \sim [7,13]\) Hz and \(\beta \sim [14,35]\) Hz frequency bands of electrodes located in the motor cortex [6]. Therefore, the spectral power of the recorded EEG signals were used as features. The Welch’s averaged modified periodogram method was used to compute the spectral power. The spectral power was computed in the frequency range [6, 40] Hz at a resolution of 1 Hz using Hanning-windowed windows of length 1 s with overlap of 0.5 s. Therefore, the number of spectral power values for each electrode is 35. These spectral power values were computed separately from EEG in the rest period [0, 3) s (and they were labeled as 0) and from EEG in the motor imagery period [3, 6) s (and they were labeled as 1 or 2 if the imagined hand was left or right, respectively). As a result, the feature vector is \(\mathbf x \in \mathbb {R}^{n \times 1}\) with a class label \(y \in \{0, 1, 2\}\) where \(n=490\) (number of electrodes \(\times \) number of spectral power values).

To reduce the dimension of the feature vector, the square of the Pearson’s correlation (r-squared) was employed to select the channels that present the higher discriminative power between rest and motor imagery [7]. The r-squared was computed independently for each electrode between each vector of spectral power values with the vector of class labels. Then, the four channels that presented the highest r-squared values were selected while the remaining electrodes were discharged and not used in the rest of the study. As a result, the feature vector is \(\mathbf x \in \mathbb {R}^{n \times 1}\) with a class label \(y \in \{0, 1, 2\}\) where \(n=140\) (four electrodes \(\times \) thirty-five spectral power values).

2.3 Self-organizing Maps (SOM)

Supervised, semi-supervised and unsupervised learning models of artificial neural networks (ANN) can be developed by Self Organizing Maps (SOM). Inspired by the visual, aural and sensory areas of the brain, were introduced by Kohonen and Somervuo [4].

These ANN models are widely used to analyze nonlinear relations between variables in high volume of data. SOM architecture allows to represent this information in a reduced space, generally of two dimensions known as a map. This new space has descriptive characteristics in the same way as in the original space, showing properties and relations that cannot be seen before, due to the high number of dimensions [8].

For training, SOM architecture must be fixed, where the number of neurons, size of the map, type of lattice and neighborhood function information were provided. Most of these parameters obey to experimental rules with some initial information. Size of the map was computed based on the inertias relation using a Multiple Correspondence Analysis (MCA), where the ratio of the two first inertias is the same to the ratio of wide and high dimensions [9]. Then, number of neurons was adjusted according with preliminary results measured by activations of the neurons. Type of lattice was preserved as hexagonal topology due to yield the same distance between neurons, and a Gaussian neighborhood function was chosen for its good performance. SOM training is developed mainly in an unsupervised manner in a process followed by three steps: one of them competitive, then a cooperative one, and finally, an adaptive. Competition stage consists on an input vector with information extracted from each subject is presented to the map, and compared with information from synaptic weights of each neuron. Euclidian distance is used for this comparison and the neuron with weights closer to the input is defined as the best matched unit (BMU).

Cooperation step is given around the BMU, where neurons around this unit or neuron are chosen to be updated, modifying its weights through the neighborhood function. Finally, adaptive stage changes the BMU weights and its neighbors using the expression:

$$\begin{aligned} w_{i}(t+1) = w_{i} + \eta (t)h_{ij}(t)(x(t)-w_{i}(t)) \end{aligned}$$
(1)

where x(t) is the input vector, \(w_{i}(t)\) are the weights of the map, \(\eta (t)\) is a learning rate coefficient, and \(h_{ij}(t)\) is the neighborhood function based on Gaussian distribution, given by:

$$\begin{aligned} h_{ij}(t+1) = exp(-d_{ij}^{2}/2\sigma ^{2}(t)) \end{aligned}$$
(2)

where \(d_{ij}(t)\) is the Euclidian distance between the j input and the BMU, and \(\sigma (t)\) is the basis of the function in the t iteration. This parameter changes during the training, beginning with a basis of four units and ending with just one unit.

Supervised Training. As the task developed by the SOM is about classification, a modification in the training was implemented. In the input vector an extra variable was included with information of the class of to which that input belongs to. This new variable makes that the training of the SOM has a guide in a supervised mode. Information of the additional variable was adjusted according to the three classes presented in the data.

Semi-supervised Training. In this case, three groups were proposed: classes one to three. This was adjusted after training, where the k-means algorithm was applied to create three groups based on weights between neurons. The k-means clustering algorithm worked developing these groups based on distances between the neurons, joining the closest neurons [10]. According to the number of activations in each group based on training data presented to the map, labels were marked for each group of neurons.

2.4 Performance Evaluation

The proposed SOM-based classification was evaluated in the three-class classification of rest versus \(left \, motor \, imagery\) versus \(right \, motor \, imagery\). The cross-validation technique with ten folds was used to measure the results of the classification [11]. In each case, nine folds were combined to create the training set, and then the generalization was measured in the validation fold that was out. Performance accuracy was measured as the percentage of correct classifications which was computed separately for each class and for all classes. Performance of the two established SOM-based classification approaches, supervised SOM and SOM+k-means, were compared with a clustering developed just by the k-means algorithm, allowing to determine differences between the use of SOM before and without the use of this algorithm.

3 Results

Figure 2 shows the classification results for the three proposed approaches. Red horizontal line points out a performance of 60% (note that the chance level of the classification accuracy is 33.33% as the number of classes is 3). Results for the k-means algorithm (Fig. 2c) show that classification is not a stronghold of the method. Indeed, results just reach a performance of 51% in mean, with a maximum of 62%. When a SOM was used in combination with the k-means algorithm (Fig. 2b) is possible to see that rate of classification was improved, showing that the mapping of the network can be used in a previous task before the classification is developed. Results with this approach reach a performance of 61% in mean with a maximum of 77%. Best results were obtained with the SOM trained in a supervised way (Fig. 2a), where the mean of the performance was 77% with a maximum of 85% for all classes.

In general, results are comparable with previous studies based on the used features from the same dataset. Classification methods obtained 80% based on lattice neural networks (LNN) [12] and 72% as maximum with the used of Support Vector Machines (SVM) with a radial basis function kernel [13]. Note however that is the present work we are evaluating the SOM method in a three-class classification, which is a more difficult tasks than the two-class classification of the previous studies. It is possible to see for the three approaches, that imagination of the right hand movement is one of the most difficult classes to match, where dispersion of the results and the low performance is notable (Fig. 2).

Fig. 2.
figure 2

Classification accuracy results obtained for each class and the total for all of them obtained with (a) Supervised SOM, (b) SOM+k-means, (c) k-means. (Color figure online)

Full size image

4 Conclusions

This work presented a novel classification model based on self-organized maps (SOM) to discriminate between mental motor imagery tasks from EEG signals for brain-computer interfaces (BCI). Traditionally, the SOM method is employed to carry out clustering analysis. In the present work, the SOM model was modified to perform supervised classification. The performance of this proposed supervised SOM classifier along with SOM+k-means and k-means alone were evaluated in a three-class classification scenario consisting of recognizing motor imagery tasks (rest, left hand and right hand) from spectral power-based features extracted from EEG signals. The classification results showed that the proposed supervised SOM yielded the higher accuracies which were above the chance level. On the basis of the presented results, supervised SOM is a promising classification model that can be used in real BCI settings to recognize motor imagery tasks.