Keywords

1 Introduction

Spiking Neural Networks (SNN) are a special class of artificial neural network, where neurons communicate by sequences of pulses. This type of neuron models provide a powerful tool for spatio-temporal analysis due to its functionality based on neuronal biological models. It has been shown that SNN can be applied not only to all problems solvable by non-spiking neural networks, also SNN are in fact computationally more powerful than perceptrons and sigmoidal gates [10]. Although they offer solutions to different problems in applied engineering, as is such as fast signal-processing, event detection, classification, speech recognition, currently, they are not very popular in the field of pattern recognition. Brain-Computer Interfaces (BCI) is a promising research field which provides a communication between humans and computers by analyzing electrical brain activity, recorded at the surface of the scalp with electroencephalography. The key part of a BCI system is how to recognize the mental tasks that a subject performs by analyzing EEG signals.

In this work we propose using SNN models in the classification of Motor Imagery (MI [12]) EEG signals. SNN are models with a high degree of realism and with the advantage of performing an analysis of spatio-temporal information. A systematic evaluation procedure was carried out to assess the performance of SNN to differentiate two motor imagery tasks from EEG signals. The results show that the proposed model achieves an accuracy on average of \(81.36\%\) which is \(11.14\%\), \(0.82\%\), \(1.91\%\) superior to the accuracy achieved with MLP (2,1), MLP (\(2n+1,1\)), and KNN, respectively.

1.1 Acquisition of EEG Signals

The dataset used in this work was provided by the Institute for Knowledge Discovery (Laboratory of Brain-Computer Interfaces) of the Graz University of Technology, in the event called “BCI Competition IV” (www.bbci.de/competition/iv/). Figure 1 shows the location of the monopolar electrodes, in addition to the selection of the 12 channels that were used to carry out the classification process mentioned in this work. Likewise, detailed information of the used dataset is shown.

Fig. 1.
figure 1

Technical details of the EEG dataset from the BCI competition IV used in this work

Full size image

The dataset is made up of 144 trials from each of the four classes: Left Hand MI, Right Hand MI, Foot MI and Tongue MI. In addition to this, we took the initial time window with a duration of 2 s to generate a fifth class, this is called Rest. MI EEG were extracted from \(t=3\,\text {s}\) to \(t=5\,\text {s}\) while Rest were extracted from \(t=0\,\text {s}\) to \(t=2\,\text {s}\). Having at the end five mental tasks with which the process of classification will be carried out.

2 Feature Extraction

In this work, two feature extraction methods were used: Power Spectral Density (PSD) and Wavelet Decomposition (WD). Each trial is composed of 22 EEG channels. From these we selected only 12. The theoretical basis of each of these methods is explained below.

2.1 Power Spectral Density (PSD)

The Power Spectral Density (PSD) of the EEG signals have been used as features to recognize among movement phases. This is because it has been well established the spectral power changes in the motor-related brain rhythms during execution, imagination o attempt to perform movements [2, 11, 14]. In addition, PSD is one of the most robust methods to estimate the spectral power and one of the standard approaches to compute frequency-based features from EEG signals recorded during motor tasks [13]. The PSD was computed based on the Welch’s averaged modified periodogram method in five band of frequency: 1–4 Hz, 5–8 Hz, 9–12 Hz, 13–30 Hz and 1–30 Hz at a resolution of 1 Hz using Hanning-windowed. After this, a PSD matrix is obtained where the number of rows corresponds to each frequency analyzed and each column corresponds to each electrode (\(P_{FreqNo X ChannelsNo}\)). The values of this matrix were normalized in a range of 0 to 1. To reduce the number of PSD values, the following operation was performed: \( features = P^{T}f \); where f is a vector that contains the frequency values (\(f_{FreqNo}\)). Therefore, the number of the features for each electrode is 60.

2.2 Discrete Wavelet Transform (DWT)

The DWT method is a method of decoding subbands using a wavelet type function, in this work we used the wavelet function known as Symlet 5. The discrete signal to be decomposed is passed through filters with different frequency of cut and a process of decimation. When a signal passes through these filters, it is split into two bands. The low pass filter extracts the common information of the signal. The high pass filter extracts the detail information of the signal. The output of the low pass filter is then decimated by two. The DWT is computed by successive low pass and high pass filtering of the discrete time-domain signal. This decomposition process was carried out up to level 5 to each channel in each of the trials. At the end, 6 new signals were obtained from each of the channels (D1, D2, D3, D4, D5 and A5), so 72 signals were obtained. In order to reduce the number of features, the variance was calculated in each of these signals, obtaining at the end, 72 features.

3 Classification

Five different classification models were employed in this work, K-Nearest Neighbors (KNN), two models of Multilayer Perceptron (MLP), Single Spiking Neuron (SSN) and Spiking Neural Network (SNN). Below we explain how the SNN works, it is a method that is not commonly used with brain signals.

3.1 Spiking Neurons

At present, it is known that biological neurons communicate through the generation and propagation of electrical pulses also called action potentials or spikes. This feature is the central paradigm of a theory of spiking neural models (SNN). In the work of Ponulak et al. [15] show that the spiking models present three main properties: (1) Information coming from many inputs and only produce a single spiking output; (2) Their probability of firing is increasing by excitatory inputs and decreased by inhibitory inputs; (3) Its dynamics are characterized by state variables, when they reach a certain state, the model generates one or more pulses. The spiking neuron model described by Izhikevich (IZ) was selected for the SNN. This model has a good biological realism as well as low computational cost. The IZ model is described by two differential equations as [7], Euler’s method was used for solving the model and its parameters were set in order to reproduce the behaviour of regular spiking neurons:

$$\begin{aligned} v' = \frac{k(v-v_{r})(v-v_{t})-u+I}{C} \end{aligned}$$
(1)
$$\begin{aligned} u' = a(b(v-v_{r})-u) \end{aligned}$$
(2)
$$\begin{aligned} \text {if } v>v_{peak}, \text { then } v\longleftarrow c, u\longleftarrow u + d \end{aligned}$$

where v is the membrane potential, u is the recovery current, I is a vector with the input current arriving to the neuron, C is the membrane capacitance, \(v_{r}\) is resting membrane potential, \(v_{t}\) is the instantaneous threshold potential, k is the rheobase resistance, \(v_{peak}\) is the spike cutoff value, a is a recovery time constant, b is the input resistance, c is the voltage reset value and d is the outwards minus inwards currents during spike which affect the after-spike behavior of the model [7]. In this work, a behavior of regular spiking neurons is used for the SN model [7]. In order to achieve this, IZ parameters are set according to Fig. 2.

Fig. 2.
figure 2

Description of the Izhiquevich model parameters

Full size image

In this type of neuronal models, the simulation must be performed using numerical methods to solve the differential equations that compose it.

Two of the proposed classification models use Spiking Neurons: the model called Single Spiking Neuron uses only one neuron to perform classification (Fig. 3a), the second model called Spiking Neural Network uses a small network of neurons formed by a hidden layer with two neurons and a single output neuron (Fig. 3b).

Fig. 3.
figure 3

Arquitecture of the two Spiking models used to classify MI from EEG

Full size image

These models were optimized using the Particle Swarm Optimization (PSO) algorithm [8]. In this method each particle of a population is taken as a set of possible weights for the SNN model, with this we proceed to make a certain number of iterations and at the end we take the best solution (set of weights) found during this process.

3.2 Multilayer Perceptron (MLP)

The multilayer perceptron is an artificial neural network formed by multiple layers, this allows solving non-linear separable problems. The MLP consists of L layers, without counting the input layer, each layer contains a certain number of perceptrons, it is not necessary that all the layers have the same number of perceptrons, this is known as the structure of the neural network, It is considered a hyperparameter. Each perceptron is consists of two parts: (1) The dot product and (2) The activation function. This transfer function can be different in each layer and when using the delta rule as a base, this function must be differentiable without having to be linear. We used two models of MLP, both with a single hidden layer, the first of them only has two neurons in the hidden layer and the second uses \(2n+1\) neurons in the hidden layer, n is the number of features. Figure 4 shows the configuration of the proposed neural network models.

Fig. 4.
figure 4

Arquitecture of the two MLP models used to classify MI from EEG

Full size image

3.3 Evaluation Procedure

For each subject, the set of trials was randomly partitioned in 75% for training and 25% for testing. To measure performance, the metrics Accuracy, Cohen’s Kappa score and F1-score were computed. Accuracy was computed as:

$$\begin{aligned} Acc=\frac{TP+TN}{TP+TN+FP+FN} \end{aligned}$$

where TP, TN, FP and FN are true positives, true negatives, false positives, and false negatives, respectively. F1 is the weighted average of the precision and recall and was computed as [5]:

$$\begin{aligned} f1 = 2\times \frac{precision\times recall}{precision+recall} \end{aligned}$$

where \(precision=TP/(TP+FP)\) and \(recall=TP/(TP+FN)\).

Table 1. Classification results for subject “A09”
Full size table

Cohen’s kappa measures the agreement between two raters who each classify N items into C mutually exclusive categories. The definition of k is: \(k=\frac{P_{o}-P_{e}}{1-P_{e}}\) where \(P_{o}\) is the relative observed agreement among raters (identical to accuracy), and \(P_{e}\) is the hypothetical probability of chance agreement, using the observed data to calculate the probabilities of each observer randomly seeing each category.

4 Recognition of Motor Imagery Tasks from EEG Signals

The binary classification process was implemented with the four mentioned classifiers, in all possible pairs using the five mental tasks. This dataset consists of 9 subjects (A01 to A09). In this stage, the performance of the SSN and SNN was compared against accuracy of the MLP models.

In the BCI area, it has been found that the best way to perform the recognition of MI tasks in EEG signals is to carry it out for each of the subjects [16]. At present, the way to generalize the recognition of patterns in signals of this type has not been found. This can be attributed to the differences in the neural connections that the brain of each subject presents. Therefore, it is important to start with the analysis of a test subject. Table 1 shows the accuracy values obtained with the EEG signals of the A01 Subject for each pair of classes in the four classification methods.

Table 2. Classification results for each subject
Full size table

The best results obtained for each pair of classes are highlighted, in most of the classification scenarios an accuracy above \(90\%\) was obtained, the best results are observed with the SNN models; only in the Left H. vs Right H. scenario the MLP showed the best performance. The best result obtained with the Spiking Neuron was \(97.22\%\) for the Right H. vs Tongue scenario, this occurs when a single neuron or a network of neurons of the Spiking type is used. In the case of the data from this test subject it can be seen that the feature extraction algorithm called PSD shows better results, however this does not always happen.

The same classification process was carried out with the data of each one of the test subjects. Table 2 shows a summary of the accuracy results obtained with each proposed method. The best results obtained in each test subject are again highlighted in this table.

In most scenarios, the models based on Spiking neurons presented the best results, in 7 of the 9 participants they surpass the performance of the MLP models, however, in the two remaining subjects the Spiking Neurons show results close to MLP models. It should be noted that in this work a basic Spiking neuron model is used, since it does not receive pulse trains, only the features in a constant way over the simulation time of the neuronal activity.

Below is the average across all subjects (Table 3) for each classifier in each classification scenario.

Table 3. Mean Accuracy across all subjects
Full size table

Similarly, in 7 of 10 classification scenarios, the SNN exceeded the MLP models, in most cases with the \(80\%\) accuracy. The best classification scenario was Rest vs. Right Hand where an average performance of \(89.20\%\) was obtained with the Single Spiking Neuron.

One aspect to highlight is that state of the art just reports the classification of two specific mental tasks: Left Hand VS. Right Hand [1, 3, 4, 6, 9]. So the binary classification of each possible pair with the five mental tasks together with the using of the SNN in the area of the classification of biological signals, are the two main contributions of this paper, we want to emphasize that there is the possibility of using another type of MI tasks, not only of the superior members.

Below in Table 4 is a comparison with the classification results presented in the state of the art, it is necessary to clarify that the subject in each work is not the same, therefore the performance of the proposed models can not be compared directly. The same thing happens with the number of trials used to train and evaluate.

Table 4. State of art (Motor Imagery)
Full size table

One of the main objectives of this work is to show the potential of the SNN as a model for classifying signals of this type. As can be seen, the performance of Spiking neural models competes with the results shown in the state of the art, although they do not have patterns with enough spatio-temporal information. It is important to note that these models are conformed with a neuron (SSN) and with 3 neurons (SNN).

5 Conclusions and Further Work

This work provides an approach to perform the classification of five different mental tasks, showing the binary discrimination between each pair of classes using two methods of feature extraction commonly used in the BCI area. Also, the use of SNN provided favorable results, showing that with a small number of neurons, an acceptable discrimination process can be obtained for the efficiently implementation of systems controlled by EEG signals.

It was observed that it is not possible to distinguish which feature extraction method provides better results. The cause of this may be the characteristics of each test subject, each test subject responds differently to the mental tasks evaluated. Another point to emphasize is that this type of neurons show acceptable results even when the features used do not contain enough spatio-temporal information, which, as mentioned above, this kind of neurons have the ability to analyse spatial-temporal information.