Keywords

1 Introduction

Brain-Computer Interfaces (BCI) are a novel emergent technology that allows an alternative interaction with the environment using only brain signals [1]. BCI systems can have several uses in the context of rehabilitation or assistance for motor disabled people, and also they can be used in ludic activities. Non-invasive methods can be used to record input signals in the form of EEG (Electroencephalography). BCI technology is of great interest, since it allows the real time characterization for motor activity to obtain information related to actions.

Recognizable changes on the EEG activity allows the description of cortical processes associated with motor actions [2]. In a moving arm, three phases can be identified: (i) at the beginning the person is resting, in a relaxation state (relax), (ii) then, there is a mental process of motor planning where internally and unconsciously the person generates electrical brain signals to prepare the body for the movement (intention), (iii) finally the execution of the movement by the arm (execution). Recognition or discrimination of these motor states is an important key in the design of BCI. Particularly, anticipate any movement execution leads towards a more natural BCI technology with more high temporal precision for the control of neuroprostetic devices or virtual applications, which allows developing new ways of inducing neural rehabilitation [3–7].

It is well established the correlation between EEG activity and motor states. The decrease/increase of oscillatory activity in specific frequency bands (ERD/S) allows to identify every motor state [8]. There are some works related to the identification of movement intention following a static decoding [9–12]; all of them try to identify movement intention before a movement execution; they make use of classical algorithms for the classification part like naive Bayesian (BSC) and linear discriminant analysis (LDA); as feature vector some of them use spectral power and others common spatial patterns. The accuracy of motor states detection is still an open issue that can be explored by the use of promising advance techniques in the area of pattern recognition. This work presents the application of a novel classification technique named Lattice Neural Networks with Dendritic Processing (LNNDP) [13]. This method has been successfully applied in other topics like [14] and recently, with brain signals [15]. However, to our knowledge, this is the first time in which this classification problem is tackled with a LNNDP.

The goal of this work is to apply and evaluate the LNNDP classification technique for the identification of 3 motor states: relax, movement intention, and movement execution. To this end, real EEG signals were recorded from healthy subjects performing a self-paced arm movement, this innovative experiment is inspired in a daily life activity. At the end, an offline signal analysis was performed, which includes signal processing, feature selection through a \(r^2\) analysis and classification. For the classification part a two-class discrimination strategy was set for training and test: (i) relax versus intention, (ii) relax versus execution, and (iii) intention versus execution. A multi-class tactic could be considered, but the results in these two-class strategies are enough for giving us an idea in how close we are to build an accurate feedback for a BCI system.

The paper is organized as follows: Sect. 2 describes the details in the methodology that supports the experiment, signal processing, and results; Sect. 3 describes the obtained results; and finally in Sect. 4 the conclusions are included.

2 Methods and Materials

2.1 Description of the EEG Dataset

Data Recording: Eighteen healthy right handed students (6 males and 12 females) voluntarily participated in this study (average age 20.3 years). None of them presented neurological or motor disease. All participants were informed about the experiment and all signed informed consent forms. EEG and EMG (Electromyographic) activity were acquired at a sampling frequency of 2048Hz and no filtering was applied. EEG signals were recorded according to the \(10-10\) international system from 21 scalp positions (Fp1, Fp2, F7, F3, Fz, F4, F8, T3, C3, Cz, C4, T4, T5, P3, Pz, P4, T6, O1, O2, A1, A2). The impedance for all electrodes was kept below \(5\,k\Omega \). The ground and reference electrodes were placed on Fpz and on the left earlobe, respectively. Bipolar EMG signals from the biceps were also recorded from both arms. The impedance for these electrodes were kept below \(20\,k\Omega \).

During the execution of the experiments, the participants were comfortably seated with both arms resting on the chair’s arm. The execution of the experiment was controlled by visual cues presented in a computer screen located in front of the participants. The experimental task consisted in the execution of many trials of natural reaching movements performed individually with the left arm or the right arm. The execution of each trial was controlled by three visual cues (see Fig. 1a). The first cue showed for three seconds the text “relax” and instructed to relax the body without performing any movement. The second cue showed for twelve seconds an image of a “left” or “right” arrow and instructed to stay relaxed for about five seconds and then to naturally move the corresponding arm towards the center of the screen. Notice that the actual movement initiation is different across trials. The last cue showed for three seconds the text “rest” and indicated to rest, move or blink while adopting the initial relaxed position. The experiment was performed in blocks of 24 trials each. Therefore, a total of 96 trials were recorded per participant.

Fig. 1.
figure 1

(a) Graphical description of the timeline of a trial during the execution of the experiments. The first cue instructed to relax, the second cue instructed to self-initiate a reaching movement with the corresponding arm, and the third cue instructed to rest. (b) Illustration of the time axis across trials. All trials are referenced to the movement onset at \(t=0\) (estimated with the EMG activity) while all of them have different initiation time at \(t_{ini}\). The figure also shows the segments used to evaluate the classification of motor states: relax segment at \([t_{ini}+1.5, t_{ini}+3]\,s\); intention segment at \([-1.5, 0]\,s\); execution segment at (0, 1]s.

Full size image

Data Pre-processing: Firstly, EEG and EMG data were re-sampled to \(256\,Hz\) and EEG signals were band-pass filtered from \(0.1\,Hz\) to \(100\,Hz\) using a zero-phase four-order Butterworth filter and re-referenced using the common average reference (CAR) filter. Subsequently, EEG and EMG data were segmented in trials starting from the first visual cue and up to the third visual cue. In consequence the length of the resulted trials was fifteen seconds. Then, the EMG signal of the moved arm of each trial was used to estimate the time instant of the movement onset. This EMG-based movement onset estimation was performed using the Hilbert transform. Subsequently, trials in which the time of the movement onset was lower than 3s and greater that 11s relative to the second visual cue were excluded and not used in the study. The time axis of all trials were then referenced to the EMG-based movement onset, thus \(t=0\) represents the start of movement execution. Finally, all trials were trimmed from the trial’s initiation up to 1s relative to the EMG-based movement onset. Therefore, all trials have the same movement initiation (\(t=0\)) but different trial’s initiation (\(t_{ini}\)) and trial’s length. An illustration of the time axis of the trials is presented in Fig. 1b. As a result of these pre-processing steps, the total number of trials across all subjects was on average \(93.8 \pm 2.0\) (minimum of 89 and maximum of 96), while the time of the trial initiation across all subjects was on average \(-10.1 \pm 1.5\) (minimum of \(-13.9\) and maximum of \(-6.0\)).

2.2 Classification of Motor States Using LNNDP

The aim of this study was to employ LNNDP to asses the recognition of motor states from brain signals in three bi-classification scenarios: (i) relax versus intention, (ii) relax versus execution, and (iii) intention versus execution.

Features Extraction and Selection: The Power Spectral Density (PSD) of the EEG signals were used as features to recognize between motor states. It is well known, the existing changes in the spectral power for alpha and beta rhythms during movement execution, intention and relax phases, especially over sensory-motor cortex areas; PSD was chosen to be one of the most robust methods and the standard approach for feature extraction. The PSD was computed based on the Welch’s averaged modified periodogram method in the frequency range between \(2\,Hz\) and \(40\,Hz\) at a resolution of \(1\,Hz\) using Hanning-windowed epochs of length 500ms. For each trial, the PSD was computed in three different segments (see Fig. 1b): (i) relax segment \([t_{ini}+1.5, t_{ini}+3]s\); (ii) intention segment \([-1.5, 0]s\), and (iii) execution segment (0, 1]s. Note that the relax segment is the second half of the total relaxation phase, this was to avoid artifacts in the EEG induced by the rest phase of the previous trail, while the intention segment includes EEG activity exclusively previous to the movement execution. For each classification scenario, a \(r^{2}\) analysis was performed to examine significant differences in the PSD between the two conditions, and subsequently to select the features (channel-frequency pairs) with the higher discriminative power. The \(r^{2}\) was computed for each electrode and frequency as the square of the Pearson’s correlation between the PSD values and their corresponding labels. Then, the eight channel-frequency pairs (from channels located above the motor cortex and from frequencies bins within the motor-related \(\alpha \) [8, 12]Hz and \(\beta \) \([13-30Hz]\) frequency bands) with the maximum values of \(r^{2}\) were selected and used as features. This resulted in a feature vector of dimension \(m = 8\). Therefore, the feature vector is \(\mathbf {x} \in \mathbb {R}^8\) with an associated class label \(\mathbf {y} \in \{relax, intention, execution\}\).

Lattice Neural Network with Dendritic Processing ( LNNDP ): Lattice Neural Networks with Dendritic Processing (LNNDP) is a recent classification method that considers computation in the dendritic structure as well as in the body of the neuron [13]. It requires no hidden layers, is capable of multiclass discrimination, presents no convergence problems and produces close separation surfaces between classes. The diagram of this model is presented in Fig. 2. The LNNDP has n input and m output neurons, where n is the number of features in the input vector, and m is the number of classes of the problem. A finite number of dendrites (\(D_{1}, ..., D_{k}\)) establish the connection between input and output neurons. The input neuron \(N_{i}\) has at most two connections on a given dendrite. The weight \(w^{l}_{ijk}\) associated between neuron \(N_{i}\) and dendrite \(D_{k}\), where \(l\in \{0, 1\}\) distinguishes between excitatory (\(l=1\), black dot in Fig. 2) and inhibitory (\(l=0\), empty dot in Fig. 2). In order to increase the tolerance to noise, it is possible to add a margin M, this margin is a number greater or equal to zero. For this work, the optimal M was selected following an strategy of exhaustive search, among 1000 values uniformly distributed between 0 and 1. This M value was computed exclusively from training data.

2.3 Evaluation Experiments

Evaluation Procedure: Classification performance was assessed for each subject independently using a ten-fold cross-validation procedure. The full set of trials were randomly partitioned into ten subsets which were used to construct mutually exclusive training and test sets. Nine of the subsets were used to train the classifier while the remaining was used to measure performance. This process was repeated until all the ten combinations of train and test sets were selected. The parameter M for the LNNDP classification method and feature extraction and selection were performed individually for each fold. For each combination, classification accuracy or CA (percentage of correct classifications) is computed, which is defined as:

$$\begin{aligned} CA = \frac{TP+TN}{TP+TN+FP+FN} . \end{aligned}$$
(1)

where TP(True Positives), TN(True Negatives), FP(False Positives), FN(False Negatives).

Fig. 2.
figure 2

Representation of a lattice neural network with dendritic computing [15]

Full size image

The metric F1-score or f1 [16] (weighted average of the precision and recall, it reaches its best value at 1 and worst at 0) was also calculated, it can be defined as follows:

$$\begin{aligned} f1 = \frac{2\times (precision\times recall)}{precision+recall} . \end{aligned}$$
(2)

The significant chance level of the classification accuracy or \(CA_{chance}\) was computed using the cumulative binomial distribution [17] at the confidence level of \(\alpha =0.05\). This can be done by assuming that the classification errors obey a binomial cumulative distribution, where for a total of \(N_{trials}\) and c classes, the probability to predict the correct class at least z times by chance is given by:

$$\begin{aligned} p(z)=\sum _{i=z}^{N_{trials}} {N_{trials}\atopwithdelims ()i} \times \left( \frac{1}{c}\right) ^i \times \left( \frac{c-1}{c}\right) ^{N_{trials}-1} . \end{aligned}$$
(3)

For this work \(c=2\) is the number of classes and \(N_{trials}=89\) is the minimum number of trials across all participants. Hence, \(CA_{chance}=62\,\%\) is the bound above which classification is significant. Finally, distributions CA where constructed for each subject and for all of them, and significant differences between the median of the distributions and \(CA_{chance}\) were assessed using the Wilcoxon signed rank test at the significant chance level of \(\alpha =0.05\).

3 Results

3.1 PSD Features Analysis

Figure 3a shows PSD averaged across all participants for electrode Cz. For classification scenarios relax vs. intention and relax vs. execution, the PSD in the \(\alpha \) and \(\beta \) frequency bands is greater in relax than in intention and execution, respectively. For scenario intention vs. execution, differences in the PSD are observed from the \(\beta \), up to higher frequency bands. To examine the differences between the two conditions, the \(r^2\) analysis across all participants is presented in Fig. 3b. For scenarios relax vs. intention and relax vs. execution, the largest differences between the two conditions are observed in electrodes located above the motor cortex (C3, Cz and C4) and in the motor-related frequency bands (\(\alpha \) and \(\beta \)), while for scenario intention vs. execution, the largest differences between the two conditions are observed across all electrodes and frequencies higher than 15 Hz. As illustration, these plots also display as white crosses the eight channel-frequency pairs with the maximum \(r^2\) which were selected as attributes to discriminate between the two conditions.

Fig. 3.
figure 3

(a) PSD and (b) \(r^2\) analysis across all participants for the three studied scenarios: relax vs. intention; relax vs. execution; intention vs. execution. White crosses is the \(r^2\) analysis represents the eight channel-frequency pairs with the maximum \(r^2\) that were selected as attributes.

Full size image

3.2 Classification Results

Figure 4 shows the distributions of CA computed for each participant and under every scenario. In scenario relax vs. intention, seven subjects (1, 6–8, 10, 13, and 17) showed that the median of their distributions of CA are higher and significantly different than the \(CA_{chance}\) (\(p<0.05\)). In scenario relax vs. execution, the median of the distributions of CA were significantly different and above chance level \(CA_{chance}\) (\(p<0.05\)) in ten of the eighteen participants (3–7, 9–12 and 18). For the last scenario intention vs. execution, the median of the distributions of CA were significantly different (\(p<0.05\)) and greater than the \(CA_{chance}\) for all the participants except for subject 13.

Fig. 4.
figure 4

Distribution of the classification accuracy for (a) relax vs. intention; (b) relax vs. execution; and (c) intention vs. execution across all subjects. Horizontal line over every bloxplot represents the median, and the black dot represents the mean. The red dotted line over the plot represents the chance level (Color figure online).

Full size image
Table 1. Summary of CA, TPR and TNR computed across all subjects
Full size table
Table 2. Summary for Precision, Recall, and f1 computed across all subjects
Full size table

Table 1 presents a summary of CA metric in each scenario. In scenario 1, the mean across all subjects is \(65.26\pm {5.73}\)%, and TPR/TNR are 60.17% and 70.95%, respectively. For scenario 2, in the average CA is \(69.07\pm {6.48}\)% across all subjects, and 67.85% and 70.75% values can be observed for TPR and TNR. In scenario 3, the average of CA is \(76.71\pm {5.24}\)%, and the specific values for TPR and TNR are 76.40% and 77.26%. Note that the mean for all scenarios are greater than \(CA_{chance}\), and particularly the last bi-class classification scenario shows the better results, even the minimal value 68.42% is higher than the chance level.

The f1 metric considers both precision and recall measures to compute the score. These values are presented in Table 2. For the three different classification scenarios the values 0.64, 0.69, and 0.77 were obtained.

4 Conclusions

In this work, the performance of the novel classification algorithm Lattice Neural Networks with Dendritic Processing (LNNDP) was evaluated in three motor states under arm movement using EEG signals. A daily life experiment of an auto-initiated movement arm over healthy participants was conducted. In every experiment EEG and EMG activity was recorded. Power spectral density was used as feature vector. First of all, through a \(r^2\) analysis, significant power spectral differences were founded between three scenarios: (i) relax vs. intention, (ii) relax vs. execution, (iii) intention vs. execution. This observation was used as feature selector, and it allowed to choose the best feature vector. The most discriminative power was observed in the electrodes around the motor cortex (C3, Cz and C4) and in the motor-related frequency bands (\(\alpha \) and \(\beta \)). For the classification and test evaluation, a two-scheme was followed to classify each condition under every scenario. The results showed that the LNNDP classification technique provides a classification accuracy significantly different and superior to chance level (\(p<0.05\), Wilcoxon signed-rank test) for seven, ten and seventeen from eighteen participants in the scenarios: (i) relax vs. intention, (ii) relax vs. execution, and (iii) intention vs. execution, respectively. The average of CA is (i) 65.26%, (ii) 69.07%, and (iii) 76.71%; in each case. The better results can be observed for the third classification scenario. However, these results cannot be compared against the related state of the art. Several important differences exist, here some of them: (i) the experiments are quite different (finger extension, curl toes, tongue movement, etc.), (ii) some classification schemes try to identify more than two motor states, (iii) and others include participants with some disable motor condition. This work shows how the classification technique Lattice Neural Network with Dendritic Processing allows to obtain a confident classification accuracy in each movement arm phase, particularly the accurate identification for movement intention can be used in the context of Brain-Computer Interfaces to trigger neurorehabilitation devices. Further schemes for the testing part can be performed; like a continuous sliding window, to detect other interesting measure: the time of movement intention initiation. It could be also interesting as a next step, to perform an on-line experiment to test the applicability over a real neurorehabilitation BCI system.