EEG Pattern Recognition: An Efficient Improvement Combination of ERD/ERS/Laterality Features to Create a Self-paced BCI System

Abstract

In this paper, a new method based on an efficient improvement combination of Event-Related Desynchronization (ERD), Event-Related Synchronization (ERS) and lateral activity of sensorimotor cortex features is presented to analyze both left and right hand motor imagery tasks. Our proposal uses delta, theta, alfa and beta rhythms to BCI system. From the spectral power, an efficient combination of ERD/ERS/laterality features was used. Because electroencephalogram signals are non-stationary type and highly vary over time and frequency, a detailed time-frequency analysis is applied. Features coming from time-frequency analysis, where eight frequency bands ranging from 0 to 32 Hz were chosen. Features vectors are classified by Gaussian classifier and the final performance is evaluated in cross-validation scheme. This novel approach was tested using the BCI competition IV data set 1. The detection of the left and right hand motor imagery task was very good, with a result of $96.4\,\%$ using BCI-Competition -IV. When comparing results from others competing methods reported in the literature, our approach resulted the best and useful to create a self-paced BCI-system.

1 Introduction

There is a channel for communicating the brain and the external environment based on EEG signals. This channel is the so-called Brain Computer Interface (BCI), which offers an effective help for people with motor disabilities [1] such as amyotrophic lateral sclerosis [2] or spinal cord injury [3]. Several studies have shown that the motor imagery area (contralateral and ipsilateral sensorimotor cortex, respectively) provides information about imagery movements of hands, feet and tongue. In this way, signals are manifested as ERD/ERS (event-related desynchronization/synchronization) [2, 4–6]. Due to the fact that the ERD/ERS patterns are opposite, imaginary movements of each hand and foot are suitable to be classified [3, 7] $\mu$ and $\beta$ rhythms registered on the sensorimotor area can be modified by executing either imaginary or observed hand and foot movements. Also, $\mu$ and $\beta$ rhythms may be used to support EEG-based BCI systems (BCIs) [8]. The selection of adequate EEG features in motor imagery movements is crucial, since it determines the accuracy of the classification. Therefore, for attaining good BCI systems, the choice of appropriate and reliable features coming from EEG signals constitutes a very important issue. This is how, in frequency domain, Pregenzer showed the importance of an appropriate preselection of EEG spectral components for accomplishing good classification results [9]. Pregenzer used different frequency bands and resolutions coming from Morlet wavelets, combined with Fisher criteria in order to classify two clases: the left and the right hand motor imagery, obtaining upto $90\,\%$ of good classification. In order to set up a realistic comparison between different methods for processing and classifying EEG signals, the BCI Competition was proposed. Specifically, Data Set I for BCI Competition IV [10] was focused on a benchmark to classify two mental tasks: right-hand imaginary movement and left-hand imaginary movement. With this benchmark, signals obtained with EEG were provided and in addition, they were obtained following the same scheme used by the other competition participants. Also, the rules and requirements were laid down. For the present work, the chief functional blocks are shown in Fig. 1. The first module “EEG signal acquisition” reads and transmits EEG signals from an Epoc-Emotiv device to a computer. The second block “signal processing” selects C3 and C4 channel and normalizes amplitudes. The third block develops a time-frequency transformation through a Short-Time Fourier Transform. The fourth block decomposes an EEG signal into 8 frequency bands. The next block “ERS/ERD detection” constitutes the core of our proposal, detecting $\alpha$, $\beta$ and $\mu$ rhythms over 8 band frequencies. Finally, the “classification motor imagery” block senses 2 mental imagery movements and no-control activity calculating the difference between C3 and C4. The rest of the paper is structured as follows. In Sect. 2, Event-Related Desynchronization/Synchronization is related to alfa,beta and mu rhythms. Materials and Methods are presented in Sect. 3. Section 4 shows and discusses experimental results. Finally, conclusions and future works are given in Sect. 5.

Fig. 1.

General functional blocks of the EEG-based BCI system

2 Event-Related Desynchronization/Event-Related Synchronization for Brain-Computer Interface

Motor imagery affects the frequency band in the interval of 0.1–32 Hz. Authors typically reference two specific bands: the $\mu$ band (8–12 Hz.) and the $\beta$ band (16–24 Hz.) [3, 7, 8, 11]. The motor imagery process has two leading steps, the first one is related to the Event-Related Desynchronization (ERD), which affects both the $\mu$ and the $\beta$ bands (decrease in activity); the second one event is related to the Event-Related Synchronization (ERS), which particularly affects the $\beta$ band. In this study, we address for the first time the use of the whole band frequency from 0 to 32 Hz., instead of only the $\mu$ and the $\beta$ bands. Our proposal is based on searching the best representative frequency interval in order to efficiently detect motor imagery self-paced cues.

3 Materials and Methods

3.1 Materials

This study reports results based on data from two sources: the first one constituted by the BCI Competition IV data sets I, provided by the Berlin Group [10]. Those data were selected because they present an asynchronous approach, which is suitable for our main purpose. The second one consists of the EEG recordings, which comes from an EPOC headset provided by Emotiv Systems¹, which is owned by our own laboratory.

BCI Competition IV Data Sets I

The BCI Competition IV Data Sets I contain the EEG signal recorded from 9 healthy people performing motor imagery tasks. The EEG data were recorded from 59 channels, at a rate of 100 samples per second and per subject. The classes of mental task are: (i) imaginary movement of the left hand, (ii) imaginary movement of the right hand and (iii) imaginary movement of any foot. Calibration data were recorded as follows: an arrow was displayed on a computer screen indicating the class of the motor imagery task to be performed, the arrow was presented for a period of 4 s, during which the subject was supposed to imagine the performance of the movement. Periods of time were interleaved with 2 s of blank screen and with 2 s of a cross in the center of the screen. The cross was superimposed to the cue, so it was displayed for 6 s.

3.2 Time-Frequency Representation

General EEG recorded data are represented by the array X (1), where X $\in \mathfrak {R}^{NxM}$, N represents the number of available channels, M represents the number of samples per channel, $x_{c,i}$ represents i channel where $i \in {1...N}$. Each signal is processed in one-second window.

(1)

Giving that the EEG signal is non-stationary type, one sample shift is taken from one window to the next one (one-second window each time).

The mean of the signal for each channel $\overline{{x}_{ci}}$ is subtracted from every $x_{ci}$ row to eliminate the offset and to produce $\widetilde{{x}_{ci}}$. The spectral power $P_{ci}$ of each channel is calculated using the Short-Fourier Transform (SFT) ($\mathscr {F}$). In order to reduce high frequency artifacts due to windowing process, a blackman window is used (2) to calculate the SFT. In order to get real power spectrum $P_{ci}$, the spectral power is multiplied by its complex conjugate ($\mathscr {F}^*$) (3).

$$\begin{aligned} \mathscr {F} \{ \widetilde{{x}_{ci}^k} \} \; = \; \sum _{n=-\infty }^{\infty } \widetilde{{x}_{ci}} W_{Blackman}[n] \pmb {e}^{-{ j\omega n}} \; \end{aligned}$$

(2)

$$\begin{aligned} P_{ci} = \mathscr {F} \{ \widetilde{{x}_{ci}} \} \;\; \cdot \;\; \mathscr {F}^*\{ \widetilde{{x}_{ci}} \} \; \end{aligned}$$

(3)

The whole EEG frequencies (between 0 and 32 Hz.) associated to the delta ($\delta$), theta ($\theta$), alpha ($\alpha$) and beta ($\beta$) rhythms, constitute the most important part of the spectral power $P_{ci}$ since the purpose is to detect ERD/ERS complexes.

3.3 Feature Extraction

From the power representation $P_{ci}$ (3), ranging from 0 to 32 Hz., we compose eight cumulative power values as: ${P}_{ci}^1 = \sum P_{ci} \in (0 - 4]\,Hz.$, ${P}_{ci}^2 = \sum P_{ci} \in (4 - 8]\,Hz.$, ${P}_{ci}^3 = \sum P_{ci} \in (8 - 12]\,Hz.$, ${P}_{ci}^4 = \sum P_{ci} \in (12 - 16]\,Hz.$, ${P}_{ci}^5 = \sum P_{ci} \in (16 - 20]\,Hz.$, ${P}_{ci}^6 = \sum P_{ci} \in (20 - 24]\,Hz.$, ${P}_{ci}^7 = \sum P_{ci} \in (24 - 28]\,Hz.$, ${P}_{ci}^8 = \sum P_{ci} \in (28 - 32]\,Hz.$ (see Fig. 2), each of them associated to the delta ($\delta$), theta ($\theta$), alpha ($\alpha$) and beta ($\beta$) rhythms.

Fig. 2.

Interval frequency division of 8 bands

3.4 Event-Related Desynchronization/Event-Related Synchronization Detection

In this article, any reference to ERD/ERS refers to the frequency combination at a giving time, rather than the sequential-spatial distributed ERD/ERS phenomenon [6]. Motor imagery tasks are detected through out ERD/ERS signals; three states were monitored from eight different frequency bands at the same time: attenuation (related to ERD), enhancement (related to ERS) and laterality (related to $C4-C3$ difference).

3.5 Imagery Movement Detection

Starting from the spectral power ${P}_{ci}^j$ (for $j=1,2,3...8$), some bands were attenuated (ERD) and others were enhanced (ERS) (performing imagery movements of hands and feets). Another parameter that was analyzed was the lateral activity (motor imagery movement: left or right hand), called laterality (defined as the difference between the C4 and the C3 channel).

For each $P_{ci}^j$ signal a low-pass FIR filter was applied in order to eliminate artifacts and to get a flat signal. Filtered $P_{ci}^j$ gives us better conditions to determine: (1) attenuated signal (ERD) or enhanced signal (ERS).

The main task was sensing when the frequency band is higher or lower than upper threshold or bottom threshold; and also sensing if the difference between two channels (C4−C3) was higher or lower than a lateral threshold.

One event happens if the signal $P_{ci}^j$ combinations were turned on (enhancing event) and one of its were turned off (attenuating event).

At the end, each $i-$channel results with an attenuating event, an enhancing event, and a lateral event, $\pmb {{T}}_{ci}^a$, $\pmb {{T}}_{ci}^e$ and $\pmb {{T}}_{ci}^l$ respectively. The eight-dimension $\pmb {{T}}_{ci}^{a,e,l}$ threshold defines a particular motor imagery movement (left or right hand imagery movement).

3.6 Classification

The whole power spectral $P_{ci}$ was used to build a describing feature vector. The feature vector was conformed by slopes (gradient) of $P_{ci}$ for a given time, particularly, around motor imagery event. In order to obtain invariance from one person to other person in the feature vector, whole power espectral matriz was centered-reduced (zero mean value and one standard deviation). The slope was estimated by the difference between left and right power spectral values $\nabla P(k)_{ci}^{CR}$ at a given k time (see Eq. 4).

$$\begin{aligned} \nabla P(k)_{ci}^{CR} = ( P(k+1)_{ci}^{k,CR} - P(k-1)_{ci}^{CR} ) \end{aligned}$$

(4)

From $\nabla P(k)_{ci}^{CR}$, we have conformed four histograms according to four orientations: h_1 $= \sum slopes \in [0^{\circ }, 45^{\circ })$, h_2 $= = \sum slopes \in [45^{\circ }, 90^{\circ })$, h_3 $= = \sum slopes \in [-45^{\circ }, 0^{\circ })$ and h_4 $= = \sum slopes \in (-90^{\circ }, -45^{\circ })$. The histograms are then weighted with the magnitude of the gradient. Finally, four histograms are obtained for each channel and each band associated to imagery movements.

The final feature descriptor vector $\overrightarrow{d}$ was conformed by histograms (8 bands multiplied by 4 histograms). A 32-dimension vector is used for classification. A Gaussian classifier (see Eq. 5) was used in cross-validation mode ($50\,\%-50\,\%$) to evaluate the classification performance.

$$\begin{aligned} p( \overrightarrow{d} / class_{i}) = (2\pi )^{-n/2} \Vert \Sigma _i \Vert ^{-1/2} \exp ^{[ -\frac{1}{2} ( \overrightarrow{d}- \overrightarrow{m_{class_i}})^{T} \Sigma _i^{-1} ( \overrightarrow{d}- \overrightarrow{m_{class_i}})] } \end{aligned}$$

(5)

4 Results and Discussions

Proposed pattern recognition methodology for motor imagery detection was applied for offline motor imagery detection with well-known BCI Competition IV data set1. To evaluate motor imagery (intention of movement), a detection rate and an associated noise are calculated from (6) and (7), where TP stands for True Positives, FP for False Positives and FN for False Negatives.

$$\begin{aligned} \displaystyle { \text{ Detection } \text{ Rate } = {\frac{TP}{(TP+FN)}}\; . } \end{aligned}$$

(6)

$$\begin{aligned} \displaystyle { \text{ Miss } \text{ Detection } \text{ Rate } = 1 - {\frac{TP}{(TP+FP)}}\; . } \end{aligned}$$

(7)

4.1 Offline Motor Imagery Detection

ERD/ERS/laterality analysis is based on the EEG activity during and after a motor imagery activity. First, the BCI Competition IV Data Sets I was analyzed (five subjects were taken: BCICIV_calib_ds1b, BCICIV_calib_ds1c, BCICIV_calib_ds1d, BCICIV_calib_ds1e and BCICIV_calib_ds1g). From Data set I (59 channels at 100 samples per second; healthy people), only the C3 and the C4 channels were selected, electrodes corresponding to the sensory-motor area ($27^{th}$ and $31^{st}$ channels from BCI Competition IV data set 1). In order to analyze the EEG recordings and the ERD/ERS/laterality complexes there in contained, the EEG recordings are transformed to a time-frequency representation (spectrogram through Short-Time Fourier Transform). Figure 3 exemplifies the detection of the imagery motor originated from the $BCICIV\_calib\_ds1b$ database. There, the initial points, indicating the beginning of the intention of movement, are shown. The initial points provided by the database are marked as ($\rhd$ marks) and the the initial points originated from our methodology are marked as ($*$). Also, it is indicated the false positive points and the true positive points. The signals coming from two different channels are drawn together in two colors: red and blue. Red color indicates C3 channel whereas the blue color indicates C4 channel. In order to illustrate in a clearer manner the detection of the imagery motor, Fig. 3(b) depicts an amplification of a region of the last Fig. (3(a)). There, it is indicated how the two types of initial points, just described, coincide. Figure 3 shows motor imagery cues detected with our proposal ($*$ marks) and with database competition ($\rhd$ marks). The thresholds $\pmb {{T}}_{ci}^a$, $\pmb {{T}}_{ci}^e$ and $\pmb {{T}}_{ci}^l$ were estimated experimentally and the values were: $\pmb {{T}}_{ci}^a=[0.20, 0, 0, 0, 0, 0, 0, 0]$, $\pmb {{T}}_{ci}^e=[5, 0.02, 0.005, 0.005, 0.0005, 0.0005, 0.0005, 0.0005]$, and $\pmb {{T}}_{ci}^l= [0.003, 0.005, 0.007, 0.007, 0.0005, 0.0003, 0.0003, 0.0003]$ for an attenuating event, an enhancing event, and a lateral event, respectively. Above thresholds were used for the five people under analysis. Our methodology detects up to $99\,\%$ (mean of $96.4\,\%$) of the imagery movements for the five evaluated dataset coming from the BCI Competition IV data. The Miss Classification represents the detection of False Positives and it was around 0.116, as shown in Table 1.

Table 1. Imagery movement detection and miss detection for BCI competition IV dataset I.

Using the BCI-Competition-IV data [10], we compared the detection performance of our proposal vs. five competitive methods reported in the literature (see Table 2). The five previous methods were: (a) Common Spatial Pattern (CSP), (b) Discriminative Common Spatial Pattern (DCSP) [12], (c) Local Temporal Common Spatial Pattern (LTCSP) [13], (d) Spectrally and Temporally Weighted Classification Method (STWCM) [14] and Time-series discrimination using feature relevance analysis in motor imagery classification (TSDFRAMI) [15]. As you can see in Table 2, our methodology proved a significant improvement on detecting the intention of movement for the right or left arm when it was applied on the data base provided by BCI-Competition IV. Further, when it was compared with the most competitive methods, it provided an improvement by going from $92.86\,\%$ (Time-series discrimination using feature relevance analysis in motor imagery classification (TSDFRAMI)) to $96.4\,\%$, i.e. $3.54\,\%$ of improvement over the best existing method for detecting imagery movements.

Fig. 3.

An example of correct cues detection: $*$ corresponds to our methodology detection, and $\rhd$ corresponds to the true competition cues (Color figure online)

Table 2. Detection performance of the most competitive methods in detection of motor imagery for BCI competition IV: Common Spatial Pattern (CSP), Discriminative Common Spatial Pattern (DCSP), Local Temporal Common Spatial Pattern (LTCSP), Spectrally and Temporally Weighted Classification Method (STWCM), Time-series discrimination using feature relevance analysis in motor imagery classification (TSDFRAMI) and the proposed method.

5 Conclusion and Future Work

A new methodology to detect and to classify imagery movements was proposed. It is based on an efficient improvement of the combination of attributes originating from the Event-Related Desynchronization (ERD), Event-Related Synchronization (ERS) and the lateral activity of the sensorimotor cortex. These attributes were calculated by analyzing time-frequency spectrograms ranging from 0 to 32 Hz due to the non-stationarity of EEG signals.

Our approach shows its efficiency by using only two channels (C3 and C4) taken from sensorimotor cortex region, instead of using the whole 59 channels. With these two channels it is possible to detect the activity of the imagery motor (activity of control) or no-activity (activity of no-control) as well as classify left and right motor imagery.

Also, our methodology proved a significant improvement on detecting the intention of movement for the right or left arm when it was applied on the data base provided by BCI-Competition IV. Further, when it was compared with the most competitive methods, it provided an improvement by going from $92.86\,\%$ (Time-series discrimination using feature relevance analysis in motor imagery classification (TSDFRAMI)) to $96.4\,\%$, i.e. $3.54\,\%$ of improvement over the best existing method for detecting imagery movements.

These results suggest that our approach may be utilize as an efficient switch between activity of control and activity of no-control. Activity of control represents wanting to make a movement with the hands while activity of no-control stands for no intention of any activity, useful to create a asynchronous self-paced BCI-system.

Future work most be focus on the way to automatically determine the following thresholds: $\pmb {{T}}_{ci}^a$, $\pmb {{T}}_{ci}^e$ and $\pmb {{T}}_{ci}^l$, for each person and each kind of test. The authors plan to improve the learning phase (time reduction) by parallel processing, as well as real-time implementation of the whole proposal BCI-asynchronous system. The goal will be to implement control mobile device i.e. tablet, cellphone; by motor imagery.