An EEG-Based Attentiveness Recognition System Using Hilbert–Huang Transform and Support Vector Machine
- 109 Downloads
Abstract
Purpose
Attentiveness recognition benefits the detection of the mental state and concentration when humans perform specific tasks. Hilbert–Huang transform (HHT) is useful for the analysis of nonlinear or nonstationary bio-signals including brainwaves. In this work, a method is proposed for the characterization of attentiveness levels by using electroencephalogram (EEG) signals and HHT analysis.
Methods
Single-channel EEG signals from the frontal area were acquired from participants at different levels of attentiveness and were decomposed into a set of intrinsic mode functions (IMF) by empirical mode decomposition (EMD). Hilbert transform analysis was applied to each IMF to obtain the marginal frequency spectrum. Then the band powers and spectral entropies (SEs) were selected as the attributes of a support vector machine (SVM) for a two-class classification task.
Results
Compared with the predictive models of approximate entropy (ApEn) and fast Fourier transform (FFT), the results show that the band powers extracted from IMF2 to IMF5 of \(\alpha\) and \(\beta\) waves and their SE can best discriminate between attentive and relaxed states with the average classification accuracy of 84.80%.
Conclusion
In conclusion, this integrated signal processing method is capable of attentiveness recognition that can offer efficient differentiation and may be used in a clinical setting for the detection of attention deficit.
Keywords
Attentiveness Electroencephalogram Hilbert-Huang transform Machine learning1 Introduction
Attention is an important feature that reflects the mental state of the brain and can be measured by using electroencephalography (EEG). The measurement of the degree of attention is mainly associated with α and β waves [1]. In particular, α waves between 8 and 13 Hz with amplitudes from 30 to 50 μV are evident on the EEG of a relaxed participant with closed eyes. The β oscillations between 14 and 30 Hz with amplitudes from 5 to 20 μV are evident during active attention. Therefore, quantifying these frequency-specific features using EEG can be used to probe the level of attentiveness [2, 3, 4].
Previous studies have shown that for EEG attentiveness recognition, using a k-nearest neighbor (KNN) classifier based on the self-assessment manikin model can yield an average accuracy of 57.03% [5], and using support vector machine (SVM) model of power spectral density resulted in an average accuracy of 76.82% [6]. In addition, the accuracy can be increased to 81% when taking into account approximate entropy using fuzzy entropy [7]. For identifying attention during the learning process, KNN combining correlation-based feature selection (CFS) yields a classification rate of 80.84% [8]. On a single subject level, the accuracy is up to 89.4% when using the integration of common spatial pattern filtering and nonlinear mutual information method [9]. Taken together, frequency-specific and nonlinear features extracted from EEG are essential for attentiveness recognition. Therefore, in this study, we proposed a method for the characterization of the levels of attentiveness based on Hilbert–Huang transform (HHT) and SVM. HHT and empirical mode decomposition (EMD) have been used to process nonlinear and nonstationary brainwave signals [10, 11] in EEG analysis and clinical applications, such as emotion recognition [12, 13], motor imaginary [14], seizure detection [15, 16, 17], anesthesia monitoring [18, 19], and arousal detection [20]. For attentiveness recognition, HHT combined with extreme learning machine (ELM) has been proposed and yielded the highest accuracy of 85.5% (average accuracy of 72.1%) [21]. SVM, a machine learning technique, gradually becomes a popular translation method for classification with high accuracy [22, 23, 24]. Given a set of training samples for supervised learning, SVM can build a predictive model of the specific EEG features to perform a classifier for attentiveness recognition.
In this study, we measured the brain activity from the frontal area with one channel EEG device during participants solving some puzzles shown on the screen or during a resting period. EEG signals were first decomposed to intrinsic mode functions (IMF) by EMD, after which the instantaneous frequencies of IMFs were obtained by HHT. The resulting marginal spectra (MS) of specific frequency bands and spectral entropy (SE) entered an SVM as the feature attributes for the characterization of attentiveness.
2 Materials and Methods
2.1 Data Collection
EEG data were measured by using a commercial mobile EEG monitor (MindWave, NeuroSky) at a sampling rate of 512 Hz [25]. The unipolar recording device has a fixed channel position on the scalp surface of the forehead (Fp1), according to the International 10/20 system [26]. An ear clip (A1) of the device was used to provide a ground reference to filter out the electrical noise.
A continuous raw EEG signal of a representative participant while conducting two tasks. The participant paid attention for 5 min (until the dotted line) and then took a break with eyes opened for another 5 min. The epochs in the red block were collected as attention data, and the epochs in the green block were collected as relaxation data for further analysis
The flowchart of the proposed method. The EEG data were treated with Hilbert–Huang analysis to obtain the time–frequency information. The extracted features were the input of the support vector machine to build a proper classifier for attentiveness recognition
2.2 Hilbert–Huang Transform
2.3 Feature Selection and Support Vector Machine
Having extracted the frequency-specific power from HHT and computed the SE, we employed the linear forward selection method to reduce the number of attributes that enter SVM.
The SVM was developed from statistical learning theory to analyze a data set for the classification of multi-classes [32, 33]. A data set is trained to acquire a mathematical model, which is used to discriminate a testing data set. For binary classification, an SVM model constructs a hyperplane that optimally separates data sets into one of two classes, and the distance from the hyperplane to the nearest data points on each side is maximized.
In this study, a Gaussian radial basis function (RBF) kernel, \(K({\mathbf{x}},{\mathbf{x}}') = { \exp }( - \gamma {\mathbf{x}} - {\mathbf{x}}'^{2} )\), was used. Both C and γ are carefully chosen to obtain optimal results.
A typical procedure of LIBSVM [33] involves several steps: (1) the input of attributes of a data set with pre-classified indices, (2) training the data to build a model, and (3) predicting the classification or information of a test data set from the model. In the c-support vector classification in this study, the attribute vectors of attention epochs were labeled as class 1 in advance, while those of relaxed epochs were labeled as class − 1. These attribute vectors with two-task labels comprised an input matrix for SVM training. After building the models, the classification of new epochs can be predicted using these models.
3 Results and Discussion
The HHT spectrum of the representative participant while conducting two tasks. The colorbar shows the magnitude of the energy distribution. The conducted task was switched at the 300th second. The time–frequency-energy information of attentive and relaxed tasks are illustrated in red and green blocks, respectively
a IMFs of an EEG epoch when the representative participant was paying attention to solve a puzzle. b IMFs of an EEG epoch when the representative participant had been instructed to relax and take a rest
Marginal spectra of IMF2–5 when the representative participant was a paying attention and b taking a rest
The impact of different attribute vectors on the accuracy of SVM models of a representative participant
# | Attributes | Accuracy (%) |
---|---|---|
1 | α-original, β-original | 85.25 |
2 | α-IMF4, β-IMF4 | 82.00 |
3 | α-IMF2, α-IMF3, α-IMF4, α-IMF5, β-IMF2, β-IMF3, β-IMF4, β-IMF5 | 88.00 |
4 | α-SE, β-SE | 86.25 |
5 | α-original, β-original, α-SE, β-SE | 87.50 |
6 | α-IMF2, α-IMF3, α-IMF4, α-IMF5, β-IMF2, β-IMF3, β-IMF4, β-IMF5, α-SE, β-SE | 93.25 |
7 | α-IMF1, α-IMF2, α-IMF3, α-IMF4, α-IMF5, β-IMF1, β-IMF2, β-IMF3, β-IMF4, β-IMF5, α-SE, β-SE | 91.25 |
8 | α-IMF2, α-IMF3, α-IMF4, α-IMF5, α-IMF6, β-IMF2, β-IMF3, β-IMF4, β-IMF5, β-IMF6, α-SE, β-SE | 91.75 |
9 | α-IMF1, α-IMF2, α-IMF3, α-IMF4, α-IMF5, α-IMF6, β-IMF1, β-IMF2, β-IMF3, β-IMF4, β-IMF5, β-IMF6, α-SE, β-SE | 91.75 |
10 | α-IMF1, α-IMF2, α-IMF3, α-IMF4, α-IMF5, α-IMF6, α-IMF7, α-IMF8, β-IMF1, β-IMF2, β-IMF3, β-IMF4, β-IMF5, β-IMF6, β-IMF7, β-IMF8, α-SE, β-SE | 91.75 |
The contour plot of SVM parameter selection using attribute #6
The accuracy of SVM models of 20 participants using attributes #6
Participant | Accuracy (%) |
---|---|
1 | 77.75 |
2 | 86.75 |
3 | 81.00 |
4 | 85.50 |
5 | 89.75 |
6 | 77.25 |
7 | 77.00 |
8 | 93.50 |
9 | 89.75 |
10 | 81.25 |
11 | 81.50 |
12 | 84.00 |
13 | 83.25 |
14 | 88.25 |
15 | 89.00 |
16 | 72.25 |
17 | 90.25 |
18 | 82.00 |
19 | 93.25 |
20 | 92.75 |
Average | 84.80 |
The accuracy of predictions with independent test data
Participant | Accuracy (%) |
---|---|
1 | 82.00 |
2 | 91.00 |
3 | 92.00 |
4 | 66.00 |
5 | 89.00 |
6 | 76.00 |
7 | 73.00 |
8 | 92.00 |
9 | 89.00 |
10 | 76.00 |
11 | 78.00 |
12 | 80.00 |
13 | 79.00 |
14 | 91.00 |
15 | 96.00 |
16 | 77.00 |
17 | 86.00 |
18 | 82.00 |
19 | 92.00 |
20 | 91.00 |
Average | 83.90 |
The boxplots of the accuracy of SVM models using HHT marginal power density, approximate entropy, and Fourier power density as the attributes
Moreover, in this method, we used one-channel data to build the predictive models and yielded good accuracy up to 93.50%, with an average accuracy of twenty participants being 84.80%. EEG recording using more channels may help improve the system; nevertheless, the single-channel EEG monitor is inexpensive, convenient and portable for the publics. Compared with the similar studies for attentiveness recognition including FFT + SVM [6] (the highest accuracy of 76.82% and the average accuracy of 75.87%) and HHT + ELM [21] (the highest accuracy of 85.50% and the average accuracy of 72.10%), our method has better performance (the highest accuracy of 93.50% and the average accuracy of 84.80%). Our results suggest that using nonlinear HHT method instead of FFT and selecting appropriate features can improve the accuracy in attention recognition. We believe that this convenient method has the potential to be used in clinical settings for the detection of attention deficit hyperactivity disorder (ADHD), or even be a therapeutic tool as part of a biofeedback training system for people who have difficulty in paying attention.
4 Conclusion
A method of feature extraction and characterization of EEG signals using HHT frequency analysis and SVM has been presented. Raw EEG data have been analyzed by HHT to obtain marginal spectra for nonlinear and nonstationary frequency information. The α and β band powers of IMF2–5 and their SEs were selected as the attributes of SVM to obtain the mean accuracy of 84.80%. We conclude that the proposed method can offer efficient differentiation for the assessment of attentiveness, showing promise in applications of attention deficit detection or biofeedback training.
Notes
Acknowledgements
This material is based upon work supported by the Ministry of Science and Technology, Taiwan, under Contract Nos. MOST 106-2221-E-008-042, MOST 106-2221-E-008-066-MY2 and MOST 105-2221-E-008-038.
Funding
Ministry of Science and Technology, Taiwan.
Compliance with Ethical Standards
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical Approval
This study, numbered 201812EM027, was approved by the Research Ethics Committee of National Taiwan University, Taiwan.
References
- 1.Klimesch, W., Doppelmayr, M., Russegger, H., Pachinger, T., & Schwaiger, J. (1998). Induced alpha band power changes in the human EEG and attention. Neuroscience Letters,244(2), 73–76.CrossRefGoogle Scholar
- 2.Lubar, J. F., Swartwood, M. O., Swartwood, J. N., & O’Donnell, P. H. (1995). Evaluation of the effectiveness of EEG neurofeedback training for ADHD in a clinical setting as measured by changes in TOVA scores, behavioral ratings, and WISC-R performance. Biofeedback and Self-Regulation,20(1), 83–99.CrossRefGoogle Scholar
- 3.Hillard, B., El-Baz, A. S., Sears, L., Tasman, A., & Sokhadze, E. M. (2013). Neurofeedback training aimed to improve focused attention and alertness in children with ADHD: A study of relative power of EEG rhythms using custom-made software application. Clinical EEG and Neuroscience,44(3), 193–202.CrossRefGoogle Scholar
- 4.Chiang, H.-S., Hsiao, K.-L., & Liu, L.-C. (2018). EEG-based detection model for evaluating and improving learning attention. Journal of Medical and Biological Engineering,38(6), 847–856.CrossRefGoogle Scholar
- 5.Li, Y., Li, X., Ratcliffe, M., Liu, L., Qi, Y., & Liu, Q. (2011). A real-time EEG-based BCI system for attention recognition in ubiquitous environment. In Proceedings of 2011 international workshop on ubiquitous affective awareness and intelligent interaction (pp. 33–40). ACM.Google Scholar
- 6.Liu, N.-H., Chiang, C.-Y., & Chu, H.-C. (2013). Recognizing the degree of human attention using EEG signals from mobile sensors. Sensors,13(8), 10273–10286.CrossRefGoogle Scholar
- 7.Xu, L., Liu, J., Xiao, G., & Jin, W. (2012). Characterization and classification of EEG attention based on fuzzy entropy. In Third international conference on digital manufacturing and automation (ICDMA) (pp. 277–280). IEEE.Google Scholar
- 8.Hu, B., Li, X., Sun, S., & Ratcliffe, M. (2016). Attention recognition in EEG-based affective learning research using CFS + KNN algorithm. IEEE/ACM Transactions on Computational Biology and Bioinformatics,15(1), 38–45.CrossRefGoogle Scholar
- 9.Hamadicharef, B., Zhang, H., Guan, C., Wang, C., Phua, K. S., Tee, K. P., et al. (2009). Learning EEG-based spectral-spatial patterns for attention level measurement. In IEEE international symposium on circuits and systems (pp. 1465–1468). IEEE.Google Scholar
- 10.Lin, C.-F., & Zhu, J.-D. (2012). Hilbert-Huang transformation-based time-frequency analysis methods in biomedical signal applications. Proceedings of the Institution of Mechanical Engineers Part H—Journal of Engineering in Medicine,226(H3), 208–216.CrossRefGoogle Scholar
- 11.Wolpaw, J. R., Birbaumer, N., Heetderks, W. J., McFarland, D. J., Peckham, P. H., Schalk, G., et al. (2000). Brain-computer interface technology: A review of the first international meeting. IEEE Transactions on Rehabilitation Engineering,8(2), 164–173.CrossRefGoogle Scholar
- 12.Zhuang, N., Zeng, Y., Tong, L., Zhang, C., Zhang, H. M., & Yan, B. (2017). Emotion recognition from EEG signals using multidimensional information in EMD domain. BioMed Research International. https://doi.org/10.1155/2017/8317357.CrossRefGoogle Scholar
- 13.Zhang, Y., Ji, X. M., & Zhang, S. H. (2016). An approach to EEG-based emotion recognition using combined feature extraction method. Neuroscience Letters,633, 152–157.CrossRefGoogle Scholar
- 14.Panoulas, K. I., Hadjileontiadis, L. J., & Panas, S. M. (2008). Hilbert-Huang spectrum as a new field for the identification of EEG event related de-/synchronization for BCI applications. In 30th annual international conference of the IEEE engineering in medicine and biology society, Vancouver, British Columbia, Canada.Google Scholar
- 15.Li, S., Zhou, W., Yuan, Q., Geng, S., & Cai, D. (2013). Feature extraction and recognition of ictal EEG using EMD and SVM. Computers in Biology and Medicine,43(7), 807–816.CrossRefGoogle Scholar
- 16.Oweis, R. J., & Abdulhay, E. W. (2011). Seizure classification in EEG signals utilizing Hilbert-Huang transform. BioMedical Engineering OnLine,10(1), 38.CrossRefGoogle Scholar
- 17.Rehman, N., Xia, Y., & Mandic, D. P. (2010). Application of multivariate empirical mode decomposition for seizure detection in EEG signals. In Annual international conference of the IEEE engineering in medicine and biology, Buenos Aires, Argentina (Vol. 1, pp. 1650–1653).Google Scholar
- 18.Shalbaf, R., Behnam, H., Sleigh, J. W., & Voss, L. J. (2012). Using the Hilbert-Huang transform to measure the electroencephalographic effect of propofol. Physiological Measurement,33(2), 271–285.CrossRefGoogle Scholar
- 19.Chen, S.-J., Peng, C.-J., Chen, Y.-C., Hwang, Y.-R., Lai, Y.-S., Fan, S.-Z., et al. (2016). Comparison of FFT and marginal spectra of EEG using empirical mode decomposition to monitor anesthesia. Computer Methods and Programs in Biomedicine,137, 77–85.CrossRefGoogle Scholar
- 20.Mahalaxmi, U., & Patnaik, M. R. (2017). EEG arousal detection using SVM and EMD based frequency detection method. International Journal of Applied Engineering Research,12(18), 7663–7674.Google Scholar
- 21.Aziz, F. A. A., Shapiai, M. I., Setiawan, N. A., & Mitsukura, Y. (2017). Classification of human concentration in EEG signals using Hilbert Huang transform. International Journal of Simulation: Systems, Science and Technology,18(1), 10.1–10.11.Google Scholar
- 22.Blankertz, B., Muller, K. R., Krusienski, D. J., Schalk, G., Wolpaw, J. R., Schlogl, A., et al. (2006). The BCI competition III: Validating alternative approaches to actual BCI problems. IEEE Transactions on Neural Systems and Rehabilitation Engineering,14(2), 153–159.CrossRefGoogle Scholar
- 23.Garrett, D., Peterson, D. A., Anderson, C. W., & Thaut, M. H. (2003). Comparison of linear, nonlinear, and feature selection methods for EEG signal classification. IEEE Transactions on Neural Systems and Rehabilitation Engineering,11(2), 141–144.CrossRefGoogle Scholar
- 24.Zhang, Y., Wei, S., Zhang, L., & Liu, C. (2019). Comparing the performance of Random Forest, SVM and their variants for ECG quality assessment combined with nonlinear features. Journal of Medical and Biological Engineering,39(3), 381–392.CrossRefGoogle Scholar
- 25.Sałabun, W. (2014). Processing and spectral analysis of the raw EEG signal from the MindWave. Przeglad Elektrotechniczny,90(2), 169–174.Google Scholar
- 26.Jasper, H. H. (1958). Report of the committee on methods of clinical examination in electroencephalography. Electroencephalography and Clinical Neurophysiology,10(2), 370–375.CrossRefGoogle Scholar
- 27.Huang, N. E., Shen, Z., Long, S. R., Wu, M. C., Shih, H. H., Zheng, Q., et al. (1998). The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis. Proceedings of the Royal Society of London A: Mathematical, Physical and Engineering Sciences,454(1971), 903–995.MathSciNetzbMATHCrossRefGoogle Scholar
- 28.Boashash, B. (1992). Estimating and interpreting the instantaneous frequency of a signal-Part I: Fundamentals. Proceedings of the IEEE,80(4), 520–538.CrossRefGoogle Scholar
- 29.Bedrosian, E. (1963). A product theorem for Hilbert transforms. Proceedings of the IEEE,51(5), 868–869.CrossRefGoogle Scholar
- 30.Powell, G. E., & Percival, I. C. (1979). A spectral entropy method for distinguishing regular and irregular motion of Hamiltonian systems. Journal of Physics A: Mathematical and General,12(11), 2053.CrossRefGoogle Scholar
- 31.Humeau-Heurtier, A., Wu, C. W., Wu, S. D., Mah, G., & Abraham, P. (2016). Refined multiscale Hilbert-Huang spectral entropy and its application to central and peripheral cardiovascular data. IEEE Transactions on Biomedical Engineering,63(11), 2405–2415.CrossRefGoogle Scholar
- 32.Lotte, F., Congedo, M., Lecuyer, A., Lamarche, F., & Arnaldi, B. (2007). A review of classification algorithms for EEG-based brain-computer interfaces. Journal of Neural Engineering,4(2), R1–R13.CrossRefGoogle Scholar
- 33.Chang, C.-C., & Lin, C.-J. (2011). LIBSVM: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology (TIST),2(3), 27.Google Scholar
- 34.Kimeldorf, G., & Wahba, G. (1971). Some results on Tchebycheffian spline functions. Journal of Mathematical Analysis and Applications,33(1), 82–95.MathSciNetzbMATHCrossRefGoogle Scholar
- 35.Sanchez, J. C., Carmena, J. M., Lebedev, M. A., Nicolelis, M. A. L., Harris, J. G., & Principe, J. C. (2004). Ascertaining the importance of neurons to develop better brain-machine interfaces. IEEE Transactions on Biomedical Engineering,51(6), 943–953.CrossRefGoogle Scholar
Copyright information
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.