Skip to main content
SpringerLink
Log in

On the Use of Brain Decoded Signals for Online User Adaptive Gesture Recognition Systems

  • Conference paper
Pervasive Computing (Pervasive 2010)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 6030))

Included in the following conference series:

Abstract

Activity and context recognition in pervasive and wearable computing ought to continuously adapt to changes typical of open-ended scenarios, such as changing users, sensor characteristics, user expectations, or user motor patterns due to learning or aging. System performance inherently relates to the user’s perception of the system behavior. Thus, the user should be guiding the adaptation process. This should be automatic, transparent, and unconscious.

We capitalize on advances in electroencephalography (EEG) signal processing that allow for error related potentials (ErrP) recognition. ErrP are emitted when a human observes an unexpected behavior in a system. We propose and evaluate a hand gesture recognition system from wearable motion sensors that adapts online by taking advantage of ErrP. Thus the gesture recognition system becomes self-aware of its performance, and can self-improve through re-occurring detection of ErrP signals.

Results show that our adaptation technique can improve the accuracy of a user independent gesture recognition system by 13.9% when ErrP recognition is perfect. When ErrP recognition errors are factored in, recognition accuracy increases by 4.9%. We characterize the boundary conditions of ErrP recognition guaranteeing beneficial adaptation. The adaptive algorithms are applicable to other forms of activity recognition, and can also use explicit user feedback rather than ErrP.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Ward, J.A.: Activity Monitoring: Continuous Recognition and Performance Evaluation. PhD thesis, ETH Zurich, Nr. 16520 (2006)

    Google Scholar 

  2. Davies, N., Siewiorek, D.P., Sukthankar, R.: Special issue: Activity-based computing. IEEE Pervasive Computing 7(2), 20–21 (2008)

    Article  Google Scholar 

  3. Ravi, N., Dandekar, N., Mysore, P., Littman, M.L.: Activity recognition from accelerometer data. American Association for Artificial Intelligence (2005)

    Google Scholar 

  4. Stiefmeier, T., Roggen, D., Ogris, G., Lukowicz, P., Tröster, G.: Wearable activity tracking in car manufacturing. IEEE Pervasive Computing Magazine 7(2), 42–50 (2008)

    Article  Google Scholar 

  5. Bao, L., Intille, S.S.: Activity recognition from user-annotated acceleration data. In: Pervasive Computing: Proc. of the 2nd Int Conference, pp. 1–17 (2004)

    Google Scholar 

  6. Lester, J., Choudhury, T., Borriello, G.: A practical approach to recognizing physical activities. In: Fishkin, K.P., Schiele, B., Nixon, P., Quigley, A. (eds.) PERVASIVE 2006. LNCS, vol. 3968, pp. 1–16. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  7. Nieuwenhuis, S., Ridderinkhof, K.R., Blom, J., Band, G.P., Kok, A.: Error-related brain potentials are differentially related to awareness of response errors: Evidence from an antisaccade task. Psychophysiology 38(5), 752–760 (2001)

    Article  Google Scholar 

  8. Yasuda, A., Sato, A., Miyawaki, K., Kumano, H., Kuboki, T.: Error-related negativity reflects detection of negative reward prediction error. Neuroreport 15(16), 2561–2565 (2004)

    Article  Google Scholar 

  9. Frank, M.J., Woroch, B.S., Curran, T.: Error-related negativity predicts reinforcement learning and conflict biases. Neuron 47(4), 495–501 (2005)

    Article  Google Scholar 

  10. Santosh, K.C., Nattee, C.: A comprehensive survey on on-line handwriting recognition technology and its real application to the nepalese natural handwriting (2009)

    Google Scholar 

  11. Tang, Y., Rose, R.: Rapid speaker adaptation using clustered maximum-likelihood linear basis with sparse training data. IEEE Transactions on Audio, Speech, and Language Processing 16(3), 607–616 (2008)

    Article  Google Scholar 

  12. Baker, J.M., Deng, L., Glass, J., Khudanpur, S., Lee, C.-H., Morgan, N., OShaughnessy, D.: Research developments and directions in speech recognition and understanding, part 1. IEEE Signal Processing Magazine 26(3), 75–80 (2009)

    Article  Google Scholar 

  13. Ohmura, R., Hashida, N., Imai, M.: Preliminary evaluation of personal adaptation techniques in accelerometer-based activity recognition. In: Proc. 13th IEEE Int. Symposium on Wearable Computers: Late Breaking Results (2009)

    Google Scholar 

  14. He, X., Zhao, Y.: Fast model selection based speaker adaptation for nonnative speech. IEEE Trans. on Speech and Audio Processing 11(4), 298–307 (2003)

    Article  MathSciNet  Google Scholar 

  15. Kunze, K., Lukowicz, P.: Using acceleration signatures from everyday activities for on-body device location. In: 2007 11th IEEE International Symposium on Wearable Computers, September 2007, pp. 115–116 (2007)

    Google Scholar 

  16. Förster, K., Roggen, D., Tröster, G.: Unsupervised classifier self-calibration through repeated context occurences: Is there robustness against sensor displacement to gain? In: Proc. 13th IEEE Int. Symposium on Wearable Computers (ISWC), pp. 77–84 (2009)

    Google Scholar 

  17. Taylor, S.F., Stern, E.R., Gehring, W.J.: Neural systems for error monitoring: Recent findings and theoretical perspectives. Neuroscientist 13(2), 160–172 (2007)

    Article  Google Scholar 

  18. Falkenstein, M., Hoormann, J., Christ, S., Hohnsbein, J.: ERP components on reaction errors and their functional significance: A tutorial. Biol. Psychol. 51(2-3), 87–107 (2000)

    Article  Google Scholar 

  19. Ferrez, P.W., Millán, J.: Error-related EEG potentials generated during simulated brain-computer interaction. IEEE Trans. Biomed. Eng. 55, 923–929 (2008)

    Article  Google Scholar 

  20. Schalk, G., Wolpaw, J.R., McFarland, D.J., Pfurtscheller, G.: EEG-based communication: Presence of an error potential. Clin. Neurophysiol. 111(12), 2138–2144 (2000)

    Article  Google Scholar 

  21. Parra, L.C., Spence, C.D., Gerson, A.D., Sajda, P.: Response error correction–A demonstration of improved human-machine performance using real-time EEG monitoring. IEEE Trans. Neural. Syst. Rehabil. Eng. 11(2), 173–177 (2003)

    Article  Google Scholar 

  22. Fatourechi, M., Bashashati, A., Ward, R.K., Birch, G.E.: EMG and EOG artifacts in brain computer interface systems: A survey. Clin. Neurophysiol. 118(3), 480–494 (2007)

    Article  Google Scholar 

  23. Chavarriaga, R., Ferrez, P.W., Millán, J.: To Err Is Human: Learning from error potentials in brain-computer interfaces. In: International Conference on Cognitive Neurodynamics (2007)

    Google Scholar 

  24. Bollon, J.M., Chavarriaga, R., Millán, J., Bessière, P.: EEG error-related potentials detection with a Bayesian filter. In: 4th International IEEE EMBS Conference on Neural Engineering, Antalya Turkey (2009)

    Google Scholar 

  25. Gehring, W.J., Goss, B., Coles, M.G.H., Meyer, D.E., Donchin, E.A.: Neural system for error-detection and compensation. Psychol. Sci. 4, 385–390 (1993)

    Article  Google Scholar 

  26. Schlögl, A., Keinrath, C., Zimmermann, D., Scherer, R., Leeb, R., Pfurtscheller, G.: A fully automated correction method of EOG artifacts in EEG recordings. Clin. Neurophysiol. 118(1), 98–104 (2007)

    Article  Google Scholar 

  27. Liu, H., Setiono, R.: A probabilistic approach to feature selection - a filter solution, pp. 319–327. Morgan Kaufmann, San Francisco

    Google Scholar 

  28. García Lopez, F., García Torres, M., Melian Batista, B., Moreno Perez, J.A., Moreno-Vega, J.M.: Solving feature subset selection problem by a parallel scatter search. European Journal of Operational Research 169(2), 477–489 (2006)

    Article  MATH  MathSciNet  Google Scholar 

  29. John, G., Langley, P.: Estimating continuous distributions in Bayesian classifiers. In: Proceedings of the Eleventh Conference on Uncertainty in Artificial Intelligence, pp. 338–345. Morgan Kaufmann, San Francisco (1995)

    Google Scholar 

  30. Castillo, E., Gutiérrez, J.M., Hadi, A.S.: Expert Systems and Probabilistic Network Models, Erste edn. Springer, New York (1996)

    Google Scholar 

  31. Aha, D.W., Kibler, D.: Instance-based learning algorithms. In: Machine Learning, pp. 37–66 (1991)

    Google Scholar 

  32. Witten, I.H., Frank, E.: Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations, 1st edn. The Morgan Kaufmann Series in Data Management Systems. Morgan Kaufmann, San Francisco (1999)

    Google Scholar 

  33. Tsymbal, A.: The problem of concept drift: Definitions and related work. Technical report, Department of Computer Science, Trinity College (2004)

    Google Scholar 

  34. Casson, A., Smith, S., Duncan, J., Rodriguez-Villegas, E.: Wearable EEG: what is it, why is it needed and what does it entail? In: Proc. IEEE Eng. Med. Biol. Soc., pp. 5867–5870 (2008)

    Google Scholar 

  35. Garipelli, G., Galán, F., Chavarriaga, R., Ferrez, P.W., Lew, E., Millán, J.: The use of Brain-Computer Interfacing for Ambient Intelligence. In: Intl. Workshop on Human Aspects in Ambient Intelligence (2007)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. IFE, Wearable Computing Lab, ETH Zurich, CH-8092, Zurich, Switzerland

    Kilian Förster, Daniel Roggen & Gerhard Tröster

  2. EPFL, CNBI, Center for Neuroprosthetics, CH-1015, Lausanne, Switzerland

    Andrea Biasiucci, Ricardo Chavarriaga & José del R. Millán

  3. Department of Informatics, Systems and Telematics (DIST), University of Genova, 16126, Genova, Italy

    Andrea Biasiucci

Authors
  1. Kilian Förster
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrea Biasiucci
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ricardo Chavarriaga
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. José del R. Millán
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Daniel Roggen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Gerhard Tröster
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Helsinki Institute for Information Technology – HIIT, Department of Computer Science,, University of Helsinki, P.O. Box 68, 00014, Finland

    Patrik Floréen

  2. German Research Center for Artificial Intelligence (DFKI), Stuhlsatzenhausweg 3, 66123, Saarbrücken, Germany

    Antonio Krüger

  3. Nokia Research Center, Palo Alto, CA, USA

    Mirjana Spasojevic

Rights and permissions

Reprints and permissions

Copyright information

© 2010 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Förster, K., Biasiucci, A., Chavarriaga, R., del R. Millán, J., Roggen, D., Tröster, G. (2010). On the Use of Brain Decoded Signals for Online User Adaptive Gesture Recognition Systems. In: Floréen, P., Krüger, A., Spasojevic, M. (eds) Pervasive Computing. Pervasive 2010. Lecture Notes in Computer Science, vol 6030. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-12654-3_25

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-12654-3_25

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-12653-6

  • Online ISBN: 978-3-642-12654-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation