Journal of Intelligent & Robotic Systems

, Volume 87, Issue 2, pp 247–263 | Cite as

Design and Implementation of a Multi Sensor Based Brain Computer Interface for a Robotic Wheelchair

  • Gurkan Kucukyildiz
  • Hasan Ocak
  • Suat Karakaya
  • Omer Sayli


In this study, design and implementation of a multi sensor based brain computer interface for disabled and/or elderly people is proposed. Developed system consists of a wheelchair, a high-power motor controller card, a Kinect camera, electromyogram (EMG) and electroencephalogram (EEG) sensors and a computer. The Kinect sensor is installed on the system to provide safe navigation for the system. Depth frames, captured by the Kinect’s infra-red (IR) camera, are processed with a custom image processing algorithm in order to detect obstacles around the wheelchair. A Consumer grade EMG device (Thalmic Labs) was used to obtain eight channels of EMG data. Four different hand movements: Fist, release, waving hand left and right are used for EMG based control of the robotic wheelchair. EMG data is first classified using artificial neural network (ANN), support vector machines and random forest schemes. The class is then decided by a rule-based scheme constructed on the individual outputs of the three classifiers. EEG based control is adopted as an alternative controller for the developed robotic wheelchair. A wireless 14-channels EEG sensor (Emotiv Epoch) is used to acquire real time EEG data. Three different cognitive tasks: Relaxing, math problem solving, text reading are defined for the EEG based control of the system. Subjects were asked to accomplish the relative cognitive task in order to control the wheelchair. During experiments, all subjects were able to control the robotic wheelchair by hand movements and track a pre-determined route with a reasonable accuracy. The results for the EEG based control of the robotic wheelchair are promising though vary depending on user experience.


Robotic wheelchair EEG EMG Navigation Brain-computer interface 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Boquete, L., Garcia, R., Barea, R., Mazo, M.: Neural control of the movements of a wheelchair. Int. J. Intell. Robot. Syst. 25, 213–226 (1999)MATHGoogle Scholar
  2. 2.
    Barea, R., Boquete, L., Mazo, M., Lopez, E.: Wheelchair guidance strategies using EOG. Int. J. Intell. Robot. Syst. 34, 279–299 (2002)MATHGoogle Scholar
  3. 3.
    Jeonghee, K., Hangue, P., Joy, B., Erica, S., Diane, R., Deborah, P., Jaimee, H., Julia, M., Beatrice, N., Dennis, W., Anne, L., Eliot, R., Mike, J., Emir, V., Maysam, G.: The tongue enables computer and wheelchair control for people with spinal cord injury. Sci. Trans. Med. 5(1), 166–213 (2013)Google Scholar
  4. 4.
    Carlson, T, Millan, J.R.: Brain-controlled wheelchairs: A robotic architecture. IEEE Robot. Autom. Mag. 20(1), 65–73 (2013)Google Scholar
  5. 5.
    Kim, K.H., Kim, H.K., Kim, J.S., Wookho, S., Lee, S.D.: An EMG-based human interface for the control of an electrically powered wheelchair. Int. J. Robot. Autom 28(1), 111–114 (2006)Google Scholar
  6. 6.
    Grasse, R., Morere, Y., Pruski, A.: Assisted navigation for persons with reduced mobility: path recognition through particle filtering (condensation algorithm). Int. J. Intell. Robot. Syst. 60, 19–57 (2010)MATHGoogle Scholar
  7. 7.
    Yoshiyuki, N., Akira, K., Kazuhiko, T.: A mechatronics vision for smart wheelchairs. Int. J. Robot. Autom. 30(1), 611–627 (2008)Google Scholar
  8. 8.
    Gulrez, T., Tognetti, A.: A sensorized garment controlled virtual robotic wheelchair. Int. J. Intell. Robot. Syst. 74, 847–868 (2014)Google Scholar
  9. 9.
    Zhenghui, G., Zhuliang, Y., Zhifang, S.: An online semi-supervised brain–computer interface. IEEE Trans. Biomed. Eng. 8, 2614–2623 (2013)Google Scholar
  10. 10.
    Christopher, B., Alric, A., Paul, D.G.: A Brain? Computer Interface (BCI) for the detection of mine-like objects in sidescan sonar imagery. IEEE J. Ocean. Eng. 41, 123–138 (2016)Google Scholar
  11. 11.
    Riccardo, B., Francesco, B., Alessandro, D.G., Danu, P., Carlotta, S.: Electroencephalogram and physiological signal analysis for assessing flow in games. IEEE Trans. Comput. Intell. AI in Games 5, 164–175 (2013)Google Scholar
  12. 12.
    Konstantin, P.G., Petr, K.G., Mikhail, I.S.: Stable sequential Kuhn-Tucker theorem in one-dimensional inverse problems of dielectric reflectometry. Proc. 8th IEEE Int. Conf. Commun. Syst. Netw. (COMSNETS) 2, 1–4 (2015)Google Scholar
  13. 13.
    Novi, Q., Zoubin, G.: very simple safe-bayesian random forest, A, IEEE Trans. Pattern Anal. Mach. Intell, 37, 6,1297–1303 (2015)Google Scholar
  14. 14.
    Scornet E.: Random forests and kernel methods. IEEE Trans Inf. Theory 62(3), 1485–1500 (2016)MathSciNetMATHGoogle Scholar
  15. 15.
    Ocak, H.: Automatic detection of epileptic seizures in EEG using discrete wavelet transform and approximate entropy. Expert Syst. Appl. 36(2), 2027–2036 (2009)Google Scholar
  16. 16.
    Sathiyanarayanan M.: MYO Armband for physiotherapy healthcare: A case study using gesture recognition application, Proceedings of the 16th IEEE International Conference on Transparent Optical Networks (ICTON), 1–6 (2016)Google Scholar
  17. 17.
    Luh, G.C., Lin, H.A., Ma, Y.S., Yen, C.J.: Intuitive muscle-gesture based robot navigation control using wearable gesture armband, Proceedings of the IEEE International Conference on Machine Learning and Cybernetics (ICMLC), 389–395 (2015)Google Scholar
  18. 18.
    Gundlapalli, V.S.K., Shaik, Y.F., Muhammad, Z.U.R., Shaik, R.A., Aimé, L. E.: Efficient signal conditioning techniques for brain activity in remote health monitoring network. IEEE Sens. J. 13 (9), 3276–3283 (2013)Google Scholar
  19. 19.
    Zhendong, L., Shun, H., Feng, D: The development of a smart house system based on Brain-Computer Interface, Proceedings of the IEEE International Conference on Robotics and Biomimetics (ROBIO), 1012–1017 (2015)Google Scholar
  20. 20.
    Shouyi, W., Jacek, G., Chaovalitwongse, W.A.: Using wireless EEG signals to assess memory workload in the n-back task. IEEE Trans. Human-Mach. Syst. 46(3), 424–434 (2016)Google Scholar
  21. 21.
    Raul, C.C., Edmundo, B.H., Federico, R.C., Roberto, M.C., Carlos, P.C.: Facial expression analysis with kinect for the diagnosis of paralysis using nottingham grading system. IEEE Latin Am. Trans. 14(12), 3418–3426 (2016)Google Scholar
  22. 22.
    Enrico, B., Pietro, P., Stefano, B., Alberto, D.B.: Sequences, reconstructing high-resolution face models from kinect depth. IEEE Trans. Inf. Forensic. Secur. 11(12), 2843–2853 (2016)Google Scholar
  23. 23.
    Seokmin, H., Adrian, D., Genaro, S., Juan, C.B., Manuel, M.C.: Three-dimensional integral-imaging display from calibrated and depth-hole filtered kinect information. J. Disp. Technol. 12 (11), 1301–1308 (2016)Google Scholar
  24. 24.
    Zhao, Z., Liu, Y., Zhang, Z.: Camera calibration with three noncollinear points under special motions. IEEE Trans. Image Process. 17(12), 2393–2402 (2008)MathSciNetGoogle Scholar
  25. 25.
    Rahman, T., Krouglicof, N.: An efficient camera calibration technique offering robustness and accuracy over a wide range of lens distortion. IEEE Trans. Image Process. 21(2), 626–637 (2012)MathSciNetGoogle Scholar
  26. 26.
    Tzung-Sz, S., Chia-Hsiang, M.: Automatic camera calibration for a multiple-sensor integrated coordinate measurement system. IEEE Trans. Robot. Autom. 17(4), 502–507 (2001)Google Scholar
  27. 27.
    Robert, J., Angelika, P., Martin, B.: Feature extraction and selection for emotion recognition from EEG. IEEE Trans. Affect. Comput. 5(3), 327–339 (2014)Google Scholar
  28. 28.
    Matsubara, T., Morimoto, J.: Bilinear modeling of EMG signals to extract user-independent features for multiuser myoelectric interface. IEEE Trans. Biomed. Eng. 60(8), 2205–2213 (2013)Google Scholar
  29. 29.
    Mahaphonchaikul, K., Sueaseenak, D., Pintavirooj, C., Sangworasil, M., Tungjitkusolmun, S.: EMG signal feature extraction based on wavelet transform, Proceedings of the 20th IEEE International Conference on Electrical Engineering/Electronics Computer Telecommunications and Information Technology (ECTI-CON), 327–331 (2010)Google Scholar
  30. 30.
    Xie B., Minn H.: Real-time sleep apnea detection by classifier combination. IEEE Trans. Inf. Technol. Biomed. 16(3), 469–477 (2012)Google Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2017

Authors and Affiliations

  • Gurkan Kucukyildiz
    • 1
  • Hasan Ocak
    • 1
  • Suat Karakaya
    • 1
  • Omer Sayli
    • 1
  1. 1.Department of Mechatronics Engineering, Umuttepe CampusKocaeli UniversityKocaeliTurkey

Personalised recommendations