Software switches: novel hands-free interaction techniques for quadriplegics based on respiration–machine interaction

  • Cagdas EsiyokEmail author
  • Ayhan Askin
  • Aliye Tosun
  • Sahin Albayrak
Long Paper


The majority of interaction techniques for individuals with disabilities to control a computer require an additional dedicated hardware like switches beyond standard computer peripherals. Furthermore, each traditional hardware switch emulates different keyboard characters or mouse clicks depending on its manufacturer. There is no commonly agreed standard on this; while some switch accessible programs expect to receive enter character, other programs expect to receive mouse left click. Switch manufacturers, therefore, distribute software drivers, which allow users to assign expected characters, with switches to overcome this standardization problem by making them adaptable. As software drivers are developed for a specific switch, they are not compatible with the other manufacturers’ switches. On the other hand, for the ones with a very limited motor activity but a complete respiration activity, sip-and-puff switches are efficient solutions. However, they are expensive and invasive systems with tubes, that have to be changed regularly due to the hygiene concern, inside users’ mouth. Although invasive respiration-based systems (sip-and-puff devices) caught enough attention by researchers over the years, using non-invasive respiration-based interaction techniques is an underestimated approach in general. In this study, we proposed two novel non-invasive interaction techniques as software switches (PuffCam and PuffMic) compatible with any switch accessible software. Both techniques are respiration operated, where a hard puff, detected by a microphone or an adapted camera, is considered as ‘switch-on’ like a puff switch. We have also collected the objective data by an evaluation software namely TestBed. A user study (conducted with 46 participants with/out disabilities) revealed that the accuracy, precision, recall, and false positive rate of our interaction techniques were quite impressive, and PuffCam performed better than PuffMic for all metrics. According to questionnaire findings, comfort assessment of interaction techniques by participants was rated quite satisfactory. All participants agreed that the idea of controlling a computer via breathing without purchasing any dedicated hardware sounded very promising. Because most interaction techniques for computer control require extra dedicated devices and there is not any adaptable (i.e., compatible with most switch accessible software) software switch to the best of our knowledge, proposed interaction techniques can help community in an open access manner without purchasing any device.


Interaction techniques Switch access Computer access Sip-and-puff device Software switch Switch accessible software Respiration–machine interface 



We would like to thank Dr. Brijnesh-Johannes Jain and Dr. Fikret Sivrikaya for their valuable suggestions. The first author holds the Ministry of National Education Scholarship of the Turkish Republic.


  1. 1.
    Anson, D., Lawler, G., Kissinger, A., Timko, M., Tuminski, J., Drew, B.: Efficacy of three head-pointing devices for a mouse emulation task. Assist. Technol. 14(2), 140–150 (2002)CrossRefGoogle Scholar
  2. 2.
    Armour, B.S., Courtney-Long, E.A., Fox, M.H., Fredine, H., Cahill, A.: Prevalence and causes of paralysis—united states, 2013. Am. J. Public Health 106(10), 1855–1857 (2016)CrossRefGoogle Scholar
  3. 3.
    Betke, M., Gips, J., Fleming, P.: The camera mouse: visual tracking of body features to provide computer access for people with severe disabilities. IEEE Trans. Neural Syst. Rehabilit. Eng. 10(1), 1–10 (2002)CrossRefGoogle Scholar
  4. 4.
    Borghetti, D., Bruni, A., Fabbrini, M., Murri, L., Sartucci, F.: A low-cost interface for control of computer functions by means of eye movements. Comput. Biol. Med. 37(12), 1765–1770 (2007)CrossRefGoogle Scholar
  5. 5.
    Cecotti, H.: Spelling with non-invasive brain–computer interfaces-current and future trends. J. Physiol. Paris 105(1–3), 106–114 (2011)CrossRefGoogle Scholar
  6. 6.
    Chen, H.C., Liu, Y.P., Chen, C.L., Chen, C.Y.: Design and feasibility study of an integrated pointing device apparatus for individuals with spinal cord injury. Appl. Ergon. 38(3), 275–283 (2007)CrossRefGoogle Scholar
  7. 7.
    Cripps, R., Lee, B., Wing, P., Weerts, E., Mackay, J., Brown, D.: A global map for traumatic spinal cord injury epidemiology: towards a living data repository for injury prevention. Spinal Cord 49(4), 493 (2011)CrossRefGoogle Scholar
  8. 8.
    Dai, L., Goldman, R., Sears, A., Lozier, J.: Speech-based cursor control: a study of grid-based solutions. In: ACM SIGACCESS Accessibility and Computing, 77–78, pp. 94–101. ACM (2004)Google Scholar
  9. 9.
    Gerdtman, C., Lindén, M.: Six-button click interface for a disabled user by an adjustable multi-level sip-and-puff switch. In: Proceedings of SIGRAD 2010: Content Aggregation and Visualization; November 25–26; 2010; Västerås; Sweden, 052, pp. 59–63. Linkoping University Electronic Press (2010)Google Scholar
  10. 10.
    Grauman, K., Betke, M., Lombardi, J., Gips, J., Bradski, G.R.: Communication via eye blinks and eyebrow raises: video-based human–computer interfaces. Univers. Access Inf. Soc. 2(4), 359–373 (2003)CrossRefGoogle Scholar
  11. 11.
    Guan, C., Thulasidas, M., Wu, J.: High performance p300 speller for brain–computer interface. In: 2004 IEEE International Workshop on Biomedical Circuits and Systems, pp. S3–S5. IEEE (2004)Google Scholar
  12. 12.
    Harada, S., Landay, J.A., Malkin, J., Li, X., Bilmes, J.A.: The vocal joystick: evaluation of voice-based cursor control techniques. In: Proceedings of the 8th International ACM SIGACCESS Conference on Computers and Accessibility, pp. 197–204. ACM (2006)Google Scholar
  13. 13.
    Huang, C.N., Chen, C.H., Chung, H.Y.: Application of facial electromyography in computer mouse access for people with disabilities. Disabil. Rehabilit. 28(4), 231–237 (2006)CrossRefGoogle Scholar
  14. 14.
    Karimullah, A.S., Sears, A.: Speech-based cursor control. In: Proceedings of the Fifth International ACM Conference on Assistive Technologies, pp. 178–185. ACM (2002)Google Scholar
  15. 15.
    Kim, E.Y., Kang, S.K., Jung, K., Kim, H.J.: Eye mouse: mouse implementation using eye tracking. In: International Conference on Consumer Electronics, pp. 207–208. IEEE (2005)Google Scholar
  16. 16.
    Kim, H., Ryu, D.: Computer control by tracking head movements for the disabled. In: Miesenberger, K., Klaus, J., Zagler, W.L., Karshmer, A.I. (eds) International Conference on Computers for Handicapped Persons, pp. 709–715. Springer (2006).Google Scholar
  17. 17.
    Kim, J., Park, H., Bruce, J., Sutton, E., Rowles, D., Pucci, D., Holbrook, J., Minocha, J., Nardone, B., West, D.: The tongue enables computer and wheelchair control for people with spinal cord injury. Sci. Transl. Med. 5(213), 213ra166 (2013)CrossRefGoogle Scholar
  18. 18.
    Lv, Z., Wu, X., Li, M., Zhang, C.: Implementation of the eog-based human computer interface system. In: 2nd International Conference on Bioinformatics and Biomedical Engineering, pp. 2188–2191. IEEE (2008)Google Scholar
  19. 19.
    Mace, M., Abdullah-al Mamun, K., Naeem, A.A., Gupta, L., Wang, S., Vaidyanathan, R.: A heterogeneous framework for real-time decoding of bioacoustic signals: applications to assistive interfaces and prosthesis control. Expert Syst. Appl. 40(13), 5049–5060 (2013)CrossRefGoogle Scholar
  20. 20.
    Manaris, B., McCauley, R., MacGyvers, V.: An intelligent interface for keyboard and mouse control. In: Proc. 14th Intl Florida AI Research Symposium (FLAIRS-01), pp. 182–188. Citeseer (2001)Google Scholar
  21. 21.
    Rosas-Cholula, G., Ramirez-Cortes, J.M., Alarcon-Aquino, V., Gomez-Gil, P., Rangel-Magdaleno, JdJ, Reyes-Garcia, C.: Gyroscope-driven mouse pointer with an emotiv eeg headset and data analysis based on empirical mode decomposition. Sensors 13(8), 10561–10583 (2013)CrossRefGoogle Scholar
  22. 22.
    Simpson, T., Broughton, C., Gauthier, M.J., Prochazka, A.: Tooth-click control of a hands-free computer interface. IEEE Trans. Biomed. Eng. 55(8), 2050–2056 (2008)CrossRefGoogle Scholar
  23. 23.
    Smith, J.D., Graham, T.: Use of eye movements for video game control. In: Proceedings of the 2006 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology, p. 20. ACM (2006)Google Scholar
  24. 24.
    Sporka, A., Kurniawan, S., Slavík, P.: Non-speech operated emulation of keyboard. In: Clarkson, J., Langdon, P., Robinson, P. (eds) Designing Accessible Technology, pp. 145–154. Springer (2006)Google Scholar
  25. 25.
    Sporka, A.J., Kurniawan, S.H., Slavík, P.: Acoustic control of mouse pointer. Univers. Access Inf. Soc. 4(3), 237–245 (2006)CrossRefGoogle Scholar
  26. 26.
    Struijk, L.N.A.: An inductive tongue computer interface for control of computers and assistive devices. IEEE Trans. Biomed. Eng. 53(12), 2594–2597 (2006)CrossRefGoogle Scholar
  27. 27.
    Sturtz, C.R.: Mouth-operated computer input device and associated methods (2010). US Patent 7,768,499Google Scholar
  28. 28.
    Sullivan, A., Bailey, J., Stokic, D.: Predictors of oral health after spinal cord injury. Spinal Cord 51(4), 300 (2013)CrossRefGoogle Scholar
  29. 29.
    Vaidyanathan, R., Chung, B., Gupta, L., Kook, H., Kota, S., West, J.D.: Tongue-movement communication and control concept for hands-free human–machine interfaces. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum. 37(4), 533–546 (2007)CrossRefGoogle Scholar
  30. 30.
    Varona, J., Manresa-Yee, C., Perales, F.J.: Hands-free vision-based interface for computer accessibility. J. Netw. Comput. Appl. 31(4), 357–374 (2008)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature 2019

Authors and Affiliations

  • Cagdas Esiyok
    • 1
    Email author
  • Ayhan Askin
    • 2
  • Aliye Tosun
    • 2
  • Sahin Albayrak
    • 1
  1. 1.Distributed Artificial Intelligence LaboratoryTechnische Universität BerlinBerlinGermany
  2. 2.Physical Medicine and Rehabilitation DepartmentIzmir Katip Celebi UniversityIzmirTurkey

Personalised recommendations