TouchSpeaker, a Multi-sensor Context-Aware Application for Mobile Devices: from Application to Implementation

  • Jona Beysens
  • Alessandro Chiumento
  • Min Li
  • Sofie Pollin


Tapping with your finger on any place on your mobile device is a promising candidate for enhanced interaction between users and their mobile device. So far the touchscreen and the accelerometer are commonly used to infer finger tap events. However, the touchscreen consumes a significant amount of power and is not always accessible (i.e., when the device is used as a running assistant). The accelerometer can be power efficient but can’t differentiate well between a variety of contexts and positions. To address these limitations, we present TouchSpeaker, a novel technique for finger tap detection on mobile devices using the built-in speakers as primary sensors. We show that a combination of the speakers with other built-in sensors can distinguish between 9 different tap events with an accuracy of 98.3%, outperforming the state of the art. In addition, a robust version is implemented resulting in a false positive rate below 1%. For power constrained devices, we propose a configuration consisting of only the speakers and the accelerometer, achieving an accuracy of 95.2%.


Mobile interaction Finger tap detection Speaker as microphone Feature extraction Classification methods 


  1. 1.
    Beysens, J., Chiumento, A., Pollin, S., & Li, M. (2016). Touchspeaker, a multi-sensor context-aware application for mobile devices. In 2016 IEEE international workshop on signal processing systems (SiPS) (pp. 23–26).Google Scholar
  2. 2.
    Daniel, P. KnockOn app brings double-tap to lock to most Androids and wake to ones with OLED displays.
  3. 3.
    Harrison, C., Schwarz, J., & Hudson, S.E. (2011). TapSense: enhancing finger interaction on touch surfaces. In Proceedings of the 24th annual ACM symposium on User interface software and technology - UIST ’11 (pp. 627–636).
  4. 4.
    HTC Canada: HTC One (M7) Specs and Reviews.
  5. 5.
    Lu, H., & Li, Y. (2015). Gesture on: enabling always-on touch gestures for fast mobile access from the device standby mode. In Proceedings of the 33rd annual ACM conference on human factors in computing systems - CHI ’15 (Vol. 1, pp. 3355–3364).
  6. 6.
    MathWorks Benelux: MATLAB Profile execution time for functions.
  7. 7.
    McGrath, W., & Li, Y. (2014). Detecting tapping motion on the side of mobile devices by probabilistically combining hand postures. In Proceedings of the 27th annual ACM symposium on User interface software and technology - UIST ’14 (pp. 215–219).
  8. 8.
  9. 9.
  10. 10.
    The Engineering Toolbox: Speed of Sound in some common Solids.
  11. 11.
    University of Waikato: Weka 3 - Data Mining with Open Source Machine Learning Software in Java.
  12. 12.
    Yacoub, M., & Yang, G.Z. (2007). Body Sensor Networks. Berlin: Springer Science & Business Media.Google Scholar
  13. 13.
    Zhang, C., Guo, A., Zhang, D., Southern, C., Arriaga, R., & Abowd, G. (2015). BeyondTouch: extending the input language with built-in sensors on commodity smartphones. In Proceedings of the 20th international conference on intelligent user interfaces - IUI ’15 (pp. 67–77).

Copyright information

© Springer Science+Business Media, LLC 2017

Authors and Affiliations

  • Jona Beysens
    • 1
  • Alessandro Chiumento
    • 1
  • Min Li
    • 2
  • Sofie Pollin
    • 1
  1. 1.Department of Electrical Engineering (ESAT)KU LeuvenLeuvenBelgium
  2. 2.NXP Semiconductors Belgium N.V.LeuvenBelgium

Personalised recommendations