Skip to main content
SpringerLink
Log in

TouchSpeaker, a Multi-sensor Context-Aware Application for Mobile Devices: from Application to Implementation

  • Published:
Journal of Signal Processing Systems Aims and scope Submit manuscript

Abstract

Tapping with your finger on any place on your mobile device is a promising candidate for enhanced interaction between users and their mobile device. So far the touchscreen and the accelerometer are commonly used to infer finger tap events. However, the touchscreen consumes a significant amount of power and is not always accessible (i.e., when the device is used as a running assistant). The accelerometer can be power efficient but can’t differentiate well between a variety of contexts and positions. To address these limitations, we present TouchSpeaker, a novel technique for finger tap detection on mobile devices using the built-in speakers as primary sensors. We show that a combination of the speakers with other built-in sensors can distinguish between 9 different tap events with an accuracy of 98.3%, outperforming the state of the art. In addition, a robust version is implemented resulting in a false positive rate below 1%. For power constrained devices, we propose a configuration consisting of only the speakers and the accelerometer, achieving an accuracy of 95.2%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10
Figure 11
Figure 12
Figure 13

Similar content being viewed by others

Notes

  1. The power consumption does not include signal processing and is measured by the app Sensor Kinetics [9].

References

  1. Beysens, J., Chiumento, A., Pollin, S., & Li, M. (2016). Touchspeaker, a multi-sensor context-aware application for mobile devices. In 2016 IEEE international workshop on signal processing systems (SiPS) (pp. 23–26).

  2. Daniel, P. KnockOn app brings double-tap to lock to most Androids and wake to ones with OLED displays. http://www.phonearena.com/news/KnockOn-app-brings-double-tap-to-lock-to-most-Androids.-and-wake-to-ones-with-OLED-displays_id75033.

  3. Harrison, C., Schwarz, J., & Hudson, S.E. (2011). TapSense: enhancing finger interaction on touch surfaces. In Proceedings of the 24th annual ACM symposium on User interface software and technology - UIST ’11 (pp. 627–636). https://doi.org/10.1145/2047196.2047279.

  4. HTC Canada: HTC One (M7) Specs and Reviews. http://www.htc.com/ca/smartphones/htc-one-m7/.

  5. Lu, H., & Li, Y. (2015). Gesture on: enabling always-on touch gestures for fast mobile access from the device standby mode. In Proceedings of the 33rd annual ACM conference on human factors in computing systems - CHI ’15 (Vol. 1, pp. 3355–3364). https://doi.org/10.1145/2702123.2702610.

  6. MathWorks Benelux: MATLAB Profile execution time for functions. http://nl.mathworks.com/help/matlab/ref/profile.html.

  7. McGrath, W., & Li, Y. (2014). Detecting tapping motion on the side of mobile devices by probabilistically combining hand postures. In Proceedings of the 27th annual ACM symposium on User interface software and technology - UIST ’14 (pp. 215–219). https://doi.org/10.1145/2642918.2647363.

  8. NDT Resource center: Attenuation of Sound Waves. https://www.nde-ed.org/EducationResources/CommunityCollege/Ultrasonics/Physics/attenuation.htm.

  9. Sensor Kinetics. https://play.google.com/store/apps/details?id=com.innoventions.sensorkinetics&hl=nl.

  10. The Engineering Toolbox: Speed of Sound in some common Solids. http://www.engineeringtoolbox.com/sound-speed-solids-d_713.html.

  11. University of Waikato: Weka 3 - Data Mining with Open Source Machine Learning Software in Java. http://www.cs.waikato.ac.nz/ml/weka/.

  12. Yacoub, M., & Yang, G.Z. (2007). Body Sensor Networks. Berlin: Springer Science & Business Media.

    Google Scholar 

  13. Zhang, C., Guo, A., Zhang, D., Southern, C., Arriaga, R., & Abowd, G. (2015). BeyondTouch: extending the input language with built-in sensors on commodity smartphones. In Proceedings of the 20th international conference on intelligent user interfaces - IUI ’15 (pp. 67–77). https://doi.org/10.1145/2678025.2701374.

Download references

Author information

Authors and Affiliations

  1. Department of Electrical Engineering (ESAT), KU Leuven, Kasteelpark Arenberg 10 Bus 2441, 3001, Leuven, Belgium

    Jona Beysens, Alessandro Chiumento & Sofie Pollin

  2. NXP Semiconductors Belgium N.V., Interleuvenlaan 80, 3001, Leuven, Belgium

    Min Li

Authors
  1. Jona Beysens
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alessandro Chiumento
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Min Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sofie Pollin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jona Beysens.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Beysens, J., Chiumento, A., Li, M. et al. TouchSpeaker, a Multi-sensor Context-Aware Application for Mobile Devices: from Application to Implementation. J Sign Process Syst 90, 1469–1478 (2018). https://doi.org/10.1007/s11265-017-1302-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11265-017-1302-2

Keywords

Search

Navigation