Inferring Touch from Motion in Real World Data

  • Pascal BissigEmail author
  • Philipp Brandes
  • Jonas Passerini
  • Roger Wattenhofer
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9482)


Most modern smartphones are equipped with motion sensors to measure the movement and orientation of the device. On Android and iOS, accessing the motion sensors does not require any special permissions. On the other hand, touch input is only available to the application currently in the foreground because it may reveal sensitive information such as passwords. In this paper, we present a side channel attack on touch input by analyzing motion sensor readings. Our data set contains more than a million gestures from 1’493 users with 615 distinct device models. To infer touch from motion inputs, we use a classifier based on the Dynamic Time Warping algorithm. The evaluation shows that our method performs significantly better than random guessing in real world usage scenarios.


Motion sensing Side-channel attack Touch input 


  1. 1.
    Müller, M.: Dynamic time warping. In: Information Retrieval for Music and Motion, pp. 69–84. Springer, Heidelberg (2007)Google Scholar
  2. 2.
    Aviv, A.J., Sapp, B., Blaze, M., Smith, J.M.: Practicality of accelerometer side channels on smartphones. In: ACSAC, pp. 41–50. ACM (2012)Google Scholar
  3. 3.
    Cai, L., Chen, H.: Touchlogger: inferring keystrokes on touch screen from smartphone motion. In: HotSec, p. 9 (2011)Google Scholar
  4. 4.
    Cai, L., Chen, H.: On the practicality of motion based keystroke inference attack. In: Katzenbeisser, S., Weippl, E., Camp, L.J., Volkamer, M., Reiter, M., Zhang, X. (eds.) Trust 2012. LNCS, vol. 7344, pp. 273–290. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  5. 5.
    Chong, M.K., Marsden, G., Gellersen, H.: Gesturepin: using discrete gestures for associating mobile devices. In: Mobile HCI, pp. 261–264. ACM (2010)Google Scholar
  6. 6.
    Hinckley, K., Song, H.: Sensor synaesthesia: touch in motion, and motion in touch. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 801–810. ACM (2011)Google Scholar
  7. 7.
    Kolly, S.M., Wattenhofer, R., Welten, S.: A personal touch: recognizing users based on touch screen behavior. In: Proceedings of the Third International Workshop on Sensing Applications on Mobile Phones, p. 1. ACM (2012)Google Scholar
  8. 8.
    Liu, J., Wang, Z., Zhong, L., Wickramasuriya, J., Vasudevan, V.: uwave: accelerometer-based personalized gesture recognition and its applications. In: PerCom, pp. 1–9. IEEE Computer Society (2009)Google Scholar
  9. 9.
    Marquardt, P., Verma, A., Carter, H., Traynor, P.: (sp)iphone: decoding vibrations from nearby keyboards using mobile phone accelerometers. In: ACM CCS, pp. 551–562. ACM (2011)Google Scholar
  10. 10.
    Michalevsky, Y., Boneh, D., Nakibly, G.: Gyrophone: recognizing speech from gyroscope signals. In: 23rd USENIX Security Symposium, pp. 1053–1067. USENIX Association, San Diego, August 2014Google Scholar
  11. 11.
    Miluzzo, E., Varshavsky, A., Balakrishnan, S., Choudhury, R.R.: Tapprints: your finger taps have fingerprints. In: MobiSys, pp. 323–336. ACM (2012)Google Scholar
  12. 12.
    Niu, Y., Chen, H.: Gesture authentication with touch input for mobile devices. In: Prasad, R., Farkas, K., Schmidt, A.U., Lioy, A., Russello, G., Luccio, F.L. (eds.) MobiSec 2011. LNICST, vol. 94, pp. 13–24. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  13. 13.
    Owusu, E., Han, J., Das, S., Perrig, A., Zhang, J.: Accessory: password inference using accelerometers on smartphones. In: Proceedings of the Twelfth Workshop on Mobile Computing Systems and Applications, HotMobile 2012, pp. 9: 1–9: 6. ACM, New York (2012)Google Scholar
  14. 14.
    Wu, J., Pan, G., Zhang, D., Qi, G., Li, S.: Gesture recognition with a 3-D accelerometer. In: Zhang, D., Portmann, M., Tan, A.-H., Indulska, J. (eds.) UIC 2009. LNCS, vol. 5585, pp. 25–38. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  15. 15.
    Zhi, X., Bai, K., Zhu, S.: Taplogger: inferring user inputs on smartphone touchscreens using on-board motion sensors. In: WISEC, pp. 113–124. ACM (2012)Google Scholar
  16. 16.
    Zheng, N., Bai, K., Huang, H., Wang, H.: You are how you touch: user verification on smartphones via tapping behaviors. In: IEEE 22nd International Conference on Network Protocols (ICNP), pp. 221–232. IEEE (2014)Google Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Pascal Bissig
    • 1
    Email author
  • Philipp Brandes
    • 1
  • Jonas Passerini
    • 1
  • Roger Wattenhofer
    • 1
  1. 1.ETH ZurichZürichSwitzerland

Personalised recommendations