Evaluation of Matching Degree Between Touch Gestures and User Mental Model Based on Event-Related Potential

  • Ningyue PengEmail author
  • Jing Ma
  • Chengqi Xue
Conference paper
Part of the Advances in Intelligent Systems and Computing book series (AISC, volume 876)


Designing gestures in multi-touch devices that are congruent with User Mental Model (UMM) is vital to remove the cognitive gulf, and reduce learning cost for users. In this study, a neurophysiological evaluation method-Event-Related Potential was used to evaluate matching degree between touch gestures and UMM. Prime-probe paradigm was employed and two experimental conditions were compared-congruence and incongruence. Results show that negative component was elicited within the time window of 300 ms to 500 ms after the onset of probe stimuli in frontal, central and parietal lobe. With time went by, more negative components were found in the incongruence condition. Hereby, the former negative component is interpreted as a representation of the decoding process, and the later one interpreted as semantic matching with UMM. Results in this study may provide guidance for those assessing the usability of multi-touch gestures.


Matching degree Touch gesture User mental model Event-related potential Evaluation 



Author 1 gratefully acknowledge the grant from the Fundamental Research Funds for the Central Universities & Postgraduate Research & Practice Innovation Program of Jiangsu Province (No. KYCX17_0056). As well, all the authors want to acknowledge the grant from Natural Science Foundation of China (No. 71271053).


  1. 1.
    Kim, Y.H., Lee, J.H.: Game interface enhancement under smartphone platform focused on touchscreen interaction, pp. 45–61. Pergamon Press, Inc. (2015)Google Scholar
  2. 2.
    Kukimoto, N., Onoue, Y., Aoki, K., Fujita, K., Koyamada, K.: HyperInfo: interactive large display for informal visual communication. In: International Conference on Network-Based Information Systems, pp. 399–404 (2014)Google Scholar
  3. 3.
    Valdes, C., et al.: Exploring the design space of gestural interaction with active tokens through user-defined gestures. In: SIGCHI Conference on Human Factors in Computing Systems, pp. 4107–4116 (2014)Google Scholar
  4. 4.
    Craik, K.J.W.: The nature of explanation. Stud. Q. J. 14(55), 44 (1967)Google Scholar
  5. 5.
    Doyle, J.K., Ford, D.N.: Mental models concepts revisited: some clarifications and a reply to Lane. Syst. Dyn. Rev. 15(4), 411–415 (1999)CrossRefGoogle Scholar
  6. 6.
    Senge, P.M.: Mental models. Plan. Rev. 20, 4–44 (1992)CrossRefGoogle Scholar
  7. 7.
    Johnsonlaird, P.N.: Mental models: towards a cognitive science of language. Inference Conscious. 47(1/2), 438–445 (1983)Google Scholar
  8. 8.
    Forrester, J.W.: Industrial dynamics. J. Oper. Res. Soc. 48(10), 1037–1041 (1997)CrossRefGoogle Scholar
  9. 9.
    Luck, S.J., Kappenman, E.S.: ERP components: the ups and downs of brainwave recordings. In: The Oxford Handbook of Event-Related Potential Components, pp. 3–30. Oxford University Press (2012)Google Scholar
  10. 10.
    Berti, S., Schröger, E.: A comparison of auditory and visual distraction effects: behavioral and event-related indices. Brain Res. Cogn. Brain Res. 10(3), 265 (2001)CrossRefGoogle Scholar
  11. 11.
    Shibata, H., Gyoba, J., Suzuki, Y.: Event-related potentials during the evaluation of the appropriateness of cooperative actions. Neurosci. Lett. 452(2), 189–193 (2009)CrossRefGoogle Scholar
  12. 12.
    Mu, Y., Kitayama, S., Han, S., Gelfand, M.J.: How culture gets embrained: cultural differences in event-related potentials of social norm violations. Proc. Natl. Acad. Sci. U. S. A. 112(50), 15348–15353 (2015)CrossRefGoogle Scholar
  13. 13.
    Reid, V.M., Striano, T.: N400 involvement in the processing of action sequences. Neurosci. Lett. 433(2), 93–97 (2008)CrossRefGoogle Scholar
  14. 14.
    Amoruso, L., et al.: N400 ERPs for actions: building meaning in context. Front. Hum. Neurosci. 7(57), 57 (2013)Google Scholar
  15. 15.
    Kutas, M., Federmeier, K.D.: Thirty years and counting: finding meaning in the N400 component of the event related brain potential (ERP). Annu. Rev. Psychol. 62(1), 621–654 (2011)CrossRefGoogle Scholar
  16. 16.
    Sifaqui, C.: Structuring user interfaces with a meta-model of mental models. Comput. Graph. 23(3), 323–330 (2013)CrossRefGoogle Scholar
  17. 17.
    Szucs, D., Soltész, F., Czigler, I., Csépe, V.: Electroencephalography effects to semantic and non-semantic mismatch in properties of visually presented single-characters: the N2b and the N400. Neurosci. Lett. 412(1), 18–23 (2007)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.School of Mechanical EngineeringSoutheast UniversityNanjingChina

Personalised recommendations