Eye Tracking Experiments on Process Model Comprehension: Lessons Learned

  • Michael ZimochEmail author
  • Rüdiger Pryss
  • Johannes Schobel
  • Manfred Reichert
Conference paper
Part of the Lecture Notes in Business Information Processing book series (LNBIP, volume 287)


For documenting business processes, there exists a plethora of process modeling languages. In this context, graphical process models are used to enhance the process comprehensibility of the stakeholders involved. The large number of available modeling languages, however, aggravates process model comprehension and increases the knowledge gap between domain and modeling experts. Upon this, one major challenge is to identify factors fostering the comprehension of process models. This paper discusses the experiences we gathered with the use of eye tracking in experiments on process model comprehension and the lessons learned in this context. The objective of the experiments was to study the comprehension of process models expressed in terms of four different modeling languages (i.e., BPMN, eGantt, EPC, and Petri Net). This paper further provides recommendations along nine identified categories that can foster related experiments on process model comprehension.


Process model comprehension Eye tracking Experiment 


  1. 1.
    Petrusel, R., Mendling, J., Reijers, H.A.: Task-specific visual cues for improving process model understanding. Inf. Softw. Technol. 79, 63–78 (2016)CrossRefGoogle Scholar
  2. 2.
    Martini, M., Pinggera, J., Neuratuer, M., Sachse, P., Furtner, M., Weber, B.: The impact of working memory and the process of process modelling on model quality: investigating experienced versus inexperienced modellers. In: Scientific Reports, vol. 6 (2016)Google Scholar
  3. 3.
    Moody, D.L.: Theoretical and practical issues in evaluating the quality of conceptual models: current state and future directions. Data Knowl. Eng. 55, 243–276 (2005)CrossRefGoogle Scholar
  4. 4.
    Mendling, J., Reijers, H.A., Cardoso, J.: What makes process models understandable? In: Alonso, G., Dadam, P., Rosemann, M. (eds.) BPM 2007. LNCS, vol. 4714, pp. 48–63. Springer, Heidelberg (2007). doi: 10.1007/978-3-540-75183-0_4 CrossRefGoogle Scholar
  5. 5.
    Turetken, O., Rompen, T., Vanderfeesten, I., Dikici, A., Moll, J.: The effect of modularity representation and presentation medium on the understandability of business process models in BPMN. In: La Rosa, M., Loos, P., Pastor, O. (eds.) BPM 2016. LNCS, vol. 9850, pp. 289–307. Springer, Cham (2016). doi: 10.1007/978-3-319-45348-4_17 CrossRefGoogle Scholar
  6. 6.
    Recker, J., Reijers, H.A., van de Wouw, S.G.: Process model comprehension: the effects of cognitive abilities, learning style and strategy. Commun. Assoc. Inf. Syst. 34, 199–222 (2014)Google Scholar
  7. 7.
    Recker, J., Dreiling, A.: Does it matter which process modelling language we teach or use? An experimental study on understanding process modelling languages without formal education. In: Proceedings of ACIS 2007, pp. 356–366 (2007)Google Scholar
  8. 8.
    Petrusel, R., Mendling, J., Reijers, H.A.: How visual cognition influences process model comprehension. Decis. Support Syst. 96, 1–16 (2017)Google Scholar
  9. 9.
    Figl, K., Recker, J.: Exploring cognitive style and task-specific preferences for process representations. Requirements Eng. 21, 63–85 (2016)CrossRefGoogle Scholar
  10. 10.
    Döhring, M., Reijers, H.A., Smirnov, S.: Configuration vs. adaptation for business process variant maintenance: an empirical study. Inf. Syst. 39, 108–133 (2014)Google Scholar
  11. 11.
    Bandara, W., Gable, G.G., Rosemann, M.: Factors and measures of business process modelling: model building through a multiple case study. Eur. J. Inf. Syst. 14, 347–360 (2005)CrossRefGoogle Scholar
  12. 12.
    Ottensooser, A., Fekete, A., Reijers, H.A., Mendling, J., Menictas, C.: Making sense of business process descriptions: an experimental comparison of graphical and textual notations. J. Syst. Softw. 85, 596–606 (2012)CrossRefGoogle Scholar
  13. 13.
    Weber, B., Pinggera, J., Neurauter, M., Zugal, S., Martini, M., Furtner, M., Sachse, P., Schnitzer, D.: Fixation patterns during process model creation: initial steps toward neuro-adaptive process modeling environments. In: Proceedings of the 2016 49th Hawaii International Conference on System Sciences, pp. 600–609 (2016)Google Scholar
  14. 14.
    Mendling, J., Reijers, H.A., van der Aalst, W.M.P.: Seven process modeling guidelines (7PMG). Inf. Softw. Technol. 52, 127–136 (2010)CrossRefGoogle Scholar
  15. 15.
    Rodrigues, R.D.A., Barros, M.D.O., Revoredo, K., Azevedo, L.G., Leopold, H.: An experiment on process model understandability using textual work instructions and BPMN models. In: 29th Brazilian Symposium on Software Engineering, pp. 41–50 (2015)Google Scholar
  16. 16.
    Petrusel, R., Mendling, J.: Eye-tracking the factors of process model comprehension tasks. In: Salinesi, C., Norrie, M.C., Pastor, Ó. (eds.) CAiSE 2013. LNCS, vol. 7908, pp. 224–239. Springer, Heidelberg (2013). doi: 10.1007/978-3-642-38709-8_15 CrossRefGoogle Scholar
  17. 17.
    Haisjackl, C., Barba, I., Zugal, S., Soffer, P., Hadar, I., Reichert, M., Pinggera, J., Weber, B.: Understanding declare models: strategies, pitfalls, empirical results. Softw. Syst. 15, 325–352 (2016)CrossRefGoogle Scholar
  18. 18.
    Recker, J.: Empirical investigation of the usefulness of gateway constructs in process models. Eur. J. Inf. Syst. 22, 673–689 (2013)Google Scholar
  19. 19.
    Sánachez-González, L., Garcia, F., Ruiz, F., Mendling, J.: Quality indicators for business process models from a gateway complexity perspective. Inf. Softw. Technol. 54, 1159–1175 (2012)CrossRefGoogle Scholar
  20. 20.
    Kock, N., Verville, J., Danesh-Pajou, A., Deluca, D.: Communication flow orientation in business process modeling and its effect on redesign success: results from a field study. Decis. Support Syst. 45, 562–575 (2009)CrossRefGoogle Scholar
  21. 21.
    Figl, K., Mendling, J., Strembeck, M.: The influence of notational deficiencies on process model comprehension. J. Assoc. Inf. Syst. 14, 312–338 (2013)Google Scholar
  22. 22.
    Recker, J., Dreiling, A.: The effects of content presentation format and user characteristics on novice developers’ understanding of process models. Commun. Assoc. Inf. Syst. 28, 65–84 (2011)Google Scholar
  23. 23.
    Figl, K., Strembeck, M.: Findings from an experiment on flow direction of business process models. In: International Workshop on EMISA 2015, pp. 59–73 (2015)Google Scholar
  24. 24.
    Figl, K., Laue, R.: Influence factors for local comprehensibility of process models. Int. J. Hum. Comput. Stud. 82, 96–110 (2015)CrossRefGoogle Scholar
  25. 25.
    List, B., Korherr, B.: An evaluation of conceptual business process modelling languages. In: Proceedings of the 2006 ACM Symposium on Applied Computing, pp. 1532–1539 (2006)Google Scholar
  26. 26.
    Pichler, P., Weber, B., Zugal, S., Pinggera, J., Mendling, J., Reijers, H.A.: Imperative versus declarative process modeling languages: an empirical investigation. In: Daniel, F., Barkaoui, K., Dustdar, S. (eds.) BPM 2011. LNBIP, vol. 99, pp. 383–394. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-28108-2_37 CrossRefGoogle Scholar
  27. 27.
    Dobesova, Z., Malcik, M.: Workflow diagrams and pupil dilatation in eye tracking testing. In: Proceedings of 13th International Conference on Emerging eLearning Techniques Applications, pp. 59–64 (2015)Google Scholar
  28. 28.
    Hogrebe, F., Gehrke, N., Nüttgens, M.: Eye tracking experiments in business process modeling: agenda setting and proof of concept. In: Proceedings of 4th International Workshop on Enterprise Modelling and Information Systems Architectures, pp. 183–188 (2011)Google Scholar
  29. 29.
    Mendling, J., Strembeck, M., Recker, J.: Factors of process model comprehension - findings from a series of experiments. Decis. Support Syst. 53, 195–206 (2012)Google Scholar
  30. 30.
    Weitlaner, D., Guettinger, A., Kohlbacher, M.: Intuitive comprehensibility of process models. In: S-BPM ONE-running Processes, vol. 360, pp. 52–71 (2013)Google Scholar
  31. 31.
    Krogstie, J.: Model-Based Development and Evolution of Information Systems. Springer, London (2012)Google Scholar
  32. 32.
    Jošt, G., Huber, J., Heričko, M., Polančič, G.: An empirical investigation of intuitive understandability of process diagrams. Comput. Stand. Interface 48, 90–111 (2016)CrossRefGoogle Scholar
  33. 33.
  34. 34.
    Majaranta, P., Aoki, H., Donegan, M., Hansen, D.W., Hansen, J.P., Hyrskykari, A., Räihä, K.: Gaze Interaction and Applications of Eye Tracking: Advances in Assistive Technologies. IGI Global, Hershey (2011)Google Scholar
  35. 35.
    Reichert, M., Dadam, P.: Adept\(_{flex}\) – supporting dynamic changes of workflows without losing control. J. Int. Inf. Syst. 10, 93–129 (1998)CrossRefGoogle Scholar
  36. 36.
    Wang, W., Ding, H., Dong, J., Ren, C.: A comparison of business process modeling methods. In: International Conference on Service Operations and Logistics, and Informatics, pp. 1136–1141 (2006)Google Scholar
  37. 37.
    Salvucci, D.D., Goldberg, J.H.: Identifying fixations and saccades in eye-tracking protocols. In: Proceedings of 2000 Symposium on Eye Tracking Research Application, pp. 71–78 (2000)Google Scholar
  38. 38.
    OMG: Business Process Management and Notation 2.0 (2016). Accessed 11 Oct 2016
  39. 39.
    van der Aalst, W.M.P.: Formalization and verification of event-driven process chains. Inf. Softw. Technol. 41, 639–650 (1999)Google Scholar
  40. 40.
    Person, J.L.: Petri Net Theory and the Modeling of Systems. Prentice Hall, Englewood (1981)Google Scholar
  41. 41.
    Sommer, M.: Zeitliche Darstellung und Modellierung von Prozessen mithilfe von Gantt-Diagrammen. Bachelors Thesis, Ulm University (2012)Google Scholar
  42. 42.
    Bernstein, V., Soffer, P.: Identifying and quantifying visual layout features of business process models. In: Gaaloul, K., Schmidt, R., Nurcan, S., Guerreiro, S., Ma, Q. (eds.) CAISE 2015. LNBIP, vol. 214, pp. 200–213. Springer, Cham (2015). doi: 10.1007/978-3-319-19237-6_13 CrossRefGoogle Scholar
  43. 43.
    Schrepfer, M., Wolf, J., Mendling, J., Reijers, H.A.: The impact of secondary notation on process model understanding. In: Persson, A., Stirna, J. (eds.) PoEM 2009. LNBIP, vol. 39, pp. 161–175. Springer, Heidelberg (2009). doi: 10.1007/978-3-642-05352-8_13 CrossRefGoogle Scholar
  44. 44.
    Zugal, S., Soffer, P., Haisjackl, C., Pinggera, J., Reichert, M., Weber, B.: Investigating expressiveness and understandability of hierarchy in declarative business process models. Softw. Syst. Model. 14, 1081–1103 (2015)CrossRefGoogle Scholar
  45. 45.
    Linden, D., Zamansky, A., Hadar, I.: How cognitively effective is a visual notation? On the inherent difficulty of operationalizing the physics of notations. In: Schmidt, R., Guédria, W., Bider, I., Guerreiro, S. (eds.) BPMDS/EMMSAD -2016. LNBIP, vol. 248, pp. 448–462. Springer, Cham (2016). doi: 10.1007/978-3-319-39429-9_28 Google Scholar
  46. 46.
    Reijers, H.A., Mendling, J., Dijkman, R.M.: Human and automatic modularizations of process models to enhance their comprehension. J. Inf. Syst. 36, 881–897 (2011)CrossRefGoogle Scholar
  47. 47.
    Koschmider, A., Reijers, H.A., Dijkman, R.: Empirical support for the usefulness of personalized process model views. In: Multikonf Wirtschaftsinformatik (2012)Google Scholar
  48. 48.
    Claes, J., Vanderfessten, I., Pinggera, J., Reihers, H.A., Weber, B., Poels, G.: A visual analysis of the process of process modeling. Inf. Syst. e-Business Manage. 13, 147–190 (2015)CrossRefGoogle Scholar
  49. 49.
    Figl, K.: Comprehension of procedural visual business process models. Bus. Inf. Syst. Eng. 59, 41–57 (2017)CrossRefGoogle Scholar
  50. 50.
    Mili, H., Tremblay, G., Jaoude, G.B., Lefebvre, É., Elabed, L., Boussaidi, G.E.: Business process modeling languages: sorting through the alphabet soup. ACM Comput. Surv. 43, 1–56 (2010)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Michael Zimoch
    • 1
    Email author
  • Rüdiger Pryss
    • 1
  • Johannes Schobel
    • 1
  • Manfred Reichert
    • 1
  1. 1.Institute of Databases and Information SystemsUlm UniversityUlmGermany

Personalised recommendations