Empirical Studies in Process Model Verification

  • Jan Mendling
Part of the Lecture Notes in Computer Science book series (LNCS, volume 5460)


Despite the large body of knowledge on formal analysis techniques for process models, in particular Petri nets, there has been a notable gap of empirical research into verification. In this paper we compare the few studies that report results from applying verification techniques to real-world process model collections. For this comparison we are particularly interested in the different approaches, their computational performance, and the number of errors found. Our comparison reveals that most of the samples have error rates of 10% to 20%. Some of the studies have established a connection between error probability and process model metrics, as well as between model understanding and both metrics and modeling competence of the model reader. Based on these results, we discuss implications and directions for future research.


Business Process Reduction Rule Business Process Model Reachability Graph Correctness Criterion 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Lindland, O., Sindre, G., Sølvberg, A.: Understanding quality in conceptual modeling. IEEE Software 11, 42–49 (1994)CrossRefGoogle Scholar
  2. 2.
    Becker, J., Rosemann, M., Uthmann, C.: Guidelines of Business Process Modeling. In: van der Aalst, W., Desel, J., Oberweis, A. (eds.) Business Process Management. Models, Techniques, and Empirical Studies, pp. 30–49. Springer, Berlin (2000)CrossRefGoogle Scholar
  3. 3.
    Krogstie, J., Sindre, G., Jørgensen, H.: Process models representing knowledge for action: a revised quality framework. Europ. J. of Inf. Systems 15, 91–102 (2006)CrossRefGoogle Scholar
  4. 4.
    Moody, D.: Theoretical and practical issues in evaluating the quality of conceptual models: current state and future directions. Data & Knowl. Eng. 55, 243–276 (2005)CrossRefGoogle Scholar
  5. 5.
    Davies, I., Green, P., Rosemann, M., Indulska, M., Gallo, S.: How do practitioners use conceptual modeling in practice? Data & Knowl. Eng. 58, 358–380 (2006)CrossRefGoogle Scholar
  6. 6.
    Boehm, B.: Software Engineering Economics. Prentice-Hall, Englewood Cliffs (1981)zbMATHGoogle Scholar
  7. 7.
    Wand, Y., Weber, R.: Research Commentary: Information Systems and Conceptual Modeling - A Research Agenda. Information Systems Research 13, 363–376 (2002)CrossRefGoogle Scholar
  8. 8.
    Rosemann, M.: Potential pitfalls of process modeling: part a. Business Process Management Journal 12, 249–254 (2006)CrossRefGoogle Scholar
  9. 9.
    Philippi, S., Hill, H.: Communication support for systems engineering - process modelling and animation with april. Journal of Sys. & Softw. 80, 1305–1316 (2007)CrossRefGoogle Scholar
  10. 10.
    van Hee, K., Sidorova, N., Somers, L., Voorhoeve, M.: Consistency in model integration. Data & Knowledge Engineering 56 (2006)Google Scholar
  11. 11.
    van Dongen, B., Vullers-Jansen, M., Verbeek, H., van der Aalst, W.: Verification of the sap reference models using epc reduction, state-space analysis, and invariants. Computers in Industry 58, 578–601 (2007)CrossRefGoogle Scholar
  12. 12.
    Mendling, J., Verbeek, H., van Dongen, B., van der Aalst, W., Neumann, G.: Detection and Prediction of Errors in EPCs of the SAP Reference Model. Data & Knowl. Eng. 64, 312–329 (2008)CrossRefGoogle Scholar
  13. 13.
    Mendling, J., Neumann, G., van der Aalst, W.: Understanding the occurrence of errors in process models based on metrics. In: Meersman, R., Tari, Z. (eds.) OTM 2007, Part I. LNCS, vol. 4803, pp. 113–130. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  14. 14.
    Vanhatalo, J., Völzer, H., Leymann, F.: Faster and more focused control-flow analysis for business process models through sese decomposition. In: Krämer, B.J., Lin, K.-J., Narasimhan, P. (eds.) ICSOC 2007. LNCS, vol. 4749, pp. 43–55. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  15. 15.
    Gruhn, V., Laue, R.: What business process modelers can learn from programmers. Science of Computer Programming 65, 4–13 (2007)MathSciNetCrossRefGoogle Scholar
  16. 16.
    Mendling, J., Reijers, H., van der Aalst, W.: Seven Process Modeling Guidelines (7PMG). In: Qut eprint (2008)Google Scholar
  17. 17.
    Adrion, W., Branstad, M., Cherniavsky, J.: Validation, verification, and testing of computer software. ACM Computing Surveys 14, 159–192 (1982)CrossRefGoogle Scholar
  18. 18.
    IEEE: IEEE Std 610.12-1990 IEEE Standard Glossary of Software Engineering Terminology. IEEE Computer Society Press (1990)Google Scholar
  19. 19.
    Valmari, A.: The state explosion problem. In: Reisig, W., Rozenberg, G. (eds.) APN 1998. LNCS, vol. 1491, pp. 429–528. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  20. 20.
    Hoppenbrouwers, S., Proper, H., van der Weide, T.: A fundamental view on the process of conceptual modeling. In: Delcambre, L.M.L., Kop, C., Mayr, H.C., Mylopoulos, J., Pastor, Ó. (eds.) ER 2005. LNCS, vol. 3716, pp. 128–143. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  21. 21.
    Boehm, B.W.: Software engineering; R & D trends and defense needs. In: Research Directions in Software Technology. MIT Press, Cambridge (1979)Google Scholar
  22. 22.
    Sommerville, I.: Software Engineering, 6th edn. Addison-Wesley, Reading (2001)zbMATHGoogle Scholar
  23. 23.
    van der Aalst, W.: Verification of Workflow Nets. In: Azéma, P., Balbo, G. (eds.) ICATPN 1997. LNCS, vol. 1248, pp. 407–426. Springer, Heidelberg (1997)CrossRefGoogle Scholar
  24. 24.
    Dehnert, J., Rittgen, P.: Relaxed Soundness of Business Processes. In: Dittrich, K.R., Geppert, A., Norrie, M.C. (eds.) CAiSE 2001. LNCS, vol. 2068, pp. 157–170. Springer, Heidelberg (2001)CrossRefGoogle Scholar
  25. 25.
    Dehnert, J., Zimmermann, A.: On the suitability of correctness criteria for business process models. In: van der Aalst, W.M.P., Benatallah, B., Casati, F., Curbera, F. (eds.) BPM 2005. LNCS, vol. 3649, pp. 386–391. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  26. 26.
    Weske, M.: Business Process Management: Concepts, Languages, Architectures. Springer, Heidelberg (2007)Google Scholar
  27. 27.
    Pnueli, A.: The Temporal Logic of Programs. In: Proceedings of the 18th IEEE Annual Symposium on the Foundations of Computer Science, pp. 46–57 (1977)Google Scholar
  28. 28.
    Kindler, E.: On the semantics of EPCs: Resolving the vicious circle. Data & Knowledge Engineering 56, 23–40 (2006)CrossRefGoogle Scholar
  29. 29.
    Mendling, J., van der Aalst, W.: Formalization and Verification of EPCs with OR-Joins Based on State and Context. In: Krogstie, J., Opdahl, A.L., Sindre, G. (eds.) CAiSE 2007 and WES 2007. LNCS, vol. 4495, pp. 439–453. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  30. 30.
    Mendling, J.: Detection and Prediction of Errors in EPC Business Process Models. PhD thesis, Vienna University of Economics and Business Administration (2007)Google Scholar
  31. 31.
    Keller, G., Nüttgens, M., Scheer, A.W.: Semantische Prozessmodellierung auf der Grundlage “Ereignisgesteuerter Prozessketten (EPK)”. Heft 89, Institut für Wirtschaftsinformatik, Saarbrücken, Germany (1992)Google Scholar
  32. 32.
    Murata, T.: Petri Nets: Properties, Analysis and Applications. Proceedings of the IEEE 77, 541–580 (1989)CrossRefGoogle Scholar
  33. 33.
    Desel, J., Esparza, J.: Free Choice Petri Nets. Cambridge Tracts in Theoretical Computer Science, vol. 40. Cambridge Univ. Press, Cambridge (1995)CrossRefzbMATHGoogle Scholar
  34. 34.
    Verbeek, H., van der Aalst, W.: Woflan 2.0: A Petri-net-based Workflow Diagnosis Tool. In: Nielsen, M., Simpson, D. (eds.) ICATPN 2000. LNCS, vol. 1825, pp. 475–484. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  35. 35.
    Verbeek, H., Basten, T., van der Aalst, W.: Diagnosing Workflow Processes using Woflan. The Computer Journal 44, 246–279 (2001)CrossRefzbMATHGoogle Scholar
  36. 36.
    Mendling, J., Moser, M., Neumann, G., Verbeek, H., van Dongen, B., van der Aalst, W.: Faulty EPCs in the SAP Reference Model. In: Dustdar, S., Fiadeiro, J.L., Sheth, A.P. (eds.) BPM 2006. LNCS, vol. 4102, pp. 451–457. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  37. 37.
    Verbeek, H., van der Aalst, W., ter Hofstede, A.: Verifying workflows with cancellation regions and or-joins: An approach based on relaxed soundness and invariants. The Computer Journal 50, 294–314 (2007)CrossRefGoogle Scholar
  38. 38.
    van Dongen, B., Jansen-Vullers, M.H.: Verification of SAP reference models. In: van der Aalst, W.M.P., Benatallah, B., Casati, F., Curbera, F. (eds.) BPM 2005. LNCS, vol. 3649, pp. 464–469. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  39. 39.
    Johnson, R., Pearson, D., Pingali, K.: The program structure tree: Computing control regions in linear time. In: Proceedings of the ACM SIGPLAN’94 Conference on Programming Language Design and Implementation (PLDI). SIGPLAN Notices, vol. 29(6), pp. 171–185 (1994)Google Scholar
  40. 40.
    Esparza, J.: Reduction and synthesis of live and bounded free choice petri nets. Information and Computation 114, 50–87 (1994)MathSciNetCrossRefzbMATHGoogle Scholar
  41. 41.
    Keller, G., Teufel, T.: SAP(R) R/3 Process Oriented Implementation: Iterative Process Prototyping. Addison-Wesley, Reading (1998)Google Scholar
  42. 42.
    Mendling, J., van der Aalst, W., van Dongen, B., Verbeek, H.: Referenzmodell: Sand im Getriebe - Webfehler. iX - Magazin für Professionelle Informationstechnik (in German), 131–133 (2006)Google Scholar
  43. 43.
    Becker, J., Schütte, R.: Handelsinformationssysteme. 2nd edn. Moderne Industrie, Landsberg/Lech (2004)Google Scholar
  44. 44.
    Scheer, A.W.: Wirtschaftsinformatik: Referenzmodelle für industrielle Geschäftsprozesse, 2nd edn. Springer, Heidelberg (1998)Google Scholar
  45. 45.
    Seidlmeier, H.: Prozessmodellierung mit ARIS. Vieweg Verlag (2002)Google Scholar
  46. 46.
    Staud, J.: Geschäftsprozessanalyse: Ereignisgesteuerte Prozessketten und Objektorientierte Geschäftsprozessmodellierung für Betriebswirtschaftliche Standardsoftware, 3rd edn. Springer, Heidelberg (2006)Google Scholar
  47. 47.
    Colom, J., Silva, M.: Convex geometry and semiflows in P/T nets, A comparative study of algorithms for computation of minimal P-semiflows. In: Rozenberg, G. (ed.) APN 1990. LNCS, vol. 483, pp. 79–112. Springer, Heidelberg (1991)CrossRefGoogle Scholar
  48. 48.
    Simon, H.: Sciences of the Artificial, 3rd edn. MIT Press, Cambridge (1996)Google Scholar
  49. 49.
    Burton-Jones, A., Meso, P.: How Good are these UML Diagrams? An Empirical Test of the Wand and Weber Good Decomposition Model. In: Applegate, L., Galliers, R., DeGross, J. (eds.) Proceedings of ICIS, pp. 101–114 (2002)Google Scholar
  50. 50.
    Mendling, J., Reijers, H., Cardoso, J.: What makes process models understandable? In: Alonso, G., Dadam, P., Rosemann, M. (eds.) BPM 2007. LNCS, vol. 4714, pp. 48–63. Springer, Heidelberg (2007)CrossRefGoogle Scholar
  51. 51.
    Mendling, J., Strembeck, M.: Influence factors of understanding business process models. In: Abramowicz, W., Fensel, D. (eds.) Proc. of the 11th International Conference on Business Information Systems (BIS 2008). LNBIP, vol. 7, pp. 142–153 (2008)Google Scholar
  52. 52.
    Laue, R., Mendling, J.: The impact of structuredness on error probability of process models. In: Kaschek, R., Kop, C., Steinberger, C., Fliedl, G. (eds.) UNISCON 2008. LNBIP, vol. 5, pp. 585–590. Springer, Heidelberg (2008)Google Scholar
  53. 53.
    Mendling, J., Reijers, H.: How to define activity labels for business process models? In: Oberweis, A., Hesse, W. (eds.) Proc. of the Third AIS SIGSAND Europe 2008. LNI (2008)Google Scholar
  54. 54.
    Ehrenfeucht, A., Rozenberg, G.: Partial (Set) 2-Structures - Part 1 and Part 2. Acta Informatica 27, 315–368 (1989)CrossRefzbMATHGoogle Scholar
  55. 55.
    Cortadella, J., Kishinevsky, M., Lavagno, L., Yakovlev, A.: Deriving petri nets from finite transition systems. IEEE Transactions on Computers 47, 859–882 (1998)MathSciNetCrossRefGoogle Scholar
  56. 56.
    Mendling, J., van Dongen, B., van der Aalst, W.: Getting Rid of the OR-Join in Business Process Models. In: Proceedings of the 11th IEEE International Enterprise Distributed Object Computing Conference (EDOC 2007), pp. 3–14 (2007)Google Scholar
  57. 57.
    Zhao, W., Hauser, R., Bhattacharya, K., Bryant, B., Cao, F.: Compiling business processes: untangling unstructured loops in irreducible flow graphs. Int. Journal of Web and Grid Services 2, 68–91 (2006)CrossRefGoogle Scholar
  58. 58.
    van der Aalst, W., Lassen, K.: Translating unstructured workflow processes to readable BPEL: Theory and implementation. Inf.& Softw. T. 50, 131–159 (2008)CrossRefGoogle Scholar
  59. 59.
    Nuseibeh, B., Easterbrook, S.: Requirements engineering: a roadmap, pp. 35–46 (2000)Google Scholar
  60. 60.
    Tsai, W., Vishnuvajjala, R.: Verification and Validation of Knowledge-Based Systems. IEEE Transactions on Knowledge and Data Engineering 11, 202–212 (1999)CrossRefGoogle Scholar
  61. 61.
    Sargent, R.: Verification and validation of simulation models, pp. 130–143 (2005)Google Scholar
  62. 62.
    Frederiks, P., van der Weide, T.: Information modeling: The process and the required competencies of its participants. Data & Knowl. Eng. 58, 4–20 (2006)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2009

Authors and Affiliations

  • Jan Mendling
    • 1
  1. 1.Humboldt-Universität zu BerlinBerlinGermany

Personalised recommendations