Designing a Decision-Making Process for Partially Observable Environments Using Markov Theory

  • Sérgio GuerreiroEmail author
Conference paper
Part of the Lecture Notes in Business Information Processing book series (LNBIP, volume 287)


This paper is motivated by the problem of deciding how to proceed in a business process when a workaround occurs. In addition to this problem, most of the times, the exact state of the business processes is not fully available to the industrial organization. Therefore, it means that something wrong happens during automatic (or manual) operation; however, the managers do not know exactly the state of the operating systems. Usually, the combination of these problems drives to management decision without enough information and thus error prone. This paper integrates Markov theory with business processes design to predict the impact of each decision in the operational environment. The solution is tested in an agro-food industrial company that transforms fresh fruit to preparations that are sold to others companies. The paper shows that anticipating the production processes changes has the benefit of minimizing lot infections or stock disrupts. The main identified limitations are the compute intensive process involved and the effort required to estimate the business processes details. In future research, this work might be (i) extended to friendly software interfaces in order to facilitate the interaction with end-users, (ii) optimized to the algorithm of informed decision-making computation, and (iii) automatic estimation of business processes details using machine learning techniques.


Actuation Business processes Instances Markov theories Models Observation 



This work was supported by national funds through Fundação para a Ciência e a Tecnologia (FCT) with reference UID/CEC/50021/2013.


  1. 1.
    van der Aalst, W.M.P.: Process Mining: Discovery, Conformance and Enhancement of Business Processes. Database Management & Information Retrieval. Springer, Heidelberg (2011)CrossRefGoogle Scholar
  2. 2.
    von Alan, R.H., March, S.T., Park, J., Ram, S.: Design science in information systems research. MIS Q. 28(1), 75–105 (2004)Google Scholar
  3. 3.
    Bertalanffy, L.V.: General Systems Theory. George Braziller, New York (1969)Google Scholar
  4. 4.
    Boehm, B.W., Madachy, R., Steece, B., et al.: Software Cost Estimation with Cocomo II with Cdrom. Prentice Hall PTR, Upper Saddle River (2000)Google Scholar
  5. 5.
    Bonhomme, P.: Marking estimation of p-time petri nets with unobservable transitions. IEEE Trans. Syst. Man Cybern. Syst. 45(3), 508–518 (2015)CrossRefGoogle Scholar
  6. 6.
    Cai, P.: Approximate POMDP planning (appl) toolkit (2010).
  7. 7.
    Cassandra, A.R.: POMDP solver (v53 2005).
  8. 8.
    De Bruyn, P., Huysmans, P., Oorts, G., Mannaert, H., Verelst, J.: Using entropy’s justificatory knowledge for a business process design theory. In: 2014 47th Hawaii International Conference on System Sciences (HICSS), pp. 3717–3726. IEEE (2014)Google Scholar
  9. 9.
    Dietz, J.L.G.: Enterprise Ontology: Theory and Methodology. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  10. 10.
    Dietz, J.L., Hoogervorst, J.A., Albani, A., Aveiro, D., Babkin, E., Barjis, J., Caetano, A., Huysmans, P., Iijima, J., van Kervel, S., et al.: The discipline of enterprise engineering. Int. J. Organ. Des. Eng. 3(1), 86–114 (2013)Google Scholar
  11. 11.
    Elovici, Y., Braha, D.: A decision-theoretic approach to data mining. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum. 33(1), 42–51 (2003)CrossRefGoogle Scholar
  12. 12.
    Fleischmann, A., Stary, C.: Whom to talk to? A stakeholder perspective on business process development. Univ. Access Inf. Soc. 11(2), 125–150 (2012). CrossRefGoogle Scholar
  13. 13.
    Franklin, G., Powell, J., Emami-Naeini, A.: Feedback Control of Dynamic Systems, 2nd edn. Addison-Wesley Publishing Company, Reading (1991)Google Scholar
  14. 14.
    Group, O.M: BPMN specification 1.2. pdf available in internet, January 2009Google Scholar
  15. 15.
    Guerreiro, S.: Decision-making in partially observable environments. In: 2014 IEEE 16th Conference on Business Informatics (CBI), vol. 1, pp. 159–166, July 2014Google Scholar
  16. 16.
    Guerreiro, S., Tribolet, J.: Conceptualizing enterprise dynamic systems control for run-time business transactions. In: European Conference on Information Systems (ECIS) 2013, paper 5 (2013)Google Scholar
  17. 17.
    Guerreiro, S.: Business rules elicitation combining Markov decision process with demo business transaction space. In: 2013 IEEE 15th Conference on Business Informatics (CBI), pp. 13–20 (2013)Google Scholar
  18. 18.
    Guizzardi, G.: On ontology, ontologies, conceptualizations, modeling languages, and (meta) models. Front. Artif. Intell. Appl. 155, 18 (2007)Google Scholar
  19. 19.
    Hoogervorst, J.A.: Enterprise Governance and Enterprise Engineering. The Enterprise Engineering Series. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  20. 20.
    Johnson, P., Ullberg, J., Buschle, M., Franke, U., Shahzad, K.: An architecture modeling framework for probabilistic prediction. Inf. Syst. e-Bus. Manag. 12(4), 595–622 (2014)CrossRefGoogle Scholar
  21. 21.
    Laudon, K.C., Laudon, J.P., et al.: Essentials of Management Information Systems. Pearson, Upper Saddle River (2011)Google Scholar
  22. 22.
    Metzger, A., Leitner, P., Ivanovi, D., Schmieders, E., Franklin, R., Carro, M., Dustdar, S., Pohl, K.: Comparing and combining predictive business process monitoring techniques. IEEE Trans. Syst. Man Cybern. Syst. 45(2), 276–290 (2015)CrossRefGoogle Scholar
  23. 23.
    Mikaelian, T., Nightingale, D.J., Rhodes, D.H., Hastings, D.E.: Real options in enterprise architecture: a holistic mapping of mechanisms and types for uncertainty management. IEEE Trans. Eng. Manag. 58(3), 457–470 (2011)CrossRefGoogle Scholar
  24. 24.
    Montiel, L., Bickel, J.: A simulation-based approach to decision making with partial information. Decis. Anal. 9(4), 329–347 (2012)CrossRefGoogle Scholar
  25. 25.
    OMG: Business process model and notation (2011).
  26. 26.
    Österle, H., Becker, J., Frank, U., Hess, T., Karagiannis, D., Krcmar, H., Loos, P., Mertens, P., Oberweis, A., Sinz, E.J.: Memorandum on design-oriented information systems research. Eur. J. Inf. Syst. 20(1), 7–10 (2011)CrossRefGoogle Scholar
  27. 27.
    Puterman, M.L.: Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley, New York (1994)CrossRefGoogle Scholar
  28. 28.
    Reijers, H.A., Limam, S., van der Aalst, W.M.P.: Product-based workflow design. J. Manag. Inf. Syst. 20(1), 229–262 (2003)Google Scholar
  29. 29.
    Rozinat, A., van der Aalst, W.M.P.: Conformance checking of processes based on monitoring real behavior. Inf. Syst. 33(1), 64–95 (2008). doi: 10.1016/ CrossRefGoogle Scholar
  30. 30.
    Russell, S., Norvig, P.: Artificial Intelligence: A Modern approach. Artificial Intelligence, 3rd edn. Prentice Hall, Englewood Cliffs (2010)Google Scholar
  31. 31.
    Smith, T.: ZMDP software for POMDP and MDP planning (2007).
  32. 32.
    Sommestad, T., Ekstedt, M., Johnson, P.: A probabilistic relational model for security risk analysis. Comput. Secur. 29(6), 659–679 (2010)CrossRefGoogle Scholar
  33. 33.
    Spaan, M.: Perseus randomized point-based approximate value iteration algorithm (2004).
  34. 34.
    van der Aalst, W.M.P., et al.: Process mining manifesto. In: Daniel, F., Barkaoui, K., Dustdar, S. (eds.) BPM 2011. LNBIP, vol. 99, pp. 169–194. Springer, Heidelberg (2012). doi: 10.1007/978-3-642-28108-2_19 CrossRefGoogle Scholar
  35. 35.
    Vanderfeesten, I., Reijers, H.A., Van der Aalst, W.M.P.: Product-based workflow support. Inf. Syst. 36(2), 517–535 (2011)CrossRefGoogle Scholar
  36. 36.
    Weber, M.: Decision making with incomplete information. Eur. J. Oper. Res. 28(1), 44–57 (1987)CrossRefGoogle Scholar
  37. 37.
    Wieringa, R.J.: Design Science Methodology for Information Systems and Software Engineering. Springer, London (2014)CrossRefGoogle Scholar
  38. 38.
    Wilf, J., Port, D.: Decisions and disasters: modeling decisions that contribute to mishaps. In: 2016 49th Hawaii International Conference on System Sciences (HICSS), pp. 5635–5641. IEEE (2016)Google Scholar
  39. 39.
    Winograd, T.: A language/action perspective on the design of cooperative work. In: Proceedings of the 1986 ACM Conference on Computer-Supported Cooperative Work, pp. 203–220. ACM (1986)Google Scholar
  40. 40.
    Winter, R.: Design science research in Europe. Eur. J. Inf. Syst. 17(5), 470–475 (2008)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Instituto Superior TécnicoUniversity of LisbonLisbonPortugal
  2. 2.INESC-IDLisbonPortugal

Personalised recommendations