Keywords

1 Introduction

One of the key challenges modern organisations face is how to make effective decisions within a dynamic environment [1]. Precise understanding of various aspects of the organisation such as goals, organisation structure, operational processes, historic data and the stakeholders of the organisation is necessary to arrive at effective decisions [2]. Current industry practice of decision making relies heavily on human experts using tools such as spreadsheets, word processors, and diagram editors. Though providing computational and visualization support, these tools are rather primitive as regards analysis capabilities. This leads to excessive cognitive burden on human experts thus adversely affecting the quality and precision of decision making [3].

In recent years, the use of enterprise modelling (EM) is widely discussed in the context of complex enterprise wide missions [4] such as Business-IT alignment, enterprise transformation, etc. A wide range of sophisticated support for comprehending various aspects of enterprises make the EM techniques credible choice for such adoptions. For example, the Zachman framework [5] is capable of representing enterprises in a structured form by visualising them along six interrogative aspects namely what, where, when, why, who and how; the ArchiMate [6] and tool ArchiFootnote 1 enable comprehensive specification and visualisation support along structural, behavioural and information aspects of the organisation. As regards the analysis capabilities, the specification such as BPMN [7], i* [8] and stock-n-flow (SnF) [9] are amenable to sophisticated analysis. For example, the process aspect can be analysed and simulated using BPMN, the high level goals and objectives can be evaluated using i*, and high level system dynamics can be simulated using Stock-and-Flow (SnF) tools such as iThinkFootnote 2. These specification and analyses abilities of EM techniques motivated us to explore suitability of EM techniques for complex dynamic decision making (CDDM).

In this paper, we present a critical evaluation of enterprise modelling techniques as an aid for CDDM. We discuss the rigorous analysis carried out using Systematic Mapping Study methodology. In particular, we evaluated 173 publications that are rigorously selected from 7622 EM related publications collated from 5 popular digital libraries namely Scopus, ACM Digital Library, IEEE Xplore, ScienceDirect and Web of Science. The systematic study identified 29 enterprise modelling techniques which were then evaluated for suitability to CDDM. The key contributions of this paper are: (i) A detailed investigation of modelling and analysis needs for addressing CDDM problem leading to enumeration of a set of requirements, (ii) A report on systematic mapping study evaluating EM techniques as aid to CDDM in the context these requirements, and (iii) A report describing the suitability of EM state-of-the-art and industry expectations vis-a-vis CDDM in enterprises.

Table 1. Specification and analysis requirements for CDDM
Full size table

2 Complex Dynamic Decision Making

The efficacy of decision making primarily depends on two key factors: (i) the ability to capture relevant information, and (ii) the ability to perform what-if and if-what analyses on available information.

We argue that an enterprise can be understood well by analyzing what an enterprise is, how it operates, why it is so and who are the stakeholders [10]. Complex dynamic decision making (CDDM) deals with organisations that typically consist of many autonomous units, organized into dynamically changing hierarchical structures, and managing goals that affect their behaviour over time. They also deal with non-linear causality, several interdependent and localized feedback loops, and indefinite time-delay between action and responses [11]. Essentially, these socio-technical characteristics [2, 11] of complex dynamic organisation puts some special demands [12] on specification in terms of desirable characteristics. Table 1 enumerates specification and analysis requirements for CDDM. We conducted a comprehensive systematic mapping study to evaluate the specification and analysis characteristics of EM techniques described in Table 1. The next section describes the detailed report of the conducted study.

3 Systematic Mapping Study

We adopted systematic mapping study (SMS) methodology proposed by Petersen et al. [13] to conduct this review. Primarily the review process steps conform to three standard phases of SMS methodology namely planning, execution and reporting. The planning phase defines the review protocol that includes research questions, search criteria (i.e., inclusion criteria, exclusion criteria and quality criteria), selection of digital libraries, and the study template for conducting review. The execution phase executes the plan through two logical steps namely exploration of digital libraries and conduct studies. The exploration of digital libraries logical step iteratively explores identified digital libraries using the search criteria described in the planning phase and identifies the relevant publications. The conduct studies logical step studies all identified publications and documents the study outcome as prescribed by the study template of review protocol. Finally, the report review phase summarizes all studies and their outcomes in a precise form. We now describe the activities and outcomes of all the three phases of EM literature review.

Table 2. Review Protocol for conducting review on EM publications
Full size table

3.1 Planning Phase

The planning phase formalize the protocol for conducting review as described in Table 2. The protocol defines two research questions RQ1 and RQ2. The research question RQ1- What are the papers on Enterprise Modelling (EM) and Enterprise Architecture (EA) that focus on enterprise modelling? identifies EM related publications. The sub-question RQ1.1 is designed to consolidate cited EM techniques. Research question RQ2 also ascertains suitability and gap of EM techniques to support CDDM.

The inclusion criterion of this review is very broad as it is designed to find all Enterprise Modeling (EM) and Enterprise Architecture (EA) related literature. The exclusion criterion is designed to eliminate EM publications that are irrelevant for this study. We consider publications that solely focus on workflow, process mining, security, and infrastructure related topics as not much relevant to CDDM. Two constraints are defined as part of quality criteria, they are: (i) paper should be aligned with the problem statement and (ii) paper should be cited by at least one refereed paper (excluding self-citation) if it is published before 2014. The former quality criterion checks the relevance and the latter validates minimum acknowledgment from research community.

The review protocol of this study chooses five digital libraries namely Scopus, ACM Digital Library, IEEE Xplore, ScienceDirect and Web of Science for identifying EM related publications. This step also specifies all digital library specific search strings for both inclusion criteria and exclusion criteria. The search strings of other digital libraries are available in appendices sectionFootnote 3. The study template described in Table 2 is an extension of the standard attributes defined by EBSE Research Group in [14]. Additional attribute EM technique referred (see Study Template row of Table 2) captures the list of cited EM techniques, and the attribute Key Findings captures the high-level description of the publication using free text form.

Fig. 1.
figure 1

Overview of execution phase

Full size image

3.2 Execution Phase

The exploration of digital libraries logical step of execution phase explores the five digital libraries using the search criteria described in Table 2 to find relevant publications. The count of identified publications for each process step is depicted in Fig. 1. The inclusion criterion of review protocol collectively selected 7622 publications (with multiple duplicate entries); total 1855 publications were short-listed by exclusion criteria; and finally 173 publications were selected after evaluating quality criteria. The second logical step conduct study is performed on 173 publications.

Fig. 2.
figure 2

Publication trends of EM literature

Full size image

3.3 Synthesis Phase

The synthesis phase analyzes all studies captured using the study template and reports the answers of two research questions formulated in the study protocol. The final outcome of the review synthesis answering two research questions is described below:

Answers to RQ 1 - What are the papers on Enterprise Modeling (EM) and Enterprise Architecture (EA) that focus on organisation modelling?

As shown in Fig. 1, 7622 publications on a range of EM and EA related topics were initially identified. The trend of publications since 1990 with 5 year time-span as a time unit is depicted in Fig. 2. It indicates an increasing trend of EM publications with a significant increase in the last 5-year slot. This trend is consistent over all 5 digital libraries. We also conducted trend analysis on final selection of 173 publications. These publications are contributed from 35 countries involving 161 institutions/organisations in time span of 1987 to 2016. The complete list of publications is presented in appendices section. The consolidation of EM techniques referred attribute of 173 publication studies collectively report 29 EM techniques as an answer to sub-question - What are the EM techniques cited by those publications? Table 3 describes identified EM techniques. The useful references associated with identified EM techniques are listed in appendices section.

Table 3. Identified EM techniques
Full size table

Answers to RQ 2: What are the modelling and analysis characteristics reported in EM? We conducted a series of detailed studies to understand the EM techniques identified by RQ 1.1. Table 4 summarizes the studies wherein each row is a representation of the suitability and gap analysis of a specific EM technique. We omitted TOGAF, EIF, SEAM and MoKI in the below list as TOGAF and SEAM primarily focus only on the method aspects whereas EIF is a set of guidance to European public administrations about the design of European public services and MoKI is an wiki site for enterprise modelling.

Table 4. Consolidated study report
Full size table

The synthesis of Table 4 asserts some characteristics of identified EM techniques. The appearance count of EM techniques in large collections selected uning inclusion criteria in Table 4 depicts the popularity of EM techniques. The Zachman Framework, Archimate, ARIS, UML are referred very frequently in EM literature whereas EEML, IEM, KAOS are not referred extensively. The column describing the appearance counts within selected in final consolidation ensure the coverage of this study. We further analyse the capabilities of EM techniques along three dimensions namely specification, visualization and analysis.

Specification: Table 4 shows a consensus among EM techniques that an enterprise should be specified using multiple views or aspects. It is visible throughout the table as all EM techniques support more than one aspect. However, we found only 6 EM techniques (i.e., Zachman Framework, ArchiMate, DoDAF, MoDAF, GERAM and UEML) out of 25 to be capable of specifying all the four aspects. We also see a major inadequacy in supporting specification of adaptability, uncertainty and temporal characteristics. No EM technique is capable of specifying adaptation and uncertainty whereas only the system dynamic model is capable of specifying the temporal property. This leads us to infer that the state-of-the-art of enterprise modeling and specification are purposive and they cover limited aspects.

Visualization: We find adequate visualization aids for most of the EM techniques. 18 EM techniques are supported by advanced visualization aids and 5 other EM techniques also supported with primitive visualization aids.

Analysis: We find considerable lacunae in machine interpretable specification, qualitative analysis and quantitative analysis. In particular, 8 EM techniques partly support machine interpretable specification, 8 EM techniques support partial quantitative analysis and 7 of the 25 identified EM techniques support partial qualitative analysis.

The above analysis shows that the state-of-the-art EM techniques fare better on documentation and visualization than quantitative and qualitative analysis. Also, there is inadequate support for socio-technical characteristics such as adaptability, uncertainty, and temporal properties.

3.4 Threats to Validity

Four kinds of validity threats namely construct validity, conclusion validity, internal validity and external validity are considered while conducting this study. The construct validity is ensured through appropriate measures. Firstly, the search-string for inclusion criteria is formulated using three sufficiently stable terminologies of EM literature namely Enterprise Model, Enterprise Modelling and Enterprise Architecture. Secondly, the search string for exclusion criteria is constructed using an iterative method considering one phrase at a time. The construct validity related to coverage assurance is addressed by considering five digital libraries namely Scopus, ScienceDirect, ACM Digital Library, IEEE Xplore and Web of Science. Finally, the construct validity is ensured through rigorous review process. In particular, three researchers from review team (i.e., authors of this paper) independently validated the review protocol specified by other researcher. The conclusion validity or reliability focuses on whether the data are collected and the analysis is conducted in a repeatable form. We defined search terms based on stable phrases, used a well-defined procedure and emphasized on standard digital libraries to make the study repeatable. The automated search capabilities of proposed digital libraries are used to a large extent for improving the repeatability. All these measures contributed to address conclusion validity. Internal validity is concerned with the analysis of the data. The analysis primarily based on the descriptive statistics, thus the threats are minimal. Finally, external validity is about generalization from this study. Since we do not draw any conclusion outside of our primary focus on evaluating EM techniques for CDDM, the external validity threats are not applicable.

4 Conclusion

We presented the necessary characteristics of enterprise specification to effectively support CDDM. A systematic mapping study was carried out to evaluate existing EM techniques for supporting CDDM in enterprises. The study concluded with key observations: (i) the majority of the existing EM techniques meet the most basic requirement of specifying relavant aspects of organisation for CDDM, (ii) existing EM techniques display a range of automated analysis also, (iii) however, none of the existing EM techniques is capable of meeting all necessary requirements of enterprise specification for CDDM, and (iv) the key concern is to specify and analyse socio-technical characteristics of complex dynamic organisation. A cursory study of Actor Model of Computation [15] reveals that it can be a good candidate for specifying socio-technical characteristics of enterprises. As a next step, we intent to conduct similar study to explore suitability of Actor languages and frameworks for CDDM in enterprises.