Keywords

1 Introduction

Interoperability is defined by the ALCTSFootnote 1 [2] as the ability of two or more systems or components to exchange information and use the exchanged information without special effort by either system. Improving interoperability depends largely on the implementation of a Collaborative Information System(CIS) by means of a software engineering process. In this work, the interoperability requirements specification is based on measurable and non-measurable quality characteristics. It is also demonstrated that the improvement proposed in software specification activity will have positive impact on the software development activity. For the validation activity, the definition of a interoperability testing sub-process is made. The next three sections presents respectively the literature reviews of requirements specification, interoperability in relation to the software process and business process performance. The two following sections are dedicated to the research design and the application respectively.

2 Literature of Software Requirements Verification and Validation

After the validation activity, the software system is delivered to the customer and is installed and put into practical use. When the validation activity reveals problems, this means that the system is not good enough for use, then further development is required to fix the identified problems [1]. Software system requirements are often classified as functional and non-functional requirements [1]. Software validation or, more generally, verification and validation is intended to show that a system both conforms to its specification and that it meets the expectations of the system customer [1, 3]. Except for small programs, systems could not be tested as a single, monolithic unit. The testing process is made up of three stages, Development testing, System testing and Acceptance testing, in which system components are tested then the integrated system is tested and finally, the system is tested on customer’s data [1]. The aim of verification is to check that the software meets its stated functional and non functional requirements [1, 3, 4]. The aim of validation is to ensure that the software meets the customer’s expectations (i.e. expectations of the organization that commissioned the system) [1, 3]. Properties expressed as quantitative measures can be naturally verified [5]. Properties that refer to subjective feeling can be difficult to verify and are a natural target for validation [5].

3 Interoperability Literature Review

A review of the literature conducted to analyze how the activities of software specification, software development and software validation [1] were carried out in the interoperability domain.

3.1 Interoperability Requirements

Approaches used to represent interoperability requirements, in the literature can be summarized as following with their limitations:

  1. 1.

    The maturity models [6–8]: Repetition, Ambiguity, Imprecision and incoherence because the needs are expressed in natural language [9].

  2. 2.

    Formal representation of interoperability requirements [9, 10]: the implementation of this approach may experiment scalability problems [11].

  3. 3.

    Interoperability requirements as problems.

    1. (a)

      Requirements are specified by the mean of collaboration processes models: the requirements are disconnected from existing systems and also to the interoperability problems to solve.

    2. (b)

      The interoperability matrix [12, 13]: do not provide a structured set of requirements [14].

3.2 Collaborative Information System (CIS) Architecture

A CIS aims at supporting “Information Systems (IS) Interoperability”, that is to say, to satisfy requirements such as data conversion, application sharing and process management [15]. The CIS should then include two different parts: connectors to be plugged into partners information systems and the concrete entity managing the collaboration: an intermediate information system. Several research works were conducted in order to find logical and technical solutions for the CIS. Approaches proposed in these works can be categorized in two groups: Model-Driven Interoperability (MDI) approaches [16, 17] and Business Process Lifecycle (BPL) based approaches [18]. It can be noticed that, in all approaches for CIS development, the proposed platforms are based on Service Oriented Architectures (SOA) [19]. Requirements specification proposed in the approaches for the development of CIS does not provide sufficient information to describe interoperability problems and then facilitate the software development activity.

3.3 SOA Testing

Since our problem is related to the verification and the validation of interoperability, we will focus primarily on testing approaches and techniques for service-based systems. For the testing of SOA applications, [20] identified four distinct testing layers: Unit testing, Service testing, Integration testing and System testing. References [21, 22] advocated for the realization of system testing at process level. The first limitation of this work [21], is the fact that the business process performance metrics used in the test process are not defined. In our opinion, although interoperability implementation is generally based on SOA, the testing approaches proposed for service-based systems are not suitable for testing inter-enterprise interoperability achievement. Indeed, these approaches [20–22] do not reference interoperability problems, which makes them useless for verifying the elimination of the latter.

4 Business Process Performance Indicators Calculation from Event Logs

The Business Process represents a chain of interrelated activities that normally must be connected with the customer requirements [23]. Process models are built in the design phase and are later used to implement the information system. Process performance measurement is an important aspect of Business Process Management (BPM). In this research, the considered process PIs are the average elapsed time, average cost and percentage of failure at the process level and represent the aggregation of the PIs for the activities [24, 25]. This choice relies on the assumption that these characteristics encompass all other types of dynamic properties of business processes [11].

An “event log” is defined as “a chronological record of computer system activities” which are saved to a file on the system [26]. Any information system using transactional systems will provide workflow information in some form and to some extent, such as tasks available, their events and the details of these events like starting/ending timestamp [27, 28]. The main data source that can be used for the calculation process PIs in the verification of interoperability requirements are the integrated event logs which are widely used in the fied of cross-organizational process mining. Cross-organizational workflow is usually distributed on different servers owned by different partners or different organizations [29, 30]. Cross-organizational workfows are enabled by web-services technology [31]. A Framework for workflow integration based on process mining is proposed in [30]. The integrated log results from the integration between the running logs of two or more organizations. In the running log collected from each organization, an event record is an 8-tuples containing Case(Id), Activity (Name), Start time, end time, Required resources, Released resources, Messages received, Messages sent. The data integrated between the running logs contains an additional column labeled “Organization”.

5 Research Design and Hypothesis

The present research work aims at developing an approach to reach inter-enterprise interoperability and to test its achievement using practices from the software engineering process. In the approach proposed in [32], interoperability requirements are specified by representing interoperability problems directly on business process models. The testing sub-process defined in this work [32] was limited to the validation of interoperability requirements but did not enable the verification because of the nature of these requirements. Four fundamental activities are identified in software process: software specification, software development, software validation and software evolution [1]. For each of the first three activities, our approach proposes to define a sub-goal and determines how to achieve it:

  1. 1.

    Software Specification. Interoperability requirements specification ensures certain characteristics: requirements are structured (i.e. understandable and identifiable) and testable.

  2. 2.

    Software Development. The structured nature of interoperability requirements will ease the logical and technical architecture definition.

  3. 3.

    Software Validation. The testability of interoperability requirements facilitates the definition of a method that enables testing the level of achievement of interoperability improvement. The objective will be to prove that the implementation of the CIS has improved the interoperability up to the desired level.

We propose a framework for the definition of a Performance Measurement System (PMS) (Fig. 1) made up, in its complete version, of cost, time and quality PIs for all the processes impacted by the implementation of the CIS. To have a comprehensive verification process, interoperability measures defined at process level will be used. Indeed, as it is explained in [11], defining measures at activity level will produce too numerous indicators and increase the PMS complexity. The framework should be understood as road-map where each state represents additional process PIs on which the verification step of the interoperability testing sub-process (Sect. 5.3) is applied using approaches defined in process mining. We assume that the PMS is constructed through a series of states where \(S_{i}\) denotes the set of indicators in the \(state_{i}\)

  • the difference \(S_{i} - S_{i-1}\) represents additional indicators defined in order to take into account new concerns defined in \(S_{i}\). Therefor \(S_{i-1}\) is always included in \(S_{i}\)

  • the directed arcs from \(S_{i-1}\) to \(S_{i}\) represents the influence of \(S_{i-1}\) to \(S_{i} - S_{i-1}\) that is also a part of \(S_{i}\).

The four states as well as their influence relationships are defined as following:

  1. 1.

    State1. The first state concerns time PIs for process instances excluding those containing loops. A loop causes a task to be executed multiple times for a given case [28, 33]. According to [34], if the maximum number of loops is high then the number of loops can greatly differs from instance to instance. In other words, the presence of loops will results in variability in the time and other process PIs between different instances of a given process. In our opinion, the origin of this variability is the random nature of reasons why process or task can fail.

  2. 2.

    State2. In the second state, the new concern is the measurement of loops in business processes. Traditional workflow systems define loops for repeating parts of the process. Tasks or sets of tasks are iterated for multiple reasons:

    • they have not yielded an expected result [35].

    • the workfow management system is configured to act automatically (i.e. pass the case back to the last performer) after a long passivity of performers [31]. Therefore, we can conclude that the time PIs of the state1 influence the failure (or loop) PIs added in state2.

  3. 3.

    State3. The process PIs added in state3 are related to time but the difference with the ones defined in state1 is the fact that they also measure the variability of time caused by loops. The group of indicators in state3 are influenced by the indicators defined in state2 because, according to [36], the quality of the process expressed in the number of failures or loops is an indicator for necessary rework. More generally, it is safe to say that quality characteristics of business process influence time (and cost).

  4. 4.

    State4. The new process PIs included in state4 are related to cost. These measures receive impact of time and failure.

Fig. 1.
figure 1

Framework for the definition of the PMS

Full size image

The rest of this paper represents the first state of the framework, the scope of the verification step is then limited to process PIs of \(state_{1}\) (i.e. time process PIs). Process mining techniques enable to automatically mines upper bounds for different key performance indicators (like waiting times, execution times etc.) of a process by taking into account both the timestamps of tasks in a log and the overall structure of the process model given as input [33, 37, 38].

5.1 Improving the Interoperability Requirements Specification

The interoperability requirements specification proposed in this work results in the definition of two categories of requirements:

  1. 1.

    Measurable Interoperability Requirements. This category concerns measurable interoperability requirements that can be verified in the testing activity. Indeed, according to [1], non-functional requirements should be written quantitatively, whenever possible, so that they can be objectively tested. The measurable interoperability requirements are metrics representing the desired interoperability level for each business process as a result of the implementation of the CIS. These measurable requirements are related three process PIs considered as sufficient to measure interoperability by [11]: the average elapsed time, average cost and percentage of failure. As explained in the beginning of Sect. 5, this study uses only the average elapsed time and will measure this indicator for all business processes impacted by the implementation of the CIS.

  2. 2.

    Non-measurable Interoperability Requirements. This category contains interoperability requirements that can be validated in the testing activity but not verified because of to their non-measurable nature. For this category, the requirements specification consists in representing directly interoperability problems in business process models, mainly in “As-is” ones. The adopted representation is based on a principle that consists of distinguishing between business activities and Non-Value-Added (NVA) activities, mainly inspired by the work done in [11]. Business activities create value in a business process. The NVA activities are defined as the components of business processes that represent efforts between partners to achieve interoperability in information exchange. According to [39], Non-value-adding work, creates no value for customer but is required in order to get the value-adding work done. One the most important principles of Business Process Reengineering (BPR) is the value-focus principle. The value-focus principle states that [40], NVA activities must be targeted for elimination in order to save time and/or money. Interoperability problems are depicted in BPMN (Business Process Model and Notation) [41] business process models. The stereotypes [42], generally used for the UML language, will be used to differentiate NVA to business activities in the BPMN process models. BPMN is more suitable for modeling collaborative processes because it helps situate the boundaries of the collaborating companies using pools. The proposed representation technique enables to overcome limitation presented in the Sect. 3.1, since the requirements will relate interoperability problems to target elements such as people, organization units and material resources which are clearly identifiable in the collaborating enterprises.

Interoperability requirements are part of non-functional requirements. Both measurable and non-measurable requirements defined in this research work are considered as non-functional requirements because of their relation to interoperability. Indeed, according to [1], non-functional requirements arise through user needs, because of budget constraints, organizational policies or the need for interoperability with other software or hardware systems.

5.2 Ease Architecture Definition

Software development is the activity where the software is designed and programmed. The requirements specification results are inputs of the design and implementation processes [1]. In order to develop solution for interoperability problems, the interoperability matrix utilizes the concepts of solution space [12, 13]. The solution space is composed of the three dimensions of the INTEROP framework. The cross of an interoperability barrier, an interoperability concern and an interoperability approach includes the set of solutions to breakdown a same interoperability barrier for a same concern and using a same approach. In order to determine correctly the solution (third dimension), there must be sufficient information to describe interoperability problems (two first dimensions). The requirements representation in process models contains the following information: tasks where interoperability problems arise and the resources (human and non-human) involved in the interoperability problems. This set of information will facilitate the design process and then improve the software development activity.

5.3 Adapt Validation Activity

In the SE process, the validation activity can be considered as a test process that can be divided into a set of test sub-processes defined to perform a specific test level (e.g. system testing, acceptance testing) or test type (e.g. usability testing, performance testing) within the context of an overall test process for a test project [43]. A test type is a group of testing activities that are focused on specific quality characteristics [43]. In the validation activity of software process, the improvements will consist on considering interoperability as a quality characteristic and developing the interoperability test type. On the basis of the recommendations given in the literature of service-based systems testing, the interoperability testing sub-process will be executed at system testing level using business process models (Sect. 3.3). The interoperability testing sub-process is divided into two steps: Verification and Validation of interoperability requirements.

Fig. 2.
figure 2

The interoperability testing sub-process

Full size image

Verification of Interoperability Requirements. The first step of the interoperability testing sub-process is the verification of measurable interoperability requirements. Each business process in the scope of the project must be executed several times to calculate its average elapsed time. As mentioned in Sect. 4, information needed for the computation of this process PI average elapsed time are the probability related to the different path and the average elapsed time of the activities in the process model. In the context of inter-enterprise collaboration these information can be obtained using the cross-organizational workflow model and the integrated event log. Applying the Framework for workflow integration based on process mining described in Sect. 4 will help to get the cross-organizational workflow model and the integrated event log. The events log analysis enable calculating process PIs that represent measurable interoperability requirements. The goal of this step is to verify if the level of interoperability is improved at the level expected from the implementation of the CIS as determined in the software specification activity. When the percentage of interoperability improvement is judged unsatisfactory then the validation step is launched in order to identify, understand and possibly resolve interoperability problems.

Validation of Interoperability Requirements. The second step of the interoperability testing sub-process is the validation of non-measurable interoperability requirements. The main input of this step (Fig. 2) is the “As-is” business process models which contain the interoperability requirements specification (Sect. 5.1). The validation is carried out by executing each process in order to verify if all the NVA activities it contains in its “As-is” version are effectively eliminated by the implementation of the CIS. The execution of a business process may reveal the presence of NVA activities. In this situation, the “As-is” business process model gives sufficient information about the interoperability problems related to the concerned NVA activities. The information will be used to fix the interoperability problems.

6 Application

The illustrative example used to demonstrate the applicability of the methodology involves a supply chain in which an interoperability investment is used to improve the quality of the collaboration. The partners involved in this collaboration are a customer (an e-commerce company), a stockist (a warehouse owner), a customs agent and the customs administration. The goal of the interoperability investment is to allow the customers to be quickly connected at a low cost and with flexibility to their partners and to the customs administration using an interoperable information system.

6.1 Interoperability Requirements Specification

The specification of both measurable and non-measurable interoperability requirements is illustrated through a goods entry (collaboration) process in which all four partners concerned by the investment participate. The goods entry process begins when the customer places an order and terminates when the stockist updates the material accounting and informs the customer that it can begin to distribute and sell the goods. For this process, the specification of the non-measurable requirements consist in representing NVA activities in the “As-is” version of the model using stereotypes (Fig. 3).

Fig. 3.
figure 3

Goods entry process- “As-is” version

Full size image

The specification of the measurable interoperability requirements for the goods entry process consist in calculating the percentage of improvement in the average elapsed time PI between “As-is” and “To-be” situations. The partners in the illustrative example provide the average elapsed time for each activity in the goods entry business process, which allows the calculation of the average elapsed time process PI of 183 min for the “As-is” situation. The “To-be” process models essentially depends on the number of identified barriers that the solution is intended to remove. In the illustrative example, the SOA-based solution was expected to remove all of the identified barriers. Therefore, to arrive at the “To-be” process model, the NVA activities are simply removed from the “As-is” model (Fig. 3). Simulation of the “To-be” process models can be used to estimate the average elapsed time process PI which has a value of 152 min. The value of the metrics representing this interoperability requirement is –17, meaning that the decrease of average elapsed time between the “As-is” and “To-be” situations for the goods entry process is expected to be of 17 percent.

6.2 Software Architecture Definition and Implementation

Consider the following activities of the goods entry process: “Transfer notification”(1), “Inform warehouse”(2), “Lead driver to warehouse”(3), “Unload the truck”(4), “Fill Material Accounting”(5), “Update Material Accounting”(6). All these activities are business ones except “Fill Material Accounting” which is a NVA activity representing a non-measurable interoperability requirement and it is hypothesized that an interoperability barrier existing somewhere in the process. Identifying software systems supporting the activities connected to the NVA activity will help in determining technological barriers between (e.g. 1 is executed using Delta which is non-inter-operable with Sage used for 5 and 6). Understanding why these systems are not inter-operable will facilitate finding solutions. Web services are an industry effort to provide platform-independent SOA using interoperable interface descriptions, protocols, and data communication [44]. In the software development activity, the decision was made to use web services architecture to implement the CIS. The main reason is that web services architecture can support the business activities while removing NVA activities identified in all the “As-is” business process models obtained in the requirements specification. The software development activity was realized using .NET Web Services implemented using the Microsoft .NET platform [45].

6.3 Interoperability Testing Sub-process Application

The interoperability testing sub-process is executed in the system testing phase after the testing of the functional requirements. The functional requirements testing allowed verifying that the implemented CIS supports the business process as described in the “To-be” models. For the goods entry process, the verification of interoperability requirements shown a reduction of 20 percent in average elapsed time, a value greater than the threshold set in the requirements specification. The validation of interoperability requirements conducted reveals that all the NVA activities identified in the “As-is” goods entry process models were removed in the implementation of the CIS. The results of verification and validation steps allow us to assert that the interoperability problems in the goods entry process were fully eliminated.

7 Conclusion

This work was aimed to improve software specification, development and validation activities of software process in projects related to interoperability. The interoperability requirements specification is based on measurable and non-measurable quality characteristics. Non-measurable requirements are defined as interoperability problems directly depicted in business process models using the concept of NVA activity. The average elapsed time process PI is used for the development of measurable interoperability requirements. It has been then demonstrated that the proposed form of interoperability requirement specification can positively impact the software development and the software validation activities. For the validation activity, the definition of a interoperability testing sub-process is made through a two-step decomposition: one step to verify measurable requirements and another to validate non-measurable ones. A perspective of this work will consist in extending the verification step to the remaining process PIs defined in [11]: average cost and percentage of failure.