1 Introduction

The Ecoknow project aims at integrating hybrid technologies in public administration as part of the effective digitization of knowledge work processes. In this context, a hybrid representation combining the declarative DCR (Dynamic Condition Response) notation [1] with textual annotations depicting the law and a simulation tool allowing to simulate the possible process executions was proposed (cf. Fig. 1). While the use of declarative notations (i.e., DCR graphs) enables flexibility and higher adaptability, their understandability is controversial especially with regards to novice users [3]. This hybrid representation (called “hybrid DCR representation”) aims to overcome the understandability limitation by offering a multi-artifact representation to help end-users to make sense of digitized models of the law.

As part of an exploratory study investigating the understandability of the hybrid DCR representation, this paper uses eye tracking to provide some first insights about the way end-users engage with the different artifacts of this hybrid representation. The outcome of this study will provide insights into the use of the DCR platform. Section 2 presents briefly the related work. Section 3 describes the research method pursued to plan and conduct the exploratory study. Section 4 unveils some first insights into the study results, and finally Sect. 5 highlights the future work and concludes the paper.

2 Related Work

The literature about hybrid process model representations can be categorized into the following groups: (a) The first group comprises hybrid representations that combine two or more notations into a single artifact. For instance, in [2] and [7] the authors combine imperative and declarative notations into a single process model. This part of the literature is beyond the scope of this paper. Instead, this work focuses on a hybrid representation that combines several artifacts. (b) The second group considers approaches that combine a graphical model with textual annotations. For instance, Pinggera et al. in [5] proposed the Literate Process Modeling technique (LiProMo). Inspired by the dual coding theory, LiProMo aims at fostering the communication at the process modeling stage by fusing textual descriptions with a process model. Likewise, Wang et al. in [8] proposed a hybrid representation that combines a process model with linked rules expressed as textual annotations. Both approaches demonstrated higher comprehension accuracy and lower mental effort. (c) The third group of the literature comprises hybrid representations that combine a graphical model with a tool that allows the execution of single traces. For instance, Zugal et al. in [10] proposed a test driven approach to support modelers in maintaining declarative process models. Hereafter, the authors introduced a tool that allows to test whether the process model complies with a set pre-defined (positive and negative) test cases. The evaluation (cf. [9]) demonstrated an increased process maintainability with reduced mental effort. Similarly, this work combines a graphical model in DCR notation with textual annotations referring to the law. In addition, the scrutinized hybrid representation includes a simulation tool that allows to simulate the execution of single traces and perceive the allowed behaviour.

3 Research Method

To investigate the factors affecting the understandability of the hybrid DCR representation, we have planned and conducted an eye tracking experiment. This section highlights the key aspects considered in the design phase of the experiment and provides insights into the measurements deployed in the analysis.

Research Question. To obtain a better understanding about the way the hybrid DCR representation is used, we formulate the following research question: “How end-users engage with the different artifacts proposed by the hybrid DCR representation?” To answer to this question we analyze the following: (a) the distribution of attention between the different artifacts, and (b) the common reading patterns seen in different groups of end-users.

Subjects and Objects. The end-users (called “subjects” in an experimental context) who took part in this study have varying levels of expertise in using DCR. They have been recruited among case-workers from Syydjurs municipality in Denmark and students/employees from Technical University of Denmark (DTU) and IT university of Copenhagen (ITU). The DCR model used in this study originates from Section §45 of the“Consolidation Act on Social Services”Footnote 1.

Design. The experiment begins with a brief training session where all the subjects receive basic guidance about the hybrid DCR representation. Throughout the experiment eight comprehension tasks are displayed sequentially. The tasks evaluate the subjects’ capacity to understand the semantics of the DCR graph as well as engage them to read the law fragments and to use the simulation. After each comprehension task, a set of questions investigating the artifacts used, the cognitive load and the subject’s emotional state are prompted. By the end of the experiment, a think aloud session is held. Finally, a post-experiment questionnaire is used to collect data about the subjects’ demographics, domain knowledge and experienceFootnote 2. For the sake of brevity, this paper will put emphasis only on the gaze data collected from the eye tracker, and the artifacts data. The rest of the data will be investigated in upcoming work. The experiment material used for this paper is available online at http://andaloussi.org/papers/DeHMiMoP2018/Material.pdf.

Fig. 1.
figure 1

A view showing the hybrid DCR representation. At the analysis phase, this view is split into 3 areas of interest, each representing a distinct artifact.

Full size image

Measures. To investigate the distribution of attention between the different artifacts, we have used the following fixation-derived measures [6]: (a) fixation count which quantifies the number of fixations on a specific area of the stimulus [4, pp. 412–415], and (b) total fixation duration which sums up the duration of all the fixations on a specific area of the stimulus [4, pp. 377–386]. Indeed, as shown in Fig. 1, the stimulus can be divided into several areas called “Areas Of Interest” (AOIs), such that each AOI corresponds to a different artifact (DCR graph, law text, simulation). Finally, we have compared the distribution of attention with the subjective artifact data used by the subjects to answer the comprehension tasks.

To investigate the common reading patterns, we have developed a new approach to analyze the transitions between the different AOIs. Given a timestamped log file containing the sequence of fixations and their corresponding AOIs, we have generated an event log and used the process mining tool DiscoFootnote 3 to discover and analyze the underlying reading pattern. More insights about the experimental settings and the measures are demonstrated in the video available online at https://youtu.be/8OsY9PYAs3I.

4 Early Results

This section provides insights about the understandability of the hybrid DCR representation. Section 4.1 investigates the distribution of attention, and Sect. 4.2 analyzes the different reading patterns.

4.1 Distribution of Attention

The data used for the analysis and the tables described in this section are available in an online spreadsheet at http://andaloussi.org/papers/DeHMiMoP2018/Analysis.xls. By looking at the total fixation duration for all participants, we have noticed that the DCR graph was the most focused artifact (duration: 881.767 s, proportion: 0.511), followed by the law text (duration: 443.251 s, proportion: 0.257), then the simulation (duration: 398.954 s, proportion: 0.231). Comparing these values with the fixation count for all participants, we have noticed a similar distribution of attention with almost similar proportions. The same observation holds with the subjective artifact data. Overall, this comparison between the artifacts shows that the graph caught most of the subjects’ attention. However, this observation does not provide enough insights into the usability of the law text and the simulation, since the different AOIs differ in size and content.

As an alternative, we have compared the subjects based on their proportions of fixation duration and fixation count on each artifact (cf. AOI artifacts Sheet). Hereafter, we have observed the following: (a) subjects with highest fixation duration and fixation count proportions on the graph (P03, P07, P09 and P10), (b) subjects with highest fixation duration and fixation count proportions on the law text (P01 and P05), and (c) subjects with highest fixation duration and fixation count proportions on the simulation (P02, P06 and P08). These observations allow distinguishing between three user profiles with varying artifacts preference (DCR graph, law text, simulation), which lead to the question: “Do the three user profiles exhibit different reading patterns?”

4.2 Reading Patterns

To investigate the reading patterns for the different user profiles, we have analyzed the transitions between the different AOIs using the process mining tool Disco. For sake of brevity, this analysis was conducted on a single question (Question 3, cf. online material). Figure 2 depicts the reading patterns for the different profiles. The activities in the model represent the different AOIs of the stimulus, the digits on the activities refer to the absolute frequency of AOI visits, the arcs between the activities indicate the transitions between the AOIs, and the digits on the arcs count the number of transitions. The arcs looping around the same activity refer to intermediate transitions to areas that are not relevant to our analysis.

Figure 2a shows that the graph profile has high abs. visit frequency on the graph and few transitions from the graph to the law text and simulation. By looking at the gaze video recordings, we have noticed that the graph profile subjects were constantly checking the graph semantics to identify the answering clues. However, since the graph is not providing clear answering clues to all the questions, some subjects have used the law text and the simulation to complement their understanding. Figure 2b shows that the simulation profile has balanced abs. visit frequency on the graph and the simulation, and more transitions between the graph and simulation compared to the transitions between the graph and the law text. By looking at the gaze video recordings, we have noticed that the simulation profile subjects were constantly checking the DCR relations surrounding the activities targeted by the simulation, which explains the high frequency of transitions between the graph and simulation AOIs. Finally, Fig. 2c shows that the law text profile has two thirds of the abs. visit frequency on the graph while the remaining is on the law text. By looking at the gaze video recordings, we have noticed that the subjects from this profile have not used the simulation, thus, the fixations on the simulation AOI for this profile were ignored. Hereafter, frequent transitions were between the graph and the law text. This observations demonstrate that each user profile has a distinct reading pattern.

Fig. 2.
figure 2

Reading patterns for the different user profiles. Higher resolution at http://andaloussi.org/papers/DeHMiMoP2018/userprofiles.pdf

Full size image

5 Conclusion and Future Work

This paper describes the approach used to investigate the understandability of the hybrid DCR representation. The analysis depicts the distribution of attention between the different artifacts, and shows that each user profile exhibits a different reading pattern. It has to be noted that the reported results are subject to limitations due to the small number of subjects, and the use of domain knowledge by some subjects (i.e., case workers from the municipality) to answer to some of the experiment questions.

As future work, the user interactions and the verbal data transcribed from the think aloud will be used to explain the transitions between the different artifacts, and to identify the circumstance of using each of them. Moreover, the verbal data will be analyzed to spot the typical challenges faced by the subjects, and the questionnaire data will be used to measure the comprehension accuracy. Finally, the subjects’ cognitive load and emotional reactions will be examined using the pupil data and the Galvanic Skin Response (GSR) data.