Keywords

1 Introduction

The success of the interactive applications is related to the quality they provide to their end users. Among the quality attributes, usability is one of the most important quality attributes [4], because it influences in the acceptability of interactive applications [1]. Several techniques have been proposed to help improving usability into the development process of interactive applications. Most of these techniques only evaluate usability through the artifacts designed on early stages or in final version of the application [7]. However, few techniques are created to assist designers in artifacts designing aiming at improving the usability of the application [12]. Therefore, assisting designers in this process ensures that user interactions are efficient, functionally correct and error tolerant [1]. Considering designing for usability in the early stages of the development process (e.g., Design phase), can help preventing early usability problems, improving the quality of use, and reducing project costs [8].

One of the artifacts created in the early stages of the development process is the Activity Diagram (AD) [14]. AD’s shows the logic of the activities that can be carried in an interactive application. In addition, this diagram is used as complement for others artifacts into the development process (e.g., scenarios, mockups and others). Therefore, it is important to help designers create these diagrams aiming at developing an application with an acceptable level of usability.

In order to support designers in this direction, we proposed the Usability Design Reading Technique for Activity Diagrams (UDRT-AD) technique. In order to propose this technique, we considered two main principles: (1) to use procedures of reading techniques, because they guide the designers in the creation process of the artifacts [13]; and, (2) to use empirical studies to assess, improve and assist in the evolution of the technique. The UDRT-AD technique assists designers in the prevention of usability problems that may influence the quality of the final application. UDRT-AD provides a set of steps that can be followed by designers. Each step consists of four elements that support the AD modeling, and improve the usability of the final application during the modeling of that AD. Moreover, the technique has application examples to help designers with little experience in both AD modeling and usability principles.

In this paper, we present an empirical study performed that aimed at analysing if the UDRT-AD is feasible when compared to the conventional approach. To do so, we verified through this empirical study: (a) the effectiveness; (b) the time spent for the modeling process of a diagram; (c) the degree of correctness of the designed diagrams; and (d) the number of possible usability problems found through the designed diagrams and that could influence the quality of the final application.

The remainder of this paper is organized as follows: Sect. 2 present the work related to this research. Section 3 provides an overview of the UDRT-AD technique proposed. Section 4 describes the planning and the execution of the empirical study. Section 5 present the results of the study. Finally, Sect. 6 presents our conclusions and future work.

2 Related Works

One of the artifacts used in the early stages of the development process is the Activity Diagram (AD) [2]. The diagram is employed to capture the dynamics of an application, without being limited to the behavior description of a particular functionality [5]. Designing an AD, thinking in the usability of the application, can support the correctness and conformity, as well as increasing the quality of the designed application by this diagram [5, 11]. Therefore, several studies have been proposed in order to ensure usability as well as verify the completeness and correctness of these diagrams [3, 5, 14]. The following are some of the proposed methods to improve the usability of interactive applications through ADs.

Valentim et al. [14] proposed the MIT 3, which is part of a set of techniques called MIT (Model Inspection Technique for Usability Evaluation). The MIT’s aims auxiliary the usability inspection through design models. Specifically, MIT 3 evaluates the usability through activity diagrams and, to do so, it provides 14 verification items that guide the inspectors during the inspection process.

Silva e Silveira [3] proposed a set of guidelines that aim at evaluating the usability in the design stage of interactive applications, using design models of the areas of Human-Computer Interaction and Software Engineering. Among the guidelines, there are those focused on evaluating the usability through activity diagrams. The technique has 16 items that assist the identification of usability defects.

However, the cited techniques only evaluate the usability of the application through activity diagrams, in other words, when these diagrams have been modeled. Therefore, it is necessary to develop techniques to assist designers in elaborating this artifact, in order to improve the quality of the final application. These techniques should enable designers to be guided in a proactive way to design and to improve the usability of interactive applications, through this diagram. Based on this, we proposed a design technique, which will be shown below.

3 Usability Design Reading Technique for Activity Diagrams - UDRT-AD

The UDRT-AD (Usability Design Reading Technique for Activity Diagrams) technique consists of a reading technique that assists designers in the process of modeling an AD’s, aimed at improving usability of the interactive applications [11]. A reading technique is a specific type of technique that provides guidelines that can be employed for assisting in the execution of a specific task [13]. This technique aims to support (or even teach) designers (experienced or not) in building an activity diagram. To do so, the technique helps identifying the diagram elements from a textual description (scenarios or use cases); and, anticipating the usability of the final application, by providing guidelines that suggest improvements of the usability during the construction of the activities diagram.

The phases to use the UDRT-AD technique are shown in Fig. 1. In phase 1, the designers should first try to understand the problem which needs to be solved. In phase 2, the designer uses the UDRT-AD technique for extract the elements, from the textual description and designing the AD. In parallel with the diagram’s design, the designers must perform the reading of the usability guidelines presented in UDRT-AD technique. They should also verify what part of the diagram is possible to support the usability of the final application. The UDRT-AD technique has six steps that guide the designers in the designing of diagrams and in anticipation of usability. These steps are: Identification of Actors, Identification of Start Node, Identification of Activities, Grouping of Activities, Transition Activities, and Identification of the End Nodes. Each step consists of: a Heuristic that helps to identify the diagram elements (Fig. 1 element A); the representation of the heuristic Element (Fig. 1 element B); an Instruction that teaches how to insert the elements in diagram (Fig. 1 element C); and, Usability Guidelines that guides how to anticipate the application’s usability during the design of the diagram (Fig. 1 Elemento D). To assist in using the technique, each Heuristic and each Instruction has an application example (Fig. 1 Element E), showing how to extract the AD element from the textual description.

Fig. 1.
figure 1

Example of using the UDRT-AD technique

Full size image

As it was exemplified on Fig. 1 element E, the swimlanes identified in the scenario are “User” and “Application”. Then, the designers identified that the activities “Execute the Activity A” and “Perform the Activity B” are respectively related to the swimlanes “User” and “Application”. In parallel the designers used the usability guidelines to anticipate usability. When using those guidelines, designers can perceive that the grouped activity presents an error situation. The designers are instructed to create another activity that allows the user to exit the error situation.

In order to check technique performance compared with the used practice in the software companies (call in this paper of conventional approach), we conducted an empirical study comparing the two approaches (UDRT-AD and conventional approach). The empirical study is described below.

4 Empirical Study

This empirical study was conducted in order to verify the feasibility of the UDRT-AD technique, and to indicate what parts of the technique need improvements. The UDRT-AD technique was compared to a conventional approach, commonly used in software companies. In a conventional approach, first the diagrams are modeled and then, perform a usability inspection through the created diagrams. In this study, the technique applied for inspecting the diagrams was MIT 3, once it was used as a basis for defining the UDRT-AD usability guidelines.

4.1 Planning

In this step, we defined the scope of the study (preparation of the approaches), we prepared the materials which were employed (consent and characterization forms, elaboration of scenarios, instructions for the study, modeling forms, and a worksheet for the annotation of the identified discrepancies for the group that applied the conventional approach), and we selected and trained the subjects. All the activities of this phase were carried out by the study moderator and reviewed by two others researchers. Furthermore, we performed a pilot study for evaluating the study artefacts. The results of the pilot study can be found in our previous work [13].

Subjects were undergraduate student volunteers from a Systems Analysis and Design class in their 5th semester at a course on Information Systems at Federal University of Amazonas. Sixteen undergraduate students consented to participate in the study. They signed a consent form and filled out a characterization form that measured their expertise with Human–Computer Interaction (HCI) Design/Evaluation (HCI) and Software Development (SD).

The characterization data was used to classify subjects into four categories (none, low, medium or high experience). A subject is considered to have: (a) high experience, if (s)he participated in more than 5 projects/evaluations in HCI in the industry; (b) medium experience, if (s)he participated from 1 to 4 projects/evaluations in HCI in the industry; (c) low experience, if (s)he participated in at least one HCI project/evaluation in the classroom; (d) none experience, if (s)he had no prior knowledge on HCI or if (s)he had some notions acquired through readings/lectures, but without practical experience. The data on experience in software development was defined similarly. Third (HCI) and fourth (SD) columns of Table 2, presented in Sect. 5, show each subject’s categorization respectively.

To decrease the bias of participants who have more experience than others, the subjects were divided into two groups with the same number of subjects and with equivalent experience. Thus, the subjects were assigned to each group at random and the groups were balanced, according to the subjects’ experience, which was reported in the characterization form. Therefore, each group was formed by eight subjects.

4.2 Execution

Before the study, we carried out a training for all study subjects. This training aimed at explaining usability concepts and how to apply an inspection method (Heuristic Evaluation [10]). Then, the subjects were divided into two groups. Each group remained in separate rooms and there was training on each technique. For Group 01, we conducted training on the UDRT-AD technique. For Group 02, we held a training on how to model a diagram and then on how to perform an inspection using the MIT 3. Next, the study artefacts were delivered individually and we did not allow communication between the subjects.

The subjects from each group were given a scenario and, from this scenario, they performed the AD modeling. The scenario employed to model the AD was booking a hotel room. In this scenario, the user carried out the activities from logging in the application to confirming the booking of the hotel room. Furthermore, the subjects in Group 01 (UDRT-AD) received the UDRT-AD technique to assist in the modeling of the activity diagram targeting at improving the usability of the interactive application. On the other hand, subjects in Group 02 (conventional approach) received a document containing instructions, based on literature [2], so that they could consult, in case a difficulty arised during the AD modeling. After performing the modeling of the AD, subjects in Group 02, carried out a usability inspection in the modeled diagram using the MIT 3 technique. During the inspection, after detecting a usability problem using the MIT 3, the subjects wrote down the defect in the discrepancy spreadsheet.

We highlight that throughout the implementation process, the subjects performed their activities individually and received no assistance from the involved researchers.

4.3 Analysis of Results

Finally, we performed the analysis of the results. In this step, the researchers analyzed the diagrams modeled by each subject. In order to perform this process, we removed the name of the participants and a code was inserted to represent them with the goal to avoid causing bias in the analysis of the results. For the analysis of the quantitative data, we employed the following indicators: effectiveness, modeling time, correctness, and usability error prevention.

The Effectiveness indicator was calculated as follows:

$$ {\text{Effectiveness }}\left( {\text{Subject X}} \right) = \frac{{({\text{NMA}} - {\text{nAnC}}) }}{{({\text{NMA}} - {\text{nAnC}}) {\text{ + nAO}}}}, {\text{were}}: $$

  • Subject X is the reference to the study subject;

  • NMA is the number of modeled activities by Subject X in the activity diagram;

  • nAnC is the number of activities that are not in the context of the scenario employed by Subject X to model the activity diagram;

  • nAO is the number of missing activities in the diagram modeled by Subject X, i.e., activities that are present in the scenario but are not present in the diagram modeled by Subject X.

The Modeling Time indicator it was calculated in hours and represents the total time spent by each subject to carry out the modeling of the AD.

The Correctness indicator checks how correct the diagrams were modeled. We classified the defects found by researchers according to the categorization of defects by Travassos et al. [13]. This categorization has been employed to classify types of defects found by inspection techniques of design models (e.g., OORTs [13] and ActCheck [5]). In the Table 1 the five categories are presented, these categories were tailored to the specific context of activity diagrams, as suggested by Travassos et al. [13].

Table 1. Defects Categories (adapted from [13])
Full size table

Finally, the Usability Error Prevention indicator verified if the UDRT-AD technique helps preventing usability problems through the modeled diagrams. To do so, we carried out an inspection with two researchers that using the MIT 3 technique over the activity diagrams modeled by subjects in the group that used UDRT-AD to design the activity diagrams.

5 Results and Discussion

In this section, the quantitative results of the empirical study are presented. The statistical analysis was carried out using the statistical tool SPSS V. 21, and α = 0.05. The choice of statistical significance and the Mann-Whitney non parametrical statistic test [9] was motivated by the small sample size used in this study [6], as suggested by Wohlin et al. [15]. We used the boxplot graph to facilite visualization of data. The results are shown below.

5.1 Effectiveness

Table 2 shows the overall results per subject and per approach. In this table is shown the number of modeled activities (fifth column), number of activities that were not present in the textual description (sixth column), and number of missing activities (seventh column). Based on that information, we performed the effectiveness calculation for each participant (eighth column). Figure 2 (Item A) shows the boxplots graph with the distribution of effectiveness per approach.

Table 2. Results per subjects designing the activity diagram
Full size table
Fig. 2.
figure 2

Boxplots with quantitative results per group

Full size image

From Fig. 2 (Item A), we can observe that the median of the UDRT-AD Group is a little higher that the median of conventional approach Group. In addition, it is possible to see through the graph that the subjects who employed the UDRT-AD have their effectiveness distributed around the median. The results of the subjects of employing the conventional approach have a greater dispersion. We compared the two samples using the Mann-Whitney test, we found no significant differences between the two groups (p = 0,382). These results suggest that the UDRT-AD and the conventional approach offer similar levels of effectiveness when employed to design an activity diagram.

5.2 Time Modeling

Table 2 (ninth column) shows the modeling time in hours per subject. The boxplots graph with the distribution of time modeling per approach (Fig. 2 Item B) suggests that UDRT-AD Group obtained time modeling was much higher than conventional approach Group. Also, the median of the UDRT-AD Group is much higher than the median of the conventional approach Group. This was confirmed by Mann-Whitney test (p = 0,000). Therefore, we can conclude that the modeling time needed for an activity diagram to be improved in terms of usability is significantly higher using the UDRT-AD than using the conventional approach.

5.3 Correctness

Figure 2 (Item C) shows the boxplots graph comparing the distribution of correctness per approach. From Fig. 2 (Item C), we can observe that the median of the group using conventional approach is a much higher than the median of the group that used UDRT-AD technique. In other words, we found fewer defects in the diagrams of the participants who used the UDRT-AD than those who used the conventional approach. However, the Mann-Whitney statistical test pointed out that there is no significant difference among the groups (p = 0,161). Therefore, we can be concluded that there is a significant difference between the number of defects found in the modeled diagrams using UDRT-AD and the number of defects found in the modeled diagrams using the conventional approach. However, we can argue that given the small sample used in this study, it is difficult to obtain statistical significance in results obtained.

Table 3 the number and types of problems identified in the modeled diagrams per subject for each group.

Table 3. Number of defects found per subjects
Full size table

Inconsistency and Ambiguity defects were not found in any of the groups. Regarding the omission defects, Group 01 – UDRT-AD – found fewer defects (07 defects - 04 of activities and 03 of elements) than Group 02 – conventional approach – (16 defects - 12 of activities and 04 of elements). The fact that Group 01 (UDRT-AD) does not have many omission defects may be related to the fact that the UDRT-AD technique uses procedures that guided the subjects to identify the activities and elements of the diagram, from the scenario used.

Group 01 (UDRT-AD) found less Fact Incorrect defects (27 defects – all defects were related to elements) than Group 02 – conventional approach – (43 defects – 02 of activities and 41 of elements). We can observe in Table 3 that most defects were related to the diagram elements. The main reason that caused this type of defect was that the subjects did not use brackets in guard conditions (expressions that decide what the next action to be executed). In Group 02 (conventional approach), since the participants only used a guide containing instructions on how to model of the activity diagram, in almost all diagrams this type of defect was found. However, Group 01 (UDRT-AD) had one heuristic that indicated that the designer had to insert brackets in the guard conditions. Nevertheless, the subjects did not follow this heuristic.

With respect to Extraneous Information defects, in Group 02 (conventional approach) fewer defects were found (03 defects – all were defects related to activities) than in Group 01 (07 defects – 05 of activities and 02 of elements). This may be due to one of the usability guidelines (D9) from the UDRT-AD technique. The D9 guideline suggests the following: “If the user is in an error condition there should be activities that help the user to correct the error”. In an attempt to recover from an error condition, the subjects tried to create new activities, which performed the same actions of other activities. One of the suggestions is to improve the guideline so subjects do not get confused.

5.4 Usability Error Prevention

This indicator checked whether the diagram modeled by the subjects from the UDRT-AD group presented potential usability problems that could affect the usability of the application. As the subjects of the Group 02 (conventional approach) had already performed the inspection on their diagrams using the MIT 3 technique, and we wanted to assess the quality of the UDRT-AD, we decided to skip the evaluation of the diagrams designed by this group.

To do this, two researchers with a high degree of industry experience, with 8 and 10 years of experience respectively, were selected to act as the inspectors of the ADs that were modeled using UDRT-AD. To carry out the inspections, the researchers employed the MIT 3 technique. The researchers had already used MIT 3 to conduct inspections in AD of other studies. Table 4 presents the results from the inspection that was held by researchers in the diagrams modeled by the UDRT-AD Group.

Table 4. Number of possible usability problems per subjects of the Group 01 (UDRT-AD)
Full size table

From Table 4, we can observe that this group, even though they used the Usability Guidelines present in UDRT-AD, the diagrams showed a high number of potential usability problems that could affect the usability of the application. These problems, if left untreated, can cause a poor usage experience for end users. Thus, we can see that the Usability Guidelines of the UDRT-AD helped predicting some possible usability problems. However, these Usability Guidelines still need to be improved, in order to assist designers in creating the AD with the least amount of usability problems possible, thus improving the final quality of interactive applications.

6 Conclusion and Future Work

This paper presents an empirical study in order to verify the feasibility of UDRT-AD technique. The UDRT-AD technique is a reading technique that assists designers in modeling activity diagrams aiming at achieving a high degree of the usability during the development of interactive applications. By analyzing the results of the study, we can observe that the UDRT-AD showed a similar effectiveness than the conventional approach. Regarding the time modeling indicator, the group that used the conventional approach had high results than the group that used the UDRT-AD. Thus, the time for modeling an activity diagram predicting usability of the application is increased using the UDRT-AD. Regarding the correctness indicator, we can observe that the UDRT-AD group has a lower number of defects than the conventional approach. Finally, regarding usability errors prevention indicator, in the diagrams designed by subjects yet were identified potential usability problems for application. The study results also showed that the steps present in UDRT-AD technique were important for subjects during diagram design, mainly, the construction examples because these visually aided in the diagram modeling. Furthermore, the UDRT-AD assists in preventing other defects that may influence the quality of the final application. As future work, we intend to implement improvements in UDRT-AD technique and we intend to develop a new version of the UDRT-AD technique. Another objective is to perform further studies to ensure technical quality for future transfer to the software industry.