Keywords

1 Introduction

The banks are in a constant search for designing new ways to interact with their users [1]. Thus, thanks to technological advances, today, it is possible that clients can perform many of their transactions through channels other than the traditional bank offices [2]. The automated teller machine (ATM) is one of these new types of emerging technologies.

Nowadays, due to rapid technological advancement, ATMs acquire new features other than the provision of cash [3, 4]. However, in many cases they have severe problems that hinder its usability [5, 6]. Therefore, companies are in a constant search for tools that allow software applications to deliver a proper experience of interaction with the user [1]. However, this information is very limited when it comes to the development of ATM interfaces, compared to the large number of existing heuristics for the development of web interfaces [7].

The best-known set of heuristics to perform usability evaluations is the proposal of Nielsen [8,9,10], who establishes ten principles to problems of usability in software products. However, this tool does not adapt to the diversity of types of software that currently exist and in many cases elude own characteristics in certain types of applications [8, 10, 12].

The paper presents the development and validation of a new proposal of heuristic principles for the evaluation of ATM interfaces. The new inspection tool was based in the heuristic existing proposals, the characteristic features of the ATM and the difficulties expressed by the users.

Finally, on the basis of the comments received, in reference to the quantity and quality of perceived problems when applying the heuristic evaluation (HE) and user tests, we can conclude that the proposal has obtained positive results.

2 Methodology to Develop Usability Heuristics

In order to create an appropriate set of heuristics for evaluation of interfaces usable ATM, in general, we use the proposal of the studies of Quiñones et al. [10, 12, 13]. These focused on developing heuristics of usability for emerging information technologies. This methodology involves several iterations of nine steps (see Fig. 1).

Fig. 1.
figure 1

Methodology for the development of heuristics.

Full size image

In relation to the “Validation stage”, Granollers [10] considers heuristic evaluation is an effective method to assess user interfaces by taking the recommendations based on user-centered design (UCD) principles; nevertheless, this technique need for adapting the heuristic set to the specific features of each interactive system. For this reason, this stage begins with the application of the heuristic evaluation from the set of heuristics of usability that we have created. In addition, Smith and Mayes [15] state that “usability is now recognized as a vital determining factor in the success of any new computer system”; so, we use usability test to analyze these aspects and complement the results of the evaluation of the ATM interfaces.

2.1 Exploratory Stage

In this first step, we collect the literature related to the subject of research. Specifically, in this activity is conducted a systematic review related aspect relevant and heuristics in the domains of ATM and similar contexts.

2.2 Experimental Stage

This second stage is to compile the information provided by the experts of the domain or users, because they are constantly interacting with the system. To achieve this task was carried out interviews in search of relevant characteristics and possible problems or non-conformities.

2.3 Descriptive Stage

In this third step, all the information collected in the previous stages is processed. The activity consisted in grouping and classifying the information on two relevant topics: heuristics identified in similar domains and the list of problems established.

2.4 Correlational Stage

This fourth step is to relate previously classified information, in other words, the characteristics of the application domain and the problems identified are associated with the found heuristics.

2.5 Selection Stage

In this fifth step, we maintain, adapt and/or discard each found heuristic. At this stage, we create the new set of ATM heuristics, so that the main problems previously identified can be covered.

2.6 Specification Stage

In this sixth step, is formally described the proposal of heuristic using standard template indicated by Quiñones et al. [11]. The fields used are:

  • ID and name: Heuristic’s identifier.

  • Definition: A brief but concise definition of heuristics.

  • Explanation: Detailed explanation of heuristics.

  • Examples: Case of compliance and/or breach of heuristics.

2.7 Pre-refinement Stage

In this seventh step, be carried out an initial refinement through the application of Expert Judgment. The purpose of this activity is that the set of heuristics is easy to understand, and that there are no major inconveniences to evaluate them.

2.8 Validation Stage

In this eighth step, the set of proposed heuristics is validated through the implementation of User Test. For which, based on the set of rectified heuristics, we applied the heuristic evaluation to a set of ATM interfaces. This evaluation allows us to obtain a list of usability problems, whose resolution allows the elaboration of new interfaces designs. Finally, we complement the results through the use of usability tests with real users in both sets of interfaces.

2.9 Refinement Stage

Finally, a second update based on feedback obtained by the heuristic evaluation carried out in the previous stage is performed. These conditions allow the final set of heuristics to be useful and easy to understand by the domain experts (Figs. 2, 3).

Fig. 2.
figure 2

Detailed list of the steps made for the development of the ATM heuristics – Part 1.

Full size image
Fig. 3.
figure 3

Detailed list of the steps made for the development of the ATM heuristics – Part 2.

Full size image

3 Case Study: Usability Heuristics for the Evaluation of the ATM Interfaces

3.1 Purpose of Study

The purpose of this study was to develop and validate a set of usability heuristics for ATM that is useful and easy to use by domain experts, and thus improve the level of usability offered by ATM interfaces.

In order to make usability tests as realistic as possible, we had the collaboration of a banking entity: the BBVA Continental Bank. BBVA provided all the necessary information, as well as collaborated in the design of the final interfaces.

We established three objectives to be developed throughout the study:

  • Display a set of usability heuristics for ATM that meets the needs of users and domain-specific.

  • Propose a new design of ATM interfaces that correspond to the problems detected by the application of the heuristic principles elaborated.

  • Validate that the level of usability obtained with the group of final interfaces is significantly higher than that achieved by the initial interfaces.

3.2 Exploratory Stage

As mentioned, the amount of information regarding heuristics focused on the characteristics of an ATM is very scarce; therefore, the systematic review was addressed under three additional domains: Banking, touchscreen-based [16] and autoservices. In order to obtain an orderly inspection, two research questions were also proposed:

  • PI1: What problems and difficulties are most reported by users when using ATM interfaces?

  • PI2: What are the heuristic existing proposals in the domains of banking, touchscreen-based and autoservices?

The systematic review conducted in the databases SCOPUS, Springer, IEEE Xplore and ACM Digital Library obtained twelve relevant articles. In Table 1, we list the relevant articles, their authors and group them according to their topic of interest.

Table 1. Topics of the relevant articles.
Full size table

3.3 Experimental Stage

We conducted an interview with nine people among users and experts in HCI and the ATM domain, trying to cover the different profiles of the clients. The interview was about thirty questions and focused on validate or expand each previously identified problem. In Table 2, we list an extract of the questions asked.

Table 2. Questions asked during the interview (extract).
Full size table

In the feedback and ideas that were obtained, it was evidenced a greater emphasis in that the flow of the transactions is very long, the client only knows which are available bills when using the ATM and that the time that shows the information on screen is very short.

3.4 Descriptive Stage

We proceeded to analyze, group and classify all the information obtained in the previous stages. Additionally, we discard those problems and heuristics related only to hardware aspects. A total of twenty-five identified problems and thirty-three heuristic principles were obtained. In Table 3 is listed an extract of the complete set of heuristic.

Table 3. Heuristics identified in similar domains (extract).
Full size table

3.5 Correlational Stage

Thus, we proceeded to relate the problems identified in the literature and interviews with the found heuristics. In Table 4, we can see the lack of specific heuristics that allow to address certain problems.

Table 4. Matches among identified problems and the existing heuristics (extract).
Full size table

3.6 Selection Stage

Then, we started with the creation of the heuristics set. From Nielsen’s proposal [19], the “Help and documentation” heuristic was eliminated because an ATM system has a time limit to complete the transaction. The implementation of a section focused on the tasks of the user is not the most appropriate in these systems because it would cause a delay in the process.

In addition, five heuristics were created in order to address specific problems that were not addressed by some principle.

3.7 Specification Stage

As a result, the first revision of the heuristic proposal has a total of nineteen heuristics that address the ATM’s characteristics. Then, we formally describe each proposed heuristic:

  • PHC01: Visibility of system status.

The system should always keep the user informed of the state of the system through appropriate feedback within reasonable time.

  • PHC02: Visibility of transaction status.

The system should inform the users about the status, successful or wrong, of the operation performed or progress of tasks at moderate intervals.

  • PHC03: Visibility and clarity of the relevant elements of the system.

The system should expose the elements of greatest importance clear and highly visible way for the user.

  • PHC04: Match between system and the real world.

The system should follow the conventions of the real world, using words, phrases and concepts easy to understand that they are familiar to the user.

  • PHC05: User control and freedom.

The system should be able to offer options that the user can easily undo his actions in situations unwanted or wrong [19].

  • PHC06: Consistency between the elements of the system.

The system should maintain a similar design style, a well-organized structure, a consistency in the functionality of each element and ensure that it is consistent across the entire system [8].

  • PHC07: Adaptation to standards.

The system should follow the established design conventions, commonly used structures, and locations of elements widely known for their continued implementation [8].

  • PHC08: Error prevention.

The system should be able to prevent the occurrence of situations that cause the occurrence of errors or confusion for users [19].

  • PHC09: Recognition rather than recall.

The user should not have to remember information from one part of the dialogue to another [8, 9].

  • PHC10: Appropriate flexibility of features.

The system should be able to adapt to users with different levels of experience. It is important that the system allows you to customize common actions to accelerate tasks.

  • PHC11: Aesthetic and minimalist design.

The system should provide content to support the achievement of the objectives or goals of the user, avoiding irrelevant, unnecessary, or complex information [25].

  • PHC12: Help users recognize, diagnose, and recover from errors.

The system should be able to express the error messages in a clear language, precisely indicate the problem and suggest concrete and simple steps to recover from the inconvenience [19].

  • PHC13: Proper distribution of the content display time.

The system should ensure that the display time of the information on the screen is long enough so that users do not have any difficulty in completing the task successfully. The allocated time should vary depending on the relevance of the type of content being exposed.

  • PHC14: Correct and expected functionality.

The system should do what it promises the user. The features must be properly implemented and must offer what the user expects from them [8].

  • PHC15: Recoverability of information against failures.

The system should be able to protect the integrity and consistency of the user’s personal, private, and financial information against abrupt system failures.

  • PHC16: Visibility of exchange rates.

The system should exhibit on initial screens the availability of the types of monetary denomination.

  • PHC17: Customization in the design of the interface.

The system should allow users to customize aesthetically design the interface so that it adapts to your visual preferences.

  • PHC18: Prevention of forgetting the bank card.

The system should ensure that, after the transaction is completed, the logic of return of bank card prevents a possible oblivion.

  • PHC19: Efficiency and agility of transactions.

The system should support that the user successfully performs the desired operation in the minimum possible time. To do this, you need to minimize the steps needed to run a task, as well as the response and execution times should reach optimal levels.

3.8 Pre-refinement Stage

The refining stage began with the application of Expert Judgement to three experts on usability issues, and with extensive knowledge in the elaboration of heuristics. Table 5 lists the encountered difficulties, as well as the heuristics related to each problem.

Table 5. Problems identified by usability experts.
Full size table

Then, we proceeded to make the necessary modifications according to the feedback received. The following changes are presented:

  1. (a)

    Change of name:

    • Before: PHC02 - Visibility of transaction status.

    • Changed to: PHC02 - Visibility of the progress and final status of the transaction.

    • Before: PHC07 - Adaptation to standards.

    • Changed to: PHC07 - Compliance with standards.

    • Before: PHC10 - Appropriate flexibility of features.

    • Changed to: PHC10 - Adaptability of the functionalities to the user profile.

  2. (b)

    Complete modification of each heuristic:

    • Before: PHC13 - Proper distribution of the content display time.

    • Changed to:

      • PHC13 - Appropriate session time distribution to display content.

    • The system should ensure that the session time is long enough for users to be able to properly view the contents and successfully complete the task. The allocated time should vary depending on the relevance of the type of content being exposed.

    • Before: PHC16 - Visibility of exchange rates.

    • Changed to:

      • PHC16 - Early-stage visibility of interaction restrictions

    • The system should display in initial screens the impediments that limit the interaction of the user, avoiding unnecessary navigation that causes loss of time and discomfort in the client.

    • Before: PHC18 - Prevention of forgetting the bank card.

    • Changed to:

      • PHC18 – Prevention of the capture of the cash and bank card.

      • The system should ensure that any effort involved in the delivery, reception, return and capture of money, as well as the bank card, prevents any inconvenience that may result in the client losing his cash or card.

3.9 Validation Stage

After obtaining the set of rectified heuristics, we proceeded to perform the heuristic evaluation, which is one of the most popular inspection methods [10, 26]. In order to focus the inspection, two sets of different interfaces were evaluated, corresponding to the screens associated with the “Bank Loan” and “Cash Withdrawal” process, respectively. In both situations, the appropriate distribution of the “Main Menu” is examined.

For the first case, the set of ATM interfaces and the evaluation process was carried out by different groups of undergraduate students participating in the course Human-Computer Interaction (INF647) taught at the Pontificia Universidad Católica del Perú. In addition, this proposal holds the fulfillment of several previous design methods carried out throughout the course.

The second case corresponds to a group of interfaces belonging to the BBVA Continental (see Fig. 4) for which was obtained the corresponding support of the relevant authorities. The profile of the members who formed the team assigned to this evaluation consisted of three usability experts, with extensive experience in the creation of evaluation heuristics and three novice evaluators.

Fig. 4.
figure 4

Set of ATM interfaces of BBVA Continental.

Full size image

After applying the heuristic evaluation in both cases, the total list of identified problems was made available by the respective authorities of the BBVA Continental. Which, depending on the resources available and the technical limitations of the hardware, solved the main obstacles encountered through a process of improving the interfaces. The improvement tasks performed for the conception of the new prototype are:

  • Increase the size of the letters of conditions on the display of “Retiro Seguro”.

  • Standardize the buttons.

  • Change the “Retiro rápido” button to “Continuar con Retiro rápido”.

  • In the option “Otros retiros” place the text “Más cuentas” next to the scroll arrow of other bank accounts.

  • In the “Main Menu”, perform a redistribution of the buttons.

  • In advertising displays, add an informative tape “Procesando…”

  • The final message for the “Cash Withdrawal” process will be divided into: “Retira tu tarjeta” y “Retira tu efectivo y voucher”.

After completing the new interface design, we proceeded to perform a cross-usability test with eight collaborators. This evaluation was carried out in a controlled laboratory environment, in which each participant was exposed to the two ATM corresponding to the interfaces to be examined. The objective of the test is to capture the level of satisfaction perceived. In the Table 6, we can see the values obtained in each prototype design.

Table 6. Average value of perceived satisfaction level.
Full size table

According to the information obtained, numerical experimentation was carried out to verify if there are significant differences in the perception of the level of satisfaction perceived between the new design proposal and the current prototype.

The base hypothesis was defined that the data follow a normal distribution. It was decided to perform the test of Saphiro-Wilk due to the small amount of data (n = 8). The results obtained validated the initial hypothesis.

In addition, the T-Student test is the appropriate statistical technique for the analysis of these variables because the samples are related. Table 7 shows that the T-Student test got a less than 5% significance level. Thus, we can conclude that the final design proposal is perceived with a higher level of satisfaction than the current prototype.

Table 7. T-Student test in related samples that examines the level of satisfaction.
Full size table

3.10 Refinement Stage

Finally, we proceeded to analyze the results obtained by the heuristic evaluation. In Fig. 5 shows the results of the percentage of the number of problems identified by each heuristic in both groups.

Fig. 5.
figure 5

Total percentage of problems identified by heuristics.

Full size image

We can observe that, for the heuristics PHC10, PHC12, PHC15, PHC16 and PHC18 not associated them any problems. We also realized that there were problems mistakenly associated to the heuristics “PCH03 – Visibility and clarity of the relevant elements of the system” y “PHC07 – Compliance with standards”.

In order to validate the first condition, a survey was carried out to the usability experts who carried out the evaluation. The results obtained allowed to certify that difficulties were not associated with these heuristics because the interfaces inspected comply with the indicated guidelines. Similarly, the results validated the utility provided by each heuristic indicated.

With regard to the second condition, due to the large number of erroneously associated difficulties, we opted to remove the PHC03 heuristic and attach its information to the PHC07 heuristic.

In summary, the Table 8 exhibits the set of heuristics final refined and validated:

Table 8. A set of refined final heuristics.
Full size table

4 Conclusions and Future Works

Usability is an aspect that increasingly takes on the different software products. Therefore, there is a need to create specific evaluation methods for each software application. The usability heuristics created by Jakob Nielsen offer acceptable but very generic results and fail to cover all aspects of specific systems.

In this study we elaborate a set of usability heuristics focused on ATM software. The results of the validation consolidated that the heuristic proposal is capable of identifying characteristic and important problems in the ATM environment. However, it is necessary to note that the results could differ in more general contexts.

Also, the perception of BBVA Continental’s specialists, who collaborated in the inspection process, has been very positive about the problems identified by the heuristic evaluation and User Test. They indicate that the problems found are coherent and correspond to aspects of the ATM, as well as, they were able to identify several problems that the UX team could not find.

As a future work, we are working on enriching the process through extending the study’s approach to creating heuristics that address the perception of customer security. The interaction with ATMs located in a banking agency is different from what a public environment can offer, such as a faucet.

Additionally, the study has a bias associated with the prototype of interfaces used as a basis in the validation phase. The scheme used is focused on the design aspects of BBVA Continental and the “Cash Withdrawal” and “Bank Loan” transactions. Consequently, we want to validate if we continue to obtain optimal results for case studies corresponding to the interfaces of other financial institutions and/or different banking transactions.

Finally, the large number of heuristics presented may prove to be a demotivating aspect in their use because it’s difficult to apply in practice [11]. So, we are working on reducing its amount through multiple refinements.