Keywords

1 Introduction

1.1 Motivation

Within the context of human-computer interaction (HCI), and especially in terms of mobile applications (“apps”), consumer behavior and use patterns strongly depend on the emotions induced with the user by interacting with an application. As a consequence of user experience, emotions are triggered and anticipated as well as retrospected rather than details of the HCI (e.g., interaction form, interaction concept, or screen design) and particular elements of the UX (e.g., usability, utility, or symbolism). Recent models of user experience models show the influence of different user experience factors to the user’s perceived emotions.

Existing research mainly focuses on positive emotions (like joy, fun, and pride) and their generation through influencing the factors of emotion [3, 11], neglecting that compared to positive emotions, negative emotions are remembered more strongly and over a longer period of time, and that they have a greater impact on the consumption behavior. Mobile apps are neither bought nor installed when the potential customer attributes negative emotions to these mobile apps. If installed and used apps cause negative emotions, users uninstall or just stop using them after a very short time due to the huge number of available alternative (free) apps. Thus, it is important to understand the causes of negative emotions from HCIs with apps. In this paper, we provide the results of an initial study on the factors of negative emotions. We have considered models of emotions from other domains and adapted them to fit the needs of software engineering by identifying the negative emotions that are most relevant to software, i.e., those emotions which are best recognized by users of software systems before, during, and after HCIs with the software systems and which are therefore measurable. We want to know what the main causes for negative emotions with respect to HCIs are.

1.2 Contribution

In this paper, we argue for the scientific community to be more concerned with the identification of the factors that evoke negative emotions. Subsequently, the software development processes should be enhanced with methods that systematically help prevent software systems to causing negative emotions. In terms of apps, this involves avoiding factors that cause negative emotions, which will help increase the apps’ acceptance and therefore positively influence consumer behavior and use patterns. Furthermore, this helps boost the appreciation of the apps’ positive aspects, which will consequently result in more positive emotions.

For this purpose, a condensed user experience model based on the works of Hassenzahl [2] and Mahlke [3] is derived. This model shows the strong interrelation between user experience factors and emotions. It is known that hedonic quality attributes directly contribute to a positive experience and emotions [1], whereas there is no research on user experience factors that lead to negative emotions to our best knowledge. We argue for a deeper research on factors that are responsible for negative emotions and present results of a first study conducted to identify drivers for negative emotions.

2 UX and its Relation to Emotions

2.1 UX Model of Hassenzahl

Hassenzahl proposed a model of user experience [2] that states how user experience evolves, what the key elements of user experience are, and identifies the interrelations of these elements. The main contribution of this model is the distinction of a product’s pragmatic and hedonic attributes, and the consequences these attributes trigger when the user is confronted with that product.

Hassenzahl’s model assumes that a product has a certain set of features. The features do not merely refer to the product’s core functionality but also include the product’s content, aesthetics, and interaction style. Persons perceiving the product’s features construct a product character based on their individual impression of the product, making this product character a subjective perception. The product character is constituted of pragmatic and hedonic product attributes. Pragmatic attributes relate to the product’s utility (relevant functionality to achieve particular goals with the help of the product) and usability (ways to access this functionality), whereas hedonic attributes emphasize an individual’s psychological well-being in terms of stimulation, identification, and evocation [2].

The assessment of a product’s appeal, based on the attributes’ relevance in a given situation, leads to emotional and behavioral consequences. The emotional consequences are the user’s satisfaction, when the product fulfills the expectations, and pleasure, when the product exceeds the expectations. However, those consequences depend on and are influenced by the situation in which the product is used (e.g., the mood of the user, stress, environmental conditions). Behavioral consequences (e.g., the user’s willingness to use the product, the amount of time spent with the product) also depend on the emotional consequences of product usage. Hence, a lack of positive emotional consequences reduces the product’s acceptance and the user’s motivation to use it.

Although Hassenzahl shows that the hedonic product attributes are the main contributors to positive emotional consequences, he emphasizes that pragmatic product attributes should not be neglected [2]. Pragmatic attributes are related to the achievement of pragmatic goals, which usually need to be fulfilled in order to also achieve hedonic goals. Hence, pragmatic attributes facilitate the achievement of hedonic goals, which in turn require the existence of hedonic attributes. Humans strive for the fulfilment of hedonic goals and perform particular activities to achieve these goals. A prerequisite for this achievement is the positive characteristic of pragmatic attributes and the successful achievement of pragmatic goals. For example, one cannot achieve the hedonic goal of feeling related to another person (identification) when the pragmatic goal of making a phone call to the other person (which provides one with the possibility to fulfil the hedonic goal) is not achieved. The successful phone call and the feeling of being related to the person who was called may lead to positive emotions as a consequence of the fulfilled pragmatic and hedonic goals.

2.2 The CUE Model

The CUE-Model (Components of User Experience) has been proposed by Mahlke and Thüring [3, 4]. According to their model, user experience has three components, namely the perception of instrumental qualities, the perception of non-instrumental qualities, and the resulting emotional reactions. Instrumental values focus on utility, which are the tasks a user wants to perform, and usability, which are the goals he wants to accomplish. Non-instrumental values contain three attributes, namely aesthetics, symbolic value, and motivational aspects. Aesthetics aim at the appearance of a product, like its color, shape, taste, or smell. Symbolic values are divided into communicative and associative aspects. Communicative aspects express individual qualities, values, and attributes, i.e., the user’s self-expressiveness. Associative aspects are related to personal memories, i.e., what the user associates with the product. Motivational aspects are the perceived ability to motivate the user, i.e., they provide stimulation. The perception of the instrumental and non-instrumental qualities through HCI influences the third component of the model, the emotional reactions.

2.3 Implications of the Models for UX Research

The CUE-model and Hassenzahl’s model are very similar. Both models show the importance of separating pragmatic (instrumental) and hedonic (non-instrumental) aspects of an HCI, and both describe their influence on the emotional and behavioral consequences. The pragmatic attributes are reflected in the instrumental values, hedonic attributes are reflected in the non-instrumental values, and the pragmatic and hedonic attributes together lead to consequences that include the user’s emotions. While this influence is conceptually highlighted, the detailed pragmatic and hedonic causes of particular emotions remain covered.

Academic research has so far focused on the positive constitution of product attributes in order to facilitate the emergence of positive emotions. To the best of our knowledge, no study has yet considered the influence of negative aspects of HCIs that consequently lead to negative emotions.

In order to research negative emotions, these emotions have to be classified, and the most prominent emotions that can be felt by users of software while and shortly after interacting with the software have to be identified.

2.4 Classification of Emotions

The study of emotions has a large history. Even the early Greeks and Romans studied the model of emotions. Initially divided into the two basic emotions of delight and reluctance (Aristoteles, Cyrenaics), more complex models have developed over the centuries. René Descartes proposed a model of six basic emotions, namely love, hatred, anxiety, pleasure, sadness and admiration [5]. Several well-known psychologists and philosophers have built their own models upon either the model of delight and reluctance (Kant, Ebbinghaus, Külpe, Freud) or defined several basic emotions (Descartes, Kast, Russell, Plutchik). To assess the emotions that arise through the usage of computer programs, especially apps, the simple model of delight and reluctance does not suffice. Delight is about looking forward to use an application, but does not necessarily imply that a user enjoys using it. Thus, more complex models need to be taken into account and the relevant emotions need to be identified.

Ekman et al. [6] derived six basic emotions from several cross-cultural studies: anger, disgust, fear, happiness, sadness, and surprise. Izard [7] identified ten basic emotions, namely anger, contempt, disgust, distress, fear, guilt, interest, joy, shame, and surprise.

A more detailed model of emotions is Plutchik’s wheel of emotions [8] that has the shape of a cone. It assumes the eight primary emotions joy, trust, fear, surprise, sadness, disgust, anger, and anticipation. The vertical dimension of the cone represents the intensity of each kind of emotion, with the emotions at the top being the most intense ones. The eight sectors of the circle each describe a different kind of primary emotion. The wheel of emotions has the coloring of a color wheel, i.e., the circle describes the relation between the emotions through supplementary (adjacent) and complementary (opposite) colors. Emotions on the opposite of the circle represent the complementary emotion whereas emotions may also be mixed up to derive dyads of emotions (e.g. joy + trust = love).

More recent research efforts have aimed at providing a finer arrangement of emotions. Parrott’s tree structure of emotions identifies six primary emotions (love, joy, surprise, anger, sadness and fear) and further refines them in secondary and tertiary emotions, resulting in a classification of about 150 emotions [9]. The HUMAINE research association defined a language called Emotion Annotation and Representation Language (EARL), which comprises 10 categories, each consisting 48 emotions [10].

The classifications by Plutchik, Parrott and HUMAINE show that emotions can strongly vary and that there is not a single emotion for the current state of mind. Comparing the primary emotions of Plutchik, Parrott, and HUMAINE results in the following five primary emotions: anger, fear, sadness, joy, and surprise. Reducing especially the negative emotions will help reduce a negative user experience. Each of those emotions can be further refined into sub-emotions (based on Plutchik, Parrott, and HUMAINE). The negative emotion anger contains the two sub-emotions frustration and dislike; fear contains nervousness and uncertainty, and sadness contains disappointment and shame.

Fear is an emotion induced by a threat perceived by humans in case of a risk in life, status, power, security or anything else valuable [6]. Sadness in contrast is characterized by feelings of despair, disadvantage, loss, helplessness, or sorrow [6]. Graham defines anger in terms of our expectations and assumptions about the world [12]. Graham states anger usually results when we are caught up “…expecting the world to be different than it is”.

2.5 Condensed UX Model

The user experience model presented in this paper was developed to facilitate the identification and reduction of negative user experience (cf. Fig. 1). Its basic principles have been taken from Mahlke’s [4], Mahlke and Thüring’s [3] and Hassenzahl’s [2] models. Emotions are influenced by the instrumental and non-instrumental characteristics of the software [2, 3]. Hassenzahl’s terms instrumental and non-instrumental characteristics have been used here, as they focus more on the characteristics of the application than Mahlke and Thüring’s terms pragmatic and hedonic. Both Hassenzahl and Mahlke consider usability and utility to be elements of instrumental qualities. Utility and usability differ in that utility provides a certain functionality, whereas usability aims at accessing this functionality.

Fig. 1.
figure 1

User experience Model based on Hassenzahl’s and Mahlke’s models

Full size image

Non-instrumental attributes aim at user satisfaction that goes “beyond the mere instrumental value” [4], i.e. it is built upon aesthetics, symbolic value, and motivational aspects. Aesthetics is defined as pleasing in appearance and pleasing to the [human] senses [13]. Focusing on the main traditional senses sight, hearing, taste, smell, and touch, it is obvious that nowadays it is not possible to perceive smell and taste through the use of smartphone applications. Like in Mahlke’s model, the factors “visual aesthetics”, “haptic quality” and “acoustic quality” are considered under the term aesthetics. The symbolic value is the value a product provides independent of its features and functionality, i.e. what does the user associate with the product (memories, prior experiences, brand image, etc.) and what does the product communicate to others (social position, status, group membership, etc.). Motivational aspects are about the ability of the product to motivate the user in performing his tasks.

Both, the instrumental and non-instrumental characteristics lead to emotions. The resulting emotions have been mentioned in the previous section and are anger, fear, sadness, joy, trust, and surprise. In contrast to Mahlke’s model, we do not consider the facets of emotional user reactions, but only the resulting emotions. This is about facilitating the identification of emotions as well as deriving relevant factors for a certain emotion. An emotion can purely be aroused by a certain instrumental characteristic (e.g. a certain function does not work as expected), a certain non-instrumental characteristic (e.g. main color of the UI is pink) or a combination of several instrumental and non-instrumental characteristics.

Instrumental and Non-instrumental characteristics have an influence on the user’s emotions whereas the influence of negative aspects of HCI leading to negative emotions as consequence of an HCI are not researched.

3 Study

The user experience Model presented in Sect. 2 shows the influence of the instrumental and non-instrumental characteristics on emotions. The goal of this study is to identify factors that are responsible for arousing negative emotions.

3.1 Setting

An iOS application for booking meeting rooms at the Fraunhofer Institute for Experimental Software Engineering (IESE) has been selected as the app to be evaluated in this study. This app was in an early stage of development, and was known to contain many negative aspects. Twelve scientific employees at Fraunhofer IESE were asked for participating in the study. They were all familiar with the process of booking meeting rooms via a web interface. Nine users participated in this study, eight of whom had neither used nor seen the application before and were first-time users. Each participant individually took part in the study, under the supervision of a moderator who sat next to the participant. Five of nine participants were male. The average age was 33.8 (max: 45, min: 25). Seven of the participants owned a smartphone when the study took place (iOS: 5, Android: 1, old Samsung OS: 1). One employee used his smartphone professionally, the rest only for private use. Six of the employees had at least one year of experience with iOS devices.

Participants received an instructional document that described the general flow of the study and the purpose of the app, illustrated by screenshots of the app. Participants were additionally given a set of tasks to perform. These tasks were introduced by a moderator and written down. The tasks and app used were constant over all participants.

The participants were asked to fill out two types of questionnaires: an application questionnaire and task questionnaires. The application questionnaire consisted of two parts. The first part inquired about demographical data and asked questions about the anticipated user experience (e.g. “What do you associate with the application?”). This part was filled out before the experiment. The second part asked the participant about his or her opinion and aroused emotions after using the app. (e.g. “What did not meet your expectations?”).

A task questionnaire was filled out after the completion of each specified task, and was intended to identify problems that occurred during the execution of the task. The task questionnaire consisted of 13 questions, which were to some extent similar or the same to those of the application questionnaire, but with a focus on the task and not on the app as a whole (e.g. “Please describe issues that arouse during task execution”). The questions covered each component of the UX model, and attempted to reveal issues that arose and functionality that the user missed. The user also chose which emotions occurred during the execution of a task from a list containing the emotions and nuances of these emotions that were discussed in Sect. 2.3 The user was allowed to mark any number of emotions (including none and several). The emotions were explained to the user to ensure a common understanding of them.

3.2 Preparation

The study has been performed in one of the meeting rooms of Fraunhofer IESE. All meeting rooms have the same appearance, so there was no special preparation necessary. The surrounding was bright and calm, subjects were not interrupted while they performed the evaluation study. Screenshots of the application were taken and a short application description has been created. The tasks to perform with the application have been chosen and the needed questionnaires were printed. The chosen tasks cover the main functionality of the application. The Moderator was sitting next to the participant. The roombooking application was installed on an iPhone 5 and each participant participated individually.

3.3 Procedure

The study itself was structured as follows: The moderator explained the process of the evaluation and provided the participant with the documents described in Sect. 3.1. The subject then familiarized him- or herself with the app by exploring the app description document and screenshots, and answered the application questionnaire, before exploring the application on his or her own for five minutes. The subject then received a set of the following four tasks to perform with the app:

  1. 1.

    Select your name from the list of employees

  2. 2.

    Book an available room tomorrow from 08:00 a.m. to 09:00 a.m.

  3. 3.

    Check if the room from Task 2 has been booked correctly.

  4. 4.

    Book an available room tomorrow from 08:30 a.m. to 09:30 a.m., add two visitors, and remove one of them again.

After a subject has performed each task, it filled in one task questionnaire. Eventually, the subject filled in the overall application questionnaire. There was no time limit for performing the tasks.

3.4 Results

The following expectations regarding the application were mentioned in the application questionnaire:

  • Fast booking of meeting rooms (7 remarks)

  • Delete a booked meeting room (3)

  • Overview of available rooms (3)

  • Faster than existing web application (2)

  • Similar functionality to web application (2)

  • Easy to use (2)

  • Conforms to Fraunhofer Corporate Identity (2).

For each task and the application itself, perceived emotions have been selected by the user. Which emotions and how often these emotions were aroused is shown in Fig. 2.

Fig. 2.
figure 2

Occurred emotions in the each task questionnaires and the application questionnaire

Full size image

The task-specific questionnaire for task 1 contained mainly positive aspects, with the statements: intuitive (2), everything fine (2), search function like in other apps (2), and simple (1), although two negative issues were mentioned: app crashes when VPN is not activated (1), and ugly person icon (1). Seven of the participants stated that they did not miss any functionality, each one mentioned the missing sorting functionality (last name vs. first name) and that the iPhone owner is not pre-selected. Summing up for task 1, seven positive issues were mentioned (5 instrumental, 2 undefined). The negative issues (4) related to both the instrumental and non-instrumental component.

For task 2, 22 negative issues have been stated, all related to usability or utility (instrumental characteristics). The end date/time of a meeting is not adapted to the start date/time (3), that it was very difficult to scroll to the desired date in the picker (1), that no recurrent meetings could be booked (1), ugly room icons(1), some problems with the wording (2), and that it is difficult to tap buttons on the top (1). In addition, the participants found the following functionality to be missing or to not work properly:

  • No feedback when meeting name has not been entered and confirm button not working (7)

  • An overview of all booked rooms is missing (1)

  • Sending of the meeting to Outlook is missing (1)

  • Available time slots of a certain room are missing (1)

  • Text field for entering the meeting name not everywhere clickable (1)

  • Booking a room for an already started meeting not possible (1)

  • Missing feedback on whether the start or end time for a meeting has been selected (1).

Like task 1, task 3 was a rather small task. Twelve negative issues were mentioned for task 3, all relating to instrumental characteristics. Participants found the name of the meeting missing in the overview of booked rooms (3), the screen looked confusing with many booked rooms (1), that details of the booked rooms were missing (1), and this functionality could not be found (1). The following functionality was mentioned as missing:

  • deletion of a meeting (2)

  • editing of a meeting (2)

  • visitors could not be removed (1)

  • only one’s own booked rooms and not all booked rooms are shown (1).

Task 4 was similar to task 2, except that two visitors should be added to the meeting. 17 negative issues were mentioned, all related to the instrumental component. Thus, problems similar to those in task 2 were experienced (missing feedback) (2), but also that there is no delete button to delete a visitor (6), it is always necessary to press “done” to select a time (2), and there was no feedback about an added visitor (1). Functionalities that were found missing:

  • using the return button on the keyboard to add a user (2)

  • visitors are not directly shown (1)

  • the missing request form for a WiFi account for visitors (1)

  • access to the address book to add visitors (1)

  • that text fields start in lowercase (1).

For the four tasks overall, 7 positive and 55 negative issues were found. Of the positive issues, 5 (71.42 %) were instrumental characteristics, while 52 (94.55 %) negative issues were instrumental.

After completing all tasks, the overall application was rated, and the participants were asked what did not meet their expectations. The most frequently given answers were: delete a booked meeting room (4), visitor announcement (3), overview of meeting rooms (3), and edit a booked meeting room (2).

All issues mentioned in the application questionnaire were negative and related to the instrumental component.

3.5 Discussion

The goal of the study was to determine the influence of the instrumental and non-instrumental characteristics on emotions, and especially whether negative aspects of an HCI also lead to negative emotions. The reported issues, expectations, and emotions make clear that this is indeed the case. Of the negative issues, 94.55 % were instrumental characteristics, and are thus related to usability and utility. This leads us to the conclusion that the instrumental component of user experience is responsible when negative emotions arise.

Considering the definitions of the emotions anger, fear, and sadness (see Sect. 2.3), anger is the emotion that appears when people are expecting the world (software) to be different than it is. Sadness is characterized by feelings of despair, disadvantage, loss, or helplessness. Fear occurs in case of risk in something valuable (life, power, control, …). The most mentioned emotion was anger where people were expecting the software to be different than it is. Considering the mentioned issues, this is due to missing functionality (e.g. “booked rooms could not be removed”) and unexpected behavior (“button to book a room inactive, but no explanation why button is inactive”). Fear and sadness occurred much less than anger. Sadness occurred due to helplessness and despair (e.g. certain functionality missing or not found, as it has been mentioned quite often). Taking the definition of fear into account, it is obvious that it occurred less often than anger, as people did not encounter a situation where they see a risk in harming them. The similar frequency of occurrences of fear and sadness cannot be explained at this point in time. There were no reasons mentioned by the participants why a certain emotion occurred. With an occurrence of 20, anger is much more perceived than fear or sadness when there is a bad usability and utility.

Tasks 2 and 4 were much more complex than task 1 and task 3, as they required more user inputs to fulfill the task. This is also represented in the mentioned emotions, that were for complex tasks much higher than for simple tasks. Figure 3 shows the relation between occurrences of negative emotions and mentioned positive/negative issues. The more negative issues have been mentioned, the more negative emotions occurred. Task 1 shows even there are negative issues present, positive issues might overcome them, resulting in no negative emotions, although task 1 was lass complex than the other tasks.

Fig. 3.
figure 3

Relation between the occurrences of perceived negative emotions and positive/negative issues.

Full size image

4 Summary and Future Work

The study illustrated in this paper provides a first glance into the relationship between user experience and negative emotions. The results show that negative emotions are experienced in case of poor usability and utility, while no conclusive evidence about the influence of negative non-instrumental characteristics was found. Once the app is fully functional and all known problems have been dealt with, this study will be repeated. This will reduce negative issues related to the instrumental component and increase the validity of future results. An interesting point is the relationship among occurring emotions and positive/negative issues. Task 1 showed a first hint, that positive issues might overcome negative issues resulting in no negative emotions. Such combinations of positive and negative issues are an interesting starting point for further research. Improvements for further studies are to ask users why certain emotions occur and what their cause is, and include positive emotions.