Usability, Quality in Use and the Model of Quality Characteristics

Abstract

In this paper, a history of usability concept is reviewed including Shackel and Richardson, Nielsen, and ISO standards to show how the usability is located among relevant quality characteristics. Secondly, the importance of subjective quality is emphasized in relation to the usability. Thirdly, the concept of quality in use is considered in relation to the usability. Finally, a new scheme on quality characteristics is presented.

1 Introduction

The UX is now a buzzword. But the concept of usability is still very important even though the connotation of UX is much wider than usability. In this paper, the author presents the historical review of usability concept, the emphasis on subjective quality, the difference between the usability and the quality in use are discussed and finally a new scheme on quality characteristics is presented.

2 Historical Review of Usability

One of the important characteristics of artifacts that constitute the everyday experience of the user is the usability. Artifacts are made to be used. All the hardware, the software and the humanware (i.e. services) are used to expand the range of our experience, thus the ability for use, i.e. the usability is very much important. Artifacts that are difficult to use will have a little meaning and will lead to the dissatisfaction.

2.1 Shackel and Richardson

In academia, the concept of usability was first systematically defined by Shackel and Richardson (1991). They listed up three positive aspects of artifacts; utility, usability and likeability, to be important. The utility means that the artifact will do what is needed functionally. In other words, it is the functionality that matches to the users’ need. The usability means the degree of the success that the user can work with the artifact. The success can be regarded as the same with the goal achievement. The likeability, a coinage by authors, is similar to the subjective feeling of suitability, thus will lead to the satisfaction. On the other hand, there is a negative aspect, the cost, including the initial cost and the running cost. The balance between the sum of utility, usability and likeability and the cost will affect the degree of acceptance, or the acceptability. If the former is equal to or larger than the latter, the artifact will be accepted and be purchased and used.

The significance of their model lies in that the usability is regarded as one of the important aspects of the artifact. But they didn’t specify the relative importance among utility, usability and likeability. In other words, if an artifact may have a high degree of utility and likeability and the sum of the two will exceed the cost, there is a question if the artifact will be accepted even though it has a low level of usability. There must be a kind of absolute threshold for each of utility, usability, likeability and cost. Furthermore, not such other characteristics as performance, safety, reliability, compatibility, etc. than utility, usability, likeability and cost are not included. There is a question whether utility, usability, likeability and cost are more important than performance, safety, reliability, compatibility, etc. or not.

2.2 Nielsen

Following Shackel and Richardson, Nielsen (1993) proposed a hierarchical model of acceptability including the usability. In his model, the influence of Shackel and Richardson can be seen. At the top, the system acceptability is located that is split into the social acceptability and the practical acceptability. The latter consists of cost, compatibility, maintenance, reliability, safety and usefulness. The usefulness is composed of the utility and the usability where the latter is further divided into the sub-characteristics such as learnability, efficiency, memorability, errors and satisfaction.

His model is more acceptable than the one proposed by Shackel and Richardson because the structure of quality characteristics are more systematically described and the location of usability is clearly specified. But we’ll have to take care that Nielsen is also the one who proposed the heuristic evaluation method. In other words, learnability and other sub-characteristics below the usability are focal points when that method is applied for evaluating the usability of artifacts. In other words, the learnability, for example, means having less problems regarding the learning. Likewise, with the exception of satisfaction, all sub-characteristics under the usability are proposed for detecting the usability problems. That is, these sub-characteristics are directing towards the zero level from the negative (minus) level. On the contrary, the utility that can be presumed as consisting of the functionality and the performance is directing towards the plus zone from the zero level because having some functionalities or higher performance can be accepted by users positively.

This reflects the situation of the usability engineering in 80 s and 90 s when managers and engineers were directed more to the utility than to the usability. Furthermore, there is another problem that the components of usability are not systematically chosen and do not cover all relevant characteristics. For example, the ease of cognition including the visual size of the target, the contrast of the target against the background, etc. is not included.

2.3 ISO9241-11

In 1998, ISO9241-11 was standardized. In this standard, the definition of usability is the “Extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use” where the effectiveness is defined as “Accuracy and completeness with which users achieve specified goals”, the efficiency is defined as “Resources expended in relation to the accuracy and completeness with which users achieve goals” and the satisfaction is defined as “Freedom from discomfort, and positive attitudes towards the use of the product”. This definition was quite influential in TC159 (Ergonomics) and was referred later in such standards as ISO13407:1999, ISO18529:2000, ISO16982:2002, ISO18152:2003, ISO9241-210:2010 and ISO20282 series with minor changes in some cases.

In the Annex B of ISO9241-11, there is a list of measures for overall usability, desired properties of the product, and some others, with the measures of effectiveness that is mostly the number or the percentage, measures of efficiency as the time and measures of satisfaction as the rating scale. This list shows that the effectiveness and the efficiency as usability components can be objectively measured while the satisfaction can be subjectively measured.

According to Bevan (2001), the origin of the concept of usability in this standard was defined at ISO TC159/SC4/WG5 meeting in 1988 as “the degree to which specified users can achieve specified goals in a particular environment effectively, efficiently, comfortably and in an acceptable manner” and “the word satisfaction was introduced for simplifying the definition as ‘freedom from discomfort and positive attitudes towards the use of the product’ to have essentially the same meaning as the phrase ‘comfortably and in an acceptable manner’”. But the author cannot understand the reason why the satisfaction was included as a part of usability even though it is a very important aspect. It could have been included in the model as an independent characteristic as in the case of Shackel and Richadson’s likeability.

The model describes the dynamic process on how the usability and its measures can be located. But we will have to be careful that usability measures are not mutually exclusive and collectively exhaustive (MECE) with each other. Especially, they are not independent with each other. Firstly, the efficiency cannot be measured when the goal was not achieved, hence the efficiency is dependent on the effectiveness. Secondly, the satisfaction will be experienced when the effectiveness and the efficiency are satisfactory and furthermore, it will be influenced by other such characteristics as reliability, safety, beauty, etc. hence the satisfaction is dependent on all these characteristics. Thus the author has been using only the effectiveness and the efficiency as the measures of usability.

2.4 Kurosu-1

Within the scope of ISO9241-11, Kurosu (2005) proposed the model of goal achievement as in Fig. 1. This model describes the effectiveness and the efficiency in the context of the goal achievement.

Fig. 1.

Model of goal achievement by Kurosu (2005)

The dotted line ends up on the way to the goal and this suggests the occurrence of the error or the user being puzzled. It means that the use of an artifact is ineffective in this case. The dashed line reaches the goal and represents the effectiveness, but it took a winding path thus is inefficient. It suggests that there was a trial and error. The straight line that reaches the goal in the shortest path means it is effective and efficient.

Anyway, this figural representation only describes the usability concept proposed in ISO9241-11. No other such quality characteristics as reliability, safety, compatibility are not included here.

2.5 ISO9241-210

ISO13407 that adopted the definition of usability of ISO9241-11 was standardized in 1999 and focused on the human centered design. It was then revised into ISO9241-210 in 2010. The definition of usability in ISO9241-210 (2010) is expanded to include the “system, product or service” from that of ISO9241-11 that was applied only to the product. In other words, ISO9241-210 covers almost all kinds of the artifact.

3 Subjective Quality Characteristics

Although the usability is very important for the artifact, some researchers also pointed out the importance of subjective quality. As an example, we saw that the likeability was juxtaposed with the usability by Shackel and Richardson in a similar sense to the satisfaction. We also found that the satisfaction was located as a component of usability in the concept structure of Nielsen and ISO9241-11. The question here is whether such subjective quality as the satisfaction be independent to the usability or be included in the usability.

3.1 Jordan

Jordan (1998, 2000) proposed a three-layered hierarchical model composed of functionality, usability and pleasure. According to Jordan, the functionality is the fundamental characteristics for a product. But the usability is also important in order for the function to be used (effectively and efficiently). His idea is more than that. He put the pleasure atop of functionality and usability, because it makes the product attractive. Then he differentiated four types of pleasure, namely, physio-pleasure, socio-pleasure, psycho-pleasure and ideo-pleasure.

The pleasure as the subjective quality characteristics is similar to the notion of satisfaction in the ideas of Nielsen and ISO9241-11, but is different from them in the sense that it is differentiated from the usability and is an independent concept. In this sense, his idea is more similar to that of likeability by Shackel and Richardson, but is more marked as being positioned at the top of other quality characteristics.

3.2 Hassenzahl

A clear differentiation between the objective quality characteristics and the subjective quality characteristics was done by Hassenzahl (2003). Using his terminology, he distinguished pragmatic attributes from hedonic attributes. Hedonic attributes is his unique terminology but has something common to the subjective quality characteristics.

3.3 Kurosu-2

For the purpose of integrating the concept of Nielsen and ISO9241-11 and clarifying the conceptual location of satisfaction, Kurosu (2006) proposed a model in Fig. 2. There are several ideas embedded in this figure.

Fig. 2.

A model of quality characteristics proposed by Kurosu (2006)

• (a) The small usability consisting of the ease of cognition and the ease of operation is a part of the big usability.

• (b) The big usability includes the utility as well as the small usability.

• (c) The concept of (big) usability has two sub-concepts: effectiveness and efficiency.

• (d) Unlike ISO9241-11, the satisfaction is located far above of all relevant quality characteristics.

• (e) There are objective quality characteristics such as reliability, cost, safety, compatibility and maintenance as well as the usability.

• (f) On the other hand, there are subjective quality characteristics such as pleasure, joy, beauty, attachment, (matching for) motivation, and (matching for) value.

• (g) All these quality characteristics are put together to the satisfaction.

3.4 Satisfaction

According to OED (third edition), the satisfaction is defined as “The action of gratifying (an appetite or desire) to the full, or of contenting (a person) by the fulfilment of a desire or the supply of a want. The fact of having been thus gratified or contented”. The important keyword here is “full”. In other words, there is a certain mental space to be filled in the human need or want. People recognize something as attractive when it seems to fill their need of want. And they tend to try to do something to get it so that the space will be filled. That is the motivation mechanism of the human being. The room to be filled could be in terms of many quality characteristics including both of objective and subjective ones, i.e. usability, reliability etc. and novelty, scarcity and beauty and cuteness.

The mechanism of need-fulfillment is rather the multiplication than the addition. Take an example of objective quality characteristics and subjective quality characteristics. In the additive model, the lack on one side can be filled by other side so that the sum of the two will exceed the threshold for acceptance. But the fact is not just so. Because the lack of subjective quality characteristics, for example, cannot be filled by the high level of objective quality characteristics. Instead, the mechanism is more of the multiplication. The low level (e.g. 0.3 where 0<=level<=1) on one side cannot be supplement by the high level (e.g. 0.8) on the other side, thus 0.3 × 0.8 = 0.24. Even if it is not the simple multiplication, the minimum rule can be applied instead to give the result of 0.3 = minimum (0.3, 0.8). This kind of logic can be viewed in Kano’s theory of attractive quality.

3.5 Kano’s Theory of Attractive Quality

Taking the satisfaction as a dependent variable, Kano et al. (1984) distinguished the attractive quality and must-be quality in relation to the needs fulfillment as an independent variable. Must-be quality will give the dissatisfaction to users when it is not fulfilled. Even when it is fulfilled, it doesn’t give users a high level of satisfaction. On the contrary, the attractive quality will be accepted even when it is not fulfilling the user needs, but when its degree of fulfillment grows it will give the user an excitement and satisfaction.

A typical example is that usability, reliability and safety as well as ordinary functionalities are must-be qualities while new functionality and good design can be perceived as attractive.

For this reason, managers, planners, engineers and designers tend to focus more on the attractive quality than on the must-be quality. But the important point is that the attractive quality can be attractive only when must-be quality as a fundamental is fulfilled. The attractive quality without any considerations on the must-be quality is a quasi-attractiveness.

3.6 Usability and Utility

A similar relationship between attractive quality and must-be quality can be found between utility and usability. The relation between utility and usability is not an addition but a multiplication. That is, the lack of usability cannot be compensated by the utility. Thus, we will have to build up a stable usability when we are developing a new functionality and improving the performance.

4 Quality in Use

Although Kurosu (2006) dealt with many quality characteristics, the artifact’s quality and the quality in use were not clearly separated in his model. The artifact’s quality is the quality characteristics of the artifact itself and can be measured without considerations on the user’s specific traits and the contextual information of the actual use of the artifact. In terms of the quality of software, ISO/IEC9126-1:1991 made a distinction between the internal quality and the external quality. The Internal quality is defined as “the totality of characteristics of the software product from an internal view and is measured and evaluated against the internal quality requirements”. And the external quality is “the quality when the software is executed, which is typically measured and evaluated while testing in a simulated environment with simulated data using external metrics”. This notion of the quality of software can be expanded to all kinds of artifact.

On the other hand, the quality in use is “the user’s view of the quality of the software product when it is used in a specific environment and a specific context of use” according to the standard.

The key difference between the artifact’s quality and the quality in use is that the former is the quality of the artifact TO BE used while the quality in use is the quality of the artifact DURING the use. Conceptually this relationship can be described as

Quality in Use = f (Quality of the Artifact, User, Context)

where the Quality of the Artifact is the sum of the internal quality and the external quality, and the Context includes the environment and the situation. And this relationship is important when we think of the UX.

4.1 ISO/IEC9126-1

ISO/IEC9126-1 was first standardized in 1991 based on the thorough study of (objective) quality characteristics. While ISO9241-11 was standardized by TC159 of ISO on the ergonomics, ISO/IEC9126-1 was standardized by JTC1 (Joint Technical Committee 1) of ISO on the information technology to “develop worldwide Information and Communication Technology (ICT) standards for business and consumer applications”. Because of this reason, ISO/IEC9126-1 was standardized in terms of the software and was not intended to be applied to the hardware and the humanware. But the author thinks that the fundamental idea of this standard can be applied to all kinds of artifacts.

In this standard, there are functionality, reliability usability, efficiency, maintainability and portability included as quality characteristics each of which has a list of sub-quality characteristics on the side of the internal and external quality (left) and there are effectiveness, productivity, safety and satisfaction on the side of the quality in use (right).

An interesting point in comparison with ISO9241-11 is that the sub-concepts of usability in ISO9241-11, i.e. the effectiveness, the efficiency and the satisfaction are split into the left side and the right side. Furthermore, the productivity on the right side is a generic concept and will be affected by the effectiveness (and possibly by the efficiency too).

4.2 ISO/IEC25010

ISO/IEC9126-1 was abolished and was renewed into ISO/IEC25010 in 2011. The quality model was changed. There are many changes from ISO/IEC9126-1 among which major ones are that the title of the left side was changed from “internal and external quality” to “system/software product quality” and that all the sub quality characteristics of usability in ISO9241-11 were moved to the right (quality in use). But it’s quite confusing that the sub quality characteristics of usability, i.e. effectiveness, efficiency and satisfaction, were all moved to the side of the quality in use even though the usability is still located on the left side, i.e. the product quality.

5 A Model of Quality Characteristics

5.1 Kurosu-3

Kurosu (2014) proposed his latest model as in Fig. 3 based on his previous model in Fig. 2 and the some concepts of ISO/IEC25010. This figure contains such ideas as follows:

Fig. 3.

Quality model of Kurosu (2014)

• (a) There are the artifact quality (the product quality in ISO/IEC25010) and the quality in use. Hence the usability is different from the quality in use.

• (b) There are objective quality characteristics and subjective quality characteristics.

• (c) As a result, there are objective artifact quality and subjective artifact quality on the left side and objective quality in use and subjective quality in use on the right side.

• (d) Objective artifact quality includes those quality characteristics that were included in the list of ISO/IEC25010. In Fig. 3, all the sub-characteristics are suppressed for the purpose of simplicity regarding functionality, performance, reliability, safety, compatibility, cost, and maintainability. For the sub-characteristics on these quality characteristics, ISO/IEC25010 can be referred.

• (e) Novelty and scarcity were added because they are objective but can be categorized as the part of attractiveness.

• (f) Subjective artifact quality is the attractiveness that includes beauty and cuteness as well as novelty and scarcity.

• (g) Objective quality in use consists of the productivity and the freedom from risk where the former includes effectiveness and efficiency. In terms of the position of productivity, the author thinks it is more reasonable to summarize all relevant sub-quality characteristics into it.

• (h) Subjective quality in use is the satisfaction and also is the meaningfulness. These two characteristics are almost identical because something meaningful will bring the satisfaction and the artifact that satisfies users will be regarded as meaningful.

• (i) Under the subjective quality in use, such Kansei quality characteristics as joyfulness, pleasure, and delightfulness and some more are included.

• (j) Objective artifact quality will influence the objective quality in use and the subjective quality in use.

• (k) Subjective artifact quality will influence the subjective quality in use.

• (l) Furthermore, objective quality in use will influence subjective quality in use. Thus the satisfaction (meaningfulness) can be regarded as the utmost quality characteristic.

5.2 Relationship to the UX

The concept of UX has two important aspects; the temporal and longitudinal viewpoint and the inclusiveness of subjective aspects. Regarding the latter, the quality in use that is dependent to the artifact’s quality is the key to the UX. Because the quality in use includes both objective quality characteristics and subjective quality characteristics, the quality in use will be the basis for understanding the UX.

6 Discussion

Based on previous ideas on the usability and other quality characteristics, the author presented a conceptual model of the quality characteristics (Fig. 3) including the usability as a part of the artifact’s quality and the quality in use. The model also includes both the objective quality characteristics and the subjective quality characteristics including the satisfaction, so that it will serve as the basis of the discussion on UX.