Keywords

1 Introduction

With the rapid development of technology, the functions of electric cooker become more and more, and the operation is more and more complex, then many ergonomics problems, such as the operation becomes hard, function definition is not clear and so on, come along. This violates the user’s pursuit of convenient, simple and efficient, it is a great strike to the user’s interest, also hinders the development of intelligent electric cooker.

In view of the above problems, it is necessary to make a scientific and reasonable evaluation on display interface for intelligent electric cooker, thus can provide a reference for the design and redesign of intelligent electric cooker, and improve human-computer interaction.

2 Determination of the Evaluation Method

The usability evaluation is one of the main methods of evaluation for display interface. There are many traditional usability evaluation methods, such as heuristic evaluation, usability testing, cognitive walk-through, action analysis, structured and unstructured interview, and questionnaire survey model. These methods have been widely applied in many evaluation processes, suitable for variety of interface design and development, and each has its benefits and drawbacks1.

With the development of cognitive science, physiology and psychology, and so on, some scholars begin to use the method of cognitive psychophysiology such as eye movement and electroencephalograph (EEG) technology, to research the usability2.

This study first proposes to use a combined means of usability testing and EEG technology to evaluate display interface of electric cookers.

This method mainly includes three steps:

  1. 1.

    Determine test plan: It mainly includes establishing the index system, designing the experiment tasks and settings the experimental process.

  2. 2.

    Evaluation process: Evaluate the evaluation objects according to the test plan.

  3. 3.

    Analyze the results and write reports: Analyze and arrange the results of test, and write related reports.

3 Establishing the Evaluation Index System

The index of the usability is the evaluation index that reflect the degree of products available for related users, in the usability research and practice [3]. According to the view of International Standardization Organization (ISO) and experts in the field that usability includes three main aspects, which is efficiency, learnability and satisfaction [4, 5, 6, 7, 8], The index system of intelligent electric cooker display interface, are constructed, through a questionnaire survey and combining with the existing research results [9], besides, the characteristics of the three chosen sample electric cooker to be tested are also taken into account. The index system is shown in Table 1, which O stands for overall goals, \( {\text{O}}_{\text{i}} (i = 1, 2, \cdots , 7) \) stands for sub-goals, \( {\text{U}}_{\text{k}} \) stands for Index layer.

Table 1. The index system of intelligent electric cooker display interface usability evaluation
Full size table

4 Determination of the Weight for Each Evaluation Index

4.1 G1 Method

This study use the G1 method to determine the weight for each evaluation index of intelligent electric cooker. G1 method is an improved method which roots in AHP method and overcomes the defect of the AHP method, put forward by Northeastern University professor Guo Yajun. The basic principle of this method is sorting all the index of the same index layer first, according to certain evaluation criteria. Then assign the importance of adjacent index quantitatively according to the identified method. The Weight Factors of each index in the same index layer can be got through calculating the results of sorting and assignment [10, 11]. For example, if the evaluation index set of one index layer is \( \left\{ {u_{1} ,u_{2} , \ldots u_{n} } \right\} \), then the steps of the method are:

Sorting the importance of the index.

If the importance of index \( \left\{ {u_{i} } \right\} \) is greater (or not less) than index \( \left\{ {u_{j} } \right\} \), according to some evaluation criteria, marked as \( \{ u_{i} \succ u_{j} \} \).

Experts in the related field are asked to choose the most important index (only one) in the evaluation index set according to some evaluation criteria. Then choose the most important index (only one) in the rest index according to the same evaluation criteria. After n times of choosing, a sole relationship of importance is determined as formula (1):

$$ \{ u_{ 1} \succ u_{ 2} \succ \cdots \succ u_{n} \} $$
(1)

Assigning \( u_{k - 1} \) and \( u_{k} \) through comparing and judging the importance of them.

If the ratio of the importance of \( u_{k - 1} \) and \( u_{k} \), namely \( w_{k - 1} / w_{k} \), is as formula (2) shown, according to the judgement of experts.

$$ w_{k - 1} / w_{k} = r_{k} ,k = n, n - 1, \cdots , 3, 2 $$
(2)

The assignment of \( r_{k} \) can refer to Table 2.

Table 2. Reference values of \( r_{k} \)
Full size table

Calculating the weight coefficient \( w_{k} \).

The weight coefficient \( w_{i} \) for index \( u_{i} \) can be calculated through formulas (3) and (4), according \( {\text{to}}\,r_{k} \), then the weight coefficients of all index in the index set \( \{ u_{ 1} , u_{ 2} , \cdots , u_{n} \} \) can be got.

$$ w_{n} = (1 + \sum\limits_{k = 2}^{n} {\prod\limits_{i = k}^{n} {r_{i} } } )^{ - 1} $$
(3)
$$ w_{k - 1} = r_{k} w_{k} ,k = n, n - 1, \cdots , 3, 2 $$
(4)

When the number of experts is more than 16, the weight coefficient of the evaluation objects changes to be stabilized, that is to say the current weight coefficient is reliable [12]. So this study hired 20 experts, considering the actual situation. The 20 experts gave their own judgment on the same issue at the same time.

4.2 The Results and Analysis of Weight Coefficient

According to the result of the research and formulas 3 and 4, it is easy to obtain the weight of each index, as follows:

$$ \{ W_{{_{{O_{1} }} }} ,W_{{o_{2} }} ,W_{{o_{3} }} \} \, = \, \{ 0. 3 2, \, 0. 3 3, \, 0. 3 5\} , $$
$$ \{ W_{{o_{11} }} ,W_{{o_{12} }} \ldots W_{{o_{14} }} \} \, = \, \{ 0. 2 4, \, 0. 2 7, \, 0. 2 6, \, 0. 2 3\} , $$
$$ \{ W_{{o_{21} }} ,W_{{o_{22} }} ,W_{{o_{23} }} \} \, = \, \{ 0. 3 4, \, 0. 3 1, \, 0. 3 5\} , $$
$$ \{ W_{{o_{31} }} ,W_{{o_{32} }} ,W_{{o_{33} }} \} \, = \, \{ 0. 3 2, \, 0. 3 3, \, 0. 3 5\} , $$
$$ \{ W_{{u_{11} }} ,W_{{u_{12} }} ,W_{{u_{13} }} \} \, = \, \{ 0. 3 6, \, 0. 3 3, \, 0. 3 1\} , $$
$$ \{ W_{{o_{21} }} ,W_{{o_{22} }} \ldots W_{{o_{27} }} \} \, = \, \{ 0. 1 4, \, 0. 1 5, \, 0. 1 6, \, 0. 1 4, \, 0. 1 1, \, 0. 1 1, \, 0. 1 6\} , $$
$$ \{ W_{{u_{31} }} ,W_{{u_{32} }} \} \, = \, \{ 0. 5 2, \, 0. 4 8\} , $$
$$ \{ W_{{u_{41} }} ,W_{{u_{42} }} \} \, = \, \{ 0. 4 6, \, 0. 5 4\} , $$
$$ \{ W_{{u_{51} }} ,W_{{u_{52} }} \ldots W_{{u_{56} }} \} \, = \, \{ 0. 1 5, \, 0. 1 7, \, 0. 1 4, \, 0. 2 1, \, 0. 20, \, 0. 1 3\} , $$
$$ \{ W_{{u_{61} }} ,W_{{u_{62} }} \} \, = \, \{ 0. 4 9, \, 0. 5 1\} , $$
$$ \{ W_{{u_{71} }} ,W_{{u_{72} }} ,W_{{u_{73} }} \} \, = \, \{ 0. 3 6, \, 0. 3 4, \, 0. 30\} . $$

5 Evaluation Case

5.1 Experiment Device

The experiment proceeded in the usability lab, and the UX office system provided a whole-process monitor. The EEG data is provided by NeuroEdu device. NeuroEdu adopts portable brain wave device as the front-end EEG measuring equipment which can capture the EEG data and other psychological state parameters of subjects. The equipment is very easy to wear, to use, safe and comfortable, stable and reliable at the same time. The main indicators to be measured is sentiment index and caution index. “Caution index” indicates the current “caution indicator” or “concentration level” of users, and reflects the level of users’ concentration. The mental status, such as upset, trance, inattention or anxiety will reduce the value of “caution index”. “Sentiment index” parameter indicates the current “tensity” of users. “Caution index” and “sentiment index” indicate the users’ caution index level and sentiment index level with a definite value among 1-100. If the value is higher than 60, then it indicates that the subject is in a high level of concentration and tension.

5.2 Experiment Evaluation Team

The evaluation team is composed of conductor and several reviewers. The conductor leads the reviewers to perform the pre-set task and evaluate the parameters that need to be evaluated. Beside this, the conductor needs to record the process of the experiment also.

The reviewers most are consumers. They can be common consumers or specialists with professional experience. The common consumers can put forward their subjective feelings or estimate. And the specialists can give some professional advises.

Nielsen and Landauer find a functional relationship between the number of usability problems and the number of participants. When there are more than 12 participants, almost all problems could be found13. So in this experiment, the number of reviewer is 12.

5.3 Evaluation Objects

Considering the characteristics of intelligent electric cooker, this study selects 3 typical electric cookers, marked as sample 1, sample 2, and sample 3. The buttons on sample 1 and sample 2 are physical buttons, and the buttons on sample 3 are touch buttons. Sample 1 has 21 adjustable programs. Sample 2 has 20 adjustable programs, and sample 3 has 12 adjustable programs. Since the versatility and complexity of display interface for intelligent electric cooker, it isn’t scientific to evaluate on the whole. Thus, according to the analysis of display interface for intelligent electric cooker, this study will evaluate from 10 aspects, such as the display interface layout, information readability, etc. As shown in Table 1.

5.4 Evaluation Process

This study uses a method that combined usability test and EEG to evaluate display interface of electric cookers. The evaluation process can divide into 4 stages.

  1. 1.

    Preparatory stage. In the preparatory stage, the tester need to prepare 3 intelligent electric cookers and set experiment tasks. According to people`s habits and the consideration of the characteristics of each selected electric cooker, this study chooses three typical tasks and one open task for each reviewer. Typical tasks are stewing, cooking congee, and reservation timing.

  2. 2.

    Reviewers recruit stage. The result of experiment has a great relationship with reviewers, so the reviewers recruit is very important. The reviewer must have a certain discernment and experience of using intelligent electronic device.

  3. 3.

    Evaluation stage. This stage is the core of the whole evaluation. It is divided into three parts. The first part is a preparation for the next parts. After the subjects wear the NeuroEdu hair band, the conductor open the video monitoring equipment. In the second part, the conductor introduces the display interface of the three intelligent electric cookers, and explains the evaluation index system to the reviewers. If the reviewers have any questions, the conductor has a responsibility to explain and demonstrate. In the third part, the reviewers perform the evaluation under the conductor’s guidance. The reviewers operate the tasks on the usability testing record chart one by one, then grade the three-class index on the user experience score chart. The scale of marks is a five-grade marking system. At the same time, the conductor records the problems during the evaluation and the number that error appears, etc.

  4. 4.

    Result output and analysis stage. After the evaluation, the tester process the evaluation data collected. The data is classified into two kinds, usability data and EEG data. Usability data contains quantitative data and qualitative data. The quantitative data includes the reviewers’ grade of three-class index, the time needed to complete the tasks and the number error appears, etc. The qualitative data includes the usability problems that reviewers find during the evaluation. The basis of quantitative data is qualitative data. The conclusion made from quantitative data could be proved by qualitative data; EEG data contains caution index and sentiment index.

6 Analysis of the Results

6.1 Analysis of Usability Results

Analysis of Qualitative Results.

By summarizing the usability problems put forward by reviewers, the three electric cookers can meet the basic needs of people, but still have some usability problems. For sample 1: the space consistency between display device and control device doesn’t conform people’s usage habits; and the reservation timing function is very complex to use. For sample 2: the motion consistency and habit consistency between display device and control device doesn’t conform to people’s habits. It causes people don’t know how to operate it. Besides, the color contrast of background and fonts is not very obvious. The button isn’t sensitive and people can’t conform the current operation status. For sample 3: the display interface is too small, and the fonts is not very clear; the variation of screen colors is unreasonable; the information display doesn’t conform to people’s habits and the functional adjustment is complex.

Analysis of Quantitative Results.

Summarize the time needed to complete the former 3 tasks and the percentage of error, the results shown as the Tables 3, 4, and 5. It can be seen from the table that sample 1 is superior to sample 2 and sample 3.

Table 3. The statistical result of the cooking task
Full size table
Table 4. The statistical result of the porridge task
Full size table
Table 5. The statistical result of the making an appointment time task
Full size table

According to the score of every index given by reviewers and the relevant index weight product, the usability score of sample 1, 2, 3 are 4.23, 3.83, and 3.82, respectively. Both users’ satisfaction and usability score can declare the usability design of sample 1 is better.

Global Analysis.

As far as the qualitative results can indicate that sample 1 has less of usability problems. As far as the quantitative results can indicate that sample 1 has a higher score and users’ satisfaction score. Sample 2 and sample 3 have a little distinction. We can draw a conclusion that the number of functions is out of proportion as usability. More functions don’t mean good usability.

6.2 Statistical Analysis of EEG Results

The EEG results are arranged as Table 6 shown. It is indicated that the usability design of these 3 electric cookers is good, because the percentage of average value over 60 for caution index and sentiment index are very low. But, on the other hand, it is indicated that when subjects operate these three electric cookers, they would feel tense. Because the percentage of maximum value over 60 for caution index and sentiment index are very high. Furthermore, the percentage of sample 3’s each index are lower than the other two sample. It has a relationship with the simple operation program, which could reduce the mental burden of users.

Table 6. The statistical result of EEG experiment
Full size table

6.3 Analysis of Global Results

From the usability test result and EEG test result, it can be seen the usability design of these three electric cookers is relatively good. But there are still some error ratio, and the users’ satisfaction score is not that high. The user would feel tense during operating the electric cooker. Therefore, these three electric cookers should do something to improve their effectiveness, efficiency and users’ satisfaction combining their characters to make human-machine interaction more friendly.

7 Conclusion

The aim of evaluation is finding the problem and solving the problem. This paper used a method that combined usability test and EEG to find problems existed in the display interface of electric cookers. This article is of great significance:

  • Using NeuroEdu equipment to evaluate the usability of electric cookers, we can monitor the changing emotion of subjects during the experiment, and we can understand the users’ needs, so that we can improve the display interface.

  • By using an innovative method that combined traditional usability test and modern electroencephalograph (EEG), we can find usability problems from both subjective and objective aspect, and solve these problems to improve the usability of intelligent electric cookers.

  • This study provides a direction for the display interface design of intelligent electric cookers. It makes the target consumers experience the product innovation as early as possible. This study can shorten the product innovation process, in order to reduce the defect after the product come into the market, improve the product quality.