Keywords

1 Introduction

Everyday, we interact with software through various user interfaces (UIs). How the UI design provides users with information is an extremely important topic. To improve UI design, we focused on how users find needed information using a device (i.e., information searching).

1.1 Information Device and Human Cognition

In this study, we define the term “information” as facts that a device stores and provides for users. Information is provided for users through a stimulus or a combination of stimuli on the device display. These stimuli have properties, such as format, color, size, and location.

The users’ process to read information is as follows. The users have to operate the device so that the needed information appears on the device display. They can perceive all stimuli in their visual field (i.e., iconic memory). From these stimuli, they find the location of a stimulus they need, based on the property settings of the stimulus, focus their attention on it, and then recognize its meaning (Anderson and Matessa 2007). From the meaning(s) of the stimulus or stimuli, they obtain their needed information.

We focused on how the users bring up their needed information on the device’s display during the first step of the process. In this step, they must utilize some declarative knowledge, that is, facts that a person knows (e.g., “Tokyo is the capital city of Japan”). Declarative knowledge about the device is acquired by the users through their experiences of information searching. In this study, we investigated the types of declarative knowledge acquired and utilized to search for information.

1.2 Two Types of Declarative Knowledge

Two types of declarative knowledge were important to our research efforts: structural knowledge and perceptual knowledge.

Structural Knowledge Previous studies have emphasized the importance for users to acquire an understanding of the hierarchical structure of information or the menu categories (Amant et al. 2007; Jacko and Salvendy 1996; Ziefle and Bay 2004, 2006). For example, Jacko and Salvendy (1996) investigated the influence of the depth of a menu structure on perceived complexity. Ziefle and Bay (2004, 2006) concluded that knowledge of the menu structure is important for the accuracy of the operations.

We defined knowledge about the structure of information as “structural knowledge.” To acquire structural knowledge means to construct a mental map of the information, which shows where each piece of information is allocated in the hierarchical information structure of the device. The users can mentally find a path from a current position to a target position in the map.

Perceptual Knowledge. Recently, UIs have provided information in a more graphical manner. The properties of a stimulus have various settings. In older UIs, stimuli are presented in the identical property settings (e.g., the format is textual, the colors are black and white, and the stimuli are sized identically). We defined the property settings of a stimulus used as a cue to finding the target information as “perceptual knowledge.”

Previous studies show that the difference in property settings of stimuli affected how users learned and operated a device (Benbasat and Todd 1993; Gittins 1986; Parush et al. 2005). Simon (1975) showed that participants determined which actions to take based on the problem state acquired perceptually. Similarly, users would utilize the perceptual knowledge while searching for information. We investigated the role of perceptual knowledge by simulating users’ cognitive processes.

2 Multi-information Display

We used a multi-information display (MID) installed in certain hybrid cars to provide drivers with driving information. The driving information was presented graphically, such as the examples shown in Fig. 1. For example, the average speed of a vehicle was presented by four stimuli: one was an icon (drive meter) format and the other three were text (AVG, 56, km/h) format, and both in white color, as shown in Fig. 1(a), and how much money was being saved each month was shown in a yellow graph with white text, as shown in Fig. 1(b). These property settings were acquired as perceptual knowledge if needed (e.g., the amount of money saved in July is presented with a yellow graph).

Fig. 1.
figure 1

Example screenshots of MID. Figure 1(a) shows the “Drive Info” in DI. Figure 1(b) shows the “Saving Money Record” in Eco (refer to Fig. 2).

Full size image

The MID information was structured as a hierarchy, as shown in Fig. 2. The information was categorized into one or both of the higher categories: “Drive Information (DI)” and “Eco.” Furthermore, they were categorized into some lower categories. The information in an identical lower category was presented together on the display. The structural knowledge in the MID referred to the information categorizations and their relationships (Fig. 2).

Fig. 2.
figure 2

Hierarchical information structure of MID. Information is shown in italic fonts, the lower categories in normal fonts, and the higher categories in bold fonts. The figure does not describe all information in MID. “EV” in figure means an electric vehicle.

Full size image

The drivers could select categories using arrow buttons on a steering wheel. They selected the higher category using the right and left buttons and the lower category using the up and down buttons. The lower category was selected in the order shown in Fig. 2 (i.e., if the driver pressed the down button when DI1 was selected, DI2 would be selected). Additionally, both higher and lower categories were selected cyclically. For example, DI1 would be selected after DI4 if the down button was pressed, and vice versa. In this study, the MID was presented on a PC monitor and operated using arrow keys on the keyboard.

During the information search task, one of the names of the target information (Table 1) was presented at first. Participants or computer models had to find the value of the target information from the MID and enter it. If the entered value was correct, the name of the next target information was presented. We prepared four sets of the target information (Table 1).

Table 1. Target information
Full size table

3 Human Data

Thirty-eight undergraduates ranging in age from 19 to 23 years (\(M = 20.579\) years, \(SD = 1.780\) years) participated, and their behavioral data were collected after training (performing eight searches for each set of target information). During training, an initial category for each search was randomly determined. The participants sufficiently, quickly, and accurately searched for the information by the fourth search.

4 Computer Simulation

4.1 Adaptive Control of Thought-Rational

Adaptive Control of Thought-Rational (ACT-R) is a cognitive architecture that consists of modules (Anderson, 2007). Figure 3 shows the structure of ACT-R. The ACT-R model receives the perceptual stimuli from the environment through perceptual modules and changes the environment using motor modules. ACT-R has two memory modules: declarative and procedural memory modules. Procedural memory includes the production rule (i.e., “IF ..., THEN ...”). Declarative memory includes declarative knowledge. The structural and/or perceptual knowledge is part of this memory. The production rule whose conditions match the current model’s state is executed.

Fig. 3.
figure 3

Structure of ACT-R

Full size image

4.2 Three Models

We created three models implementing different types of knowledge in declarative memory, namely the Structural Knowledge model (SK model), the Perceptual Knowledge model (PK model), and the Mixed Knowledge model (MK model). In the following section, we discuss the types of knowledge in declarative memory, as summarized in Table 2, and the search heuristics for each model.

Table 2. Declarative knowledge of each model
Full size table

SK Model. The SK model included perfect structural knowledge, which constructed the information structure in Fig. 2. Its structural knowledge consisted of link and order knowledge. Link knowledge was the parent-child relationships between the information and the categories, such as “Vehicle Speed is categorized as Digital Speed Meter” and “Simple Display is categorized as DI” (refer to Fig. 2). The order knowledge was the order of the lower categories, such as “Digital Speed Meter is the third lower category” and “Simple Display is the fourth.”

For the SK model, we used a means-ends analysis as a heuristic. In this heuristic, the model applied an operation that would reduce the difference between the goal and the current state. The SK model calculated the shortest path from the currently presented category (i.e., current state) to the category including the target information (i.e., goal state), based on the information structure constructed from the link and order knowledge.

For this heuristic, the SK model had to identify the currently presented category based on the presented stimuli. The SK model had no perceptual knowledge. So it recognized the meaning of the stimuli and identified to which information the meaning corresponded. Therefore, we gave the SK model the knowledge to associate stimulus meanings with the information names.

PK Model. The PK model included only perceptual knowledge, which included the property settings of the stimulus important for information searching. Its perceptual knowledge consisted of the target-display and DI/Eco-identification knowledge. The target-display knowledge consisted of the property settings of the stimulus used to identify the lower category that included the target information, such as “Vehicle Speed is presented with a big text stimulus,” and “Saving Money in July is presented with a graph stimulus.” Note that the knowledge included the property settings of a significant stimulus in the display, not the stimulus representing the target information. The DI/Eco-identification knowledge was the property settings of the stimulus used to identify the currently presented higher category, such as “Graph is the property setting used in Eco” and “big meter is the property setting used in Eco.”

The PK model used a heuristic in which it compared the property settings in the target-display knowledge with the property settings of the stimuli currently presented. If the model was not able to find the stimulus that had the searching property settings, it switched to the next lower category. The model repeated this comparison until it found the stimulus that had the property settings in the target-display knowledge (i.e., the model found the lower category that included the target information). We defined this heuristic as a perceptual comparison. Additionally, the PK model switched to the other higher category when the presented category returned to the initial one.

MK Model. The MK model included both perceptual and partial structural knowledge. The perceptual knowledge was identical to that of the PK model. The structural knowledge of the MK model was the higher-category knowledge that showed to which higher category the target information belonged, such as “Vehicle Speed belongs to DI” and “Current Eco Ranking belongs to Eco.”

The MK model used a means-ends analysis and a perceptual comparison. First, the model used a means-ends analysis. It detected whether the currently presented higher category, identified using the DI/Eco-identification knowledge, was the higher category that included the target information, shown by the higher-category knowledge. If it was not, the model changed to the other higher category. Then, it used a perceptual comparison until the lower category including the target information was presented.

4.3 Simulation Results

We ran each model 450 times for each set of the target information.

Search Sequence. Using the Levenshtein distance, we compared the search sequence of the categories of each model and those of the human participants. The distance was obtained using the minimal number of operations (insertions and deletions) needed to transfer from one sequence to another. The smaller distance meant that the sequences used by the model and human participants were more similar. Figure 4 shows the mean distance between each model and the human participants. A significant difference between the models was observed (\(F (2, 111) = 40.073, p < .001\)). The distance of the MK model was smaller than those of other models (ps \(\le .001\)). The distance of the SK model was smaller than that of the PK model (\(p < .001\)). These results indicate that the MK model’s sequence was the nearest to the human data, and the PK model was the farthest.

Fig. 4.
figure 4

Mean Levenshtein distance between each model and the human data

Full size image

Additionally, no human participant achieved a sequence that was identical to that of the PK model. Considering the farthest distance, the PK model could not explain the process of the users’ information search. After that, we compared the SK and MK models.

Observation Time. Because of the limitation of the analysis method, we used the data from the human participants whose search sequences were identical to those of any model for the target information 1 and 3 (T1 and T3 in Table 1). The details are summarized in Table 3. We calculated the observation time per lower category for the human participants and for each model. We conducted a linear regression analysis. The MK model predicted the observation time in the human data (\(r^2 = .951\)) to be more than that done by the SK model (\(r^2 = .330\)).

Table 3. Details of the analyzed data
Full size table

5 Discussion

We compared the behavior of the three computer models to investigate the types of declarative knowledge utilized for information search. The MK model could explain the behavior of the human participants, which means that the users utilized both perceptual and partial structural knowledge for information searching.

The PK model, which included only perceptual knowledge, failed to predict the search sequence of the human participants. The PK model observed all lower categories in the higher category selected initially, even when the higher category did not include the target information; the human participants avoided such unnecessary searching. This result implies that the users utilize the structural knowledge to judge whether the current higher category included the target information.

The SK model, which implemented only structural knowledge, could not explain the human participants’ time spent observing each lower category. The model took more time to identify the currently presented category when the search started and when the higher category was changed. Because of the lack of perceptual knowledge, the model had to recognize the meaning of the stimuli for the identification. More time was needed to move attention and to understand the meaning of the stimulus. The human participants did not need such a long time. Therefore, they must have utilized perceptual knowledge to judge whether the currently presented category included the target information.

5.1 Cognitive and Physical Load

The results of our simulation show the following user information search strategy. The users narrowed the search area using structural knowledge, and then used perceptual knowledge to judge whether the currently presented category included the target information. Based on the results of several previous studies, it seems the rational strategy.

The means-ends analysis imposes a heavy cognitive load (Sweller 1988). To calculate the shortest path mentally, the users have to keep in mind the goal and current states, and the intermediate states if needed. However, the calculated path is the shortest and most accurate if the structural knowledge is perfect. The users can reach the target information without extraneous key presses, which reduces their physical load.

In contrast, the perceptual comparison reduces the cognitive load (Kirsh and Maglio 1994; Simon 1975). The problem state kept in the users’ mind is minimized because the stimuli in the environment are used for perceptual comparison. However, the physical load is increased. The users have to continue to press keys until they find the stimulus that matches with the perceptual knowledge without knowing which category includes the target information and how far category includes the target information.

Our study suggests that users balanced cognitive and physical loads.

5.2 Implications

We should consider both structural and perceptual aspects when designing a UI. The structure of the information is particularly important for classifying information into higher categories. Users utilized structural knowledge when they decide whether they had to switch the higher category or not. Users can easily acquire structural knowledge when an information classification in a device is similar to users’ classification.

Users utilized perceptual knowledge to judge whether the currently presented category included the target information. Therefore, a highly similar arrangement of stimuli confuses users. Thus, we should provide significant features that identify each category.

In addition, a computer simulation approach can act as a low-cost usability evaluation. We can calculate the cognitive and physical load of users without recruiting many participants.