Keywords

1 Introduction

Demands on production performance, cost efficiency, personal safety and environmental considerations, among others, push the creation of even more intelligent industrial systems. From the beginning of the industrial era, technological advances have changed the users’ behavior and tasks performed. For each new generation, machines are becoming more and more automated. Availability of information technology in terms of sensors, computing performance, and software capabilities increases over time, and so does the use of information systems enabled by the big amount of data produced.

This direct the machine users (i.e. operators) activities, towards information processing and spatial awareness, rather than controlling the machine. They spend more time monitoring and gaining information from built in, or retrofitted information systems. For example, Global Positioning System (GPS) based navigation systems, aided visual systems using cameras, Industrial Control System (ICS), etc.

Current industrial approach to present the increasing information in vehicles is to add more instrumentation and graphical displays. A tractor can, for example, contain more than five different displays. One for the main instrumentation cluster, one for tractor configuration and information, one for the precision farming system, one for the implement attached to the machine and one for monitoring cameras. This does not include additional screens, such as the operators’ cell phone or additional implement displays. While future scenarios involve augmented reality and see-through interfaces [1, 2], there is currently a limitation in cabin space.

Even at the current state, there are issues with user acceptance. For example: When the author were making a pilot field study at a larger Swedish farm and asked the operator about the usability of the touch screen, the operator answered: “I don’t use the touch screen, only the joystick”. In this case, the machine was a potato harvester attached to a tractor, where the touch screen was used to configure and show information from the harvester’s implement. The user was regarded a skilled machine operator in terms of driving and operating the equipage, but he was limited in his computer related interaction. Instead, the manager of the farm made the configuration and tuning of the system.

The observation above is supported by other studies addressing obstacles for operators to embrace new technology. In agriculture, for example, Kutter et al. express that precision farming is less frequent than expected [3]. Precision farming systems are information intelligence systems that include positioning information, maps, and sensory data in order to provide farm managers and operators a better overview and control of their production. Ultimately this leads to decision support systems that optimize production and reduce the use of treatments that has an environmental impact [4, 5]. Although precision farming has existed for decades, its main adopters are younger full-time farmers with higher education, operating at larger farms [6], making the potential benefits not fully exploited. Assuming that other industrial vehicle domains have a similar situation there is still much to be done in terms of operator interaction design with these types of system.

1.1 The Challenge of Gaining Understanding

Creating interaction with these systems require a diligent understanding of operator’s behavior [7]. However, doing field studies can take a significant amount of time [8]. Also, participatory techniques can be difficult to perform, because in many of these vehicles the cabin is too small to ride along with the operator. Thus, one is limited to observe from a distance, making it difficult to detect operator attention and actions that are not visible. Performing interviews or surveys with users is an approach that can give lots of valuable information. However, when they are made separate from the actual operation the answers may not fully include the actually performed operation, or be biased by opinions [9]. As argued by Nielsen, designers should “pay attention to what users do, not what they say” [10, 11].

One method to acquire knowledge about a user’s behavior and attention is by using eye tracking. Eye tracking is carried out using a camera that measures the eyes focus point, i.e. the gaze fixations [12]. These fixations can then be referenced against information in front of the user, for example, a monitor presenting a web page. This makes it possible to observe where the user puts his or hers focus of attention and analyze the information seeking pattern.

In literature, many eye tracking examples relates to marketing and different types information retrieval studies, such as forms evaluation and web pages [12]. In road vehicle situations eye tracking has been frequently used to study drivers behavior and attention [13]. Eye tracking has also been used in different off-highway vehicle setups, some examples from different areas being: Forestry, where eye tracking has been used to identify dwell patterns and focus areas for harvester operators [14]. In mining, where it has been used to track user attention and, combined with additional sensors, identifying factors for operators ergonomics [15]. Eye tracking has been used in agriculture to measure differences between drivers experience levels and to measure how a higher level of automation increases utilization of vehicle display systems [16]. It has moreover been used in trains, for example, to measure how in-train traffic management systems affect the visual attention to the outside of the cab [17]. In the property construction area, “considerably less attention has been paid to its potential to improve the design and construction processes” according to Yosefi et al. [18].

Traditional eye tracking has been related with bungle some in equipment as well as time-consuming methods. Historically, equipment such as cameras had to be mounted in cabins, taking up valuable production time when doing field studies. With the availability of head-worn tracker solutions, the tracker follows the user movements while looking around. This is beneficial in applications where the user is not focusing at one single area, such as a form or a screen. Instead, it can record how the user looks around in the scenery, such as in mobile machines, were the driver can have a wide area of attention and a frequent change of focus. Still, head worn trackers have until recently been associated with a high cost and have thus been a tool for researchers or enterprises with deep pockets that spend significant efforts in user understanding. Also, recommendations on the number of participants to include in studies are above 30 people for quantitative conclusions and six users for qualitative studies [19]. Not only do this consume time in the field, it also makes analysis of data an effort consuming activity. Combining a gaze map with 30 users on a static screen is considerably easier than combining 30 people performing a freely moving activity.

1.2 Re-Setting the Scenery

With shortened time to market and a design practice that is becoming increasingly incorporated into the product development [20], it is of interest to understand how we can complement more rigorous user studies with lightweight approaches. In the user experience community (UX), cost effective methods are proposed where practitioners quickly go out to gain user understanding, for example when approaching persons in a coffee shop to ask for feedback on a design, also known as guerilla HCI [21, 22].

What if the same type of cost effective methods could be used to gain understanding in other domains, more specifically for heavy vehicles? Though, in a café, it can be assumed a visitor’s participation will not interfere with the work of a complete team.

Here we come back to eye tracking, where availability of simpler and cheaper equipment [23, 24], opens up for a more widespread use [25], as a probing tool to identify where additional usability testing resources can be focused. With eye tracking plus a recording of the user’s operation, it is possible to get a view of the attention and how the user is performing the task at hand. If the operator could quickly be hooked up with the equipment and then continue with the work, such approach could offer a time and effort efficient method to conduct field studies. Giving minimal interference to operators and their team’s ongoing work and a tool to collect, and share, user perception of a product or a solution, with a physically distributed engineering and design team.

Additionally, Rozalinski et al. present that even simplified methods of eye tracking and a limited test sample “allowed us to discover several usability problems […] not discoverable through traditional usability testing methods. Moreover, eye tracking provided a more detailed understanding of usability issues already uncovered by traditional usability testing” [26].

The rest of this paper will present an experience report where eye tracking was performed in a guerilla usability testing manner. This combination of approaches within the industrial setting is uncommon in literature and show how practitioners can quickly get real field data. The used technical setup will be presented as well as the practice where industrial sites were approached to find users to study. Following thereafter is a presentation of the recording analysis and findings, a discussion of results including practical lessons learned and finally a conclusion.

2 Study Setup

The purpose of this study was to evaluate how eye tracking can be used to make quick probing studies of operator attention when using off-highway machines, as well as to gain knowledge about their use of the digital information systems in vehicles. For this study, a diverse set of machines and environments were approached, rather than a higher amount of a specific type. The machines were:

  • A forestry harvester, performing thinning operations.

  • A wheel loader working in a stone crusher zone, moving and loading crushed material to trucks.

  • An excavator performing supportive functions during leveling of a shaft.

  • An articulated dump truck transporting soil from a digging zone to an unloading zone.

  • A wheeled excavator working with leveling of a property ground.

  • A mobile crane, lifting prefabricated concrete walls at a property building.

The study was done at different working sites located in central region of Sweden. Swedish was the natural language for all participants. The conductor, also Swedish, stemmed from the industrial vehicle domain, thus having basic knowledge about machines, operation process, domain terminology etc. All machines visited were operated by male operators, ranging from young adults to middle-aged men. Unfortunately, we didn’t meet any females.

The field work was constrained in time, motivated, as described above, by an interest to get real-life information quickly and without costly setups and long planning. In practice that meant that 4 of the vehicles was visited in one day. The other at separate days.

A mix of operators worked alone and with other people/machines. The harvester worked alone. The excavators and the crane worked with other persons. The wheel loader and dump truck were mostly alone, though occasionally approached by other machines. This mix was looked-for, it gave a diverse sample. A lower variety would be of interest, in a detailed study.

For some of the above machines, there was enough space for an additional passenger that could ride along and observe the operator behavior and interaction, in this case, the dump truck and the harvester. In the other machines, there wasn’t enough space to fit more than the operator.

On-road moving vehicles, such as trucks, were deliberately not included as these type of vehicles work in different conditions and with a different working pattern.

Our assumption was that, using eye trackers that record what is in front of the operator, analysis can be made of real word operation in a detailed manner where it is difficult observe from aside or through testimonials, in areas such as users attention while doing operation, how they interact with other workers and to what level the systems built-in Human Machine Interface (HMI) is used. Observing the user’s gaze can also give input on factors in the HMI, for example, if excessive time is spent comprehending the HMI, or if information is not detected, such as notification or alarm messages. The diversity and size of the sample set would not give quantitative results, but prove the study approach and give input of interest for further, more detailed, studies.

3 Technical Setup

The eye tracking equipment was based on a pair of eye tracking glasses from Pupil labs [27]. The glasses capture the pupil position using emission of infrared light that illuminates the user’s eye and detects the pupil position based on the reflected light [24]. This is done via two eye cameras, making it possible to track both eyes for higher accuracy. These cameras are placed on an adjustable arm at the lower outer side of the spectacle frame. In the center of the spectacles frame there is a camera pointing forward, the world camera. This camera records what’s in front of the operator, in other words, what the operator sees.

The gaze tracking and eyesight recording are done via software on a connected computer. A lightweight laptop computer was used, running Ubuntu 14.04 and the pupil labs open source software (Pupil Capture v.0.7.3).

The glasses features a 120 Hz infrared camera for eye capture rate and a full HD (1080p) scene in front. However, in the recordings, the resolution was limited to 60 Hz framerate and HD (720p) resolution. This was selected to reduce the file size of the recordings and to preserve computer battery lifetime, due to the high CPU load when using highest settings. Sound was also recorded, using the laptops built-in microphones.

The analysis was performed in the accompanying player application, Pupil Player. The windows version v.0.6.9 was used. This application can play back the video from the front camera together with the eye tracking data overlaid on the recorded image.

4 On-Site and User Approach

The first visit was the forest thinning site. It involved a few hours of travel, hence the operator was contacted in advance and the study was briefly described. This was the only visit where the operator knew about the evaluation in advance. The subsequent site was contacted via phone to clear the visit with site management before approaching the site, the rest of the sites were approached directly.

At the site, the responsible manager was approached for clearance before approaching any operator. The purpose of the visit was described, how the eye tracking study should be performed as well as the ambition to introduce as minimal as possible disruption to ongoing production. Everyone approached was very positive to the purpose and every visited place accepted the request to study one or more operators. For the machines approached there was occasionally a shorter time period where the machine wasn’t operating, for example, while waiting for material or for others work. Unless when escorted by local managers, this was the time when machine operators were approached.

In order to produce valid data, the glasses have to be calibrated. This calibration was done with the operator staying in the cabin. Not only did this make the setup sequence quick. It also avoided the glasses to become repositioned on the head while entering the vehicle and it helped to get world camera set up in a good angle, to record the correct part of the visible area.

The calibration process was performed using the recording applications screen calibration process, where the computer display was held in height of the operator’s head and as close as possible to the front windscreen. This type of calibration process does not result in the highest accuracy when looking at longer distances, it was, however, a tradeoff for getting accuracy when looking inside the cabin. The glasses have an accuracy of 0.6% in good conditions [24] and the eye gaze position accuracy provided what can be considered as a satisfying result for the need, to capture attention to different areas and not specific details in the scenery or cabin.

Once calibrated the recording was started and the computer stowed away in the cabin. The whole setup procedure was completed a few minutes, depending on a number of questions asked by the operator. After activation of the recording, the operator was left to continue the work. The length of each recording session followed the natural breaks occurring during in operation.

4.1 User Acceptance

After shortly presenting the study, including the eye tracking equipment, the operators were asked if they were interested in participating. Neither of them had ever tried anything like this before, but pleasingly all were positive to be part of the test.

The use of glasses turned out to be a very suitable technology. It was natural for the user to put them on and easy to calibrate. They also seemed to give some amusement, with operators giving comment to other colleagues like “terminator is here” when wearing the glasses. The glasses was not reported to disturb their operation. However, some users reported them as a bit unaccustomed to wear and in the beginning they glanced at the eye tracking recording cameras, visible in their outer lower field of vision.

When natural breaks occurred, or when it was possible to ride along in the vehicle, questions were asked to gain more understanding. This also gave an impression about the operators, where the harvester, wheeled excavator and crane operator showed a genuine interest in their machine and technology. The excavator and Dump Truck operator seemed knowledgeable but didn’t show the same interest. The wheel loader operator seemed least interested. However, this can be biased by the operator’s social appearance. Firm information about user knowledge, background etc. would require formal questions, thus counteracting the aim of minimal interruption.

5 Gaze Recording Analysis

Apart from evaluating the acceptance and use of eye tracking as a method and tool to conduct quick on-site studies with different types of heavy machines, there was a purpose for the study as such. The purpose was to perform a mix of phenomenological research [28] and objective indications. The phenomenological research objective was to acquire information on operators work in natural conditions and to get an understanding about their steps of operation and focus of attention. The following section will provide examples of analysis results from each of the visited machine.

5.1 Forestry Harvester

The forestry harvester was performing thinning in the forest. The purpose of the thinning procedure is to select which trees to keep for further growth and which trees to remove in order to give the kept trees improved conditions and room for further growth. The selection of trees to keep and remove is in some cases done before actual felling, but in this case, it was done by the operator while driving.

During felling and movement of the crane, the focus was mostly on the harvester head, as exemplified in Fig. 1, with glances in other directions to check felling directions as well to ensure enough open space for the tree being fed through the harvester head.

Fig. 1.
figure 1

Forestry harvester and operators gaze points when processing a birch tree.

Full size image

The display, placed centrally in the lower part of the windscreen, was glanced upon shortly to verify that the selection of assortment was correct and that the fed length was reasonable. In other words, making sure everything was operating as expected. This was only done for trees considered to have a value, pulpwood was just cut in the pre-set length without checking the screen. However, when feeding of the tree was not proceeding as expected, the display was used to find out information about what could have gone wrong. For example, if the display showed that the diameter or length wasn’t as expected, this provided an indication that additional stems had been taken in the harvester head and thus obstructing the length measurement system.

In addition to the selection of trees to fell, the operator must also plan the travel path for the machine. Whether or not he will reach a tree from the current passage or have to go another way. During movement of the vehicle, the gaze moves around more freely. Sweeping from identification of the tree to fell, if the crane reaches there, scanning of the ground for good passage, estimating if trees are possible to pass or if the wheel will hit them in tight passages or during reversing. The operator must also choose how to fell the trees in a good way, both to avoid damages on trees that should be kept, as well as being able to collect cut trees into piles which the forwarder can later come and collect. Still, the focus is mostly ahead, because the whole cabin can be rotated instead of only the crane.

Figure 2 shows the level of attention through the cabin windows and at different areas in the cabin. Areas are divided into left window, front window, right window, upper window, lower part of the front window and the computer display.

Fig. 2.
figure 2

Level of attention in different areas for the harvester.

Full size image

Being a machine so computerized and with the display placed centrally in front of the operator, its attention was remarkably lower than expected. There is, however, a possible reason, related to the task performed. During thinning trees with less value are removed, while the maximum output of each tree is the focus during final felling. Because of this, according to the operator, the thinning work involve less screen usage than final harvesting. In other words, there wasn’t much focus on selecting optimal lengths and cuts, which is one of the primary information on the display.

5.2 Wheeled Excavator

Next was the wheeled excavator, it was adding gravel to a property ground in order to level it. Gravel was collected from one pile and transported it to the leveling area. Reaching the unloading area the operator scanned the surface, looking for the next place to fill, see example in Fig. 3. Then the gravel was poured out and thereafter evened with the shovel. The final adjustment was done manually by another person.

Fig. 3.
figure 3

Level of attention in different areas for the wheeled excavator.

Full size image

In a site like this, with several machines and persons working simultaneously, the operator reported that he routinely checked the rearview mirrors before starting to roll the excavator. This was also confirmed by the eye tracking. However, during leveling, the focus was on the task at hand, with only a few indications that the operator was looking at something else than the shovel operation and the partner on the ground.

The machine was equipped with a height sensing system, giving the operator guidance on the height of the shovel. The final measuring was however done by the external person using laser measuring, not making it necessary for the operator to do precision operation based on the machine measurement indication. Additionally, the measurement system also had a useful feature where the whole screen shifted color between blue, green and red, depending if the shovel was too low, at correct height or too high. This signal was probably indication enough for the operator’s peripheral view while having focus on the task at hand.

In relation to the harvester, the excavator had much higher values on display usage, in other words, use of precision positioning system, see Fig. 3. This despite having ground support. In terms of attention, the excavator also shows relatively high numbers at the sides, even though the operator had the possibility to rotate the cabin with the boom, in the same way as the harvester. One reason for this was the wheeling. The path was partly over a narrow road built using sleepers, with the purpose to even the ground pressure. There the operator looked out of the side windows to ensure the correct wheel placement. Another reason was the communication with the ground person where the operator often rotated the machine and looked through the side window, not to have the boom close to persons.

5.3 Mobile Crane

The mobile crane was lifting concrete pre-fabricated wall segments from a truck to a building ground. This was performed in close range to the mobile crane and with good visibility. Unfortunately, for eye tracking recording, the weather changed from cloudy to sunny before recording was started, i.e. the sunlight became strong enough to affect the pupil detection. Although no valid gaze plots could be made, the recording from the front camera could still give valuable information. For example about the sequence of operation and communication between the operator and other personnel when attaching wall segments and placing them.

5.4 Wheel Loader

The wheel loader was transporting material from a stone crusher to a pile as well as to passing trucks that came to collect gravel. During loading, the attention was on the bucket. Presumably to watch the bucket filling up, but also to watch the angle of the bucket when lifting material from the pile. During unloading the focus was on the gravel falling out of the bucket, in order to obtain a steady flow and match the falling gravels placement.

Figure 4 shows the areas of attention for the operator. The mapping of visual attention was analyzed during transportation to the loading/unloading area and loading of a dump truck flatbed. As shown by the numbers, the major focus was ahead of the machine. Very little attention was focused through the side windows, even while the loading involved reversing operations. Though, some attention was put at the rearview mirrors which were visible through the front windows. It is also noticeable that the instrumentation, placed low in the cabin and containing an integrated reversing camera, was never visible during the session analyzed. This may indicate that placement of information providers, in this case reversing camera and mirrors, can affect usage. We will come back to this for the articulated dump truck.

Fig. 4.
figure 4

Level of attention in different areas for the wheel loader.

Full size image

5.5 Excavator

The excavator was supporting ground personnel to level a shaft, wheeling to get material and distributing it on places indicated by the ground personnel. During wheeling, the operator’s attention was freely moving as the transportation was straight ahead without obstacles. During unloading the attention was however swiftly moving between the ground leader, indicating by gestures how much material to pour out, to observation of the gravel falling out from the shovel and where it landed.

Figure 5 shows the attention of the operator during the supporting face. As the work was carried out in a shaft on the left side of the machine the attention is focused on the left side. And since the workers in the shaft controlled depth and alignment of the ground, the vehicle’s information system was hardly used at all. Using the video recording, it was also noted, in terms of input to the system and ergonomics, that the operator did quite excessive shakes of the joysticks. This in order to get an even pour of gravel.

Fig. 5.
figure 5

Level of attention in different areas for the excavator.

Full size image

5.6 Articulated Dump Truck

Lastly, the articulated dump truck whose task was more related to vehicle driving than control of machine operation, such as crane arm operation. Still, while the control of the dump truck isn’t related to advanced processing control systems, the display systems usage was surprisingly high in, comparison with the other vehicles, see Fig. 6.

Fig. 6.
figure 6

Level of attention in different areas for articulated dump truck.

Full size image

One reason for this was due to the reversing and unloading operations, where the operator constantly switched focus between the rear view mirrors and the reversing camera display. Looking into the mirrors make up for 11% and 8% of the side windows gaze time respectively. In relation to the wheel loader, it can be noted that the placement of the reversing camera display, more aligned with the mirrors could be one factor for the higher utilization. It can also be noted that the placement of the rear view mirrors makes the operator look more on the sides.

5.7 General Findings

Sampling on a diverse set of machines opens up for general findings where it is possible to detect similarities in use, or differences, between different machines. Such examples are when problems occur. Then the operators looked at the display for additional information. Another example is during normal movement of crane or machine, where the digital information systems were used to a very low degree, if used at all. This fact shall probably be seen as reassuring since focus should preferably be outside the vehicle rather than on screens inside the vehicle. As told by the operators the display was however used more frequently in certain scenarios, like when the crane lifts material to locations with limited visibility. The earlier mentioned difference in display placement and use of reversing camera is also an example showing how learnings can be discovered by matching different designs.

An additional area affecting all of these machine types is safety, where operation should be safe for both operators and surroundings. Neither machine had any system for detection of surrounding persons or objects, so the operator had to keep track of people in range of the machine. However, when doing work that required some form of precision, the gaze mapping clearly showed a focused attention to the task at hand and the persons directly involved in the task, resulting in a limited attention to the surroundings. The surrounding environment does, of course, affect the possibility of someone being in the vicinity of the machine, for example being alone in the forest versus a building site with several machines and persons close by.

For all machines, the machine structure caused limited visibility, but it was most noticeable for the wheel loader, with its bucket in front of the operator. Additionally, the visibility was very limited when doing operation with the bucket raised high, for example when unloading material in the truck bed.

6 Measuring Display Usage

One objective of the study was to measure attention to the graphical display interface in the different vehicles. This measurement was done by observing a clip from each machines recording, each containing a few sequences of repeating operation. For example, tree cuts, lifts, transportation loops, etc. For each of these clips, the time when the user was having a focus on any graphical display in the cabin was measured, starting with a fixation on the display and ending with a fixation outside the display. For calculation, the frame counter presented by the pupil player software was used.

The result is listed in Table 1, indicating the time spent looking at the vehicle graphical display(s) in percent of the total time for each recording clip. In some cases, the numbers in the table differ in relation to the numbers in the figures above (Figs. 2, 3, 4, 5 and 6). The reason for this is that the measurement of display usage was performed over another timespan than the measurement of where the operator looked in the whole cabin/surrounding environment.

Table 1. Percentage gaze time on vehicle display when performing operation for the measured time
Full size table

The numbers were generally lower than what was expected prior to the study. It was expected that vehicles with richer information systems that presented process and site information should have at least 5% display attention. Such vehicles were the harvester with its centrally placed display presenting key metrics presented, as well as the wheeled excavator using a precision positioning system.

The other systems had a more basic information system (instrumentation type and reversing cameras. It was expected that the usage for these vehicles should be low. In this respect, the dump truck stands out with the highest registered usage at 7%, and the wheel loader, who did not use the built-in display at all.

One vehicle is missing from the above table, the mobile crane. As mentioned above this vehicle was left out because of weather conditions, making the recorded data untrustworthy for measurement. However, the indication was that screen usage was limited. Something that was also supported by the operator when describing that he did not use the support from the vehicle information system when lifting in close range and good visibility. The aiding systems, that display height, load, wind, and pressure on supporting legs was, however, more frequently used when lifting longer distances and when direct visibility is hindered.

As a diverse set of vehicles was selected for the study, findings cannot be considered quantitative representative, they rather provide an indication for future more detailed research. Both for individual machine types and also for identification of possible similarities between machines. Following is an example that illustrates this.

As mentioned, the dump truck has the highest display usage. This while being the vehicle with the lowest level of information system support. The other graphical information system in the vehicle was a dot matrix display in the center console. However, the secondary display was frequently used for a specific activity, reversing. During reversing, attention was divided between the mirrors and the reversing camera. Subsequently, the attention to the reversing camera display increases to 23.8% when looking only at reversing time, which occurred 4 times during the clip.

Simultaneously, the wheel loader had no display usage. The wheel loader was also equipped with reversing camera capabilities. The operation pattern between the dump truck and the wheel loader did also have similarities: Something is loaded in the bucket of the wheel loader, or the tipping body of the dump truck, the machine thereafter reverses, travels to the unloading zone and unloads. What could then be the reason for this difference in attention? One possible explanation could be the cabin design. In the dump truck, the rear view camera display is placed at the same level as the rear view mirrors. Making the eye movement flow naturally between the mirrors and the display. In the wheel loader, on the other hand, the display showing the reversing camera is placed in the lower right instrumentation cluster, far away from the operator’s normal attention. Perhaps this makes the time and effort needed to look at the display too high.

More data can be extracted from the recorded material. Still, the paper exemplifies that also a limited study and analysis can give insights and pinpoint future detailed studies. It shall be noted that great care shall be taken when comparing the figures and before making firm conclusions. The use of different information sources must be put into context. For example, the excavator did not have to use any assistive functionality for the work he was doing. The display usage indicated by the numbers was instead used during crawling, to change from slow speed into fast speed.

7 Discussion

Looking into the imminent future, development investments in heavy vehicles will to a high degree be in the digital domain. Studies indicate that up to 40% of a cars production cost is due to electronics and software [29]. Heavy vehicles are also be connected and integrated into the information value chain. For examples through wireless real-time productivity improvements in construction sites [30] and information driven farming [31].

With more functionality realized through electronics and software it can be expected that more information will have to be presented to the operator, already today the HMI in a car can provide access to more than 700 functions [32]. Still, the attention of the operator cannot be too diverted from the task at hand. Sørensen et al. mention that, in farming, the “increasing use of computers and the dramatic increase in the use of the internet have to some degree improved and eased the task of handling and processing of internal information as well as acquiring external information. However, the acquisition and analysis of information still proves a demanding task” [31]. Neither does the availability of data warrant the understanding or usefulness of the data to the user [33].

As more information has to be presented, space in vehicle cabins can be a limiting factor and as window area to look outside of the cabin is of the essence, thus putting demands on efficient communication through limited display sizes. One method is to combine different information providers together. However, as described by Holstein et al. there are challenges in building homogenous interfaces consisting of a diversified set of haptic and visual interfaces and software providers [34]. Sanches et al. state that “the need for research that informs the design of effective, intuitive, and efficient displays is a pressing one.” [16].

It can also be assumed that the connected vehicles will be updated more frequently and that agile development methods will be increasingly used. This is already happening in the automotive industry where continuous software deployment is starting to become practiced. Such examples are the Tesla vehicles, or the recent Volvo XC90 platform, where software is deployed every two days [35].

To aid the design of these information driven and more continuously improved systems there is a need for efficient validation of usefulness and stakeholder acceptance. Buur et al. observed, doing user studies of forklift operators, that the skilled operators perform many operations in parallel, as well as, tasks far beyond the everyday notion of ‘lifting’, ‘carrying’ and ‘dropping’ a load [36]. The knowledge must therefore be based system use and adoption “in the wild”, thus giving a true understanding of use to complement laboratory tests and prior assumptions [37, 38]. Benyon notes that designers need to understand the people who will use their systems and products. They need to understand the activities that people want to undertake and the contexts in which those activities take place [39]. With a diligent understanding of operator behavior, the systems could be made better at combining and presenting relevant and valid information for the current task, as well as automating tasks, thus easing operator’s task fulfillment [7].

The purpose of this paper is not to provide design guidance, but rather a way to quickly gain insights. The combination of eye tracking and guerilla approach is, to our knowledge, uncommon in industry settings. Eye tracking can give a lot of qualitative and quantitative information on comparable interfaces. But instead of studying a single machine type, to draw more objective conclusions, we targeted diversity, to get input on the general approach in industrial vehicle settings. This is a big area in itself, as only in the area of advanced agriculture and construction there are over 800 K vehicles created yearly [40]. We also believe that the combination of guerilla and eye tracking can be considered universal and useful in many settings needing a lightweight approach.

Additionally, we perceived eye tracking as a door opener to approach users, making it interesting and easy for the operators to accept our request. This, in turn, gave good informal discussions. It shall be noted that social context, gender, organizational structure, culture etc. could affect the accessibility of working sites as well as the openness of operators and would be interesting to study further.

Availability of affordable eye tracking equipment and light weight processes opens up for new ways to acquire information. Not only could more research organizations do first person studies, it could also be used to expand the potential sample ground for product development teams. For example, a global sales and support organization with access to eye tracking equipment could provide first person views from a diverse global user group and more detailed information about occurring issues.

However, the use of non-trained usability professional doing operator recordings may open up to ethical considerations, as recording data from real operation could impose personal or professional issues. Common guidelines, such as the principles of research ethics, states that participants in studies should be informed of the purpose, be asked for consent, be handled with highest possible confidentiality, and, that the result should only be transferred or used by others that commit to the same principles as the person(s) performing the study [41]. One implication in this regard could be that persons with limited training in user studies might not follow the ethical guidelines. Establishing a formal structure, including consent forms etc. can be one solution. Though, even in professional settings this is not commonly used. Only 45% of usability professional have participants to sign audio/video consent forms [42]. Also, having a process that is heavy on the formality, may harm the objective of evaluations that are minimal-intrusive and easily performed. As it will interfere with operators work and they could potentially be deterrent from participation by “legal disclaimers”. It can also be difficult to guarantee that recorded data gets handled in a de-personified way when sharing information.

In this study, the use of vehicle information systems was rather limited according to the gaze patterns recorded, less than expected in advance. This may partly relate to the activities being performed by the operators, an assumption supported by the conversations with the studied operators. The harvester operator, both types of excavator operators and the mobile crane operator all said that the use of vehicle information systems differs depending on the task performed. Furthermore, they described that the display usage was relatively low for the tasks performed during recording.

Nevertheless, as indicated by the study, efficient vehicle information systems are highly valuable in certain situations. For example, to get specific information, such as an indication of the correct height of ground when using the excavator. Or, to find out what is going on when the automation systems aren’t behaving as expected. Additionally, the information systems are used for increased visibility, for example when reversing or when the crane operator had limited visibility.

The design of the user interfaces, as well as cabins, is in this sense a vital factor for efficient interaction, as possibly indicated by the utilization of the reversing camera between the dump truck and wheel loader. This difference in usage may also affect the ergonomic situation for the user. Edger et al. points out that need for the operator to look in extreme angles causes operator injuries [15]. Using cameras or additional sensors to extend visualization around the vehicle could assist the operator, not only in increased situational awareness but also in terms of less operator strain. Given that the cabin is appropriately designed to support usage.

Looking into the future, new technologies, such as see-through interfaces and augmented reality, could also increase productivity, visibility, and placement of information. For example, placing information in line of sight using Head-up-displays has indicated benefits in information exchange and productivity for heavy vehicles [2]. But for these new means of interaction, there is still a strong need to understand the user attention and the task performed in daily operation.

7.1 Lessons Learned from the Setup and Usage

As we learn from the mistakes made and the way we overcome problems, a section of practical lessons learned is included below.

It is recommended to have a lightweight recording computer with a touch display as it eases operation in tight cabin spaces. The initial approach was to use an Intel Compute Stick as recording computer, together with a Wi-Fi connected tablet for calibration and configuration. This would give a small and lightweight solution. This approach was however disbanded, because of low processing power and connectivity issues with the tablet. After several tryouts, a working setup was established using a touch screen enabled laptop computer running Linux.

The recordings were saved to a USB stick. This enabled easy transfer of information to the analysis computer as well as the possibility to add more storage when needed. During the initial runs, the software crashed after a longer period of usage, resulting in corrupt recordings. This was related to the fat32 file system, used per default on USB sticks. Reformatting the USB sticks with a file system supporting a larger file size limit solved this issue (NTFS). A separate hard drive would have been another option. But this makes the solution more bungle some. The limited power supplied from the USB outlet was also an issue, causing the glasses to turn off. The use of a USB cable with two ports for power consumption solved this issue.

Sometimes the recordings showed a slightly staggered position in regards to what could be considered the operator’s visual focus. This was probably due to the working environment, where the operator is exposed to shocks and vibrations, resulting in the glasses slight movement on the operator’s head. In these cases, the eye fix position was retrospectively recalibrated against what could be considered credible focal points, by manually adjusting the offset in the X and Y axis plane. For more detailed tracking, for example, to track exactly what was read on an operator screen, this process of recalibration could raise uncertainties on data validity and a more secure way to fit the glasses on the operators head would be needed.

The CPU performance required for video recording and gaze tracking is rather high, thus quickly draining the computer battery. For longer recording sessions it is advised to have some sort of charging equipment that can be attached to the machine. It was however not used for these evaluations, as it would have added more time and complexity during the setup phase.

The optics on the world camera provide a good field of view, still, the visual area to cover is even bigger. Additional cameras added at strategical places could provide additional useful information. For example, detailed visibility of display content, even when the user looks in another direction. Or, how the controls are used by the operator. The drawback, again, is increased vehicle setup effort as equipment needs to be mounted in the cabin. This would also extend the time needed for analysis because several sources of information would have had to be stitched together.

8 Conclusions

The paper presents a way for designers to gather information using relatively affordable equipment and a method of acquiring data that doesn’t interfere with the subject of study. This was done by combining the guerilla HCI approach on heavy vehicle user studies and relatively cheap eye tracking equipment. This combination of approaches within the industrial setting is uncommon in literature and shows how practitioners can quickly acquire real field data, with limited interference in ongoing work. The experience report demonstrates that it is possible to observe an operators work and attention, even when it is not possible to be physically located together with the operator in the cabin. The limited sample spectrum for this specific study makes it difficult to ground design directions. But it indicates the usefulness of quick sample studies to gain knowledge and identify areas for further observation, or limitations in existing designs. For example, the difference in use based on display placement, how interaction minimizing focus on the screen can provide useful assistance, or how limited visibility might affect the use.

Moreover, the study as such contributes to the understanding of operator use of vehicle information systems and how the tracking result can be used to learn about operator behavior and attention in addition to the statistical data than can be extracted. While sheer numbers of HMI display usage show low levels, the systems were used in certain situations, for example when the machine was not performing as expected or for reversing camera usage. Also, cabin design is something that might prohibit vehicle HMIs where, for example, placement of displays out of sight of operator attention area may result in information being left unattended.

9 Future Work

As a continuation of this work, it is of interest to perform a larger analysis with more vehicles and operators and more objectives for measurement. It is also of interest to refine the method and technical approach. For example using visual tags to automatically detect areas of interest, making it easier to analyze longer sessions. Or integrating additional sensor or cameras to further enhance quality, while still facilitating a quick and easy setup. It is also of interest to target the studies to specific activities that are more information system related, for example, when making machine configuration and administration, or when performing precision control.