Keywords

1 Introduction

Distracted driving contributed to an estimated 421,000 accidents in the year 2012 [8]. It is estimated that 9 % of drivers are using a cell phone or manipulating an electronic device while operating a motor vehicle. According to the National Highway Traffic Safety Administration, this is almost 660,000 drivers that are driving distracted at any particular time [15]. The “100 Car Naturalistic Study” found that secondary tasks were performed during 40 % of all trips [8]. Secondary and tertiary tasks are activities that are not immediately important to the act of driving - including eating, using a cell phone, talking with a passenger, and interacting with in-vehicle infotainment systems (IVIS). Distraction contributed to 78 % of accidents or near-accidents in the 100 car naturalistic study [8]. This study was conducted prior to the more recent increase in touchscreen interfaces, which contribute to longer periods of time that the eyes are off the road. It is difficult to find accurate data on true causes of distracted driving accidents, but one can muse that the IVIS touchscreen displays play a role.

A new study by Virginia Tech Transportation Institute (VTTI) and Insurance Institute for Highway Safety (IIHS) examined the relationship between cell phone usage and accidents or near-accidents. Not surprisingly, drivers were 17 % more likely to be involved in an accident or near-accident when reaching for, interacting with, or talking on their cell phone. It is interesting to note, however, that while talking on the phone, drivers had their eyes off the road only slightly less than their baseline driving. This stresses the importance of cognitive load and begs the question of when distraction was highest - and when the accidents or near accidents occurred. The researchers also took this a step further and found that drivers were unlikely to engage in other distracting behavior when using their cell phone. When not using their cell phone, however, drivers still engaged in distracting behavior. The biggest distracting behavior besides cell phone use was talking with a passenger [9].

All of the distractions in the vehicle prompt research questions such as “What is causing distracted driving and how can we prevent it?”, and “What technologies can be implemented to lower distracted driving?”. To answer these questions we decided to conduct user studies. The studies must be effectively designed so that not only can researchers have appropriate metrics to measure, but also have tangible data to apply to future technologies and studies. The more common method for conducting driving studies is to use a driving simulator. The simulators are typically relatively cheap, portable, and have minimal hardware installations. Some simulators, such as the National Advanced Driving Simulator in Iowa, are much more involved and help to mimic actual driving [7]. These expensive technologies include the movement of an actual car, 360-degree views, and give a nearly-perfect experience of actual driving. The driving simulators are able to provide some insight to how drivers will respond to distractions however all driving simulators currently share the same limitation, it is not truly driving. Thus ideally, to truly observe driver road behavior a study should include an actual car, on a real road, to most closely exemplify the driving experience. This is known as a naturalistic study.

When setting out to create our naturalistic driving study, we looked at others who have tackled the problem. It was surprising to find that there had been little research regarding naturalistic driving study design principles and guidelines to set standards. Within distracted driving research, we encounter standards in everything - subjective measures (Likert) of feeling distracted, Lane Change Task (LCT) to calculate lane deviations, International Organization of Standards (ISO) standards, etc. A number of naturalistic driving studies have been conducted, but little in terms of standards are available to help streamline the validity of these studies. As such, we decided to compile a preliminary list of naturalistic driving study principles based on the literature and conduct a pilot study to put them to the test. We present the lessons learned from our study in an attempt to create a set of standards to be adopted by all others conducting a naturalistic driving study experience.

2 Literature Review

2.1 Distracted Driving

The issue of driver distraction has been a growing problem in recent years and is still a serious problem today, resulting in numerous accidents and road fatalities [2, 5]. Driver distraction is viewed as the diversion of attention from activities that are necessary to safely operate a vehicle [5]. In the VTTI study, 100 cars were unobtrusively equipped with sensors and video cameras for 12–13 months. The study showed that almost 80 % of all crashes and 65 % of near-crashes involved the driver looking away from the roadway right before the crash [8]. Therefore, it is evident that many accidents occur when there is some form of distraction away from the forward roadway. The issue of distracted driving is a daunting task that is extremely difficult to completely study and address considering the plethora of distractions in the vehicle.

2.2 Laws and Responses

Reducing the use of mobile devices while driving has been the major focus in the effort to address distracted driving. As phone prevalence and use in the car has increased, governmental agencies and regulators have attempted to enforce texting while driving and phone use bans in some states in the US [4]. This was predicted to stop or scare drivers from phone use in the car. It has become apparent, however, that prohibiting drivers from texting has led to more distractions and accidents [1, 6]. Some of these bans actually increase the number of accidents on the road since drivers who use their phone while driving make the effort to hold their phones down and out of the sight of police officers, which essentially reduces their peripheral view thus leading to more accidents. It is very important to address the drivers needs while keeping them safe and less distracted from the primary task of driving [1]. As it become apparent regardless of laws and repercussions we cannot directly stop drivers from engaging in tasks deemed as distracting, many entities have moved to how do we provide them with the technologies that will safely assist them in accomplishing their intended tasks in a seamless manner.

2.3 Solutions

These revelations have prompted several entities, including car manufacturers, mobile app developers, etc., to dive deep into alternatives, such as eyes-free, hands-free technologies. There are numerous solutions that aim to limit driver distraction with respect to phone use in the car. A few of these applications include DriveSafe.ly, Cellcontrol, DriveScribe, tXtBlocker, Voiceing among many others. Some require users to look at or touch the phone, while others have a hands-free, eyes-free driven interface. Other applications aim to block phone use while the vehicle is in motion. As a result of the handheld phone use bans in some states, this has led to many of these hands-free, eyes free applications where users can use their voice to send and receive messages and applications that block some phone activities in a moving vehicle. Research has shown that applications such as Voiceing are one of the few applications that truly support hands-free, eyes free. Some applications have hands-free features but still require the user to touch the device at specific times. Voiceing, previously called VoiceTEXT is a hands-free, eyes free application developed by researchers at Clemson University that allows users to send and receive messages using their voice [10]. The advantage that Voiceing has over many other voice applications is that transcription from human voice to text is not the main focus of the application. The Voiceing system works by sending a recording of the sender’s actual voice to the recipient of the message. The transcribed text is also available for later viewing if necessary. The notion of emphasizing on the recording of verbal messages is the key difference with Voiceing. In addition, some of these other applications read emails, texts and other incoming media out loud to the driver. They also allow drivers to utter a response and transcribe the message in order the limit the need to manually input text with the phone. These applications aim to limit distraction and touching the phone while operating a motor vehicle.

Despite the technological advances, a recent study sponsored by AAA Foundation for Traffic Safety claimed that even hands-free, eyes-free solutions to texting and tertiary tasks was still mentally demanding and hindered driving performance [12]. As more and more weigh in to address the current issue of distracted driving attributed to mobile phone use along with IVIS steadily increasing in number, the need to study and develop feasible strategies to keep drivers safe is dire. Yet, there currently exist little guidance in terms of standards for designing a naturalistic driving study to assess this phenomenon.

3 Road Study Design: Simulator vs. Naturalistic Study

Numerous studies are conducted across the country to explore the impacts of secondary tasks on driving performance. The majority of these studies are done with the assistance of a simulator. Simulator studies have competitive advantages to naturalistic studies; the biggest factor is safety. Simulator environments can be controlled, the simulator software records data, and it is much easier to analyze and replicate the experiment. A more direct method to evaluating the driving experience, however, is captured by naturalistic studies in the car. A naturalistic environment entails studying a driver in as much of their natural driving state as possible. In this natural environment, there are many factors not present in a simulator study: limited repeatability, uncontrollable variables, crashes and injury to participants, data prone to noise that do not perform well in varying conditions, etc. [7]. Because of these limitations, along with the high risk-low fidelity nature of naturalistic driving studies, such studies are few. This can be problematic for researchers designing naturalistic studies for the ever changing landscape of IVIS and other secondary in-car tasks performed by drivers as there is little guidance to the process. In addition, there are also numerous challenges transferring knowledge gained from research into effective practices [3].

Simulator studies involve the use of a steering wheel, accelerator, and brake. Some systems are more advanced and have an entire cabin of a car incorporated to the design. A lower-budget system has a single front screen that shows a driving environment and reacts to driver input. Advanced systems like NADS are an entire 360° view for a much more realistic experience. The simulator studies have a secondary task, like Lane Change Task (LCT) that attempt to mimic real driving behavior. Simulator studies are very easy to control. The conditions are always the same since it is an indoor environment, the road surface is exactly the same, and even the tasks can be manipulated to the exact same moment for every participant. There are very limited confounding variables that would influence results since simulator studies allow so much researcher input. The naturalistic studies have many other variables that can cause difficulties, as we discuss below.

4 Design Principles

Due to the high-risk nature of a naturalistic study and the amount of resources, logistics and manpower required a detailed design strategy is a necessity. There are many elements that researchers must explore in planning and designing a naturalistic driving study. The principle design guidelines that we will focus on below, will place emphasis on the study design and data collections as those were the sections we found more challenging to navigate as we designed our naturalistic driving study. As the origins of most user studies are derived from a question, we will begin our exploration of design principles with the research question. The 100-Car Naturalistic Driving Study [8] is one of the most cited naturalistic road studies. As such we use [14] “Design of the in-vehicle driving behavior and crash risk study: in support of the SHRP 2 Naturalistic Driving Study” as a benchmark to guide the design principles we put forth. Below we will present some of the design principles we found in the literature along with our pilot design, which was our attempt to implement these principles in a small scale study.

4.1 Research Question

Research questions guide the process of a naturalistic study design. Strategic planned research questions will allow for data produced in the study to answer these questions within the known project constraints and limitations such as funding and time. In a naturalistic driving/road study, research questions can be organized in at least 11 different categories [14] (see Table 1).

Table 1. Research Question Category
Full size table

The research questions we developed for our study fall within the Infotainment-system-based and nomadic-device-based category:

  • Of the three following mobile telecommunication methods: manually texting, voice texting (Siri/Vlingo), and Voiceing, which offers a less distracting form of communicating while driving?

  • Is Voiceing less distracting form of communication for a driver than voice texting?

  • Is Voiceing less distracting form of communication for a driver than manual texting?

4.2 Design Plan

Reference [14] paper defines the design plan by two broad areas the participant sample population and the data collection site. In addition we have included the study content.

Participants Sample Population. In determining the sample population there are several factors to consider in addition to a standard study design. In recruiting participants you must weigh the pros-and cons of using a representative versus risk-prone sample. So for example participant sample population you will have to consider the correlation age to traffic infractions as it relates to research question. In addition the sample population will also affect the vehicle model selection as the effects of vehicle selection to traffic infractions as it relates to research question (sedan vs. truck): passenger cars (sedans, coupes, hatchbacks, and station wagons), pickup trucks, sport utility vehicles (SUVs; including crossover vehicles), and minivans. If your study resources allows for participant compensation the total cost of data collection versus the cost per data-year and the cost per participant versus the ability to enhance participant attraction and retention via meaningful compensation amounts has to be considered.

Data Collection Sites. In a naturalistic road driver study you want your data collection site to resemble the natural environment in which your study is situated as much as possible. There are several things that factor into the data collection site selection to determine feasibility such as geography, weather, state/county law, road types, obstructions, land usage (e.g., urban versus rural) etc. Thus the [14] paper recommends the use of a data collection site coordinator. If doing a naturalistic study within an entity with limited resources it is not always feasible to have a site contractors as used and described in the VVTI study. However having a dedicated team member to fulfill similar responsibilities, we later discovered, is essential. Selecting and securing a data collection site proved to be a challenge in our study. Because our study was a pilot we choose to do one without a coordinator. Due to our location there were limited road options that fit the outlined study requirements in terms of road distance, location, traffic activity, etc. However once we were able to identify roads that would suitable, several additional barriers were encountered as we worked to secure access to a road, included the state passing a no texting while driving law. In our study, the data collection site used, a low traffic rural road lined with trees homes and shrubs, revealed a major flaw in our study design. The rural road along with heavy tree coverage gravely impacted cell reception for several of our study participants. Thus proving to be very problematic for our study as participants struggled to receive and send text’s.

Content. Many naturalistic studies look to evaluate drivers’ road performance with varying environmental conditions. In our study we were interested in studying the distraction caused by varying methods of texting. Thus for this we used a texting script used by the researcher and the participant. It is important that the length, style and subject matter of the text was perceived as realistic communication amongst our targeted audience. Figure 1 below is an example of one such exchange.

Fig. 1.
figure 1

Sample text dialog from a distracted driving study

Full size image

Data Collection. Driver Demographics and Vehicle Inventory.

Driver demographic collected would vary based upon research question and target demographic, however, this will not be very different from the type of driver demographic information a researcher might collect for a simulated study. Most naturalistic driving studies are conducted from vehicles that are similar to those statistically in use among targeted demographic in the studied user case. For example in our study, distracted driving traffic incidents that have occurred more frequently were in sedans. Ideally, for a naturalistic study it is best for participants to use their own vehicle because it reduces the amount of variability in the results that can be attributed to the lack of familiarity to the vehicle. However because our study was a pilot, with limited resources we rented a sedan. This was to reduce the chances for error and time to set up and take down the study instrumentation in participant’s vehicles. In our study, the majority of drivers drove a sedan so the learning gap should not be that large as a result of direct manipulation. Direct manipulation is a well-known Human-Computer Interaction principle [11] and a good e.g. is the task of driving. The basic driving task is relatively easy to understand regardless of vehicle or country as a result of it being a direct manipulation task. For example, there are specific critical driving functions a driver can perform in every vehicle (drive forward, turn left, turn right, reverse etc.). Therefore driving is transferrable across vehicles as a result of direct manipulation. It is obvious that the notion of IVIS and additional vehicle features may lead to a completely new and much more detailed discussion, but in reference to the basic task of driving, this is very similar across most vehicles. Since distracted driving was a primary interest to the study, in addition to collection basic vehicle inventory information such as make, model, safety features, etc. we were also interested in infotainment features the participants had from previous experience.

Driver Assessment. Regardless of the research question, driver assessment is an important step in a naturalistic driving study. Understanding study participants driving functionalities under best-case scenario serves as a control and a baseline to gauge and adapt study results per participant. Because of the heighten risk factors associated with naturalistic driving studies driver assessment also allows for a means for participant selection as you can target individual differences or impairments as they relate to cognition; visual perception; various visual–cognitive, physical, and psychomotor abilities; personality factors; sleep- related factors; medicines and medical conditions; driving knowledge; and history. Depending upon the nature of the study a researcher might choose to categorize and separately administer Visual Perception Assessments, Visual-Cognitive Assessments, Cognitive and Psychomotor Assessments, physical Assessments.

Data Acquisition System. As previously stated naturalistic study design like most studies are driven by their question. As such to help illustrate a setup method we will describe our set up below. In a four door sedan, we used four GoPro cameras to provide visual feedback (Fig. 2), a dashboard camera in a separate surveillance vehicle, a steering column camera facing the speedometer, a camera on the back of the rearview mirror facing the road ahead (to measure lane deviation) and finally a dashboard camera facing the driver. An audio signal that could be heard on all cameras was used to sync the cameras’ for each study run for the purpose of data analysis. In simulated road studies visual stimulus, such as road barriers or signs to adjust speed are frequently used to detect moments of distraction, and assess when the participants cognitive load is being impacted such as in the case of inattention blindness. Inattention blindness occurs when an observer fails to see an unexpected object or event in plain view because their attention is diverted to something else. Thus they can look at, but not see an object or event. We used three Adafruit NeoPixel Digital RGB LED strips lights that were programmed to flash randomly in the vehicle. In the vehicle the strips were set such that the driver could see all three strips, without having to turn their head, equally using peripheral and standard vision. One strip was placed across the front dashboard, the second to the left side of the vehicle and the third on the right side of the vehicle (Fig. 3). Study participants were instructed when they saw the LEDs on either side of the vehicle flashed green to slow down and engage the turn signal for that respective side of the vehicle. Participants were instructed when they saw the LED strip in the front flashed red they were come to a complete stop. The LED strips were programmed to flash while the participant is engaged in a conversation via texting using both voice texting and manual input.

Fig. 2.
figure 2

Data acquisition system

Full size image
Fig. 3.
figure 3

Adafruit NeoPixel digital RGB LED strips light set up (Color figure online)

Full size image

5 Discussion

There were several challenges encountered from our guided pilot study that will provide feedback in how we design future naturalistic driving road studies. One of the most problematic of the challenges we encountered was phone reception on out test road. We decided to have participants use their personal mobile phone to reduce the amount of variability/unknown in the data collected that could be attributed to the participant using a phone they were not accustomed to. This correlated with the test road selected resulting in several participants having poor to no cell reception along the test road. As a result of some quick thinking by the team we were still able to get some good data by timing the sent and received text to correspond with moments of coverage. Another challenge encountered was with the design of our stimuli, and their placement in the vehicle. For example, our stimuli were visual and we conducted studies over the course of a six-hour day we needed to better factor in the position of the sun and visibility. It would have been beneficial to the study to do a pre-pilot study just on the positioning of the LED strips in the vehicle. Lastly, due to several adjustments during the course of the study that required a bit more time than originally calculated, we ran into a challenge in charging the cameras while out in the field. After the first day of the study the team adjusted by scheduling more dead time between study participants to allow for ample time to charge the cameras. There are, however, many other ways we could have redesigned the study to avoid these problems in the first place. Guidelines for designing a naturalistic study would have surely helped our team execute the study more effectively from the start.

6 Conclusion

From our research and attempts to conduct our naturalistic pilot study there were several lessons learned. Not only lessons, but also we have identified a set of guidelines that may be very helpful to those conducting or planning to conduct a naturalistic driving experiment. Extensive pilot testing is one thing that became very evident in the process of trying to conduct a pilot study. We mentioned the issues with phone service, the sensitivity with LED lights and issues with the GoPro’s keeping the charge. Even though we conducted numerous pilot tests, these are events that would not necessarily show up until later on. This one is almost self-explained, but the more testing the better, and researchers will also need to think outside of the box even more when designing a naturalistic study. For e.g. even though two phone carriers worked during pilot test, it was important to potentially examine the phone reception of all major phone carriers on the test track. From our design principles, it is clear there are numerous factors that researchers need to be aware of when conducting naturalistic experiments including, but not limited to participant sample population, driver assessment and the data acquisition system. These guidelines and tips presented in this paper are not meant to be a definitive solution for every single experiment, but a guide for researchers in this space, that may help to limit some of the potential problems that could arise when conducting a naturalistic driving experiment.