## Abstract

In this study, we propose an evaluation method for an Advanced Rider Assistance System (ARAS) for two-wheeled vehicles, combining riding simulator experiments and computer simulations in terms of cost-benefit analysis. This evaluation method focuses on the collision warning system at intersections using an ARAS for two-wheeled vehicles. The study was carried out experiments with 30 test subjects who have two-wheeled vehicle licenses and are not novice riders. To quantify the accident-reduction effect, a Monte-Carlo simulation based on a time-series reliability model was used. Based on the collision probability results derived from the Monte-Carlo Simulation, the overall error probability as a human-machine system was calculated based on an integrated error model. In addition, cost-benefit analysis was conducted to quantify the social benefits and costs of introducing the ARAS to the market. As a result, we confirmed that the system can be beneficial after 4 years when introduced into the market.

## Introduction

Advanced Driver Assistance Systems (ADAS) for four-wheeled vehicle safety such as Forward Vehicle Collision Warning System (FVCWS) and Lane Departure Warning System (LDWS) are already in practical use. Likewise, ADAS using Information and Communication Technology (ICT) such as Vehicle-to-Everything communication (V2X) are currently being developed [1,2,3]. Owing to the spread of such active safety systems, collision accidents of four-wheeled vehicles are decreasing year by year [4, 5], yet collision accidents of two-wheeled vehicles are still occurring with high rates [6].

In the case of two-wheeled vehicles, Riding Simulators (RS) have also been developed to train novice riders and to analyze riding safety [7, 8]. Since 2008, Advanced Rider Assistance System (ARAS) and On-Bike Information Systems (OBIS) have been actively studied by the SAFERIDER project [9] for two-wheeled vehicle riders using RS. The SAFERIDER team studied five key ARAS functions: velocity alert, curve warning, frontal collision warning, intersection support, and lane change support. However, there is little previous research on ARAS for two-wheeled vehicles [10, 11], and it is not an active field compared with studies for four-wheeled vehicle safety. Thus, even though the vehicle dynamics and severity of injury in collision accidents are significantly different from four-wheeled vehicles, ARAS evaluation methods for two-wheeled vehicles have not been studied. Therefore, we are focusing on an evaluation method for upcoming ARAS for two-wheeled vehicles. This study focuses on the collision warning system of an ARAS for two-wheeled vehicles, and evaluates the accident-reduction effect through combing RS experiments [12, 13], computer simulation and cost-benefit analysis. The simulation was performed to quantify the accident-reduction effect using a model that can evaluate comprehensive safety as a human-machine system. The accident-reduction effects can be quantified by combining RS experiments and computer simulation. But quantification of collision-reduction effects is not sufficient to evaluate the societal benefits of introducing the system into the market for practical use. Although there are some studies that evaluated the accident-reduction effect of such systems [14], there are few evaluation methods that consider the social benefits. Cost-benefit analysis is typically used in evaluating the public utility of certain services or systems as a predictor which evaluates whether a new service or system will become beneficial to society, or not.

In this study, we quantitatively evaluate the effects of the ARAS not only in terms of accident-reduction effects of the system based on RS experiments and computer simulation methods, but also the social benefit of introducing ARAS to society. To evaluate the usefulness of the system according to the benefits of practical use in society, we propose an evaluation method for ARAS combining riding simulator experiments and computer simulation in terms of cost-benefit analysis.

This study has the following two purposes:

- 1.
Propose a quantification method of total collision reduction effect as a human-machine system through riding simulator experiments and computer simulation.

This purpose aims to quantify not only collision-reduction effect through the usage of ARAS but also to quantify the total reduction of accidents which combines collision due to both human errors and system errors. We developed an RS and investigated the collision-reduction effects of ARAS and riding behaviors (see Sect. 2). In addition, accuracy of quantifying the collision-reduction effects was improved by using a time-series error model (see Sect. 4.1). The total collision accident probability was calculated by quantified summation of collision probability of both human errors and system errors based on the integrated error model (see Sect. 4.2).

- 2.
Propose an evaluation method to quantify the social benefit of market introduction of new ARAS for two-wheeled vehicles using cost-benefit analysis.

This purpose aims to quantify the social benefit generated by practical use of new ARAS for two-wheeled vehicles. For this purpose, we quantify the social benefit and cost. Benefit was calculated as the summation of the estimated value of the reduction of human loss and material loss. Human loss was calculated according to the severity of injury due to collision accidents based on simulated collision velocity (see Sect. 4.3). Material loss was calculated based on the average repair cost of vehicles according to each target collision accident pattern. Social cost is defined as the total cost of system introduction in this paper. Finally, we evaluate whether the system is beneficial to society by comparing the cost and benefit (see Sect. 5).

## Riding Simulator Experiment to Evaluate the Accident Reduction Effect of ARAS

### Experiment Device (Riding Simulator)

In order to investigate the riding behaviors and collision-reduction effect on the riders in risky situations when using the system, an RS experiment was conducted. The RS experiment considered variable conditions, scenario manipulation, and session time, with the participation of many test subjects. To carry out the experiment, we constructed an RS which has a main-PC for calculating the vehicle dynamics and simulating the variable scenarios, a sub-PC for transmission of graphics, an electric fan for increasing the reality of driving velocity, and an actual motorcycle frame which can measure the angle of steering, level of acceleration and brake pressure. Figure 1 shows the configuration of the RS which we constructed for this experiment. In particular, for the projection of the road environment, a Head Mounted Display (HMD) was used to increase the reality of driving. We simulated the experiment conditions and calculated two-wheeled vehicle dynamics using MatLab R2013a-Simulink and linked it to Car-Sim 8.2.2. We used “DS-nano” designed by Misaki Design as a platform software for driving simulation.

### Experiment Design for Evaluating ARAS

In Japan, collision accidents of two-wheeled vehicles occur most frequently at intersections, and many collision accidents occur when the two-wheeled vehicle riders are driving on the priority road. Therefore, we focused on collision accident patterns at intersections when a rider is driving on the priority road. We conducted experiments under two different conditions: ARAS is in use and not in use. Figure 2a shows a risky scenario in the riding simulator experiment. The risky scenarios were designed to include an approaching vehicle suddenly coming out from a blind spot of the intersection.

To investigate the negative effects of system usage caused by the rider’s over-dependence on the system, we added a system malfunction scenario for the final session of the experiment. During normal operation, the simulated collision warning system at intersections provides a notification to riders about other approaching vehicles that can’t be seen by the rider. The system notifies riders about approaching vehicles using both a visual icon and an audio alarm. The system provides both audio alarm and a visual icon at the same time to increase the level of the Situation Awareness (SA) in risky situations. The visual icon is assumed to be projected on a Head-Up Display (HUD) while driving on a public road. Icons were displayed at the center of the rider’s view while wearing the HMD. The timing of alarms was set as “Time to Collision (TTC) = 4.8 seconds” according to the approaching vehicle, based on the Advanced Safety Vehicle (ASV) guidelines [15] in Japan. The provided visual icon is shown in Fig. 2b, c and the audio alarm was approximately 500 Hz tone with one time (presenting time = 1.5 [s]) per risky situation.

### Experiment Procedure

We conducted experiments with 30 male test subjects (Mean age = 22.1, SD = 1.47) who have Japanese two-wheeled vehicle licenses and are not novice riders. We recruited the test subjects with at least one years of the two-wheeled vehicle driving experience. The average driving experience of the subjects was 3.02 ± 1.66 years and driving frequency is more than three days a week. During the RS experiments, test subjects were required to drive to the final destination while performing actions of collision avoidance in risky situations. We required test subjects to perform the Uchida-Kraepelin test [16] as a sub-task while driving in order to reproduce the mental workload of actual driving. To reduce the learning effect as much as possible, dummy scenarios with no risk were also included in the same road environment. Table 1 shows the experiment conditions per one test subject to investigate riding behaviors when using Human-Machine Interface (HMI) and not using HMI. In addition, the experiment scenarios were randomized for each test subject. We allowed the test subjects adequate rest time during the experiment when they requested a break. Likewise, the test subjects were given enough rest after completing the experiment to prevent motion sickness due to wearing the HMD during the experiment.

Before starting the RS experiments, we provided sufficient explanation about the experiment and obtained preliminary informed consent. All test subjects joined the experiment after signing informed consent, and the content and experiment process was reviewed and approved by the Experiment Ethics Committee of Kagawa University.

## Investigation Results of Riding Behaviors Using Riding Simulator

We investigated riding behaviors in terms of parameters that are related to the risk of traffic accidents including Reaction Time (RT), Average Deceleration (AD). In this study, RT means the time for the rider to start braking after detecting dangerous objects or risky situations. This is a very common and important parameter in the studies of traffic engineering and traffic accidents. Riders can prevent accidents by shortening their RT. AD indicates the average value of deceleration starting from responsive brake operation until the vehicle stops completely and it is also a critical parameter when braking operation in risky situations. If AD is high, the rider can stop in short time in a short distance. This parameter is highly relevant to the RT. If RT is short, the rider can quickly detect risky situations and stop the vehicle with safe deceleration.

### Result of Reaction Time (RT)

One-way ANOVA was performed to confirm the changes of RT according to HMI usage and at HMI malfunction. The result of one-way ANOVA for RT is shown in Fig. 3. Each group (at with and without HMI, and HMI malfunctioning) has a significant difference (F (2, 147) = 252.055, *p* < .001, η^{2} = 0.774). As a result of the homogeneity of variances, the distribution of RT has not an equal variance (p < .001). Also, the sample size of each condition of RT was unequaled. Therefore, the post hoc analysis to compare the RT of each condition can be used Games Howell, Dunnett T3, Dunnett C, and Tamhane T2 method [17]. The multi-comparison of RT was conducted using Dunnett T3 method to earn conservative results. As a result, when using HMI, the RT is shorter than without HMI (*p* < .001) as the result of post hoc tests (Dunnett T3). The mean RT at HMI malfunction is longer than the other situations, and when we compare with “without HMI” situation, there is no significant difference (*p* = .100). This result also suggests even at “with HMI” the risk of collision will increase if the system does not operate properly.

### Result of Average Deceleration (AD)

One-way ANOVA was performed to confirm the changes of AD according to HMI usage and when HMI is malfunctioning. The result of one-way ANOVA for AD is shown in Fig. 4. Each group (at with and without HMI, and HMI malfunctioning) has a significant difference (F (2, 147) = 67.239, *p* < .001, η^{2} = 0.478). The trend of changing AD according to HMI usage is almost same as RT. We think it is because the correlation between RT and AD is high (In this study, Pearson correlation = .703, p < .001). As a result of the homogeneity of variances, the distribution of AD has an equal variance (*p* = .687). Also, the sample size of each condition of AD was unequaled. Therefore, the post hoc analysis to compare the AD of each condition can be used Fisher’s LSD, Scheffe, Bonferroni, etc. for post hoc analysis of AD [17]. The multi-comparison of AD was conducted using Scheffe method to earn conservative results. When using HMI, the AD is lower than without HMI (*p* < .001) as the result of post hoc tests (Scheffe). Slow deceleration makes the long braking distance, but the test subjects are able to stop the vehicle safely because they are more aware of risk than without HMI. When HMI malfunctioning, the AD is higher than when with HMI (p < .001), but there is no significant difference compared to without HMI (*p* = .989). Fast deceleration allows riders to stop the vehicle in a short time in a short distance. However, in the experiment, many test subjects have collided because they have not enough time and braking distance due to long RT.

## Simulation to Quantify Collision Probability and Collision Velocity

Computer simulation of riding behaviors using experiment results data was carried out to compensate for a statistically insufficient amount of data from the riding simulator experiment. We also quantified the total collision probability, considering both the collision probability related to human error and the collision probability related to system error. Figure 5 is a flowchart showing the riding simulator experiment and the computer simulation.

This process consists of four steps. First, we investigated riding behaviors of test subjects in RS experiments. Second, we performed Monte-Carlo simulation based on a time series reliability model to quantify collision probability and collision velocity when riders are in a risky situation. Third, we estimated the total collision probability based on the integrated error model as a human-machine system. Finally, the fourth step estimated the injury level of the riders in case of a collision accident according to the total collision probability and the collision velocity.

### Estimation of Collision Probability Using Monte-Carlo Simulation Based on Time-Series Reliability Model

We carried out Monte-Carlo simulation based on a time series reliability model [16]. In this study, the time-series reliability model represents the rider errors and environmental errors on a time-series axis. If these two statuses (rider & environment) are both “error” at the same time, a collision accident is judged to have happened. Figure 6 shows the concept of our time series reliability model. We conducted a Monte-Carlo simulation based on the time series reliability model 100,000 times to each experiment condition (With HMI, Without HMI, HMI malfunction) to quantify collision probability and collision velocity. The time series reliability model can quantify the collision probability using RT, which is the time latency between recognition of a risky situation and braking, and Time for Collision Avoidance (*T*_{ca}) based on the experiment data such as RT, AD, and etc. *T*_{ca} represents the maximum time margin during which braking operation must be performed to avoid a collision. In other words, a collision accident will not occur if the RT is smaller than *T*_{ca}. The *T*_{ca} was estimated using the following formulas. Here, *x*_{1} is the distance between vehicles when the riders recognized the risky situation and the ego vehicle starts braking, and *x*_{2} is the distance traveled from when the rider starts deceleration until the vehicle stops completely. *D* is the total distance from the position of the ego vehicle when the rider recognizes the risky situation until the collision point. The initial velocity of the ego vehicle is *v*_{0}, *a*_{1} is the deceleration until the start of braking operation, and *a*_{2} is AD during braking operation.

Before performing the simulation, we pre-processed the data with appropriate probability distributions for RT and *T*_{ca}. Collecting statistically reliable data is very important, and the reliability of data is highly related to the amount of data. Statistical reliability requires a large amount of data, but obtaining such in a RS experiment data requires a lot of time and cost. So, we processed the riding behavior data from the RS experiment to improve the reliability of data through increasing the amount of data with the appropriate probability distribution. When examining the state variables to explain each subject’s driving behavior, they will show statistically normal distribution if we obtained more than 50 data based on the central limit theorem (CLT), but there were not many data that followed the normal distribution. This happened because some properties have different distributions such as Lognormal, Beta, Weibull distribution, etc. Therefore, assuming a single probability distribution when performing computer simulations can output strange results. So, we tried to apply an appropriate probability distribution for each state variable in the simulation.

### Estimation of Total Collision Probability as a Human-Machine System Based on the Integrated Error Model

To evaluate the accident-reduction effect including system malfunction situations, we used an integrated error model [18] which can consider human error and system error simultaneously. Figure 7 (left) shows the concept of an integrated error model which calculates the total collision probability without using the system. Figure 7 (right) shows total collision probability with the system. Here, system error indicates the probability of system malfunction and human error indicates the collision probability caused by human behavior. The reason for including the system malfunction scenario is that we need to consider both human and system errors to evaluate the collision-reduction effect. In this study, we assumed the error probability of the system is 1% according to results of previous research [19]. But actually, a 1% system error probability is unrealistically high. Through the assumption of a high system error probability, we can evaluate the system with greater strictness than standards for system evaluations such as the SIL-3 of IEC 61508 or the ASIL-D of ISO 26262.

Here, *Es* is the error probability of the system, *Er*0 is the error probability of the rider without using the system, *Er*1 is the error probability of the rider with the system (human error), *Er*2 is the error probability of the rider when the system is in a state of error (system error: malfunction). Therefore, Area A means the human error probability when the system is normal, and Area B means the error probability when the system is malfunctioning. Then, the total area of A and B indicates the total collision probability caused by both human error and system error. Finally, the overall error probability as a human-machine system is calculated using the following formula.

### Estimation of Collision Velocity Using Monte-Carlo Simulation

In addition to estimating the collision probability, the collision velocity was also simulated to estimate the degree of injury to the riders if a collision accident occurred. The collision velocity was also simulated using the Monte-Carlo simulation method, and the simulation was performed by random sampling from the probability distribution data of subject riding behavior when riders were in a risky situation during the experiment. Eq. (9) is the formula used in the Monte-Carlo simulation for estimating the collision velocity. Eq. (7) is the formula for calculating the velocity of the ego vehicle when the collision occurred. Eq. (8) is the formula for calculating the velocity of the other vehicle when the collision occurred.

Here, *v*_{0} means the velocity of the ego vehicle before the rider starts braking. Next, *a*_{2} is the average deceleration from when braking starts until the vehicle stops completely. The alarm timing means the system notification timing for alerting the riders to a dangerous situation, and RT means the rider’s reaction time until braking operation starts. In these formulas, *V*_{1} means the velocity of the other vehicle. When the approaching vehicle doesn’t brake in the risky scenario, the vehicle’s velocity at the time of collision was simply input to the equation for estimating the collision velocity.

## Cost-Benefit Analysis to Evaluate ARAS as Social Benefit

Generally, cost-benefit analysis is used to evaluate large-scale matters such as public utilities and political decisions like environmental policy [20, 21]. Through this analysis method, we can confirm whether the analysis subject has social benefits or not, in terms of the benefit ratio (benefit / cost). Likewise, we can also evaluate the newly proposed ARAS from a social standpoint. Therefore, we propose combining methods to include riding simulator experiments, computer simulation methods, and cost-benefit analysis to evaluate the accident-reduction effect from a social perspective.

In this study, “cost” represents the assumed price of an ARAS multiplied by the number of two-wheeled vehicles in Japan. We considered that introducing the system would reduce the number of traffic accidents, therefore reducing both human loss and material loss. This total reduced loss (human loss + material loss) is regarded as the “benefit”. The cost-benefit analysis conducted in this study has four steps.

Figure 8 shows a flow chart of cost-benefit analysis that follows the computer simulation steps in this study (Fig. 5). First, we investigated accident data. Second, we calculated the benefit according to insurance data and severity of injury. Third, we calculated the cost of system introduction. Finally, the fourth step is comparing the cost and benefit.

Step 1: Investigation of accident data.

The target accident pattern of the system used in this study was collision accidents at intersections, so we conducted a survey focusing on accidents that occurred at intersections while two-wheeled vehicles were traveling on priority roads based on the statistical traffic accident database as reported by the Institute for Traffic Accident Research and Data Analysis (ITARDA) [22] in Japan.

Step 2: Calculate the benefit according to insurance data and severity of injury.

To quantify the benefit, we estimated the amount of human loss and material loss that will be reduced by introducing ARAS into the market. Human loss was calculated by estimating the rider’s injury severity level based on collision velocity [23]. Severity is usually divided into 4 stages from S0 to S3. S0 is defined as no injuries, but S0 is not applicable in accidents involving two-wheeled vehicles. This is because two-wheeled vehicle riders are most likely to suffer injuries when accidents occur. Next, S1 is defined as mild injury, S2 is moderate injury, and S3 is severe injury. To estimate the severity of injury, we calculated human damage due to accidents based on automobile insurance data [24]. We used this data to calculate the amount of insurance money paid per person for each level of severity. In the insurance data from the general insurance association of Japan, we could find the total insurance payment of each injury severity level (From Severity 1 to 3) at a year. However, the specific amount of insurance payment for each severity per person was not found. So, we simply divide the total insurance payment from the number of the accident at a year. The dividend amount was assumed as an average insurance payment to a person when the accident occurred for each severity level. Table 2 shows the relationship between collision velocity and injury severity level according to a previous study [23] and the human loss of each severity level was calculated as the average amount of insurance payment per person of the severity level for the year in the insurance data [24].

Step 3: Calculate the cost of system introduction.

In addition, it is necessary to consider not only human loss but also material loss. Material loss was calculated based on the average repair amount of various two-wheeled vehicles. In this paper, the average motorcycle repair cost is set as JPY 600,000.

Step 4: Compare the cost and benefit.

Next, by combining the material loss and human loss, we can calculate the total loss. Reduction of this loss through introducing ARAS is the benefit. In other words, to calculate the total benefit, we multiplied the number of accidents for each level of severity and the human loss for each level of severity, both with and without using the system. We also calculated the material loss by multiplying the average repair amount by the number of accidents under each condition, both with and without using the system. Material loss of each condition and human loss are added to calculate the total amount of damage under each condition. Finally, the difference between the total damage amount with and without using the system is the benefit of introducing ARAS. We also considered the cost required for system installation, defined as system cost for society.

In this paper, we assumed that ARAS will be installed in all two-wheeled vehicles in Japan within 10 years. We also assumed the market penetration rate, the discount rate, and system price as required parameters for cost calculation. The penetration rate was assumed to be 10% each year. This rate determines how much the system will spread within the market in one year. The discount rate is also an important factor, because this rate estimates future prices according to the current value. For example, if the discount rate is 4%, a value of $100 will be equal to the current value of $96 twelve months later. In the field of economics, it is commonly assumed that the annual discount rate is 4%, so we also set the discount rate at 4%. System price was based on the following formula (10), and the price includes both the system and necessary infrastructure. We used these parameters to calculate the annual cost required for installing the system, as shown below. Here, *Ac* is annual cost, *Sp* is the system price, *Dr* is the discount rate, and *Ep* is an extension period.

Finally, we compare cost and benefit to clarify whether the accident reduction effect due to system introduction is socially beneficial or not.

## Evaluation Results

### Estimating Collision Probability and Collision Velocity in Computer Simulations

We sampled experiment data in a database of RS experiment results and simulated collision probability and collision velocity using a time-series reliability model to calculate the collision probability. In this step, we simulated collision probability through random sampling to assign values in the equation using the probability distribution which is most similar to values of RT and *T*_{ca} in RS experiments. Results showed that collision probability is 9.0% without using the system, 0.97% with the system, and 11.91% in cases of system malfunction. In the integrated error model to quantify the total collision probability, *Er*0 represents collision probability without using the system, and *Er*1 represents the collision probability while using the system. *Er*2 represents collision probability when the system malfunctions. Next, we applied these values (*Er*0, *Er*1, *Er*2) to the eq. (6) and calculated the overall error probability as a human-machine system. As a result, the total error probability is 1.08%, which is a summation of 0.96% due to human error (Area A) and 0.12% due to system error (Area B).

The simulation for calculating collision velocity was also carried out. Table 3 shows the distribution of collision velocity in relation to each system condition. The severity level is assigned based on collision velocity, and the number of accidents in each velocity range was calculated by multiplying the simulation distribution and actual accident data. In order to estimate the probability of collision while actually using the system, the collision probability derived from the simulation is substituted based on the actual number of target pattern traffic accidents in 2017 (100%), allowing us to estimate collision probability while using the system. According to accident statistics in Japan [22], a total of 15,656 accidents involving two-wheeled vehicles occurred at intersections in 2017. This actual accident data corresponds to the number of accidents without using the system in this experiment. As a result, the human error probability is 10.67%, and the system error probability is 1.32%. Therefore, the overall error probability of both the system and rider while using the system is 11.99%. We applied the simulation results to actual accident data to estimate the number of collisions. Table 3 shows that the number of collisions with the system (human error) would be 1670, with the system (system error) is 207, and the total is 1878.

### Result of Cost-Benefit Analysis

The number of accidents in each velocity range was calculated in this section. Figure 9 shows the comparison of benefit and cost over 10 years of system introduction in the market. Benefit is defined as the decrease of human and material loss due to the accident-reduction effects of system introduction. System introduction cost is calculated according to eq. (10) in Sect. 5, by multiplying the estimated annual system cost and the number of installed systems. As a result, we confirmed that when the system cost is JPY 50,000, the benefit ratio exceeds 1.0 after just 4 years of practical system use. Therefore, system usage provides benefit to society after 4 years.

## Conclusion

In this study, we proposed an evaluation method for ARAS which combines a riding simulator experiment, computer simulation, and cost-benefit analysis. Through our research, we reached the following conclusions:

- 1.
Propose a quantification method of total collision reduction effect as a human-machine system through riding simulator experiments and computer simulation.

We conducted an evaluation experiment for a collision warning ARAS to quantify accident reduction effects through Monte-Carlo simulation using a time-series reliability model. In this simulation, we calculated the reduction of collision probability and collision velocity to estimate the social benefit. We also evaluated the accident reduction effect as a human-machine system using our original integrated error model. Results confirmed that introducing the system into the market would have an accident reduction effect even considering possible system malfunction.

- 2.
Propose an evaluation method to quantify the social benefit of introduction of new ARAS for two-wheeled vehicles using cost-benefit analysis.

We calculated both the human loss and material loss according to accident statistics data to estimate the benefit due to the accident reduction effect. In addition, we calculated the long-term cost using parameters which set the cost for introducing the system into the market over 10 years. Results confirmed that the benefit ratio exceeds 1.0 after 4 years of practical use, meaning that societal benefit can be gained through ARAS introduction. Using these results, we have proposed a method to evaluate ARAS that considers accident reduction effects and social benefit combining riding simulator experiments, computer simulation, and cost-benefit analysis.

In this paper, we proposed a multi-step method to evaluate ARAS from a social standpoint. Our method can evaluate ARAS with appropriate time and cost by using both RS experiments and computer simulation.

In order to perform computer simulation to supplement the statistical reliability of a limited number of experiments, we pre-processed data describing the riding behaviors of test subjects using statistical techniques. Pre-processing was necessary because the riding behavior data obtained in the experiments for simulations was not quite sufficient for meaningful computer simulation. Overall trends of riding behaviors of experiment test subjects were used to verify the probability distribution with maximum similarity, and the number of data was increased after curve fitting with the appropriate probability distribution. The central limit theorem suggests that at least 50 samples are required for statistical reliability in order to confirm overall trends of riding behaviors, but our method is efficient enough compared to the time and cost used in actual driving experiments. We believe this will increase the reliability of computer simulation, and we think our evaluation method is adequate for evaluating and comparing new ARAS.

In our next study, we will investigate the relationship of bio-signals and driving performance to develop a new ARAS. Furthermore, future studies will be performed using this system for other patterns of traffic accidents.

## References

- 1.
Morsink, P.L.J., Cseh, C., Gietelink, O.J., Miglietta, M.: “Preliminary design of an application for communication based longitudinal control in the Car TALK 2000 project” in E-Safety conference. Lyon, France (2002)

- 2.
Kowshik, H., Caveney, D., Kumar, P.: Provable systemwide safety in intelligent intersections. IEEE Trans. Veh. Technol.

**60**(3), 804–818 (2011) - 3.
Ryosuke Okuda, Yuki Kajiwara and Kazuaki Terashima, “A Survey of Technical Trend of ADAS and Autonomous Driving” in 2014 International Symposium on VLSI Technology, Systems and Application (VLSI-TSA) (Hsinchu, Taiwan, 2014)

- 4.
Lu, M.: Kees Wevers and rob van Der Heijden, “technical feasibility of advanced driver assistance systems (ADAS) for road traffic safety”. Transp. Plan. Technol.

**28**(3), 167–187 (2005) - 5.
Vaa, T., Penttinen, M., Spyropoulou, I.: Intelligent transport systems and effects on road traffic accidents: state of the art. IET Intell. Transp. Syst.

**1**(2), 81–88 (2007) - 6.
International Motorcycle Manufacturers Association: HHRT Motorcycle Safety: IMMA’s Contribution to the Decade of Action for Road Safety 2011–20. 2010

- 7.
L. Nehaoua, S. Hima, H. Arioui, N. Seguy and S. Espie: Design and Modeling of a New Motorcycle Riding Simulator. In 2007 American Control Conference (New York, USA, 2007)

- 8.
Cossalter, V., Lot, R., Massaro, M.: The chatter of racing motorcycles. Int. J. Vehicle Mechanics Mobil.

**46**(4), 339–353 (2008) - 9.
Bekiaris, E.D., Spadoni, A., Nikolaou, S.I.: “SAFERIDER Project: New Safety and Comfort in Powered Two Wheelers” in 2009 2nd Conference on Human System Interactions. Italy, Catania (2009)

- 10.
J. P. Frederik Diederichs, Marco Fontana, Giacomo Bencini, Stella Nikolaou, Roberto Montanari, Andrea Spadoni, Harald Widlroither and Niccolò BaldanziniSmith, “New HMI Concept for Motorcycles–The Saferider Approach” in International Conference on Engineering Psychology and Cognitive Ergonomics (San Diego, USA, 2009)

- 11.
Bertolazzi, E., Biral, F., Da Lio, M., Saroldi, A., Tango, F.: Supporting drivers in keeping safe speed and safe distance: the SASPENCE subproject within the European framework Programme 6 integrating project PReVENT. IEEE Trans. Intell. Transp. Syst.

**11**(3), 525–538 (2009) - 12.
Biondi, F., Rossi, R., Gastaldi, M., Mulatti, C.: Beeping ADAS: Reflexive effect on driver’s behavior. Transp. Res. Part F Traffic Psychol. Behav.

**25**(Part A), 27–33 (2014) - 13.
Lee, J., Ozaki, I., Kishino, S., Suzuki, K., Nakajima, M.: Human-machine interface system for motorcyclists: quantification of total reliability as human-machine system. In: Advances in Human Aspects of Transportation. USA, Orlando (2018)

- 14.
Morita, K., Tanaka, N., Aoki, Y., Enomoto, M., Yasumoto, M., Sekine, M.: Quantitative evaluation of adaptive driving beam using “a survey simulator to evaluate safety systems (ASSESS)” to reduce pedestrian accidents during nighttime. Trans. Soc. Automotive Eng. Jpn.

**45**(1), 127–134 (2014) - 15.
Ministry of Land, Infrastructure, Transport and Tourism in Japan, “Advanced Safety Vehicle Guideline”, 2011

- 16.
Suzuki, K., Mochizuki, M., Miichi, Y., Yamada, K.: Methodologies for evaluating the effectiveness of collision avoidance assistance system. Trans. Jpn Soc. Mechan. Eng.

**82**(835), (2016) - 17.
Shingala, M.C., Rajyaguru, A.: Comparison of post hoc tests for unequal variance. Int. J. New Technol. Sci. Eng.

**2**(5), 22–33 (2015) - 18.
Mochizuki, M., Nii, Y., Suzuki, K.: Kiichi Yamada, “evaluation of the distraction reduction by driving support Systems in a Collision at intersection based on the time series reliability model”. Trans. Soc. Instrum. Control Eng.

**49**(11), 1074–1081 (2013) - 19.
Mochizuki, M., Kubotani, H., Nii, Y., Suzuki, K.: Evaluation of the Effectiveness of Collision-prevention Support Information at a Blind Intersection and Analysis Concerning the Acceptable Reliability of the Sensors. Trans. Soc. Automotive Eng. Jpn.

**44**(2), 621–626 (2013) - 20.
Ajao, K.R.: Oyindamola Oladosu and Olubunmi Popoola, “using HOMER power optimization software for cost-benefit analysis of hybrid-solar power generation relative to utility cost in Nigeria”. International Journal of Research & Reviews in Applied Sciences.

**7**(1), 96–102 (2011) - 21.
Takahashi, M., Deguchi, A.: Method for cost-benefit EvaluationSystem of impact by forming Compact City. J. City Plan. Inst. Jpn.

**42**(3), 487–492 (2007) - 22.
Institute for Traffic Accident Research and Data Analysis, “Traffic Statistics”, 2017

- 23.
Arai, Y., Hasegawa, M., Harigae, T.: “Research on Severity Class Evaluation Based on Various Crash Situations Involved with Motorcycles for ISO 26262”, in SAE/JSAE 2016 Small Engine Technology Conference & Exhibition. USA, South Carolina (2016)

- 24.
The General Insurance Association of Japan, “Automobile insurance data”, 2012

## Author information

## Additional information

### Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

## Electronic supplementary material

### ESM 1

(DOCX 28 kb)

## Rights and permissions

**Open Access** This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.

## About this article

### Cite this article

Lee, J., Ozaki, I., Kishino, S. *et al.* Evaluation Method of ARAS Combining Simulator Experiment and Computer Simulation in Terms of Cost-Benefit Analysis.
*Int. J. ITS Res.* (2020). https://doi.org/10.1007/s13177-019-00215-z

Received:

Revised:

Accepted:

Published:

### Keywords

- Cost-benefit analysis
- Evaluation method
- Advanced rider assistance system (ARAS)
- Human-machine Interface (HMI)
- Two-wheeled vehicles
- Riding simulator
- Time-series reliability model
- Integrated error model