Big data for assisted and autonomous driving

Abstract

If intelligent algorithms are the rockets of self-driving cars, then the data that is used to train and validate these algorithms can be considered the fuel on which these cars are running. Indeed, developing algorithms for self-driving cars requires many iterations on large sets of data which are collected under various real-case driving scenarios, and this is especially so as these algorithms increasingly incorporate deep neural nets.

A standard approach to developing and testing algorithms for autonomous driving in a timely and cost-efficient manner is to carry out re-simulations. In re-simulation campaigns, based on the raw sensor data collected in a limited number of physical test drives, many additional virtual test drives are created on which the algorithms are then carried out, optimized and tested against the range of scenarios. The sheer amount of data that is involved in this process poses very new challenges to this standard approach for meeting functional and non-functional requirements.

A typical development process requires to carry out many re-simulations, but even a single re-simulation is computationally expensive. In conventional IT environments where re-simulations are carried out on individual workstations one at a time, this implies correspondingly large times to results. Data is collected in test drives world-wide, where even at a single testing site, data volumes exceed the storage and processingcapacities. Typically, because of the large size, data cannot be easily moved from the location of storage to the re-simulation environment, and re-simulation campaigns on cross-site data become extremely difficult.