Using Physics-Based M&S for Training and Testing Machine Learning Algorithms

  • Justin CarrilloEmail author
  • Burhman Gates
  • Gabe Monroe
  • Brent Newell
  • Phillip Durst
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11472)


Machine learning algorithms have been used to successfully solve many complex and diverse problems especially in the domain of unmanned vehicle systems. However, machine learning algorithms require training data that contain extensive variations in specimens. These variations include variations of the sensor settings, terrain conditions, environmental conditions, and variants of the objects of interest themselves. Capturing training specimens that span these variants is time consuming, expensive, and in some cases impossible. Training data in a narrow range of variations lead to decreases in performance such as overfitting. Therefore, collecting training data is often the limiting factor in developing robust machine learning algorithms for applications such as object detection and classification. Another time-consuming task is labeling training specimens with metadata needed in some training approaches. In this paper, we demonstrate using a physics-based modeling and simulation (M&S) capability to generate simulated training data spanning variations in sensor settings, terrain conditions, and environmental conditions that include a versatile automated labeling process. The product of prior efforts, the Virtual Autonomous Navigation Environment (VANE), is a high-fidelity physics-based M&S tool for simulating sensors commonly used on unmanned ground vehicles (UGVs) and unmanned aerial vehicles (UAVs).


VANE Modeling Simulation UGV UAV High-Fidelity Physics-Based 


  1. 1.
    Fields, M., et al.: Simulation tools for robotics research and assessment. In: Proceedings of the SPIE, vol. 9837 (2016)Google Scholar
  2. 2.
    Koenig, N., Howard, A.: Design and use paradigms for gazebo, and open-source multi-robot simulator. In: IROS, vol. 3, pp. 2149–2154. IEEE, USA (2004)Google Scholar
  3. 3.
    Michal, D.S., Etzkorn, L.: A comparison of player/stage/gazebo and microsoft robotics developer studio. In: Proceedings of the 49th Annual Southeast Regional Conference, pp. 60–66. ACM, USA (2011)Google Scholar
  4. 4.
    Brewer, R., Schaefer, K.E., Avery, E.: Robotic interactive visualization experimentation technology (RIVET): game-based simulation for human-robot interaction research. In: 2015 Winter Simulation Conference (WSC), pp. 3224–3225 (2015)Google Scholar
  5. 5.
    Elmquist, A., Negrut, D.: Virtual sensing for autonomous vehicle simulation in chrono. Simulation-Based Engineering Lab University of Wisconsin-Madison, Technical report TR-2016-13 (2017)Google Scholar
  6. 6.
    Pharr, M., Jakob, W., Humphreys, G.: Physically Based Rendering: From Theory to Implementation, 3rd edn. Morgan Kaufmann, Burlington (2017)Google Scholar
  7. 7.
    Nagaraju, K., et al.: High-Fidelity Modeling to Support Route Clearance. North Atlantic Treaty Organization (NATO) Science and Technology Organization (STO) STO-MP-MSG-143 (2016)Google Scholar
  8. 8.
    Goodin, C., et al.: Unmanned ground vehicle simulation with the virtual autonomous navigation environment. In: International Conference on Military Technologies (2017)Google Scholar
  9. 9.
    Durst, P., Goodin, C.: High fidelity modelling and simulation of inertial sensors commonly used by autonomous mobile robots. World J. Modell. Simul. 8, 172–184 (2012)Google Scholar
  10. 10.
    Goodin, C., Durst, P., Prevost, Z., Compton, P.: A probabilistic model for simulating the effect of airborne dust on ground-based LiDAR. In: SPIE Defense, Security, and Sensing. International Society for Optics and Photonics (2013)Google Scholar
  11. 11.
    Lee, M., Valisetty, R., Breuer, A., Kirt, K., Panneton, B., Brown, S.: Current and future applications of machine learning for the US Army. US Army Research Laboratory, Technical report ARL-TR-8345 (2018)Google Scholar
  12. 12.
    Carrillo, J., Davis, J., Osorio, J., Goodin, C., Durst, J.: High-fidelity physics-based modeling and simulation for training and testing convolutional neural networks for UGV systems. J. Unmanned Veh. Syst. (in review)Google Scholar
  13. 13.
    Harney, R.: Combat Systems. Volume 1. Sensor Elements. Part I. Sensor Functional Characteristics (2004)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Justin Carrillo
    • 1
    Email author
  • Burhman Gates
    • 1
  • Gabe Monroe
    • 1
  • Brent Newell
    • 1
  • Phillip Durst
    • 1
  1. 1.U.S. Army Engineer Research and Development CenterVicksburgUSA

Personalised recommendations