Skip to main content
SpringerLink
Log in
Book cover

Autonomy and Artificial Intelligence: A Threat or Savior? pp 103–127Cite as

Verification Challenges for Autonomous Systems

  • Chapter
  • First Online:

Abstract

In this chapter, some research challenges in the verification of autonomous systems are outlined. The objective was to identify existing available verification tools and their associated gaps, additional challenges for which there are no tools, and to make suggestions for directions in which progress may profitably be made. The chapter briefly touches on existing research to begin addressing these problems but there are more unexplored research challenges than there are programs underway to explore them. This chapter concludes with an enumeration of the unexplored challenges.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   159.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    Immobot—a robot that is not capable of moving from one location to another within its environment but is capable of modifying its environment in some way, e.g. a smart house.

References

  • IEEE Standard Ontologies for Robotics and Automation (2015a), IEEE Std 1872-2015, 60 pages.

    Google Scholar 

  • IEEE Standard Ontologies for Robotics and Automation (2015b), P1872/D3, 55 pages.

    Google Scholar 

  • Albus, J., Huang, H. M., Messina, E., Murphy, K., Juberts, M., Lacaze, A., et al. (2000). 4D/RCS: A reference model architecture for unmanned vehicle systems version 2.0.

    Google Scholar 

  • Barltrop, K. J., Friberg, K. H., & Horvath, G. A. (2008). Automated generation and assessment of autonomous systems test cases. Aerospace Conference (pp. 1–10). IEEE.

    Google Scholar 

  • Billman, L., & Steinberg, M. (2007). Human system performance metrics for evaluation of mixed-initiative hterogeneous autonomys systems. Proceedings of 2007 Workshop on Performance Metrics for Intelligent Systems (pp. 120–126). ACM.

    Google Scholar 

  • Castillo-Effen, M., & Visnevski, N. A. (2009). Analysis of autonomous deconfliction in unmanned aircraft systems for testing and evaluation. Aerospace Conference (pp. 1–12). IEEE.

    Google Scholar 

  • Chaki, S., & Giampapa, J. A. (2013). Probabilistic verification of coordinated multi-robot missions. In Model Checking Software (pp. 135–153). Springer.

    Google Scholar 

  • Cleary, M. E., Abramsom, M., Adams, M. B., & Kolitz, S. (2001). Metrics for embedded collaborative intelligent systems. NIST Special Publication.

    Google Scholar 

  • Defense Acquisition University Press. (2001, January). Systems Engineering Fundamentals. Retrieved 2016, from http://ocw.mit.edu/courses/aeronautics-and-astronautics/16-885j-aircraft-systems-engineering-fall-2005/readings/sefguide_01_01.pdf

  • Dunbabin, M., & Marques, L. (2012, March). Robots for environmental monitoring: Significant advancements and applications. IEEE Robotics and Automation Magazine, 19(1), pp. 24–39.

    Google Scholar 

  • Durst, P. J., Gray, W., Nikitenko, A., Caetano, J., Trentini, M., & King, R. (2014). A framework for predicting the mission-specific performance of autonomous unmanned systems. IEEE/RSM International Conference on Intelligent Robots and Systems, (pp. 1962–1969).

    Google Scholar 

  • Engelberger, J. (1974). Three million hours of robot field experience. Industrial Robot: An International Journal, 164–168.

    Google Scholar 

  • Ferri, G., Ferreira, F., Djapic, V., Petillot, Y., Palau, M., & Winfield, A. (2016). The eurathlon 2015 grand challenge: The first outdoor multi-domain search and rescue robotics competition - a marine perspective. Marine Technology Science Journal, 81–97(17).

    Google Scholar 

  • Gehr, J. D. (2009). Evaluating situation awareness of autonomous systems. In Performance Evaluation and Benchmarking of Intelligent Systems (pp. 93–111). Springer.

    Google Scholar 

  • Huang, H. M., Albus, J. S., Messina, R. L., Wade, R. L., & English, R. (2004). Specifying autonomy levels for unmanned systems: Interim report. Defence and Security (pp. 386–397). International Society for Optics and Photonics.

    Google Scholar 

  • Huang, H. M., Pavek, K., Novak, B., Albus, J., & Messina, E. (2005). A framework for autonomous levels for unmanned Systems (ALFUS). Proceedings of the AUVSI's Unmanned Systems North America.

    Google Scholar 

  • Jacoff, A., Huang, H. M., Messina, E., Virts, A., & Downs, A. (2010). Comprehensive standard test suites for the performance evaluation of mobile robots. Proceedings of the 10th Performance Metrics for Intelligent Systems Workshop. ACM.

    Google Scholar 

  • Kress-Gazit, H., Wongpiromsarn, T., & Topcu, U. (1989). Correct, reactive, high-level robot control. Robotics and Automation Magazine, 18(3), pp. 65–74.

    Google Scholar 

  • McWilliams, G. T., Brown, M. A., Lamm, R. D., Guerra, C. J., Avery, P. A., Kozak, K. C., et al. (2007). Evaluation of autonomy in recent ground vehicles using the autonomy levelos for unmanned systems (alfus) framework. Proceedings of the 2007 Workshop on Performance Metrics for Intelligent Systems (pp. 54–61). ACM.

    Google Scholar 

  • Miller, S., van den Berg, J., Fritz, M., Darrell, T., Goldberg, K., & Abbeel, P. (2011). A geometric approach to robotic laundry folding. International Journal of Robotics Research, 31(2), 249–267.

    Google Scholar 

  • Murphy, R. & Shields, J. (2012). Task Force Report: The Role of Autonomy in DoD Systems. Department of Defense, Defense Science Board.

    Google Scholar 

  • Paull, L., Severac, G., Raffo, G. V., Angel, J. M., Boley, H., Durst, P. J., et al. (2012). Towards an ontology for autonomous robots. IEEE/RSJ International Conference on Intelligent Robots and Systems, (pp. 1359–1364).

    Google Scholar 

  • Pecheur, C. (2000). Validation and verification of autonomy software at NASA. National Aeronautics and Space Administration.

    Google Scholar 

  • Schultz, A. C., Grefenstett, J. J., & De Jong, K. A. (1993, October). Test and evaluation by genetic algorithms. IEEE Expert, 8(5), 9–14.

    Google Scholar 

  • Sholes, E. (2007). Evolution of a uav autonomy classification taxonomy. Aerospace Conference (pp. 1–16). IEEE.

    Google Scholar 

  • Smith, B., Millar, W., Dunphy, J., Tung, Y. W., Nayak, P., Gamble, E., et al. (1999). Validation and verification of the remote agent for spacecraft autonomy. Aerospace Conference. 1, pp. 449–468. IEEE.

    Google Scholar 

  • Smithers, T. (1995). On quantitative performance measures of robot behavior. In The Biology and Technology of Intelligent autonomous Agends (pp. 21–52). Springer.

    Google Scholar 

  • Steinberg, M. (2006). Intelligent autonomy for unmanned naval systems. Defense and Security Symposiukm (pp. 623013–623013). International Society for Optics and Photonics.

    Google Scholar 

  • Vassev, E., & Hinchey, M. (2013). On the autonomy requirements for space missions. IEEE 16th International Symposium on Object/Component/Service-Oriented Real-Time Distributed Computing, (pp. 1–10).

    Google Scholar 

Download references

Acknowledgements

The authors would like to thank Andrew Bouchard and Richard Tatum at the Naval Surface Warfare Center in Panama City, Florida, for their help with early version of this paper, and the Verification of Autonomous Systems Working Group, whose efforts help define the terminology and identify these challenges. Thanks, are also due to the United States Naval Research Laboratory and the Office of Naval Research for supporting this research.

Author information

Authors and Affiliations

  1. U.S. Naval Research Laboratory, Washington, DC, USA

    Signe A. Redfield

  2. Dalhousie University, Halifax, NS, Canada

    Mae L. Seto

Authors
  1. Signe A. Redfield
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mae L. Seto
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Signe A. Redfield .

Editor information

Editors and Affiliations

  1. Paine College, Augusta, Georgia, USA

    W.F. Lawless

  2. Naval Research Laboratory, Washington, District of Columbia, USA

    Ranjeev Mittu

  3. Naval Research Laboratory, Washington, District of Columbia, USA

    Donald Sofge

  4. U.S. Army Research Laboratory, Adelphi, Maryland, USA

    Stephen Russell

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this chapter

Cite this chapter

Redfield, S.A., Seto, M.L. (2017). Verification Challenges for Autonomous Systems. In: Lawless, W., Mittu, R., Sofge, D., Russell, S. (eds) Autonomy and Artificial Intelligence: A Threat or Savior?. Springer, Cham. https://doi.org/10.1007/978-3-319-59719-5_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-59719-5_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-59718-8

  • Online ISBN: 978-3-319-59719-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation