Skip to main content
SpringerLink
Log in
Book cover

International Conference on Software Engineering and Formal Methods

SEFM 2017: Software Engineering and Formal Methods pp 174–179Cite as

Generalized Oracle for Testing Machine Learning Computer Programs

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 10729))

Abstract

Computation results of machine learning programs are not possible to be anticipated, because the results are sensitive to distribution of data in input dataset. Additionally, these computer programs sometimes adopt randomized algorithms for finding sub-optimal solutions or improving runtime efficiencies to reach solutions. The computation is probabilistic and the results vary from execution to execution even for a same input. The characteristics imply that no deterministic test oracle exists to check correctness of programs. This paper studies how a notion of oracles is elaborated so that these programs can be tested, and shows a systematic way of deriving testing properties from mathematical formulations of given machine learning problems.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   107.00
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Bishop, C.M.: Pattern Recognition and Machine Learning. Springer, New York (2006)

    MATH  Google Scholar 

  2. Chen, T.Y., Chung, S.C., Yiu, S.M.: Metamorphic testing - a new approach for generating next test cases, HKUST-CS98-01. The Hong Kong University of Science and Technology (1998)

    Google Scholar 

  3. Guderlei, R., Mayer, J., Schneckenburger, C., Fleischer, F.: Testing randomized software by means of statistical hypothesis tests. In: Proceedings of SOQUA 2007, pp. 46–54 (2007)

    Google Scholar 

  4. Nakajima, S., Bui, H.N.: Dataset coverage for testing machine learning computer programs. In: Proceedings of 23rd APSEC, pp. 297–304 (2016)

    Google Scholar 

  5. Weyuker, E.J.: On testing non-testable programs. Comput. J. 25(4), 465–470 (1982)

    Article  Google Scholar 

  6. Xie, X., Ho, J.W.K., Murphy, C., Kaiser, G., Xu, B., Chen, T.Y.: Testing and validating machine learning classifiers by metamorphic testing. J. Syst. Softw. 84(4), 544–558 (2011)

    Article  Google Scholar 

  7. Zhu, X.: Machine teaching: an inverse problem to machine learning and an approach toward optimal education. In: Proceedings of 29th AAAI, pp. 4083–4087 (2015)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. National Institute of Informatics, Tokyo, Japan

    Shin Nakajima

Authors
  1. Shin Nakajima
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shin Nakajima .

Editor information

Editors and Affiliations

  1. Nazarbayev University, Astana, Kazakhstan

    Antonio Cerone

  2. Fondazione Bruno Kessler, Povo, Italy

    Marco Roveri

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Nakajima, S. (2018). Generalized Oracle for Testing Machine Learning Computer Programs. In: Cerone, A., Roveri, M. (eds) Software Engineering and Formal Methods. SEFM 2017. Lecture Notes in Computer Science(), vol 10729. Springer, Cham. https://doi.org/10.1007/978-3-319-74781-1_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-74781-1_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-74780-4

  • Online ISBN: 978-3-319-74781-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation