Skip to main content
SpringerLink
Log in

How to Specify It!

A Guide to Writing Properties of Pure Functions

  • Conference paper
  • First Online:
Trends in Functional Programming (TFP 2019)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 12053))

Included in the following conference series:

Abstract

Property-based testing tools test software against a specification, rather than a set of examples. This tutorial paper presents five generic approaches to writing such specifications (for purely functional code). We discuss the costs, benefits, and bug-finding power of each approach, with reference to a simple example with eight buggy variants. The lessons learned should help the reader to develop effective property-based tests in the future.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://hackage.haskell.org/package/QuickCheck.

  2. 2.

    http://hackage.haskell.org/package/QuickCheck.

  3. 3.

    https://hypothesis.works/articles/what-is-property-based-testing/.

  4. 4.

    https://en.wikipedia.org/wiki/QuickCheck.

  5. 5.

    https://fscheck.github.io/FsCheck/.

  6. 6.

    https://pypi.org/project/hypothesis/.

  7. 7.

    Recall that we have not imposed any balance condition on our trees. If we were to repeat this entire exercise for balanced trees, then we would need a stronger invariant to capture the balance condition, but we would still face the same problem in this property, since balance conditions don’t require a unique tree shape. Both trees in this example are balanced—they are just different balanced representations of the same information.

  8. 8.

    http://metwiki.net/MET19/.

  9. 9.

    http://mbt-workshop.org/.

  10. 10.

    https://conf.researchr.org/series/a-most.

  11. 11.

    A company founded in 2006 by the author and Thomas Arts, to commercialize property based testing. See http://quviq.com.

References

  1. Amaral, C., Florido, M., Santos Costa, V.: PrologCheck – property-based testing in prolog. In: Codish, M., Sumii, E. (eds.) FLOPS 2014. LNCS, vol. 8475, pp. 1–17. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-07151-0_1

    Chapter  Google Scholar 

  2. Arts, T., Hughes, J., Johansson, J., Wiger, U.T.: Testing telecoms software with Quviq QuickCheck. In: Feeley, M., Trinder, P.W. (eds.) Proceedings of the 2006 ACM SIGPLAN Workshop on Erlang, Portland, Oregon, USA, 16 September 2006, pp. 2–10. ACM (2006)

    Google Scholar 

  3. Barr, E.T., Harman, M., McMinn, P., Shahbaz, M., Yoo, S.: The oracle problem in software testing: a survey. IEEE Trans. Soft. Eng. 41(5), 507–525 (2015)

    Article  Google Scholar 

  4. Braquehais, R.M.: Tools for discovery, refinement and generalization of functional properties by enumerative testing. Ph.D. thesis, University of York, UK (2017)

    Google Scholar 

  5. Broy, M., Jonsson, B., Katoen, J.-P., Leucker, M., Pretschner, A. (eds.): Model-Based Testing of Reactive Systems. LNCS, vol. 3472. Springer, Heidelberg (2005). https://doi.org/10.1007/b137241

    Book  MATH  Google Scholar 

  6. Chen, T.Y., Cheung, S.C., Yiu, S.M.: Metamorphic testing: a new approach for generating next test cases. Technical report, HKUST-CS98-01, Department of Computer Science, Hong Kong (1998)

    Google Scholar 

  7. Chen, T.Y., et al.: Metamorphic testing: a review of challenges and opportunities. ACM Comput. Surv. 51(1), 4:1–4:27 (2018)

    Article  Google Scholar 

  8. Claessen, K.: Inductive testing. Private communication. https://docs.google.com/presentation/d/1pejW9foV4ZAw5e03kYR3urNQsIPobomY_5HshxZQpLc/edit?usp=drivesdk

  9. Claessen, K., Hughes, J.: QuickCheck: a lightweight tool for random testing of Haskell programs. In: Proceedings of 5th ACM SIGPLAN International Conference on Functional Programming, ICFP 2000 (2000)

    Google Scholar 

  10. Lindley, S., McBride, C., Trinder, P., Sannella, D. (eds.): A List of Successes That Can Change the World. LNCS, vol. 9600. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-30936-1

    Book  Google Scholar 

  11. Floyd, R.W.: Assigning meanings to programs. In: Colburn, T.R., Fetzer, J.H., Rankin, T.L. (eds.) Program Verification, vol. 14, pp. 65–81. Springer, Dordrecht (1993). https://doi.org/10.1007/978-94-011-1793-7_4

    Chapter  Google Scholar 

  12. Godefroid, P., Klarlund, N., Sen, K.: DART: directed automated random testing. ACM SIGPLAN Not. 40, 213–223 (2005)

    Article  Google Scholar 

  13. Guttag, J.V., Horning, J.J.: The algebraic specification of abstract data types. Acta Inform. 10(1), 27–52 (1978)

    Article  MathSciNet  Google Scholar 

  14. Hoare, C.A.: Proof of correctness of data representations. Acta Inform. 1(4), 271–281 (1972)

    Article  Google Scholar 

  15. Hoare, C.A.R.: An axiomatic basis for computer programming. Commun. ACM 12(10), 576–580 (1969)

    Article  Google Scholar 

  16. Hughes, J.: Experiences with QuickCheck: testing the hard stuff and staying sane. In: Lindley et al. [10], pp. 169–186 (2016)

    Google Scholar 

  17. Löscher, A., Sagonas, K.: Targeted property-based testing. In: Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis, pp. 46–56. ACM (2017)

    Google Scholar 

  18. Meyer, B.: Applying ‘design by contract’. Computer 25(10), 40–51 (1992)

    Article  Google Scholar 

  19. Nilsson, R.: ScalaCheck: The Definitive Guide. Artima Press, Mountain View (2014)

    Google Scholar 

  20. Papadakis, M., Sagonas, K.: A proper integration of types and function specifications with property-based testing. In: Proceedings of the 10th ACM SIGPLAN Workshop on Erlang, pp. 39–50. ACM (2011)

    Google Scholar 

  21. Pike, L.: SmartCheck: automatic and efficient counterexample reduction and generalization. In: Swierstra, W. (ed.) Proceedings of the 2014 ACM SIGPLAN Symposium on Haskell, Gothenburg, Sweden, 4–5 September 2014, pp. 53–64. ACM (2014)

    Google Scholar 

  22. Polya, G.: How To Solve It! A System of Thinking Which Can Help You Solve Any Problem. Princeton University Press, Princeton (1945)

    Google Scholar 

  23. Runciman, C., Naylor, M., Lindblad, F.: SmallCheck and lazy SmallCheck: automatic exhaustive testing for small values. In: Gill, A. (ed.) Proceedings of the 1st ACM SIGPLAN Symposium on Haskell, Haskell 2008, Victoria, BC, Canada, 25 September 2008, pp. 37–48. ACM (2008)

    Google Scholar 

  24. Sen, K., Marinov, D., Agha, G.: CUTE: a concolic unit testing engine for C. ACM SIGSOFT Softw. Eng. Notes 30, 263–272 (2005)

    Article  Google Scholar 

  25. Spivey, J.M.: Understanding Z: A Specification Language and Its Formal Semantics, vol. 3. Cambridge University Press, Cambridge (1988)

    MATH  Google Scholar 

Download references

Acknowledgements

I’m grateful to the anonymous referees for many useful suggested improvements, and to Vetenskapsrådet for funding this work under the SyTeC grant.

Author information

Authors and Affiliations

  1. Chalmers University of Technology and Quviq AB, Göteborg, Sweden

    John Hughes

Authors
  1. John Hughes
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to John Hughes .

Editor information

Editors and Affiliations

  1. University of British Columbia, Vancouver, BC, Canada

    William J. Bowman

  2. University of British Columbia, Vancouver, BC, Canada

    Ronald Garcia

A Metamorphic Properties

A Metamorphic Properties

figure au

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Hughes, J. (2020). How to Specify It!. In: Bowman, W., Garcia, R. (eds) Trends in Functional Programming. TFP 2019. Lecture Notes in Computer Science(), vol 12053. Springer, Cham. https://doi.org/10.1007/978-3-030-47147-7_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-47147-7_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-47146-0

  • Online ISBN: 978-3-030-47147-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation