Skip to main content
SpringerLink
Log in
Book cover

Towards Integrated Web, Mobile, and IoT Technology pp 1–20Cite as

Metamorphic Testing of Mapping Software

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Business Information Processing ((LNBIP,volume 347))

Abstract

Mapping software is difficult to test because it is very costly to evaluate its output. This difficulty is generally known as the oracle problem, a fundamental challenge in software testing. In this paper, we propose a metamorphic testing strategy to alleviate the oracle problem in testing mapping software. We first conduct a case study to test Google Maps, the most popular web mapping service. The results of the case study show that our testing approach is effective, with the detection of several real-life bugs that can hardly be exposed under conventional testing paradigms. Following this, we conduct an analysis of the system OpenStreetMap, well-known open-source mapping software built and maintained by a community of users. We show the potential of metamorphic testing for such systems. These case studies show that metamorphic testing can be applied to mapping software for both verification and validation purposes.

An initial version of this paper was presented at HICSS-51 [1].

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Brown, J., Zhou, Z.Q., Chow, Y.-W.: Metamorphic testing of navigation software: a pilot study with Google Maps. In: Proceedings of the 51st Annual Hawaii International Conference on System Sciences (HICSS-51), pp. 5687–5696 (2018). http://hdl.handle.net/10125/50602

  2. The world factbook: Central Intelligence Agency (2013). https://www.cia.gov/library/publications/the-world-factbook/fields/2085.html

  3. BuiltWith: Mapping usage statistics (2017). https://trends.builtwith.com/mapping

  4. OpenStreetMap stats report: OpenStreetMap Foundation (2018). https://www.openstreetmap.org/stats/data_stats.html

  5. Stats: OpenStreetMap Foundation (2018). https://wiki.openstreetmap.org/wiki/Stats

  6. List of OSM-based services: OpenStreetMap Foundation (2018). https://wiki.openstreetmap.org/wiki/List_of_OSM-based_services

  7. OSM Internet Links: OpenStreetMap Foundation (2018). https://wiki.openstreetmap.org/wiki/OSM_Internet_Links

  8. Barr, E.T., Harman, M., McMinn, P., Shahbaz, M., Yoo, S.: The oracle problem in software testing: a survey. IEEE Trans. Softw. Eng. 41(5), 507–525 (2015)

    Article  Google Scholar 

  9. Wright, M., Stallings, D., Dunn, D.: The effectiveness of global positioning system electronic navigation. In: Proceedings of IEEE SoutheastCon, pp. 62–67 (2003)

    Google Scholar 

  10. Chen, T.Y., Tse, T.H., Zhou, Z.Q.: Fault-based testing without the need of oracles. Inf. Softw. Technol. 45(1), 1–9 (2003)

    Article  Google Scholar 

  11. Chen, T.Y., Kuo, F.-C., Zhou, Z.Q.: An effective testing method for end-user programmers. ACM SIGSOFT Softw. Eng. Notes 30(4), 1–5 (2005)

    Google Scholar 

  12. Liu, H., Kuo, F.-C., Towey, D., Chen, T.Y.: How effectively does metamorphic testing alleviate the oracle problem? IEEE Trans. Softw. Eng. 40(1), 4–22 (2014)

    Article  Google Scholar 

  13. Lindvall, M., Ganesan, D., Árdal, R., Wiegand, R.E.: Metamorphic model-based testing applied on NASA DAT – an experience report. In: Proceedings of the 37th International Conference on Software Engineering (ICSE 2015), pp. 129–138 (2015)

    Google Scholar 

  14. Zhou, Z.Q., Xiang, S., Chen, T.Y.: Metamorphic testing for software quality assessment: a study of search engines. IEEE Trans. Softw. Eng. 42(3), 264–284 (2016)

    Article  Google Scholar 

  15. Segura, S., Fraser, G., Sanchez, A.B., Ruiz-Cortés, A.: A survey on metamorphic testing. IEEE Trans. Softw. Eng. 42(9), 805–824 (2016)

    Article  Google Scholar 

  16. Chen, T.Y., et al.: Metamorphic testing for cybersecurity. Computer 49(6), 48–55 (2016)

    Article  Google Scholar 

  17. Kanewala, U., Pullum, L.L., Segura, S., Towey, D., Zhou, Z.Q.: Message from the workshop chairs. In: Proceedings of the IEEE/ACM 1st International Workshop on Metamorphic Testing (ICSE MET 2016), in Conjunction with the 38th International Conference on Software Engineering (ICSE). ACM Press (2016)

    Google Scholar 

  18. Jarman, D.C., Zhou, Z.Q., Chen, T.Y.: Metamorphic testing for Adobe data analytics software. In: Proceedings of the IEEE/ACM 2nd International Workshop on Metamorphic Testing (ICSE MET 2017), in Conjunction with the 39th International Conference on Software Engineering (ICSE), pp. 21–27 (2017)

    Google Scholar 

  19. Ding, J., Hu, X.-H., Gudivada, V.: A machine learning based framework for verification and validation of massive scale image data. IEEE Trans. Big Data. https://doi.org/10.1109/TBDATA.2017.2680460

  20. Pezzè, M., Young, M.: Software Testing and Analysis: Process, Principles, and Techniques. Wiley, New York (2008)

    MATH  Google Scholar 

  21. Chen, T.Y., Huang, D.H., Tse, T.H., Zhou, Z.Q.: Case studies on the selection of useful relations in metamorphic testing. In: Proceedings of the 4th Ibero-American Symposium on Software Engineering and Knowledge Engineering (JIISIC 2004). Polytechnic University of Madrid, pp. 569–583 (2004)

    Google Scholar 

  22. Goodman, J.L.: The space shuttle and GPS: a safety-critical navigation upgrade. In: Erdogmus, H., Weng, T. (eds.) ICCBSS 2003. LNCS, vol. 2580, pp. 92–100. Springer, Heidelberg (2003). https://doi.org/10.1007/3-540-36465-X_9

    Chapter  Google Scholar 

  23. Nisner, P.D., Johannessen, R.: Ten million data points from TSO-approved aviation GPS receivers: results of analysis and applications to design and use in aviation. Navigation 47(1), 43–50 (2000)

    Article  Google Scholar 

  24. Elleuch, W., Wali, A., Alimi, A.M.: Mining road map from big database of GPS data. In: 14th International Conference on Hybrid Intelligent Systems (HIS), pp. 193–198. IEEE (2014)

    Google Scholar 

  25. Elleuch, W., Wali, A., Alimi, A.M.: Collection and exploration of GPS based vehicle traces database. In: 4th International Conference on Advanced Logistics and Transport (ICALT), pp. 275–280. IEEE (2015)

    Google Scholar 

  26. Chen, T.Y., Cheung, S.C., Yiu, S.M.: Metamorphic testing: a new approach for generating next test cases, Technical report HKUST-CS98-01, Department of Computer Science, Hong Kong Univ. of Science and Technology (1998)

    Google Scholar 

  27. Goodman, J.L.: A software perspective on GNSS receiver integration and operation. In: Rycroft, M. (ed.) Satellite Navigation Systems. Space Studies, vol. 8, pp. 119–126. Springer, Dordrecht (2003). https://doi.org/10.1007/978-94-017-0401-4_13

    Chapter  Google Scholar 

  28. Luxen, D., Vetter, C.: Real-time routing with OpenStreetMap data. In: Proceedings of the 19th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems (GIS 2011), pp. 513–516. ACM (2011)

    Google Scholar 

  29. Graf, F., Kriegel, H.-P., Renz, M., Schubert, M.: MARiO: multi-attribute routing in open street map. In: Pfoser, D. (ed.) SSTD 2011. LNCS, vol. 6849, pp. 486–490. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-22922-0_36

    Chapter  Google Scholar 

  30. Cipeluch, B., Jacob, R., Winstanley, A., Mooney, P.: Comparison of the accuracy of OpenStreetMap for Ireland with Google Maps and Bing Maps. In: Proceedings of the 9th International Symposium on Spatial Accuracy Assessment in Natural Resources and Environmental Sciences, pp. 337–341 (2010)

    Google Scholar 

  31. Chen, T.Y., et al.: Metamorphic testing: a review of challenges and opportunities. ACM Comput. Surv. 51(1), 4:1–4:27 (2018)

    Article  Google Scholar 

  32. Zhou, Z.Q., Sun, L., Chen, T.Y., Towey, D.: Metamorphic relations for enhancing system understanding and use. IEEE Trans. Softw. Eng. https://doi.org/10.1109/TSE.2018.2876433

  33. Zhou, Z.Q., Sun, L.: Metamorphic testing of driverless cars. Commun. ACM 62(3), 61–67 (2019)

    Article  Google Scholar 

  34. Segura, S., Parejo, J.A., Troya, J., Ruiz-Cortés, A.: Metamorphic testing of RESTful Web APIs. IEEE Trans. Softw. Eng. 44(11), 1083–1099 (2018)

    Article  Google Scholar 

  35. Google Maps Directions API: Google (2016). https://developers.google.com/maps/documentation/directions/intro

Download references

Acknowledgements

This work was supported in part by a linkage grant of the Australian Research Council (project ID: LP160101691) and an Australian Government Research Training Program scholarship.

Author information

Authors and Affiliations

  1. Institute of Cybersecurity and Cryptology, School of Computing and Information Technology, University of Wollongong, Wollongong, NSW, 2522, Australia

    Joshua Brown, Zhi Quan Zhou & Yang-Wai Chow

Authors
  1. Joshua Brown
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhi Quan Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yang-Wai Chow
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhi Quan Zhou .

Editor information

Editors and Affiliations

  1. Department of Information Systems, University of Agder, Kristiansand, Norway

    Tim A. Majchrzak

  2. ISISTAN-UNICEN-CONICET, Tandil, Argentina

    Cristian Mateos

  3. University of Bologna, Bologna, Italy

    Francesco Poggi

  4. Kristiania University College, Oslo, Norway

    Tor-Morten Grønli

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Brown, J., Zhou, Z.Q., Chow, YW. (2019). Metamorphic Testing of Mapping Software. In: Majchrzak, T., Mateos, C., Poggi, F., Grønli, TM. (eds) Towards Integrated Web, Mobile, and IoT Technology. Lecture Notes in Business Information Processing, vol 347. Springer, Cham. https://doi.org/10.1007/978-3-030-28430-5_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-28430-5_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-28429-9

  • Online ISBN: 978-3-030-28430-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation