Skip to main content
SpringerLink
Log in

Clustering Study of Crowdsourced Test Report with Multi-source Heterogeneous Information

  • Conference paper
  • First Online:
Data Mining and Big Data (DMBD 2019)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1071))

Included in the following conference series:

Abstract

Crowdsourced testing is an emerging testing method in the field of software testing and industrial practice. Crowdsourced testing can provide a more realistic user experience. But crowdsourced workers are independent of each other, they may submit test reports for the same issue, resulting in highly redundant test reports submitted. In addition, crowdsourced test reports with multi-source heterogeneous information tend to have short text descriptions, but the screenshots are rich, and using only text information can lead to information bias in test reports. In view of this, this paper attempts to use the screenshot information in the crowdsourced test report to assist the text information to cluster the crowdsourced test report. Firstly, the text similarity and screenshot similarity of crowdsourced test reports are calculated respectively, then the similarity between crowdsourced test reports is weighted. Finally, test reports are grouped by clustering algorithm based on similarity measure. Testers only need to audit the test report as the representative, which greatly reduces the pressure of the tester’s report audit. The final experimental results show that the effective use of the screenshot information in the test report can achieve higher clustering accuracy.

Supported by National Natural Science Foundation of China (No. 61573362, 61773384, and 61502212), and National Key Research and Development Program of China (No. 2018YFB1003802-01).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Yao, X.J., Gong, D.W., Zhang, G.J.: Constrained multi-objective test data generation based on set evolution. IET Softw. 9(4), 103–108 (2015)

    Article  Google Scholar 

  2. Yao, X.J., Gong, D.W., Wang, W.L.: Test data generation for multiple paths based on local evolution. Chin. J. Electron. 24, 46–51 (2015)

    Article  Google Scholar 

  3. Feng, Y., Jones, J.A., Chen, Z.Y., Fang, C.R.: Multi-objective test report prioritization using image understanding. In: ACM International Conference on Automated Software Engineering, pp. 202–213. ACM, Singapore (2016)

    Google Scholar 

  4. Jiang, H., Chen, X., He, T., Chen, Z., Li, X.: Fuzzy clustering of crowdsourced test reports for apps. ACM Trans. Internet Technol. 18(2), 1–28 (2018)

    Article  Google Scholar 

  5. Gong, D.W., Zhang, W.Q., Yao, X.J.: Evolutionary generation of test data for many paths coverage based on grouping. J. Syst. Softw. 84(12), 2222–2233 (2018)

    Article  Google Scholar 

  6. Yao, X.J., Harman, M., Jia, Y.: A study of equivalent and stubborn mutation operators using human analysis of equivalence. In: Proceedings of 36th International Conference on Software Engineering, ICSE, Hyderabad, pp. 919–930 (2014)

    Google Scholar 

  7. Gong, D.W., Yao, X.J.: Testability transformation based on equivalence of target statements. Neural Comput. Appl. 21, 1871–1882 (2012)

    Article  Google Scholar 

  8. Howe, J.: The rise of crowdsourcing. Wired Mag. 14(6), 1–4 (2006)

    Google Scholar 

  9. Liu, D., Bias, R.G., Lease, M., Kuipers, R.: Crowdsourcing for usability testing. Proc. Am. Soc. Inf. Sci. Technol. 49(1), 1–10 (2013)

    Google Scholar 

  10. Pastore, F., Mariani, L., Fraser, G.: CrowdOracles: can the crowd solve the oracle problem? In: IEEE 6th International Conference on Software Testing. Verification and Validation, pp. 342–351. IEEE Computer Society, Luxembourg (2013)

    Google Scholar 

  11. Dolstra, E., Vliegendhart, R., Pouwelse, J.: Crowdsourcing gui tests. In: 2013 IEEE 6th International Conference on Software Testing. Verification and Validation, pp. 332–341. IEEE Computer Society, Luxembourg (2013)

    Google Scholar 

  12. Musson, R., Richards, J., Fisher, D., Bird, C., Bussone, B., Ganguly, S.: Leveraging the crowd: how 48,000 users helped improve lync performance. IEEE Softw. 30(4), 38–45 (2013)

    Article  Google Scholar 

  13. Nebeling, M., Speicher, M., Grossniklaus, M., Norrie, M.C.: Crowdsourced web site evaluation with crowdstudy. In: Brambilla, M., Tokuda, T., Tolksdorf, R. (eds.) ICWE 2012. LNCS, vol. 7387, pp. 494–497. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-31753-8_52

    Chapter  Google Scholar 

  14. Runeson, P., Alexandersson, M., Nyholm, O.: Detection of duplicate defect reports using natural language processing. In: International Conference on Software Engineering, pp. 499–510. IEEE Computer Society, Minneapolis (2007)

    Google Scholar 

  15. Wang, X., Zhang, L., Xie, T., Anvik, J., Sun, J.: An approach to detecting duplicate bug reports using natural language and execution information. In: International Conference on Software Engineering, pp. 461–470. IEEE Computer Society, Leipzig (2008)

    Google Scholar 

  16. Jalbert, N., Weimer, W.: Automated duplicate detection for bug tracking systems. In: IEEE International Conference on Dependable Systems and Networks with Ftcs and DCC, pp. 52–61. IEEE, Anchorage (2008)

    Google Scholar 

  17. Tian, Y., Sun, C., Lo, D.: Improved duplicate bug report identification. Softw. Maint. Reengineering 94(3), 385–390 (2012)

    Google Scholar 

  18. Sun, C., Lo, D., Wang, X., Jiang, J., Khoo, S.C.: A discriminative model approach for accurate duplicate bug report retrieval. In: Proceedings of the IEEE International Conference on Software Engineering, pp. 45–54. IEEE Computer Society, Cape Town (2010)

    Google Scholar 

  19. Sun, C., Lo, D., Khoo, S.C., Jiang, J.: Towards more accurate retrieval of duplicate bug reports. In: International Conference on Automated Software Engineering, ASE, USA, pp. 253–262 (2011)

    Google Scholar 

  20. Wang, J., Cui, Q., Wang, Q, Wang, S.: Towards effectively test report classification to assist crowdsourced testing. In: ACM International Symposium on Empirical Software Engineering and Measurement. ACM, Spain (2016)

    Google Scholar 

  21. Wang, J., Wang, S., Cui, Q., Wang, Q.: Local-based active classification of test report to assist crowdsourced testing. In: ACM International Conference on Automated Software Engineering, pp. 190–201. ACM, Singapore (2016)

    Google Scholar 

  22. Feng, Y., Chen, Z., Jones, J.A., Fang, C.R., Xu, B.: Test report prioritization to assist crowdsourced testing. In: Proceeding of the 2015 10th Joint Meeting on Foundations of Software Engineering, pp. 225–236. ACM, Bergamo (2015)

    Google Scholar 

  23. Che, W., Li, Z., Liu, T.: LTP: a Chinese language technology platform. In: International Conference on Computational Linguistics. Demonstrations, pp. 23–27. Chinese Information Processing Society of China, Beijing (2010)

    Google Scholar 

  24. Bhattacharyya, A.: On a measure of divergence between two statistical populations defined by their probability distributions. Bull. Calcutta Math. Soc. 35, 99–109 (1943)

    MathSciNet  MATH  Google Scholar 

  25. Gopalan, R.P., Krishna, A.: Duplicate bug report detection using clustering. In: International Conference on Software Engineering, pp. 104–109. IEEE Computer Society, Hyderabad (2014)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Mathematics, China University of Mining and Technology, Xuzhou, 221116, China

    Yan Yang & Xiangjuan Yao

  2. School of Information and Control Engineering, China University of Mining and Technology, Xuzhou, 221116, China

    Dunwei Gong

Authors
  1. Yan Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiangjuan Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dunwei Gong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiangjuan Yao .

Editor information

Editors and Affiliations

  1. Department of Machine Intelligence, Peking University, Beijing, China

    Ying Tan

  2. Department of Computer Science and Engineering, Southern University of Science and Technology, Shenzhen, China

    Yuhui Shi

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Yang, Y., Yao, X., Gong, D. (2019). Clustering Study of Crowdsourced Test Report with Multi-source Heterogeneous Information. In: Tan, Y., Shi, Y. (eds) Data Mining and Big Data. DMBD 2019. Communications in Computer and Information Science, vol 1071. Springer, Singapore. https://doi.org/10.1007/978-981-32-9563-6_14

Download citation

  • DOI: https://doi.org/10.1007/978-981-32-9563-6_14

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-32-9562-9

  • Online ISBN: 978-981-32-9563-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation