Advertisement

Invalid bug reports complicate the software aging situation

  • Xiaoxue Wu
  • Wei ZhengEmail author
  • Minchao Pu
  • Jie Chen
  • Dejun Mu
Article
  • 6 Downloads

Abstract

Symptoms of software aging include performance degradation and failure occurrence increasing when software systems run for a period of time. Therefore, software aging is closely related to system performance. Understanding and analyzing performance issues in the software system is critical to mastering software aging information. Instead of focusing on normal valid bug reports (VBRs), this paper advocates the usage of invalid bug reports (IBRs) to capture software aging signals. We use performance bugs that are highly related to software aging as an example to construct a binary classification model for bug report classification. We conduct a rigorous evaluation of the constructed models via different performance measures (i.e., recall, precision, F1-score, AUC). Then, the model is used to predict the performance bug reports (PBRs) in IBRs, and a manual analysis of the prediction results is conducted to identify aging-related bug reports (ABRs). The final results show that the ratio of PBRs in IBRs ranges from 4.9 to 42.18% for the two real open-source projects HDFS and HBase when considering five different classifiers. Among these five classifiers, Support Vector Machine (SVM) classifier can achieve the best performance. The ratios of PBRs in IBRs by using this classifier are 11.1% and 15.35% for these two datasets and the performances in terms of F1-score are 85% and 74%. Further analysis of the predicted PBRs of IBRs in the project HDFS is conducted through a manual user case study; some surprising findings revealing the relationship between IBRs, PBRs, and ABRs are presented: (1) Around 50% of the PBRs in IBRs are related to software aging; (2) components that undertake major tasks are more prone to aging problems; (3) more than 50% ARBs lead to timeout, 33% ARBs are caused by improper control of memory or threats, and 29% ARBs are caused by inappropriate management of file operation or disk usage; (4) hard to reproduce is the major reason that ARBs are usually closed as invalid because many aging-related bugs would temporarily disappear by restarting the system.

Keywords

Software aging Invalid bug report Performance-related bug report Aging related bug report Text mining 

Notes

References

  1. Agrawal, A., & Menzies, T. (2018). Is better data better than better data miners?. In Proceedings of the 40th international conference on software engineering (pp. 1050–1061): IEEE.Google Scholar
  2. Apach hadoop project. (2019). http://hadoop.apache.org.
  3. Apach hbase project. (2019). http://hadoop.apache.org/hbase.
  4. Bao, Y., Sun, X., Trivedi, K.S. (2005). A workload-based analysis of software aging, and rejuvenation. IEEE Transactions on Reliability, 54(3), 541–548.CrossRefGoogle Scholar
  5. Breiman, L. (2001). Random forests. Machine Learning, 45(1), 5–32.CrossRefGoogle Scholar
  6. Chang, C.C., & Lin, C.J. (2011). Libsvm: a library for support vector machines. ACM Transactions on Intelligent Systems and Technology (TIST), 2(3), 27.Google Scholar
  7. Cortes, C., & Vapnik, V. (1995). Support-vector networks. Machine Learning, 20(3), 273–297.zbMATHGoogle Scholar
  8. Cotroneo, D., Natella, R., Pietrantuono, R. (2011). Is software aging related to software metrics. In IEEE second international workshop on software aging and rejuvenation.Google Scholar
  9. Cotroneo, D., Natella, R., Pietrantuono, R. (2013). Predicting aging-related bugs using software complexity metrics. Performance Evaluation, 70(3), 163–178.CrossRefGoogle Scholar
  10. Cotroneo, D., Pietrantuono, R., Russo, S., Trivedi, K. (2016). How do bugs surface? A comprehensive study on the characteristics of software bugs manifestation. Journal of Systems and Software, 113(C), 27–43.CrossRefGoogle Scholar
  11. Cucchiara, A. (2012). Applied logistic regression. Technometrics, 34(3), 358–359.CrossRefGoogle Scholar
  12. Domingos, P., & Pazzani, M. (1997). On the optimality of the simple Bayesian classifier under zero-one loss. Machine Learning, 29(2-3), 103–130.CrossRefGoogle Scholar
  13. Elshishiny, H., Deraz, S., Bahy, O. (2008). Mining software aging patterns by artificial neural networks. In Iapr workshop on artificial neural networks in pattern recognition.Google Scholar
  14. Fan, R.E., Chang, K.W., Hsieh, C.J. (2008). Wang: Liblinear: a library for large linear classification. Journal of Machine Learning Research, 9(9), 1871–1874.zbMATHGoogle Scholar
  15. Fan, Y., Xia, X., Lo, D., Hassan, A.E. (2018). Chaff from the wheat: characterizing and determining valid bug reports. IEEE Transactions on Software Engineering, PP(99), 1–1.CrossRefGoogle Scholar
  16. Gardner, M.W., & Dorling, S.R. (1998). Artificial neural networks (the multilayer perceptron): a review of applications in the atmospheric sciences. Atmospheric Environment, 32(14–15), 2627–2636.CrossRefGoogle Scholar
  17. Ghoneim, S.A., & Fahmy, H.M.A. (2003). Evaluation of the drm and the time for preventive maintenance for aging software. Software Quality Journal, 11(1), 57–75.CrossRefGoogle Scholar
  18. Goseva-Popstojanova, K., & Tyo, J. (2018). Identification of security related bug reports via text mining using supervised and unsupervised classification. International Conference on Software Quality, Reliability, and Security, QRS 2018 pp. 344–355.Google Scholar
  19. Grottke, M., Dong, S.K., Mansharamani, R., Nambiar, M., Natella, R., Trivedi, K.S. (2016). Recovery from software failures caused by mandelbugs. IEEE Transactions on Reliability, 65(1), 70–87.CrossRefGoogle Scholar
  20. Gunawi, H.S., & Hao, M. (2014). Leesatapornwongsa: what bugs live in the cloud? a study of 3000+ issues in cloud systems. In Proceedings of the ACM symposium on cloud computing.Google Scholar
  21. Han, X., Tingting, Y., David, L. (2018). Perflearner: learning from bug reports to understand and generate performance test frames. In Proceedings of the 33rd ACM/IEEE international conference on automated software engineering.Google Scholar
  22. Huang, Y., Kintala, C., Kolettis, N., Fulton, N.D. (1995). Software rejuvenation: analysis, module and applications. In International symposium on fault-tolerant computing.Google Scholar
  23. Kotsiantis, S.B., & Kanellopoulos, D. (2006). Data preprocessing for supervised leaning. International Journal of Computer Science, 1(2), 111–117.Google Scholar
  24. Kumar, L., & Ashish, S. (2018). Feature selection techniques to counter class imbalance problem for aging related bug prediction: aging related bug prediction. In Proceedings of the 11th innovations in software engineering conference.Google Scholar
  25. Lindorfer, M., Neugschwandtner, M., Platzer, C. (2015). Marvin: efficient and comprehensive mobile app classification through static and dynamic analysis. In IEEE computer software and applications conference.Google Scholar
  26. Lu, S., Park, S., Seo, E. (2008). Learning from mistakes: a comprehensive study on real world concurrency bug characteristics. In Proceedings of the international conference on architectural support for programming languages and operating systems (ASPLOS).Google Scholar
  27. Mani, S., Sankaran, A., Aralikatte, R. (2018). Deeptriage: exploring the effectiveness of deep learning for bug triaging. arXiv:1801.01275.
  28. Ng, A.Y. (2004). Feature selection, l 1 vs. l 2 regularization, and rotational invariance. In Proceedings of the twenty-first international conference on machine learning.Google Scholar
  29. Peters, F., Tun, T., Yu, Y., Nuseibeh, B. (2017). Text filtering and ranking for security bug report prediction. IEEE Transactions on Software Engineering, PP (99), 1–1.Google Scholar
  30. Qiao, Y., Zheng, Z., Yunyu, F. (2018). Two-level rejuvenation for android smartphones and its optimization. IEEE Transactions on Reliability.Google Scholar
  31. Qin, F., Zheng, Z., Li, X., Yu, Q., Trivedi, K.S. (2017). An empirical investigation of fault triggers in android operating system. In IEEE Pacific rim international symposium on dependable computing.Google Scholar
  32. Qin, F., Zheng, Z., Qiao, Y., Trivedi, K.S. (2018). Studying aging-related bug prediction using cross-project models. IEEE Transactions on Reliability.Google Scholar
  33. Romano, J. (2006). Appropriate statistics for ordinal level data: should we really be using t-test and cohen’sd for evaluating group differences on the nsse and other surveys. In Annual meeting of the Florida association of institutional research.Google Scholar
  34. Scikit learn. (2019). https://scikit-learn.org/.
  35. Shu, R., Xia, T., Williams, L., Menzies, T. (2019). Better security bug report classification via hyperparameter optimization. In International conference on automated software engineering, 2019 (pp. 1–12): IEEE/ACM.Google Scholar
  36. Storn, R., & Price, K. (1997). Differential evolution–a simple and efficient heuristic for global optimization over continuous spaces. Journal of Global Optimization, 11(4), 341–359.MathSciNetCrossRefGoogle Scholar
  37. Umesh, I.M., & Srinivasan, G.N. (2017). Dynamic software aging detection-based fault tolerant software rejuvenation model for virtualized environment. In Proceedings of the international conference on data engineering and communication technology. Springer, Singapore.Google Scholar
  38. Wang, Z., Wu, C., Yuan, X., Wang, Z., Li, J. (2018). Using local clocks to reproduce concurrency bugs. IEEE Transactions on Software Engineering, 44(11), 1112–1128.CrossRefGoogle Scholar
  39. Yang, X.L., Lo, D., Xia, X., Huang, Q., Sun, J.L. (2017). High-impact bug report identification with imbalanced learning strategies. Journal of Computer Science and Technology, 32, 181–198.CrossRefGoogle Scholar
  40. Yu, T., Wei, W., Xue, H., Hayes, J. (2018). Conpredictor: concurrency defect prediction in real-world applications. IEEE Transactions on Software Engineering, PP (99), 1–1.CrossRefGoogle Scholar
  41. Zhang, Y., Lo, D., Xia, X., Sun, J. (2015). An empirical study of classifier combination for cross-project defect prediction. In IEEE computer software and applications conference.Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2020

Authors and Affiliations

  1. 1.Northwestern Polytechnical UniversityXi’anChina

Personalised recommendations