Journal of Central South University

, Volume 25, Issue 5, pp 1129–1143 | Cite as

RevRec: A two-layer reviewer recommendation algorithm in pull-based development model

  • Cheng Yang (杨程)
  • Xun-hui Zhang (张迅晖)
  • Ling-bin Zeng (曾令斌)
  • Qiang Fan (范强)
  • Tao Wang (王涛)
  • Yue Yu (余跃)
  • Gang Yin (尹刚)
  • Huai-min Wang (王怀民)


Code review is an important process to reduce code defects and improve software quality. In social coding communities like GitHub, as everyone can submit Pull-Requests, code review plays a more important role than ever before, and the process is quite time-consuming. Therefore, finding and recommending proper reviewers for the emerging Pull-Requests becomes a vital task. However, most of the current studies mainly focus on recommending reviewers by checking whether they will participate or not without differentiating the participation types. In this paper, we develop a two-layer reviewer recommendation model to recommend reviewers for Pull-Requests (PRs) in GitHub projects from the technical and managerial perspectives. For the first layer, we recommend suitable developers to review the target PRs based on a hybrid recommendation method. For the second layer, after getting the recommendation results from the first layer, we specify whether the target developer will technically or managerially participate in the reviewing process. We conducted experiments on two popular projects in GitHub, and tested the approach using PRs created between February 2016 and February 2017. The results show that the first layer of our recommendation model performs better than the previous work, and the second layer can effectively differentiate the types of participation.

Key words

Pull-Request code reviewer recommendation GitHub open source community 

RevREC:一个基于Pull-Request 开发模型的双层审阅人推荐算法


代码审查是减少代码缺陷和提高软件质量的重要过程。在像GitHub 这样的社交编码社区,由 于每个人都可以提交Pull-Request,所以代码审查扮演着比以往更重要的角色,而且这个过程非常耗 时。因此,寻找并推荐正确的评审人员来应对新兴的Pull-Request 成为一项重要任务。然而,目前大 部分的研究主要集中在评估人员是否参与,并没有对人员参与的类型进行区分。在本文中,我们开发 了一个两层审阅人推荐模型,从技术和管理角度为GitHub 项目中的Pull-Request(PR)推荐审阅人。对 于第一层,我们根据混合推荐方法推荐合适的审阅人对目标PR 进行审阅。对于第二层,在从第一层 获得推荐结果之后,我们指定被推荐的审阅人是技术还是管理上参与审阅过程。我们在GitHub 的两 个热门项目上进行了实验,并使用2016 年2 月至2017 年2 月期间创建的PR 来测试该方法。结果显 示,我们的推荐模型的第一层比以前的工作表现得更好,第二层可以有效地区分参与类型。


Pull-Request 代码审阅人推荐 GitHub 开源社区 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. [1]
    BOEHM B, ROMBACH H D, ZELKOWITZ M V. Software defect reduction top 10 list [M]// Foundations of Empirical Software Engineering: the Legacy of Victor R. Basili, Springer, Verlag New York, Inc., 2005.CrossRefGoogle Scholar
  2. [2]
    THONGTANUNAM P, TANTITHAMTHAVORN C, KULA R G, YOSHIDA V, IIDA H, MATSUMOTO K I. Who should review my code? A file location-based code-reviewer recommendation approach for modern code review [C]// Software Analysis, Evolution and Reengineering (SANER), 2015 IEEE 22nd International Conference on. IEEE, 2015: 141–150.CrossRefGoogle Scholar
  3. [3]
    KOLLANUS S, KOSKINEN J. Survey of software inspection research [J]. The Open Software Engineering Journal, 2009, 3(1): 15–34.CrossRefGoogle Scholar
  4. [4]
    RIGBY P C, STOREY M A. Understanding broadcast based peer review on open source software projects [C]// Proceedings of the 33rd International Conference on Software Engineering. ACM, 2011: 541–550.Google Scholar
  5. [5]
    BALACHANDRAN V. Reducing human effort and improving quality in peer code reviews using automatic static analysis and reviewer recommendation [C]// Software Engineering (ICSE), 2013 35th International Conference on. IEEE, 2013: 931–940.CrossRefGoogle Scholar
  6. [6]
    JEONG G, KIM S, ZIMMERMANN T, YI K. Improving code review by predicting reviewers and acceptance of patches [M]// Research on Software Analysis for Error-free Computing Center Tech-Memo (ROSAEC MEMO 2009-006), 2009: 1–18.Google Scholar
  7. [7]
    YU Y, WANG H, YIN G, WANG T. Reviewer recommendation for pull-requests in GitHub: What can we learn from code review and bug assignment? [J]. Information and Software Technology, 2016, 74: 204–218.CrossRefGoogle Scholar
  8. [8]
    BROY M, DENERT E. Pioneers and their contributions to software engineering [M]. Berlin Heidelberg, Springer: 2001.CrossRefzbMATHGoogle Scholar
  9. [9]
    MCLNTOSH S, KAMEI Y, ADAMS B, HASSAN A E. The impact of code review coverage and code review participation on software quality: A case study of the qt, vtk, and itk projects [C]// Proceedings of the 11th Working Conference on Mining Software Repositories. ACM, 2014: 192–201.Google Scholar
  10. [10]
    MCINTOSH S, KAMEI Y, ADAMS B, HASSAN A E. An empirical study of the impact of modern code review practices on software quality [J]. Empirical Software Engineering, 2016, 21(5): 2146–2189.CrossRefGoogle Scholar
  11. [11]
    BOSU A, CARVER J C. Peer code review to prevent security vulnerabilities: An empirical evaluation [C]// Software Security and Reliability-Companion (SERE-C), 2013 IEEE 7th International Conference on. IEEE, 2013: 229–230.CrossRefGoogle Scholar
  12. [12]
    MORALES R, MCLNTOSH S, KHOMH F. Do code review practices impact design quality? a case study of the qt, vtk, and itk projects [C]// Software Analysis, Evolution and Reengineering (SANER), 2015 IEEE 22nd International Conference on. IEEE, 2015: 171–180.CrossRefGoogle Scholar
  13. [13]
    MUKADAM M, BIRD C, RIGBY P C. Gerrit software code review data from android [C]// Mining Software Repositories (MSR), 2013 10th IEEE Working Conference on. IEEE, 2013: 45–48.CrossRefGoogle Scholar
  14. [14]
    GOUSIOS G, ZAIDMAN A, STOREY M A, van DEURSEN A. Work practices and challenges in pull-based development: the integrator's perspective [C]// Proceedings of the 37th International Conference on Software Engineering: Volume 1. IEEE, 2015: 358–368.Google Scholar
  15. [15]
    PHAM R, SINGER L, LISKIN O, FIGUEIRAFILHO F, SCHNEIDER K. Creating a shared understanding of testing culture on a social coding site [C]// Software Engineering (ICSE), 2013 35th International Conference on. IEEE, 2013: 112–121.CrossRefGoogle Scholar
  16. [16]
    ZHU J, ZHOU M, MOCKUS A. Effectiveness of code contribution: From patch-based to pull-request-based tools [C]// Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering. ACM, 2016: 871–882.CrossRefGoogle Scholar
  17. [17]
    YU Y, WANG H, YIN G, LING C X. Reviewer recommender of pull-requests in GitHub [C]// Software Maintenance and Evolution (ICSME), 2014 IEEE International Conference on. IEEE, 2014: 609–612.CrossRefGoogle Scholar
  18. [18]
    YANG C, ZHANG X, ZENG L, FAN Q, YIN G, WANG H. An empirical study of reviewer recommendation in pull-based development model [C]// Proceedings of the 9th Asia-Pacific Symposium on Internetware. ACM, 2017: 14.Google Scholar
  19. [19]
    JEONG G, KIM S, ZIMMERMANN T. Improving bug triage with bug tossing graphs [C]// Proceedings of the 7th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on the Foundations of Software Engineering. ACM, 2009: 111–120.Google Scholar
  20. [20]
    KAGDI H, POSHYVANYK D. Who can help me with this change request? [C]// Program Comprehension, 2009. ICPC'09. IEEE 17th International Conference on. IEEE, 2009: 273–277.CrossRefGoogle Scholar
  21. [21]
    BHATTACHARYA P, NEAMTIU I. Fine-grained incremental learning and multi-feature tossing graphs to improve bug triaging [C]// Software Maintenance (ICSM), 2010 IEEE International Conference on. IEEE, 2010: 1–10.Google Scholar
  22. [22]
    CANFORA G, CERULO L. Supporting change request assignment in open source development [C]// Proceedings of the 2006 ACM Symposium on Applied Computing. ACM, 2006: 1767–1772.CrossRefGoogle Scholar
  23. [23]
    LINARES-VÁSQUEZ M, HOSSEN K, DANG H, KAGDI H, GETHERS M, POSHYVANYK D. Triaging incoming change requests: Bug or commit history, or code authorship? [C]// Software Maintenance (ICSM), 2012 28th IEEE International Conference on. IEEE, 2012: 451–460.CrossRefGoogle Scholar
  24. [24]
    JONSSON L, BORG M, BROMAN D, SANDAHL K, ELDH S, RUNESON P. Automated bug assignment: Ensemble-based machine learning in large scale industrial contexts [J]. Empirical Software Engineering, 2016, 21(4): 1533–1578.CrossRefGoogle Scholar
  25. [25]
    TAMRAWI A, NGUYEN T T, AL-KOFAHI J M, NGUYEN T N. Fuzzy set and cache-based approach for bug triaging [C]// Proceedings of the 19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering. ACM, 2011: 365–375.Google Scholar
  26. [26]
    XUAN J, JIANG H, REN Z, YAN J, LUO Z. Automatic bug triage using semi-supervised text classification [J]. arXiv: 1704.04769, 2017.Google Scholar
  27. [27]
    HAN D, ZHUO H, XIA L, LI L. Permission and role automatic assigning of user in role-based access control [J]. Journal of Central South University, 2012, 19(4): 1049–1056.CrossRefGoogle Scholar
  28. [28]
    ANVIK J, HIEW L, MURPHY G C. Who should fix this bug? [C]// Proceedings of the 28th International Conference on Software Engineering. ACM, 2006: 361–370.Google Scholar
  29. [29]
    SHOKRIPOUR R, ANVIK J, KASIRUN Z M, ZAMANI S. Improving automatic bug assignment using time-metadata in term-weighting [J]. IET Software, 2014, 8(6): 269–278.CrossRefGoogle Scholar
  30. [30]
    ZHANG X, WANG T, YIN G, YANG C, YU Y, WANG H. DevRec: A developer recommendation system for open source repositories [C]// International Conference on Software Reuse. Springer, 2017: 3–11.Google Scholar
  31. [31]
    ZHANG X, WANG T, YIN G, YANG C, WANG H. Who will be interested in? A contributor recommendation approach for open source projects [C]// Proceedings of the 29th International Conference on Software Engineering & Knowledge Engineering, 10.182931SEKE2017-067.Google Scholar
  32. [32]
    THONGTANUNAM P, KULA R G, CRUZ A E C, YOSHIDA N, IIDA H. Improving code review effectiveness through reviewer recommendations [C]// Proceedings of the 7th International Workshop on Cooperative and Human Aspects of Software Engineering. ACM, 2014: 119–122.Google Scholar
  33. [33]
    RAHMAN M M, ROY C K, COLLINS J A. CoRReCT: Code reviewer recommendation in GitHub based on cross-project and technology experience [C]// Software Engineering Companion (ICSE-C), IEEE/ACM International Conference on. IEEE, 2016: 222–231.Google Scholar
  34. [34]
    YU Y, WANG H, YIN G, LING C X. Who should review this pull-request: Reviewer recommendation to expedite crowd collaboration [C]// Software Engineering Conference (APSEC), 2014 21st Asia-Pacific. IEEE, 2014, 1: 335–342.CrossRefGoogle Scholar
  35. [35]
    ZANJANI M B, KAGDI H, BIRD C. Automatically recommending peer reviewers in modern code review [J]. IEEE Transactions on Software Engineering, 2016, 42(6): 530–543.CrossRefGoogle Scholar
  36. [36]
    XIA Z, SUN H, JIANG J, WANG X, LIU X. A hybrid approach to code reviewer recommendation with collaborative filtering [C]// 2017 6th International Workshop on Software Mining (Software Mining). IEEE, 2017: 24–31.CrossRefGoogle Scholar

Copyright information

© Central South University Press and Springer-Verlag GmbH Germany, part of Springer Nature 2018

Authors and Affiliations

  • Cheng Yang (杨程)
    • 1
  • Xun-hui Zhang (张迅晖)
    • 1
  • Ling-bin Zeng (曾令斌)
    • 1
  • Qiang Fan (范强)
    • 1
  • Tao Wang (王涛)
    • 1
  • Yue Yu (余跃)
    • 1
  • Gang Yin (尹刚)
    • 1
  • Huai-min Wang (王怀民)
    • 1
  1. 1.National Laboratory for Parallel and Distributed Processing, College of ComputerNational University of Defense TechnologyChangshaChina

Personalised recommendations