1. State Key Laboratory of Software Engineering, School of Computer, Wuhan University, Wuhan 430072, China
2. School of Software, Dalian University of Technology, Dalian 116621, China
3. Microsoft Research Asia, Beijing 100080, China
4. School of Electrical Engineering and Computer Science, The University of Newcastle, Callaghan NSW 2308, Australia
A bug tracking system provides a collaborative platform for the developer crowd. After a bug report is submitted, developers can make comments to supplement the details of the bug report. Due to the large number of developers and bug reports, it is hard to determine which developer (also called commenter) is able to comment on a particular bug report. We refer to the problem of recommending developers for commenting on bug reports as commenter recommendation. In this paper, we perform an empirical analysis on commenter recommendation based on five-year bug reports of four open source projects. First, we preliminarily analyze bug comments and commenters in three categories, the relationship between commenters and fixers, the data scale of comments, and the collaboration on bug commenting. Second, we design a recommendation approach via ranking developers in the crowd to reduce the manual effort of identifying commenters. In this approach, we formulize the commenter recommendation problem as a multi-label recommendation task and leverage both developer collaboration and bug content to find out appropriate commenters. Experimental results show that our approach can effectively recommend commenters; 41% to 75% of the recall value is achieved for top-10 recommendation. Our empirical analysis on bug commenting can help developers understand and improve the process of fixing bugs.
This work was supported by National Natural Science Foundation of China (Grant Nos. 61502345, 61403057, 61370144, 61272089) and New Century Excellent Talents in University (Grant No. NCET-13-0073).
(Color online) A part of bug report for bug 597930 in Firefox and its two comments. This bug describes an error about a regular expression for the location bar.
(Color online) Illustration of the number of bugs with a specific number of commenters or comments for Firefox. For example, a point in the illustration with 5 comments ($x$-axis) and 800 bugs ($y$-axis) denotes that there exist 800 bugs in Firefox, each of which consists of 5 comments.
Illustration of comments for top 100 developers in five-year bugs in Firefox. Diameter of a circle denotes the average number of comments per bug for each developer. For example, the largest circles denote developers who contribute to over four comments per bug. Note that the vertical axis is on a log scale. Top 100 developers refer to the 100 developers who make comments on the largest numbers of bug reports.
(Color online) Illustration of commenter recommendation via ranking for the developer crowd. In collaboration ranking, developers are ranked based on historical collaboration; in capability ranking, developers are ranked based on specific capability on bug reports. The final recommendation is formed by integrating the above two developer rankings.
(Color online) Example of bug field based collaborative filtering for developer ranking.
(Color online) Measurement comparison in capability ranking on Firefox based on Recall@5 and Recall@10. Each result is an average of results in ten rounds.
(Color online) Performance of Recall@5 in the integrated approach when changing the weight factor on four projects. Each result is an average of results in ten rounds.
Copyright 2020 Science China Press Co., Ltd. 《中国科学》杂志社有限责任公司 版权所有