Abstract
The purpose of this working group was to identify which role controlled experiments play in the field of empirical Software Engineering. The discussions resulted in a list of motivational factors, challenges, and improvements suggestions. The main outcome is that, although the empirical Software Engineering community, over the last 14 years, has matured with regard to doing controlled experiments, there is still room for improvement. By now the community has understood under which conditions it is possible to empirically evaluate Software Engineering methods, techniques, and tools, but the way controlled experiments are designed, performed, and reported still lacks the level of quality of other disciplines such as social sciences or medicine. The generalizability of the results of controlled experiments is one major concern. Furthermore, more emphasis should be put on the role of empirical Software Engineering education.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Carver, J., Jaccheri, L., Morasca, S., Shull, F.: Issues Using Students in Empirical Studies in Software Engineering Education. In: Proceedings of 2003 International Symposium on Software Metrics (METRICS 2003), September, pp. 239–249 (2003)
Dybå, T., Kampenes, V.B., Sjøberg, D.: A Systematic Review of Statistical Power in Software Engineering Experiments. Information and Software Technology 48(8), 745–755 (2006)
Höst, M., Regnell, B., Wohlin, C.: Using Students as Subjects - A Comparative Study of Students and Professionals in Lead-Time Impact Assessment. Empirical Software Engineering 5(3), 201–214 (2000)
Jedlitschka, A., Pfahl, D.: Reporting Guidelines for Controlled Experiments in Software Engineering. In: Proc. of ACM/IEEE Intern. Symposium on Software Engineering 2005 (ISESE2005), Noosa Heads, Australia, Nov. 2005, pp. 95–104. IEEE Computer Society Press, Los Alamitos (2005)
Kitchenham, B.A., Pfleeger, S.L., Pickard, L.M., Jones, P.W., Hoaglin, D.C., El Emam, K., Rosenberg, J.: Preliminary guidelines for empirical research in software engineering. IEEE Transactions on Software Engineering 28(8), 721–734 (2002)
Carolyn, B.: Seaman, Qualitative Methods in Empirical Studies of Software Engineering. IEEE Transactions on Software Engineering 25(4), 557–572 (1999)
Sjøberg, D., Hannay, J.E., Hansen, O., Kampenes, V.B., Karahasanovic, A., Liborg, N.-K., Rekdal, A.C.: A Survey of Controlled Experiments in Software Engineering. IEEE Transactions on Software Engineering 31(9), 733–753 (2004)
Editor information
Rights and permissions
Copyright information
© 2007 Springer Berlin Heidelberg
About this chapter
Cite this chapter
Jedlitschka, A., Briand, L.C. (2007). The Role of Controlled Experiments Working Group Results . In: Basili, V.R., Rombach, D., Schneider, K., Kitchenham, B., Pfahl, D., Selby, R.W. (eds) Empirical Software Engineering Issues. Critical Assessment and Future Directions. Lecture Notes in Computer Science, vol 4336. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-71301-2_19
Download citation
DOI: https://doi.org/10.1007/978-3-540-71301-2_19
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-71300-5
Online ISBN: 978-3-540-71301-2
eBook Packages: Computer ScienceComputer Science (R0)