Abstract
Peer review is the main mechanism for quality evaluation and peer-mentoring in the research community. Yet, it has been criticized with respect to its summative function, as being prone to bias and inconsistency and approaches had been proposed to improve it (e.g. double blind review). However, relatively less attention has been paid on how well it meets its formative objective, i.e. providing useful feedback to help the authors improve their quality of work. In our previous work we proposed a modified peer review process, which involved a back-evaluation of reviews by the authors. This paper reports the results of a study of the application of this peer review process to support a group of teachers in Chile engage in group peer mentorship in the context of a summer continuing education course. The objectives are to find out if authors reciprocate their reviews feedback in the back-evaluation given to their reviewers, and if the review length affects the helpfulness and authors’ satisfaction with the reviews. Our results showed that peers did not reciprocate their ratings and review length did not affect peers’ satisfaction with the reviews.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Adewoyin, O., Vassileva, J.: Can online peer-review systems support group mentorship? In: Lane, H., Yacef, K., Mostow, J., Pavlik, P. (eds.) AIED 2013. LNCS, vol. 7926, pp. 737–741. Springer, Heidelberg (2013)
Adewoyin, O., Vassileva, J.: Ethics of scientific peer review: are we judging or helping the review recipients? In: IEEE International Symposium on Ethics in Engineering, Science, and Technology, Ethics 2014 (2014)
Cestone, C.M., Levine, R.E., Lane, D.R.: Peer assessment and evaluation in team-based learning. New Directions Teaching Learn. 116, 69–78 (2008)
Farrell, K.: Collegial feedback on teaching: a guide to peer review. Centre for the Study of Higher Education, The University of Melbourne (2011)
Fernandez, C.E., Yu, J.: Peer review of teaching. J. Chiropractic Educ. 21(2), 154–161 (2007)
Gutknecht-Gmeiner, M.: Peer review in education. Peer review in initial VET, Leornado da Vinci project, Austria (2005)
Houry, D., Green, S., Callaham, M.: Does mentoring new peer reviewers improve review quality? A randomized trial. BMC Med. Educ. 12(1), 83 (2012)
Hutchings, P.: Peer review of teaching. “From Idea to Prototype”. In: AAHE Bulletin, November 1994
Kronick, D.A.: Peer review in 18th century scientific journalism. J. Am. Medical Assoc. (JAMA) 263, 1321–1322 (1990)
Levine, R.E.: Peer evaluation in team-based learning (2010). https://training.health.ufl.edu/handouts/FacDev/TBL_Chapter9.pdf. (Accessed on 17 October 2015)
Pearce, J., Mulder, R., Baik, C.: Involving students in peer review: case studies and practical strategies for University teaching. University of Melbourne, Victoria (2009). http://www.cshe.unimelb.edu.au/resources_teach/teaching_in_practice/docs/Student_Peer_Review.pdf. (Accessed on 17 October 2015)
Ranalli, B.: A prehistory of peer review: religious blueprints from the hartlib circle spontaneous generations. J. History Philos. Sci. 5(1), 12–18 (2011)
Sachs, J., Parsell, M.: Peer review of learning and teaching in higher education. British J. Educ. Technol. 45(3) (2014)
Schroter, S., Black, N., Evans, S., Carpenter, J., Godlee, F., Smith, R.: Effects of training on quality of peer review: randomised controlled trial. BMJ 2004(328), 673 (2004)
Searby, M., Ewers, T.: An evaluation of the use of peer assessment in higher education: a case study in the school of music. Assess. Eval. Higher Educ. 22(4), 371–383 (1997). Kingston University
Smith, R.: Peer review: a flawed process at the heart of science and journals. J. R. Soc. Med. 99, 178–182 (2006)
Turner, S.; Perez-Quinones, M.A., Chase, J.: Peer review in CS2: conceptual learning. In: SIGCSE 2010 Proceedings of the 41st ACM Technical Symposium on Computer Science Education, pp. 331–335
Van Rooyen, S., Godlee, F., Evans, S., Black, N., Smith, R.: Effect of open peer review on quality of reviews and on reviewers’ recommendations: a randomised trial. Br. Med. J. 318, 23–27 (1999)
Vygotsky, L.S.: Mind in Society: The Development of Higher Psychological Processes. Harvard University Press, Cambridge (1978)
Wagner, W., Steinzor, R.: Rescuing Science from Politics Regulation and the Distortion of Scientific Research, 1st edn. Cambridge University Press, Cambridge (2006)
Xiong, W., Litman, D., Schunn, C.: Assessing reviewers’ performance based on mining problem localization in peer-review data. In: Proceedings of the Third International Conference on Educational Data Mining (EDM 2010), pp. 211–220 (2010)
Acknowledgements
We thank Paulina Sepúlveda for implementing the peer review system and to Abelino Jiménez and Josefina Hernández for initial data preprocessing. We thank Francisco Gutierrez for his help with the translation of user comments from Spanish to English. This research was possible with funding from NSERC Discovery Grants Program to the 3rd author and from Basal Funds for Centers of Excellence Project BF 0003 from the Associative Research Program of CONICYT to the 2nd author.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Adewoyin, O., Araya, R., Vassileva, J. (2016). Peer Review in Mentorship: Perception of the Helpfulness of Review and Reciprocal Ratings. In: Micarelli, A., Stamper, J., Panourgia, K. (eds) Intelligent Tutoring Systems. ITS 2016. Lecture Notes in Computer Science(), vol 9684. Springer, Cham. https://doi.org/10.1007/978-3-319-39583-8_31
Download citation
DOI: https://doi.org/10.1007/978-3-319-39583-8_31
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-39582-1
Online ISBN: 978-3-319-39583-8
eBook Packages: Computer ScienceComputer Science (R0)