Performant Peer Review for Design Science Manuscripts: A Pilot Study on Dedicated Highlighters

  • Oscar Díaz
  • Jeremías P. ContellEmail author
  • Haritz Medina
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11491)


Peer review is under pressure. Demand for reviews is outstripping supply where reviewers tend to be busy people who contribute voluntarily. Authors highly value reviews, yet complain about the time it takes to get feedback to the point of putting research timeliness at stake. Though part of the review process has been moved to the Web, the review itself is still often conducted with the only help of a yellow highlighter, physical or digital. This work looks for more performant highlighters that account for the review specifics. Peer review does not stop at spotting the manuscript (de)merits, it also strives for manuscript improvement and gatekeeping. These functions are conducted within an often tacit research-quality framework, and frequently in a discontinuous way. Unfortunately, when it comes to support review practices, current facilities fall short. This work introduces a set of requirements for review-dedicated highlighters. These requirements are instantiated and evaluated through Review&Go, a color-coding highlighter that generates a review draft out of the reviewer’s highlighting activities. The aim is to offer representational guidance to enhance context/cognitive awareness so that reviewers can exert less effort while offering valuable and timely reviews.


Peer review Annotation Design Science Research 



This work is supported by the University of the Basque Country under contract US17/13. Haritz Medina enjoys a grant from the same University.


  1. 1.
    Adobe: Importing and exporting comments (2017). Accessed 22 Mar 2019
  2. 2.
    Basili, V.R., Caldiera, G., Rombach, H.D.: Goal question metric paradigm. Encycl. Softw. Eng. 1, 528–532 (1994)Google Scholar
  3. 3.
    Clarke, M.: Reducing the peer-reviewer’s burden (2010). Accessed 22 Mar 2019
  4. 4.
    DeMaria, A.N.: Peer review: the weakest link. J. Am. Coll. Cardiol. 55(11), 1161–1162 (2010)CrossRefGoogle Scholar
  5. 5.
    Díaz, O., Contell, Jeremías P., Venable, J.R.: Strategic reading in design science: let root-cause analysis guide your readings. In: Maedche, A., vom Brocke, J., Hevner, A. (eds.) DESRIST 2017. LNCS, vol. 10243, pp. 231–246. Springer, Cham (2017). Scholar
  6. 6.
    Enago: Experts’ take on peer review evaluation (PRE) (2018). Accessed 22 Mar 2019
  7. 7.
    Grant, R.P.: On peer review (2010). Accessed 22 Mar 2019
  8. 8.
    Gregor, S., Hevner, A.R.: Positioning and presenting design science research for maximum impact. MIS Q. 37(2), 337–355 (2013)CrossRefGoogle Scholar
  9. 9.
    Hames, I.: Peer review in a rapidly evolving publishing landscape. In: Academic and Professional Publishing, pp. 15–52. Elsevier (2012)Google Scholar
  10. 10.
    Hanson, B., Panning, J., Townsend, R., Wooden, P.: Annotation tool facilitates peer review. In: EOS, vol. 98 (2017)Google Scholar
  11. 11.
    Johannesson, P., Perjons, E.: An Introduction to Design Science. Springer, Cham (2014). Scholar
  12. 12.
    Lawrence, M.: Familiarity: A UX consideration (2017). (accessed 22 Mar. 2019)
  13. 13.
    Lovejoy, T.I., Revenson, T.A., France, C.R.: Reviewing manuscripts for peer-review journals: A primer for novice and seasoned reviewers. Ann. Behav. Med. 42(1), 1–13 (2011)CrossRefGoogle Scholar
  14. 14.
    McEwan, E.K.: Seven Strategies of Highly Effective Readers: Using Cognitive Research to Boost K-8 Achievement. Corwin press, Thousand Oaks (2004)Google Scholar
  15. 15.
    Nicol, D.: From monologue to dialogue: improving written feedback processes in mass higher education. Assess. Eval. High. Educ. 35(5), 501–517 (2010)CrossRefGoogle Scholar
  16. 16.
    Nicol, D.: Good designs for Written Feedback to Students. McKeachie’s Teaching Tips: Strategies, Research, and Theory for College and University Teachers, pp. 108–124. Cengage Learning (2010)Google Scholar
  17. 17.
    Publishing Research Consortium (PRC): Peer review survey 2015 (2016). Accessed 22 Mar 2019
  18. 18.
    Smith, R.: What is peer review? is peer review effective? classical peer review: an empty gun. Breast Cancer Res. 12(Suppl 4), 13 (2010)CrossRefGoogle Scholar
  19. 19.
    Spyns, P., Vidal, M.-E.: Scientific Peer Reviewing. Practical Hints and Best Practices. Springer, Cham (2015). Scholar
  20. 20.
    Tennant, J.P., Dugan, J.M., Graziotin, D., et al.: A multi-disciplinary perspective on emergent and future innovations in peer review. F1000Research 6, 1151 (2017)CrossRefGoogle Scholar
  21. 21.
    Tenopir, C., et al.: Research Publication Characteristics and Their Relative Values: A (2010)Google Scholar
  22. 22.
    Tornatzky, L.G., Klein, K.J.: Innovation characteristics and innovation adoption-implementation: A meta-analysis of findings. IEEE Trans. Eng. Manage. EM 29(1), 28–45 (1982)CrossRefGoogle Scholar
  23. 23.
    Vaishnavi, V.K., Kuechler, W.: Design Science Research Methods and Patterns: Innovating Information and Communication Technology. CRC Press, Boca Raton (2015)CrossRefGoogle Scholar
  24. 24.
    Venable, John R.: Five and ten years on: have DSR standards changed? In: Donnellan, B., Helfert, M., Kenneally, J., VanderMeer, D., Rothenberger, M., Winter, R. (eds.) DESRIST 2015. LNCS, vol. 9073, pp. 264–279. Springer, Cham (2015). Scholar
  25. 25.
    Ware, M.: Online submission and peer-review systems. Learn. Publish. 18(4), 245–250 (2005)CrossRefGoogle Scholar
  26. 26.
    Ware, M.: Peer review: recent experience and future directions. New Rev. Inf. Netw. 16(1), 23–53 (2011)CrossRefGoogle Scholar
  27. 27.
    Ware, M., Mabe, M.: The STM report: An overview of scientific and scholarly journals publishing (2009). Accessed 22 Mar 2019

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Oscar Díaz
    • 1
  • Jeremías P. Contell
    • 1
    Email author
  • Haritz Medina
    • 1
  1. 1.ONEKIN Web Engineering GroupUniversity of the Basque Country (UPV/EHU)San SebastiánSpain

Personalised recommendations