Peer Assessment Improvement Using Fuzzy Logic

  • Mohamed El AlaouiEmail author
  • Khalid El Yassini
  • Hussain Ben-Azza
Conference paper
Part of the Lecture Notes in Intelligent Transportation and Infrastructure book series (LNITI)


Peer assessment, consists of a prearrangement between learners to consider and specify the level, value, or quality of a product or performance or other equal-status learners. The practice imposes itself when trying to evaluate a large number of students, teachers are practically obliged to use peer assessment, especially in Massive Open Online Courses (MOOCs). However, the novice students, unlike their teachers, are not formed to assess others contributions. Therefore, their evaluations are unreliable and may be biased. Here we try to improve the peer assessment outcome, using fuzzy logic to model opinions, those opinions are weighed according to their validity, then aggregated in order to achieve consensus, hence reliable evaluation.


Peer assessment Validity Reliability Group decision making Massive open online course Fuzzy logic Weighting opinions 


  1. 1.
    Lewis Gaillet, L.: A Foreshadowing of modern theories and practices of collaborative Learning: The work of scottish rhetorician george Jardine. In: Presented at the 43rd Annual Meeting of the Conference on College Composition and Communication, Cincinnati OH, Mar 19 (1992)Google Scholar
  2. 2.
    García-Peñalvo, F.J., Fidalgo-Blanco, Á., Sein-Echaluce, M.L.: An adaptive hybrid MOOC model: disrupting the MOOC concept in higher education. Telemat. Inform. 35, 1018–1030 (2018)CrossRefGoogle Scholar
  3. 3.
    Giovannella, C., Martens, A., Zualkernan, I.: Grand challenge problem 1: people centered smart “cities” through smart city learning. In: Grand Challenge Problems in Technology-Enhanced Learning II: MOOCs and Beyond. pp. 7–12. Springer, Cham (2016)Google Scholar
  4. 4.
    Haber, J.: MOOCs. The MIT Press, Cambridge, Massachusetts (2014)CrossRefGoogle Scholar
  5. 5.
    Alario-Hoyos, C., Pérez-Sanagustín, M., Delgado-Kloos, C.G.H.A.P., Muñoz-Organero, M., Rodríguez-de-las-Heras, A.: Analysing the impact of Built-In and external social tools in a MOOC on educational technologies. In: Scaling up Learning for Sustained Impact. pp. 5–18. Springer, Berlin, Heidelberg (2013)Google Scholar
  6. 6.
    Formanek, M., Wenger, M.C., Buxner, S.R., Impey, C.D., Sonam, T.: Insights about large-scale online peer assessment from an analysis of an astronomy MOOC. Comput. Educ. 113, 243–262 (2017)CrossRefGoogle Scholar
  7. 7.
    Ho, D., McAllister, S.: Are health professional competency assessments transferable across cultures? a preliminary validity study. Assess. Eval. High. Educ. 43, 1069–1083 (2018)CrossRefGoogle Scholar
  8. 8.
    Wilson, M.J., Diao, M.M., Huang, L.: ‘I’m not here to learn how to mark someone else’s stuff’: an investigation of an online peer-to-peer review workshop tool. Assess. Eval. High. Educ. 40, 15–32 (2015)CrossRefGoogle Scholar
  9. 9.
    Usher, M., Barak, M.: Peer assessment in a project-based engineering course: comparing between on-campus and online learning environments. Assess. Eval. High. Educ. 43, 745–759 (2018)CrossRefGoogle Scholar
  10. 10.
    Bordel, B., Alcarria, R., Martín, D., Sánchez-de-Rivera, D.: Improving MOOC student learning through enhanced peer-to-peer tasks. In: Digital Education: Out to the World and Back to the Campus. pp. 140–149. Springer, Cham (2017)CrossRefGoogle Scholar
  11. 11.
    Mulder, R., Baik, C., Naylor, R., Pearce, J.: How does student peer review influence perceptions, engagement and academic outcomes? A case study. Assess. Eval. High. Educ. 39, 657–677 (2014)CrossRefGoogle Scholar
  12. 12.
    Meek, S.E.M., Blakemore, L., Marks, L.: Is peer review an appropriate form of assessment in a MOOC? Student participation and performance in formative peer review. Assess. Eval. High. Educ. 42, 1000–1013 (2017)CrossRefGoogle Scholar
  13. 13.
    Suen, H.K.: Peer assessment for massive open online courses (MOOCs). Int. Rev. Res. Open Distrib. Learn. 15, 312–327 (2014)Google Scholar
  14. 14.
    Ashton, S., Davies, R.S.: Using scaffolded rubrics to improve peer assessment in a MOOC writing course. Distance Educ. 36, 312–334 (2015)CrossRefGoogle Scholar
  15. 15.
    Love, K.G.: Comparison of peer assessment methods: reliability, validity, friendship bias, and user reaction. J. Appl. Psychol. 66, 451–457 (1981)CrossRefGoogle Scholar
  16. 16.
    Cho, K., Schunn, C.D., Wilson, R.W.: Validity and reliability of scaffolded peer assessment of writing from instructor and student perspectives. J. Educ. Psychol. 98, 891–901 (2006)CrossRefGoogle Scholar
  17. 17.
    Speyer, R., Pilz, W., Van Der Kruis, J., Brunings, J.W.: Reliability and validity of student peer assessment in medical education: a systematic review. Med. Teach. 33, e572–e585 (2011)CrossRefGoogle Scholar
  18. 18.
    Schunn, C., Godley, A., DeMartino, S.: The reliability and validity of peer review of writing in high school AP english classes. J. Adolesc. Adult Lit. 60, 13–23 (2016)CrossRefGoogle Scholar
  19. 19.
    Salehi, M., Masoule, Z.S.: An investigation of the reliability and validity of peer, self-, and teacher assessment. South. Afr. Linguist. Appl. Lang. Stud. 35, 1–15 (2017)CrossRefGoogle Scholar
  20. 20.
    Yoon, H.B., Park, W.B., Myung, S.-J., Moon, S.H., Park, J.-B.: Validity and reliability assessment of a peer evaluation method in team-based learning classes. Korean J. Med. Educ. 30, 23–29 (2018)CrossRefGoogle Scholar
  21. 21.
    James, S., Pan, L., Wilkin, T., Yin, L.: Online peer marking with aggregation functions. In: 2017 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE). pp. 1–6 (2017)Google Scholar
  22. 22.
    Kearney, E.M.: Assessing learning. In: On Becoming a Teacher. pp. 85–89. Sense Publishers, Rotterdam (2013)CrossRefGoogle Scholar
  23. 23.
    Sale, D.: Assessing learning. In: The Challenge of Reframing Engineering Education. pp. 59–80. Springer, Singapore (2014)Google Scholar
  24. 24.
    Ettarres, Y.: Evaluation of online assignments and quizzes using Bayesian networks. In: Innovations in Smart Learning. pp. 39–44. Springer, Singapore (2017)Google Scholar
  25. 25.
    Govindarajan, K., Boulanger, D., Seanosky, J., Bell, J., Pinnell, C., Kumar, V.S., Kinshuk.: Assessing learners’ progress in a smart learning environment using bio-inspired clustering mechanism. In: Innovations in Smart Learning. pp. 49–58. Springer, Singapore (2017)Google Scholar
  26. 26.
    Zhu, M., Sari, A., Lee, M.M.: A systematic review of research methods and topics of the empirical MOOC literature (2014–2016). Internet High. Educ. 37, 31–39 (2018)CrossRefGoogle Scholar
  27. 27.
    Staubitz, T., Petrick, D., Bauer, M., Renz, J., Meinel, C.: Improving the peer assessment experience on MOOC platforms. In: Proceedings of the Third (2016) ACM Conference on Learning @ Scale. pp. 389–398. ACM, New York, NY, USA (2016)Google Scholar
  28. 28.
    Rust, C.: A briefing on assessment of large groups. In: LTSN Generic Centre: Assessment Series (2001)Google Scholar
  29. 29.
    Bali, M.: A new scholar’s perspective on open peer review. Teach. High. Educ. 20, 857–863 (2015)CrossRefGoogle Scholar
  30. 30.
    Soh, K.C.: Peer review: has it a future? Eur. J. High. Educ. 3, 129–139 (2013)CrossRefGoogle Scholar
  31. 31.
    Millard, W.B.: The wisdom of crowds, the madness of crowds: rethinking peer review in the web era. Ann. Emerg. Med. 57, A13–A20 (2011)Google Scholar
  32. 32.
    Clase, K.L., Gundlach, E., Pelaez, N.J.: Calibrated peer review for computer-assisted learning of biological research competencies. Biochem. Mol. Biol. Educ. Bimon. Publ. Int. Union Biochem. Mol. Biol. 38, 290–295 (2010)CrossRefGoogle Scholar
  33. 33.
    Purcell, M.E., Hawtin, M.: Piloting external peer review as a model for performance improvement in third-sector organizations. Nonprofit Manag. Leadersh. 20, 357–374CrossRefGoogle Scholar
  34. 34.
    Wu, J.: Empirical analysis of evaluation of english teachers’ educational ability under MOOC environment. In: 2018 International Conference on Intelligent Transportation, Big Data Smart City (ICITBS). pp. 303–306 (2018)Google Scholar
  35. 35.
    Yin, Z.: Educational ability evaluation of japanese language teacher under MOOC environment. In: 2018 International Conference on Intelligent Transportation, Big Data Smart City (ICITBS). pp. 299–302 (2018)Google Scholar
  36. 36.
    Koç, E.S.: An evaluation of the effectiveness of committees of teachers according to the teachers’ views, ankara province sample. Procedia - Soc. Behav. Sci. 174, 3–9 (2015)CrossRefGoogle Scholar
  37. 37.
    Song, Y., Hu, Z., Gehringer, E.F.: Collusion in educational peer assessment: How much do we need to worry about it?. In: 2017 IEEE Frontiers in Education Conference (FIE). pp. 1–8 (2017)Google Scholar
  38. 38.
    Gielen, S., Dochy, F., Onghena, P., Struyven, K., Smeets, S.: Goals of peer assessment and their associated quality concepts. Stud. High. Educ. 36, 719–735 (2011)CrossRefGoogle Scholar
  39. 39.
    Luo, H., Robinson, A.C., Park, J.-Y.: Peer grading in a MOOC: reliability, validity, and perceived effects. J. Asynchronous Learn. Netw. 18, 1–14 (2014)Google Scholar
  40. 40.
    Derrick, G.: The Evaluators’ Eye: Impact Assessment and Academic Peer Review. Palgrave Macmillan (2018)Google Scholar
  41. 41.
    Roberts, T.S. (ed.): Self, Peer and Group Assessment in E-learning. Information Science Publishing, Hershey, PA (2006)Google Scholar
  42. 42.
    Zheng, Q., Chen, L., Burgos, D.: Emergence and development of MOOCs. In: The Development of MOOCs in China. pp. 11–24. Springer, Singapore (2018)Google Scholar
  43. 43.
    Waks, L.J.: The Evolution and Evaluation of Massive Open Online Courses: MOOCs in Motion. Palgrave Macmillan US (2016)Google Scholar
  44. 44.
    Jackson, M., Marks, L.: Improving the effectiveness of feedback by use of assessed reflections and withholding of grades. Assess. Eval. High. Educ. 41, 532–547 (2016)CrossRefGoogle Scholar
  45. 45.
    Gamage, D., Whiting, M., Rajapakshe, T., Thilakarathne, H., Perera, I., Fernando, S.: Improving Assessment on MOOCs Through Peer Identification and Aligned Incentives. pp. 315–318 (2017). ArXiv170306169 CsGoogle Scholar
  46. 46.
    Lui, A., Andrade, H.: Student Peer Assessment. In: Gunstone, R. (ed.) Encyclopedia of science education, pp. 1003–1005. Springer, Netherlands, Dordrecht (2015)CrossRefGoogle Scholar
  47. 47.
    Adachi, C., Tai, J.H.-M., Dawson, P.: Academics’ perceptions of the benefits and challenges of self and peer assessment in higher education. Assess. Eval. High. Educ. 43, 294–306 (2018)CrossRefGoogle Scholar
  48. 48.
    Alias, M., Masek, A., Salleh, H.H.M.: Self, peer and teacher assessments in problem based learning: are they in agreements? Procedia - Soc. Behav. Sci. 204, 309–317 (2015)CrossRefGoogle Scholar
  49. 49.
    Jones, I., Alcock, L.: Peer assessment without assessment criteria. Stud. High. Educ. 39, 1774–1787 (2014)CrossRefGoogle Scholar
  50. 50.
    Orsmond, P., Merry, S., Reiling, K.: The importance of marking criteria in the use of peer assessment. Assess. Eval. High. Educ. 21, 239–250 (1996)CrossRefGoogle Scholar
  51. 51.
    Li, L.: The role of anonymity in peer assessment. Assess. Eval. High. Educ. 42, 645–656 (2017)CrossRefGoogle Scholar
  52. 52.
    Sridharan, B., Muttakin, M.B., Mihret, D.G.: Students’ perceptions of peer assessment effectiveness: an explorative study. Account. Educ. 27, 259–285 (2018)CrossRefGoogle Scholar
  53. 53.
    Pitt, E., Winstone, N.: The impact of anonymous marking on students’ perceptions of fairness, feedback and relationships with lecturers. Assess. Eval. High. Educ. 43, 1183–1193 (2018)CrossRefGoogle Scholar
  54. 54.
    Rotsaert, T., Panadero, E., Schellens, T.: Anonymity as an instructional scaffold in peer assessment: its effects on peer feedback quality and evolution in students’ perceptions about peer assessment skills. Eur. J. Psychol. Educ. 33, 75–99 (2018)CrossRefGoogle Scholar
  55. 55.
    Wahid, U., Chatti, M.A., Schroeder, U.: A systematic analysis of peer assessment in the MOOC era and future perspectives. In: Presented at the eLmL 2016, The Eighth International Conference on Mobile, Hybrid, and On-line Learning Apr 24 (2016)Google Scholar
  56. 56.
    Gielen, S., Dochy, F., Onghena, P.: An inventory of peer assessment diversity. Assess. Eval. High. Educ. 36, 137–155 (2011)CrossRefGoogle Scholar
  57. 57.
    Xiong, Y., Suen, H.K.: Assessment approaches in massive open online courses: possibilities, challenges and future directions. Int. Rev. Educ. 64, 241–263 (2018)CrossRefGoogle Scholar
  58. 58.
    Zadeh, L.A.: Fuzzy sets. Inf. Control 8, 338–353 (1965)CrossRefGoogle Scholar
  59. 59.
    Bellman, R.E., Zadeh, L.A.: Decision-making in a fuzzy environment. Manag. Sci. 17, B141–B164 (1970)MathSciNetCrossRefGoogle Scholar
  60. 60.
    Capuano, N., Loia, V., Orciuoli, F.: A fuzzy group decision making model for ordinal peer assessment. IEEE Trans. Learn. Technol. 10, 247–259 (2017)CrossRefGoogle Scholar
  61. 61.
    Lubis, F.F., Rosmansyah, Y., Supangkat, S.H.: Experience in learners review to determine attribute relation for course completion. In: 2016 International Conference on ICT For Smart Society (ICISS). pp. 32–36 (2016)Google Scholar
  62. 62.
    Ospina-Delgado, J., Zorio-Grima, A.: Innovation at universities: a fuzzy-set approach for MOOC-intensiveness. J. Bus. Res. 69, 1325–1328 (2016)CrossRefGoogle Scholar
  63. 63.
    El Alaoui, M.: SMART grid evaluation using fuzzy numbers and TOPSIS. IOP Conf. Ser. Mater. Sci. Eng. 353, 012019 (2018)CrossRefGoogle Scholar
  64. 64.
    El Alaoui, M., Ben-Azza, H., Zahi, A.: New multi-criteria decision-making based on fuzzy similarity, distance and ranking. In: Proceedings of the Third International Afro-European Conference for Industrial Advancement—AECIA 2016. pp. 138–148. Springer, Cham (2016)Google Scholar
  65. 65.
    Lee, H.-S.: Optimal consensus of fuzzy opinions under group decision making environment. Fuzzy Sets Syst. 132, 303–315 (2002)MathSciNetCrossRefGoogle Scholar
  66. 66.
    Chen, C.-T.: Extensions of the TOPSIS for group decision-making under fuzzy environment. Fuzzy Sets Syst. 114, 1–9 (2000)CrossRefGoogle Scholar
  67. 67.
    Chen, C.-T., Lin, C.-T., Huang, S.-F.: A fuzzy approach for supplier evaluation and selection in supply chain management. Int. J. Prod. Econ. 102, 289–301 (2006)CrossRefGoogle Scholar
  68. 68.
    Skalna, I., Rębiasz, B., Gaweł, B., Basiura, B., Duda, J., Opiła, J., Pełech-Pilichowsk, T.: Advances in Fuzzy Decision Making—Theory and Practice. Springer International Publishing (2015)Google Scholar
  69. 69.
    El Alaoui, M., Ben-Azza, H., El Yassini, K.: Optimal weighting method for fuzzy opinions. In: Presented at the International Conference on Industrial Engineering and Operations Management, Paris, France July 26 (2018)Google Scholar
  70. 70.
    Chai, K.C., Tay, K.M., Lim, C.P.: A new fuzzy peer assessment methodology for cooperative learning of students. Appl. Soft Comput. 32, 468–480 (2015)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Mohamed El Alaoui
    • 1
    Email author
  • Khalid El Yassini
    • 2
  • Hussain Ben-Azza
    • 1
  1. 1.Moulay Ismail UniversityENSAM MeknesMorocco
  2. 2.Faculty of Science MeknesMoulay Ismail UniversityMeknesMorocco

Personalised recommendations