Evaluating the Use of Open Badges in an Open Learning Environment

  • Jose Luis Santos
  • Sven Charleer
  • Gonzalo Parra
  • Joris Klerkx
  • Erik Duval
  • Katrien Verbert
Part of the Lecture Notes in Computer Science book series (LNCS, volume 8095)


This paper reports on our ongoing research around learning analytics. We focus on how learning analytics can be used to increase student motivation and the use of badges as a way to aggregate learning activity being a representation of their goals and progress along the course. The context of this work is an open learning environment, based on wikis, blogs, twitter, an activity stream mash-up and an open badges system. Our evaluation analyses perceived usefulness and usability of the system, as well as the impact on student motivation. Our results indicate that badges are useful to motivate students while activity streams have the potential to activate students.


Augmented Reality Student Activity Usefulness Perception Activity Stream Likert Score 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Eales, R.T.J., Hall, T., Bannon, L.J.: The motivation is the message: comparing cscl in different settings. In: Proc. of CSCL 2002, International Society of the Learning Sciences, pp. 310–317 (2002)Google Scholar
  2. 2.
    Santos, J.L., Verbert, K., Govaerts, S., Duval, E.: Addressing learner issues with stepup!: An evaluation. In: Proc. of LAK 2013. ACM (accepted, 2013)Google Scholar
  3. 3.
    Volet, S., Vauras, M.: Interpersonal Regulation of Learning and Motivation: Methodological Advances. Routledge (2013)Google Scholar
  4. 4.
    Williams, K.C., Williams, C.C.: Five key ingredients for improving student motivation. Research in Higher Education Journal 12, 1–23 (2011)Google Scholar
  5. 5.
    Savage, N., Birch, R., Noussi, E.: Motivation of engineering students in higher education. Engineering Education 6(2), 39–46 (2012)CrossRefGoogle Scholar
  6. 6.
    Di Serio, Á., Ibáñez, M.B., Kloos, C.D.: Impact of an augmented reality system on studentsmotivation for a visual art course. Computers & Education (2012)Google Scholar
  7. 7.
    Hsiao, I.-H., Brusilovsky, P.: Motivational social visualizations for personalized E-learning. In: Ravenscroft, A., Lindstaedt, S., Kloos, C.D., Hernández-Leo, D. (eds.) EC-TEL 2012. LNCS, vol. 7563, pp. 153–165. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  8. 8.
    Parra, G., Klerkx, J., Duval, E.: What should i read next? awareness of relevant publications through a community of practice. In: Proc. of CHI EA 2013. ACM (accepted, 2013)Google Scholar
  9. 9.
    Wenger, E.: Communities of practice: Learning, meaning, and identity. Cambridge University Press (1999)Google Scholar
  10. 10.
    Govaerts, S., Verbert, K., Duval, E., Pardo, A.: The student activity meter for awareness and self-reflection. In: Proc. of CHI EA 2012, pp. 869–884. ACM (2012)Google Scholar
  11. 11.
    Young, J.R.: Badges earned online pose challenge to traditional college diplomas. Chronicle of Higher Education (2012)Google Scholar
  12. 12.
    Goligoski, E.: Motivating the learner: Mozillas open badges program. Access to Knowledge: A Course Journal 4(1) (2012)Google Scholar
  13. 13.
    Domínguez, A., Saenz-de Navarrete, J., de Marcos, L., Fernández-Sanz, L., Pagés, C., Martínez-Herráiz, J.J.: Gamifying learning experiences: Practical implications and outcomes. Computers & Education 63, 380–392 (2013)CrossRefGoogle Scholar
  14. 14.
    Michel, C., Lavoué, E., Pietrac, L.: A dashboard to regulate project-based learning. In: Ravenscroft, A., Lindstaedt, S., Kloos, C.D., Hernández-Leo, D. (eds.) EC-TEL 2012. LNCS, vol. 7563, pp. 250–263. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  15. 15.
    Olson, J.F., Howison, J., Carley, K.M.: Paying attention to each other in visible work communities: Modeling bursty systems of multiple activity streams. In: IEEE SocialCom, pp. 276–281 (2010)Google Scholar
  16. 16.
    Li, I., Dey, A., Forlizzi, J., Höök, K., Medynskiy, Y.: Personal informatics and hci: design, theory, and social implications. In: Proc. of CHI EA 2011, pp. 2417–2420. ACM (2011)Google Scholar
  17. 17.
    Li, I., Dey, A., Forlizzi, J.: A stage-based model of personal informatics systems. In: Proc. of CHI 2010, pp. 557–566. ACM (2010)Google Scholar
  18. 18.
    Verbert, K., Duval, E., Klerkx, J., Govaerts, S., Santos, J.L.: Learning analytics dashboard applications. American Behavioral Scientist (accepted, 2013)Google Scholar
  19. 19.
    Podgorelec, V., Kuhar, S.: Taking advantage of education data: Advanced data analysis and reporting in virtual learning environments. Electronics and Electrical Engineering 114(8), 111–116 (2011)CrossRefGoogle Scholar
  20. 20.
    Leony, D., Pardo, A., de la Fuente Valentín, L., de Castro, D.S., Kloos, C.D.: Glass: A learning analytics visualization tool. In: Proc. of LAK 2012, pp. 162–163. ACM (2012)Google Scholar
  21. 21.
    Gutiérrez Rojas, I., Crespo García, R.M., Delgado Kloos, C.: Enhancing orchestration of lab sessions by means of awareness mechanisms. In: Ravenscroft, A., Lindstaedt, S., Kloos, C.D., Hernández-Leo, D. (eds.) EC-TEL 2012. LNCS, vol. 7563, pp. 113–125. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  22. 22.
    Hamari, J., Koivisto, J.: Social motivations to use gamification: an empirical study of gamifying exercise. In: Proc. of ECIS 2013 (2013)Google Scholar
  23. 23.
    Grosseck, G., Holotescu, C.: Can we use twitter for educational activities. In: eLSE08 (2008),
  24. 24.
    Zeng, X., Mehdi, Q.H., Gough, N.E.: Shape of the story: Story visualization techniques. In: Proc. of IV 2003, pp. 144–149. IEEE (2003)Google Scholar
  25. 25.
    Dawson, S., Bakharia, A., Heathcote, E.: Snapp: Realising the affordances of real-time sna within networked learning environments. In: Proc. of NLC 2010, pp. 125–133 (2010)Google Scholar
  26. 26.
    Kosba, E., Dimitrova, V., Boyle, R.: Using student and group models to support teachers in web-based distance education. In: Ardissono, L., Brna, P., Mitrović, A. (eds.) UM 2005. LNCS (LNAI), vol. 3538, pp. 124–133. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  27. 27.
    Naaman, M., Boase, J., Lai, C.H.: Is it really about me?: message content in social awareness streams. In: Proc. of CSCW 2010, pp. 189–192. ACM (2010)Google Scholar
  28. 28.
    Brooke, J.: Sus-a quick and dirty usability scale. Usability Evaluation in Industry, 189–194 (1996)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Jose Luis Santos
    • 1
  • Sven Charleer
    • 1
  • Gonzalo Parra
    • 1
  • Joris Klerkx
    • 1
  • Erik Duval
    • 1
  • Katrien Verbert
    • 2
  1. 1.Dept. of Computer ScienceKU LeuvenLeuvenBelgium
  2. 2.Dept. of Computer ScienceEindhoven University of TechnologyEindhovenThe Netherlands

Personalised recommendations