Advertisement

Adaptation of Kirkpatrick’s four level model of training criteria to assessment of learning outcomes and program evaluation in Higher Education

  • Ludmila Praslova
Article

Abstract

Assessment of educational effectiveness provides vitally important feedback to Institutions of Higher Education. It also provides important information to external stakeholders, such as prospective students, parents, governmental and local regulatory entities, professional and regional accrediting organizations, and representatives of the workforce. However, selecting appropriate indicators of educational effectiveness of programs and institutions is a difficult task, especially when criteria of effectiveness are not well defined. This article proposes a comprehensive and systematic approach to aligning criteria for educational effectiveness with specific indicators of achievement of these criteria by adapting a popular organizational training evaluation framework, the Kirkpatrick’s four level model of training criteria (Kirkpatrick 1959; 1976; 1996), to assessment in Higher Education. The four level model consists of reaction, learning, behavior and results criteria. Adaptation of this model to Higher Education helps to clarify the criteria and create plans for assessment of educational outcomes in which specific instruments and indicators are linked to corresponding criteria. This provides a rich context for understanding the role of various indicators in the overall mosaic of assessment. It also provides Institutions of Higher Education rich and multilevel feedback regarding the effectiveness of their effort to serve their multiple stakeholders. The importance of such feedback is contextualized both in the reality of stakeholder pressures and in theoretical understanding of colleges and universities as open systems according to the systems theory (Katz and Kahn 1966). Although the focus of this article is on Higher Education, core principles and ideas will be applicable to different types and levels of educational programs.

Keywords

Assessment Evaluation Program evaluation Higher Education Education Criteria 

References

  1. Allen, M. J. (2006). Assessing general education programs (2006). Bolton: Anker.Google Scholar
  2. Alliger, G. M., Tannenbaum, S. I., Bennett, W., Jr., Traver, H., & Shotland, A. (1997). A meta-analysis of relations among training criteria. Personnel Psychology, 50, 341–358.CrossRefGoogle Scholar
  3. Arthur, W., Jr., Bennett, W. J., Edens, P. S., & Bell, S. T. (2003a). Effectiveness of training in organizations: a meta-analysis of design and evaluation features. The Journal of Applied Psychology, 88, 234–245.CrossRefGoogle Scholar
  4. Arthur, W., Jr., Tubre, T. C., Paul, D. S., & Edens, P. S. (2003b). Teaching effectiveness: the relationship between reaction and learning criteria. Educational Psychology, 23, 275–285.CrossRefGoogle Scholar
  5. Bers, T. H. (2008). The role of institutional assessment in assessing student learning outcomes. New Directions for Higher Education, 141, 31–39. doi: 10.1002/he.291.CrossRefGoogle Scholar
  6. Biesta, G. (2009). Good education in an age of measurement: on the need to reconnect with the question of purpose in education. Educational Assessment, Evaluation and Accountability, 21, 33–46. doi: 10.1007/s11092-008-9064-9.CrossRefGoogle Scholar
  7. Boyer, E. L., & Hechinger, F. M. (1981). Higher learning in the nation’s service. New York: Carnegie Foundation for the Advancement of Teaching.Google Scholar
  8. Brittingham, B., O’Brien, P. M., & Alig, J. (2008). Accreditation and institutional research: the traditional role and new dimensions. New Directions for Higher Education, 141, 69–76. doi: 10.1002/he.294.CrossRefGoogle Scholar
  9. Colby, A., Ehrlich, T., Beaumont, E., & Stephens, J. (2003). Educating citizens: Preparing America’s undergraduates for lives of moral and civic responsibility. San Francisco: Jossey-Bass.Google Scholar
  10. Dalton, J. C., Russell, T. R., & Kline, S. (2004). Editors’ notes. New Directions for Institutional Research, 122, p1–p2.CrossRefGoogle Scholar
  11. De Roulet, D., Praslova, L., Carmody, T., Roberson, M., & Wilson, M. (2009). Assessing core curriculum. Working document, Vanguard University of Southern California.Google Scholar
  12. Dysvik, A., & Martinsen, Ø. L. (2008). The relationship between trainees’ evaluation of teaching and trainee performance among Norwegian executive students. Educational Psychology, 28, 747–756. doi: 10.1080/01443410802259253.CrossRefGoogle Scholar
  13. Ewell, P. T. (2001). Accreditation and student learning outcomes: A proposed point of departure. Council for Higher Education Accreditation, Washington, DC.Google Scholar
  14. Ewell, P. T. (2006). Making the grade: How boards can ensure academic quality. Washington: Association of Governing Boards of Colleges and Universities.Google Scholar
  15. Halpern, D. F., & Hakel, M. D. (2003). Applying the science of learning to the university and beyond: teaching for long-term retention and transfer. Change, July/August, 2–13.Google Scholar
  16. Hansen, J. B. (1994). Applying systems theory to systemic change: a generic model for educational reform. Paper presented the Annual Meeting of the American Educational Research Association, New Orleans, LA, April 1994.Google Scholar
  17. Karpenko, O., Bershadskaia, M., & Voznesenskaia, I. (2009). The PISA international study and problems of the development of Higher Education. Russian Education & Society; 51(4), 68–91. doi: 10.2753/RES1060-9393510404.CrossRefGoogle Scholar
  18. Katz, D., & Kahn, R. L. (1966). The social psychology of organizations. New York: Wiley.Google Scholar
  19. Kirkpatrick, D. L. (1959). Techniques for evaluating training programs. Journal of the American Society of Training Directors, 13, 3–9.Google Scholar
  20. Kirkpatrick, D. L. (1976). Evaluation of training. In R. L. Craig (Ed.), Training and development handbook: A guide to human resource development (2nd ed., pp. 301–319). New York: McGraw-Hill.Google Scholar
  21. Kirkpatrick, D. L. (1996). Invited reaction: reaction to Holton article. Human Resource Development Quarterly, 7, 23–25.CrossRefGoogle Scholar
  22. Landy, F. J., & Conte, J. M. (2007). Work in the 21 century. An introduction to industrial and organizational psychology (2nd ed.). Malden: Blackwell.Google Scholar
  23. Russell, T. R. (2004). Assessing outcomes of character-building programs: problems and prospects. New Directions for Institutional Research, 122, p105–p110. doi: 10.1002/ir.113.CrossRefGoogle Scholar
  24. Salas, E., & Cannon-Bowers, J. A. (2001). The science of training: a decade of progress. Annual Review of Psychology, 52, 471–499.CrossRefGoogle Scholar
  25. Sax, L. J. (2004). Citizenship development and the American college student. New Directions for Institutional Research, 122, 65–80. doi: 10.1002/ir.110.CrossRefGoogle Scholar
  26. Toutkoushian, R. K. (2005). What can institutional research do to help colleges meet the workforce needs of states and nations? Research in Higher Education, 46, 955–984. doi: 10.1007/s11162-005-6935-5.CrossRefGoogle Scholar
  27. Van Buren, M. E., & Erskine, W. (2002). The 2002 ASTD state of the industry report. Alexandria: American Society of Training and Development.Google Scholar
  28. Voorhees, R. A., & Harvey, L. (2005). Higher education and workforce development: a strategic role for institutional research. New Directions for Institutional Research, 128, 5–12.CrossRefGoogle Scholar

Copyright information

© Springer Science + Business Media, LLC 2010

Authors and Affiliations

  1. 1.Vanguard University of Southern CaliforniaCosta MesaUSA

Personalised recommendations