Skip to main content
SpringerLink
Log in
Book cover

International Scholarships in Higher Education pp 281–304Cite as

Palgrave Macmillan

Tracing the Spark that Lights a Flame: A Review of Methodologies to Measure the Outcomes of International Scholarships

  • Chapter
  • First Online:

Abstract

This chapter reviews current methodologies used to measure and evaluate the effects of international scholarship programs for higher education. Just as these programs are diverse in their design, there is no “one size fits all” approach to evaluation. The overview of methodologies uses several analytic lenses and identifies their respective challenges: understanding the theory of change; choosing the unit of analysis; the timeline for evaluation; and approaches to quantitative and qualitative data collection. The chapter concludes with the importance of transparency in relaying evaluation outcomes to key audiences to improve programs and influence research in the field.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   139.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    While the report conducted by IIE was internal, the topics that emerged were discussed in a roundtable held at the Institute in December 2012 and were outlined on IIE’s blog: http://www.iie.org/Blog/2012/December/Alumni-Tracking#.V-CODc6cHIU

References

  • Amos, L.B., Windham, A, de los Reyes, I.B., Jones, W., and Baran, V. (2009). Delivering on the promise: An impact evaluation of the Gates Millennium Scholars Program. Washington, DC: American Institutes for Research.

    Google Scholar 

  • Banerjee, A., Cole, S., Duflo, E. and Linden, L. (2005). Remedying education: Evidence from two randomized experiments in India, No. w11904, National Bureau of Economic Research. Available at: http://www.nber.org/papers/w11904.pdf (Accessed 23 Jan 2017).

  • Bamberger, M. (1999). Ethical Issues in Conducting Evaluation in International Settings. New Directions for Evaluation, 82, pp. 89–97.

    Article  Google Scholar 

  • Bamberger, M., Rugh, J. and Mabry, L. (2011). RealWorld evaluation: Working under budget, time, data, and political constraints. Thousand Oaks, CA: SAGE Publications, Inc.

    Google Scholar 

  • Bhandari, R. and Belyavina, R. (2011). Evaluating and measuring the impact of citizen diplomacy: Current status and future directions. New York: Institute of International Education.

    Google Scholar 

  • Boeren, A., Bakhuisen, K., Christian-Mak, A.M., Musch, V. and Pettersen, K. (2008). Donor policies and implementation modalities with regard to international postgraduate programmes targeting scholars from developing countries. Brussels: Vlaamse Interuniversitaire Raad.

    Google Scholar 

  • CARE. (2009). Girls’ leadership development in action: CARE’s experience from the field. Available from: http://www.care.org/sites/default/files/documents/EDU-GE-2012-Girls-Leadership-Development-in-Action.pdf (Accessed 1 August 2016).

  • Center for Theory of Change. (2016). Center for Theory of Change home page. Available from: www.theoryofchange.org (Accessed 1 September 2016).

  • Chen, P., Weiss, F. L., Nicholson, H. J. and Girls Incorporated (2010). Girls Study Girls Inc.: Engaging Girls in Evaluation through Participatory Action Research. American Journal of Community Psychology, 46(1–2), pp. 228–237.

    Article  Google Scholar 

  • Chesterfield, R., and Dant, W. (2013). Evaluation of LAC higher education scholarships program: Final report. Washington, DC: U.S. Agency for International Development.

    Google Scholar 

  • Chouinard, J.A. and Cousins, J.B. (2009). A Review and Synthesis of Current Research on Cross-Cultural Evaluation. American Journal of Evaluation, 30(4), pp. 457–494.

    Article  Google Scholar 

  • CIDA. (2005). Evaluation of the Canadian Francophone Scholarship Program (CFSP), 1987–2005. Quebec: CIDA.

    Google Scholar 

  • Cook, T.D., Scriven, M., Coryn, C.L. and Evergreen, S.D. (2009). Contemporary thinking about causation in evaluation: A dialogue with Tom Cook and Michael Scriven. American Journal of Evaluation, 31(1), pp. 105–117.

    Article  Google Scholar 

  • Cosentino, C., Rangarajan, A., Sloan, M., Fortson, J., Moorthy, A., Humpage-Liuzzi, S. and Thomas, C. (2015). Monitoring, Evaluation, and Learning Design for The MasterCard Foundation Scholars Program. Washington, DC: Mathematica Policy Research.

    Google Scholar 

  • Creed, C., Perraton, H and Waage, J. (2012). Examining development evaluation in higher education interventions: a preliminary study. London: London International Development Centre.

    Google Scholar 

  • Gertler, P.J., Martinez, S., Premand, P., Rawlings, L.B. and Vermeersch, C.M. (2011). Impact evaluation in practice. Washington, DC: The World Bank.

    Google Scholar 

  • Hansel, B. and Chen, Z. (2008). AFS long term impact study, 20 to 25 years after the exchange experience, AFS alumni are compared with their peers, Report 1. New York: AFS International.

    Google Scholar 

  • Hesse-Biber, S.N. (2010). Mixed methods research: Merging theory with practice. New York: Guilford Press.

    Google Scholar 

  • Hofmann-Pinilla, A. and Kallick Russell, J. (2009). Evaluation of the Leadership Development for Mobilizing Reproductive Health Program, Final Report Executive Summary. New York: Research Center for Leadership Action.

    Google Scholar 

  • Institute of International Education. (2015). Schlumberger Faculty for the Future: Program impact study. New York: Institute of International Education.

    Google Scholar 

  • Institute of International Education. (2016). LOTUS Annual Progress Report FY2016. Cairo, Egypt: IIE Cairo.

    Google Scholar 

  • Jadad, A.R. and Enkin, M. (2007). Randomized controlled trials: questions, answers, and musings (2nd ed). Malden, MA: Blackwell Publishing.

    Book  Google Scholar 

  • Kirkpatrick, D.L. (1979). Techniques for evaluating training programs. Training and Development Journal, 33 (6), pp. 78–92.

    Google Scholar 

  • Kirkpatrick, D. L. (1994). Evaluating training programs: the four levels. San Francisco: Berrett-Koehler.

    Google Scholar 

  • Kusek, J.Z. and Rist, R.C. (2004). Ten steps to a results-based monitoring and evaluation system: A handbook for development practitioners. Washington, DC: The World Bank.

    Book  Google Scholar 

  • Marsh, R., Baxter, A., Cliff, R., Di Genova, L., Jamison, A. and Madden, M. (2016). Career choices, return paths and social contributions: The African alumni project, Abridged report. Toronto: The MasterCard Foundation.

    Google Scholar 

  • Martel, M. and Bhandari, R. (2016). Social Justice and Sustainable Change: The Impacts of Higher Education, Ford Foundation International Fellowships Program Alumni Tracking Study Report No. 1. New York: Institute of International Education.

    Google Scholar 

  • Martinez, A., Epstein, C. and Parsad, A. (2015). Evaluation of the National Science Foundation’s Partnerships for International Research and Education (PIRE) Program, Volume 1: Final report. Cambridge, MA: Abt Associates.

    Google Scholar 

  • Mason, L., Powers, C. and Donnelly, S. (2015). The Boren Awards: A Report of Oral Language Proficiency Gains during Academic Study Abroad. New York, NY: Institute of International Education.

    Google Scholar 

  • Mawer, M. (2014). A study of research methodology used in evaluations of international scholarship schemes for higher education. London: Commonwealth Scholarship Commission in the United Kingdom.

    Google Scholar 

  • Mayne, J. (2011). Addressing cause and effect in simple and complex settings through contribution analysis, in Forss, K., Mara, M., and Schwartz, R. (eds), Evaluating the complex: Attribution, contribution and beyond. New York: Transaction Publishers.

    Google Scholar 

  • Perna, L., Orosz, K., Gopaul, B., Jumakulov, Z., Ashirbekov, A., and Kishkentayaeva, M. (2014). Promoting human capital development: A typology of international scholarship programs in higher education. Educational Researcher, 43(2), pp. 66–73.

    Google Scholar 

  • PPMI. (2012). Interim evaluation of Erasmus Mundus II (2009–2013), Brussels: European Commission Directorate-General of Education and Culture.

    Google Scholar 

  • Ramboll Management Consulting. (2012). Final report: Evaluation of NPT and NICHE. Berlin: Ramboll Management Consulting.

    Google Scholar 

  • Research Solutions International. (2016). The Benjamin A. Gilman International Scholarship Program Evaluation Report. Washington, DC: U.S. Department of State, Bureau of Educational and Cultural Affairs.

    Google Scholar 

  • Rotem, A., Zinovieff, M., and Goubarev, A. (2010). A framework for evaluating the impact of the United Nations fellowship programmes. Human Resources for Health, 8(7). Available at: https://human-resources-health.biomedcentral.com/articles/10.1186/1478-4491-8-7 (Accessed 23 Jan 2017).

  • Scott, J., and Carrington, P. J. (2011). The SAGE handbook of social network analysis. Thousand Oaks, CA: SAGE publications.

    Google Scholar 

  • SRI International. (2005). Outcome assessment of the visiting Fulbright Student Program. Washington, DC: U.S. Department of State, Bureau of Educational and Cultural Affairs.

    Google Scholar 

  • The MasterCard Foundation. (2016). Education in Support of Social Transformation: Learning from the First Five Years of The MasterCard Foundation Scholars Program. Toronto: The MasterCard Foundation.

    Google Scholar 

  • Tvaruzkova, M. (2012). Evaluation: Advances in measuring the effectiveness of networks. Paper presented at the Global Leadership Consortium, December 11, 2012, Washington, DC.

    Google Scholar 

  • USAID. (2004). Generations of quiet progress: the development impact of US long-term university training on in Africa from 1963–2003, Report prepared by Aguirre International. Washington, DC: USAID.

    Google Scholar 

  • Uyeki, E. (1993). As Others See Us: A Comparison of Japanese and American Fulbrighters. New York: Institute of International Education.

    Google Scholar 

  • Valuy, E. and Martel, M. (2016). HER initiative to lead change: The power of education. New York: Institute of International Education.

    Google Scholar 

  • Valuy, E. (2016). Centroamerica Adelante Final Report. New York: Institute of International Education.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of International Education (IIE), New York, NY, USA

    Mirka Martel

Authors
  1. Mirka Martel
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Heller School for Social Policy and Management, Brandeis University, Waltham, Massachusetts, USA

    Joan R. Dassin

  2. University of California, Institute for the Study of Societal Issues, Berkeley, California, USA

    Robin R. Marsh

  3. The Association of Commonwealth Universities, London, United Kingdom

    Matt Mawer

Rights and permissions

Reprints and permissions

Copyright information

© 2018 The Author(s)

About this chapter

Cite this chapter

Martel, M. (2018). Tracing the Spark that Lights a Flame: A Review of Methodologies to Measure the Outcomes of International Scholarships. In: Dassin, J., Marsh, R., Mawer, M. (eds) International Scholarships in Higher Education. Palgrave Macmillan, Cham. https://doi.org/10.1007/978-3-319-62734-2_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-62734-2_14

  • Published:

  • Publisher Name: Palgrave Macmillan, Cham

  • Print ISBN: 978-3-319-62733-5

  • Online ISBN: 978-3-319-62734-2

  • eBook Packages: EducationEducation (R0)

Publish with us

Policies and ethics

Search

Navigation