Advertisement

An International Perspective of Monitoring Educational Research Quality: Commonalities and Differences

  • Richard K. Coll
  • Wen-Hua Chang
  • Justin Dillon
  • Rosária Justi
  • Eduardo Mortimer
  • Kim Chwee Daniel Tan
  • David F. Treagust
  • Webb Paul
Chapter

This chapter considers the notion of educational research quality and evaluation from an international perspective. We consider how and why these approaches differ from the US-based Gold Standard design (i.e., research based on randomized controlled trials [RCTs] mimicking third-stage drug trials; see Shelley, Yore, & Hand, Chap. 1). The Gold Standard is based on the assumption that RCT design alone, regardless of other factors, provides the desired quality.

We suggest here that the notion of quality in research and the mechanisms used to evaluate research quality are highly dependent on the overarching aim of education. To illustrate, the governments of many countries see education, especially science education, as a key component in economic progress and as a means of delivering on social services (Coll & Taylor, 2008). Hence, there are a number of reasons why we need to evaluate research quality in science education. We need to provide evidence that our science education regimes (and vocational education and training) do in fact produce outputs in terms of qualified people needed to drive economic success. There is then the notion of accountability; the expenditure of taxpayer monies by government—especially in the area of education—is subject to much public scrutiny and often to criticism. There also is accountability to specific legalization in which governments require the education sector to deliver on education aims, such as scientific literacy. In each of these examples, we need to be as sure as we can that the research findings are trustworthy—to use Guba and Lincoln's (1989) term—or believable. New curricula and teaching and learning approaches often prove highly controversial (e.g., Bell, Jones, & Carr, 1995; Coll & Taylor; Matthews, 1994), and education stakeholders—including government—naturally want to see convincing evidence that costly educational interventions actually work.

Keywords

Educational Research Research Output Research Quality International Perspective South African Journal 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Antman, E. M., Lau, J., Kupelnick, B., Mosteller, F.,&Chalmers, T. C. (1992). A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts: Treatments for myocardial infarction. Journal of the American Medical Association, 268(2), 240–248.CrossRefGoogle Scholar
  2. Bell, B., Jones, A.,&Carr, M. (1995). The development of the recent national New Zealand science curriculum. Studies in Science Education, 26(1), 73–105.CrossRefGoogle Scholar
  3. Brazil Conselho Nacional de Desenvolvimento Científico e Tecnológico [National Council of Scientific and Technological Development]. (n.d.). Homepage. Retrieved July 4, 2008, from http://www.cnpq.br/english/cnpq/index.htm
  4. Brazil Coordenação de Aperfeiçoamento de Pessoal de Nível Superior [Coordination for Improvement of University Level Staff]. (2006). Homepage. Retrieved April 17, 2008, from http://www.capes.gov.br/ [in Portuguese]
  5. Cheng, Y.-J. (2006, Fall). Enhancing the quality of group proposals in science education. Paper presented at the annual workshops for Writing NSC Project Proposals held in several universities in Taiwan.Google Scholar
  6. Coll, R. K.,&Taylor, N. (Eds.) (2008). Science education in context: An international examination of the influence of context on science curricula development and implementation. Rotterdam, The Netherlands: Sense.Google Scholar
  7. Excluding the many for the sake of the few. (2007, February 2). The Times Higher Education Supplement, p. 12. Available from http://www.timeshighereducation.co.uk/story.asp?sectionco de=26&storycode=207676)
  8. Guba, E. G.,&Lincoln, Y. S. (1989). Fourth generation evaluation. Newbury Park, CA: Sage.Google Scholar
  9. Ioannidis, J. P. A. (2005). Why most published research findings are false. Public Library of Science Medicine, 2(8), e124. Retrieved from http://dx.doi.org/10.1371%2Fjournal. pmed.0020124Google Scholar
  10. Johnson, B.,&Cross, M. (2004). Academic leadership under siege: Possibilities and limits of executive deanship. South African Journal of Higher Education, 18(2), 34–58.Google Scholar
  11. Koh, T. S., Tan, K. C. D.,&Cheah, H. M. (2008). Science education in Singapore: Meeting the challenges ahead. In R. K. Coll&N. Taylor (Eds.), Science education in context: An international examination of the influence of context on science curricula development and implementation (pp. 283–290). Rotterdam, The Netherlands: Sense.Google Scholar
  12. Lane, B. (2007, August 1). Our papers have influence. The Australian Higher Education Section, p. 22. Lipsett, A. (2007, February 2). Casualties rise even before the RAE starts. The Times Higher Education Supplement, p. 6. Available from http://www.timeshighereducation.co.uk/story.asp?sectioncode=26&storycode=207665)
  13. Mather, J. (2001). RQF blamed for spike in discovery applications. Campus Review, 17(40), 1–14.Google Scholar
  14. Matthews, M. R. (Ed.). (1994). Science teaching: The role of history and philosophy of science. New York: Routledge.Google Scholar
  15. New Zealand Tertiary Education Commission. (2004). Performance-based research fund (PBRF) — Evaluating research excellence—The 2003 assessment. Retrieved April 17, 2008, from http://www.tec.govt.nz/templates/standard.aspx?id=589
  16. New Zealand Tertiary Education Commission. (2007). Development of the performance-based research fund. Retrieved April 29, 2008, from http://www.minedu.govt.nz/index.cfm?layout= document&documentid=7503
  17. New Zealand Tertiary Education Commission. (n.d.-a). PBRF Quality evaluation 2006 — Questions and answers? Retrieved April 29, 2008, from http://www.tec.govt.nz/templates/ StandardSummary.aspx?id=1206
  18. New Zealand Tertiary Education Commission. (n.d.-b). Performance-Based Research Fund — Evaluating research excellence — The 2006 assessment. Retrieved April 29, 2008, from http://www.tec.govt.nz/upload/downloads/pbrf-full-report-2006.pdf
  19. No Child Left Behind Act of 2001. Pub. L. No. 107–110, 115 Stat. 1425. (2002).Google Scholar
  20. Roberts, G. (2003). Review of research assessment. Report by Sir Gareth Roberts to the UK funding bodies. Bristol, Avon, UK: Higher Education Funding Council for England.Google Scholar
  21. Shulman, L. S. (1997). Disciplines of inquiry in education: A new overview. In R. M. Jaeger (Ed.), Complementary methods for research in education (2nd edn., pp. 3–31). Washington, DC: American Educational Research Association.Google Scholar
  22. Singapore Centre for Research in Pedagogy and Practice. (n.d.). Core research program. Retrieved April 17, 2008, from http://www.crpp.nie.edu.sg/course/view.php?id=306
  23. Singapore Learning Sciences Lab. (2006). Research overview. Retrieved April 17, 2008, from http://lsl.nie.edu.sg/overview.htm
  24. Singapore Ministry of Education. (2006). Education in Singapore. Retrieved April 18, 2008, from http://www.moe.gov.sg/about/files/moe_corporate_brochure_2006.pdf
  25. Singapore Ministry of Finance. (2007). Budget highlights financial year 2007: Ready for the future, ready for the world. Retrieved April 17, 2008, from http://www.mof.gov.sg/budget_2007/budget_ speech/downloads/FY2007_Budget_Highlights.pdf
  26. Singapore National Research Foundation. (2007). Call for proposals. Retrieved April 17, 2008, from https://rita.nrf.gov.sg/IDM/default.aspx
  27. South Africa Human Sciences Research Council. (n.d.). Homepage. Retrieved July 4, 2008, from http://www.hsrc.ac.za/index.phtml
  28. South Africa National Research Foundation. (n.d.-a). Evaluation and rating facts and figures.Retrieved April 20, 2008, from http://evaluation.nrf.ac.za/Content/Facts/factsfigures.htm
  29. South Africa National Research Foundation. (n.d.-b). Homepage. Retrieved April 17, 2008, from http://www.nrf.ac.za/
  30. South Africa National Research Foundation. (n.d.-c). The NRF focus area programme. Retrieved April 20, 2008, from http://www.nrf.ac.za/focusareas/
  31. South Africa National Research Foundation. (n.d.-d). NRF key performance indicator report 2003/04. Retrieved April 17, 2008, from http:www.nrf.ac.za/publications/annrep/04/intro.htm
  32. South African Government Gazette. (2003). Policy and procedures for measurement of research output of public higher education institutions. Retrieved April 17, 2008, from http://www.uct. ac.za/downloads/uct.ac.za/research/office/pubcount/307.pdf
  33. Stewart, P. (2007). Re-envisioning the academic profession in the shadow of corporate manageri-alism. Journal of Higher Education in Africa, 5(1), 131–147.Google Scholar
  34. Taiwan Department of Science Education. (2006). Mission. Retrieved April 17, 2008, from http://www.nsc.gov.tw/sci/mp.asp?mp=1
  35. Taiwan National Science Council. (2006). Request for proposals. Retrieved April 17, 2008, from http://www.nsc.gov.tw/sci/lp.asp?ctNode=1622&CtUnit=1224&BaseDSD=7 [in Chinese]
  36. Taiwan National Science Council. (n.d.-a). Homepage. Retrieved April 17, 2008, from http://web.nsc.gov.tw/default.asp?mp=7
  37. Taiwan National Science Council. (n.d.-b). Organization. Retrieved June 20, 2008, from http://web.nsc.gov.tw/ct.asp?xItem=14932&CtNode=3417
  38. United Kingdom Research Assessment Exercise. (2001). What is RAE 2001? Retrieved April 17, 2008, from http://www.hero.ac.uk/rae/AboutUs/
  39. United Kingdom Research Assessment Exercise. (2006). Panel criteria and working methods: Panel K. Retrieved April 17, 2008, from http://www.rae.ac.uk/pubs/2006/01/docs/k45.pdf
  40. United Kingdom Research Assessment Exercise. (n.d.-a). History of the RAE. Retrieved April 17, 2008, from http://www.rae.ac.uk/aboutus/history.asp
  41. United Kingdom Research Assessment Exercise. (n.d.-b). Making a submission to the RAE 2008. Retrieved April 17, 2008, from http://www.rae.ac.uk/aboutus/subs.asp
  42. United Kingdom Research Assessment Exercise. (n.d.-c). Panel members for the 2008 RAE. Retrieved April 17, 2008, from http://www.rae.ac.uk/panels/members/
  43. United Kingdom Research Assessment Exercise. (n.d.-d). Panels. Retrieved April 17, 2008, from http://www.rae.ac.uk/panels/
  44. United Kingdom Research Assessment Exercise. (n.d.-e). Quality profiles. Retrieved April 14, 2008, from http://www.rae.ac.uk/aboutus/quality.asp
  45. United Kingdom Research Assessment Exercise. (n.d.-f). RAE 2008: Research assessment exercise. Retrieved April 17, 2008, from http://www.rae.ac.uk/

Copyright information

© Springer Science + Business Media B.V 2009

Authors and Affiliations

  • Richard K. Coll
    • 1
  • Wen-Hua Chang
    • 2
  • Justin Dillon
    • 3
  • Rosária Justi
    • 4
  • Eduardo Mortimer
    • 5
  • Kim Chwee Daniel Tan
  • David F. Treagust
    • 6
  • Webb Paul
    • 7
  1. 1.Centre for Science & Technology Education ResearchUniversity of WaikatoHamiltonNew Zealand
  2. 2.Department of Life ScienceNational Taiwan Normal UniversityTaipei
  3. 3.Science & Technology Education GroupKing's College LondonUK
  4. 4.Chemistry Department and Faculty of EducationUniversidade Federal de Minas GeraisBrazil
  5. 5.Faculty of EducationUniversidade Federal de Minas GeraisBrazil
  6. 6.Science & Mathematics Education CentreCurtin University of TechnologyPerthAustralia
  7. 7.Faculty of EducationNelson Mandela Metropolitan UniversitySouth Africa

Personalised recommendations