Advertisement

Speaking Truth to Power with Powerful Results: Impacting Public Awareness and Public Policy

  • Mack C. ShelleyIIEmail author
Chapter

This part of the book focuses specifically on the public policy issues of: (a) the ways in which global education funding patterns reflect governmental—and perhaps societal—priorities; (b) the role of research ethics boards in enforcing public policy norms regarding what is appropriate for science and literacy education research; (c) rules and expectations established by national legislative action and by professionalassociations for maintaining the security of the voluminous sets of data needed for sustained research excellence in science and literacy education research; (d) how qualitative research studies can be employed to provide broader and more lasting impacts on public policy making through systematic research reviews, secondary analysis, comparative case studies, and metasynthesis; and (e) how Gold Standard(s) inform education experts and policy makers about what should be donewith research findings. This chapter is intended to elaborate many of the points that have been made earlier in this book and perhaps to foreshadow an action agenda for education researchers and those who seek to influence the shape and direction of public policy. One of the major lines of argument is the need for eclecticism—in methodology, subject matter expertise, and policy agendas. Consistent with that theme of the virtue and necessity of eclectic approaches, and to honor the need for truth in advertising, it may be helpful to know that the author of this chapter is a faculty member with a joint appointment in a department of statistics and in a department of political science, with about 30 years of experience with statistical consulting, and with a background in public policy, program evaluation, and public administration. That background may help explain where this chapter is coming from]—as a somewhat eclectic, multifaceted exploration of a topic that is very much at the interface of several disciplines and multiple research methodologies.

Keywords

Mathematics Education Education Research Hierarchical Linear Model Educational Researcher Data Analysis Method 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Agresti, A.,&Finlay, B. (1997). Statistical methods for the social sciences (3rd edn.). Upper Saddle River, NJ: Prentice Hall.Google Scholar
  2. American Statistical Association. (2007). Using statistics effectively in mathematics education research. Arlington, VA: Author. Available from http://www.amstat.org/research_grants/pdfs/ SMERReport.pdfGoogle Scholar
  3. Arnold, C. L. (1992). An introduction to hierarchical linear models. Measurement&Evaluation in Counseling&Development, 25(2), 58–90.Google Scholar
  4. Best Evidence Encyclopedia. (n.d.). Homepage. Retrieved July 7, 2008, from http://www.bestevidence.org/
  5. Bloom, H. S., Richburg-Hayes, L.,&Black, A. R. (2007). Using covariates to improve precision for studies that randomize schools to evaluate educational interventions. Educational Evaluation and Policy Analysis, 29 (1), 30–59.CrossRefGoogle Scholar
  6. Bock, R. D. (Ed.). (1989). Multilevel analysis of educational data. San Diego, CA: Academic Press.Google Scholar
  7. Bollen, K. A. (1989). Structural equations with latent variables. New York: Wiley.Google Scholar
  8. Bollen, K. A.,&Curran, P. J. (2005). Latent curve models: A structural equation perspective. New York: Wiley.Google Scholar
  9. Bollen, K. A.,&Long, J. S. (Eds.). (1993). Testing structural equation models. Newbury Park, CA: Sage.Google Scholar
  10. Brewer, D. J.,&Goldhaber, D. D. (2008). Examining the incentives in educational research.Phi Delta Kappan, 89(5), 361–364.Google Scholar
  11. Briggs, D. C. (2008). Comments on Slavin: Synthesizing causal inferences. Educational Researcher, 37(1), 15–22.CrossRefGoogle Scholar
  12. Bryk, A. S.,&Raudenbush, S. W. (1987). Application of hierarchical linear models to assessing change. Psychological Bulletin, 101(1), 147–158.CrossRefGoogle Scholar
  13. Bryk, A. S.,&Raudenbush, S. W. (1989). Toward a more appropriate conceptualization of research on school effects: A three level hierarchical linear model. In R. D. Bock (Ed.),Multilevel analysis of educational data. San Diego, CA: Academic Press.Google Scholar
  14. Bryk, A. S.,&Raudenbush, S. W. (1992). Hierarchical linear models: Applications and data analysis methods. Newbury Park, CA: Sage.Google Scholar
  15. Byrne, B. M. (1998). Structural equation modeling with LISREL, PRELIS, and SIMPLIS: Basic concepts, applications, and programming. Mahwah, NJ: Lawrence Erlbaum.Google Scholar
  16. Byrne, B. M. (2001). Structural equation modeling with AMOS: Basic concepts, applications and programming. Mahwah, NJ: Lawrence Erlbaum.Google Scholar
  17. Campbell Collaboration. (n.d.). Homepage. Retrieved October 3, 2007, from http://www.campbellcollaboration.org
  18. Chatterji, M. (2008). Comments on Slavin: Synthesizing evidence from impact evaluations in education to inform action. Educational Researcher, 37(1), 23–26.CrossRefGoogle Scholar
  19. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd edn.). Hillsdale, NJ:Lawrence Erlbaum.Google Scholar
  20. Cohn, D. (2006). Jumping into the political fray: Academics and policy-making. Institute for Research on Public Policy (IRPP) Matters, 7(3), 8–36. Retrieved from http://www.irpp.org/pm/index.htmGoogle Scholar
  21. Comprehensive School Reform Quality Center. (2006). Homepage. Retrieved July 21, 2008, from http://www.csrq.org/Google Scholar
  22. Cooper, H.,&Hedges, L. V. (Eds.) (1994). The handbook of research synthesis. New York: Russell Sage Foundation.Google Scholar
  23. Donner, A.,&Klar, N. (2000). Design and analysis of cluster randomized trials in health research. London: Arnold.Google Scholar
  24. Dynarski, M. (2008). Comments on Slavin: Bringing answers to educators: Guiding principles for research syntheses. Educational Researcher, 37(1), 27–29.CrossRefGoogle Scholar
  25. Education Sciences Reform Act of 2002. Pub. L. No. 107–279, 116 Stat. 1940. (2002).Google Scholar
  26. Evidence for Policy and Practice Information and Co-ordinating Centre. (n.d.). Homepage Retrieved June 20, 2008, from http://eppi.ioe.ac.uk/cms/
  27. Fusarelli, L. D. (2008). Flying (partially) blind: School leaders' use of research in decision making.Phi Delta Kappan, 89(5), 365–358.Google Scholar
  28. Gamse, B. C., Bloom, H. S., Kemple, J. J.,&Jacob, R. T. (2008). Reading first impact study:Interim report. (NCEE 2008–4016). Washington, DC: National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, U.S. Department of Education. Available from http://ies.ed.gov/ncee/pubs/20084016/index.asp
  29. Goldstein, H. (1987). Multilevel models in educational and social research. New York: Oxford University Press.Google Scholar
  30. Goldstein, H. (2003). Multilevel statistical models (3rd edn.). London: Hodder Arnold.Google Scholar
  31. Green, J. L.,&Skukauskaité, A. (2008). Comments on Slavin: Becoming critical readers: Issues in transparency, representation, and warranting of claims. Educational Researcher, 37(1), 30–40.CrossRefGoogle Scholar
  32. Hedges, L. V. (2007a). Correcting a significance test for clustering. Journal of Educational& Behavioral Statistics, 32(2), 151–179.CrossRefGoogle Scholar
  33. Hedges, L. V. (2007b). Effect sizes in cluster-randomized designs. Journal of Educational & Behavioral Statistics, 32(4), 341–370.CrossRefGoogle Scholar
  34. Hedges, L. V.,&Hedberg, E. C. (2007). Intraclass correlation values for planning group-randomized trials in education. Educational Evaluation&Policy Analysis, 29(1), 60–87.CrossRefGoogle Scholar
  35. Henig, J. R. (2008). The evolving relationship between researchers and public policy. Phi Delta Kappan, 89(5), 357–360.Google Scholar
  36. Hess, F. M. (2008). The politics of knowledge. Phi Delta Kappan, 89(5), 354–356.Google Scholar
  37. Howell, D. C. (2007). Statistical methods for psychology (6th edn.). Pacific Grove, CA: Duxbury.Google Scholar
  38. Hox, J. (2002). Multilevel analysis: Techniques and applications. Mahwah, NJ: Lawrence Erlbaum.Google Scholar
  39. Hunter, J. E.,&Schmidt, F. L. (2004). Methods of meta-analysis: Correcting error and bias in research findings (2nd edn.). Thousand Oaks, CA: Sage.Google Scholar
  40. Ingersoll, R. M. (2008). A researcher encounters the policy realm: A personal tale. Phi Delta Kappan, 85(5), 369–371.Google Scholar
  41. Jöreskog, K. G.,&Sörbom, D. (1996a). LISREL 8: Structural equation modeling with the SIMPLIS command language. Chicago: Scientific Software International.Google Scholar
  42. Jöreskog, K. G.,&Sörbom, D. (1996b). LISREL 8: User's reference guide. Chicago: Scientific Software International.Google Scholar
  43. Killip, S., Mahfoud, Z.,&Pearce, K. (2004). What is an intracluster correlation coefficient? Crucial concepts for primary care researchers. Annals of Family Medicine, 2(3), 204–208.CrossRefGoogle Scholar
  44. Kim, J. S. (2008). Research and the reading wars. Phi Delta Kappan, 85(5), 372–375.Google Scholar
  45. Kingdon, J. W. (1995). Agendas, alternatives, and public policies (2nd edn.). New York:v HarperCollins.Google Scholar
  46. Lee, V. E.,&Bryk, A. S. (1989). A multilevel model of the social distribution of high school achievement. Sociology of Education, 62 (3), 172–192.CrossRefGoogle Scholar
  47. Lenth, R. V. (2006). Java applets for power and sample size. Available from http://www.cs.uiowa.edu/~rlenth/Power/Google Scholar
  48. Lipsey, M. W.,&Wilson, D. B. (2001). Practical meta-analysis. Thousand Oaks, CA: Sage.Google Scholar
  49. Loehlin, J. C. (2003). Latent variable models: An introduction to factor, path, and structural equation analysis (4th edn.). Mahwah, NJ: Lawrence Erlbaum.Google Scholar
  50. Murphy, K. R.,&Myors, B. (2003). Statistical power analysis: A simple and general model for traditional and modern hypothesis tests (2nd edn.). Mahwah, NJ: Lawrence Erlbaum.Google Scholar
  51. Murray, D. M. (1998). Design and analysis of group-randomized trials. New York: Oxford University Press.Google Scholar
  52. National Mathematics Advisory Panel. (2008). Foundations for success: The final report of the national mathematics advisory panel. Washington, DC: US Department of Education. Available from http://www.ed.gov/about/bdscomm/list/mathpanel/report/final-report.pdfGoogle Scholar
  53. No Child Left Behind Act of 2001. Pub. L. No. 107–110, 115 Stat. 1425. (2002).Google Scholar
  54. Ragin, C. C., Nagel, J.,&White, P. (2004). Workshop on scientific foundations of qualitative research. Available from http://www.nsf.gov/pubs/2004/nsf04219/start.htm
  55. Raudenbush, S. W. (1997). Statistical analysis and optimal design for cluster randomized trials. Psychological Methods, 2(2), 173–185.CrossRefGoogle Scholar
  56. Raudenbush, S. W. (2005). Learning from attempts to improve schooling: The contribution of methodological diversity. Educational Researcher, 34(5), 25–31.CrossRefGoogle Scholar
  57. Raudenbush, S. W.,&Bryk, A. S. (1986). A hierarchical model for studying school effects. Sociology of Education, 59(1), 1–17.CrossRefGoogle Scholar
  58. Raudenbush, S. W.,&Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd edn.). Thousand Oaks, CA: Sage.Google Scholar
  59. Raudenbush, S. W., Bryk, A. S., Cheong, Y. F.,&Congdon, R. (2001). HLM5: Hierarchical linear and nonlinear modeling. Lincolnwood, IL: Scientific Software International Inc.Google Scholar
  60. Raudenbush, S. W.,&Liu, X. (2000). Statistical power and optimal design for multisite randomized trials. Psychological Methods, 5(2), 199–213.CrossRefGoogle Scholar
  61. Raudenbush, S. W.,&Liu, X. (2001). Effects of study duration, frequency of observation, and sample size on power in studies of group differences in polynomial change. Psychological Methods, 6(4), 387–401.CrossRefGoogle Scholar
  62. Raudenbush, S. W., Martinez, A.,&Spybrook, J. (2007). Strategies for improving precision in group-randomized experiments. Educational Evaluation&Policy Analysis, 29(1), 5–29.CrossRefGoogle Scholar
  63. Raudenbush, S. W., Spybrook, J., Liu, X., Congdon, R.,&Martinez, A. (2006). Optimal Design (Version 1.76) [Software]. Available from http://sitemaker.umich.edu/group-based/ optimal_design_software
  64. Reise, S. P.,&Duan, N. (Eds.). (2002). Multilevel modeling: Methodological advances, issues, and applications. Mahwah, NJ: Lawrence Erlbaum.Google Scholar
  65. Rotherham, A. J. (2008). The translators: The media and school choice research. Phi Delta Kappan, 89(5), 376–379.Google Scholar
  66. Schmitt, L. N. T.,&Whitsett, M. D. (2008). Using evaluation data to strike a balance between stakeholders and accountability systems. In T. Berry&R. M. Eddy (Eds.), Consequences of No Child Left Behind for educational evaluation (pp. 47–58). San Francisco: Jossey-Bass.Google Scholar
  67. Schochet, P. Z. (2005). Statistical power for random assignment evaluations of education programs. (Contract No. ED-01-CO-0038-0009; MPR Reference No. 6046-310). Princeton, NJ: Mathematica Policy Research, Inc. Available from http://www.mathematica-mpr.com/publica-tions/PDFs/statisticalpower.pdf
  68. Shadish, W. R., Cook, T. D.,&Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin.Google Scholar
  69. Slavin, R. E. (2008a). Perspectives on evidence-based research in education—What works? Issues in synthesizing educational program evaluations. Educational Researcher, 37(1), 5–14.CrossRefGoogle Scholar
  70. Slavin, R. E. (2008b). Response to Comments: Evidence-based reform in education: Which evidence counts? Educational Researcher, 37(1), 47–50.CrossRefGoogle Scholar
  71. Sloane, F. (2008). Comments on Slavin: Through the looking glass: Experiments, quasi-experi-ments, and the medical model. Educational Researcher, 37(1), 41–46.CrossRefGoogle Scholar
  72. Toynbee, A. J. (1934–1961). A study of history. London: Oxford University Press.Google Scholar
  73. Trenholm, C., Devaney, B., Fortson, K., Clark, M., Quay, L.,&Wheeler, J. (2008). Impacts of abstinence education on teen sexual activity, risk of pregnancy, and risk of sexually transmitted diseases. Journal of Policy Analysis and Management, 27(2), 255–276.CrossRefGoogle Scholar
  74. United States Department of Education. (2003). Identifying and implementing educational practices supported by rigorous evidence: A user friendly guide. Washington, DC: Institute of Education Sciences, National Center for Education Evaluation and Regional Assistance. Available from http://ies.ed.gov/ncee/pubs/evidence_based/evidence_based.aspGoogle Scholar
  75. United States Institute of Education Sciences. (2008). Education research grants CFDA number: 84.305A. Retrieved June 17, 2008, from http://ies.ed.gov/funding/pdf/2009_84305A.pdf
  76. United States Institute of Education Sciences. (n.d.-a). Homepage. Retrieved June 17, 2008, from http://ies.ed.gov/
  77. United States Institute of Education Sciences. (n.d.-b). What Works Clearinghouse: Homepage. Retrieved June 20, 2008, from http://ies.ed.gov/ncee/wwc/
  78. United States Institute of Education Sciences. (n.d.-c). What Works Clearinghouse: Technical assistance. Retrieved July 8, 2008, from http://ies.ed.gov/ncee/wwc/tech_assistance/
  79. United States Institute of Education Sciences. (n.d.-d). What Works Clearinghouse: Technical working papers. Retrieved July 8, 2008, from http://ies.ed.gov/ncee/wwc/twp.asp
  80. United States National Research Council. (2002). Scientific research in education. Committee on Scientific Principles for Education Research. R. J. Shavelson&L. Towne (Eds.). Center for Education, Division of Behavioral and Social Sciences and Education. Washington, DC: The National Academies Press.Google Scholar
  81. W. T. Grant Foundation and University of Michigan (n.d.). Building capacity to evaluate group-level interventions. Retrieved July 21, 2008, from http://sitemaker.umich.edu/group-based/ home
  82. Wildavsky, A. (1979). Speaking truth to power: The art and craft of policy analysis. Boston: Little, Brown.Google Scholar
  83. Ye, N. (Ed.). (2003). The handbook of data mining. Mahwah, NJ: Lawrence Erlbaum.Google Scholar

Copyright information

© Springer Science + Business Media B.V 2009

Authors and Affiliations

There are no affiliations available

Personalised recommendations