How Research Messages Get Sidetracked by Governments

Abstract

Politicians and governments have agenda, sometimes at odds with the facts associated with educational phenomena. Further, educational research is hard to do and leaves room for ambiguity in creating policy out of research, allowing newspapers, in general, and politicians, in particular, to misinterpret educational phenomena. This often leads to inappropriate policies. For example, interpretations of the results of PISA tests (The Program for International Student Assessment) are highly political, often not trustworthy, and commonly misleading. Four examples of problems with PISA interpretations by government are given: the misunderstanding of the relationship between PISA and a nation’s economic performance; the data hidden when only the mean scores of nations are reported; the meaning of variance accounted for in interpreting PISA test scores; and the conclusion that better standards for educational achievement will improve America’s performance on the PISA tests. Discussed as well is the fact that political expediency and government policy often affect such issues as the field testing of instructional programs and their assessment; the setting of goals for achievement on commonly used assessments; the overuse of simple main effects to interpret data, along with a lack of understanding of interactions; a failure to understand the effects of context on the implementation of policy; inadequate estimates of the costs associated with policy implementation; inadequate understanding of the effects of tax credits on education; an overconcern with educational outcomes and a corresponding lack of concern for educational inputs; and an overreliance on standardization.