Pleasures, Power, and Pitfalls of Writing up Mathematics Education Research

Abstract

Research in mathematics education consists of both doing a study and reporting its thesis and conclusions. Communicating one’s research clearly and adequately is at least as important as conducting it proficiently. At ICME-13, we each conducted a workshop on academic writing in which we drew upon our experiences as writers and editors to acquaint novice researchers with potential benefits and costs of reporting in our field. In this chapter, we present some of the points made in our workshops. We look at the nature of research in our field, the elements of a research report, its structure, the importance of choosing an appropriate publication outlet, and some pitfalls of academic writing. The chapter ends by reprinting a satirical editorial that underlines some of the ways in which a research report can go wrong.

1 Why Publish?

Once you have completed an investigation on some topic of mathematics education—research that may have taken several years of your life—you may find yourself under pressure to disseminate the results of your hard work in a published paper (or more than one). Why is this process important, and what are some of the potential pitfalls in this type of academic writing? This chapter explores answers to both of those questions.

One issue that is relevant to the publication of research reports is the nature of the word academic as it appears in “academic writing.” The meaning of this word resonates with a concern through the decades (Sowder 1990; Teppo 1998) that research in mathematics education has not been particularly fruitful in its implementation in mathematics classrooms. Is “academic” writing irrelevant in the field? Dorier (2008) pointed out that there are two contrasting meanings of the word: One of these is “related to education and scholarship,” and the other is “not related to a real or practical situation and therefore irrelevant” (p. 40). These concerns go to the very heart of why we do research in mathematics education. If the goal of research is to improve the teaching and learning of mathematics in some way, then it is important that our writing be academic in the scholarly sense, with the relevance of our work in the lives of mathematics teachers and learners on full display.

Do these concerns imply that development of theory in our field is less important than practical relevance and implementation? On the contrary, we regard theory as a collection of lenses through which the complex phenomena of mathematics education may be observed. Theory permeates all aspects of the research process, from the initial choices of questions to be addressed, through suitable methods of data collection and analysis, to the conclusions reached as a result of the research, and future directions that the research might take.

Thus, in a sense, theory is the glue that holds the study together and guarantees its integratedness, as well as defining its boundaries and limitations. Without these aspects, the relevance of research could be severely compromised. Academic writing concerning research can be both scholarly and empirically significant. Some of these issues are developed further in the following sections.

2 Quality Criteria

Dissemination is a vital element of the scientific research process. In this section, we address some aspects of research that have been considered essential in shaping what counts as good quality in mathematics education research. Although the following lists are historical—in the sense that they were compiled several decades ago and thus may be considered part of the history of our field—they are timeless because they outline elements of an ongoing scientific endeavor. After a general introduction to the research itself in this section, in the sections that follow we put forward some possible structures and essential elements in writing about research.

2.1 What Is Research in Mathematics Education?

Mathematics education research is a relatively young field in comparison with the millennia of mathematics research: the two fields are related but distinct (Kilpatrick 2008; Presmeg 2014). In the 1990s there were strong efforts to identify what makes mathematics education research distinctive, what its components are, and how we can identify quality in this field (Sierpinska and Kilpatrick 1998). These are still significant questions, especially for the necessary dissemination of one’s research through publication. There are different types of research, and quality criteria are of necessity specific to these types. Nevertheless, basic foundational components can be identified. Some types of research used in education generally were already identified by Jaeger in 1988:

• Historical methods

• Philosophical inquiry methods

• Ethnographic research

• Case study methods

• Survey research methods

• Comparative experimental methods

• Quasi-experimental methods.

These types are still current in mathematics education research. A trend that started in the decade of the 2000s, however, is to use a mixed methods design, which acknowledges the different purposes of quantitative and qualitative research and that these types of research may complement each other in order to give a fuller picture of some phenomenon.

What, then, is research in mathematics education specifically? In a discussion group at the 13 Annual Meeting of the International Group for the Psychology of Mathematics Education in Paris in 1989, Kath Hart proposed that the nature of educational research is disciplined enquiry, repeating (and ‘briticizing’) the term introduced by Lee Cronbach and Patrick Suppes (1969; see also Kilpatrick 1992). Research in education entails the following essential components:

• 1. There is a problem.

• 2. There is evidence/data.

• 3. The work can be replicated.

• 4. The work is reported.

• 5. There is a theory (Hart 1993, p. 411).

The importance of publication may be inferred from this list. Criteria for judging the quality of mathematics education research were formulated already in these early years of our field. One such report identified the following characteristics: relevance, validity, objectivity, originality, rigor and precision, predictability, reproducibility, and relatedness (Kilpatrick and Sierpinska 1993). Another identified elements that are not unlike these, in different terms: worthwhileness, goodness of fit, competence, openness (awareness of researcher biases, and full reporting tempered by ethics), credibility (grounded in data, with evidence), and intangible qualities such as lucidity, conciseness, and originality (Lester and Cooney 1994; see also Lester and Lambdin 1993).

In unpacking some of these terms, essential elements that relate to writing up research may be highlighted. Worthwhileness indicates that an author should make the significance of the study clear. Why was it important to address this particular question? What does this study add to what was already known from previous research? The author should thus take into account the prior state of the field, as reflected in relevant literature. Goodness of fit relates to the suitability of choices regarding theory, the broad umbrella of methodology, and specific methods of data collection and analysis within this methodology. The relative merits and purposes of quantitative and qualitative methodologies have been debated through the decades (e.g., Lester 2007), and it is now recognized that a combination of methodologies is possible and may exploit the benefits of each, in what has come to known as “mixed methods” (e.g., Keller and Buchholtz 2015). Examples of methodology and methods, with appropriate theories, are presented in books such as that by Bikner-Ahsbahs et al. (2015), in which theoretical chapters are paired with empirical chapters that illustrate the implementation of theories and methods in particular studies. All aspects of the research need to hang together in an integrated whole.

Competence, openness, and credibility are characteristics of the researcher, as manifested in the writer’s awareness of the need to be open to his or her biases, and to provide evidence for each claim made as a result of the empirical work. The sensitivity of the researcher to ethical issues is also relevant here: There may be ethical reasons not to report some aspects of the research (e.g., inadvertent data gathered without permission). Further, conflict of interests may be a source of bias that may detract from the validity and objectivity of the conclusions. In providing evidence for claims made, there are three possible sources of evidence, namely, the data themselves, evidence from trustworthy literature, and logical argument, which relates to the rationality and lucidity of the writing. However, beliefs and values of the researcher are inevitably implicated in this process:

How researchers go about convincing others of the claims they make and how they defend their claims on ethical and practical grounds are, only in part, matters of marshaling adequate contextualized evidence embedded in sets of beliefs and theories. Indeed, convincing others is also a matter of persuading them to accept the values the researcher holds about the objects and phenomena being studied as well as about the very purpose of research itself. (Lester and Wiliam 2000, p. 136)

From the foregoing lists, for the purposes of academic writing in order to disseminate the results of empirical research, some further essential elements may be inferred. The following section addresses aspects that editors and reviewers may look for in deciding what will be published in mathematics education journals.¹

2.2 What Are the Essential Elements in a Research Report in Our Field?

Nuts-and-bolts matters to be taken into account by prospective authors include the following:

• Select an issue on which the report will center; even if the research study addressed several issues, the report needs a focus.

• Decide on a publication venue (e.g., journal) and become acquainted with its style, formatting, and referencing, making sure that these are appropriate for your study.

• Think through the structure of the paper, and write a first draft according to the resulting outline (see the next section for a possible outline of a structure).

• Supply evidence or justification for every statement. Justification may be presented in one of the three ways: by referring to relevant literature, by evidence from data, or by logical argument.

• Be sure the manuscript is checked for fluency and accuracy in language (including citations and the reference list) before it is submitted.

After you have produced the first draft of your manuscript, put it aside for several days and then return to it prepared to make revisions. Find some colleagues willing to read the revised draft and provide suggestions for revision. Revise the manuscript in light of your colleagues’ suggestions. Proofread the manuscript and submit it to the journal. Respond to the feedback you receive. The following are components of a well-structured research paper:

• There is a well-written abstract, describing the area of the research, its design, extent, and the main results.

• The introduction provides a succinct but interesting background to the main issues.

• The issues addressed emerge unambiguously from the introduction. Research questions may be presented towards the end of the introduction, or after the next component. Meanings of key terms are clarified.

• There is a succinct, scholarly review of relevant literature (international in nature, using original papers if possible) leading to a clear conceptual framework. The unique contribution to knowledge in the field of the study should be clear.

• Details are provided for the empirical design, for the sample of participants, and of how the design was implemented. Are the data representative? Were there various stages in the research?

• There is a summary of actual data obtained, and a detailed report of how the analysis of the data was carried out.

• Finally, there is discussion of the implications and limitations of the research, and of possible directions for carrying the line of research forward in the future. The discussion should make clear how the conclusions of the report address the main issues.

2.3 One Possible Structure for a Report on Empirical Research

The foregoing points may be distilled in the following effective structure,² which could be used for a thesis or a research report (and see also the similar structure presented in Fig. 20.8 in Chap. 20 of this volume).

This structure includes a zooming-in feature: Points 1 and 7 are quite general in nature, progressively becoming more specific and detailed as the internal aspects of this particular study are described in Points 3, 4, and 5. The results in Point 6 are related back to the literature and theory in Point 2. Finally, the conclusions outline what has been accomplished in addressing the problem with which the research started in the first place.

This structure resonates well with an outline put forward by Brown and Dowling (1998), in which a theoretical field (in the “theoretical domain”) and an empirical field (in the “empirical domain”) sandwich the central phases of the research, zooming in with increasing specializing and localizing, respectively, to the details of the research study, which are supplied in the middle of the report.

2.4 An Example of a Mission Statement of a Research Journal

In deciding on a suitable venue for publication, it is important to take into the account the purpose and nature of various journals. The following is the original and current mission statement of Educational Studies in Mathematics (ESM), which is published in every issue of this journal:

ESM presents new ideas and developments which are considered to be of major importance to those working in the field of mathematics education. It seeks to reflect both the variety of research concerns within the field and the range of methods used to study them. It deals with didactical, methodological and pedagogical subjects rather than with specific programs for teaching mathematics. All papers are strictly refereed and the emphasis is on high-level articles which are of more than local or national interest.

Different journals have different aims and reasons for their existence, as the following section illustrates. Thus choice of journal is an important concern that should be taken into account at the outset.

3 An Activity That Illustrates Why Choice of Journal Is Important

Because unpublished writing is protected, in one workshop preceding ICME-13 we reviewed a published paper from a respected journal, For the Learning of Mathematics (FLM), using the criteria from another respected journal that has a different purpose and focus, the Journal for Research in Mathematics Education (JRME). The paper from FLM is a delightful account by Jenny Houssart in 2001, titled “Rival Classroom Discourses and Inquiry Mathematics: ‘The Whisperers.’” Jenny served in the role of classroom helper, observing while sitting at the back of a mathematics classroom (of a “bottom set” of pupils) in a British primary school, during a year of participation research in which she was engaged. She had not intended to concentrate on “the whisperers”—her observation of the events reported in the paper was serendipitous. Thus she did not record the boys’ words verbatim, e.g., using a tape recorder. Her research interest was in “inquiry classrooms” and the culture of inquiry. The following is an anecdote from the paper:

A class of nine- and ten-year old children were working on fractions of shapes. Two worksheets were given out and the children were asked to look first at the one starting with a rectangle. This caused some confusion as one sheet started with a circle and one with a square. In response to this the teacher asked, ‘Do you know what a rectangle is?’ He pointed to the sheet in question and one of the children said, ‘But it’s a square.’ The teacher looked again at the sheet, admitted that it was a square, and apologised for calling it a rectangle. As the teacher started to explain the sheet, one of the children sighed slightly and said in a whisper, ‘Well, anyway, it is a rectangle’. (Houssart 2001, p. 2)

The following are some details of the research and events reported in the paper:

Data sources: The words of four boys whispering at the back of the room; interviews and discussions with the teacher; documents such as lesson plans and pupils’ work.Methodology: Qualitative: participant observation once a week for an entire school year.Authors of literature invoked and cited in the paper: Bauersfeld, Brissenden, Cobb and Yackel, Pimm, Richards.

The paper does not start with an abstract. In the activity of critiquing this paper, we used basic questions considered useful for authors and reviewers associated with JRME, for the purpose of deciding whether Houssart’s paper was suitable for publication in JRME. These questions were as follows:

• Does the research deepen our understanding of important issues? Does it have the potential to lead the field in new directions?

• Do the research questions pertain to issues of significant theoretical or practical concern? Are they well-grounded in theory or in prior research?

• Is there an appropriate match between the research questions and the methods and analyses employed?

• Does the conduct of the study include the effective application of appropriate data collection, analysis, and interpretation techniques?

• Are the claims and conclusions in the manuscript justified, and do they logically follow from the data or information presented?

• Is the writing clear, lucid, and well-organized?

Many of the current journals in mathematics education (including JRME) present four alternative choices for reviewers’ recommendations to the editor after they have critiqued a manuscript:

• Accept the manuscript for publication as it is;

• Suggest minor revisions, after which the manuscript is likely to be accepted (although not always) after review by the editor and possibly one or more other reviewers;

• Suggest major revisions, after which a new reviewing cycle will be carried out, and the manuscript may or may not be accepted;

• Reject the manuscript.

In practice the first of these alternatives is rarely used; in most cases authors can be helped to strengthen their writing by some form of revision, either major or minor in nature. A decision of minor revisions implies that the write-up is basically sound, requiring only superficial changes. Major revisions might entail a complete restructuring of the manuscript, perhaps with a request for additional information. The difference between a decision of major revisions and one of rejection is that in the former case the editor believes that the researcher has the materials from the research to bring the manuscript into a publishable form; while in the latter case the editor believes that no amount of re-writing will remedy elements that are missing or unsuitable. Thus elements from the research itself, and the nature of the write-up, are both entailed in decisions about whether a manuscript should be published.

In the case of workshop participants’ critique of “The Whisperers” both the nature of the research, and the academic aspects of the write-up, were taken into account in discussion of the manuscript. If it had been only the writing that needed changes (e.g., adding an abstract, bringing in more relevant literature, etc.), then clearly the decision could be major revisions needed—or as some participants suggested initially, even minor revisions. However, in this case there were issues with the research itself, as addressed by some of the six categories of JRME questions. Let us consider these points in turn:

• Yes. The research does deepen our understanding by raising significant issues regarding the teacher’s beliefs about the abilities of the whisperers; this paper does have the potential to lead the field in different directions;

• No. There were no research questions formulated by the researcher at the outset, concerning the whisperers;

• Thus it was not appropriate to ask for a match between research questions and the methods and analyses employed; the observations were serendipitous;

• No. The data presented were anecdotal, without formal audio or video recording; thus it could not be considered that there were “techniques of effective and appropriate data collection, analysis, and interpretation”;

• Yes and no. Some of the claims and conclusions in the manuscript appeared to be justified and to follow logically from the information presented; however, the informal nature of the boys’ words that were overheard, might be considered a negative aspect if the study is taken to be an example of formal research;

• Yes. The writing is clear, lucid, and well-organized, within the parameters of the nature of the study.

Thus, despite the significance of the issues highlighted, it is apparent that no amount of re-writing would bring the manuscript into a form that would be acceptable for publication in JRME. The unanimous final decision resulting from the review by the workshop participants was that this paper should be rejected for JRME. An appropriate response to the author of the manuscript was that she “should consider resubmission of the manuscript to a different journal [not JRME], such as For the Learning of Mathematics.” It was acknowledged that FLM’s purpose was a wider, more exploratory one than that of a stringent research journal such as JRME. Both purposes are important for the furthering of the field of mathematics education. The merits of Houssart’s observations in her paper were acknowledged to be important for mathematics education, in that the teacher of ‘the whisperers’ considered these four boys to be less able mathematically, yet their whispered comments often manifested insight and ability. The message is clear: Choice of the correct journal for a manuscript is a significant issue.

4 Pitfalls

There are many pitfalls in academic writing into which unprepared writers can fall. For example, they may omit relevant information, assuming that because they are familiar with the issue under investigation or the conditions under which the data were gathered, the reader will be too. Another problem commonly faced by novices arises from an attempt to squeeze a dissertation’s worth of research into a single journal article, thereby confusing the reader by raising too many issues and providing too much information. The scope of the report should be clear at the outset, and if it raises more than two or three main issues, the writer should consider writing more than one article. Sometimes writers forget either that they are telling a story—and therefore the parts need to fit together—or that they are also reporting research—and therefore the claims they make need to be supported with evidence. By the time they reach the conclusion section, they also may have forgotten the rationale they gave for the study and the research questions they posed at the outset, thereby leaving the reader dissatisfied.

We end with a deeply ironic piece written as an editorial for the JRME more than three decades ago (Kilpatrick 1985) and reprinted by permission of the National Council of Teachers of Mathematics. The editorial raises a number of pitfalls that are unfortunately all too common in academic writing in our field.

5 Editorial Sarcasm

Irony is a powerful literary device that may take various forms, several of which are present in this piece:

• antiphrasis—broad use of words that convey a meaning opposite to what is intended;

• epitrope—ironical permission to act in a certain way;

• paralipsis—phrases that reveal some aspect despite a proposal not to do so. (Joseph 2005, p. 138)

The following editorial gives some ironic hints about why we consider the reporting of scientific research to be a significant issue worthy of time and care. It seems to have arisen because the editor lost patience after receiving too many submissions of unpublishable manuscripts.

Editors and reviewers are reputed to be busy people, but that is a fiction. The JRME editors and reviewers lead dreary lives of unremitting sloth. If you plan to submit a manuscript to the JRME, here are several easy ways that you can bring stimulation and challenge to some idle minds.First, do not bother to read the journal itself and pay no attention to what the Publication Manual of the American Psychological Association (Third Edition) has to say about preparing a manuscript. Be a free spirit. Try to convey the impression that you are not a person to be bothered with petty details of style and format.Second, see if you can give your manuscript the portentous tone of a dissertation or project report. Start with the title; it should be as long as possible. Mention every variable you studied, along with the names of the instruments you used. If the title still seems too short, try adding “A Report of a Study Designed in an Attempt to Investigate Various Factors That Might Be Associated With….” If that is not enough, add the name of the institution and the city where you did the study. Either omit the abstract altogether (you’re a busy person, right?) or, better, stretch it out to at least 250 words so that you can allude to results that are not included in the body of the report, thereby providing the reviewers with more of a challenge. In setting up tables and figures, just remember that they should not be easily interpreted on their own. Verbatim copies of computer printouts usually make wonderful tables. If a table seems too stark, you can add a dozen or so cryptic footnotes. Most tables should be discussed entry-by-entry in the text, but the virtuoso writer will include at least one table or figure that is not cited anywhere.

Third, do not let the organization be obvious. Avoid headings or subheadings that might reveal too much. Use Introduction at the beginning and put Results somewhere in the middle, if you wish, but report the results all the way to the end, as they occur to you. Aim at a stream-of-consciousness effect. If you want to give the purpose of the study, follow Agatha Christie’s style and tell your secret at the end. You can, however, give the manuscript a nice absurdist touch by omitting any statement of purpose.

Finally, do not let anyone else read your manuscript before you submit it. After all, who are you writing for if not yourself? Proofreading the manuscript carefully and letting colleagues look it over might suggest that you were eager to have it published. If you follow the simple suggestions above, you will not need to worry about publication. And you can know the satisfaction of having given some extra work to the editors and reviewers.