Keywords

1 Introduction

I am honored and grateful to have been awarded the 2013 AERA Distinguished Researcher award. I am gratified as well that the timing of the award allows me to unveil a new set of tools for understanding and supporting powerful classroom instruction. This article also allows me to reflect on, and exemplify, two themes that have been central to my work from my beginnings in educational research almost forty years ago:

  1. (1)

    research and practice can and should live in productive synergy, with each enhancing the other;

  2. (2)

    research focused on teaching and learning in one particular discipline can, if carefully framed, yield insights that have implications across a broad spectrum of disciplines.

In what follows I shall briefly review two main lines of my work over the years: a decade of work on problem solving, and two decades of work on modeling the teaching process. My focus in those reviews will be on elaborating the research-practice dialectic, and showing how results first derived in mathematics apply more generally. Having done so, I will turn my attention to work I have been involved with over the past half dozen years, which has been focused on understanding the attributes of powerful mathematics classrooms. Here too, I will show how an expanding R&P agenda has enriched both the research and the practices with which it has lived in happy synergy. Then, having elaborated on the nature of the work and framed it within mathematics, I will indicate how it yields a set of hypotheses for understanding and enhancing teaching in all disciplines.

2 Case 1: Problem Solving in Mathematics and Beyond

Perhaps the best way to summarize my problem solving work (see, e.g., Schoenfeld 1985, 1992) is that it consisted of a decade-long series of design experiments aimed at understanding and enhancing students’ mathematical problem solving.

Here a brief theoretical detour on the nature of design experiments is in order—my impression is that a significant number of researchers take “design experiment” to mean something like “cycles of design and implementation that result in an improved educational intervention.” While design experiments do include such cycles, there is more to them than that. They are about improving both theory and practice.

So, just what is a design experiment? As I understand it (see, e.g., Cobb et al. 2003; Schoenfeld 2006),

  1. a.

    One has a “local theory” about learning, which suggests some aspects of design.

  2. b.

    One crafts a theory-based intervention (“the local theory says that this intervention ought to work in the following ways, to enhance understanding in these ways”).

  3. c.

    On the basis of implementing the intervention and carefully observing its impact, one (i) refines the local theory and (ii) refines the intervention.

That is, design experiments are as much about theory as they are about design. That said, let me briefly recap the evolution of my problem solving work. Prior to building my first problem solving course, I had conducted some experimentation, and done some theorizing, about what it would take to make effective use of particular problem solving strategies. I took these ideas into the first version of the course, where I found that some things seemed to work, but some didn’t. Close observations of my students’ struggles provided a refined understanding of aspects of metacognition, which I could then better theorize. The same was the case for laboratory studies and ongoing instruction, aimed at understanding and then having an impact on belief systems. Over the course of a decade, my evolving understanding resulted in the following theoretical perspective:

If one seeks the reason(s) for someone’s success or failure in a mathematical problem solving attempt, the cause of that success or failure will be located in one or more of that person’s

  1. A.

    mathematical knowledge and resources

  2. B.

    access to productive “heuristic” strategies for making progress on challenging problems

  3. C.

    monitoring and self-regulation (aspects of metacognition), and

  4. D.

    belief systems regarding mathematics, and one’s sense of self as thinker in general and a doer of mathematics in particular (in more current language, one’s mathematical identity). (See, e.g., Schoenfeld 1985).

At the same time, the problem solving course improved, both profiting from the research and contributing to it. In that way, then, research and practice lived in productive synergy. (This is fundamental assumption 1.)

It goes without saying that the problem solving work was not conducted in a theoretical vacuum. There was, of course, a large body of research on domain-specific knowledge in a wide variety of content areas. As I was elaborating on productive strategies (heuristics) for engaging in mathematics, others were elaborating on productive strategies in other domains (e.g., in reading (Palincsar and Brown 1984), writing (Scardamalia and Bereiter 1983), science (diSessa 1983), and artificial intelligence (Newell 1983)). Clearly, metacognition has a domain-specific component (that is, the more knowledge you have, the more effective one can be in monitoring and self-regulation), but aspects of it are domain-general (e.g., Brown 1987). Similarly, some beliefs about self may be general, some mathematics-specific; but there are surely analogs in other domains. Thus, when the mathematical problem solving work was well established, I could claim with confidence that A through D above were essential components of mathematical problem solving. But, given parallel work in other domains, I could already suggest the following with some confidence:

If one seeks the reason(s) for someone’s success or failure in a problem solving attempt in any knowledge-rich domain, the cause of that success or failure will be located in one or more of that person’s

  1. A.

    domain-specific knowledge and resources

  2. B.

    access to productive “heuristic” strategies for making progress on challenging problems in that domain

  3. C.

    monitoring and self-regulation (aspects of metacognition), and

  4. D.

    belief systems regarding that domain, and one’s sense of self as thinker in general and a doer of that domain in particular (in more current language, one’s domain-specific identity).

That conjecture has held, in that resources, strategies, metacognition, and beliefs are generally acknowledged to be essential parts of sense making in every field, and no additional arenas have been declared as essentialFootnote 1 (see, e.g., Collins et al. 1989). I take this to be a case in point of fundamental assumption 2, that the research originally done in mathematics had direct analogues in other content domains.

3 Case 2: Decision Making in Mathematics Teaching and Beyond

My second example concerns the theory of decision making put forth in my book How We Think (Schoenfeld 2010). My research group, known as the “Functions Group,” had been conducting a series of studies of mathematics tutoring (see, e.g., Schoenfeld et al. 1992) in order to develop the tools for studying teaching. My first case study of teaching occurred when a student in our teacher preparation program was dissatisfied with a lesson he had taught, but couldn’t figure out why things had gone wrong. The coordinator of the program, who was also a member of Functions Group, suggested that he bring a video of his lesson to us for analysis.

My goal was to do more than help the student, though that was important. Ultimately, my intention was to be able to model this mathematics teacher’s classroom decision making, on a moment-by moment basis—and then others’.

Let me unpack the central ideas in the previous sentence. First, my intention was to build an analytical model of the student’s teaching. The reason for modeling is that modeling is a rigorous way to test theoretical ideas. It’s one thing to say “I think this is why the teacher did what he did;” it’s quite something else to say “here is a general model of the decision making process during teaching, into which I insert a characterization of the key aspects of this teacher’s teaching. The model makes the same decisions that the individual did.” Models are falsifiable, which means that one’s explanations are put to the test.Footnote 2 Second, the state of the art had advanced to the point where we might have a chance of understanding and modeling teaching as it happens, rather than studying behavior in the laboratory. When the state of the art allows for it, the idea is to push the theory-practice dialectic. Third, yes, I began by modeling mathematics teaching—because my grounding is in mathematics and it is what I know best—but, it was reasonable to expect that the essentials of teachers’ decision making (if framed the right way) would be akin to teachers’ decision making in other fields.Footnote 3

To summarize a book in a paragraph, the key idea in How We Think (Schoenfeld 2010) is that people’s moment-by-moment decision making in: teaching; in medicine; in fact, in all knowledge-rich domains, can be modeled as a function of their:

  • Resources (especially their knowledge, but also the tools at their disposal);

  • Orientations (a generalization of beliefs, including values and preferences); and

  • Goals (which are often chosen on the basis of orientations and available resources).

I will note, as before, that the theoretical ideas summarized above evolved in productive dialectic with practice. The first attempt at modeling was motivated by a problem from practice, and potential limits to the scope of our models were posed by other cases of “real world” teaching. Moreover, as our understandings evolved, we embarked on experimental work related to professional development (see, e.g., Arcavi and Schoenfeld 2008). It is one thing to say, theoretically, that orientations are central and belief change is slow; it is quite something else to try to achieve belief change. In attempting to do so and studying what happens, one develops a more nuanced understanding of the growth and change of belief systems. Thus, the work modeling decision making during mathematics teaching was, again, a case example of the productive research-and practice synergy (that is, fundamental assumption 1).

Re fundamental assumption 2, I will note again that while I was focused on mathematics teaching in the early phases of modeling, the architecture of the decision making process was constructed so as to be general. As in the case of problem solving, what was known about decision making in a wide range of knowledge-intensive fields (e.g., teaching in other domains, but also medical decision making; see, e.g., Groopman 2007; Szolovits et al. 1988) was entirely consistent with what was known in mathematics; it stood to reason that with the right framing, a theory of decision making during mathematics teaching could be seen as an instance of something more general (and, the applications to other fields would be straightforward). How We Think makes a plausibility case, with ties to the broader literature and some fully worked out examples. Time will tell how robust the claims will be.

4 Case 3: Documenting and Supporting Productive Teaching in Mathematics

4.1 (3A): What Counts in Mathematics Teaching?

Here we come to the core example of this paper. The question here is: can one identify the key aspects of powerful mathematics classrooms—classrooms that produce students who do well on tests of mathematical content and problem solving?

The motivation for this work was the observation (a half dozen years ago, before the MET study (Bill and Melinda Gates Foundation 2012) was conducted) that, although many of us have strong opinions about what makes for ‘good teaching’ in mathematics and other classrooms, there was precious little evidence to support those beliefs. Ideally, one would like to have tools to measure classroom practices, and tools to measure student performance, to explore the relationship between the two: do classrooms that score high on the dimensions of purported importance produce students who score high on tests of mathematical thinking?

To explore this issue, we needed a classroom measure that (a) was comprehensive, (b) focused on key aspects of mathematical sense making, (c) contained a relatively small number of important dimensions (so that, among other things, the implications for professional development would be clear), and (d) could be used in perhaps twice real time to code classroom data (that is, an hour of observations and note-taking would require an additional hour to code the score), so that large-scale data analysis would be feasible. As explained in Schoenfeld (2013), although there were many schemes for classroom analysis—e.g., Beeby et al. 1980; Danielson 2011; Institute for Research on Policy Education and Practice 2011; Junker et al., 2004; Marder and Walkington 2012; PACT Consortium 2012; Pianta et al. 2008; University of Michigan 2006—no schemes with attributes (a) through (d) above were available. Thus we set about building a theoretical framework and a classroom analysis rubric to match. The details of how we arrived at the theoretical frame are given in Schoenfeld (2013). In Table 26.1, I set forth our hypotheses regarding the key dimensions of powerful mathematics classrooms.Footnote 4 Again, this distillation is grounded in the literature (see Schoenfeld 2013; Schoenfeld et al. 2014). The key feature here is the distillation into a small number of dimensions.

Table 26.1 The five dimensions of mathematically powerful classrooms
Full size table

To explore the hypothesis that these are indeed (one productive organization of) the dimensions of powerful classrooms, one needs a rubric for classroom observations. The team has produced such a rubric, called the “Teaching for Robust Understanding of Mathematics” (or TRU Math) rubric.Footnote 5 Before summarizing its essence I must emphasize that this rubric was developed a research tool, and that it is not intended for administrative use in evaluating teachers. This is for at least three reasons: (1) validation of the rubric through research is in its very early stages; (2) although the summary rubric given below seems straightforward, the actual use of the rubric requires training; and (3) we would much rather focus on working productively with teachers, as opposed to rating them. As seen later in this paper, we have an approach to professional development that focuses on engaging teachers in productive activities and conversations concerning the five dimensions of TRU Math.

The full TRU Math rubric contains sub-rubrics for characterizing episodes of: whole-class instruction; small group work; student presentations; and individual student work. Using the rubric involves parsing classroom activities into a sequence of ‘episodes’ of no more than five minutes each in duration, assigning scores to each episode using the relevant sub-rubric, and then computing a weighted average of scores. The summary rubric, which is not used for scoring, is given in Fig. 26.1. The summary rubric does provide a clear sense of the kinds of classroom activities that will score high or low along each of the dimensions.

Fig. 26.1
figure 1

The TRU Math Summary Rubric

Full size image

As noted above, to explore the relationship between classroom practices and student performance, one needs robust measures of student performance—in particular, measures of content, concepts, reasoning and problem solving. This, fortunately, has been a focus of our work since the early 1990s. The Balanced Assessment Project (Hugh Burkhardt, Alan Schoenfeld, Judah Schwartz, and Sandra Wilcox, principal investigators) was first funded in 1992 to construct assessments in line with the 1989 NCTM Standards, and versions of the project (the Mathematics Assessment Resource Service and the Toolkit for Change, with Burkhardt, Schoenfeld, and Wilcox as PIs, and the Mathematics Assessment Project) have continued to the present day.Footnote 6 In sum, we now have a set of tools—both independent and dependent measures—for the empirical exploration of research-based hypotheses regarding powerful instruction and its impact. (See the left hand side of Fig. 26.3.) Very preliminary data analyses suggest that the relationship exists, but large n studies are really what is called for.

4.2 (3B): Supporting Effective Mathematics Teaching

The next logical question is how to enhance teachers’ proficiency along those five dimensions (and study the impact of that work on teachers and students). Here the Mathematics Assessment Project and the Algebra Teaching Study have produced two sets of tools. The first is a set of “Classroom Challenges” or “Formative Assessment Lessons” designed to support teachers engagement in formative assessment focused on core content. To put things simply, doing formative assessment is hard: it calls for a set of pedagogical habits of mind and pedagogical content knowledge that most teachers do not have, and which are not simple to acquire. For that reason the Mathematics Assessment Project developed a series of 100 Formative Assessment Lessons (FALs) whose purpose it is to scaffold teachers in teaching formatively. The content-oriented FALs are grounded in research on what students find difficult. They begin with tasks designed to elicit student thinking—to reveal bases of understanding that can be built upon, and to reveal misunderstandings that need to be addressed. The lessons indicate common patterns of student responses to the tasks, and ways to deal with them; they also contain activities that support the teacher in further assessing student understanding, and building on it.

It is worth discussing the project’s design methodology here, for it too reflects a form of research-and-practice dialectic. Project designers based at the University of Nottingham craft a draft version of a lesson, which is then piloted in local schools. Team members observe the lessons (using a feedback rubric) and make suggestions for refinement to the design team, which then modifies the draft. When the draft is deemed solid, it is sent to three observation centers in the US (in California, Michigan, and Rhode Island). The lessons are taught in a range of classrooms, and the observers document how they are working (or not) using a standard protocol. Observation forms are returned to Nottingham, where they are compiled by and discussed by the team. Revisions are then made by a team member other than the person who designed the lesson. This results in an “alpha” version. The alpha version is then distributed to the observation centers in the US, where the observation process is repeated and the compiled feedback is used in the creation of a beta version. See Mathematics Assessment Project (2014) for detail.

To date there have been more than two million Formative Assessment Lesson downloads from the project website, http://map.mathshell.org/.

The second set of materials we offer by way of support for teachers is the “TRU Math Conversation Guide” (Baldinger and Louie 2014). This tool, intended for teacher-coach conversations (or, better, professional learning communities), addresses each of the five dimensions of TRU Math by raising a series of questions that teachers might consider with regard to lesson planning, debriefing, and thoughts about where to go next. In the conversation guide, the approach to the five dimensions is reframed as in Table 26.2.

Table 26.2 Framing questions in the TRU Math Conversation Guide
Full size table

These framing questions are mere overtures to conversations; the questions are elaborated in the Conversation Guide. Figure 26.2 shows the conversational elaboration of the third dimension, Access to Mathematical Content.

Fig. 26.2
figure 2

Access to mathematical content, in the TRU Math Conversation Guide

Full size image

The Formative Assessment Lessons and the TRU Math Conversation Guide constitute the Professional Development part of the TRU Math Suite. As discussed below, they constitute the right hand side of Fig. 26.3.

Fig. 26.3
figure 3

The research, practice, and development dialectic in mathematics

Full size image

5 Discussion: The Current State, and Possibly Productive Next Steps

Figure 26.3 offers the “big picture” view of the enterprise as I have described it thus far.

In concluding I would like to review the enterprise described in Fig. 26.3 with regard to the framing assumptions outlined at the beginning of this article.

Framing Assumption 1: Research and practice can and should live in productive synergy, with each enhancing the other.

The examples discussed in this article provide substantial backing for this assumption. In the problem solving work, the course and the theory I was developing were mutually enriching, with the theory suggesting ideas for implementation and, at times, my intuitions as a teacher suggesting things for me to explore theoretically. The result was a ‘virtuous cycle’ of discovery, including the refinement of ideas about strategy implementation, metacognition, and belief systems. The work on teacher modeling was inspired by problems of practice, and refined by it; in turn, the theoretical work suggested avenues for the improvement of practice. In the current body of work, it is worth noting that every object and every arrow in Fig. 26.3 represents or embodies a productive dialectic between theory and practice.

Framing Assumption 2: research focused on teaching and learning in one particular discipline can, if carefully framed, yield insights that have implications across a broad spectrum of disciplines.

As indicated above, the problem solving work was done in mathematics but had obvious analogues in other disciplines. The theoretical claims with regard to generality have stood the test of time. Similarly, the decision making work was done in the context of a vast literature on decision making; mathematics teaching was the focus of my core examples, but the modeling was framed in ways that could be abstracted. I believe the same is the case with regard to the issues encapsulated in Fig. 26.3. The TRU Math scheme is, of necessity, grounded in the specifics of mathematics teaching and learning. Thus, dimension 1 (“the mathematics”) is fundamentally mathematical, just as the “resources” in the original problem solving work and teacher modeling were fundamentally mathematical. But the other dimensions of TRU Math—cognitive demand; access to meaningful engagement with the content; agency, authority, and identity; and the uses of assessment—while ‘tinged’ with mathematics when one looks at mathematics instruction, are general. That is, in a writing (or literature, or physics) class they would be ‘tinged’ with writing (or literature, or physics) in the same ways.

Hence, to use mathematical language, one can think of “The Mathematics” in the TRU Math work as a variable—call it X, where X = “any particular discipline.” In the new scheme, called the “TRU X framework,” the first dimension, “The X,” would be “the extent to which this discipline (“X”) comes alive in the classroom as described in content standards or other documents, with students having the opportunity to develop productive disciplinary habits of mind.” The other three tools (represented by the dark boxes) in Fig. 26.3 would be fleshed out analogously, resulting in Fig. 26.4.

Fig. 26.4
figure 4

The research, practice, and development dialectic in general

Full size image

This, of course, is just a conjecture at this point. I do think, however, that working on it will be a productive enterprise. I look forward to doing so, with colleagues from across the educational spectrum, in the years to come.