Introduction

Teachers’ interactions with curricular resources are changing. Studies such as the TIMSS report (Hooper et al. 2015) show that though mathematics teachers in most countries still rely on a textbook for most of their curricular decisions, it is quite common for teachers to supplement the curriculum with learning resources available on the Internet. In some contexts, teachers do not have a single textbook to rely on and are responsible for the construction of teaching sequences. However, the web does not hold semantic information, which is necessary if one wishes to search for learning resources according to subtle epistemic or didactic features. Relying on existing search engines to find web-based learning resources may seriously compromise the integrity and the balance of the intended curriculum.

We are developing tools to address this shortcoming. We study the use of a pair of coupled tools—a browser extension for tagging various aspects of learning resources on the web (i.e., associating metadata with URLs, see Fig. 1) and an open-source dashboard-configuration tool (https://keshif.me/, see Fig. 2) to view and navigate collections of tagged learning resources. One of the uses that we envision for these tools is that teachers will tag resources that they have used or are planning to use, and the metadata that they associate with shared resources will support other teachers in their search for activities that are as suitable as possible for their own particular teaching contexts and pedagogical preferences.

Fig. 1
figure 1

Tagging tool, showing two of the categories of metadata

Full size image
Fig. 2
figure 2

Dashboard showing a preview of what will remain after filtering on Role: practice. Upper middle section shows titles of all the tasks, highlighting in orange those that match the filter. Other sections indicate the number of tasks that will remain, e.g., of the 20 tasks in the “Phenomena” chapter, seven match the filter (upper left section)

Full size image

A major design challenge we face is deciding on categories of metadata that are relevant for teachers’ curricular design. The literature on the evolving relationship of teachers with learning resources suggests some research-based answers; however, mathematics teachers and researchers of mathematics education may have very different perspectives on curricular design, and a tool whose design is based on researchers’ perceptions of teachers’ needs will not be used if it does not address the needs of teachers as perceived by teachers. Indeed, a tagging experiment (Cooper and Olsher 2018) conducted with seven practicing high school teachers revealed some fundamental differences in researchers’ and teachers’ perspectives on didactic metadata and on tagging. These differences suggest both challenges and opportunities for teachers’ productive use of these tools. In the current research, we report on an experiment conducted with the same teachers a few weeks later, in which they were asked to construct a short teaching sequence, using the dashboard to select resources according to tagged metadata. We also include findings from a similar experiment conducted with pre-service teacher candidates using the same dashboard tool with similar categories of metadata.

Literature review

Recent publications reflect the ongoing interest of the mathematics education community in textbooks (e.g., Fan et al. 2013), acknowledging them as a crucial factor in shaping the curriculum that is taught in classrooms. However, with the availability of technology, and the coming of age of the e-textbook, the nature of teachers’ interactions with textbooks is changing (Pepin et al. 2015). Even where printed textbooks are still the main vehicle of the curriculum, many teachers are beginning to integrate interactive computerized activities in their teaching (Hooper et al. 2015). In some contexts, this may simply be in response to institutional expectations to “integrate technology” in instruction, and in others, it may be with well-defined pedagogical goals in mind—as a way to support inquiry learning (e.g., Yerushalmy 2016), or as a means toward personalized learning for students with special needs (e.g., Hasselbring and Glaser 2000). Some teachers may address this need by authoring their own digital activities, but the web is flooded with interactive learning resources, and searching for existing activities ought to be more efficient than developing activities from scratch. Yet searching the web may be a frustrating experience for critical teachers. Research has demonstrated that “the right fit between the technology, the software, and the instruction is essential for implementation” (Hooper et al. 2015, p. 76), but existing search engines are not designed to help achieve such a fit. Search engines are sensitive to the title of a web page, to the text that appears in it, and to any metadata that the author may have included in the HTML file, and search results are organized according to some notion of ranking. Thus, teachers are limited to rather coarse-grained searches—by mathematical topic, possibly including some keywords such as “inquiry” or “advanced students,” which may or may not prove to be useful in focusing the search—and will typically consider only the most high-ranking (i.e., popular) results of their search. In this state of affairs, teachers are not required to be explicit about the didactic characteristics of tasks, or about the kind of interactions they would like their students to have with the mathematics.

Changes in the way developers and researchers conceptualize mathematics textbooks may also have an impact on teachers’ relationship with instructional resources, particularly as the e-textbook comes of age. Chazan and Yerushalmy (2014) argue that changes in the role of textbooks in specifying what is to be taught, and changes in the processes for authoring and publishing textbooks, “have the potential to shift the role of teachers in the curriculum development process” (p. 63). Pepin et al. (2015) distinguish between three models of currently available e-textbooks—dynamic, evolving or “living,” and interactive. In the “dynamic” model, a static textbook (traditional or digital) is linked to other learning objects. In the “living” model, textbooks are dynamically and cumulatively authored by a community—often a community of teachers (e.g., Gueudet et al. 2013). The third model of e-textbooks—interactive—is based on a toolkit model and is anchored in a set of learning objects, where tasks and interactives can be linked and combined in different ways, affording what Chazan and Yerushalmy (2003) call “object-oriented navigation through the book.” Remillard (2016), borrowing from Usiskin’s travel image (2010), describes such navigation by means of the metaphor of cartography; “A mass of [curricular] roads has been built, both superhighways and tiny lanes, but the choices of how to travel them are vast and few trips have been preplanned” (p. 200)—a situation that places teachers firmly in the role of curriculum designers.

In the first two models of e-textbooks, teachers must find (or author) learning objects, and in all models (including interactive) they must select and sequence learning objects, actions which may be crucial for learning. For example, the order in which learners engage in instructional activities has been shown to influence ways in which inherent discontinuities in the content are handled (Yerushalmy and Chazan 2008). Teachers’ use of curriculum resources is a complex process, involving “interpreting, authoring, appropriating, adapting, framing and reframing the content” (Remillard 2016, p. 197). In this era—abundant with web-based material—some studies have attended to considerations that guide teachers in the selection of instructional resources (Siedel and Stylianides 2018) and the role of metadata in the choices they make (Abramovich et al. 2013). Siedel and Stylianides (2018) interviewed teachers and categorized their predispositions in selecting instructional resources into six key themes: student-driven selection, teacher-driven, mathematics-driven, constraints-driven, resource-driven, and culture-driven. Abramovich et al. (2013) have found that teachers, in determining whether to download a resource, appear to rely on complex heuristics that are influenced by many factors and that the number of ratings for a resource more strongly predicts downloads than do the mean rating levels. Such dispositions will come into play when teachers search for tasks to supplement a given learning sequence or plan learning paths through a flexible e-textbook (or from scratch). Hence, they will require appropriate tools to do this in an informed manner. They will also require professional training to develop a sensitivity to mathematical and pedagogical aspects of learning resources and to make effective use of this sensitivity. The design of such tools—both as instructional aids and as vehicles of professional development—is crucial, since it can influence the ways in which teachers’ curricular practices will develop.

In summary, teachers often lack a language to articulate their considerations for the curricular decisions they make and lack tools to make such decisions in an informed manner. The solution we propose, which underlies the research reported herein, is based on tagging, a term that in information systems means assigning metadata to items in order to support semantic searching. With the shift to various models of e-textbooks, it seems natural that authors should tag didactic aspects of their resources; however, this is not currently feasible—there are no standards and no appropriate tools for this. Our development and research seek to investigate an alternate model, where teachers are responsible for tagging. The envisioned usage is: When a teacher finds an internet-based learning resource that she thinks she might want to return to, rather than bookmarking it, she will tag it according to prescribed categories of metadata. The union of resources tagged by a large community of teachers will serve them all in their role as curriculum designers.

A crucial question of design is: What aspects of learning resources should be tagged. This is a delicate issue, since our aim is not only to support teachers’ current practices, but also to support, and even encourage, a transformation in the ways that teachers think and talk about curricular design. The affordance of tools for such a goal has two aspects—as instructional tools and as vehicles of professional development. Though these roles are intertwined, focusing on one or the other has methodological implications. Research on teachers’ use of these tools in authentic instructional contexts is ongoing and will be reported elsewhere. In the current study, we focus on affordances of the tools for professional development of teachers—both practicing and pre-service.

Prior to the research reported herein, we conducted a tagging experiment with seven practicing high school teachers enrolled in a graduate course on uses of technology in teaching mathematics (Cooper and Olsher 2018). This set the stage for the experiment reported herein, where the same seven teachers, a few weeks later, used tagged metadata to search for tasks for particular didactic purposes. This article aims to investigate how a tool for navigating a collection of tagged learning resources—implementing particular categories of didactic metadata—can contribute to the development of teachers’ curricular design.

Theoretical frameworks and research question

Commognitive epistemic stance

Our general epistemic stance is commognitive (Sfard 2008). In this perspective, fields of human knowledge are viewed as patterns of participation in various discourses that are typical of particular communities (teachers, educational researchers, etc.). The notion of discourse is broad, encompassing all facets of communication (speech, gesture, practice, etc.). Particular discourses, such as teachers’ discourse of curricular design—patterns of communication (including action) pertaining to the planning and preparation of curricula—are determined and studied through four facets: typical keywords and the ways in which they are used (e.g., whole-class discussion), visual mediators (e.g., representations of a curriculum), narratives that can be endorsed or rejected (e.g., “it is advisable to begin a new topic with a whole-class discussion”), and recurring routines (e.g., Google search for appropriate instructional resources). Learning in this perspective is conceived as changes in one’s patterns of communication. In this Vygotskian approach, learning is a societal process, beginning through communication with others, and gradually internalized as communication with oneself, hence the welding of the two terms communication and cognition into commognition.

Boundary crossing

Our work with teachers has revealed that their curricular discourse is often quite different from that of educational researchers. Differences may take the form of keywords or visual mediators used by one community and not the other, or used differently across communities, narratives that are endorsed by one community and rejected by the other, or different routines of instructional design. We have framed these differences (Cooper and Olsher 2018) as a boundary between communities (Akkerman and Bakker 2011), defined as “sociocultural differences that give rise to discontinuities in action and interaction” (p. 139). This framework is compatible with our commognitive stance, where the notion of boundary can be reframed as differences in discourse that give rise to commognitive conflict—“seemingly conflicting narratives [that] are originating from different discourses… that differ in their use of words, in the rules of substantiation, and so forth. Such discourses… do not share criteria for deciding whether a given narrative should be endorsed” (Sfard 2008, p. 257). Akkerman and Bakker (2011) have reviewed literature on boundary interactions and have described how learning on boundaries may take place through acts of boundary crossing, conceived as transitions and interactions between the discourses of the involved communities. In discursive terms, this would be conceived as interdiscursive communication—“the use of elements in one discourse and social practice which carry institutional and social meanings from other discourses and social practices” (Candlin and Maley 1997, p. 212). This approach to learning is consistent with Sfard’s definition as changes in an individual’s discourse. It also resonates with how we see the relationship between development, research, and practice; we recognize that the tools and categories of metadata that we provide cannot dictate a particular discourse of curricular design and that any changes in teachers’ discourse will necessarily draw on the perspectives of both communities. Though learning on boundaries usually affects both involved communities, and indeed teachers may contribute to the learning of the research community in many ways, our focus in the current article is on affordances for changes in teachers’ discourse of curricular design.

Mediation of boundary object

Boundary crossing typically takes place through the mediation of boundary objects (Star 1989). Such objects, which may be concrete or abstract, “both inhabit several intersecting worlds and satisfy the informational requirements of each of them…. [They are] both plastic enough to adapt to local needs and the constraints of the several parties employing them, yet robust enough to maintain a common identity across sites” (Star and Griesemer 1989, p. 393). In our context—teachers using tools to construct sequences of learning resources—we recognize three nested boundary objects, which encompass three facets of discourse of curricular design.

Metadata (keywords)

The categories of metadata were chosen to represent the researchers’ perspective on curricular design. These categories are prescribed and are visible in the tagging and browsing tools. As such, they “have a structure that is common enough to make them recognizable across worlds” (ibid., pp. 140–141). Yet they still carry interpretative flexibility (a necessary condition for a boundary object), in the sense that they can be used by teachers differently from the researchers’ intentions.

Tools (visual mediators)

The categories of metadata are embedded in tools, which serve as visual mediators of curricular design. These tools have practical flexibility, in the sense that teachers can pick and choose which of the categories to make use of in their search for tasks.

Assignments (routines)

Finally, teachers may develop routines that they employ in the activity of selecting and sequencing learning resources as required in the professional development assignments.

Research question

In this framing, our research question is: What are the affordances of a tool for navigating a collection of tagged learning resources—implementing particular categories of didactic metadata—for the development of teachers’ discourse of curricular design?

Analytical frameworks

Categories of metadata: supporting two notions of coherence

The categories of didactic metadata implemented in the tagging tool and in the dashboard act as keywords in the teachers’ developing discourse and, as such, serve as an analytical framework. In this section, we describe the main categories of metadata and discuss their inspiration and rationale; however, as our research is conducted in cycles of design/research/redesign, the rationale for the initial design is less interesting than the empirical affordances of the design for learning (and their implications for redesign), which is the focus of this article.

A major challenge of teachers’ work as co-designers of the curriculum is maintaining instructional coherence. Pepin et al. (2015) have elaborated the notion of coherence in the context of producing a textbook and distinguish between two types of coherence. Coherence of the design encompasses aspects such as mathematical correctness, epistemological stance to mathematical topics, sequencing that avoids gaps in the mathematical progression, consistent handling of mathematical objects, and consistency with national curricula. These aspects of coherence are constituted in the textbook’s expositions, its tasks, and ways in which technology is made available to students. Coherence in use is the coherence of what teachers actually propose to their students, drawing on the textbook, or on other curricular material.

To support teachers in maintaining both types of coherence—of-design and in-use—we provide two classes of metadata categories, which we call design categories (i.e., more or less objective, decontextualized aspects of learning resources) and usage categories (i.e., aspects of how the individual tagger envisions the pedagogical use that the resources will be put to in a particular context). Some categories are tagged numerically, on a scale of 0–3 (e.g., the extent to which students are required to make inferences and draw conclusions), and some are tagged categorically (e.g., which of four different class arrangements is the resource most suited to). None of the categories is compulsory for tagging.

Categories of design

Curricular coverage


We follow Schwartz et al. (1995) in our conceptualization of the mathematics curriculum as applying mathematical and general actions in particular mathematical domains (see Fig. 1), applying operations to mathematical objects. The framework of Balanced Assessment (ibid.) was developed by a large research forum that comprised researchers from the University of California at Berkeley (Alan Schoenfeld, PI), Michigan State University, the Shell Mathematics Centre of the University of Nottingham, and the Educational Technology Center of the Harvard University Graduate School of Education. According to this framework, a balanced curriculum should cover all relevant combinations of actions, objects, and operations. This supports a modular approach to curricular design; tagging the mathematical nature of the task and avoiding categories such as grade and difficulty levels imply that resource can be used in different instructional contexts. The categories are:


Actions: Every mathematical activity involves four mathematical and general actions to varying degrees: constructing a model, manipulating mathematical objects, inferring or drawing conclusions, and communicating. Following the initial pilots with teachers, we decided to avoid the category of communication, since teachers tended to see this action in all mathematical activities. It is to some extent replaced by role of technology in explanation (see categories of use below). We added a new action—using technology—which has become relevant since the framework was first proposed. Each of these actions is tagged on a scale of 0–3, indicating the extent to which it is elicited.


Mathematical domains: number/quantity, shape/space, pattern/function, chance/data.


Mathematical objects: The list is long and includes numbers, functions, equations, various geometric objects, etc.


Operations on mathematical objects: vary from object to object and include: analyze (e.g., rate of change of functions and properties of geometric objects), compare (e.g., numbers and functions), construct new objects from old (e.g., adding/subtracting numbers or functions), invariant-preserving manipulation (e.g., solving equations, simplifying expressions, rotating geometric objects), and constructing parametric families of objects (e.g., symbolic representation of a family of lines through a given point).


Representational modality of mathematical objects


Yerushalmy (2006) has demonstrated the importance of linked multimodal representations of mathematical objects (e.g., functions) in interactive learning resources. Each modality (verbal, numeric, symbolic, graphic) is tagged separately on a scale of 0–3.


Types of resource


Include: text, video, mathematical aid, and dynamic sketch. Since resource may include more than one type, each is tagged on a scale of 0–3.


Curricular relevance


Learning resources is tagged on a scale of 0–3, indicating the curricular specificity of a task, i.e., how relevant it is for the requirements of a particular curriculum. Sinclair and Jakiw (2005), who coined this term, have shown that this facet of resources tends to be negatively correlated with a second category—relevant for variety of mathematical topics (what they call mathematical expressivity)—richness of mathematical ideas, representations, and approaches, also tagged on a scale of 0–3. As a result of this correlation, teachers often find it difficult to include “rich” tasks in a prescribed curriculum. The inclusion of both these tagging categories was intended to allow teachers to search for resources that have a high level of both curricular specificity and mathematical expressivity, which might encourage them to take advantage of the richness of mathematical activity that is supported by technology.


Supports heterogeneous classroom


Tagged on a scale of 0–3, was added following the request of teachers who participated in the tagging experiment (see Cooper and Olsher 2018 for details).

Categories of use

Categories of usage indicate contextualized aspects of the use that resources may be put to. Such categories somewhat contradict the modular approach that underlies the categories of curricular coverage, yet teachers felt that such categories can be significant for them.


Curricular role


May be one of the following: opening a topic, practice, homework, assessment, or enrichment.


Class arrangement


May be one of the following: whole class, individual, pairs, or groups.


Role of technology in explanations


When technology is available, teachers may have different expectations of its role in substantiating students’ claims and accordingly may require one of the following: an explanation that relies exclusively on technology, an explanation that makes some use of technology, or a completely non-technological explanation.

Development of teachers’ discourse of curricular design

Of the four learning mechanisms associated with boundary crossing (Akkerman and Bakker 2011), three are relevant in our context.

Coordination

Boundary crossing requires that the participating communities coordinate their different discourses. In our context, this entails translation of keywords, e.g., when teachers use categories of metadata (i.e., keywords from the researchers’ discourse of curricular design) as filters in their search for resources that meet didactical criteria expressed in their own words.

Reflection

This is a mechanism in which participants “come to realize and explicate differences between practices and thus to learn something new about their own and others’ practices” (pp. 144–145). This mechanism takes place through two complementary processes. Perspective making involves “making explicit one’s understanding and knowledge of a particular issue” (ibid., p. 145) and may “reflexively give access to implicit and unstated assumptions” (Boland and Tenkasi 1995, p. 364). For example, using metadata to search for learning resources may encourage teachers to explicate considerations that are tacit in their current practice. Perspective taking involves “taking of the other into account, in light of a reflexive knowledge of one’s own perspective” (ibid., p. 362) and may result in teachers adopting or adapting researchers’ categories of metadata.

Transformation

Ultimately, boundary crossing may entail a transformation in practice, i.e., in the ways in which teachers go about selecting and sequencing instructional activities. This may involve processes of hybridization, whereby “ingredients from different contexts are combined into something new and unfamiliar” (Akkerman and Bakker 2011, p. 148) to create a new cultural form—new tools, signs or concepts (Engeström et al. 1995), or a new practice that stands in between established practices (Konkola et al. 2007). In our context, this refers to ways in which teachers use and modify tools that are provided by the researchers.

Research setting and methodology

We conducted our research with a group of seven experienced high school teachers of mathematics, who were enrolled in a 2-year MEd program. The experiment was conducted in the context of a one-semester course on uses of technology in teaching, which was dedicated to various aspects of tagging didactic metadata. To get acquainted with the categories of metadata, the participating teachers tagged one chapter of an e-textbook (Yerushalmy et al. 2002/2015), after a brief introduction of the framework by the first author. This was followed by a critical discussion (2 h) on practical affordances and challenges of tagging didactic metadata (Cooper and Olsher 2018). Next, we selected a topic that all of the participants teach—trigonometric equations—and each participant searched the internet to find and tag nine tasks to their liking—three that they deemed appropriate for opening the topic, three for practice, and three for assessment. This created a shared collection of 63 tagged tasks. The main focus of the current research is on the assignment that followed, which was intended to approximate instructional practices of sequencing learning activities: working in three small groups (2, 2, and 3 participants), use a dashboard, which supports filtering tasks by metadata, to select three tasks—one for opening the topic, one for practice, and one for assessment. Each group presented their work, describing how they made use of the metadata to find appropriate tasks, and how satisfied they were with the results (~ 25 min each). This was followed by a critical discussion with their peers, facilitated by the first author (~ 15 min each). Finally, each group submitted a written report of their work. The presentations and discussions were audio-recorded, along with video capture of the use of the dashboards, and were fully transcribed and selectively translated by the authors. These transcriptions, along with the written reports, serve as the main source of data.

We conducted a similar experiment with 23 pre-service mathematics teacher candidates enrolled in a teacher-preparation program in our department. The intervention took place during one session (4 h) of a course on processes of teaching mathematics. In preparation for the experiment, three chapters (74 tasks) from a high school precalculus textbook (Yerushalmy 1996) were converted to HTML and were tagged by members of our research team. The chapters were “characterizing and analyzing processes”—20 tasks; “the derivative”—44 tasks; “the integral”—10 tasks. The assignments were a condensed version of the main experiment:

  • Before the lesson, a preparatory assignment: From one of the chapters, select three tasks—one that is suitable (in your opinion) for opening the topic, one that is suitable for practice, and one that is suitable for assessment. Explain your selections—what guided you in choosing a task for each one of the teaching stages? Students worked on this task individually and submitted their work.

  • During the lesson, students were given basic training on the tagging tool and were given some time to acquaint themselves with the categories of metadata by tagging some of the tasks that they had selected in the preparatory assignment. They were then given basic training on using the dashboard to navigate the tasks by metadata and were instructed to repeat the assignment of selecting three tasks, this time making use of the dashboard to browse and filter the tagged collection of tasks. Students worked in pairs, and three pairs presented their work, which was then discussed and critiqued with their peers, coordinated by the authors. The discussion was audio-recorded, transcribed, and selectively translated by the authors.

In both research contexts, the preliminary assignment was intended to approximate teachers’ current practice, where they must select tasks applying a variety of instructional considerations—some of which may be tacit. Repeating the assignment with tools for representing didactic metadata allowed the participants to verbalize and discuss affordances of these tools, which in turn allowed us to analyze affordances of this activity for the development of their discourse of curricular design.

We first analyzed the data from the perspective of our framework of categories of metadata, characterizing ways in which the participants made use of metadata to express their considerations in selecting tasks. We then considered our findings from the perspective of boundary crossing, searching for events of reflection (perspective making and perspective taking) and of hybridization of practice. The three cases of the in-service teachers are described and analyzed comprehensively, while the three cases from the pre-service research are presented selectively, where they contribute additional insight.

We recognize that the observed patterns of the teachers’ participation in the discourse of curricular design may or may not be internalized and sustained over time. Yet, the concern of our current research is affordances of particular tools for the development of teachers’ discourse, and we consider even short-term changes in discourse to indicate such affordances.

Analysis

Our analysis has two parts, corresponding to our two analytical frameworks. We begin with an analysis of teachers’ emerging discourse of curricular design and then proceed to analyze this emergence of discourse as acts of boundary crossing.

Analysis: teachers’ emerging discourse of curricular design

For each of the six groups of teachers, we begin with a schematic description of their routines—how they used the tool to find tasks—analyzing their work through the analytical framework of categories of metadata (two notions of coherence). We note that all of the groups followed a similar routine of applying filters until a small number of tasks remained, reviewing some or all of the remaining tasks, and selecting one of them. Yet there were significant differences in their choice of categories on which to filter, the values on which they filtered, and the order in which they applied filters. Tables summarize the filters that were applied in searching for each of the tasks and indicate the number of tasks that remained after filtering. This is followed by excerpts that describe the rationale for their work. In the next section (Discussion), this analysis is discussed from the perspective of learning through boundary crossing.

Group 1—Farid and Rick (practicing teachers, 3-month intervention)

Role

Filter 1

Filter 2

Filter 3

Tasks left

Opening

Role: Opening

Using technology: 3

Graphic rep.: 3

1

Practice

Role: Practice

Using technology: 3

 

2

Assessment

Role: Assessment

Graphic rep.: 0

Dynamic sketch: 3

1

Farid and Rick began each of their searches by filtering on the keyword role. For example, to find a task for opening a topic, they first limited their search to tasks that had been tagged as opening a topic. They filtered only on extreme values, e.g., when they wanted a task that did or did not make use of graphic representation, they filtered on values 3 (when searching for a task to open a topic) or 0 (when searching for an assessment task). As a result, they required only 2–3 filters to reduce the collection to 1–2 tasks, from which they made their selections. They explained this practice as: “… it was efficient. We arrived [at a short collection of tasks] immediately.” Their decision to search for a dynamic sketch for an assessment task is noteworthy; they explained it as follows: “Assessment is usually individual work, where [the teacher] is not beside [the student], so a dynamic sketch can help. For practice, if he has questions he can ask… if he gets stuck you can direct him… dynamic sketch is for individual work… he doesn’t receive [immediate] feedback from the teacher.” We also note that all three tasks that they eventually selected had been chosen and tagged for the database by Farid himself. They found this very surprising (recall, each of the seven teachers selected and tagged nine of the 63 tasks), since they were not consciously searching for their own tasks, Farid did not have a clear recollection of how exactly he had tagged his nine tasks, and furthermore, the dashboard-based search was co-conducted with Rick, who was unfamiliar with Farid’s tasks. They both felt that the tasks they ultimately found were suitable; in fact, Rick considered Farid’s tasks “beautiful.”

Group 2—Jon and Kim (practicing teachers, 3-month intervention)

Role

Filter 1

Filter 2

Filter 3

Tasks left

Opening

Dynamic sketch: 3

Curricular relevance: 3

Role: Opening

4

Practice

Curricular relevance: 3

Class arrangement: pairs

 

7

Assessment

Curricular relevance: 3

Inferring: 3

 

2

Jon and Kim, like Farid and Rick, were pleased to find that a small number of filters could efficiently reduce the collection to a manageable size: “with two or three fields you’re left with just a few [tasks]. That’s excellent. You filter quickly and you’re left with 2 or 3 tasks, and you don’t need another filter, because it’s enough of a selection… It was efficient.”

They did not rely heavily on the tagged role of tasks, using this category only once (for opening a topic), and then only as the third filter. They sought curricular relevance (level 3) in all of their searches. For opening a topic, they agreed among themselves that it would be appropriate to make use of a dynamic sketch of the unit circle, and chose dynamic sketch (level 3) as their first filter. Not surprisingly, this led them to a task that had been selected and tagged by one of them. For practice, they were not certain whether they would rather arrange the class for individual work or for pair work. They first filtered on individual, but found that this left many tasks (26), which would require further filtering. They decided to select a pair activity instead, which reduced to seven tasks, from which they selected the one that had the word “YouTube” in its title, which they found curious: “We said: who tags a video as practice? It doesn’t seem right.” However, upon reviewing the resource, they came to appreciate that problems solved by a skillful teacher and recorded on video could serve as a kind of practice. They found this to be an enriching experience: “It can change your style of teaching. From a state where you’re set in your ways about how to teach, you see something that someone else thought was suitable, and you say wow. This is very positive, because you want to refresh your instruction, with regard to considerations for selecting tasks.” Later, they elaborated two different orientations to searching for resources: “If you’re set on what you want… then stick to your original plan [e.g. of using a dynamic sketch of the unit circle to open the topic]. But when it came to practice, where we weren’t sure what we wanted… it changed our original thinking… If you know what you want, you’ll disqualify resources quickly. Otherwise, you’ll give them more thought.”

For assessment, they felt that students should be required to make inferences or draw conclusions, and accordingly filtered on this category.

Group 3—Jim, Ada, and Alan (practicing teachers, 3-month intervention)

Role

Filter 1

Filter 2

Filter 3

Filter 4

Filter 5

Tasks left

Opening

Video: 1–3

Supports hetero class: 1–3

Graphic rep.: 1–3

Curricular relevance: 1–3

Class arrange: pairs

1

Practice

Dynamic sketch: 1–2

Symbolic rep.: 1–2

Curricular relevance: 1–2

  

2

Assessment

Analyze: 1–2

Inferring: 1–2

Graphic rep.: 1–2

Using technology: 1–2

Dynamic sketch: 1–2

3

Unlike the others, this group did not value efficiency of the process. On the contrary, “At first we over-filtered. We applied a lot of filters, and reduced quickly to 1 task, or sometimes none, and we found it wasn’t effective. And we weren’t satisfied with the tasks that we found.” They decided to apply longer sequences of filters (3–5), feeling that in this manner they would be able to attend to more didactical aspects of the resources. However, in a collection of 63 tasks, applying five filters may not leave any tasks. Therefore, they were less strict with the tagged value, often filtering on a range of values (e.g., any positive value—between 1 and 3). They felt that this was a worthwhile trade-off—perhaps the tasks they found would not have the relevant characteristics to the greatest extent, yet they would have many desirable characteristics. Furthermore, they were concerned that strict filtering was liable to remove tasks that in retrospect they would consider suitable. In particular, they found that filtering on role was misleading—many of the tasks that were tagged for one role they ultimately considered suitable for other roles as well (false-negative effect). They also felt that the tagged role is not only highly subjective, it is also contextual and interdependent: “It depends on the sequencing. When you consider how you opened the topic, it can influence what you do in practice [activities]… you feel there’s something the opening didn’t cover, and you need to address it in the practice… We wanted the practice to touch on the topic differently [from the opening tasks].”

Of all the groups, they were the most explicit about why they stopped filtering when they were left with 3–4 tasks, proceeding to review them individually. Besides the “false-negative” effect of over-filtering mentioned above, they noted that it is difficult to frame all didactic considerations in terms of metadata and that the “human factor” of individual preferences might be difficult to articulate. Yet the main reason is that they did not see the activity only as a means for selecting suitable tasks, but also as an opportunity to extend their curricular repertoire. “We tried not to reduce [the tasks] to what was natural for us… We wanted to arrive at things that would surprise us,” much as Jon and Kim were surprised by the video tagged as practice. Hence, they made a point of reviewing as many resources as they felt were feasible (3–4). An example of something that surprised them is a video tagged for assessment, which, once reviewed, suggested to them how assessment could be conducted interactively, with a teacher asking questions related to what is shown in the video.

Applying longer sequences of filters, they came to realize that the order in which they apply them is consequential—the first categories that they apply will determine which 3–4 resources they eventually review: “If you do the order badly, you lose tasks that you would have liked to keep and review. We understood that we must prioritize the filters.”

Regarding the particular categories of metadata that this group used: A video resource was considered suitable for opening a topic since it does not rely heavily on prior knowledge; a dynamic sketch was considered appropriate for practicing a familiar topic, with the expectation that it would invite students to make connections between symbolic and graphic representations; assessment should challenge students at higher cognitive levels, hence the filtering on analysis and on inferring.

Like Farid and Rick, this group also ended up locating their own tasks, yet there was a mismatch between the role they tagged and the role for which the task was eventually selected; their opening task, which they described as “an opening task par excellence,” had been tagged by Alan as practice! Nevertheless, they reported that “working as a team, [Alan] had no problem taking it for opening the topic.” For practice they found a task tagged by Jim as assessment, and for assessment they couldn’t decide among two tasks, one tagged by Kim and one by Alan. Like Farid and Rick, they were surprised by this “coincidence” and were particularly surprised by their own flexibility regarding role (selecting for opening the topic a task that Alan himself had tagged as practice).

Group 4—Ruth and Tami (pre-service teachers, 4-hour intervention)

Role

Filter 1

Filter 2

Filter 3

Filter 4

Tasks left

Opening

Graphic rep: 3

Role: Opening

Acceptable explanations: Not only tech

 

2

Practice

Verbal rep: 3

Role: practice

Class arrangement: individual

 

2

Assessment

Inferring: 3

Class arrangement: individual

Graphic rep: 3

Role: assessment

2

Ruth and Tami presented their selection of three tasks from the Derivative chapter of the precalculus textbook. Similarly to Farid and Rick and to Jim, Ada, and Alan, in all three cases they applied filters until two tasks remained. They made consistent use of the representation category of metadata, which they explained as follows: for opening a topic (where they sought graphic representation): “because we thought about the type of students [we teach…], for an opening task […] it’s something they need to see. For these students specifically, it can help if they see it graphically… also because we want to ask the students “why” […] and often the answer comes from the visual representation that they see in front of their eyes. Say derivative, if I show them a graph […] it’ll be easier for them to understand why…”; for practice: “it was important for us that there be verbal representation, a bit more explanation than there had been up to now”; and for assessment: “maybe it’ll be easier for them to apply if we leave them the graphical representation.”

They also made use of the role category in each of their searchers, though not as the first filter. They reached conclusions similar to Jim’s: “at first we filtered according to what others tagged … by role, and we saw that [they were] not necessarily tasks that we would have chosen [for an opening task]. So we said we’d start with our criteria, and take it from there.” They articulated how the activity encouraged them to reflect on their instructional practices: “we were left with a lot of tasks [after filtering on inferring and on class arrangement], so we started to think what else is important for us in assessment.”

Group 5—Saib and Ula (pre-service teachers, 4-h intervention)

Role

Filter 1

Filter 2

Filter 3

Filter 4

Tasks left

Opening

Numeric rep: 3

Operations—transform/build a family: 0

Operations—rate of change: 2

Operations—on/with functions: 0

1

Saib and Ula presented their selection of a single opening task for the Integral chapter. They intentionally filtered only on categories of design: “role of opening task… we didn’t go for that, and we weren’t particularly interested in [the tagging of] class arrangement, though we did want [the task] to be individual or group work, but we didn’t select [that category].” They intentionally filtered until only one task remained: “we were left with two tasks, not many, but we tried to filter further.” They tried to explain their rationale for the categories they used: “[We wanted students to] focus on the function… we tried to connect two things, numerical representation, which is not symbolic like you see on exams and on standard questions… But we still want them to perform mathematical operations [on functions…], it’s connected to the concept of integral, and they should do something.”

Group 6—Jack and Pete (pre-service teachers, 4-h intervention)

Role

Filter 1

Filter 2

Filter 3

Filter 4

Tasks left

Assessment

Verbal rep.: 3

Graphic rep.: 3

Numeric rep.: 2–3

Symbolic rep.: 3

2

Jack and Pete presented their selection of a single assessment task for the Derivative chapter. They applied a sequence of filters that left two tasks, which they then reviewed to select one. They filtered only on the category of representations, feeling that for assessment, all four representations of a function should be present. When asked, they indicated that for opening a topic, they would search for a task that does not include a symbolic representation.

Analysis: teachers’ learning through acts of boundary crossing

Translation of keywords

Using researchers’ categories of metadata to search for learning resources required that the teachers coordinate their discourse of curricular design with the researchers’, translating between their criteria for task selection and available categories of metadata.

Many of the teachers indicated that they were curious to see if their filtering would lead them to familiar tasks that they themselves had initially chosen and tagged. This can be seen as a validation of the translation process, as follows: First, teachers used tacit didactic criteria to choose tasks, and through tagging, coded these criteria in categories of metadata. Now, these same teachers are translating their didactic criteria into metadata (using the dashboard to filter tasks). Will this “double translation” lead them back to their chosen tasks? The fact that it did in two cases out of three (the cases of Farid and Rick, Jim, Ada, and Alan) provides some measure of validation for the usefulness of the categories to express the teachers’ curricular discourse. This is also validated by the fact that many of the teachers were able to express their curricular considerations in terms of the prescribed categories of metadata, as follows:

  • The categories of representations were used by some teachers to differentiate among the various tasks. Rick and Farid used graphic representation to differentiate between opening tasks (where it should be present) and assessment task (where it should be avoided), while Jack and Pete used symbolic representation for the same purpose. Ruth and Tami used verbal representation in their search for a practice task and thought that graphic representation would be appropriate for the kind of sense-making they try to encourage in opening a topic.

  • Categories of the type of resource were used in various ways. Some teachers sought dynamic sketches: Rick and Farid for assessment, seeing in a dynamic sketch a kind of “teacher proxy,” while Jim, Ada, and Alan considered dynamic sketches to be appropriate for practice, for their invitation for students to make connections between symbolic and graphic representations that they expect to be present in dynamic sketches. For opening a topic, they sought a resource that would not require prior knowledge and considered video to be an appropriate type. Kim and Jon used this category for finding an opening task, though in their case they had a clear image in mind of the resource they were looking for (unit circle).

  • In the category of actions, Kim and Jon, and also Jim, Ada, and Alan, considered inferring to be an appropriate filter for assessment.

Nevertheless, we note that there were cases where teachers used keywords in ways different from the researchers’ intentions. For example, we thought of verbal representation (of a function) as a qualitative description, not as “a bit more explanation,” as understood by group 4. We do not see such cases as failure in translation. Rather, the ability to work smoothly across this “boundary,” where teachers can make effective use of tools without fully adopting the researchers’ discourse, is an aspect of the boundary object’s flexibility that allows collaboration without complete consensus between the parties.

Reflection across teachers’ and researchers’ perspectives

Teachers were quite explicit about how the task and the tool encouraged them to reflect on their curricular practices. Ruth and Tami, for example, reported that “we were left with a lot of tasks [after filtering on inferring and on class arrangement], so we started to think what else is important for us in assessment.” Similarly, Jim, Ada, and Alan reported that “we understood that we must prioritize the filters.” This is a case of what Boland and Tenkasi (1995) refer to as perspective making; the possibility of filtering the collection, together with the necessity of deciding which categories of metadata are most relevant for various stages in the curriculum, encouraged the participants to reflect on their curricular considerations, and to be explicit about their own didactics. Furthermore, this setting encouraged boundary crossing, in bringing teachers to articulate “what is important” in terms of the researchers’ categories of metadata. This is what Boland and Tenkasi (1995) have called perspective taking. This aspect of boundary crossing was particularly explicit in the case of Olga and Dave (whose work was not presented in class), who were alone in (mis)interpreting the assignment as using metadata to locate the tasks that they had initially preselected without access to metadata.

To further describe the nature of reflection that was taking place, we recall an aspect of the boundary as it was revealed in the tagging experiment (Cooper and Olsher 2018). Generally speaking, teachers tended to view particular tagged tasks as being suitable for specific points in a linear curriculum, while the researchers tended to see them as modular building blocks, which can be used at different points in a curriculum for different purposes. Jim, Ada, and Alan had the opportunity to reflect on the researchers’ perspective when their search for a task for opening a topic led them to one that Alan himself had tagged as practice, and their search for a practice task led them to a task that Jim had tagged as assessment, strongly suggesting that tasks may be appropriate for various different points in the curriculum. Though our focus in this article is on the learning of the participating teachers, we note our own boundary crossing (on the part of the researchers) regarding this dichotomy. Reflecting on the stance expressed by Jim, Ada, and Alan—“we wanted the practice to touch on the topic differently [from the opening task]”), we are coming to accept that a task’s suitability for practice or for assessment in a particular teacher’s classroom relies not only on its decontextualized didactic characteristics, but also on the particulars of how the learning sequence unfolded, and what kind of tasks the students encountered previously. This has implications for the redesign of our tools; while tagging is conducted at the low level of individual tasks, it might make sense to support searching at a coarser grain size—sequences of tasks that fit together “coherently.”

Hybridization of routines

Of the six groups of teachers, only one (Saib and Ula) used the dashboard to pinpoint a single task. All the others filtered only until a manageable number of tasks remained (usually 3–4), which were then reviewed, and one was selected from among them. We view this as hybridization in the following sense: It makes use of the researchers’ categories of metadata to ensure that the selected tasks will have the desired didactic characteristics, while at the same time it allows teachers to apply a “human factor,” as they are accustomed to doing, which might be difficult to capture in pre-defined categories of metadata. Nevertheless, we note that this emerging hybrid practice did not always draw on both perspectives substantially. For example, Jon and Kim, who did not really care if their practice task was arranged as an individual or a pair activity, eventually filtered on “pairs” simply because this filtering reduced the number of remaining tasks more effectively, and in so doing seem to have been attending to the metadata somewhat superficially.

Many of the teachers reiterated an aspect of their discourse that was prominent in the tagging experiment (ibid.), whereby the efficiency of instructional procedures is of utmost importance, and expressed satisfaction with the fact that 2–3 filters were sufficient to bring the collection of tasks down to size. To this effect, many of the teachers filtered on extreme values (e.g., Farid and Rick consistently filtered on values of 0 or 3). However, Jim, Ada, and Alan took a different view, more consistent with the researchers’ approach, whereby tagging and filtering should be comprehensive. They valued inefficiency in filtering, for its ability to locate tasks that match many of the searcher’s didactic criteria without exhausting the collection, and to this effect often filtered on a range of values (e.g., 1–3).

Another act of hybridization had to do with the objective/subjective nature of tagged metadata. As a rule, teachers tended to agree with the researchers in viewing the categories of design as more objective, while the categories of use were considered to be more contextual and individual (Cooper and Olsher 2018). We expected this realization to lead teachers to avoid categories of use as filters, recognizing them as unreliable, and indeed this was often the case—Jon and Kim relied mainly on categories of design, and almost completely avoided the category of role, Jim, Ada, and Alan actually found that filtering on role would have excluded tasks that they eventually selected, Saib and Ula were also very explicit on their avoidance of role and class arrangement, and Ruth and Tami, for similar reasons, made use of role only after applying more reliable filters. However, an unexpected affordance of use categories emerged; Jon and Kim, and Jim, Ada, and Alan even more explicitly, spoke of the opportunity to learn from other teachers through tagged metadata. By combining categories of design (e.g., video resource) and of use (e.g., role), they gained a glimpse of other teachers’ curricular discourse, e.g., video resources are used by other teachers for practice or for assessment. We have reported (Cooper and Olsher 2018) that some teachers, based on a tagging experiment, were hoping that tagged metadata would contribute to peer-learning, yet their vision of how this might come about would have involved a significant redesign of the tools, to allow taggers to add extensive free-text descriptions of how and why they used tasks as they did. The experiment reported here provided the opportunity to transform this idea into a new practice. Drawing on the researchers’ goal of didactically informed task selection, along with the teachers’ goal of learning from the wisdom of others, a hybrid practice emerged. By searching for surprising combinations of metadata values, without modification to the design of the dashboard, teachers were able to gain new curricular insights from their peers, mediated by the boundary object.

Discussion, conclusions, and ways forward

We have demonstrated that the participants in our research—seven practicing teachers in a graduate course and a class of university students in a teacher-preparation program—were able, under the conditions of the intervention, to make use of our categories of metadata in an interactive dashboard to search for opening, practice, and assessment tasks. From the commognitive perspective, the teachers were learning in the sense that their discourse of curricular design was undergoing a change. The exposure to a new discourse took place through boundary crossing—situations that invited them to engage with the researchers’ discourse of curricular design. They did not always use the metadata or the tools in the manner that we had intended; sometimes categories of metadata were interpreted differently from our intentions, and the considerations that teachers applied for selecting tasks were not always aligned with our perspectives on curricular design. Yet this is to be expected. If teachers in professional development were to blindly adopt researchers’ perspectives, we would have suspected it as being ritualistic participation in a new discourse, which Sfard (2008) defines as participation whose goal is alignment with others and social approval, discourse that it is unlikely to be sustained over time. In contrast, emergent discourse and practice that draw on the perspectives of teachers, as well as those of researchers, seem to be explorative and are more likely to be sustained. We do not claim that the categories of metadata that we have been using are “optimal,” yet they were instrumental in contributing to the development of teachers’ discourse (communication and practice) regarding didactically informed task selection. We now summarize in what ways the metadata categories, the tool, and the activities we provided, supported learning through boundary crossing.

  • Many of the categories of metadata were sufficiently familiar to the teachers to be comprehensible, while at the same time aligned with our perspectives.

  • The dashboard supported the emergence of a hybrid practice of filtering with the goal of reducing a collection of tasks to a manageable size. This allowed participants to make use of new search schemes (filtering on metadata), while retaining the familiar and productive routine of reviewing a small number of tasks and choosing the most appropriate from among them. In particular, this allowed the application of idiosyncratic considerations alongside the prescribed categories of metadata.

  • The activity encouraged participants to make explicit the tacit considerations that they apply to curricular decisions and to reflect on them when attempting to translate their perspectives into the researchers’ language of metadata.

  • Comparing the search criteria across tasks invited reflection on subtle differences among tasks for opening a topic, practice, and assessment.

Our focus in this article has been on affordances for teachers’ professional development. Accordingly, we have focused attention on a scaffolded short-term intervention, which can reveal only initial stages in a learning process. Eventually, the tools we develop should support a sustainable change in teachers’ instructional practices. This poses a practical challenge—how to create a world where a significant collection of learning resources is tagged. One possible model, which has been suggested in this article, is communities of teachers contributing to a shared corpus of collectively tagged resources. Such a model poses a challenge; we have shown an inherent flexibility in interpreting categories of metadata. What is the relevance of tagged metadata, if the tagger and the searcher do not share meanings of the metadata categories? A way out of this conundrum, discussed by Cooper and Olsher (2018), is supporting the emergence of communities of practice and making metadata visible only within communities—possibly teachers in a single school or district, or any group of teachers who come to share meanings of metadata. Yet this is not the only model. Eventually, we envision a standardized set of metadata categories, with meanings that are shared across large communities. When this is achieved, developers will be able to tag their learning resources, or even embed metadata in their resources in a manner that is accessible to standard search engines. In so doing, they will be able to convey their didactic intentions to prospective users, who may, of course, use them in ways that are different from the intentions of the designer. In this approach, as in others, the development of teachers’ discourse of curricular design pertaining to informed selection of tasks will be crucial. We view our current work as a step toward a sustainable solution. In describing the ways in which categories of metadata support the professional development of prospective users, we are laying guidelines for any future initiative to make metadata available at scale, whether by means of standardized categories or by any other means that might emerge.