1 Introduction

Thanks to new digital technology, methods for finding and analysing research literature have become dramatically more powerful over the last decades. In the field of educational science, for example, extensive online literature databases are available that can be searched through sophisticated and fine-grained queries. The results of such queries, which include detailed data on the literature found as well as full texts in many cases, can easily be stored in software for literature management. Such literature management systems, then, offer an excellent starting point for literature analysis. Setting up a literature study in this way nowadays is much less laborious than has been the case for centuries. No wonder that literature review studies have become widespread, also in our field of mathematics education research.

But there is more. Now that bibliographical data are more accessible and better documented than was the case in the past, new bibliometric techniques are developed and applied to the results of literature search queries. Through bibliometric clustering, for example, clusters of related publications in a domain can be identified, and their consistency can be quantified. Amazing new means of literature study are emerging from these and other bibliometric techniques.

However, these bibliometric algorithms do not address anything concerning the topic of interest in the literature study. Therefore, to guarantee the content validity of the literature clusters identified through bibliometric techniques, these clusters should be interpreted and evaluated by domain-specific experts, who know the field of study. In this way, this type of bibliometrics-based literature study gradually gets the character of a man–machine interaction, in which the bibliometric results are submitted to a process of meaning making by human experts, who are familiar with the substance of elements in their field, and who can check whether the biometrical findings make sense.

It is this interplay, which essentially holds for all scientific literature study, that we explore in this paper for the field of mathematics education. For this purpose, we carried out a case study consisting of a literature study based on one of the most important research literature repositories, and the application of advanced bibliometric tools to identify biometrical clusters, complemented by a more qualitative approach, drawing on our own knowledge of the field. As such, our aim in this paper is an exploration of the value of bibliometric techniques for mathematics education research through triangulating bibliometrics and expert findings.

To assess this research method, we considered the paradigmatic case of instrumental orchestration. In the light of the expertise needed for the ‘human touch’ in comparing bibliometric findings with expert views, we chose this topic because of two of the authors’ knowledge of the literature in this field. Also, as researchers involved in the development of this notion (Trouche 2004; Drijvers, Doorman, Boon, Reed, and Gravemeijer 2010; Drijvers and Trouche 2008; Trouche and Drijvers 2010, 2014), we are very interested in finding out how this theoretical view on teachers’ practices developed over time and was used. These two authors from within the field invited as co-author an expert in the field of bibliometrics analysis who contributed to this analysis through the use of specific bibliometric tools (Grauwin and Jensen 2011).

The paper is structured as follows. In Sect. 2, we set out the theoretical background of the study, including notions from bibliometric methods and of instrumental orchestration, and formulate the research questions. In Sect. 3, we describe our methods. Section 4 contains the findings, including the biometrical clustering of literature and expert interpretations. In the final section, we revisit the research questions and identify some future directions for exploring the coordination of algorithmic bibliometric methods and more qualitative human expertise.

2 Theoretical background

The theoretical background of this paper includes two main elements, one from bibliometrics, referring to the methodological innovation we discuss, and one on instrumental orchestration, which is the topic of our case study.

2.1 Notions from bibliometric methods

Bibliometrics, the measurement of all aspects related to the publication and reading of books and documents (Otlet 1934), offers statistical methods to analyse books, articles, and other publications. Bibliometric data include, among other things, descriptive metadata characterizing the source document, such as author, country, institution, keywords, language, publication source, publication year, references cited, references sources and subject category.

Once these data are available, bibliometrics are used to explore relationships among academic journal citations. Usually, this type of citation analysis involves examining how many times the publication under investigation is referred to or cited in other, more recent, documents, as a means to assess the source’s merit or impact. Also, papers can be compared in terms of their co-citation ratio. The limitation of this aspect, of course, is that such an analysis can be carried out only some years after publication. If the goal is to sketch the current landscape of a research field, therefore, another approach needs to be used.

Bibliometric Coupling (BC) is such an approach. The basic idea here is not to consider how and where a specific publication is cited, but to which other publications the publication itself refers. Then, two publications can be compared through Kessler’s (1963) omega measure (ω) for similarity in references: if R1 and R2 are the sets of references of two papers 1 and 2, respectively, and the bars || refer to the number of elements in a set, the ‘reference similarity’ of R1 and R2 is defined as the number of shared references divided by the square roots of the product of the numbers of references in each paper:

$$ \omega_{1,2} = \frac{{|R_{1} \cap R_{2} |}}{{\sqrt {\left| {R_{1} \left| \times \right|R_{2} } \right|} }}. $$

Kessler’s omega provides the relative degree of overlap between the references of each pair of publications. If two publications do not share any reference, then omega equals zero. If they have identical references, the strength of their connection is maximal, and omega equals 1. This BC approach is consistent with the idea of ‘mother resources’: the more papers share references, the more they are considered close. Once the values of omega are calculated for a set of publications, network maps may be made to provide visual output.

Once biometrical coupling has provided a measure of ‘how close each pair of papers is’, one can try to identify clusters within the ‘community of publications’. BC clusters can be identified through community detection algorithms, resulting in a partition of the publications into clusters. Basically, the algorithm groups publications belonging to the same ‘dense’—in terms of links—region of the BC network. The quality of the cluster partitioning can be quantified by its network modularity Q (Girvan and Newman 2004), which reflects, roughly speaking, the number of links within clusters (as opposed to crossing between clusters), minus the expected number of such links if the network were randomly produced. This measure takes values ranging from − 1 to 1, a value of 0 meaning that the links are randomly distributed. The higher Q, the more meaningful the partitioning, the partition being often considered meaningful for values of Q higher than 0.3 or 0.4. We compute the graph partition using the efficient heuristic Louvain algorithm presented by Blondel et al. (2008). Applying the Louvain algorithm yields a partition of the network into clusters (e.g., see Fig. 6). Simple frequency analysis then allows the characterization of each cluster though its more frequent items (keywords, author, etc.). The significativity σ of the presence of a given item into a cluster is computed by comparing its frequency f in the cluster to its frequency f0 within the whole corpus. More precisely, we use the normalized deviation definition:

$$ \sigma = \sqrt N \frac{{f - f_{0} }}{{\sqrt {f_{0} \left( {1 - f_{0} } \right)} }}. $$

The value of σ can be computed for each of the cluster members. In this way, the ‘most significative’ cluster element is identified for each of the clusters. Also, a cluster’s most representative paper can be identified, i.e., the paper that is most closely aligned with the core content of the cluster. It is defined as the paper with the highest in-degree, where the in-degree of a paper is defined as the number of papers within the cluster it is connected to by shared references; Table 4 provides examples.

To go further into the details of biometrical clustering and the quality of the resulting partition is beyond the scope of this paper. As is the case for literature review study (see Sect. 1), however, bibliometric analyses have clearly become much more efficient and accessible through the availability of digital technology. In addition to automatically calculating values of similarity and clusters, for example, cleaning up reference data can be automatized to a large extent. As a consequence, bibliometrics has much to offer educational research. As its use in mathematics education so far has been limited, one of the aims of this paper is to explore the value of its application in this domain; a value that, we expect, might be considerable.

2.2 Instrumental orchestration

Since the 1990s, powerful digital technology has entered the mathematics classroom, and teachers have been confronted with the question how to integrate it into their teaching. It soon became clear that learning mathematics with and through technology is not self-evident and requires a process of instrumental genesis, i.e., the subtle processes of appropriating digital tools for teaching mathematics. Such appropriation includes both adopting these tools (i.e., developing new teaching techniques) and adapting them to a teacher’s own needs and habits. A rethinking and re-arranging of traditional teaching formats is needed. It is in this context that the notion of Instrumental Orchestration (IO) emerged. To quote the title of one of its sources, IO is a framework for “managing the complexity of human/machine interactions in computerized learning” from the perspective of a teacher (Trouche 2004).

Instrumental orchestration was mentioned for the first time in 2002 already, in a special issue of ZDM (volume 34, issue 5) dedicated to the integration of Computer Algebra Systems in mathematics education. Guin and Trouche (2002) defined an instrumental orchestration as follows:

We will call instrumental orchestration a plan of action, partaking in a didactic exploitation system which an institution (the school institution, in this case) organizes with the view of guiding students’ instrumented action. Instrumented orchestration is defined by four components:

  • A set of individuals;

  • A set of objectives (related to the achievement of a type of task or the arrangement of a work-environment);

  • A didactic configuration (that is to say a general structure of the plan of action);

  • A set of exploitation of this configuration. (p. 208)

What we retain from this definition is the a priori feature of an orchestration as the systematic arrangement by an intentional agent. Furthermore, the notion of IO initially had a theoretical character: it emerged as a modelling of the teaching process in a technological environment, involving artefacts and humans. To add an ad-hoc perspective to the model’s main elements of didactic configuration and exploitation modes, and to highlight that an instrumental orchestration is a living entity rather than something a teacher prepares beforehand, Drijvers et al. (2010) added didactical performance to the model of didactic configuration and exploitation mode as a third layer, comparable with the musical performance in the metaphor of instrumentation:

A didactical performance involves the ad hoc decisions taken while teaching on how to actually perform in the chosen didactic configuration and exploitation mode: what question to pose now, how to do justice to (or to set aside) any particular student input, how to deal with an unexpected aspect of the mathematical task or the technological tool, or other emerging goals (p. 215)

Initially, the so-called Sherpa student orchestration (Guin and Trouche 2002) acted as the exemplary one. In this orchestration, a student operates a device, which is visible to the whole class, under the guidance of the teacher. Indeed, it evidences the different ways a teacher may take into account students’ artefacts for implementing a given mathematical situation. Gradually, the repertoire of instrumental orchestrations, and didactical configurations in particular, has been extended. For example, Fig. 1 provides an overview of orchestration types observed in whole class teaching and in supporting students during their individual interactions with digital technology (Drijvers, Tacoma, Besamusca, Doorman, and Boon 2013). These types of reports opened the view on a wide range of possible applications of the instrumental orchestration framework.

Fig. 1
figure 1

Types of whole-class and individual orchestrations (Drijvers et al. 2013, p. 998)

Full size image

Gradually, the notion of instrumental orchestration became more widespread outside the French CAS research community. It was also applied to other types of digital environments, including spreadsheets (Haspekian 2014) and dynamic geometry systems (Erfjord 2011), and was compared and confronted with other frameworks, such as the notion of webbing used in the UK (Trouche and Drijvers 2014). In the meantime, teachers’ roles in technology-rich mathematics education have received other attention, not using the instrumental orchestration framework (e.g., see Dillenbourg and Jermann 2010; Ruthven 2014; Stein, Engle, Smith, and Hughes 2008). This raises the question of how the notion of IO ‘travelled through’ the world of mathematics education research, how it was used, and how it was adapted. To investigate this journey through the use of bibliometrics, therefore, is the goal of the paper.

2.3 Research questions

In line with our goal of exploring the value of bibliometrics for research in mathematics education for the case of instrumental orchestration, we drew up the following research questions.

  1. 1.

    How can recent bibliometric methods be used in mathematics education research?

    As we applied this overall methodological approach to the paradigmatic case of instrumental orchestration, the second research question concerns the outcomes of this application.

  2. 2.

    How did the notion of instrumental orchestration develop over time and how was it used in research practices in the field of mathematics education?

3 Methods

The study’s methods included four steps, (1) a preliminary, explorative step, (2) a systematic identification of different corpora, (3) the application of bibliographic tools, and (4) the domain expert interpretation of these findings. In the first step, we explored the number of hits for the query ‘instrumental orchestration’ or its French translation, ‘orchestration instrumentale’, in full texts available on Google Scholar (672 hits, August 21, 2019) and on Academia (9854 hits, August 1, 2019). Google Scholar made it clear that two publications seem central in the field: Trouche (2004) introduces the notion of instrumental orchestration, which is further developed in Drijvers et al. (2010). Figure 2 shows the number of citations over time, as presented in Google Scholar, and shows how these citation numbers are not yet decreasing at this point in time. This suggests the notion of IO is still ‘alive’ in the mathematics education research community. Even if Scholar and Academia provided us with an impression of the work done in this field, the literature corpus is very heterogeneous, and many entries were not described in enough detail to act as input for using bibliometric tools.

Fig. 2
figure 2

Distribution over time of the 511 citations of the Trouche (2004) paper (above) and the 298 citations of Drijvers et al. (2010) (below) as provided by Google Scholar (July 30, 2019)

Full size image

As a second step in our study, therefore, we started a more systematic literature review through the same query, including both journal papers and book chapters, applied to the educational databases Educational Research Complete, Eric, PsychInfo, Scopus, and Web of Science. This search led to 12, 13, 11, 189, and 8 hits, respectively. For several reasons, we decided to focus on Scopus: first, the corpus found in Scopus was drastically larger than that of the other databases and included the other databases’ results. Second, the bibliometric tools need metadata that, for example, were not all available in Eric. In Scopus, we set out a two-phase procedure of selecting a corpus of ‘core’ publications, and building an extended corpus by adding all publications citing the core publications. For the core corpus, we limited the query in Scopus to entries’ titles, abstracts and key words. Initially, this led to 22 publications (August 27, 2019). A quick scan led to the deletion of two items that were out of scope, and one that appeared non-existing, even if Scopus provided the title, abstract et cetera copied from another existing paper. This way, we retained a core corpus of 19 publications (see Sect. 4.1). In the next phase, we selected in Scopus all papers referring to at least one of these 19 core publications through the ‘View Cited By’ option. We found 234 items (August 27, 2019). After cross checking the papers—which is doable for such a number of publications—we decided to include them all, even though the relationship with instrumental orchestration was tenuous in some cases. This corpus is called the Layer-1 extended corpus, or simply the extended corpus. Of course, one can iterate this procedure, by collecting papers that cite Layer-1 papers, resulting in the so-called Layer-2 corpus (1017 items), and a next iteration in the Layer-3 of 4224 publications. Through these iterations, the corpus size increased, and the relationship to the original notion of Instrumental Orchestration became less evident. For this reason, we decided to focus our analysis on the core corpus and the extended corpus.

In the study’s third step we aimed to get a more structured insight of the literature through the use of the bibliometrics toolsFootnote 1 designed by one of the authors (Grauwin and Jensen 2011), which we applied to both the core literature corpus and the extended corpus. The bibliometric tools identified author, country, institution, keyword, language, publication source, publication year, references cited, references source, subject category and subject subcategory as metadata characterizing the corpus papers. The references data were cleaned up semi-automatically. For example, we found more than 40 name variants for “ZDM Mathematics Education” (such as “ZDM”, “Zentralblatt fur Didaktik der Mathematik”, “ZDM Math Educ”, etc.) in the Scopus metadata, which were all set to the same full title. After doing so, the method of Bibliographic Coupling was applied. Kessler’s (1963) measure omega was calculated for each pair of items in the core corpus, and likewise for the extended corpus. As a next step, BC clusters were identified. Each cluster’s description includes the following: the number of publications, publication years, most frequent keywords, categories, journal sources, author countries, reference, reference source, most representative paper, and most cited papers and authors. Network graphs were made for the data and the clusters. For the core corpus, this clustering procedure led to the identification of three clusters, and for the extended corpus, after deleting two papers that did not share any reference with the other papers of the corpus, we ended up with five. The values of the quality measure Q of the cluster partitioning were Q = 0.22 for the core corpus, and Q = 0.18 for the extended corpus. Both values can be considered weak but may be explained by the relatively small number of literature items involved. This implies that running the partition algorithm several times might result in slightly different partitions. However, some repeated applications of the clustering to slightly different core literature sets led to similar clusters, which is reassuring, and provided the domain experts with a suitable canvas to describe the landscape of the research being carried out in the field of interest, in this case Instrumental Orchestration. Still, care was needed in interpreting the meaning of the clusters.

The fourth and final phase concerned setting up these ‘interpretations with care’. This is where the expertise of the authors came in—which was the main reason for choosing the topic of instrumental orchestration. The challenge to the experts was to direct the automated results of the quantitative method towards a deeper interpretation and understanding of the literature. For this purpose, a first step was to label each cluster as a means of identifying its meaning. The software itself provides different means to label the clusters: the most frequent or significant authors, keywords, title words, journal sources, references or categories may be used as a label, or the most significant item according to the definition given in Sect. 2.1. Each of these labels adds to a cluster’s identity, but is not always useful, nor does it tell the whole story. For example, for the three core corpus clusters, “the most frequent authors’ keyword”, led to each cluster having the same label “instrumental orchestration”, which is neither surprising nor informative. The choice of “the most significant keyword” led to the labels mathematical instrument; classroom teaching practice; and instrumentalization, for clusters 1, 2 and 3, respectively. This example shows that identifying the meaning of each cluster and its common key aspects in the light of the research questions was a delicate task for the experts. For a small corpus, such as our 19-item core corpus, such an analysis is relatively easy. For a greater number of papers, such as the extended corpus, the analysis is more complex, and a method to deal with the bibliometrics tools was needed. The method we chose consisted of the two authors with a mathematics education background independently studying the bibliometric data for each of the clusters through the following activities:

  • Studying the most significant keywords and the most representative papers;

  • Comparing this information with the other information given by the software, to try to make sense of each cluster, looking for surprising or contradicting information, and considering the most frequently cited articles;

  • Looking for the papers recognized in the field, to investigate the cluster content itself;

  • Comparing this with our own knowledge of the trends of the field.

After this, we put together the results of this analysis, and compared and discussed them until we agreed on a characterization of each cluster in the form of a cluster label and a one-phrase description. In this method, our independent characterizations of the five clusters matched very well and were close to the most significant keyword. After doing so, we encapsulated our results in the form of so-called ‘identity cards’ for each of the clusters, which embodied the interpretations that we, as researchers involved in the field, attached to the bibliometric results.

4 Results of the bibliometric literature study

In this section, we first describe the core literature corpus of 19 publications and the extended corpus of 234 publications. Also, we compare the two to identify trends over time, taking into account that the extended corpus is more recent than the core corpus, as the extended corpus contains papers citing core corpus papers (Sect. 4.1). Next, we present the results of the clustering technique for these two corpora, including our interpretations (Sect. 4.2).

4.1 A comparative description of the core corpus and the extended corpus

The corpus of core publications consisted of 19 items, which are marked by (*) in this paper’s reference list. The extended corpus included 234 publications. As a first global comparison of the two corpora, Table 1 provides the top-3 ratings of the subject categories and subcategories to which its members belong, according to the Scopus indices. As is manifest in the sum of the percentages, papers can be in more than one category. The two corpora are comparable in terms of subject categories and subcategories: the papers on IO are categorized in the fields of social sciences, education, mathematics, and computer science. The latter is somewhat surprising, as our impression is that the corpora focus on the use of digital technology in mathematics education rather than on computer science; apparently, this is considered a computer science application.

Table 1 Top-3 ratings of subject categories and subcategories of the core corpus and the extended corpus
Full size table

As a second way to describe and compare the two corpora, Fig. 3 shows the publication years of the items in each of them. The fact that the Layer-1 publications are more recent than the core corpus papers, of course, follows from that fact that the first refer to the latter, so are necessarily later. The left graph on the core corpus suggests three periods in which instrumental orchestration core publications appeared. The first period might reflect the introduction of the notion of instrumental orchestration, the second one its appropriation and development, and the third one its further extension and dissemination.

Fig. 3
figure 3

Publication years of items of the core corpus (left) and the extended corpus (right)

Full size image

To reflect the geographical spread of the notion of IO, Fig. 4 shows the distribution of the residence countries of the authors represented in the core corpus and the extended corpus. In the core corpus, authors from France are very present, and the geographical distribution is limited. In the Layer-1 corpus, there are more countries represented, even if South America, Asia and Africa are not-at-all or hardly represented. This may be due partly to the overall capacity of the mathematics education research community in these continents, but other factors might play a role too, such as the dominating role of the English language and the limited access to journals included in Scopus in some countries. For example, Brazil, Mexico and China are absent, while we know from our experience that IO is studied in these countries.

Fig. 4
figure 4

Residence country of authors form the core corpus (left) and the extended corpus (right)

Full size image

As a next step, Table 2 shows the three most cited papers in both the core corpus and the extended corpus. These three already provide an impression of the field. The first one, by Artigue (2002), was influential in introducing the notion of instrumentation and instrumental genesis. As such, it set the scene for instrumental orchestration. The second one, chronologically, was the paper by Trouche (2004) in which the notion of instrumental orchestration was introduced and discussed in detail. The most recent one, by Drijvers et al. (2010), contributed to building up a repertoire of observed instrumental orchestrations and further developed the model. Not surprisingly, these three papers are cited relatively less frequently in the extended corpus than in the core corpus: in most publications in the core corpus, IO plays a more central role than in many publications within the extended corpus. As an aside, if we compare the citation figures in Scopus to the ones provided by Google Scholar (see Fig. 2), we notice remarkable differences: 112 vs. 511 for Trouche (2004), and 76 vs. 298 for Drijvers et al. (2010). Apparently, Scopus contains only a fraction of all publications referring to these two papers.

Table 2 Citation data for the three most cited papers in the core corpus and the extended corpus
Full size table

As a final step in the description of the corpora, Fig. 5 represents the number of papers each of the authors wrote in a Wordle graphic for each of the four successive corpora. The figure clearly shows how the number of authors grows over time, suggesting that the notion of instrumental orchestration was disseminated further. In the meantime, the initial authors became less central over time. Our interpretation is that in many of the publications in Layers 2 and 3, instrumental orchestration played only an indirect role, through referring to a paper that referred to a core corpus paper. This phenomenon was one of the reasons for focusing the analysis on the core corpus and the extended corpus.

Fig. 5
figure 5

Wordle representing the authors’ frequency in the core corpus (top left), the Layer-1 extended corpus (top right), Layer-2 (bottom left) and Layer-3 (bottom right) corpora

Full size image

To summarize the results of the descriptive literature study in the light of the research questions, the development of IO over time can be characterized by the word diversification: diversification from algebra/calculus with CAS to a variety of topics and tools, a diversification from the classroom to distance learning (without addressing Moocs). In the meanwhile, IO remains based in mathematics education, and in some parts of the world. Concerning the way IO is used, the focus seems to be on a growing repertoire of didactical configurations and exploitation modes, whereas didactical performance is hardly addressed.

4.2 Results from the bibliographic coupling clustering

This section contains the results of the bibliographic coupling clustering for both the core and the extended corpus, visualized in network graphs. For each of the two, we present our interpretations of these clusters in the form of ‘identity cards’.

4.2.1 Identifying the clusters of the core corpus

The BC cluster network for the core corpus led to three clusters. As each cluster integrates a small number of papers, the identity cards shown in Table 3 are structured according to just a few features, namely, the dates of the design of these papers, the authors and years of the papers themselves, and their top-3 references. The cluster numbers have been assigned to reflect chronology.

Table 3 Cluster description of the three core corpus clusters
Full size table

Our knowledge of the field led us to label the three clusters according to the three phases in the genesis of the notion of IO described in Sect. 2:

  • Cluster 1 contains papers introducing the notion of IO. Its top references reveal the roots of IO, as follows: the idea of computational transposition (Balacheff 1994), the integration and viability of digital tools (Chevallard 1992), and the issues of CAS integration (Guin and Trouche 1998). These aspects are also coherent with the label “mathematical instrument” given by the software;

  • Cluster 2 contains papers developing the notion of IO, including the notion of didactical performance. Its top references address the issues of creating meanings (Doerr 2000) and of techniques for using digital tools in mathematics teaching, which is coherent with the label of “classroom teaching practice” suggested by the software;

  • Cluster 3 contains papers using the notion of IO in various contexts of mathematics teaching. Its top references are the core references in the field of instrumentation and IO. Using IO in the classroom often leads to adapting the digital tools to a given didactical goal, which is coherent with the label of “instrumentalisation” given by the software.

Finally, the different labels given by the software propose several features for each cluster, informing the researcher who seeks to aggregate these features into an integrated label, using his/her own knowledge of the field. This integrative work leads us to attribute to the three clusters the labels introducing IO, developing IO, and using IO. Figure 6 depicts this cluster structure, with node sizes that are proportional to the number of publications included in it, and line thickness that is proportional to the average similarity (in terms of shared references) between publications from two linked clusters.

Fig. 6
figure 6

The network obtained by applying the BC method to the core corpus (N = 19) and using the knowledge of the field

Full size image

4.2.2 Identifying the clusters of the extended corpus

As indicated in the “Methods” section, we excluded two papers of the extended corpus that did not share any reference with the other papers of the corpus from the analysis presented here. After this exclusion, the BC cluster network for the extended corpus led to five clusters. It is impossible, of course, to give the whole list of papers included in each cluster. To compensate for this omission, Table 4 shows more features than Table 3. The cluster numbers have been assigned to reflect chronology. In this Table, the results provided by the software are printed in black, whereas our interpretations and inferences are displayed in grey. To summarize, Fig. 7 shows the BC cluster network for the extended corpus, consisting of five clusters with the labels we assigned, and the number of items included.

Table 4 Cluster description of the five extended corpus clusters
Full size table
Fig. 7
figure 7

The network of five clusters resulting from application of the BC method to the extended corpus (N = 232)

Full size image

4.2.3 Further interpretations ad reflections

The findings presented in the previous section, and on the five bibliometric clusters identified in the extended corpus in particular, led the two authors from the field of mathematics education to further reflect on them and to connect them to their own research experience.

Related to cluster 1, labelled Managing Teaching Complexity, we mention the issues in crossing language borders, as we experienced them in the cases of Arabic, Chinese, Portuguese, and Spanish languages. Facing the challenge of translating the name of a concept, particularly in the case of a metaphor like instrumental orchestration, one needs to cross cultural boundaries. This reflects the underlying complexity of the concept (e.g., see Fig. 8), and, as such, is an opportunity to deepen the concept of IO (Wang, Salinas, and Trouche 2019).

Fig. 8
figure 8

Thinking of the possible Chinese translations of IO (Shao, in preperation)

Full size image

Both cluster 1 and cluster 3 (labelled Teaching with Technology) stress the design of instrumental orchestrations. What should not be neglected here are teachers’ and students’ gestures. We expect orchestrations targeting “embodied instrumentation” (Drijvers 2019) to be potentially powerful, and to deserve further investigation. Also related to the design of IOs is the need to chain them into coherent learning trajectories (Drijvers, Gitirana, Monaghan, & Okumus 2019), and the wish to focus on student-centred orchestrations, rather than on the teacher being at the centre. What is crucial, though, is to elaborate on the teacher’s didactical performance, an element of instrumental orchestration that has been somewhat neglected so far.

Cluster 2, labelled Designing Living Resources, also matches experiences concerning teachers’ resources outside the field of mathematics education. For example, the work by Gourlet (2018) in primary school convincingly shows the variety of artefacts that come into play (both available and under construction), and the subtlety of the variables of IO, based on the teacher’s positions and gestures (Fig. 9).

Fig. 9
figure 9

A variety of digital and non-digital artefacts to take into account the critical role of a teacher’s positions and gestures (Gourlet, 2018)

Full size image

Cluster 3 suggests new developments of IO, due to the metamorphosis of teaching environment (e.g., the MOOC case, with a diversity of actors, designers, teachers, tutors: see Panero, Aldon, Trgalová, and Trouche 2017), new methodological tools, such as agile methods for design (Trouche, in press), and the extension to other educational contexts, such as universities (Orosco 2019).

Looking through the lens of cluster 4 on Adult Learners led us to consider the issue of teacher education and teacher professional development, i.e., how to support teachers in designing and using IO. This lens led Lucena (2018) to introduce the notion of meta-orchestration, i.e., orchestrating the teachers’ geneses of IO in the context of teacher education. We believe the IO model would be a very powerful starting point for a lesson-study type of professional development. Together, teachers can design and pilot instrumental orchestrations, or ‘techniques’, if one would like to simplify the vocabulary.

Thinking through the lens of cluster 5 on Interacting with Computers led us to reflect on the mutual interaction between mathematics education and software engineering (Bellemain and Trouche 2016): there is a dialectic between what digital technology allows, from a mathematical point of view, and what types of orchestrations it invites. The notion of IO, for example, might inspire software designers to pay more attention to communication opportunities to facilitate different types of orchestrations.

5 Conclusion and discussion

In this study, we set out to explore the value of bibliometric techniques for mathematics education research through triangulating bibliometrics and expert findings for the case of instrumental orchestration. We addressed the questions of (1) how recent bibliometric methods can be used in mathematics education research, and (2) how the notion of instrumental orchestration developed over time and was used in research practices in the field of mathematics education. To answer them, we carried out a literature study, including the use of bibliometric tools, and interpreted the results from an expert research experience perspective.

In answering the first research question, we would first like to express our appreciation for the BC clustering technique, which enabled us to have a more detailed, structured, and insightful look into the literature corpus, and, as a consequence, to study the field under consideration. To us as researchers, the method of a systematic literature search with the application of bibliometric tools combined with a more qualitative and expert experience-based approach was interesting and challenging. We conclude that the BC clustering provided a valuable and sense-making sketch of the ‘landscape’ of the topic under study, in this case instrumental orchestration. Concerning the ‘how’ in the research question, the method we developed for triangulating the bibliometric findings with expert interpretations, resulting in compact ‘identity cards’, proved valuable and seems applicable to other fields of study.

As for the second research question on the development of the notion of IO over time, the core corpus clusters revealed the general trend, from the introduction of the notion, via its further development in terms of extending the model and extending the repertoire of types of orchestrations, to its further use in different educational contexts. For the extended corpus, the five clusters confirmed and further nuanced this global picture. The first cluster, Managing Teaching Complexity, sets the scene for the introduction of IO. The cluster represents the initial exploration of the opportunities of IO as a means to deal with the new challenges digital technology offers to teachers. The second cluster, Designing Living Resources, focuses on a further development of the notion of IO, as part of the teachers’ process of dealing with all the available resources for teaching and with her own processes of documentational genesis. The third cluster, Teaching with Technology, takes into account a more practice-oriented stance and focuses on ways in which teachers can benefit from the notion of IO in their teaching, as well as in its preparation and the reflection on it. Clusters 4 and 5 are smaller in size and show how the notion of IO has been taken into account in broader contexts, such as the context of adult learning, and the context of learning with digital technology in general.

The results also shed light on the different ways in which IO has been used in research practices, which was also part of the second research question. In cluster 2 on the Design of Living Resources, IO forms the basis of the more encompassing theoretical lens of documentational genesis (Gueudet and Trouche 2009), and as such acts as a theoretical building block in the frame of the whole landscape of resources of different natures. In cluster 3 on Teaching with Technology, on the contrary, IO is used in a very practical way to inform teachers on possible ways to integrate digital technology in their teaching, and offers a framework to reflect on what they do in class, as well as how to set the scene for these teaching goals. Clearly, in this progression gradually the role of the teacher came more into view, focusing on what she is really doing rather than prescribing what she should do, and acknowledging the importance of the teacher’s preparation before, and reflection after, teaching. The clusters 4 and 5 illustrate how the notion of IO has travelled to other contexts than just mathematics education, and also may play a role in research on broader topics such as adult education and ICT-rich education.

Before we discuss these conclusions, we briefly address the study’s limitations. Our literature study might be biased by neglecting non-English publications and publications in less prestigious journals. For example, a search in Academia leads to 2524 hits for ‘orquestación instrumental’, the Spanish expression, which were not included in our study. Including other languages might be an interesting next step. As a second limitation, we struggled with the methods of bringing in our personal research experience. As a solution, we tried to report on our explorative approach as candidly as possible (see Sect. 3), but improvements on its methodological rigor are needed. This said, we do believe in the power of this human–machine interaction.

To start the discussion on the findings of this paper, we notice that the bibliometric clustering techniques proved to be quite powerful and helpful, and inspired the domain experts to synthesize the field in a way that would not have been possible otherwise. Just accepting the quantitative results delivered by the bibliometric techniques would lead to less valid results, as would be the case for only expert interpretations. Thus, we feel the method described here to integrate machine power and human expertise may be applicable to other topics in the field of mathematics education research.

On the case of instrumental orchestration, the results on the development and use of the notion of IO we found are quite common in our research field: theoretical elaboration, practical elaboration, widening the scope of applications, these all seem natural for a sensible theoretical notion. In fact, it would be interesting to investigate in future research whether this method would lead to a similar pattern for other topics in our field. For the current case of instrumental orchestration, each of the clusters suggests a direction for further developments and deepening the notion. Let us briefly address the two directions we consider most relevant.

First, the widening range of contexts in which researchers refer to the notion of instrumental orchestration suggests that it might be interesting to be less tightly connected to the instrumental genesis framework, and to consider orchestration in general. By doing so, the differences and similarities with other views on orchestration (e.g., Dillenbourg and Jermann 2010; Stein, Engle, Smith, & Hughes 2008) might be a productive way to further develop the notion.

Second, the notion of instrumental orchestration might be a very suitable starting point for teachers’ professional development. How about teachers, in a lesson-study like setting, who collaboratively design (chains of) orchestrations, that might even be called co-orchestrations, not only because of the co-design, but also because the actual performance is determined by both the teacher and the students? We would be very interested to see the impact of such a professional development approach being investigated.