Introduction

Although Evidence-Based Instructional Practices (EBIP) generally improve students’ performance or persistence in STEM, lecturing remains the most common instructional practice in postsecondary settings (Freeman et al. 2014). As education researchers develop EBIP, they frequently find that efforts to disseminate their EBIP to other faculty are met with frustration; education research frequently fails to cross the so-called research-to-practice “valley of death” or propagate from faculty to faculty (Henderson et al. 2011; Moldonado 2011; Spalter-Roth et al. 2011). Such failures have provoked researchers to study the barriers to change and identify effective strategies for creating systemic change in STEM post-secondary education (Borrego et al. 2013a; Henderson et al. 2011; Kezar et al. 2015). How can we effectively spread the use of EBIP from isolated pockets of change initiators to STEM faculty at large?

At Midwestern University, we have created two distinct, but philosophically linked programs to create institutional change in instructional practices in STEM. The first program (Program I) is based in the College of Engineering and was conceived as a mechanism to improve faculty instruction in large-enrollment (>200 students per semester) courses across the college. The second program (Program II) is a spin-off of Program I that is STEM-inclusive, marshaling faculty from multiple colleges to stimulate the use of EBIP across STEM. Drawing from the literature review by Henderson et al. (2011), the goal of these programs was to create a supportive environment that would enable groups of faculty to emergently adopt EBIP. Both programs sought to create Communities of Practice (CoPs; Wenger 1998) among STEM faculty who were jointly responsible for improving a course or set of courses. These CoPs were given mentors: advocates for improving the quality of teaching on our campus. As we tracked which EBIP the CoPs were adopting, we observed that CoPs that shared a common mentor began adopting similar EBIP (Herman et al. 2015).

Therefore, we present a social network analysis (SNA) of Programs I/II to investigate interactions among its faculty members. SNA reveals that an individual’s performance and their access to information and use of practices can be predicted by the characteristics of their social network and the organizational structure around them (Borgatti et al. 2009; Daly 2010; Feld 1981; Quardokus and Henderson 2015). Although there are many ways to characterize relationships in a social network, guided by structural holes theory (Burt 1992), we focus on two types of interpersonal relationships: strong ties and bridging ties. Strong ties refer to direct connections between two people, usually developed over an extended history of trust and interaction (Granovetter 1973). Bridging ties represent indirect connections—two individuals may not connect to each other directly but can influence each other through a third person who has connection with both of them (Burt 1992). Prior research suggests that strong ties are generally better for deep learning and creating institutionalized knowledge; while bridging ties can span structural holes in a social network and consequently induce innovation, creativity, and the spread of new ideas in the network (Burt 2004; Kezar 2014a).

In this paper, we continue our examination of whether the social network structures of Programs I/II align with the social network structures posited by structural holes theory. In prior studies, we used the Classroom Observation Protocol for Undergraduate STEM (Smith et al. 2013) to document that faculty in Program I/II CoPs were statistically significantly more likely to spend time engaging their students in active learning during lectures than their peers (Tomkin, et al. 2018). Using SNA, we examined the social network structures of individual CoPs. We found that the faculty CoPs in Programs I/II that began to use EBIP had strong ties in their teaching networks, whereas CoPs that did not use EBIP did not have these types of ties (Ma et al. 2016; Ma et al. 2018). These results suggest that strong ties were important for supporting the adoption of EBIP by CoPs; however, these studies did not explore how the CoPs learned about EBIP or how EBIP spread between CoPs. In this study, we examine the social network of Programs I/II holistically, to examine the interconnections between all participants to test our hypothesis that the Program I/II mentors provide the bridging ties that connect the disparate CoPs, potentially creating pathways for knowledge and practices to spread.

Background

To investigate the role of mentors in Program I/II, we present the background of our change effort from the lens of Communities of Practice, the learning theory that guided the design of Programs I/II, and social network analysis. We first describe Programs I/II in greater detail, and then provide brief overviews of the literature on CoPs and SNA.

Structure of Programs I/II

In a review of change efforts to promote the use of EBIP among STEM faculty, Henderson et al. (2011) propose that these efforts vary along two axes: final condition and structural focus.Footnote 1 Final condition indicates whether the change is prescribed or emergent, while structural focus refers to individualistic or environmental change (Henderson et al. 2011). Prescribed, individualistic change efforts often follow a develop-disseminate model where an innovator develops an EBIP and attempts to get others to use the same EBIP using dissemination pathways such as publications and workshops (Borrego and Henderson 2014). In contrast, an emergent, individualistic change strategy may provide training to help faculty members become reflective practitioners capable of identifying which EBIP would be appropriate for their own classroom. Prescribed, environmental change efforts may take the form of instituting policy requiring all faculty teach using a specific EBIP. Emergent, environmental change efforts instead focus on empowering stakeholders in an organization to change environmental features such as climate, culture, or routines to facilitate the spread of EBIP (e.g., Wieman et al. 2010).

STEM faculty have generally focused on develop-disseminate change strategies when promoting the adoption of EBIP (Henderson et al. 2011). These strategies may be ineffective, though, because they frequently do not address the core issues of motivating why a change is needed or underlying faculty identities, epistemological beliefs, or commitments (henceforth simply called beliefs) about what makes instruction effective (Fixsen et al. 2005; Brownell and Tanner 2012; Henderson et al. 2015). If faculty misunderstand core components of an EBIP, they may implement EBIP with poor fidelity to the effective practice (e.g., classroom response systems became coercive tools for attendance; Borrego and Henderson 2014).

Conversely, emergent and environmental changes are uncommon, yet they appear to align more naturally with findings from organizational change and SNA (Kezar 2014a). Because of the relative lack of these emergent and environmental change efforts, there is a significant gap in our understanding of whether they do indeed align with the findings from organizational change and SNA (Borrego and Henderson 2014). Our study aims to provide some evidence to fill this gap in the literature.

Programs I/II are an emergent and environmental change effort. These programs organize faculty into CoPs and let the CoPs emergently identify the problems they want to address and the EBIP solutions they want to pursue. The use of CoPs supported by mentors provides an environment that supports faculty through their change efforts.

Program I is a competitive, internal grant program created by the College of Engineering to improve the quality of the large lecture courses and to enable faculty to explore teaching practices such as improving students’ teamwork or design skills. A call for proposals is issued to all faculty once a year and faculty submit grant proposals to participate. In these proposals, faculty articulate why their proposed changes to a course or set of courses are strategic and innovative. More importantly, the faculty must demonstrate that there is a team of at least three faculty members who believe that the proposed course changes are worth pursuing. This requirement for submitting a proposal is the basis for creating a CoP that will execute the proposed innovation. CoPs whose proposal are accepted are given funding to pay for teaching assistants or faculty summer salary to support them as they execute innovations.

For three years (Fall 2012 through Spring 2015), Program I funded 17 CoPs from a range of engineering disciplines (see Table 1). These CoPs are mentored by a mixture of faculty development personnel from the College of Engineering and engineering faculty members who have been identified as Education Innovation Fellows (EIFs). These fellows were chosen because of their track record as advocates for high quality teaching in the college and their knowledge of EBIP. Each mentor was assigned to advise between two and four CoPs. The mentors also convened on a weekly basis to discuss how each of the CoPs was performing and what innovations each CoP was executing. Mentors described these meetings as essential for learning about the successes of the various CoPs and provided them with examples and stories to share with the CoPs that they mentored (Cross et al. 2018).

Table 1 Demographic characteristics of participants
Full size table

Program II is an externally funded program that was inspired by Program I. This program began in Spring 2014 and is a STEM-inclusive program that focused on studying the scalability and transferability of Program I beyond the College of Engineering. Program II added five new CoPs in the physical/biological sciences and mathematics. The CoPs in Program II were formed by engaging faculty from across the university to work collaboratively in teaching. The emphasis in Program II is on creating CoPs, although moderate funding (<$15 K) was provided to the CoPs. The Principal Investigators (PIs) on the Program II grant (two of which were also EIFs) mentor each of the CoPs in the same way that the EIFs mentor teams in Program I.

Communities of Practice and Faculty Teaching Practices

The concept of Communities of Practice (Lesser and Storck 2001; Wenger et al. 2002) provide a framework for understanding how organizations can learn to implement new practices in a particular context. One of the key challenges for changing teaching practices is that decision-making during instruction and curriculum development are driven by faculty’s beliefs about teaching (Borrego et al. 2013b; Hasweh 1996; Luft and Roehrig 2007; Tsai 2002; Yerrick et al. 1997). Unfortunately, because EBIP value student-centric learning and independent student action rather than the content expertise of the instructor, the implicit beliefs of instructors frequently lead them to act in ways that make EBIP ineffective and subsequently cause them to abandon using EBIP (Brownell and Tanner 2012; Kegan and Lahey 2009; Reeve 2009). Consequently, changing teaching practices requires changing beliefs.

CoPs provide an environment for challenging these resistant beliefs, surrounding faculty with respected colleagues, thus mitigating the perception of identity threat (Wenger et al. 2002). CoPs accomplish this by providing a highly collaborative organizational structure that is intended to last and thereby can promote long-term situated learning (Lave and Wenger 1991; Wenger 1998; Wenger et al. 2002). Faculty begin their participation in a CoP on the periphery, participating in, and observing, the practices of the community. Through this process, CoPs foster mutual trust and reflective engagement (Wenger 1998; Wenger et al. 2002), and in such a context, beliefs about teaching can change implicitly (Davenport and Prusak 2000; Hildreth and Kimble 2002). In addition, participation in CoPs can decrease the learning curve for novices, reduce creation of redundant resources or reenactments of failures, and promote creativity (Lesser and Storck 2001). Prior efforts suggest that CoP models can be effective in curriculum reform (Finelli and Millunchick 2013; Villachia et al. 2013).

Social Network Analysis and Faculty Teaching Practices

Individuals are embedded in complex social relationships and interactions; therefore, who they know often shapes what they know (Borgatti et al. 2009; Daly 2010; Feld 1981; Wasserman and Faust 1994). Social network analysis has shown great potential in helping social science researchers understand patterns of relationships and various social structures in the field of sociology, anthropology, psychology, economics, and many more (Borgatti et al. 2009; Freeman 2004). However, only recently education policy makers and researchers have begun to recognize the value of SNA in studying education reforms or change efforts in postsecondary education (Daly 2010; Kezar 2014a; Brennan 2010; Weaver et al. 2015).

In a social network, individuals form closely-related clusters or groups with others who have similar social status or share similar interests, ideas, or activities; whereas those who do not have much in common tend to be far apart in the network (Feld 1981; Wasserman and Faust 1994). When individuals work within dense clusters of strong network ties, they generally converge toward the beliefs and knowledge of those within their network (Coleman 1988; McPherson et al. 2001; Walker et al. 1997). This convergence can enable members of the group to gain expertise in the knowledge and skills of their cluster (Adler and Kwon 2002; Daly 2010; Walker et al. 1997).

Because social connections can be resources as well as constraints, these strong ties may limit the introduction of new information into these dense clusters (Daly 2010; Walker et al. 1997). The theory of structural holes argues that homogeneity of information, uniformity of ideas, and density of connections are generally higher within a group of people as compared to that in between two groups of people (Burt 1992; Walker et al. 1997). Burt (2004) found that people who can span structural holes were more likely to make their voice heard, spread their ideas in the network, and receive positive evaluations from colleagues on their ideas. So, individuals who possess social ties with multiple clusters can bridge these clusters enabling new ideas and information to flow between the clusters. This influx of ideas through bridging ties can spark creativity and innovation (Kezar 2014a; Daly 2010).

So far, a majority of SNA studies on strong ties or bridging ties in educational research have focused on elementary and secondary settings, revealing how social networks can improve students’ outcomes or increase the use of desired teaching methods among instructors (Judson and Lawson 2007; Neal et al. 2011; Penuel et al. 2009). For example, elementary school teachers who productively changed their teaching methods had more connections with experienced and novice teachers than teachers who did not productively change their teaching methods (Penuel et al. 2009). Bridging ties (e.g., being connected through a common mentor) have also been found to be important in helping teachers change their teaching practices. Neal et al. (2011) found that teachers were more likely to change their teaching practices if they saw a peer using a teaching method than if they saw a mentor using the same technique. They concluded that the decision-making of peers with the same mentor created a greater sense of confidence and value (i.e., “if my peer can do it, so can I”) than other social connections (Neal et al. 2011).

While these studies provide insights into the importance of social networks for improving teaching, faculty in post-secondary contexts have multiple social networks such as research collaborations or departmental affiliations. Many studies have relied on co-authorship or citation networks to study faculty social networks, but these networks may not appropriate for studying faculty teaching social networks (Kezar 2014a; Xian and Madhavan 2014). Indeed, Quardokus and Henderson (2015) argued that networks such as departmental affiliation are poor proxies for faculty teaching networks and Andrews et al. (2016) show that discipline-based education researchers maintain teaching networks that transcend their individual departments. Although, few studies have examined how the teaching social networks of faculty influence their teaching practices, we describe the few that we know of.

Spalter-Roth et al. (2010) found that faculty primarily teach in isolation and thus do not participate in the creation of teaching knowledge and lack the social capital to effectively change their practices. Middleton et al. (2015) found evidence suggesting that STEM faculty who have more teaching network connections are more likely to use learner-centric approaches. Both of these studies suggest that most faculty teaching networks are too sparse to benefit from either strong or bridging network ties. Andrews et al. (2016) found that faculty change their teaching practices in response to colleague-colleague interactions, especially with colleagues deemed as opinion leaders. Opinion leaders were identified as faculty who were seen as excellent teachers or had unique teaching expertise because they conducted discipline-based education research. These opinion leaders were also more likely to have teaching ties outside their own departments.

Purpose of Current Study and Research Questions

Mirroring national trends, Programs I/II and this study were motivated by historic failures to spread teaching innovations on our campus. Although we have many discipline-based education researchers on our campus, they have had limited impact on the teaching practices of faculty at large. Indeed, prior to Programs I/II some of the mentors of this program had been instrumental in incorporating peer instruction in two large lecture courses in their department but courses in their own department or other departments had not followed their lead for 20 years. After Programs I/II began, 16 large lecture courses associated with the program began using peer instruction and personal response systems for the first time (Herman and Mena 2015). Because, we have had a long history of siloed innovation on our campus, we seek to explore whether the environmental supports provided by Programs I/II may have positioned these mentors to become effective change leaders by providing vital social network structures.

In previous papers, we documented the spread of EBIP between CoPs (Herman and Mena, 2015; Herman et al. 2015) and that COPs that adopted EBIP had statistically significantly higher density, higher connectedness, and lower centralization than CoPs that did not adopt EBIP (Ma et al. 2016; Ma et al. 2018). We determined which CoPs adopted EBIP by triangulating internal reviews provided by mentors, evaluations provided by an external evaluation team, and publication of education scholarship by the CoPs (Ma et al. 2016; Ma et al. 2018). These findings suggested that the adoption of EBIP was supported by strong, distributed intra-CoP ties, but did not explain how EBIP might transfer between CoPs. This study seeks to explore whether Programs I/II when viewed holistically provide the bridging ties that provide a potential explanation for this spread of EBIP and whether the mentors were the primary source of these indirect ties between individuals. Additionally, this study seeks to explore whether an emergent and environmental change effort provides network structures that bridge the structural holes between groups, which is important for improved performance and innovation.

Specifically, we investigate two research questions: 1) How important are the mentors in creating the observed social network structure of Programs I/II? and 2) Do the Programs I/II mentors play a unique role in bridging diverse CoPs?

Research Methods and Results

We answer the first research question by examining the network-level characteristics of the social network using Monte Carlo simulations. We answer the second research question by examining how the demographic characteristics of faculty interplay with their bridging role.

Data Collection

We construct and analyze a teaching social network of all individuals in Programs I/II to elucidate which attributes of individuals may be important in supporting the spread of knowledge of EBIP across CoPs and across the program. All research procedures were approved by Midwestern University’s Institutional Review Board.

Sample Population

In Programs I/II, 142 faculty members from Midwestern University participated in 22 faculty CoPs through Spring 2015 (when data collection was completed). These faculty CoPs were created by Programs I/II starting in Fall 2012. The faculty who participated in these 22 CoPs and the faculty mentors of these CoPs comprise the study population. CoPs had a minimum of 3 members, an average of 5.9 members, and a maximum of 10 members. The sociograms (e.g., Fig. 3) show only those participants that completed the sociometric survey.

Sociometric Survey

The sociometric survey (see appendix) was derived from the survey used by Quardokus and Henderson (2015) to document the hidden teaching networks in STEM departments. Their survey was designed to identify the teaching networks of faculty within a department by asking faculty to identify with whom they talked about teaching and the frequency of their interactions. We modified the Quardokus and Henderson (2015) survey to match the constraints of our project. First, whereas the original survey was designed to map the networks of departments with the largest department being 44 members, Programs I/II span 15 departments and 142 faculty with only subsets of faculty from each department. Some departments had multiple CoPs and some CoPs spanned up to four departments. Consequently, faculty were asked to describe the frequency of their interactions with every member of Programs I/II. Second, during pilot testing, we found that faculty did not like the terminology of daily, weekly, and monthly as originally used by Quardokus and Henderson (2015). They preferred ratings of never, occasionally (1–5 times per year), and frequently (more than 5 times per year). Third, because the structure of our intervention was to create CoPs, we distinguished between conversations and collaboration as collaboration is more indicative of deeper involvement with a CoP whereas talking can occur outside a CoP. Consequently, we added an additional category for types of interactions and provided more definitions of teaching, talking, and collaboration than did Quardokus and Henderson (2015). Faculty were explicitly told that collaborations are any interaction that moved beyond conversations and led to action such as co-teaching a course, redesigning a course, or co-designing a survey or test.

Sampling Method

Data collection and analysis were conducted by an educational psychology graduate student as an external research assistant. She was not affiliated with Programs I/II, and her dissertation advisor was likewise not affiliated with the programs. Seeking to attain a minimal response rate of 70% for SNA surveys as suggested by Borgatti et al. (2006), she needed to administer the sociometric survey with three sampling passes. First, to minimize time that participants would need to spend completing the survey, the research assistant administered the survey on paper during CoP meetings of the CoPs that gave her permission to attend their weekly meeting. Participants were given 15 min to take the survey after she explained the task. Twenty-six paper survey responses were collected during the first stage. Because many CoPs did not provide permission to attend their weekly meetings, she ported the survey to an online survey through Google forms and collected electronic responses through mass e-mail solicitation. Fifty-three electronic survey responses were collected during the second stage. Last, she identified the participants who had not taken the survey in the first two stages and administered the survey through individual e-mail solicitation. Twenty-three electronic survey responses were collected during the third stage. Since the change in modality (in-person versus online surveys) may have impacted how participants completed the surveys, we compared the mean number of interactions participants reported in each modality. Participants reported 22.5 and 21.4 interactions on average, respectively, revealing similar depth of responses across the modalities.

A total of 102 responses were received during data collection (response rate = 72%). Two responses were not included in the data analysis because one respondent left the university partway through the study interval and another respondent submitted an incomplete survey. The final dataset includes 100 participants from 15 departments. The demographic characteristics of the participants are presented in Table 1.

Social Network Measures

SNA can use both network-level measures and individual-level measures. Network-level measures describe the relationships between all members of a social network and how well each member of the network can access any other member (i.e., network cohesion). Individual-level measures describe the types of social interactions and roles that each individual plays in the context of the whole network. Network ties between two individuals can be treated as directed (information or relationship originates from a person and reaches another person) or undirected (information or relationships are treated as reciprocal). Because dyad reciprocity did not vary significantly between CoPs (Ma et al. 2018), we treat all ties as undirected in our data.

Network-Level Measures

Based on prior studies indicating that density, connectedness, and centralization (Borgatti et al. 2002) were sufficient to distinguish between CoPs that adopted EBIP from those that did not (i.e., additional network measures such as breadth were nearly perfectly correlated with the chosen three measures; Ma et al. 2018), we use these same metrics for the present study. Density is calculated as the actual number of ties in a network divided by the possible number of ties. It indicates the degree of dyadic connection in a network. While a general indicator of the amount of social connections in a network, it cannot distinguish whether a network is globally structured as a single cluster or several smaller clusters. Connectedness complements density by indicating whether a network is globally cohesive (i.e., all nodes can reach each other). Connectedness is calculated as the proportion of pairs of nodes that reach each other by any path.

The final indicator of network cohesion is the centralization of nodes. Centralization indicates the extent to which ties are associated with one or a few nodes in the network. It is calculated as the variation in the number of ties between each node and the node that has the highest degree (Freeman 1979). In a highly centralized network, each member of the network connects to a central person who moderates all communication. However, in a distributed network, each member can reach out to others directly.

Figure 1 illustrates these first three metrics to facilitate comparisons. Network 1 is a sparse network with few interconnections (low density) and nodes that are unable to access other nodes in the network (low connectedness). Network 2 is more dense than Network 1, but some nodes still cannot access each other. Network 3 has the same density as Network 1, but all nodes can reach each other (high connectedness) and no node has more than one additional connection compared with other nodes (low centralization). Network 4 has a single central node that connects all other nodes (high centrality). In Network 5, all nodes connect with every other node, creating high density and connectedness but low centralization (no node is the central focal point). A network that supports learning and increased performance among all group members will generally approach a structure similar to Network 5 (Kezar 2014a). Each of these network metrics is normalized by dividing the maximum metric value possible (e.g., density is calculated as proportion of actual ties divided by all possible ties existing in the network), so final metric scores are all between 0 and 1.

Fig. 1
figure 1

Examples of how different network parameters reveal characteristics of the network

Full size image

Individual-Level Measures

We used betweenness centrality to identify individuals who play a bridging role between individuals or clusters of individuals. Betweenness centrality measures the number of times a node lies on the shortest path between two other nodes and represents how often a person serves as the bridge for two other unconnected people in the network (Freeman 1979). There are different ways to be a bridge—one could serve as the liaison between two unconnected people or simply act as the go-between for ideas and knowledge.

Figure 2 displays three networks with nodes labeled with numbers. The betweenness score for Node 1 varies across these three networks while density and connectedness remain constant across all three networks. In Network 1, Node 1 does not connect any two nodes, so it has low betweenness. In Network 2, Nodes 5 does not connect directly to Node 2. Nodes 1 and 3 both serve as the bridge between Nodes 2 and 5, so Node 1 has a medium level of betweenness. In Network 3, no two nodes share direct connections with each other, except for Node 1. Because Node 1 is the connecting bridge between all other nodes, it has high betweenness. While centralization and betweenness are related, centralization describes a characteristic of a whole network but betweenness describes a characteristic of an individual.

Fig. 2
figure 2

Examples of how the positioning of a node within a network affects its betweenness

Full size image

Table 2 summarizes the four SNA metrics and offers hypotheses for how the metrics for the CoP mentors will differ from those of all other participants, if the mentors play a bridging role. All metrics are calculated using UCINET (Borgatti et al. 2002). We generated the first three hypotheses with respect to Research Question 1 and the last hypothesis with respect to Research Question 2.

Table 2 Summary of SNA measures discussed in this paper
Full size table

Data Visualizations

Our survey documented the frequency and nature of interactions that a STEM faculty member has with another faculty member either through conversations about teaching or active collaborations on teaching. These conversations and collaborations are represented with a network tie (line) that connects the two network nodes (circles) that represent the two faculty members. We created two social networks—one is a Conversation Network, which is based on whether two faculty members have ever talked about teaching; and the other is a Collaboration Network, which is based on whether two faculty members actively collaborated on teaching. The Collaboration Network is a strict subset of the Conversation Network as collaboration requires conversation but goes beyond it.

We provide visualizations of the Conversation and Collaboration Networks using Gephi (available at http://gephi.org/). The arrangement of nodes in these visualization is based on each individual’s average number of ties with others, using ForceAtlas2 layout algorithm (Jacomy et al. 2014) with Lin-Log model (Noack 2007). The position of a node is determined by the interaction of attraction force and repulsion force acting on the node. The algorithm moves connected nodes closer together and pushes unconnected nodes apart. The layout minimizes the energy needed to satisfy these attraction and repulsion forces. The size of the circle is determined by the number of ties a faculty member had relative to the average number of ties each node had in the network (i.e., larger nodes are more connected). The color of a node indicates the faculty role: orange nodes are mentors, blue nodes are other participants. Similarly, the color of ties between nodes indicates whether that tie includes a mentor: orange lines are ties that include a mentor, blue lines are ties that do not include a mentor.

Figure 3 is one visualization of the Collaboration Network of Programs I/II (i.e., ties represent social interactions that faculty classified as collaborations). A visualization of the Conversation Network of Programs I/II (i.e., ties represent social interactions that faculty classified as conversations) can be found in Fig. 4. The ForceAtlas2 layout causes nodes to cluster into affinity groups based on their network ties. Given that Program I/II encouraged faculty to join CoPs, we should expect to see nodes from the same CoP cluster together. We have added dotted ovals onto Fig. 3 around clusters of nodes that were indeed members of the same CoP. As shown in the figure, four CoP participants (marked by blue circles) located outside their CoP clusters and one CoP participant crossed two CoP clusters. This overlay reveals that 95% of nodes cluster with their CoPs, providing evidence for the validity of our data for studying the social structure of Programs I/II.

Fig. 3
figure 3

A visualization of the Collaboration Network of Programs I/II. Dotted lines around clusters of nodes delineate the CoPs. Since not all participants of Programs I/II completed the sociometric survey, some CoPs appear to have fewer than three members

Full size image
Fig. 4
figure 4

The Conversation Network of Programs I/II with (left) and without (right) mentors. Orange ties are connected to orange mentor nodes

Full size image

Limitations

The study relies on faculty self-report about their social interactions. Dyad reciprocity for conversations/collaboration was high (~90%), providing evidence that faculty reports were reliable across respondents. The study also includes data only from those faculty who completed the sociometric survey. The sample likely over-represents faculty who found Programs I/II to be valuable and thus were willing to complete the lengthy survey; all mentors completed the survey and CoPs that did not adopt EBIP had slightly lower response rates (the response rate of CoP participants who adopted EBIP was 80%, while the response rate of CoP participants who did not adopt EBIP was 68%). Finally, we did not collect social network data during the first two years of the program, so we cannot explore whether the existence of social network ties preceded the spread of teaching innovations or when faculty joined the network, limiting our ability to make causal claims about the effects of these ties. Consequently, we focus our analysis on the alignment between the structure of Programs I/II and the claims of structural holes theory.

Methods for Research Question 1: Mentors and Overall Network Structure

The overarching hypothesis of this study is that the faculty mentors play a unique bridging role in creating a social network that enables the spread of knowledge about teaching innovations across the network. Based on this hypothesis, we would expect that the mentors increase the general connectivity of the network (i.e., mentors add more density and connectedness to the network than other faculty members) and that they are a primary conduit for the flow of information (i.e., mentors create a more centralized network than other faculty members). As a preliminary observation, we compared the network characteristics of the whole network with and without the mentors (See Table 3). This initial comparison revealed that each of the network characteristics decreased as hypothesized, with degree centralization in particular decreasing to less than half its original value. To provide a qualitative understanding of how these changes manifest in the social network, we present and describe visualizations of the conversation and collaboration networks.

Table 3 Network cohesion metrics for the Conversation and Collaboration Networks of Programs I/II with and without mentors
Full size table

Conversation Network

Figure 4 displays the Conversation Network of Programs I/II with mentors (left) and without mentors (right). These visualizations reveal that no nodes or clusters of nodes are separated from the larger network even when the mentors are removed (i.e., the conversation network maintains perfect connectedness). Additionally, the visualization reveals that most of the mentors are positioned toward the center of the network and appear to be generally larger than the other nodes in the network (increasing the centrality and density of the network respectively). In accordance with this observation, removing the mentors creates structural holes in the center of the network (note the increased white space in the center of the diagram on the right).

Collaboration Network

Figure 5 displays the Collaboration Network of Programs I/II with mentors (left) and without mentors (right). These visualizations reveal that removing the mentors creates more nodes or clusters of nodes that are separated from the larger network (i.e., a significant reduction in connectedness). This increased isolation of nodes in the Collaboration Network contrasts with the perfect connectedness in the Conversation Network. Like the Conversation Network, the mentors appear larger and more centrally located in the network visualization, increasing density and centralization.

Fig. 5
figure 5

The Collaboration Network of Programs I/II with (left) and without (right) mentors. Orange ties are connected to orange mentor nodes

Full size image

Monte Carlo Permutation Test

To explore whether these changes in network metrics were because of the mentor’s role in the network or simply a byproduct of removing ten participants from the network, we used a Monte Carlo permutation test (Carsey and Harden 2013) to compare the Program I/II network without the 10 mentors (mentor-less network) with networks that removed an equivalent number of random participants (10) from the network (simulated networks). Because tenured professors are more established and therefore more likely to have broader social networks than junior tenure-track faculty, we controlled for academic rank by using stratified random sampling. Since the mentors consisted of 2 full professors, 4 associate professors, and 4 non-tenure-track faculty members (distinguishing between senior and junior non-tenure-track faculty in this data set was not possible as titles for these faculty do not necessarily reflect seniority), we randomly removed 2 full professors, 4 associate professors, and 4 non-tenure-track faculty members from the simulated networks. This stratified sampling approach increases the likelihood that mentors will be removed from these simulated networks. This should produce comparison networks that are more similar to our test case than a purely random sampling approach; as a consequence, our estimates for the impact of mentors on the network is made more conservative.

We created 50 of these simulated networks for both the Conversation and Collaboration Networks, a sample sufficiently large to obtain less than 1% standard error of measurement on our estimates of the mean value of our chosen network characteristics. The 50 simulated networks were generated to model the probability distribution of the Conversation Network after removing ten members. By modeling the probability distribution of these sub-networks, we can statistically evaluate the role of the ten mentors (Carsey and Harden 2013).

We computed means and standard deviations of selected network metrics for the simulated networks to measure the likelihood of obtaining the network characteristics found in the mentor-less network. These means and standard deviations were used to perform one-sample t-tests to compare the simulated networks with the mentor-less network (Carsey and Harden 2013). We chose a conservative α-value of 0.001 for statistical significance to minimize the likelihood of Type I error for three reasons: our standard error of measurement was 1%, we perform multiple comparisons for significance with these permutation tests, and we have previously demonstrated the relationship of these cohesions metrics with the adoption of EBIP among CoPs (Ma et al. 2018).

Results for Research Question 1: Mentors and Overall Network Structure

Conversation Network

The one-sample t-test indicated that the density of the Conversation Network decreased significantly by 23%, after removing ten mentors (density = 0.17) as compared to removing ten random faculty members (density = 0.22), difference = 0.05, t(49) = 34.94, p < .001, 95% CI [0.048, 0.055]. The centralization of the network also decreased significantly by 48% after removing the ten mentors (centralization = 0.29) as compared to removing ten random faculty members (centralization = 0.60), difference = 0.31, t(49) = 34.69, p < .001, 95% CI [0.29, 0.33]. The analysis showed that the Conversation Network was fully connected regardless of who was removed from the network (see Table 4). These findings also indicate that the mentors provide more network ties than other faculty members and create a central point through which information can more easily flow.

Table 4 One sample t-tests for the network cohesion metrics for the Conversation Network of Programs I/II without mentors versus without ten random members
Full size table

Collaboration Network

Performing the same set of one-sample t-tests on the Collaboration Network indicated that all three cohesion measures showed a significant decrease after removing ten mentors as compared to removing ten random faculty members (see Table 5)—density of the Collaboration Network decreased by 31%, difference = 0.015, t(49) = 28.29, p < .001, 95% CI [0.014, 0.016]; centralization decreased by 62%, difference = 0.16, t(49) = 49.68, p < .001, 95% CI [0.15, 0.17]; and connectedness decreased by 19%, difference = 0.14, t(49) = 24.00, p < .001, 95% CI [0.13, 0.16]. These findings confirm that the mentors provide more network ties than other faculty members and create a central point through which information can more easily flow. Further, without the mentors, some faculty members become completely disconnected from the information pathways with respect to collaborations provided by Programs I/II.

Table 5 One sample t-tests for the network cohesion metrics for the Collaboration Network of Programs I/II without mentors versus without ten random members
Full size table

Methods for Research Question 2: Bridging Role of Mentors

Figures 5 provides a qualitative understanding of the betweenness of the mentors. For example, the removal of the mentor closest to the bottom of the visualization isolates the two lower nodes from the rest of the network. Consequently, this mentor is between these nodes and the rest of the network. Similarly, the isolated cluster of five nodes on the far left of the visualizations (see right visualization) is only connected to the rest of the network through mentors. This isolation of clusters of individuals reveals part of the bridging role that mentors play. Almost 70% of collaborative ties (100 of 142) that go outside the CoPs are to CoP mentors. Without mentors, there are no pathways for information and educational practices to spread to these isolated CoPs.

A two-level Poisson regression model was created to examine whether mentors had statistically significantly higher betweenness centrality than other faculty members in the Collaboration and Conversation Networks. The reason that Poisson regression analysis was used is that betweenness centrality scores derived from the network followed a Poisson distribution. Individual faculty members were the first level of regression and a faculty member’s department was the second level of regression. Department was entered as a second-level random variable, indicating that faculty are more likely to have betweenness with other faculty members from their own department. We controlled for faculty members’ gender, position as Program I/II mentor or regular participant, academic rank, and whether they are a member of a CoP that adopted EBIP or one that did not adopt EBIP by entering these attributes as fixed effects. We controlled for the amount of funding that each CoP received EBIP by entering funding as a covariate. Because we do not have any prior evidence about the role of betweenness in faculty’s interactions, we are concerned with both Type I and Type II errors; we do not want to unintentionally miss potentially significant relationships that we did not expect to find nor do we want to falsely reject our null hypothesis. Consequently, we chose an α-value of 0.05 for statistical significance for this test.

Results for Research Question 2: Individual-Level Characteristics

Conversation Network

The Poisson regression model for the Conversation Network indicated that CoP mentors had higher betweenness centrality than other CoP participants, t(76) = 5.19, p < .001; mentors were more likely to bridge two unconnected faculty members in the network. No other predictors were found to be significant.

Collaboration Network

The Poisson regression model for the Collaboration Network indicated that CoP mentors had higher betweenness centrality than CoP participants, t(76) = 7.68, p < .001. Members of CoPs that adopted EBIP had higher betweenness centrality than members from CoPs that did not adopt EBIP, t(76) = 2.34, p = .022, confirming the conversation network findings. Assistant professors had lower betweenness centrality than associate professors, t(76) = −2.43, p = .017, and non-tenure-track faculty, t(76) = −2.30, p = .023. This finding confirms our assumption in research question 1 that junior faculty are likely to be less well connected than their colleagues. No other predictors were found to be significant.

Synthesizing Findings from Prior Research and this Study

To fully understand the structure of the social network of Programs I/II, we consider the findings from this analysis in conjunction with findings from our prior research (Ma et al. 2016; Ma et al. 2018). We compare the density and centralization of the CoPs that adopted EBIP in Programs I/II with the whole network metrics calculated in this study.

First, while the Conversation and Collaboration Networks of the individual CoPs are quite dense (revealing strong ties within CoPs), the Conversation and Collaboration Networks of the whole network are sparse (revealing existence of structural holes across CoPs). About 98% of possible connections are present (average density of 0.98) in the Conversation Networks of the CoPs that adopted EBIP, but the Whole Conversation Network has an average density of 0.23. The average density of the Collaboration Network of CoPs that adopted EBIP is 0.74, while the density of the Whole Collaboration Network is 0.05. This disparity reveals that the network consists of many clusters of closely connected individuals who are only loosely connected with individuals outside their clusters. This description aligns with what can be visually observed in Fig. 3.

Second, we observe that the CoPs that adopted EBIP are less centralized (centralization = 0.04) in their conversations than in their collaborations (centralization = 0.39), which indicates a core-periphery structure. Conversely, the whole teaching network is more centralized (centralization = 0.63) in its conversations than in its collaborations (centralization = 0.27). We can interpret these disparities to mean that effective CoPs generally include everyone in conversations, but a core of individuals significantly drive the implementation of EBIP within the CoP. In contrast, conversations across the whole program are facilitated by a few individuals while collaborations are dispersed across the network.

Discussion

The findings from this study suggest that the program mentors play a critical bridging role between CoPs. The Monte Carlo analysis answers the first research question by revealing that the mentors significantly increase the density and connectedness of the whole network more than any other random selection of faculty members, strengthening an otherwise weakly connected network of individuals. While the mentors provided modest increases to the density and connectedness of the network, their presence doubled the centralization of the network (as compared with randomly selected faculty members). This dramatic increase in centralization suggests that the mentors primarily increased density and connectedness by connecting otherwise unconnected individuals.

The analysis of betweenness answered the second research question and corroborated these findings. For the Conversation Network, one’s status as a mentor was the only consistent attribute of a faculty member that increased their betweenness score. Similarly, for the Collaboration Network, one’s status as a mentor was the strongest attribute of a faculty member that increased their betweenness score. These findings indicate that the mentors played an important role in plugging structural holes in the social network of Programs I/II that would otherwise have hindered the spread of knowledge and practices across the network. These findings do not provide final proof that the mentors were responsible for spreading EBIP across Programs I/II but it does provide evidence that they are the most likely individuals to spread information between CoPs.

Creating interconnected networks of CoPs holds promise. Institutional change often proceeds through three stages of change: mobilize, transition, and institutionalize (Kezar 2014b). Creating CoPs disrupts faculty social networks, mobilizing faculty and creating the need for them to rethink their teaching practices. CoPs and their mentors provide the context for faculty to learn about EBIP and provide support for them as they transition to using them. The network of CoPs creates a new network structure in which these EBIP are valued, providing the social support to institutionalize and sustain changes. Future research should explore how to generate and support these social networks.

These observations reveal that while the members of CoPs that adopted EBIP are strongly connected within their own CoPs, they are only loosely connected to other CoPs and that this loose connection is maintained by a small number of participants in Programs I/II. While the social network structures of the CoPs that adopted EBIP aligned with the concepts of legitimate-peripheral participation (i.e., all members are connected and participatory, but a few members are more central, driving collaboration), the broader structure of Programs I/II does not align as well with common organizational learning theories. The findings do resonate with prior SNA findings, though. Future research will need to explore what types of involvement from mentors best promote the bridging ties that we seek to create.

The “Hidden” Ties that Guide STEM Faculty Teaching Practices

Our belief is that the lack of both strong and bridging social ties around teaching has been an otherwise hidden structure that has thwarted systemic change in undergraduate STEM teaching (Kezar 2014a). We explore this assertion through a four-quadrant model of approaches to creating change in faculty teaching practices (Fig. 6). According to SNA, social networks create systems that constrain which values and practices are sustainable (Daly 2010). As long as the network structure stays the same, the system will produce the same result (Daly 2010).

Fig. 6
figure 6

A four-quadrant model of teaching networks. Efforts to improve faculty teaching practices tacitly promote certain types of network ties, but generally fail to promote both the strong and bridging teaching ties needed for sustainable change

Full size image

Historically, teaching is done in intense isolation at the university level (Quadrant 1) with faculty maintaining sole jurisdiction over their courses with little input from their peers (Costin 1972; Kezar 2005; Lammers and Murphy 2002). This lack of connectivity enables the status quo that resists EBIP: the lack of strong ties does not provide the social support needed for faculty to learn and persist in using and the lack of bridging ties reduces their exposure to peers who are successfully using EBIP and can spark creativity (Borrego et al. 2013b; Henderson et al. 2012).

Centers for teaching and learning (Quadrant 2) disrupt the status quo by providing bridging teaching ties, but generally fail to provide the strong ties with other faculty that can sustainably change beliefs and practices (Henderson et al. 2012). The creation of discipline-based education research hubs (Quadrant 3) can create strong ties among faculty that lead to improved teaching, but these hubs frequently fail to create bridging ties that enabled their ideas to spread.

Programs I/II made course-centric CoPs connected by mentors, creating both clusters of strong ties and bridging ties between those clusters (Quadrant 4). The CoPs created the strong ties that provided a context for situated learning and motivational support. The bridging ties facilitated by CoP mentors provided avenues to expose CoPs to peers who were using EBIP, combatting the silo effect (Neal et al. 2011). Critically, the presence of both types of ties protects against the potential pitfalls of having only one type of tie. Strong ties support the convergence of a community toward implementing best practices effectively but can lead to stagnation and a lack of innovation as the community lacks regular exposure to new ideas from diverse peers. Complementarily, bridging ties alone can spark creativity and innovation but do not provide the ongoing support that is needed for faculty to adopt EBIP. Therefore, both strong ties and bridging ties are needed for the spread and sustainment of EBIP in STEM faculty’s teaching network.

Future Work

Having demonstrated that faculty mentors in Program I/II create bridging ties across CoPs, future studies will seek to rigorously document whether the spread of specific EBIP can be attributed to the mentorship relationships. We are collecting data on what teaching practices each CoP was using during each semester of the program. We will connect this longitudinal teaching practices data with our social networks to study more fully whether the spread of innovations can best be attributed to the bridging role that the mentors play. Our findings suggest two potentially promising research directions: how can we find or create effective mentors and how do CoPs that adopt EBIP form and grow?

First, Fig. 7 changes the color scheme of our visualization slightly, coding inter-CoP ties as red and intra-CoP ties as black. Like Figs. 4 and 5, we display the same network with and without the program mentors. Removing the mentors, again reveals a dramatic loss of inter-CoP connectivity, but it also reveals that are a few individuals in the network who maintain robust inter-CoP teaching ties. Based on our findings, it is likely that these few individuals might make ideal candidates for future program mentors. Notably, three faculty members have six or more inter-CoP connections. Two of these faculty members (marked as red circles in Fig. 6) had indeed become mentors in Programs I/II since data collection for this study completed. Future work could more fully explore whether effective faculty mentors are “found” or “made.”

Fig. 7
figure 7

The Collaboration Network of Programs I/II with and without mentors. Red ties are inter-CoP ties. Black ties are intra-Cop ties

Full size image

Our anecdotal experiences with running Programs I/II suggests that mentors can be most effective when they are mentoring a small number of CoPs (e.g., two) that are outside their primary discipline. By engaging with multiple CoPs, the mentors gain a diversity of perspectives of what the teams are doing well that they can then share with the CoPs that they mentor and with other mentors. By forcing mentors out of their own disciplines, they can create new bridging ties across the program. Beyond these new ties, moving mentors out of their discipline allows the CoP members to maintain their status as content-knowledge experts with autonomy to make decisions, creating a flatter social structure in which the mentor is seen as a valuable resource rather than as an intrusive or controlling authority figure. Future research could explore the tradeoffs between the number of CoPs that mentors connect and how their relationship with the CoPs should be framed.

Second, the betweenness analysis revealed that the tenure-track assistant professors had lower betweenness than tenured faculty and non-tenure-track faculty. This finding suggests that the assistant professors operated primarily on the periphery of teaching innovations in Programs I/II and did not play a role in supporting the spread of innovation. Because assistant professors may more readily accept the norms of a CoP, future research should explore how CoPs can successfully integrate new assistant professors, socialize them into using EBIP, and enable them to transition toward more core roles.

Conclusion

This study provides critical early evidence for the value of social network analysis for uncovering the otherwise hidden social interactions and structures (Quardokus and Henderson 2015) in post-secondary STEM education. By improving understanding of these structures, we will be better equipped to guide and study future change efforts to increase the use of EBIP. We have shown that the CoP mentors of Programs I/II played a unique role within the social interactions of those programs. They provided a vital bridge between CoPs, facilitating the transfer of knowledge across CoP boundaries, potentially increasing the spread of EBIP.