Introduction

Changes to modes of production and consumption are needed to prevent dangerous interference with climate systems and to ensure the sustainable use of natural resources. Community-based initiatives (CBIs) are actively inducing such changes in behaviour and are vital learning and testing grounds for innovations before they become widely applicable (Seyfang and Haxeltine 2012; Forrest and Wiek 2014, 2015; Seyfang and Longhurst 2016; Luederitz et al. 2017). However, while such initiatives have been regarded as promising in contributing to behavioural change, their contribution may only be locally relevant (Mattijssen et al. 2018), face several significant barriers (Middlemiss and Parrish 2010; Buttoud et al. 2011; Broto and Bulkeley 2013; Feola and Nunes 2014; Byrne et al. 2017), and require consistent, evidence-based policy support to become more widespread and structurally embedded (Seyfang et al. 2013).

To understand the role, possibilities, and challenges of CBIs in societal transitions, it is necessary to conduct international research and development projects that typically follow a case study strategy (e.g. Eicker et al. 2015; Hicks and Ison 2018): The research data are collected from multiple cases and distributed in partner countries. Such studies provide versatile and feasible results, thus creating a sound basis for informed policy interventions (Frantzeskaki et al. 2016). As in all case studies, but especially in multi-national studies, the case selection is critical to the research process.

The following typical disadvantages have been detected in case selection: Often, the focus is predominantly on the ‘novel and avant-garde’ cases (Martin and Sunley 2001); the representativeness of the chosen cases is rarely explicated properly (Markusen 1999, 2003; Yin 2013); As a result, multi-site comparative studies have succumbed to more ‘easily challengeable, isolated single case studies’, in which the ‘subjective view of single researchers is narrowing the case study selection’ (Markusen 1999, 2003). To avoid such pitfalls, several recommendations for proper case study selection have been made, including explicating the primary motivations for selecting cases (James 2006), properly contextualising the case studies (Poulis and Plakoyiannaki 2013), and applying a dynamic, iterative selection procedure instead of a linear, step-by-step procedure (Fox-Wolfgramm 1997; James 2006; Montibeller et al. 2008). A dynamic iterative case selection process should begin with the conscious and consistent creation of a common ground for case selection, i.e. with the learning process where purposes, theoretical standing points, and methodological requirements for the case selection are consistently created among the researcher group (Santos et al. 2015).

These disadvantages and recommendations are especially relevant in multi-disciplinary research projects on societal issues, like CBIs. In the beginning, researchers might have very different ideas about what constitutes a community-based initiative. Even if the CBI is explicitly defined, it might include a rather wide variety of initiatives, for example, in terms of size, domain, and geographical location. Conflicting views on the project focus may considerably hinder scientifically sound case selection and lead to the selection of a biased set of cases (Seawright and Gerring 2008). Studies on CBIs have often followed a single case strategy (e.g. Feola and Nunes 2014; Burnside-Lawry and Carvalho 2016; Gomez Mestres and Lien 2017; Schlappa 2017). ‘Theoretical sampling’ (Blichfeldt and Halkier 2014) is performed to find a case that will likely enrich the extant knowledge from a specific theoretical perspective. However, criteria used in the case selection are often vaguely defined in studies on CBIs, with a few exceptions. For example, Forrest and Wiek (2015) explicitly defined four theory-driven criteria to guide case selection on CBIs: (i) place-based, (ii) ‘small’ (population less than 10,000), (iii) community-based (they seek to change and are implemented by the community), and (iv) have sustainability as a primary goal. Forrest and Wiek (2015) sought, similar to the majority of other studies, only well-established and successful cases, as these are supposed to be more informative. Studies that follow a multiple case strategy use additional criteria to approach representativeness of the case set in the whole population of CBIs. Then researchers examine the cases’ geographical location in the rural–urban continuum (Seyfang 2001; Fudge and Peters 2009) or the north–south continuum (Gomez Mestres and Lien 2017) and life span (according to classes ‘mature’ or ‘early stage’) (Phillipson and Symes 2015; Saraiva et al. 2016). Furthermore, several pragmatic criteria, like access to data or domain of the initiative, are listed as case selection criteria on CBIs (e.g. Seyfang and Longhurst 2016; Morais and Silvestre 2018).

A specific feature of studies on CBIs, especially in large international case studies, is that the ‘population’ for the sample of the case studies is not known; therefore, no straightforward sampling frame for case selection exists. Two strategies have been used to respond to this problem: First, the most commonly followed strategy is to simply rely on existing professional networks and databases (e.g. Feola and Nunes 2014; Burnside-Lawry and Carvalho 2016; Seyfang and Longhurst 2016; Hicks and Ison 2018; Morais and Silvestre 2018). Second, a less common strategy is to conduct an iterative process, i.e. first by mapping the overall initiative population and then by narrowing down the sample from the broad range identified in the initial mapping exercise (Seyfang et al. 2013). During such a process, a detailed understanding of the case contexts, including case languages, is a prerequisite for the successful mapping and selection of cases to achieve a broad and diverse case set. Therefore, individual researchers of a multi-national case study must conduct preliminary mapping of the population and final selection of the cases in their own country, rather independently. If they do not share a common understanding of the project aims and case selection procedure, the case set of the project may turn out irrelevant.

Despite the noticed need for sound common ground, little is understood regarding how to conduct the process of common ground creation among research groups and what methodologies can be used to enhance the collaborative process therein. In this study, we are responding to this lack of knowledge. The present study has dual objectives. The first objective is to illustrate and evaluate a new procedure for applying participative methods to the identification of common ground at the beginning of a case selection. The second objective is to map and rank case selection criteria of CBIs using this new procedure.

With regard to the first objective, the new procedure applies the large group cognitive mapping method known as contextual content cognitive mapping (3CM, Kearney and Kaplan 1997) and stochastic priority calculations. Hereafter, this procedure is referred to as stochastic cognitive mapping (sCM). Phases of sCM are embedded in the process of common ground creation (Lacroix et al. 2016), as described in Fig. 1. The methodological orientation and details of sCM are described in the ‘Materials and Methods’ section. A European multi-disciplinary study on CBIs in societal change towards sustainability served as an example to illustrate sCM. The performance of sCM is evaluated based on participants’ feedback, and by comparing an initial list of case selection criteria produced by the sCM at the onset of the project with the criteria that were finally used in the example project.

Fig. 1
figure 1

A process for building common ground for selecting cases on community-based initiatives (CBIs). Phase labels (1–8) are borrowed from Lacroix et al. (2016); bulleted points refer to case selection stages in the example project of this study. This paper illustrates a procedure, stochastic cognitive mapping (sCM) for the first part of the process, i.e. for identification of common ground. Particular methods of the sCM are depicted in the bold font

Full size image

With regard to the second objective of this study, researchers and other experts on CBIs served as informants to produce a list of case selection criteria using sCM. The applied procedure is unique in that it created openly nuanced common ground for the case selection. The use of sCM in the example project led to the creation of different criteria sets for predominantly different purposes. The use of sCM also resulted in a snowball sampling of the CBIs that does not rely on preexisting knowledge of domains.

Materials and methods

The context of the study

The EU-Seventh Framework Project TESS (Towards European Societal Sustainability)Footnote 1 provided the context for developing and illustrating sCM for common ground creation in a multi-disciplinary and multi-national case study. The project focused on the CBIs in transitioning towards a green economy. The research team consisted of more than 30 researchers from different disciplines (sociology, geography, economics, natural sciences, agriculture, and forest sciences).

The purpose of the first work package of the TESS project was to conduct a participatory exercise within the research group in order to select community-based initiatives that were to be further analysed in more detail during the following phases of the project. Altogether, 320 community-based initiatives were identified in regions of six European countries (Finland, Germany, Italy, Romania, Scotland, Spain), of which 48 were selected as cases. Furthermore, 15 cases were selected as key initiatives for in-depth qualitative studies.

A selection procedure was started by the sCM as illustrated and evaluated in this paper. The sCM resulted in a prioritised list of criterion candidates for case selection. Thereafter, and partly overlapping with the sCM procedure, selection of CBIs for the TESS project proceeded as a collaborative process among the researchers. The entire process of case selection is described in more detail in the supplementary materials.

Methodological orientation

The case selection is a participatory decision-making process among a research group. Different definitions for the decision-making processes exist (e.g. Belton and Stewart 2002; Carmona et al. 2013). Vacik et al. (2014) reduced the decision process to three general phases: problem identification, problem structuring, and problem solving. Identification ‘involves the acquisition and analysis of information to understand and to define the different decision problems by identifying goals and objectives, management alternatives, related policies, resources, conflicts and interactions’.

A variety of methods and approaches have been recommended for supporting phases of the decision process (see reviews from Kangas et al. 2006; Martins and Borges 2007; Vacik et al. 2014; Marttunen et al. 2016). Cognitive mapping (CM) has been increasingly proposed as a ‘soft’ tool methodology for the participatory problem identification phase in planning and decision-making (e.g. Mouratiadou and Moran 2007; Gray et al. 2015).

The results of CM can be fruitfully integrated with ‘hard’ multiple criteria decision analysis (e.g. Hjortsø 2004; Ferreira et al. 2015) if needed. The foundation of CM lies in cognitive psychology, a discipline examining how human beings receive, record, and use information. It aids the researcher in clarifying and describing people’s conceptual ideas regarding their environment, which are recorded in a graphic form that shows the concepts and their interconnections. The direction between concepts (causal and/or temporal relationships) and weights (or importance) of relationships can be included in the map. The terminology regarding such approaches varies to some extent (Eden 1992; Eden and Ackermann 1998; Montibeller and Belton 2006), and often, the term causal map refers to a map where direction and/or weight is also included.

CM methods and approaches have been developed especially for enhancing complex multi-objective and participatory decision-making problems (Mingers and Rosenhead 2004). CM allows the open articulation of perspectives and values, which is seen as a key in value-focused thinking (Keeney 1992) that emphasises creating decision options from values instead of problems. It proposes that decision-makers creatively compose a list of objectives (hopes, wishes), alternatives, and problems (barriers). Then, the list is used to elaborate a means-end hierarchy by asking questions such as the following: Why is this objective important? Is it a means to reach a more important objective or is it of value in itself? The results of cognitive maps can be presented in the form of an objective hierarchy to enhance value-focused thinking, to which multiple criteria decision analysis may then be applied to aid in making optimal decisions (Mendoza and Prabhu 2005; Innes et al. 2005; Almeida et al. 2014).

At best, and according to the original ideas of CM, the construction of the objective hierarchy is an iterative process in which the final model is collaboratively designed. In cases where there is a large group of participants and at which more general information about participants’ concept structure is aimed, the joint concept map cannot be easily iteratively constructed and evaluated due to the number of participants and group dynamics, such as domination, that hamper equal communication. Such ‘large group’ situations have been identified as an area requiring methodological development (Mingers and Rosenhead 2004). Also, geographically dispersed and heterogenous groups of participants, along with the limited time availability, could make discursive cognitive mapping difficult and thus require methodological development.

In large group situations, the synthesis from individual maps can be constructed by stochastic measures based on the following assumptions: (1) the analyst can understand the meanings of individual concepts, even though participants probably use varying words, accurately enough to merge concepts that hold the same meaning; (2) a concept raised by one or several of the participants may be of importance to participants who have not mentioned this particular concept in their own concept map; and (3) when many or a majority of participants who have mentioned a pair of concepts see them close to each other in their concept structure, those items are probably also close to each other in the concept structure of the one who did not initially mention those concepts. For example, Sironen et al. (2013) concluded that the experts did not consider the new items that were originally missing from their concept lists less important, which indicates the existence of cognitive biases (Festinger 1957; Simon et al. 2004) and supports the second assumption above.

For approaching stochastic concept inquiry and aggregation, Kearney and Kaplan (1997) proposed a method that they called the 3CM method based on assumptions 1–3 above. The method has later been successfully applied, e.g., by Kearney et al. (1999) and Tikkanen et al. (2006). These methods were based on qualitative in-depth interviews, which are rather time-consuming and, as such, not suitable as a participation technique for gathering information about the mental structure of large groups and in situations where participants are located far from each other, as in multi-national projects.

Furthermore, the relative importance of attributes given by respondents was not calculated in the studies mentioned above. Information on the importance of concepts in the map would be useful if the CM results are to be applied in multi-criteria decision analysis. When priority information is collected from individuals together with the personal cognitive mapping task, they can give weights only for the concepts included in their personal maps, not for the concepts raised by other respondents, even though, if asked, they might see those concepts as very important. This problem can be alleviated by using a sequence of inquiries, i.e. by collecting information at first only by concepts, then qualitatively merging responses and sending a prioritisation task including all concepts (Myllyviita et al. 2014). In some associations, such a procedure may be too time-consuming, or a single-phase procedure may reasonably identify the most essential aspects to be jointly discussed later. Then, where part of the preference information is lacking, stochastic measures can be used to estimate priorities for the elements not included in all individual concept maps. Sironen et al. (2013) illustrated and reflected models for stochastic prioritisation for partly overlapping concept lists.

Data collection

From the above-described standing points, the procedure of sCM reinforces 3CM with the following new elements: (i) an email inquiry is used to collect information about individual CMs, (ii) the Simple Multi-Attribute Rating technique (SMART, von Winterfeldt and Edwards 1986) is used to collect and calculate individual-level preference information, and (iii) stochastic calculations are used to estimate the priorities of the whole group in the situation of missing priority information. The procedure occurred as follows:

As the first phase of sCM, an email inquiry concerning case initiatives was constructed and sent to researchers and to their closest experts on CBIs. The inquiry as a whole was aimed at constructing a joint understanding and motivation towards the initiatives within the research group. Using an email inquiry enabled transparent overview of all aspects that the whole research group considered important to be taken into account in case selection. In face-to-face meetings, such mapping is difficult because not all researchers are able to travel to all meetings, and group dynamics can hamper equal discussion in meetings.

The inquiry was first sent to one person in each of the eight partner organisations, and it was advised to be forwarded to 4–8 persons working within the TESS project and other experts who were very familiar with the project. The inquiry was fully completed by 25 respondents. It included open-ended questions and an Excel worksheet about 3CM cognitive mapping questions complemented with a ranking task by SMART.

Participants were first asked to freely consider aspects around case study selection and then to conduct the following six tasks:

  1. (1)

    Name as many issues as possible by continuing the following sentence: ‘the important feature of a case study is to ……’ (see examples of the responses to this task in Fig. 2 and column C in Table 1). Important issues of case studies were asked for, instead of asking directly about case selection criteria, in order to obtain a more diverse picture for common ground, including values, objectives, and measures. According to the principles of value-focused thinking (Keeney 1992), aspects of upper levels of such objective hierarchy can then be applied as selection criteria;

  2. (2)

    Rank the issues by assigning 100 points to the most important issue and 0–100 to others in relation to the most important (column D in Table 1);

  3. (3)

    Cluster the issues by copying the sentences and pasting them into the same group according to the similarity of their contents;

  4. (4)

    Give short headings to the groups (see examples from column F in Table 1 and Fig. 2);

  5. (5)

    Evaluate the importance of the groups by assigning 100 to the most important group and 0–100 to others in relation to the most important (column G in Table 1); and

  6. (6)

    Explain what connects issues in the same group (column H in Table 1).

Fig. 2
figure 2

A cognitive map of two researchers who responded to the inquiry. The oval refers to the overall cognitive mapping task, boxes are group headings as given by the respondents, and texts without frames are direct quotations from the respondents on the issues to be taken into account in case selection

Full size image
Table 1 Contents of the data sheet that served as a starting point for the calculation operations of the stochastic cognitive mapping (sCM) to select criterion candidates (CC) for case selection
Full size table

Identifying and visualising cognitive maps

In the first phase of the analysis, issues raised by respondents were targeted by qualitative coding. Respondents’ original individual cognitive maps were visualised before performing the analyses (Fig. 2). Thereafter, analysis of the issues was conducted in two stages. The first stage involved merging qualitative issues (column C in Table 1) that had the same meaning into classes. Columns C, F, and H in Table 1 served for this content analysis; column I, which locates original issues into conducted data-driven content classes, has 27 classes altogether in this case. Those classes depict criterion candidates (CC) for case selection. Classes were constructed by the two first authors of this paper using a data-driven qualitative content analysis, which develops categories from the raw data instead of locating contents into the predefined categories (e.g. Green 2004). Both authors first made the classification independently and then focused on issues classified differently and aimed to find joint classification. The nominal variable CC in column I in Table 1 depicts the classes that formed the starting point to the following quantitative analysis.

The first part of the quantitative analysis of sCM constructed descriptive statistics (Fig. 3), a two-dimensional map (Fig. 4a), and a dendrogram (Fig. 4b) of criterion candidates, depicting the overall concept structure of all participants. In Fig. 4, the proximity information between concepts is calculated by participants’ responses to the request of grouping issues (column E in Table 1). The proximity, scaling, and clustering analysis followed the following procedure (Tikkanen et al. 2006).

Fig. 3
figure 3

A list of criterion candidates for selecting cases among community-based sustainability initiatives. Candidates are based on the qualitative coding of the issues that researchers of the example project listed in their cognitive maps. Candidates are sorted and labelled according to how many researchers gave issues that were coded into the particular criterion candidate

Full size image
Fig. 4
figure 4

Multi-dimensional scaling (a) and hierarchical clustering (b) of issues that researchers saw as important criterion candidates to be taken into account in case selection. In a, texts in the corners outside the grids indicate diagonal dimensions of interpretation. In b, the numbers in brackets indicate the importance of criterion candidates in the hierarchy levels calculated as means of priority values of criterion candidates in clusters (distribution 0-mean, shown in Fig. 5). Criterion candidates 7 (scientific relevance) and 22 (impacts in general) were excluded from the figures due to lack of specific information

Full size image

Matrix operations were computed with the data producing a symmetric matrix depicting the reciprocal proximity of the criterion candidates on the overall cognitive map. The first stage involved cross-tabulation of the variable CC (27 criterion candidates) with the variable GROUP ID (a nominal variable depicting issues grouped together by respondents into 121 groups). The 27 × 121 matrix thus obtained was transformed into a dichotomous matrix by denoting with 1 all the cells greater than 0. The dichotomous matrix was then denoted by X, and its elements were denoted by xij, in which i = 1, ..., 27 and j = 1, ..., 121. Thus, xij = 1, if the criterion candidate i is in the said group j; otherwise, xij = 0. In matrix X, the row sums Pixij indicate how many groups mention the criterion candidate i. In the next stage, the matrix Y was formed by multiplying the matrix X by its transpose, i.e. Y = XXT. Now, Y is a proximity matrix, whose elements yij, i = 1, ..., 27, j = 1, ..., 27, indicate how many times the interviewees/respondents had combined the criterion candidates i and j in the same group (Scott 1991). Using matrix Y, new proximity matrix Z was formed, and its elements zij, i = 1, ..., 27, j = 1, ..., 27, depict the percentage of the groups containing the said row’s criterion candidate i, and in which the said criterion candidate had been classified into the same group with the column criterion candidate j. In other words, z′ij = y′ij/P′ixij. Because matrix Z is asymmetrical, it needs to be transformed into a symmetrical matrix Z′ such that z′ji = (zij + zji)/2. The analysis does not allow the use of an asymmetrical matrix because belonging to a group is not a directed relation between the objectives. The symmetrized proximity matrix Z′ obtained was used as the base data for the analyses. Before the computations, the diagonal of Z′ was given the value 100 because every criterion candidate will always be grouped in the same group with itself.

The SPSS software and its multi-dimensional scaling PROXSCAL (Busing et al. 1997; Commandeur and Heiser 1993; Anon. 1999, 2014) and a hierarchical cluster analysis using Ward’s linkages were used in mapping and grouping the criterion candidates (Johnson 1967; Anon. 1999, 2014) using Z′ as data. The results of the analysis are presented in the form of a two-dimensional scatterplot (Fig. 4a), with the stress-1 value as a measure of goodness of fit (Anon. 2014), and a dendrogram (Fig. 4b) that graphically illustrates grouping stages when the various criterion candidates have become grouped into the same group.

Calculating priories for criterion candidates

The second phase of the procedure was to calculate the relative importance of criterion candidates and then for computationally constructed clusters of the hierarchical cluster analysis. The priority given to each criterion candidate by each respondent (participant level) was calculated from the initial priorities assessed by the respondents to the issues. If there were several issues from one respondent classified in the same class, i.e. a criterion candidate, then an average was used. Second, because each respondent gave priority assessments only to a part of all 27 criterion candidates, the global (group level) priorities were calculated stochastically.

Missing priority values were estimated for respondents following the procedure presented by Sironen et al. (2013). The procedure is based on Monte-Carlo simulations using distributional probability assumptions of missing values. There are different potential distributional assumptions to guide simulations. In this study, we used the two most prominent distributions according to Sironen et al. (2013), as estimated in the following way.

The priorities from the respondents were used as a basis of the generations, and they were kept fixed in further calculations. Random realisations were simulated for every missing priority of each of the respondents in turn by using two different distributions. To do this, a uniform distribution in an interval from 0 to the mean value of the range each respondent had given for his/her criterion candidates and from 0 to 100 was produced. The random generation was repeated 1000 times for each priority for both methods. Thus, the assigned respondent-level priorities were

\( {\widehat{a}}_i=\exp \left({\widehat{\alpha}}_i\right)/{\sum}_i\exp \left({\widehat{\alpha}}_i\right) \)where \( {\widehat{a}}_i \) denotes the estimate of the priority of the criterion candidate i in the priority scale and\( \widehat{\alpha_i} \)denotes the estimate of the criterion candidate value on the logarithmic scale (see, e.g., Kangas et al. 1998; Alho et al. 2001; Leskinen 2001) for more details, in addition to Sironen et al. 2013). Next, the total weights of the priorities were calculated as an arithmetic mean of the 25 respondents separately for both distributional assumptions. Each respondent received the same weight in these calculations. Priorities for the main clusters of criterion candidates in the cluster analysis were calculated as arithmetic means of criterion candidates in the particular cluster.

Evaluation of the procedure

The evaluation of sCM is based on (i) a feedback survey to researchers and other experts in TESS on CBIs and (ii) evaluation of the effectiveness of the method by comparing the results of sCM with the selection criteria obtained in the intensive and iterative case selection process that lasted almost one year (described in more details in the supplementary materials). sCM was a starting point of this iterative process. The idea of this comparison is that sCM could be seen as functional if it was able to capture essential criterion candidates, the ones that were finally useful in the actualised selection process.

The feedback questionnaire was targeted to all participating researchers immediately after finishing the case selection. It included interval-scale evaluation tasks on the process as a whole and on the applied methods in particular. The methods’ evaluation focused simultaneously on the performance and on the suitability of the method in the semantic differential scale. The ZEF-solutions (2006) survey tool was used in data collection and visualisation of results (Fig. 5b).

Fig. 5
figure 5

Scaled importance of criterion candidates for case selection (a) and researchers’ ex-post evaluation of the methodology of stochastic cognitive mapping (sCM) (b). In a, two distributional assumptions were used to estimate missing values for respondents: uniform distributions in an interval 0 to the mean value of the range each expert had given to the priorities and in an interval 0–100. The importance figures are arranged by the series 0-mean. Numbers in brackets indicate rank of the criterion candidate in different series; Single and double asterisks indicate criterion where ranking order differs considerably between series. Star symbols on the right side of the bars correspond to criterion candidates finally applied as criteria of case selection after all phases of the selection process (see more in the supplementary materials). In b, the evaluation was conducted by a web-based survey tool (ZEF-solutions 2006) with a semantic differential scale. Respondents were asked to graphically locate their evaluation between evaluative words shown in the axis labels. Solid axis lines depict alternative space and dotted lines are in the middle of the continuum (= no positive nor negative connotation)

Full size image

Results

Cognitive mapping results

The respondents listed a total of 361 issues in their cognitive maps, ranging individually from 5 to 33, with the average being 14 issues. Furthermore, respondents grouped those issues into 121 groups. The number of issues in a group varied from 1 to 12, with the average being 6.

Figure 2 gives two examples of different maps constructed by respondents. Those example respondents mentioned a total of 11 (respondent a) and 14 (respondent b) issues to be taken into account when selecting case initiatives for the research project. Respondent (a) is worried about the representativeness and relevance of the initiative data and the generalisation possibilities of the study results, whereas respondent (b) assigns important features of case study initiatives, which are used directly as various criteria of initiatives themselves for selecting the case study initiatives.

Those 361 messages were classified in a data-driven analysis into 27 criterion candidates (see Fig. 3). Most of the respondents emphasised that case initiatives should have upscaling potential and have to give support to the social capital of the community. From the technical criteria of initiative selection, the availability of data was the one mentioned most often.

Two criterion candidates, ‘impacts in general’ and ‘scientific relevance’, were excluded from the following analysis steps approaching the structure behind the criteria set because they were seen as concepts that were too general and so not informative as case selection criteria. Thus, altogether, 25 criterion candidates were targeted for the following analysis step.

Multi-dimensional scaling (PROXCAL) of criterion candidates produced a two-dimensional, qualitatively meaningful interpretable solution, albeit the fit was not very high (Fig. 4a, stress-1 = 0.3015). Labelling the dimensions is based on the analysis of which aspects are common for criterion candidates located at both ends of the continuums. The first dimension (diagonal in Fig. 4a) relates to the sampling strategy, from purposeful searching for avant-garde and influential initiatives to a statistically representative and practically feasible case set. The second dimension locates criterion candidates with respect to the aspects to be used as sampling or searching criteria: on one end of the continuum are criterion candidates describing the initiative’s internal features, while the other end of this continuum is focusing on the potential external impacts of initiatives to innovations, carbon reduction, social impacts, etc. The locations of individual criterion candidates are scattered on the map, and compact clusters cannot be noticed.

Hierarchical clustering of criterion candidates is presented in Fig. 4b. In the uppermost level, clustering separated two clusters consistent with the first dimension of multi-dimensional scaling described above: (1) The rationale for the first cluster can be conducted from a ‘sampling strategy’. It includes criterion candidates describing the representativeness of a case set and the practical feasibility of cases. Additionally, ‘upscaling potential’ and ‘inspire innovations’ were classified in this cluster because they are key aspects to be analysed in the project. (2) The second cluster includes requirements used in ‘searching’ case initiatives.

Two different viewpoints were emphasised as search criteria, indicating different research orientations among the research group. The first viewpoint focuses on cases having positive impacts. The sub-cluster environmental impact is rather tight and clearly visible in clustering results. This orientates the case search towards initiatives indicating a green economy orientation and thus being most potentially successful in terms of carbon reduction and environmental impacts. The sub-cluster around social impacts is looser and includes a variety of aspects towards societal change. Social learning and capital were in this cluster, embedded in internal and external support structures and measured, e.g., by community cohesion and peer networking with other initiatives.

The second viewpoint assigns characteristics for a CBI itself to serve as a case in this particular study. These include openness (transparency), organisational structure, size, establishment and evolution of cases, and existence of noticeable success factors as features that can be used to define requirements of proper CBIs. In addition to these characteristics of proper cases, some called for a more radical orientation towards case selection: initiatives should be approaching radical change in order to be worth studying. Thus, case initiatives should be genuinely grassroots-oriented, emphasise solidarity, and support mainly the local community. Surprisingly, the requirement to focus on mature cases was also clustered together with these radical criterion candidates: mature initiatives that have survived over time maintain a certain stability and changes in power, making them important as a study partner. From another perspective, very new initiatives are not suitable because they cannot provide the data required in the study.

Prioritisation results

The prioritisation task given to respondents was meant to give more precise information on which aspects researchers and experts on CBIs saw as the most important to guide case selection (Fig. 5a).

The researchers who responded considered the following to be the most important criterion candidates to be taken into account in case selection: success factors, commitment to change, environmental impacts, support to community, data availability, and community cohesion. Thus, the selection procedure should seek initiatives that are supposed to be successful in terms of environmental measures and where the internal cohesion of the initiative is tight and members of the initiative are committed to the values of the initiative. Furthermore, proper case initiatives should be genuinely community-oriented, so that gains of CBIs’ actions should stay and spread inside the community. Equity, an external support structure, internal organisation and peer networking, and maturity were not seen as proportionally as important as the ones listed above. These lower priorities indicate that case selection could include both new and mature cases, which may have different organisational principles, and that both autonomous and externally supported initiatives could be accepted as case studies. Surprisingly, equity was the least valued criterion candidate to be considered in case selection.

Distributional assumption did not remarkably influence the ranking order of the criterion candidates, notwithstanding the criterion ‘impacts in general’. Indeed, this is more a common denominator of more specific criteria and cannot serve as a concrete criterion in case selection (thus, this criterion candidate, like scientific relevance, was excluded from MDS and clustering calculations). A similar but milder influence of distributional assumptions was also found regarding the criterion candidates ‘catalyst of change’, ‘variability’, ‘willingness’, and ‘size’. Weights of the these criterion candidates (6–14 in importance order) were so near to each other, especially when using the 0–100 model, that the change in the distribution model caused this jump.

Priority calculations were also conducted for the cluster hierarchies (Fig. 4b). Respondents evaluated practical issues relating to the initiatives’ ability to serve data as the most important viewpoint to be ensured in the case selection process. The second most important aspect was that the initiative set should include initiatives that have positive environmental impacts. Additionally, it was seen as important that the initiative selection also enabled representative and valid results. Issues of radical change and the power of initiatives were seen as least important in the case selection phase, albeit raised on the table on behalf of some researchers in the group.

Evaluation of the stochastic cognitive mapping procedure

More than half (13) of the 25 experts who served as informants on sCM responded to a web-based feedback survey regarding the procedure and its methods (Fig. 5b). All except one saw the overall approach of the case selection procedure as relevant and functional, and sCM was seen as supporting the approach and objectives of the project. Regarding particular methods, a simple cognitive mapping method was considered suitable to the task, and it was also considered as performing well. Evaluations on the prioritisation method of SMART were more neutral. One evaluator did not like numerical prioritisation at all.

Ability of the procedure to identify essential criterion candidates for case selection

The described procedure resulted in a rather logical map of criterion candidates for case selection and priorities for the candidates that served as a starting point for the case selection in a sequence of partner meetings in the example project. To evaluate the capability of sCM to capture relevant selection criterion candidates in the very beginning of the process, the results from the exercise were compared against the final actualised case selection and the criteria used therein. The decision-making regarding cases was an iterative process, where joint valuations and decisions were gradually achieved. Throughout the case selection process, it was necessary to refine the original definitions a number of times (for a more detailed description of the selection process, see the supplementary materials of this study, Haara et al. 2014, and Tikkanen and Haara 2015).

In Fig. 5a, star symbols depict criteria that were finally used in the case selection process. It is evident that the sCM task captured well the selection criterion candidates later applied as criteria. The actualised selection of supportive cases was aimed at fulfilling ‘a sampling strategy’ by taking into account representativeness and feasibility, which were seen as the most important aspects of sCM. The most significant difference between results of sCM and the final outcome of the selection process was that in the beginning of the project, researchers oriented intuitively predominantly towards the search of successful cases, with regard to the main objectives of the research (especially upscaling potential and success factors of initiatives in our example). During the subsequent discourse, upscaling potential and success factors were excluded from the actual selection process. It was considered that it is impossible to approximate success factors beforehand without also having less successful initiatives in the sample. Another difference between results of sCM and its actualisation was in relation to duration as a selection criterion: in the beginning, researchers also wanted to accept very new initiatives into the case set, but preliminary interviews with case candidates quickly showed that cases must have survived some time before they can provide the required data for the research.

Discussion

In this paper, we presented a procedure, sCM, for enhancing the consistent and transparent identification of issues that need to be considered during the case selection process in multi-disciplinary and multi-national research projects on CBIs. In the example project, the researchers displayed two orientations towards the case selection derived from the sCM: (1) From a ‘sampling strategy’ perspective, they requested that the case selection should be representative to ensure the generalisability of the results and they were worried about the availability of data and the initiative’s willingness to participate with the researchers. (2) From a ‘search strategy’ perspective, researchers wanted to select cases that have demonstrated, or that would most likely demonstrate, upscaling potential and positive environmental impacts.

Examples of both strategies are evident in previous CBI studies. The search strategy is commonly applied (e.g. Seyfang 2001; Fudge and Peters 2009; Middlemiss and Parrish 2010; Forrest and Wiek 2015; Saraiva et al. 2016; Burnside-Lawry and Carvalho 2016; Gomez Mestres and Lien 2017; Schlappa 2017). Such a case selection has been labelled, for example, theoretical (Blichfeldt and Halkier 2014) or strategic (Hicks and Ison 2018) sampling. Representativeness, a key argument of the sampling strategy, is a less common objective of case selection of CBIs. Seyfang et al. (2013) used snowball sampling, and both Seyfang and Longhurst (2016) and Morais and Silvestre (2018) first used existing networks to map a CBI population for their studies, and then purposive sampling to select representative cases.

Compared with that in those previous studies, the use of sCM in the present study resulted in the consistent and transparent integration of search and sampling strategies: snowball sampling was used to map the CBI population, and stratified random sampling was used to select cases for the project. Finally, key initiatives were selected from the case set, to be used in the following in-depth qualitative analyses. This was done jointly by the researchers of the project using explicit criteria and a transparent decision-making process.

The most highly ranked case selection criteria by sCM were success factors, commitment to change, environmental impacts, and support of community. These criteria were also listed among the case selection criteria used in the study conducted by Forrest and Wiek (2015), who have argued that more can be learned from successful cases than from unsuccessful ones. Several of their other criteria (i.e. sustainability as a primary goal, size, and the notion that the case CBIs should be well established) were among the criterion candidates in the present study as well, although not near the top in the ranking order. Necessary pragmatic criteria, such as data availability and CBI’s willingness, have further been discussed in previous studies (e.g. Seyfang and Longhurst 2016), as well as in the sCM example of this study. The use of the sCM provided a more nuanced picture than previous studies about the case selection and researchers’ different intentions towards CBI research. Respondents listed a wide collection of attributes that constitute a good case initiative, including inclusiveness, openness, and ability to inspire innovations. Several researchers emphasised that some initiatives in the case set should exhibit a radical orientation towards societal change.

Thus, the common ground identification by sCM raised somewhat competing and conflicting views to the case selection; however, these views are consistent with two views found in other studies on case study research, namely, intentional focus on ‘novel and avant-garde’ cases (Martin and Sunley 2001) and a call for representativeness in statistical terms (James 2006) without consistent consideration for the underlying rationale of the study. Thus, our results reinforce the idea that joint discussion of case selection strategy requires significant attention in the early phases of multi-disciplinary research projects on societal issues. For example, relying on preexisting networks of initiatives in case selection may not result in the best case set in studies on CBIs. Therefore, decision-support procedures and methods, such as sCM, could contribute to discussions on case selection.

sCM integrates email enquiry, SMART, and a stochastic analysis of missing priority values with previously developed 3CM cognitive mapping (Kearney and Kaplan 1997). According to the feedback results, participants gave a positive evaluation for the sCM procedure. In particular, the email-based cognitive mapping task was considered to be beneficial. Opinions of the quantitative ranking tasks were more diverse. This finding requires further study. The sCM procedure managed to capture the majority of criterion candidates chosen as criteria for the case selection in the project. While we cannot explicitly evaluate to what extent the multi-stage selection procedure was finally influenced by the sCM, it is evident that the actualised procedure is unique in applying snowball sampling to define case population and in using different sets of criteria to narrow down the sampling to two samples for predominantly different research tasks.

The procedure of sCM was based on the qualitative coding of rather short responses that participants provided through the email inquiry. Such an analysis is always subjective by nature. In this study, two researchers first independently performed the analysis and then verified the differently understood classifications and jointly re-iterated the analysis (see more about triangulation options for qualitative content analysis in Denzin 2009).

The main results of the sCM are visual presentations of multi-dimensional scaling by PROXCAL and a hierarchical cluster analysis by Ward’s method. Those algorithms are widely applied in statistical packages. PROXCAL was originally developed for the symmetric data structure (Commandeur and Heiser 1993) and thus is a natural choice for sCM. In this study, the stress value of the scaling remained relatively high. In the methodological literature, there is no unanimous guidance on a sufficient stress level. In addition, critiques towards scaling algorithms’ capability to reach a global optimum have been presented (e.g. Everitt and Rabe-Hesketh 1997). A recommended solution is to assign the same data for the analysis with different methods. In this study, consistent and transparent comparisons between results of MDS and hierarchical clustering increased the reliability of the findings.

The stochastic procedure of sCM for calculating missing priority values for respondents applied the distributional assumptions proposed by Sironen et al. (2013). The estimation of priority values to criterion candidates, that the particular respondent did not evaluate, was conducted according to the priorities given by the same respondent on other candidates. Further testing and methodological development is needed regarding other distributional assumptions.

Based on the results, we recommend a relevant mixed-method procedure, such as sCM, to be applied as a first procedure to identify common ground for case selection in large research projects. The main advantage of sCM is that it does not restrict concepts and wording to be raised in a discussion. Another advantage is that the respondents can conduct prioritisation simultaneously with an individual cognitive mapping task, in contrast to most other multi-criteria analysis methods that separate problem structuring and prioritisation phases (e.g. Marttunen et al. 2016). A possible disadvantage of sCM is related to the coding of rather short qualitative messages by the analysts. This may include misunderstandings and simplify diversity of thinking unnecessarily which can result in exclusion of essential aspects from the discussion. sCM, like any other similar approach, is meant to be used to support a collaborative learning among researchers of case studies. Even though we did not measure learning outcomes of the sCM, we expect that a careful collaborative reflection on the results of sCM could shorten the time needed in the learning process and slacken potentially conflicting preassumptions about the role of cases among the researchers.

Conclusions

The applied sCM procedure resulted in context-specific case selection criteria in the study of the role of CBIs in the shift towards sustainable societies. The procedure raised different perspectives and transparency in discussions about the objectives and methodological needs of the research and formed a good basis for selecting research cases. Therefore, the reported criteria are also applicable to similar case studies on CBIs.

According to the experiences of this study, sCM is a promising procedure to be applied in the beginning phase of multiple case study selections. In particular, an email inquiry for the 3CM cognitive mapping worked well and enabled partners to conceptualise different starting points, hopes, and worries in regard to the case selection. The method uses a normal spreadsheet and thus is easily adaptable. A specific graphical interface for the sCM tasks should make the procedure even more attractive and functional.

The sCM procedure does not produce directly usable criteria for case selection. It is rather meant to create a foundation upon which the discussion and iterative learning process for case setting will be based. The procedure should lead to consistent sequences of project meetings, including prioritising case selection criteria, piloting case interviews, and random and or purposive samplings.

In addition to case study selection, the methodology of sCM might also contribute to many other decision-making processes in which there is a need to jointly define criteria for decision-making with a diverse group of participants. For example, this procedure may contribute to the writing process of large multi-national research project applications, and it can be adapted for project evaluators in search of potentially successful research applications, or even in the ex-post evaluation of a project’s achievements.