Motivational theorists have long subscribed to the idea that human behavior is fundamentally driven by needs and goals. A goal perspective provides us with insights on the organization of affect, cognition, and behavior in specific contexts, and how these may change depending on different goals (Dweck 1992). The interest in goals is also prominent in the educational realm, which led to a boom of research with a focus on achievement goals (Kiefer and Ryan 2008; Mansfield 2012). Although this research provided significant insights into the role of motivation, it does not provide a holistic view of the goals pursued in academic contexts. Students pursue multiple goals in the classroom (Lemos 1996; Mansfield 2009, 2010, 2012; Solmon 2006), all of which need to be considered to understand students’ motivations and behaviors. Many prominent researchers argue that social goals should be regarded with the same importance as achievement goals (e.g., Covington 2000; Dowson and McInerney 2001; Urdan and Maehr 1995), as they too have implications for academic adjustment and success. For instance, studies have shown that social goals are related to academic achievement (Anderman and Anderman 1999), school engagement (Kiefer and Ryan 2008; Shim et al. 2013), academic help-seeking (Roussel et al. 2011; Ryan and Shin 2011), and learning strategies (King and Ganotice 2014). At this point it should be noted that the term social goal is a rather broad term, under which many types of social goals fall (e.g., prosocial, popularity, status, social development goals). Urdan and Maehr (1995) stated that there is a critical need for research to untangle and investigate the various social goals, as these could have different consequences for students’ motivation and behavior.

Intrigued by social goals and their role in socio-academic contexts, we opted to pursue this line of research for a larger project. At the beginning of every research endeavor, familiarization with the relevant theories and current research is essential. It is an important step before conducting primary research as unnecessary research is avoided, current knowledge gaps are exposed, and it can help with the interpretation of later findings. Furthermore, funding bodies that provide research grants often require a literature review to assess the significance of the proposed project (Siddaway et al. 2018). Customary within our research group, this process is completed by conducting a thorough systematic review. In addition to being beneficial merely to the authors, systematic reviews also provide other researchers and practitioners with a clear summary of findings and critical reflections thereof. Considering that the research on social goals and academic success dates back nearly 30 years, we deemed this to be an ideal time to provide a systematic overview of the entire research.

1 Purpose of Review

Our main aim was to produce a comprehensive review, which adequately displays the significance of social goals for academic success. Gough et al. (2012) describe the different types of systematic reviews by exploring their aims and approaches, which include the role of theory, aggregative versus configurative reviews, and further ideological and theoretical assumptions. The purpose of the current review was to further theoretical understanding of the current phenomena by developing concepts and arranging information in a configurative way. As such we were interested in exploratively investigating the role of social goals on academic success, by identifying patterns in a heterogeneous range of empirical findings. With such reviews, the review question is rather open, concepts are emergent, procedures less formal, theoretical inferences are drawn, and insightful information is synthesized (Brunton et al. 2017). Levinsson and Prøitz (2017) found that configurative reviews are rarely used in education, although they can be very beneficial for academic researchers, especially at the start of new research projects. Commonly researchers gather information from introductory sections of empirical journal articles, without considering that this information is cherry-picked to support the rationale and hypotheses of a research study. In order to thoroughly inform research and practice, a configurative and systematic summary of empirical findings are needed.

2 Methods

Although the aim of our systematic review was to learn something new about the relation between social goals and academic success, we did not tread into the process blindly. A paramount yet often overlooked step in the systematic review process is the exploration of relevant theoretical frameworks. Even though the theoretical framework does not need to be explicitly stated in the systematic review, it is of essential importance as it lays the foundation for every step of the process (Grant and Osanloo 2014). We thus first spent time understanding the theoretical backgrounds and approaches with which prominent research articles explored social goals and investigated their relation to academic success. This initial step helped us throughout review process, as it allowed us to better understanding the research questions and results presented in the articles, revealed interconnections with bordering topics, and gave us a more structured thought process. Naturally during the course of the systematic review, we underwent a learning process in which we gathered new theoretical knowledge and also updated previously held notions.

Before starting with the systematic review, we checked whether there are already existing reviews on the topic. We initially checked the Cochrane Database of Systematic Reviews and PROSPERO, which revealed no registered systematic reviews on the current topic. Being skeptical that these databases include non-medical reviews or that social scientists would register their reviews on these databases, we opted to search for existing systematic reviews on social goals through the Web of Science Core Collection. We identified two narrative reviews, which specifically related to social goals in the academic context (Dawes 2017; Urdan and Maehr 1995), and one narrative review on social goals (Erdley and Asher 1999). We decided that a current systematic review was warranted to provide an updated and more holistic view of the literature on social goals in relation to academic adjustment and success. We drafted a protocol, which included background and aims of the review, as well as selection criteria, search strategy, screening and data extraction methods, and plan for data synthesis. As our research agenda follows a configurative approach, we adapted the protocol iteratively when certain methods and procedures were found to be incompatible (see Gough et al. 2012); these changes are reflected transparently throughout the review. We did not register the systematic review on any database; upon request to the corresponding author the protocol can be acquired.

3 Literature Search

Systematic review searches should be objective, rigorous, and inclusive, yet also achieve a balance between comprehensiveness and relevance (Booth 2011; Owen et al. 2016). Selecting the “right” keywords that find this balance is not always easy and may require more thought than simply using the terms of the research question. A particular problem within psychology and educational research may lie within the used constructs, as the (in)consistency in terminology, definition, and content of constructs is a plight known to many researchers. The déjà-variable (Hagger 2014) and the jangle fallacy (Block 1995) are phenomena in which similar constructs are referred to by different names; this presents a particular challenge for systematic reviews, as entire literatures may be neglected if only a surface approach is taken to identify construct terminologies (Hagger 2014). Researchers may also be lured into relying on hierarchical or umbrella terms, in which a range of common concepts are covered with a single word. This is problematic, as literature which uses specific and detailed terminology instead of umbrella terms will be overlooked.

We were faced with such dilemmas when we decided to embark on a systematic review which investigates social goals; relying on the one term (and its synonyms) was deemed insufficient to comprehensively extract all appropriate articles. We thus referred back to the three identified reviews on the topic and systematically extracted all types of social goals that were mention in these articles. As the dates of the reviews range from 1995 to 2017, we assumed that they would encompass a range of approaches and specific social goals. We acknowledge that this is by no means extensive and other conceptualizations of goals exist (e.g., Chulef et al. 2001; Ford 1992; McCollum 2005). Nonetheless our search can be deemed both systematic and comprehensive, and resulted in 42 keywords for the term social goals. This large number of keywords might seem unusual, but specifically address our quest to investigate the role of various social goals for student’s academic success (as requested by Urdan and Maehr 1995).

For the second part of the search string, we used general keywords contextual to the field of academia to reflect the differential definitions and operationalizations that exist of academic success (e.g., achievement, effort, engagement). Keeping the keywords for academic success broad, meant our systematic review would take on a rather open nature. This delineates from most other reviews in which the outcome is more narrowly set. In retrospect, we found this to be quite effortful as we had to keep updating our own conceptualization of academic success and apply these to further decision-making processes. Nonetheless, we maintain that this allowed us to develop a well-rounded systematic review, in which our pre-existing knowledge did not bias our exploration of the topic. In Appendix A are our final keywords, embedded in a Boolean search string as they were used. In addition to combining or terms with the OR and AND operators, we added an asterisk (*) to the term goal to include single and plural forms.

To locate relevant articles, in March 2018 we entered our search string in the following electronic bibliographic databases: Web of Science Core Collection, Scopus, and PsycINFO. These were entered as free-text terms and thus applied to title, abstract, and keywords (depending on database). It is advisable to use multiple databases, as variations in content, journals, and period covered exists even in renowned scientific electronic databases (Falagas et al. 2008). In January 2019 we conducted an update as our initial search was more than six months ago; we entered the same keywords into Web of Science Core Collection and Scopus.

4 Selection Criteria

To be included in this review, articles were required to

  • Be relevant for the topic under investigation. Commonly excluded for being topic irrelevant were articles, which used the social goal keywords in a different way (e.g., global education goals), or only focused on social goals without explicating any academic relevance (e.g., social goals of bullies), or focused on non-social achievement goals (e.g., mastery goals).

  • Be published in a peer-reviewed journal. Dissertations, conference papers, editorials, books, and book chapters were excluded. If after extensive research it was unclear whether or not a journal was peer-reviewed it was excluded.

  • Report empirical studies. Review articles and theoretical papers were excluded.

  • To constitute of a population of students, articles which focused on teachers or parents were excluded. No age restrictions were set.

  • Not examine special populations (e.g., children with disabilities, ADHD).

  • Provide full-texts in English. We acknowledge that this introduces a language bias, but due to limited resources we were unable to translate non-English articles. Although both authors are bilingual, we felt the inclusion of specifically German articles would arguably introduce more bias.

We opted not to impose a publication date restriction; thus, the search covered articles from the first available date until March 2018.

5 Study Selection

Appendix B provides a flow diagram of the study selection process, which has been adapted from Moher et al. (2009). All potential articles obtained via the electronic database searches were imported into EPPI-Reviewer 4, and duplicate articles were removed. A title and abstract screening ensued, which resulted in the exclusion of all articles that did not meet the selection criteria. If these did not provide sufficient information, the article was shifted into the next phase. For articles that were excluded on the bases that they were not empirical, a backward reference list checking was conducted. Specifically, the titles of articles in the listed references were screened and resulted in the addition of a few new articles. Reference list checking is acknowledged as a worthwhile component of a balanced search strategy in numerous systematic review guidelines (Atkinson et al. 2015).

We were able to locate all but three articles via university libraries and online searches (e.g., ResearchGate). Full-text versions of the preliminarily included articles were obtained and screened for eligibility based on the same selection criteria. Wishing to explore research gaps in the area, as well as having an interest in the developmental changes of social goals, we originally intended to keep the level of education very broad (primary, secondary, and tertiary). During the full-text screening we were however reminded of the dissimilarities between these academic contexts, as well as school and university students; we also realized that the addressed research questions in tertiary education varied from the rest (e.g., cheating behavior, cross-cultural adjustment). Thus, articles dealing with tertiary education students were also eliminated at this point.

Although we initially planned to include both quantitative and qualitative articles, we came to realize during the full-text screening that this may be more problematic than first anticipated. Qualitative articles are often excluded from systematic reviews, although their use can increase the worth and understanding of synthesized results (CRD 2009; Dixon-Woods et al. 2006; Sheldon 2005). While strides have been made in guiding the systematic review process of qualitative research, epistemological and methodological challenges remain prominent (CRD 2009; Dixon-Woods et al. 2006). Reviewing quantitative research in conjunction with qualitative data is even more challenging, as qualitative and quantitative research varies in epistemological, theoretical, and methodological underpinnings (Yilmaz 2013). With an increased interest in mixed-methods research (Johnson and Onwuegbuzie 2004; Morgan 2007), the development of appropriate systematic review methodologies needs to be boosted. Due to the differential methodologies described for systematic reviews of qualitative and quantitative articles and a lack of clear guidance concerning their convergence, we opted to exclude all qualitative and mixed-method articles at this stage. To not lose vital information provided by these qualitative articles, we incorporated some of their findings into other sections of the review (e.g., introduction).

During the full-text screening we came to realize that we had a rather idealistic plan of conducting a comprehensive yet broad systematic review. To not compromise on the depth of the review, we opted to narrow the breadth of the review. Nonetheless, having a broader initial review question subsequently followed by a narrower one, allows us to create a synthesis in which studies can be understood within a wider context of research topics and methods (see Gough et al. 2012). Furthermore, we maintain that the inclusion of multiple social goals as well as different academic success and adjustment variables still provides a relatively broad information bank, from which theories can be explored and developed. Our review thus followed an iterative yet systematic process.

6 Data Extraction

We created an initial codebook, which included numerous categories of information to be extracted from each article. Piloting the codebook ensures that all relevant data is captured and that resources are not wasted on extracting unrequired information (CRD 2009). After piloting the codebook on some of the included articles we realized adaptations needed to be made. We carefully deliberated on which information needs to be extracted to accurately map the articles, indicate research gaps, and provide relevant information for a well-rounded synthesis on the current topic. Information regarding the identification of articles was already incorporated in EPPI-Reviewer when articles were first identified. We included both open and categorical coding schemes to extract theoretical information (i.e., overarching aim, social goal type and approach, research questions, hypotheses), participant details (i.e., number, age range, education level, continent of study), methodological aspects (i.e., design, time periods, variables, social goal measurement tools), and findings (i.e., main results, short conclusions).

Extracting theoretical information from the articles, such as the overarching aim and research questions was fairly simple. Finding the suggested hypotheses was a bit more complicated, as many articles did not explicitly report these in one section. Surprisingly, almost a third of the articles did not mention a priori hypotheses. Uniformly extracting the description of the participants also required some maneuvering. We found that the seemingly simple step of extracting the number of participants required some tact, as articles differentially reported these numbers (e.g., before-after exclusion, attrition, multiple studies). Studies differentially described the age of participants, with some reporting only the mean, others the age range, and some not mentioning the age at all (i.e., reporting only the grade level). We opted not to extract additional descriptive participant data, such as socio-economic status and sex-ratio, as these were not central to the posed research question and results of the included articles. Attributing study design was easily completed with a closed categorical coding scheme, whilst listing all the included variables required an open coding scheme.

Extracting which measurements (i.e., scales and questionnaires) were used to assess social goals with their respective references was constructive for our review. Engaging with the operationalizations provided us with a deeper understanding of the concepts, lead to new insights about the various forms of conceptualizations, and also revealed stark inconsistencies albeit sharing the same term. To extract the main results, we combed through the results section of the article, whilst at the same time having the research question(s) at hand. With this strategy we did not extract information about the descriptive or preliminary analyses, but specifically focused on the important analyses pertaining only to social goals. Although we did not extract any information from the discussion, we did find it useful to read this section as it provided us with confirmation that we extracted the main results correctly and allowed us to place them in a bigger theoretical context. As a demonstrative example, Table 1 shows a summary of some of the extracted data from five of the included articles.

Table 1 Summary of extracted data from five reviewed articles
Full size table

7 Synthesis

In our protocol we stated that we would conduct a narrative synthesis, as this would be most appropriate to the array of (quantitative) studies we hoped to include in the systematic review. A narrative synthesis is a textual approach to the systematic review, which involves summarizing and explaining the findings of multiple studies with primarily words and text (Popay et al. 2006). We have since come to realize that the term ‘narrative synthesis’ is rather generic, describing a collection of methods for synthesizing data narratively (Snilstveit et al. 2012). Upon inspection of the range of available methods (see Barnett-Page and Thomas 2009; Dixon-Woods et al. 2005), we decided that we would use a thematic analysis (synthesis), as it is a good method when dealing with a broad range of findings. A thematic analysis involves creating summaries of prominent and recurrent themes in the articles in a systematic way. We aimed to create an intertwined web of results from all the studies.

The synthesis is probably the most cumbersome step in the systematic review, as the content, results, and surrounding theories become central and generic guidelines can only be adopted to a certain extent. A challenge in the thematic synthesis was that educational and psychological studies often boasted a high number of variables and investigated complicated relations. We found that only few studies included the same social goals and academic outcomes, and the ones that did often reported contradictory results. Although “vote counting” has received some criticism, Popay et al. (2006) describe it as a useful descriptive tool in which studies are categorized as showing significant or non-significant results. However, due to the reported influences of various individual and contextual factors drawing such simple conclusions was not easy. To do justice to the articles, we additionally had to find a balance between elaborating and highlighting the key results. A common fallacy during the synthesis is simply summarizing the findings from each study, without reaching a meta-perspective. Siddaway et al. (2018) maintain that the findings need to be interpreted, integrated, and critiqued in order to advance theoretical understanding.

8 Risk of Bias and Quality Assessment

Upon inspection of popular risk of bias (e.g., Cochrane Collaboration Risk of Bias Tool) and quality assessment tools (e.g., NHLBI and STROBE checklists), we found these to be unsuitable for the majority of articles included in the current systematic review. We were unable to apply these tools, originally developed for randomized controlled trials in the health sciences, without tweaks to non-experimental social science studies. Revising these tools was deemed beyond the scope of the current systematic review. Interestingly, a moderate portion of systematic reviews do not conduct risk of bias analyses and many syntheses remain uninformed by the results of such analyses (Katikireddi et al. 2015). Some authors and methodologists reject the idea that a quality assessment needs to be conducted for articles that are included in configurative reviews, instead highlighting the need to prioritize relevance and contribution towards the synthesis (see Gough et al. 2012). As our review attempts to explore and generate theories on social goals in the academic context, we place a higher value on emergent concepts through a range of study contributions than precision by avoiding bias.

Furthermore, for a study to be included in our review it needed to be published in a peer-reviewed journal. Peer-reviews help validate research and raise the quality of articles by increasing robustness, legibility, and usefulness (Springer International Publishing AG 2018). Peer-reviews usually address aspects reflected in traditional quality assessment tools, such as reporting, validity, statistical tools, and interpretations (see Ramos-Álvarez et al. 2008). Although there is no guarantee that individual peer-reviewers adequately scrutinize each article, it has become a well-established method that the scientific community relies on. Quality assessment and risk of bias tools can also not account for frequently committed questionable research practices, such as selective reporting of variables, rounding down p-values, adjusting hypotheses after analyzing results, or falsifying data (see John et al. 2012).

9 Quality Assurance of the Systematic Review

The PRISMA statement is not a quality assurance instrument but does provide authors with a guide on how to transparently and excellently report their systematic review (Moher et al. 2009). The checklist provides a simple list with points corresponding to each section of the review (e.g., title—convey the type of review, information sources—name all databases and date searched). The majority of the items can be easily implemented, even for reviews within the field of psychology and education research. We followed this checklist and only deviated on certain points, such as those that referred to PICOS as it does not align with our review question. PICO(S) is limited in its applicability to reviews whose aim is not to assess the impact of an intervention (Brunton et al. 2017).

10 Experience and Communication

Some guidelines on systematic reviews propose that authors not only provide detailed descriptions of the review process, but also information about their experience with systematic reviews (see Atkinson et al. 2015). Our review team consisted of the two authors who worked closely together throughout the process. The second author has published and supervised numerous systematic reviews, whilst this is the first systematic review conducted by the first author. While an expert brings knowledge and skills, a novice viewpoint can ensure that the continuously advancing methods and tools for conducting a systematic review are incorporated into the process. As with any research endeavors, critical discussions, experience sharing, and help-seeking form part of the systematic review process. Working on a systematic review can at times feel tedious and endless, yet simply discussing the steps and challenges with others provides a new boost of enthusiasm. Conducting a systematic review is a time-consuming endeavor, which is not comparable to the process of writing an empirical article. Although numerous books and articles exist for self-study, having contact with an experienced author is invaluable.

11 Conclusion

This chapter details a current systematic review conducted in the realm of educational research concerning the role of social goals in academic adjustment and success. Unfortunately, reporting the findings of our systematic review and our synthesis is beyond the scope of the current chapter. Yet through methodological reflections and explicit descriptions, we hope to provide guidance and inspiration to researchers who wish to conduct a systematic review. In our example, we illustrate a possible strategy for keyword selection, setting selection criteria, conducting the study selection and data extraction. Once a precise question or aim has been set, selecting the keywords becomes a critical point with important consequences for the progression of the systematic review; unsuitable and/or limited keywords result in the loss of a comprehensive perspective that will pervade throughout the review. We recommend that selecting the keywords should be an iterative process, accompanied by careful consideration and reflection. Throughout the review process, each new stage should be accompanied by a pilot phase to ensure appropriateness as new insights emerge (e.g., selection criteria, data extraction, thematic synthesis). We also wish to highlight the importance of moving beyond mere summarizing of studies in systematic reviews, and instead striving for a meta-perspective that allows for the results to contribute to a larger theoretical and practical context. Whilst conducting a configurative review, the initial protocol should not be viewed as a restraint; we were able to adjust the review process to the emerging needs and information obtained during the individual steps. Intensive reflection and meticulous documentation allow for necessary flexibility during the review process, whilst remaining systematic. The configurative approach is well suited for synthesizing the comprehensive and diverse studies often encountered in educational research, and could prove to be useful for future systematic reviews in the field.