Keywords

1 Introduction

Research is an estimated $1.6T/year world enterprise [1], supporting a community of approximately 11 million active researchers [2] and, most importantly, fueling a large fraction of wealth creation in our modern economy [3]. Despite its importance, however, it is practiced largely as an “art” [4], passed down from one generation to the next. We learn how to do research from our professors, managers, mentors and fellow researchers, just as they did from theirs.

In recent years, a community has been growing around a field that might broadly be called the “science” of research [5–8] – the understanding of the human and intellectual processes associated with research and its societal impact. Until now, however, the two communities (the practitioners or “artists” of research and the “scientists” of research) have advanced with minimal interaction, despite the possibility that they might benefit each other enormously. Artists of research care deeply about how effective they are, and what better way to improve their effectiveness than to apply scientific principles; while scientists of research care deeply about their scientific under-standing of research, and what better way to test that understanding than to try to apply it to improving how research is actually done.

The Art & Science of Science and Technology Forum & Roundtable held at Sandia National Laboratories [9] acknowledges opportunities to discover and apply principles governing effective research throughout the research environment. To expand upon those principles laid forward from the Forum & Roundtable, we will focus on two research hypotheses and objectives.

Our first working hypothesis is that divergent and convergent thinking [10] are the foundational yin and yang of research. Research, broadly defined as the production of new and useful ideas and knowledge, proceeds via iterative, nested, and complementary cycles of idea generation followed by idea filtering, refining and retention [11]. For simplicity, we use for these complementary cycles the common terms divergent and convergent thinking, with the understanding that they are related (but not identical) to other terms: blind variation and selective retention (BVSR) [12, 13], abductive versus deductive reasoning, generative versus analytic thinking, discovery versus hypothesis-driven science [14], creativity versus intelligence, thinking fast versus thinking slow [15], foraging versus sensemaking [16], exploration versus exploitation [17], and learning versus performing [18].

Fig. 1.
figure 1

Our working hypothesis is that research proceeds as iterative, nested, and complementary cycles of divergent and convergent thinking. Shown is a schematic of one such cycle, along with examples of the challenges that each kind of thinking faces at the three levels of the research enterprise: the cognitive science level of the individual researcher; the social network level of the research team; and the “research ecosystem” level of the research institution and the larger national and international community of researchers.

Full size image

Our second working hypothesis is that divergent and convergent thinking occur at multiple levels of the research enterprise: the cognitive science level of the individual researcher; the social network level of the research team; and the “research eco-system” level consisting of the research institution and the larger national and international community of researchers. These three levels map to the micro, meso, and macro scales associated with research [5].

At all three levels of the research enterprise (individual researchers, research team, research ecosystem), we give examples of challenges to divergent and convergent thinking faced by the art of research – challenges that in turn represent opportunities for the harnessing of the science of research. Our hope is first, by articulating these challenges and opportunities, to catalyze active work on them in both communities and, second, by articulating these working hypotheses, to catalyze active work to test them (Fig. 1).

2 Individual Researchers: Overcoming Human Cognitive Constraints and Biases

2.1 Divergent Thinking: Overcoming Idea Fixation

Divergent thinking, in essence, is the creation of new ideas, mostly (perhaps always [19]) through the recombination of pre-existing ideas. But humans have cognitive constraints and biases [15] which can make divergent thinking difficult, among them idea fixation [20]. Though extremely productive researchers de-fixate themselves at key stages of their research process, such researchers are rare and are generally unable to teach others how to do the same.

There is thus opportunity for understanding the cognitive basis for idea fixation and then engineering strategies for idea de-fixation. Of particular interest are strategies associated with engineered exposure to new ideas. The ideas should be far enough away in analogical space [21] to catalyze shifts in perspective – either because they come from different disciplines or from different “translational” (science, technology, applications) communities. The ideas should not be so far away in analogical space, however, that conceptual and language gaps are too difficult to bridge.

This exposure to “optimal-analogic- distance ideas” strategy seems obvious in principle, and is in fact practiced by many in a qualitative way. However, advances in modern data analytics, combined with the sheer quantity of digitized knowledge, open up new opportunities for making this practice more quantitative. One opportunity might be scientometric clustering analyses of publications based on bibliographic connectivity. Another opportunity might be lexical clustering analyses based on syntactic/semantic regularities [22], word-order-based discovery of underlying (“latent”) constituent topic areas [23], and mutual compressibility [24]. These analyses could lead to algorithms that go beyond those that power today’s search [25] and recommendation [26] engines by feeding researchers ideas not just within their comfort zone, but optimally distant from their comfort zone.

2.2 Convergent Thinking: Overcoming Sloppy Thinking

Convergent thinking is the selection from newly generated ideas those worth pursuing through logic and analysis. Of course, easier said than done, because human cognition is subject to sloppy thinking and errors of logic and analysis.

There is thus opportunity for understanding the cognitive basis for these errors and for developing strategies to correct for them. Of particular interest is a strategy that might be called the “research narrative” strategy. Research narratives – storylines which knit together background, hypothesis, methodology, analysis, findings and implications – are essentially tools for logical thinking. They are important at the end of a research project, when a paper is being written for the scientific community and posterity. But research narratives are just as important at the beginning of a research project. Emerging cognitive science suggests that narrative and stories are the evolutionary optimal tools for communicating not only with others but even with ourselves [27]. A coarse storyboard of the title, abstract, figures, and key references of the anticipated outcome of a research project forces clarification of many of its aspects – including those that have been hypothesized [13] to be critical sub-components of creativity, such as originality, perceived utility, and surprisingness.

This “research narrative” strategy seems obvious in principle, and, just as the “exposure to optimal-analogic-distance ideas” strategy just discussed, is also practiced by many in a qualitative way. But modern data analytics creates opportunity to practice it more quantitatively. For example, machines might someday dispassionately evaluate research narratives just as they are beginning to dispassionately evaluate essays in academic writing courses [28]. Or, perhaps more likely, a combination of machines and humans might someday efficiently and accurately evaluate research narratives via machine curation of Yelp-like peer reviews.

2.3 Balancing Divergent and Convergent Thinking

Divergent and convergent thinking are by themselves difficult, but perhaps even more difficult is our ability to know when to switch between the two. On a large scale, the history of science is replete with scientists who were on the wrong track and would have been more productive switching from convergent to divergent thinking [29]. But the history of science also has its share of scientists who prematurely abandoned ideas which later proved to be correct [30].

On a small scale, researchers with an immediate narrative of what to expect in one day’s laboratory experiment or theoretical calculation, upon being confronted with something unexpected, must choose whether to stay the course (convergent thinking), whether to treat the unexpected as an opportunity to reconsider possibilities (divergent thinking), or whether to withhold judgment while waiting for additional data or in-sight.

To some extent, we all gravitate towards thinking styles with which we are most comfortable, and researchers are no different. Those who are more comfortable thinking divergently will tend to reconsider too soon; those more comfortable thinking convergently will tend to stay the course too long; and perhaps a rare few will be comfortable doing neither.

In fact, because of our modern education system’s emphasis on deducing single answers using logical thinking, modern researchers might be biased towards convergent thinking. To avoid this bias, some institutions that value creativity now deliberately hire on the basis not of grade point average and SAT scores, but of more balanced thinking styles [31].

There is thus opportunity to understand and engineer strategies to compensate for intrinsic biases towards either divergent or convergent thinking. For example, at a qualitative level, the research narratives discussed earlier might not just be powerful tools for logical, convergent thinking, but might also be powerful tools for under-standing when to cycle between divergent and convergent thinking. If the train of thought that follows from one or more divergent new ideas does not hold up to the cold logic (or mathematics) of the research narrative, then it very likely would benefit from new ideas and divergent thinking.

At a quantitative level, some of the lexical analytical techniques mentioned earlier, applied in real time to evolving research narratives and other generated knowledge trails, might be able to discover not only whether divergent or convergent thinking is happening, but whether divergent or convergent thinking is appropriate for the stage of the problem at hand.

3 Research Teams: Overcoming Social Constraints and Biases

3.1 Divergent Thinking: Overcoming Strong Links

Groups can draw upon the diverse ideas of individuals to create new ideas. And, because much of the knowledge of individuals is tacit [32] and not accessible in formal codified form, closely interacting groups which can share this tacit knowledge informally can be yet more productive. MIT’s Building 20 [33], Bell Labs’ “Infinite Corridor” [9], the Janelia Farm Research Campus [34], Pixar’s Emeryville campus [35] and Las Vegas’ Downtown Project [36] are examples of how informal interactions probabilistically enhanced through intentionally engineered or serendipitously designed physical spaces are thought to enhance divergent thinking and tighter communities through understanding the psychosocial space as clearly as the physical space [37].

However, research teams also bring inefficiencies to divergent thinking. When individuals on a team become too familiar with each other’s knowledge domains and ways of thinking, they no longer serve as sources of new ideas to each other. Moreover, homophily is common in social networks: we seek those who think as we do and avoid those who do not think as we do [38]. For divergent thinking, exposure to the less familiar is important, and weak links [39] in one’s social network can be more powerful than strong links.

Thus, similar to the opportunity identified at the individual researcher level, data analytics may provide an opportunity to identify not just ideas that are an optimal analogic distance away from the current team’s ideas, but people who are an optimal analogic distance away from people in the current team.

3.2 Convergent Thinking: Overcoming Groupthink

Just as with divergent thinking, convergent thinking in research teams can in some situations be more but in other situations be less productive than convergent thinking in individual researchers.

Convergent thinking requires logical deductive thinking, the deeper and more first- principles the more accurate and often the more surprising [40]. On the one hand, multiple minds can find and fix reasoning errors to which an individual researcher might be blind [41]. On the other hand, the depth of knowledge necessary for such thinking, and the degree to which the knowledge is tacit and difficult to articulate, the more easily it can be done within a single mind than across multiple minds. For example, the research narrative discussed earlier, a powerful tool for convergent thinking, benefits from input and criticism from multiple minds but in the end is usually most tightly and coherently articulated by fewer, even single, minds.

Moreover, teams are subject to group- think, in which groups, because of various social biases, converge prematurely and inaccurately on less-good ideas [42]. Such social biases probably evolved in humanity’s pre-history for good reason: there are many situations for which quick consensus, conflict avoidance, and social cohesion are more important than accuracy. Those situations likely do not include among them research and the accurate convergence onto the best ideas, however. There is thus opportunity to understand the conditions under which teams can over-come groupthink.

3.3 Distributing Divergent and Convergent Thinking Between Individuals and Teams

Most importantly, research teams have more options for accomplishing divergent and convergent thinking than do individuals. Teams are composed of individuals. Hence, if some aspect of thinking is best done by a team or by individuals, teams can in principle assign it to the appropriate level. For example, if individuals are indeed relatively stronger at convergent thinking while teams are relatively stronger at diver-gent thinking, it could be optimal for divergent thinking to be performed more at the team level, but for convergent thinking to be performed more at the individual level [43]. To take advantage of this strategy, however, it will be necessary to first under-stand more deeply the relative strengths and weakness of individuals and teams at convergent and divergent thinking for what types of problems, in what situations and environments, and using what interaction tools.

Teams also have more options in how their individual members are rewarded. Individual researchers not in a team would individually bear the consequences of risky too-divergent thinking, but in a team could actually be rewarded for taking on such risk. However, research teams have fewer options for oscillating back and forth between divergent and convergent thinking during the life cycle of a research project. They inherently have more inertia, and thus the decision of what kind of thinking to emphasize and at what level, individual or team, is more serious.

For all the above reasons, team leadership is crucial. Throughout the life cycle of a project, a team will move through various quadrants of individual/team diver-gent/convergent thinking, with opportunity for the team and its leader to optimally allocate resources across those quadrants.

For example, with modern data analytics, can we quantify: where in its life cycle a research project is; the degree to which divergent or convergent thinking is needed; and how well the team’s current composition and cognitive constructs [44] match the desired degree of divergent or convergent thinking? Just as the “quantified self” movement [45] seeks to use physical technology to monitor the manifold pulses of a person’s daily life to optimize health and productivity; a “quantified team” movement might seek to use data analytics technology to monitor the manifold pulses of a re-search team’s daily life [46], to better match the team’s composition and organization to the research problem at hand, and ultimately to optimize the research team’s health and productivity.

Indeed, understanding how to optimize the balance between individual and team, and between divergent and convergent thinking, might borrow from advances in emerging models for information foraging [16]. For example, if useful information is “patchy,” a forager might first seek to look broadly for useful patches, and then focus in on a few of the most useful patches. Or, for example, the risk associated with not finding a patch in a particular time horizon, or the amount of resources allocated for the foraging, might determine which stage of the foraging is best done by individuals or by a team.

4 Research Ecosystems: Understanding and Assessing

At the research institution and community- of-researcher level, there are outsized opportunities for optimization, because it is this level that defines the overall research ecosystem. Individual researchers, research teams, and research managers/leaders are drawn from, and engage in divergent and convergent thinking within, the research ecosystem.

Note that in introducing the phrase “research ecosystem” we deliberately make the metaphor to “biological ecosystem” and hence to the importance of both the individual researchers and the environment which sets the boundary and interaction conditions for the researchers. There are opportunities for understanding and assessing both.

4.1 Individual Researchers: Overcoming Performance Fixation

Regarding the individual researchers within the research environment, hiring and nurturing are both key.

With respect to hiring, the imperfect correlation between school grades and creativity is well known [47]. One possible reason: grade point average (GPA) and scholastic aptitude test (SAT) scores select for strength in convergent rather than divergent thinking, while in research both are necessary. There is thus opportunity to devise new measures that go beyond GPA and SAT scores for assessing separately those qualities which underlie excellence in divergent or convergent thinking [31].

With respect to nurturing, the profound difference between learning and performance goals is well known [18]. Learning goals are more compatible with an out-ward/community orientation, and with an openness to new ideas and divergent think-ing. Performance goals are more compatible with an inward/self orientation, and with a focus on known correct ideas and convergent thinking. In other words, divergent and convergent thinking applies not just to research problems but to researchers themselves. Particularly as knowledge landscapes change increasingly rapidly, it is not just a researcher’s competence in knowledge domains of current importance that is important, but his/her “absorptive capacity” [48] for assimilating and building on new ideas and thus for building competence in knowledge domains that will be important in the future.

One key opportunity is thus: can we measure qualities such as absorptive capacity, and then incorporate improvements in those qualities into metrics for the success of research projects themselves? The analogy here is to sustainable manufacturing, but applied to the “manufacturing” of knowledge: the output of research is not just knowledge, but also a strengthening of the researchers themselves. Though the ultimate goal is long-term output of great research, the proximate goal to accomplish that research output sustainably is researchers with high and continuously increasing absorptive capacity.

4.2 Research Environment: Overcoming Narrow Measures of Research Impact and Overcoming Existential/Social Stress

Regarding the research environment surrounding the individual researchers, the easy tautology is that a great research environment is one which produces the best research that a given set of researchers is capable of, and that assessing the research environment is equivalent to assessing research impact itself.

Assessing research impact is difficult, however. In science, the current state-of-the- art assessment metric is bibliometrics of formal written documents: publication and citation counts, H and other indexes, and journal impact factors. We know, however, that bibliometrics is limited [49]. It does not measure research impact on the larger world beyond formal written documents, especially the worlds of engineered products and human capital [50]. It converges slowly, because of the slow cycle time of formal written documents. And it is inaccurate, partly because of “obliteration by incorporation” [51] and partly because it is subject to human cognitive biases.

One opportunity is thus to move towards a more real-time and holistic view of re-search impact that goes beyond bibliometrics and beyond formal written documents. Data analytics is sure to be at the forefront of this move. Identifying the most valuable pieces of knowledge and the sources of those pieces of knowledge are at the heart of modern search analytics. Overall, extending such analytics to the narrower world of research should in principle be possible.

Limitations in our understanding of how to assess research impact, however, should not keep us from improving our understanding of how to assess research environment. Both efforts in parallel will feed on each other, and enable causal connections to be made.

Understanding research environment is non-trivial. From the discussion throughout this article of the importance of divergent and convergent thinking in individual researchers and research teams, it seems clear that a great research environment is not one in which researchers are “comfortable.” Divergent and convergent thinking require individuals and teams to go beyond their intellectual comfort zones into Kuhn’s “essential tension” [52]. This is another example of how understanding and fostering the psychosocial space of the research environment can have positive impacts on the communication, collaboration, and innovation of the research ecosystem.

5 A Vision for the Future

Research, the manufacture of knowledge, is complex, but not significantly more complex than the manufacturing of other goods and services requiring a high level of creativity. As science (understanding) and technology (tools) continue to be developed and applied to the manufacturing of those other goods and services, it is natural, perhaps inevitable, that they will also be applied to research.

At every level at which research is done, there are opportunities for improved understanding and improved tools. At the individual researcher level, examples are: how to overcome idea fixation and sloppy thinking, and how to balance divergent and convergent thinking. At the research team level, examples are: how to overcome strong links and groupthink, and how to optimally distribute divergent and convergent thinking across individuals and teams. At the research institution level, examples are: how to overcome GPA and performance fixation, and how to overcome (or harness) existential/social stress.

Moreover, we have focused in this article only on the direction from “science to art,” in which the emerging science of research is harnessed to improve the art of research. Ultimately, even greater opportunity will be unleashed when the other direction from “art to science” is also exercised simultaneously and synergistically – when improvements in how research is actually done are used to test our understanding of how re-search is done.

We recognize that all research institutions are different and will have a different landscape of actionable possibilities. Many research institutions will likely share a discomfort towards opening themselves up to social scientific study. However, there are existence proofs of research laboratories opening themselves up at least to social science observation [53] if not yet experimentation. And, most importantly, the benefits to these research institutions of a science-based approach to research are potentially enormous: an enhanced productivity that, despite declining funding for physical science research, might nonetheless enable breakthroughs needed for humanity-scale grand challenges.