Reflections: DMDU and Public Policy for Uncertain Times

  • Steven W. PopperEmail author
Open Access


  • Public policy has always confronted future uncertainties. Projecting likely futures has been viewed as best practice for assessing proposed plans even though few would expect exactly those futures to occur. But in an era of deep uncertainties in which prior rules of thumb are no longer believed likely to hold true in years to come, sufficient diligence for policy analysis demands a different standard.

  • DMDU approaches collectively represent an evolving capacity to deal with the challenge of the future by providing a technology of complexity, especially in the analysis of problems in public policy.

  • New approaches to policy analysis may provide the means to enable policy processes better suited to deep uncertainty and dynamic change. And the recognition by policymakers that there exist (and that they should demand) new means for analysis that comport better to the emerging needs of policy would, in turn, allow more rapid diffusion of technique into realms not yet exposed to means for decisionmaking under deep uncertainty.

  • The interaction between analysts and policymakers requires contact between two distinct cultures. DMDU applications collectively present a body of theory and practice with the potential for providing a common vocabulary to the work of both analysts and of those charged with policy design and implementation during uncertain times.

  • For a problem requiring treatment by DMDU approaches, “any job worth doing is worth doing superficially.” An analysis based on an initial fast and simple exploratory model will frequently elucidate many of the major interactions between policy choices and the problem system. (Exploratory Modeling is discussed extensively in Chaps.  2,  7, and  15.) Those factors that appear most salient may then be examined in more detailed fashion in later model revisions and recalibrations.

  • By developing a basis for planning and operating under conditions of deep uncertainty, complexity, or both, DMDU approaches support governance and decisionmaking in an environment characterized by multiplying centers of authoritative expertise and interests, as well as the diffusion of government responsibility beyond the control of any one agency.

16.1 Introduction

The preceding chapters of this book taken together provide a snapshot of the emerging field of DMDU research and analysis. They have been selected to showcase a broad range of approaches currently at the forefront of the field’s development. In doing so, they draw from a field that is itself in motion, as these various approaches all undergo refinement, development, evolution, and even possible combination. Each year broadens the spectrum of policy questions to which they have been applied.

The application chapters were selected to illustrate a growing ability to address problems that had previously been treated either with assumptions of greater certainty than was warranted by the actual state of knowledge, or were found to be intractable using more traditional analytical techniques. They present evidence in support of the contention in Chap.  1 that DMDU approaches may enhance the understanding of our choices and aid decisionmaking to more reliably achieve long-term objectives in the presence of deep uncertainty.

Beyond these specific instances, to what character of problem could DMDU methods be applied more ubiquitously, and in addressing themselves to such questions, what should DMDU practitioners seek to provide? This chapter steps back from the examination of individual approaches to take account of the broader movement underway. In particular, while Chap.  1 spoke of decisionmaking in general, the other chapters make clear that many of the applications lie in the realm of public policy. This is not just an artifact of chance or funding, but rather points toward specific needs that DMDU approaches may satisfy. Shifting the focus from the individual methodological trees of prior chapters to the larger DMDU forest they represent, what role may these approaches and the emerging field play in addressing some of the pressing questions presented to analysts by societies increasingly aware of the dynamic of change?

This chapter seeks to provide a thematic connection between the theory and application chapters that preceded it and the emerging needs of public policy decisionmaking processes. It first takes an excursion into the relationship between science and technology to discuss the emergence of tools for addressing issues of complexity. It then discusses the challenges now faced by institutions for democratic governance and suggests the value of achieving greater integration between the operation of those institutions and the activities of analysts seeking to support their decision processes. The chapter concludes with the contention that the world of analysis and that of policy operate as two different cultures. It lays out those characteristics of DMDU approaches discussed in this book that may bridge the gaps between them.

16.2 Viewing DMDU Approaches as a “Technology of Complexity”

It is a persistent human predisposition to view the present as a time of troubles, while in retrospect, the past appears an era of comparative calm and certainty. Yet, a case can be made that today we are increasingly confronted in our public life by challenges of a depth, breadth, complexity, and rapidity of emergence that are hard to match in the prior record.1 Two subjective factors may also make our condition unprecedented. The first is that our social and political systems are predicated upon the axiom that management is both possible and expected. We recognize the element of chance, the occurrence of the unexpected, and the potential for our affairs to go awry through force majeure, but there is an implicit expectation that the institutions we have put in place will deal with and overcome such exigencies through our skill in understanding and managing the underlying systems. The prospect of not being able to do so is not only dismaying to the individual but collectively undermines the basis of consent within society.

The second subjective factor is that, unlike previous times during which people may have viewed themselves as passing through a temporary crisis before the world would either resume its normal course or settle into a new equilibrium from which new and understandable rules of thumb could be deduced, we have no such expectation. Those elements leading to the deep uncertainties we confront show no signs of abating. Indeed, we can well accept that the next generation may in their turn view our times as being milder versions of their own experience of turbulence and change. The contradiction between these two perceptions gives rise to frustration and anxiety.

In Chap.  1, several sources of deep uncertainty were laid out, including fundamental indeterminacy or unpredictability of system outcomes. That is, beyond just lacking a desired level of knowledge or failing on our part to be sufficiently diligent, a condition of complexity or emergence beyond any means for meaningful anticipation may be hardwired into the systems within which we operate. Seeing emergence, nonlinear response, and unpredictability due to path dependence—in a word, systemic complexity—as being an inherent property in some systems themselves (rather than the more colloquial use of the term, which would have it be synonymous with the complication arising from keeping track of a very large set of forces, factors, and their interactions,2) began with the work of Poincaré in the late nineteenth century but became widely documented in the work of scientists in the last four decades of the twentieth century.3 By the end of that century, there were even research institutions and groups devoted to elaborating the science of complexity.4

This book was constructed to showcase different DMDU approaches and their practical application. There is method in this. The book’s twinned chapters on theory and practice reproduce in microcosm the intertwined histories of science with its theories and technology with its tools and outputs. With the one representing the application of a rigorous system for posing questions and evaluating answers, and the other consisting of “activities, directed to the satisfaction of human needs which produce alteration in the material world,”5 to consider them together is to recognize how dependent each often is on the other. While the popular imagination perceives a linear path, with science opening the way for new discovery, application, and innovation, it is frequently the case that the work of the technologist provides the scientist not only with the tools with which to conduct inquiries, but even with newly revealed phenomenology challenging prior explanations or requiring new ones.6 The advent of the telescope, microscope, steam engine, and semiconductor all preceded the major developments in the sciences of astronomy, biology, optics, thermodynamics, and solid-state physics.

One can argue about the degree to which the project of working out a science of complexity may have met its objectives or the distance it may have travelled toward achieving them. But to analogize to the more nuanced interaction between science and technology perceptible in other fields, we may ask whether complexity science had available a sufficient “technology of complexity” to support this effort—and whether a need for one was perceived.

A case can be made that efforts by the authors represented in this volume and others constitute such a parallel journey of discovery. To be sure, while aware of the developing field of complexity studies, few of these methodologists and practitioners were motivated principally to contribute to the complexity science enterprise. Neither is it the case that all problems (perhaps not even the majority) benefiting from DMDU analysis necessarily have complex system phenomena at their core. As in other fields, the focus was not so much on understanding the nature of complexity but to deal with its consequences in systems exhibiting complex behaviors—as well as the effects stemming from the other sources of deep uncertainty in real-world application. Rather, the unifying driver was the desire to craft analytically sound systems for addressing the mounting catalog of “wicked” problems confronting decisionmakers in contexts both small and large.7 This is what technologists do: Develop tools to enhance observation as well as provide the means for maneuver and manipulation within the newly observable space that their tools have illuminated. But in doing so, they have also allowed us to provide analysis of policy in regimes exhibiting the path dependence, nonlinear response, and emergence characteristics of complexity.

The ability to navigate along deeply uncertain pathways has become a pressing need and a hallmark of our times. But it might also be one we have long been familiar with.

16.3 DMDU and Policy in an Era of Perpetual Change

The Third Ape’s Problem

A fruit-laden tree grows on a prehistoric savanna. A bipedal apehuman comes to the edge of the brush and ponders the risk of crossing the grasslands to gather a meal. Is a lion lurking in the high cover? She notices the swish of a tail: swishing tail  active lion  danger. She withdraws. Later, a second ape long familiar with this patch comes by. No swishing tail, yet she knows that every so often a troop mate has been lost in gathering fruit here. How often? She considers the risks, balances her need, and determines whether to cross. Now, a third ape approaches this stretch of savanna for the first time. She can draw upon neither concrete information nor familiarity with this patch and the probability of danger as a basis for logical deduction. She is, however, quite hungry, and her survival hangs in the balance. She searches for familiar patterns and weighs them against her experience. She takes a few steps forward and then looks for changes in the patterns. How far can she proceed and still scramble back to safety? Where might a lion hide in this brush? Is that movement over there solely due to the wind? She ventures forth step by step, updating information, planning for contingencies, perhaps tossing a rock or two to probe for any lurking predators.

In this manner, she proceeds into a potentially terrifying unknown. 8

Technology arises in application. A branch fallen from a tree remains only that until it is picked up by a human hand and transformed into a spear, javelin, awl, hoe, flail, walking stick, roasting spit, cudgel, or lever. This book presents many candidate technologies for creating tools to solve the third ape’s problem in its modern manifestations. Hers was the problem of the individual. Ours is in many respects the problem of societies and of their constituent organizations and institutions. And more than a problem of analysis, it is a problem for governance as well.

Surrounding many of the issues confronting us today, from the large issues of global warming, wealth and income inequalities, and the future of work, to local decisions regarding infrastructure, regulation, and mobility, we may perceive a more general issue rooted in democratic institutions of governance. From the USA to the UK, from Israel to India, from the states emerging from the shadow of the Cold War to the superstructure of the European Union, democratic institutions (and ultimately the principles upon which they are predicated) are faced with challenges of several kinds that are proving difficult to resolve. True, it has never been easy to deal with clashes of values, conflicting interests, or manifestations of uncertainty, nor with the ensuing policy disagreements. But today, there appears to be a generalized perception in many instances of an inability on the part of the democratic governmental process to either meet these challenges fully or to respond in a timely manner. Despite the varying degree to which this statement might be true for particular issues, what appears to have become widespread is the growing and often implicit perception on the part of the publics whom governments are established to serve that their leaders are incapable of addressing these concerns.

Resolving such issues through means designed to minimize or entirely obviate the role for democratic institutional decisionmaking may undermine values we prize. Markets and market mechanisms are magnificent human artifacts for collating information leading to the efficient allocation of resources. But despite their capacity for fostering allocative efficiency, they were not designed nor are they capable of producing decisions on policy with which we might feel comfortable. While we do care about cost and efficiencies, we also care about equity and balancing other interests, not only among social groupings but across generations. Mechanisms for doing so have often been suggested, but few have addressed the problem of how to avoid further incentivizing the forces seeking to alter market structure in such a way that the outcomes are skewed in a direction best suited to the interests of the economically powerful, who may then translate such market power into political power. On the other hand, authoritarian forms of governance, aided by the technocratic efforts of analysts applying the most scientific of principles, may well achieve many specific priorities, but good governance has never been shown to be among them.

Many countries have looked to democratically elected (and democratically operated) governments to fulfill the role of sovereign policy authority. The role of the analyst who addresses the category of wicked problems or the technologist of complexity who applies his or her skills to the DMDU problem space within the democratic ecosystem, is to enhance and shore up the ability of democratic governance to perform the crucial role of policymaking. They fulfill this function by providing the means to perform two important tasks: (1) make visible and comprehensible those phenomena that are relevant to the decisions being considered but may, by their sheer number and complex interactions, be difficult to observe directly or share widely, and (2) illuminate the limits of the practical trade space within which the political apparatus may find and select short-term policies that may still be reasonably assured not to compromise the ability to achieve longer-term policy goals.

This may hold even more sway in an era in which the roles of hierarchies and authorities may themselves be subject not only to question but the emergence of other forms of informal but nonetheless powerful authority. Previously, governments enjoyed an advantage in having had the means to speak with authoritative expertise on many issues. Now, however, the ease of information gathering means barriers are lower for others to also gain the knowledge necessary to establish strongly held, well-argued positions. Further, the concomitant ease of communications means that such centers of expertise may actively coalesce communities of interest into tangible constituencies, acquire information enabling them to put forward positions and proposals, and communicate these positions widely to gain further adherents. To some extent, this comes at the expense of previous centers of authority as their former concentration (if not monopoly) of expertise, information, and unity of agency ebbs. For it is not only authority that has become dispersed, it is responsibility as well. One of the characteristics of canonically wicked problems is that their solution does not lie clearly with only one, or even a few, public agencies. It is difficult to tell who is responsible, for example, for eliminating roadway fatalities from the US transportation system.9 At the very least, such responsibility is shared among federal, state, and local agencies, to say nothing of the myriad professional and civic society groups dedicated to this goal.

To the extent that the preceding diagnosis is accurate, business as usual in the analysis and debate of public policy does not appear to be a prescription for success going forward. We would be better advised to consider how best to conduct effective public policy planning, choice, and decisionmaking under conditions that appear now to be more likely characteristic of our future than of our collective past. If, for example, the Dutch decision model of shared responsibility is a potential solution and could be made more ubiquitous, then (as in Chap.  14) it requires a platform providing an integrated system of analysis and decision that is capable of handling not only the full range of real-world complication but its attendant uncertainties as well—a role for DMDU-inspired applications.

16.4 Integrating Analytical and Policy Processes

A basic model of individual decisionmaking, originally developed to explain why pilots of one nation can come to dominate the trained pilots of another, sets decisions within a recursive process that is calibrated by updating knowledge about existing conditions and new developments (Boyd 1996). Figure 16.1 shows a cycle of observing the environment and situation, orienting within that frame of reference, reviewing a set of possible actions before deciding upon one course, and then acting (at which point the loop begins once more with results from the preceding action being measured and assessed)—the OODA loop.10
Fig. 16.1

Observe, orient, decide, and act (OODA) loop

Some have seen the OODA loop as a model for decisionmaking within organizations, as well as at the level of the individual.11 For our purposes, it marries the active with the analytical. An organization wishing to operate strategically must have an ability to scan the horizon (observe) and understand not only the factors that give rise to these observations but also what implications they have for its missions (orient).

The acts of observation and orientation are central to most DMDU approaches in application. Generally referred to as “monitoring,” perhaps not surprisingly this theme comes out most strongly in this book’s chapters from contributors who work in the Netherlands12: Climate and hydraulics interact with slow-moving dynamics that may take decades to show observably certain outcomes but may also result in sudden, catastrophic loss. In such cases, those seeking to perform observation and orientation may not be afforded the luxury of examining actual outcomes. Much of their work must be done by having previously identified early warning metrics and signals for action before resolutions are achieved or through the extensive use of simulations and other modeling. This DMDU work especially considers the role of observation and orientation as precursor to, and trigger of, decision and action.

While this all appears intuitive, the difficulty lies in how to make an OODA approach to public policy operational. This synthesis is not readily observable in most public agency decisionmaking processes (see Chap.  13). Instead, most policy processes are based upon a series of tacitly agreed fictions that in the past either served as useful first approximations to reality or were concessions to the limitations existing in the milieu within which decisionmaking needed to occur. Thus, we see an emphasis on the production of plans rather than on the process of planning itself as a principal tool for effective policy formulation. We also often take an implicit “once-and-done” stance toward policy. That is, having arrived at a policy decision selecting a set of preferred actions, there is a tendency to regard the problem as having been solved.13 Of course, all involved in such decisions recognize that reality may all too often diverge from plan, but we put off as long as possible having once more to address the inefficiency and ineffectuality that builds up over time until a later round of policymaking is seen to be required.14

A policy process that comes closer to the ideal represented by the OODA paradigm would recognize the close relationship between plans and planning, between product and process. It would recognize the value of and enhance the potential for more adaptive approaches to policy formulation. (It would create a more widely shared realization that any policy implemented to address a sufficiently complex issue could also be viewed as a policy experiment in the sense that observation of initial outcomes provides opportunity for learning and adaptation.) Instead of policy practices that tend to be confrontational and exclusionary, the means for engagement and inclusion would be incorporated in a manner that still permitted the process to move toward formulation and implementation. And the inherent uncertainties, rather than being at best implicit and at worst disregarded, would instead be explicitly addressed by decisionmakers and policy officials within their deliberative processes.

Analytical support to policy may bring in a further subtle dysfunction. This takes form in a self-reinforcing feedback loop between the way policy processes are currently construed and the requests and demands they place upon the analysts tasked with supporting them. The policy planners and decisionmakers usually ask only of their analysts what they have traditionally provided. Similarly, if the analytical modes themselves reinforce an implicit predisposition toward decision mechanisms predicated on prediction, optimization, and requiring an upfront winnowing of acknowledged assumptions to a small, analytically tractable set, then there is little hope that the decision processes in their turn can pass far beyond the traditional approaches so long in practice and increasingly less capable of satisfying the demands placed upon them.

In this sense, there is an intertwined process. New approaches to policy analysis may provide the means to enable different approaches to policy. And the recognition by policymakers that there exist (and that they should demand) new means for analyses that comport better to the emerging needs of policy would, in turn, allow more rapid diffusion of technique into realms not yet exposed to tools that support decisionmaking under deep uncertainty. The chapters of this volume more than hint at the possibility of a co-evolutionary relationship between changing policy processes and enabling new analytical approaches and tools.15

This theme applies to all applications presented in this volume and certainly is one that each of the represented approaches has been used to address. One of the goals of the book, certainly in the chapters illustrating applications, is to go beyond a narrow intent to enhance training of DMDU methodologists alone. Wider use of DMDU techniques and a greater realization of their latent potential for better supporting policymaking during trying and troubling times also require educating the potential recipients of DMDU outputs and analyses.

16.5 The Two Cultures of Policy and Analysis

If analysts and policymakers are to become partners in devising governance processes in better accord with new conditions while adhering to democratic principles, it would be helpful if they could speak a common language. If their activities are to become more integrated, and therefore less sequential and mutually isolated, there needs to be basis for synchronization. The difficulties of both challenges should not be minimized; nevertheless, the potential for DMDU approaches to provide necessary common ground is worth examining.

To one familiar with the relationship between policymakers and their planning staffs on one hand and scholars and analysts on the other, the obstacles to achieving closer integration between policymaking and analysis would appear to be considerable. Between the two communities of analysis and policy lies a gap in cultures. The traditional culture of analysis is numerate, reductionist in character, systematic in approach, and ultimately rooted in deductive logic. The culture of policy, however, is more narrative based, focused on details, nuanced, and situation-specific in process—and its debates are often framed in the logic of abductive reasoning: “How will we be affected if present trends continue? What could go wrong if we follow this course or that? If the circumstances we most fear come to pass, how will we cope?”

This cultural gulf explains, in part, the frustration felt by analysts who present significant findings with potentially large implications for policy and are thanked for their efforts but next find the door to the chamber in which policy deliberations occur closed in their face. The translation of analytical output into policy failed to occur.

Ultimately, the responsibility for doing so rests on the shoulders of the analysts (principally because their relationship to policy authorities is not that of equals) and will require of them greater sensitivity to culture. Presenting findings couched in a language and structure consistent with scientific norms suitable for academic publication often presents the policymakers with a problem. Such findings may be perceived by them as being less a boon and more an addition of yet more confusion to the welter of information through which they must wade to frame the knowledge base required for policy. If left with the burden of themselves translating research outputs into the terms in which policy is deliberated, meaningful analytical insights may fail to gain entry into the conversation on the other side of the door—or wall—that lies between analysis and policy.

Whether the authors of DMDU applications conceive their role in these terms, the effect of their collective effort has been to create a body of theory and practice with the potential for providing commonality to the work of analysts and of those charged with policy design and implementation during uncertain times. To a greater or lesser degree, most if not all the techniques presented in this book have implicit in them several postures toward analytical practice and the role of analysis in the support of decisionmaking that collectively hold promise for supporting new approaches to both analysis, policy, and the relationship of each toward the other.

16.6 DMDU Principles Enhance the Integration of Analysis with Policy

This book’s introduction laid out some commonalities of process among the DMDU approaches presented in subsequent chapters. If we now look to both the predispositions that inform those steps as well as what ensues as the result of having followed them, we can derive several attributes that would themselves appear to be necessary to change the way both analysts and policymakers view their roles given the challenges we now face. While there exists a tight relationship among many of these principles, we list them individually to make them explicit.

Begin from the End. Good analytical practice always seeks to understand the problem being addressed and to frame it appropriately. In many cases, however, there is a tendency after the initial framing to focus primarily on the challenging task of model building that can subtly shift focus. This results from a too narrow view of the role of models, principally seeing them as necessarily representing the most accurate representations of the systems being modeled, and of what constitutes a valid model.16 To be sure, a DMDU study may begin with an existing model as a legacy and build from there. Often, however, no such model exists and so must be created or an existing one transformed. DMDU approaches construct such an analytical framework backwards: building up what is necessary to answer the questions at the heart of the policy decision (which itself is brought directly into the modeling framework) rather than the more frequent practice of first creating an artifact that attempts to capture all details of the phenomenology involved.17 The reason for this is simple: a good deal of the detail that would be required for the most accurate system simulator may be shown through rigorous analysis not to be required to provide a full exploration of the decision to be informed in situations involving deep uncertainty. This characteristic of looking through the “wrong” end of the traditional analytical telescope not only affirms the policy question as the lodestone for all that follows, it also improves the likelihood that results stemming from analysis will fit easily into the policy debate through use of the same language and in addressing the same core issues. (Most of the Part I chapters of this book, especially Chaps.  2 and  6, call out this aspect of DMDU analysis.)

Characterization of Uncertainty. Operating under conditions of deep uncertainty deprives us of high confidence in probability density functions. Instead, the unknowns are characterized by the degree to which they may affect decisions over the choices we face in view of our objectives. (Both examples in Chap.  7 illustrate this point by their use of Scenario Discovery.) That is, DMDU analysis looks across uncertainties and seeks to understand what we would need to believe was true to advocate one course of action over another. The objective then becomes to look for alternatives within the potential solutions set (or to add to that set through modification, hybridization, conditional statements, or novelty) for those short-term courses of action that would appear least vulnerable to the uncertainties present and yet still perform sufficiently well according to our criteria for good outcomes.

This posture also shifts the entire thrust of the analysis from seeking agreement on assumptions to one of agreeing on solutions (Kalra et al. 2014). This is a powerful technique for getting beyond incendiary debates over whose assessment of presently unknowable facts is the most accurate—debates that cannot be resolved to the satisfaction of all parties. The lack of agreement on a set of most valid assumptions greatly inhibits progress toward finding solutions. Instead, the DMDU approach will carry all sets of assumptions forward into the analysis stage and explore courses of action that may satisfy all parties.18

Multi-objective Analysis. A DMDU analysis will typically range over several classes of factors to explore. Among these are assumptions about external uncertainties, the differential efficacy of policy and potential policy instruments, alternative specifications of the underlying system model or causal relationships, and the metrics by which outcomes are to be evaluated.19 While all are crucial to enabling a more comprehensive form of analysis given deep uncertainty and contentious policy debates, the last of these is worth highlighting as an important aid to achieving not only greater correspondence between analytic and policy processes, but also in addressing the phenomena of more widely distributed authoritative expertise and more diffuse government responsibility. This aspect of DMDU analyses allows their authors to confront the multi-attribute character of wicked problems explicitly.

When there are multiple parties to a complex policy issue, there will rarely be widely shared agreement on a single bottom-line metric sufficient to evaluate outcomes. Usually, there are several assessment criteria that may even be in opposition to one another. Even if all may agree that we have an interest, for example, in both economic growth and environmental sustainability, we may argue about priorities among the outcomes we value. The wider scope for utilizing multiple measures in a DMDU analysis naturally accommodates a wider perspective on normative outcomes. In most cases, this leads to a concomitant shift from searches for optimal outcomes (if exploration of the range of uncertainties has not already disabused too great faith being placed in optimality as a criterion) to an alternative posture of satisficing.20 In this case, policy sets minimum or maximum threshold levels for selected measures by which to evaluate how well different proposed solutions may meet policy goals across a wide range of plausible futures. (This also more closely resembles the hedging approach taken by senior decisionmakers confronting deep uncertainty than do modes of maximization.) This will be a key to achieving consensus on plans for confronting the major policy issues of today and the future.

The practical routines of DMDU applications will by their nature support a multi-attribute focus. Such studies regularly seek engagement with stakeholder groups, planners, and decisionmakers through the use of real-time analytical support (the water planning example in Chap.  7), workshops (Chap.  8), games (Chap.  9), and Assumption-Based Planning-inspired pre-plan exercises (Ecola et al. 2018).

Iteration. Preceding chapters have demonstrated that most DMDU analyses are inherently iterative. They are applied as a recursive process, similar to the OODA loop discussed above, refining not only the analytical product but also the analytical infrastructure for conducting the analysis. This is the characteristic that enables the “backward” approach to analysis that allows testing and refining of underlying models to bring out greater detail when required and to eliminate the need for more detailed modeling of those aspects of the system that appear to have little or no effect on our normative decisions or the characterization of uncertainties.

In fact, we might coin the dictum that for a problem requiring treatment by DMDU approaches, “any job worth doing is worth doing superficially.” That is, an initial fast and simple approach to modeling21 as well as in formulating potential solutions often makes clear many of the interactions between system and policy that may then be fleshed out as warranted in later revisions.

The resource conservation approach to modeling taken by DMDU approaches is more than an efficiency measure. It has the potential for enfranchising categories of knowledge that might otherwise not be codified in a form to fit easily within more traditional analytical modes. This can take several forms. On one end of the spectrum, science generates more knowledge than appears in academic papers. Scientists gain considerable experience and develop hypotheses well before they are positioned to report the conclusive, null-hypothesis-falsifying experiment that will yield a publishable formal result. For sufficiently complex phenomena, such as those involving climate change or potential consequences for human societies of creating genetically modified organisms, it may be nearly impossible to perform a single experiment or even a connected series that will lay all questions to rest. This circumstance places scientists in an uncomfortable position. They may possess considerable knowledge with relevance to policy, but potentially risk their academic standing and possibly transgressing professional norms if they lack confidence that their knowledge, if publicly shared, will be viewed correctly as being informed by experience but not itself representing proven scientific truth. But in an iterative process in which this knowledge becomes one of the means used to define potential ranges of values for subsequent exploration, much of this concern is reduced. Their direct experience and any hypotheses may be shared, weighed, and considered appropriately with considerably less risk than earlier, more definitive statements would generate.

At the other end of the spectrum, a good deal of knowledge is not easily codified or formalized. One of the characteristics of many DMDU techniques is to allow multi-dimensional phenomena to be rendered in lower dimensional spaces graphically. This “dimensional collapse”22 allows observation of connections, relationships, and systemic phenomena that had not been previously visible within the bewildering forest of uncertainties and assumptions. It also makes them shareable. Rather than constructing a deus ex machina, a technocratic analysis designed to dictate policy prescriptions, DMDU instead most often provides output that may serve as a “digital campfire”—an occasion for those with different perspectives as well as knowledge that may not previously have found enfranchisement in formal analysis to view DMDU results and share among themselves the significance they find and the insight such results elicit.

The advances in computer technology that make possible the drive toward artificial intelligence also enable the tools of DMDU. The crucial distinction is that in the latter case, the resulting tools seek to preserve and enhance the human-in-the-loop and not eliminate the need for human decision and intervention by making predictions.

Accessibility and Transparency. The enhanced ability to share analytical output and insights provides one further attribute for a more effective engagement of analysis with policy under deep uncertainty. An aspect of uncertainty that particularly bedevils hierarchic organizations as well as cross-organizational decisionmaking is that of uncertainty absorption. As noted by March and Simon (1958), the condition exists when lower levels within an organization possess much information about the quality of their information (how trustworthy may be its source, how well validated, etc.), but this nuance is difficult to convey to the higher levels without having the information channels seize up from sheer overload. The nuance is lost, and uncertainties and indeterminacies are either dropped or presented as solid fact. This can lead to senior-level decisions being reached based on misunderstood premises.

It is in the nature of most DMDU approaches, however, that while they may perform analytically the dimensional collapses that yield observations regarding systemic regularities among actions, assumptions, and outcomes, they also retain a “drill down” capacity. That is, it is possible to easily call up the individual cases from which these massive ensembles are composed to make them available for detailed examination. It is possible, therefore, to provide not only transparency into what cases and instances yielded the observed results, but also to make these underlying data available for further questioning if the audience receiving the DMDU output deems it valuable. The nature of DMDU analysis and the framework it creates make possible ongoing and more effective “red teaming”23 of the analytical output from these techniques.

In a different vein, the fact that the apparatus for DMDU analysis has been constructed to address complicated policy issues, and to be carried out iteratively in evaluating alternative assumptions and evolving potential courses of action, means that the analytical tooling remains useful even after the initial results have been obtained. If we are truly entering an era of testing the implications of assumptions, seeking robust solutions, and approaching policy with greater flexibility and adaptability partly due to a recognition of greater potential for change and surprise, then it stands to reason that the machinery used to conduct policy analysis should continue to be employed, updated with new knowledge, and engaged in the process of navigating the policy adaptations almost certain to be necessary. This is an extension of the deliberation with analysis approach (Chap.  2) beyond the period of policy planning. The nature of DMDU methodology creates analytical artifacts that may continue to provide value within a more integral connection between analysis and policy brought about through a desire to maintain policy effectiveness across the fluctuations that the future may bring.

16.7 Conclusion

This chapter suggests that if one looks at the DMDU applications in Part II of this book, as well as the presentation of approaches in Part I, one may perceive an academic field that has consciously dedicated itself to being relevant to the world of public policy. It has produced a growing library of real-world applications. Any such application will have been intended to serve limited purposes. But viewed in aggregate, the DMDU field is pointed toward supporting governance and decisionmaking in an environment characterized by multiplying centers of authoritative expertise and interests, diffusion of government responsibility beyond the control of any one agency due to the inherent complication of the problems being faced, and creating tools to operate under conditions of deep uncertainty, complexity or both. As such, the field may have relevance more broadly in helping provide means for evolving an approach to governance in which policy and analysis become more integrated than has previously been the norm.

The perceived inability of democratic governance to address the most vexing challenges we face has had its own effect on the public policy sphere. One of the themes that is heard most clearly in the heated debates over policy and in the political upheavals manifested in much of the democratic world in recent years is a cry for justice. Indeed, it is this that makes so many issues, from trade to immigration, and on through health and access to opportunity, contentious. Many are now participating in what they perceive as a quest for justice for themselves and for those with whom they identify. Lempert et al. (2013) elicit from Amartya Sen’s The Idea of Justice (2009) key points that would appear basic to any new synthesis of analysis and policy in support of democratic governance that may flourish in an era of accelerating change. Among these, ethical [policy] reasoning should recognize as fundamental attributes the existence of both diversity of priorities, goals, and values, and an irreducible uncertainty regarding the consequences of our actions. This suggests the value of seeking agreement on which non-ideal options could be viewed as the most just from a range of perspectives and circumstances. The only way to achieve this goal in a participatory democracy is to conduct public deliberation. And this deliberation will work best when it enables re-examination and iterative assessment and permits clear explication of reasoning, logic, and underlying values. This keystone is an “open impartiality” that accepts the legitimacy and importance of the views of others. And these properties are inherent in not only the applications of DMDU but in the approaches themselves.

In many respects, the practice of DMDU analysis is just entering its adolescence. Many questions remain regarding the role of models, how to harness sufficient computational power to fully exploit the latent possibilities for analysis, how to apply DMDU concepts when formal models don’t exist, and how the formalisms presented in this book may evolve and be tested in wider application. But the fundamental approach that motivates them all is an intention to move the practice of analysis, as well as the decision processes, policy deliberations, and governance applications DMDU analysts seek to support, in directions that would appear most needful to confront the challenges we will face in years to come.


  1. 1.

    Or, one might say that, while other eras may also have witnessed periods of upheaval (1914–1945 comes to mind), we should determine to meet our own challenges armed with more fitting tools than were available to our forebears.

  2. 2.

    The US space program of the 1960s is an example of a highly complicated system (that is, one with many moving parts) that nonetheless was not complex in the sense that it could be divided into projects directed toward subsystems that upon recombination would successfully succeed in its goal of manned flight to the Moon. An analytically complex system is not susceptible to this degree of predictability and control. For a discussion of complexity and policy, see Mitchell (2009).

  3. 3.

    Prominent among these are Ilya Prigogine in chemical thermodynamics, Benoit Mandelbrot in mathematics, and Norbert Wiener in cybernetics, to name only a few.

  4. 4.

    Perhaps the best known of these is the Santa Fe Institute, founded in 1984 (

  5. 5.

    Childe (1954).

  6. 6.

    The most vigorous statement of the linear hypothesis was that of Vannevar Bush in 1945’s Science, the Endless Frontier. The work of Derek de Solla Price and other historians of science and technology has provided numerous examples refuting unidirectionality from science to technology.

  7. 7.

    Rittel and Webber (1973) “Dilemmas in a General Theory of Planning.” Policy Sciences, 4, 155–169.

  8. 8.

    Abstracted from Popper (2003), Lempert and Popper (2004).

  9. 9.

    See Ecola et al. (2018).

  10. 10.

    Given the phenomena the original OODA loop was designed to explain, it is also a convenient structure for amalgamating into one framework both the fast- and slow-thinking aspects of human cognition discussed by Kahneman (2011); see also Klein (1999) on recognition-primed decisionmaking.

  11. 11.

    See, for example, Angerman (2004).

  12. 12.

    See especially the matched Chaps.  3,  4,  8,  9, as well as the second example in Chap.  11, and the history laid out in Chap.  14. Monitoring and triggering are at the core of the planned adaptation case studies provided in Chap.  13.

  13. 13.

    It is useful to contrast the intrinsic approach to policy implicit in the Kyoto round of climate accords to that which informed the most recent Paris accords. The former was imbued with the sense that the signatories were setting policy for decades to come, while the latter was built around a more sophisticated and recursive policy decision process (Popper 2015). Within this book, the second example in Chap.  7 provides an illustration of how DMDU applications could assist in framing and navigating the type of analysis and policy cycle envisioned in the Paris Agreement.

  14. 14.

    Classic instances of this phenomenon include the 30-year cycles in the USA between reforms to the tax code or retirement pension policies (i.e., Social Security).

  15. 15.

    Chapter  13 presents four examples that serve as case studies for a planned adaptation within a nexus of analysis and policy. Chapter  14 gives an exceedingly valuable, detailed history of one such system that stemmed from the cross product of a political disposition toward consensus decisionmaking and the support provided by DMDU analysis.

  16. 16.

    Davis, Paul K. and Steven W. Popper (in review). “The Uncertainty Analysis That Policymakers Should Demand.”

  17. 17.

    This approach favors fast simple models (FSM) rather than the more detailed modeling necessary for techniques with less capacity for and less emphasis on iteration. See the Glossary for more detail.

  18. 18.

    This satisfaction is often bought by embedding conditionality at key decision points into the policies that emerge from the process. Resolution of differences therefore does not rest upon agreement among parties. Rather, it stems from the all-too-human principle that, of course, I am willing to concede that should my opponents prove correct we will shift the implementation of the policy to a different course, because I believe that they are almost certain to be wrong.

  19. 19.

    These are laid out in the introduction to this book (Chap.  1) and discussed in operational terms elsewhere, especially in the paired chapters on RDM (Chaps.  2 and  7) and DAP (Chaps.  3 and  8).

  20. 20.

    Simon (1956).

  21. 21.

    See Davis and Bigelow (1998) on the values of multiresolution modeling, Davis and Bigelow (2003) on the use of metamodels, and van Grol et al. (2006) on the use of fast simple models.

  22. 22.

    See the discussion of Scenario Discovery in Chap.  2.

  23. 23.

    A red team is traditionally a specially constituted group, different in membership from the group that developed a plan, report, or conclusion, that is charged with trying to find the holes or flaws in the original product. DMDU techniques may be thought of as a way to make red-teaming part of the analytical process itself.


  1. Angerman, W. S. (2004). Coming full circle with Boyd’s OODA loop ideas: An analysis of innovation diffusion and evolution, Wright-Patterson Air Force Base, Ohio: Air Force Institute of Technology, Air University, Department of the Air Force, AFIT/GIR/ENV/04 M-01.Google Scholar
  2. Boyd, J. R. (1996). “The Essence of Winning and Losing,” briefing, Chet Richards and Chuck Spinney, eds.Google Scholar
  3. Bush, V. (1945). Science, the endless frontier: A report to the President by Vannevar Bush, director of the Office of scientific research and development. Washington: US Government Printing Office.CrossRefGoogle Scholar
  4. Childe, V. G. (1954). Early forms of society. In S. Charles, E. J. Holmyard, & A. R. Hall (Eds.), The Oxford history of technology. London: Oxford University Press.Google Scholar
  5. Davis, P. K., Bigelow, J. H. (1998). Experiments in multiresolution modeling (MRM). RAND Corporation, MR-1004-DARPA.
  6. Davis, P. K., Bigelow, J. H. (2003). Motivated metamodels: Synthesis of cause-effect reasoning and statistical metamodeling, RAND Corporation, MR-1570-AF.Google Scholar
  7. Ecola, L., Popper, S. W. Silberglitt, R., Fraade-Blanar, L. (2018) The road to zero: A vision for achieving zero roadway deaths by 2050. National Safety Council and the RAND Corporation, RR-2333-NSC. As of August 31, 2018.
  8. Kahneman, D. (2011). Thinking, fast and slow. New York: Farrar, Straus and Giroux.Google Scholar
  9. Kalra, N., Hallegatte, S., Lempert, R., Brown, C., Fozzard, A., Gill, S., Shah, S. (2014). Agreeing on robust decisions: A new process for decision making under deep uncertainty. WPS-6906, World Bank.Google Scholar
  10. Klein, G. A. (1999). Sources of power: How people make decisions. Massachusetts: MIT Press.Google Scholar
  11. Lempert, R. J., Groves, D. G., Fischbach, J. R. (2013). Is it ethical to use a single probability density function? RAND Corporation WR-992-NSF.Google Scholar
  12. March, J. G., & Simon, H. A. (1958). Organizations. Oxford, England: Wiley.Google Scholar
  13. Mitchell, S. D. (2009). Unsimple truths: Science, complexity, and policy. Chicago: University of Chicago Press.CrossRefGoogle Scholar
  14. Popper, S. W. (2003). The third ape’s problem: a parable of reasoning under deep uncertainty, RAND P-8080. This may also be found in abbreviated form in Robert J. Lempert and Steven W. Popper, “High performance government for an uncertain world”, in Robert Klitgaard and Paul C. Light (2004), High Performance Government: Structure, Leadership, Incentives, (RAND: Santa Monica), pp. 113–135.Google Scholar
  15. Popper, S. W. (2015). Paris gets the (decision) science right. December.
  16. Rittel, H. W. J., & Webber, M. W. (1973). Dilemmas in a general theory of planning. Policy Sciences, 4, 155–169.CrossRefGoogle Scholar
  17. Sen, A. (2009). The idea of justice. Cambridge: Massachusetts, Belknap Press.Google Scholar
  18. Simon, H. A. (1956). Rational choice and the structure of the environment. Psychological Review, 63(2), 129–138.CrossRefGoogle Scholar
  19. van Grol, R., Walker W., de Jong, G., Rahman, A. (2006). Using a meta-model to analyse sustainable transport policies for Europe: The SUMMA project’s fast simple model. In 21st European Conference on Operational Research, Reykjavik, Iceland, 2–5 July 2006.Google Scholar

Copyright information

© The Author(s) 2019

Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.

The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.

Authors and Affiliations

  1. 1.Pardee RAND Graduate SchoolRAND CorporationSanta MonicaUSA

Personalised recommendations