figure a

A key implication of the abundance of promising technologies in various stages of development is that AFS transformation is less likely to be limited by science-based discovery than by human agency. What key players do in response to the wealth of options they face will ultimately determine the path(s) we follow. The key to generating, adapting, and scaling fit-for-purpose AVC innovations to advance HERS goals, therefore, turns on the coordinated exercise of human agency. The many diverse actors within value chains must both empower each other and hold all parties accountable. Innovation co-creation is a strategic game in which each party’s actions respond to others’ behaviors. Given human agency, the challenge is to structure incentives and constraints—and to identify and use KPMs necessary to monitor progress and adjust course—so as to steer actors towards mutually beneficial actions that generate cooperative outcomes superior to the typically inferior outcomes that emerge from self-interested, non-cooperative behaviors.Footnote 1 The social process of negotiating and co-creating innovation is, therefore, as important as the science of the underlying advances.

The social process of co-creating beneficial innovation has two parts. The first is participatory dialogue and coordinated action. As we emphasized earlier, AFSs are highly decentralized, populated by myriad private actors: farmers, firm managers, chefs, food consumers, non-profit agencies, and public stewards associated with governments and civil society organizations. Each actor acts at least partly out of self-interest, guided by market signals and non-market norms and constrained by regulatory, legal, and resource limits on their actions. As we seek to accelerate the development and diffusion of beneficial innovations, it is imperative to recognize and engage with the diverse motivations people exhibit and the varied constraints they face. The task is to reconcile objectives and constraints to improve coordination and mutually reinforcing productive behaviors.

That requires dialogue. No one actor can reasonably expect (or be expected) to anticipate all the spillover effects or contingencies of a technological or institutional change in complex and heterogeneous systems. The wisdom of crowds can be tapped to improve foresight and tradeoffs analysis and to induce collective buy-in to enable progress (Surowiecki 2005). Top-down prescriptions rarely work, not even so-called expert guidance, especially in systems that demand deep contextualization and tailored bundling, much less cooperation and broad buy-in.

But the wisdom of crowds only works when no one actor has excessive, durable power, else the distorted views of the powerful persist and lead to sub-optimal outcomes (Golub and Jackson 2010). Processes that are only superficially participatory thus too often postpone—even compound—the problems that require negotiated resolution. Authentic, broad-based participation is a key reason for the impressive successes of the Science and Technology Backyards program in China, which naturally fosters innovation co-creation by researchers, farmers, input supply companies, landscape managers, etc. (see Box B).

Engaging and empowering a broad range of stakeholders is increasingly feasible with digital advances. Building public support on the demand side for beneficial innovations—among workers whose safety is imperiled, consumers whose health is compromised, farmers whose lands or livelihoods are at risk, etc.—is as important as engaging the supply side represented by researchers, policymakers, and investors. We are optimistic that as data become more plentiful and readily accessible, as the transparency of AVCs advances, and as awareness broadens and deepens of the adverse spillovers associated with current AFSs, often-marginalized elements of civil society will increasingly assert their demands. More reliable KPMs will also make it increasingly possible to identify tradeoffs and to negotiate socio-technical innovation bundles that can address multiple interests’ legitimate, but disparate, concerns. Few individual innovations offer Pareto improvements; but by bundling innovations, Pareto improvements become feasible. The question is whether we have the will, and institutions, to guide us towards the imposition of strategies that reward some at the expense of others—as too often happens now—or instead towards approaches that ensure no groups get left behind. That is a sociopolitical choice.

The risk is that continued concentrated power poses grave threats to beneficial innovation. We may luck into benevolent exercise of power, whether by governments, large companies, or civil society organizations. But benevolence and an appetite for power are at best imperfectly correlated. So rules to prevent excessive concentration of power matter for innovation.

This concern about concentrated power and stakeholder access links agri-food innovation strategy to seemingly disconnected topics like the financing of political campaigns, or conflict of interest regulations to restrict civil servants’ non-blinded financial interests in and hiring by private companies. The political economy of, for example, agricultural policies in the US and Europe have too often favored powerful, vested interests able to use a variety of ethically suspect maneuvers to postpone overdue reforms and discourage innovations that might threaten their short-term interests. When the powerful are the ones most likely to lose out from innovation, their natural response is to obstruct. Checks on concentrated power are essential to maintain a vibrant, innovative economy.

Hence the importance surrounding the formulation (and unending reformulation) of rules, or more broadly institutions, which Nobel Laureate Douglass North (1991) famously defined as “the humanly devised constraints that structure political, economic and social interaction.” Some of the most important institutional innovations are those that rectify imbalances of power so as to amplify the voices of marginalized subpopulations and hold the relatively powerful accountable for their actions and consequences.

Concentrated power is a risk in both public and private spheres. Governments are obviously key actors in shaping AVC innovations. But states have often been slow to act and are often too easily captured by special interests. Businesses therefore play a vital role, as they may be more nimble than governments, although private firms can equally be the very special interests that capture public policy. This is perhaps especially true in AFSs, within which private businesses increasingly set food standards—related to equity, nutrition, safety, and sustainability—that are stricter than those mandated by governments, which increasingly set public standards that conform to industry-led ones (Reardon et al. 2009; Swinnen 2016; Barrett et al. in press). Firms also appear to reap most of the economic rewards from such standards (Meemken et al. 2021). This reflects growing company awareness not only of social responsibility, but equally of financial self-interest in advancing AFS innovations that enhance resilience and sustainability while promoting equity, inclusion, and healthy diets. Consumers, employees, and investors all increasingly expect firms to do more than merely maximize financial profits, and they reward them for doing so.

Publicly-funded research for development actors also has a critical role to play, and innovations are needed in the way in which research for development itself is prioritized, formulated, and implemented. Too often there are lock-ins that give primacy to priorities and practices for incremental change, and for component rather than system innovation (ISPC 2018). Much of the groundswell on the necessity of AFS transformation arises, however, from the clear need for systemic action at scale, called by different names in different contexts: “end-to-end approaches” in the development donor community, or “climate-resilient development pathways” in the climate change community, for example. We still lack comprehensive analytical frameworks associated with such approaches that can guide the co-creation of multi-dimensional, bundled actions in practice. And surely no rote, cookbook approach will ever prove feasible. Nevertheless, as we have outlined here, we know many of the elements needed. Per ISPC (2018) these include:

  • New partnerships among all actors committed to moving the AFS transformation agenda forward

  • Theories of change that reflect the transformational agenda of the broader design objectives—SDGs to 2030, HERS at our longer-term horizons—and that acknowledge and respect actor-specific objectives as well

  • Clearer understanding of the economic, socio-cultural, and environmental dimensions of new, possibly highly disruptive technologies

  • Reframing the narrative concerning how we can collectively exploit scientific advances and what new capabilities will need to be built.

The second part of the social process, following from dialogue, is coordinated actions for shared management of AFSs. Coordination must typically be loose, in the sense of operating not through centralized decision-making but rather via structured, self-interested behavior within guardrails enshrined in institutions built through participatory dialogue, such that agents’ individual, uncoordinated actions occur as if they were carefully choreographed.Footnote 2 This is no easy task. But the to-each-his-own approach prevailing in most places in recent decades has left us on a course to climate, environmental, health, and social ruin.

The objective of the social process is to induce decentralized behaviors throughout AVCs that together drive beneficial innovations that steadily transition systems towards the HERS objectives. This requires the dialogue that is the first part. But ultimately, the key is this second part: agreed actions that together comprise a set of shared responsibilities and the KPMs and enforcement mechanisms necessary to adjust and enforce those agreements.

What are key elements of dialogues that generate the set of agreed shared responsibilities necessary to co-create beneficial socio-technical bundles? We articulate them here and then illustrate them with an example.

  • Identify system objectives as well as each actor’s key incentives. While the HERS design objectives apply in all contexts, these commonly need supplementation for a specific context. Moreover, one must not naively ignore the reality that different actors pursue varied goals: profit, political power, social or environmental outcomes, etc. Co-management works best when all parties acknowledge the diversity of stakeholder objectives. Forthrightness helps to build mechanisms that can accommodate all parties’ interests. Failure to do so often leads to defection from agreements and non-cooperation, with adverse results for all parties.

  • Articulate shared responsibilities. What actions can/must different actors/organizations take in order to elicit and scale beneficial innovation in a given AFS? And what actions does each actor need others to take in order to induce these necessary actions? Identifying these shared responsibilities for mutual action is an essential step of the process.

  • Agree on key performance measures. Organizations inevitably manage to what they measure. A manageable dashboardFootnote 3 of reliable, low-cost, transparent measures of KPMs that track progress towards goals (step 1) and success in fulfilling actors’ responsibilities (step 2) is therefore essential.Footnote 4

  • Develop open monitoring and enforcement mechanisms. Trust is among the scarcest and most valuable renewable resources in any society (Barrett 1997; Ostrom 2010). Checks and balances are needed to prevent any actor from shirking its agreed responsibilities. But for enforcement to be credible, the consequences of failure to perform must be transparent and agreed upon.

The outcomes of these four key steps necessarily vary by food system, by technology domain, and over time. The incentives, shared responsibilities, indicators, and monitoring and enforcement mechanisms necessary for building a transformative digital stack in emerging and diversifying systems in South and Southeast Asia will surely differ considerably from those needed to guide de-agrarianization of protein production for consumers in the industrialized and consolidated systems of much of North America. Those will be different still from those needed to adapt and diffuse biofortified, nitrogen-fixing, or stress-tolerant staple-crop varieties in rural and traditional systems in sub-Saharan Africa. We cannot be prescriptive about specific actions, only about the necessity of dialogue and of agreement on shared responsibilities, KPMs, and monitoring and enforcement mechanisms.

Our panel developed one admittedly general example, abstracted from any specific system, simply to illustrate the idea. In Fig. 1, we sketch out an actor-specific action agenda that food manufacturers and retailers could pursue to help drive beneficial innovations aligned with HERS objectives. This simple framework results in an enumeration of actions individual firms—and industry groups—can take that can help drive food systems in needed directions. It likewise identifies actions they need counterparties (e.g., consumer groups, governments) to take so that those firm actions are both feasible and compatible with the firm’s own financial incentives. A key part of the point of the dialogues is to acknowledge each parties’ legitimate, if idiosyncratic objectives. Even those private companies that authentically commit to shared societal goals, such as the SDGs or longer-run HERS objectives, must provide shareholders with satisfactory returns on equity, taking into consideration not just near-term profits, but also goodwill, market share, risk management, and stakeholder (e.g., employee, community, regulator) satisfaction goals, as well. So all parties ultimately need to identify strategies that can make good behavior sufficiently profitable to persist in the face of market pressures.

Fig. 1
figure 1

An illustrative, coordinated-action agenda regarding food product standards from the standpoint of food processors, manufacturers, and retailers. Identify distinct actors, the complementary actions each needs to take in order to generate mutually reinforcing beneficial responses, and sample key performance measures (KPMs) to track

Full size image

For each action, there should be an explicit KPM. The Food Systems Dashboard curates a rich list of KPMs at food system type or country level. A key to good KPMs is that they follow a standard, accepted best-measurement practice, are available reasonably universally and promptly, and are inexpensive to gather and distribute. The dashboard of KPMs appropriate to any dialogue of AFS partners will necessarily vary by system and objectives.

There exist several implications of the need for multiple parties in these social dialogues, so as to be able to agree to reciprocal action obligations. One is the necessity of building and maintaining trust and transparency. This can be difficult. It requires courageous leaders. Another is the need for adequate, locally knowledgeable scientific capacity to credibly engage on the research needs. In some places, such as much of sub-Saharan Africa, this requires collective investment in building the required, local scientific capacity where little currently exists. Third, this almost always requires cross-sector dialogue and coordination, as few of the actions and reciprocal actions needed to enable progress get confined to the boundaries of organizational charts.

Progress is feasible but fundamentally depends on AFS innovations to obviate pressing natural resource constraints and address looming food demand growth. As the Matt Ridley quote that opened the report emphasizes, “[I]nnovation is the most important fact about the modern world, but one of the least well understood” (Ridley 2020). People too often assume that science alone can, and will, rescue us. But translating first-rate scientific research into human progress relies inexorably on human goodwill, cooperation, and ingenuity; on incentives crafted so as to induce mutually reinforcing individual and collective behaviors; and on shared commitment both to common goals and to coordinated-action agendas that diverse stakeholders can agree on through mutually respectful dialogue. In short, human agency ultimately drives the innovation process. The most expeditious and likely path to co-creation of the socio-technical innovation bundles needed to navigate humanity away from clear and present climate, environmental, health, and socioeconomic dangers and to HERS AFSs ultimately rely on human dialogue and action. We have met past challenges and prevailed. We can do so again. But we cannot afford to delay.