Towards more effective and transferable transition experiments: learning through stratification

  • John-Oliver EnglerEmail author
  • Heike Zimmermann
  • Daniel J. Lang
  • Robert L. Feller
  • Henrik von Wehrden
Original Article
Part of the following topical collections:
  1. Sustainability Transitions, Management, and Governance


We argue that the increasing use of experimental approaches in transition and transformational research on potential sustainability solutions could substantially profit from application of the principles of stratification and stratified experimental design. We illustrate our proposition with three worked examples of hypothetical transition experiments and argue that the use of stratification can produce three essential benefits in transition experiments: (1) increased methodological validity and better comparability of experimental results, (2) potential scalability of sustainability solutions tested, and (3) better assessment of potential transferability of sustainability solutions to other systems. Moreover, stratification and stratified design of transition experiments might be employed to address some of the recently raised criticisms regarding reproducibility of results and the appropriate use of statistical methodology in scientific experimentation in general and transition experimentation in particular.


Stratification Experimental design Transition experiments Sustainability Statistical testing 



The authors gratefully acknowledge funding from the State of Lower Saxony (Niedersächsisches Ministerium für Wissenschaft und Kultur) and the Volkswagen Foundation in line with the research project “Bridging the Great Divide” (Grant number VWZN3188). The manuscript benefited from various discussions with Nigel Forrest, Aditya Ghosh, Cynthia Girling, Beatrice John, Manfred Laubichler, Jon Salter, Arnim Wiek, and Lauren Withycombe-Keeler, particularly with regard to the concept and use of transition experiments in sustainability science.

Compliance with ethical standards

Conflict of interest

The authors declare that they have no conflict of interest.


  1. Abson DJ, Fischer J, Leventon J, Newig J, Schomerus T, Vilsmaier U, von Wehrden H, Abernethy P, Ives CD, Jager NW, Lang DJ (2017) Leverage points for sustainability transformation. Ambio 46(1):30–39Google Scholar
  2. Antikainen R, Alhola K, Jääskeläinen T (2017) Experiments as a means towards sustainable societies—lessons learnt and future outlooks from a finnish perspective. J Clean Product 169:216–224Google Scholar
  3. Auten G, Carroll R (1999) The effect of income taxes on household income. Rev Econ Stat 81:681–693Google Scholar
  4. Banneheka SG, Routledge RD, Schwarz CJ (1997) Stratified two-sample tag-recovery census of closed populations. Biometrics 53:1212–1224Google Scholar
  5. Banting D, Doshi H, Li J, Missios P (2005) Report on the benefits and costs of green roof technology for the city of Toronto, Ryerson University. Online Accessed 30 Mar 2009Google Scholar
  6. Baumol W (1952) Welfare economics and the theory of the state. Harvard University Press, CambridgeGoogle Scholar
  7. Bernstein MJ, Wiek A, Brundiers K, Pearson K, Minowitz A, Kay B, Golub A (2016) Mitigating urban sprawl effects—a collaborative tree and shade intervention in Phoenix, Arizona, USA. Local Environ 21(4):414–431Google Scholar
  8. BGD [Bridging the Great Divide] (2018) Bridging the great divide in sustainability science - linking high-performance modeling and transition experiments to foster transformational change towards sustainability.
  9. Bird W (2007) Natural thinking, Technical report June 2007. Royal Society for the Protection of Birds, OxfordGoogle Scholar
  10. Bowler DE, Buyung-Ali L, Knight TM, Pullin AS (2010) Urban greening to cool towns and cities: a systematic review of the empirical evidence. Landsc Urban Plan 97(3):147–155Google Scholar
  11. Brandt P, Ernst A, Gralla F, Luederitz C, Lang DJ, Newig J, Reinert F, Abson DJ, von Wehrden H (2013) A review of transdisciplinary research in sustainability science. Ecol Econ 92:1–15Google Scholar
  12. Brazel A, Gober P, Lee SJ, Grossman-Clarke S, Zehnder J, Hedquist B, Comparri E (2007) Determinants of changes in the regional urban heat island in metropolitan Phoenix (Arizona, USA) between 1990 and 2004. Clim Res 33(2):171–182Google Scholar
  13. Bunce R, Barr C, Whittaker H (1983) A stratification system for ecological sampling. In: Fuller R (ed) Ecological mapping from ground, air and space. Cambridge Press, Cambridge, pp 39–46Google Scholar
  14. Campbell DT (1957) Factors relevant to the validity of experiments in social settings. Psychol Bull 54(4):297–312Google Scholar
  15. Campbell JB (1996) Introduction to remote sensing. The Guilford Press, New YorkGoogle Scholar
  16. Caniglia G, Schäpke N, Lang DJ, Abson DJ, Luederitz C, Wiek A, Laubichler MD, Gralla F, von Wehrden H (2017) Experiments and evidence in sustainability science: a typology. J Clean Product 169:39–47Google Scholar
  17. Caso C, Gil M (1988) The Gini-Simpson Index of diversity—estimation in the stratified sampling. Commun Stat Theory Methods 17:2981–2995Google Scholar
  18. Churchman CW (1967) Wicked problems. Manag Sci 14(4):141–146Google Scholar
  19. Cilliers S (2010) Social Aspects of Urban Biodiversity—An Overview. In: Müller N, Werner P, Kelcey JG (eds) Urban Biodiversity and Design, 81–100. Wiley-Blackwell, New YorkGoogle Scholar
  20. Clark WC (2002) Science and technology for sustainable development: consensus report of the Mexico City synthesis workshop, May 20–23, 2002. Initiative on Science and Technology for Sustainability, CambridgeGoogle Scholar
  21. Cochran WG (1977) Sampling techniques. Wiley, New YorkGoogle Scholar
  22. Coenen L, Benneworth P, Truffer B (2012) Toward a spatial perspective on sustainability transitions. Res Policy 41(6):968–979Google Scholar
  23. Cormack R (1988) Statistical challenges in the environmental sciences: a personal view. J R Stat Soc A 151:201–210Google Scholar
  24. Currie BA, Bass B (2005) Estimate of air pollution mitigation with green plants and green roofs using the UFORE model. In: Proceedings of third annual greening rooftops for sustainable communities conference, awards and trade show, Washington, DC, May 4–6, 2005Google Scholar
  25. Davis AR, Doyle R (2015) Transforming household consumption: from backcasting to HomeLabs experiments. Ann Assoc Am Geogr 105(2):425–436Google Scholar
  26. de Bruijne M, van de Riet O, de Haan A, Koppenjan J (2010) Dealing with dilemma’s: how can experiments contribute to a more sustainable mobility system? Eur J Transp Infrastruct Res 10(3):274–289Google Scholar
  27. Dekker K (2007) Social capital, neighbourhood attachment and participation in distressed urban areas. A case study in The Hague and Utrecht, the Netherlands. Hous Stud 22(3):355–379Google Scholar
  28. Duflo E, Kremer M (2005) Use of randomization in the evaluation of development effectiveness. In: Pitman GK, Feinstein ON, Ingram GK (eds) Evaluating development effectiveness, vol 7 of World Bank Series on evaluation and development. Transaction Publishers, New BrunswickGoogle Scholar
  29. Dwivedi A, Mohan BK (2018) Impact of green roof on micro climate to reduce Urban Heat Island. Remote Sens Appl Soc Environ 10:56–69Google Scholar
  30. English Nature (2003) Green roofs: their existing status and potential for conserving biodiversity in urban areas. English Nature Report No. 498, PeterboroughGoogle Scholar
  31. Fox WE, McCollum DW, Mitchell JE, Swanson LE, Kreuter UP, Tanaka JA, Evans GR, Theodore Heintz H, Breckenridge RP, Geissler PH (2009) An integrated social, economic, and ecological conceptual (ISEEC) framework for considering rangeland sustainability. Soc Nat Resour 22(7):593–606Google Scholar
  32. Frantzeskaki N, Wittmayer J, Loorbach D (2014) The role of partnerships in ‘realising’ urban sustainability in Rotterdam’s City Ports Area, The Netherlands. J Clean Prod 65:406–417Google Scholar
  33. Fünfschilling L, Truffer B (2014) The structuration of socio-technical regimes—conceptual foundations from institutional theory. Res Policy 43:772–791Google Scholar
  34. Funtowicz SO, Ravetz JR (1991) A new scientific methodology for global environmental issues. In: Costanza R (ed) Ecological economics: the science and management of sustainability. Columbia University Press, New York, pp 137–152Google Scholar
  35. Funtowicz SO, Ravetz JR (1993) Science for the post-normal age. Futures 31(7):735–755Google Scholar
  36. Gardener M (1976) Mathematical games: on the fabric of inductive logic, and some probability paradoxes. Sci Am 234(3):119–124Google Scholar
  37. Geels FW (2011) The multi-level perspective on sustainability transitions: responses to seven criticisms. Environ Innov Soc Transit 1(1):24–40Google Scholar
  38. Getter KL, Rowe DB (2006) The role of green roofs in sustainable development. Hortic Sci 41:1276–1286Google Scholar
  39. Graham I, Cooney T, De Bacquer D (2015) Risk stratification and risk assessment. In: Gielen S, De Backer G, Piepoli M, Wood D (eds) The ESC textbook of preventive cardiology. Oxford University Press, OxfordGoogle Scholar
  40. Hampton SE, Strasser CA, Tewksbury JJ, Gram WK, Budden AE, Batcheller AL, Duke CS, Porter JH (2013) Big data and the future of ecology. Front Ecol Environ 11(3):156–162Google Scholar
  41. Hedayat AS, Stufken J (1998) Sampling designs to control selection probabilities of contiguous units. J Stat Plan Inference 72:333–345Google Scholar
  42. Heinrichs H, Michelsen G (eds) (2014) Nachhaltigkeitswissenschaften. Springer, BerlinGoogle Scholar
  43. Hogan J, Lancaster T (2004) Instrumental variables and inverse probability weighting for causal inference from longitudinal observational studies. Stat Methods Med Res 13:17–48Google Scholar
  44. Hölscher K, Wittmayer JM, Loorbach D (2018) Transition versus transformation: what’s the difference? Environ Innov Soc Transit 27:1–3Google Scholar
  45. Jessen T (1945) The master sample of agriculture: II, design. J Am Stat Assoc 40:45–56Google Scholar
  46. Kates RW, Clark WC, Corell R, Hall JM, Jäger CC, Lowe I, McCarthy JJ, Schellnhuber HJ, Bolin B, Dickson NM, Faucheux S, Gallopin GC, Grübler A, Huntley B, Jäger J, Jodha NS, Kasperson RE, Mabogunje A, Matson P, Mooney H, Moore B III, O’Riordan T, Svedin U (2001) Sustain Sci. Science 292:641–642Google Scholar
  47. Kivimaa P, Hildén M, Huitema D, Jordan A, Newig J (2017) Experiments in climate governance—a systematic review of research on energy and built environment transitions. J Clean Prod 169:17–29Google Scholar
  48. Korpela K, Hartig T (1996) Restorative qualities of favorite places. J Environ Psychol 16(3):221–233Google Scholar
  49. Korpela KM, Hartig T, Kaiser FG, Fuhrer U (2001) Restorative experience and self-regulation in favorite places. Environ Behav 33(4):572–589Google Scholar
  50. Lackey RT (1972) Response of physical and chemical parameters to eliminating thermal stratification in a reservoir. J Am Water Resour Assoc 8(3):589–599Google Scholar
  51. Lang DJ, Wiek A, Bergmann M, Stauffacher M, Martens P, Moll P, Swilling M, Thomas CJ (2012) Transdisciplinary research in sustainability science: practice, principles, and challenges. Sustain Sci 7:25–43Google Scholar
  52. Lang DJ, Wiek A, von Wehrden H (2017) Bridging divides in sustainability science. Sustain Sci 12(6):875–879Google Scholar
  53. Leek J, McShane B, Gelman A, Colquhoun D, Nuijten MB, Goodman S (2017) Five ways to fix statistics. Nature 551:557–559Google Scholar
  54. Levin K, Cashore B, Bernstein S, Auld G (2012) Overcoming the tragedy of super wicked problems: constraining our future selves to ameliorate global climate change. Policy Sci 45(2):123–152Google Scholar
  55. Loorbach D, Rotmans J (2010) The practice of transition management: examples and lessons from four distinct cases. Futures 42(3):237–246Google Scholar
  56. Loorbach D, Frantzeskaki N, Avelino F (2017) Sustainability transitions research: transforming science and practice for societal change. Annu Rev Environ Resour 42:599–626Google Scholar
  57. Luck M, Wu J (2002) A gradient analysis of urban landscape pattern: a case study from the Phoenix metropolitan region, Arizona, USA. Landsc Ecol 17(4):327–339Google Scholar
  58. Luederitz C, Schäpke N, Wiek A, Lang DJ, Bergmann M, Bos JJ, Burch S, Davies A, Evans J, König A, Farrelly MA, Forrest N, Frantzeskaki N, Gibson RB, Kay B, Loorbach D, McCormick K, Parodi O, Rauschmeyer F, Schneidewind U, Stauffacher M, Stelzer F, Trencher G, Venjakob J, Vergragt PJ, von Wehrden H, Westley FR (2017) Learning through evaluation—a tentative evaluative scheme for sustainability transition experiments. J Clean Prod 169:61–76Google Scholar
  59. Markard J, Truffer B (2008) Technological innovation systems and the multi-level perspective: towards an integrated framework. Res Policy 37:596–615Google Scholar
  60. McShane BB, Gal D (2017) Statistical significance and the dichotomization of evidence. J Am Stat Assoc 112(519):885–895Google Scholar
  61. Meadows DH (1999) Leverage points: places to intervene in a system. The Sustainability Institute, Hartland.
  62. Meinzen-Dick R (2007) Beyond panaceas in water institutions. Proc Natl Acad Sci 25:39Google Scholar
  63. Millard A (2010) Cultural aspects of urban biodiversity. In: Müller N, Werner P, Kelcey JG (eds) Urban biodiversity and design. Wiley-Blackwell, Hoboken, pp 56–80Google Scholar
  64. Miller CB (2004) Biological oceanography. Blackwell Publishing, HobokenGoogle Scholar
  65. Neyman J (1934) On the two different aspects of the representative method: the method of stratified sampling and the method of purposive selection. J R Stat Soc 97:558–625Google Scholar
  66. Opsomer JD, Nusser SM (1999) Sample designs for watershed assessment. J Agric Biol Environ Stat 4:429–442Google Scholar
  67. Ostrom E (2009) A general framework for analyzing sustainability of social-ecological systems. Science 325:419–422Google Scholar
  68. Porter N, Claassen M, Timmermans J (2015) Transition experiments in Amsterdam: conceptual and empirical analysis of two transition experiments in the WATERgraafsmeer Program. Technol Forecast Soc Chang 90:525–537Google Scholar
  69. Ridder G, Moffitt R (2007) The econometrics of data combination. Handbook of econometrics, chap 75, vol 6B. Elsevier, pp 5469–5547Google Scholar
  70. Rip A, Kemp R (1998) Technological change. In: Rayner S, Malone E (eds) Human choice and climate change, vol 2. Battelle, Columbus, pp 327–392Google Scholar
  71. Rittel HWJ, Webber MM (1973) Dilemmas in a general theory of planning. Policy Sci 4(2):155–169Google Scholar
  72. Salsburg D (2001) The lady tasting tea: how statistics revolutionized science in the twentieth century. W.H. Freemann and Company, New YorkGoogle Scholar
  73. Schäpke N, Omann I, Wittmayer JM, Steenbergen F, van Mock M (2017) Linking transitions to sustainability: a study of the societal effects of transition management. Sustainability 9(5):737Google Scholar
  74. Scholz RW, Stauffacher M, Bösch S, Krütli P (2004) Mobilität und zukunftsfähige Stadtentwicklung: Freizeit in der Stadt Basel, Verlag Rüegger AG, Zürich in Zusammenarbeit mit Pabst Science Publishers, LengerichGoogle Scholar
  75. Sengers F, Wieczorek AJ, Raven R (2016) Experimenting for sustainability transitions: a systematic literature review. Technol Forecast Soc Change. Google Scholar
  76. Shove E, Walker G (2007) Commentary: caution! Transitions ahead: politics, practice, and sustainable transition management. Environ Plan A 39:763–770Google Scholar
  77. Simpson EH (1951) The interpretation of interaction in contingency tables. J R Stat Soc B 13:238–241Google Scholar
  78. Smith A, Voß J-P, Grin J (2010) Innovation studies and sustainability transitions: the allure of the multi-level perspective and its challenges. Res Policy 39:435–448Google Scholar
  79. Spangenberg JH (2011) Sustainability science: a review, an analysis and some empirical lessons. Environ Conserv 38(3):275–287Google Scholar
  80. Stephan F (1948) History of the uses of modern sampling procedures. J Am Stat Assoc 43:12–39Google Scholar
  81. Stovin V (2010) The potential of green roofs to manage urban storm water. Water Environ J 24:192–199Google Scholar
  82. Trost JE (1986) Statistically nonrepresentative stratified sampling: a sampling technique for qualitative studies. Qual Sociol 9(1):54–57Google Scholar
  83. Turnheim B, Kivimaa P, Berkhout F (2018) Experiments and beyond: an emerging agenda for climate governance innovation. In: Turnheim B, Kivimaa P, Berkhout F (eds) Innovating climate governance—moving beyond experimentation. Cambridge University Press, UKGoogle Scholar
  84. van den Bosch S (2010) Transition experiments: exploring societal changes towards sustainability, Ph.D. Dissertation, Erasmus University Rotterdam, retrieved from on 27 Oct 2017
  85. von Wehrden H, Schultner J, Abson DJ (2015) A call for statistical editors in ecology. Trends Ecol Evol 30(6):293–294Google Scholar
  86. von Wehrden H, Luederitz C, Leventon J, Russel S (2017) Methodological challenges in sustainability science: a call for method plurality, procedural rigor and longitudinal research. Chall Sustain 5(1):35–42Google Scholar
  87. Wasserstein RL, Lazar NA (2016) The ASA’s statement on p-values: context, process, and purpose. Am Stat 70(2):129–133Google Scholar
  88. Weber KM, Rohracher H (2012) Legitimizing research, technology and innovation policies for transformative change: combining insights from innovation systems and multi-level perspective in a comprehensive ‘failures’ framework. Res Policy 41(6):1037–1047Google Scholar
  89. Weiland S, Bleicher A, Polzin C, Rauschmeyer F, Rode J (2017) The nature of experiments for sustainability transformations: a search for common ground. J Clean Prod 169:30–38Google Scholar
  90. Whittow JB (1984) The penguin dictionary of physical geography. Penguin Books, LondonGoogle Scholar
  91. Wiek A, Withycombe L, Redman CL (2011) Key competencies in sustainability: a reference framework for academic program development. Sustain Sci 6:203–218Google Scholar
  92. Wittig R (2010) Biodiversity of urban-industrial areas and its evaluation—a critical review. In: Müller N, Werner P, Kelcey JG (eds) Urban Biodiversity and Design. Wiley-Blackwell, Hobobken, pp 35–55Google Scholar

Copyright information

© Springer Japan KK, part of Springer Nature 2019

Authors and Affiliations

  • John-Oliver Engler
    • 1
    Email author
  • Heike Zimmermann
    • 1
  • Daniel J. Lang
    • 1
  • Robert L. Feller
    • 1
  • Henrik von Wehrden
    • 1
  1. 1.Faculty of Sustainability, Center for Global Sustainability and Cultural TransformationLeuphana University of LüneburgLüneburgGermany

Personalised recommendations