Abstract
With the growing importance of Foresight in strategic decision-making, there is parallel growth in interest in ways in which we might evaluate Foresight activities and outputs, and thus learn how to improve their effectiveness. But general principles for such evaluation have not yet been formulated. The purpose of this chapter is to identify the basic criteria and methods used in evaluating national Foresight exercises. Reviewing a range of evaluation efforts, it presents some interesting results for specific tasks, themes examined, methods and findings. A key motivation for these evaluations was to provide feedback to national Foresight organizations and identify areas for further development. Evaluation is used to guide Foresight and to learn lessons that can inform future projects (for example, lessons concerning simplifying implementation, involving business and social organizations, recognizing the objectives of participants and the need for better compliance with the methodology). We suggest a general methodology for assessing national Foresight programmes, together with a framework that can be used to improve the comparability of results of Foresight evaluation initiatives in different countries, and even to promote the standardization and upgrading of evaluation procedures.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Behavioural additionality is the difference in actors’ behaviour resulting from the Foresight intervention (Georghiou et al. 2004b).
- 2.
It is a framework of the adjusted integrated Foresight management model.
- 3.
The main reasons for that choice were the success of the evaluation procedures and the openness of information. The latter is a crucial requirement: evaluation results are sometimes classified. For example, evaluation reports on “FUTUR” and “Vision 2023” were not published, although the sufficient minimum of information was presented in several papers (e.g. Cuhls and Georghiou 2004; Dursun et al. 2011).
- 4.
Bundesministerium für Bildung und Forschung, BMBF.
- 5.
For main themes were analysed: Create Open Access to Tomorrow’s World of Learning, Living in the Networked World: Individual and Secure, Healthy and Vital throughout Life through Prevention, Understanding Thought Processes (Cuhls and Georghiou 2004).
- 6.
There is a wide range of indicators for the last two topics evaluation: specific criteria are developed and applied to meet a particular project’s needs (e.g. Johnston 2012; Chan and Daim 2012; Miles 2012; Kappel 2001). In-depth analysis of Foresight results and impact is beyond the bounds of the research: it’s an issue for further development.
- 7.
Capability to influence on the situation in national innovation system (Meissner and Cervantes 2008).
References
Alsan A, Öner MA (2004) Comparison of national Foresight studies by integrated Foresight management model. Futures 36(8):889–902
Amanatidou E, Guy K (2008) Interpreting Foresight process impacts: steps towards the development of a framework conceptualising the dynamics of ‘Foresight systems’. Technol Forecast Soc Change 75(4):539–557
Bezold C (2010) Lessons from using scenarios for strategic Foresight. Technol Forecast Soc Change 77:1513–1518
Bohanec M, Rajkovič V, Semolič B, Pogačnik A (1995) Knowledge-based portfolio analysis for project evaluation. Inf Manage 28:293–302
Calof J (2011) Evaluating future technology assessment – Canadian Case Study. Paper presented at the 4th international Seville conference on future-oriented technology analysis (FTA), Seville, 12–13 May. http://ftp.jrc.es/EURdoc/JRC64647.pdf
Calof J, Smith JE (2008) Critical success factors for government led Foresight. Paper presented at the third international Seville seminar on future oriented technology analysis: impacts and implications for policy and decision-making, Seville, 16–17 October
Chan L, Daim T (2012) Exploring the impact of technology Foresight studies on innovation: case of BRIC countries. Futures 44(6):618–630
Clean Air Partnership (CAP) (2010) Evaluation toolkit: a guide for environmental NGOs. http://www.cleanairpartnership.org/building_evaluation_capacity_private
Cuhls K (2003) From forecasting to Foresight processes—new participative Foresight activities in Germany. J Forecast 22:93–111
Cuhls K, Georghiou L (2004) Evaluating a participative Foresight process: “Futur—the German research dialogue”. Res Eval 13(3):143–153
Destatte P (2007) Evaluation of Foresight: how to take long term impacts into consideration? FOR-LEARN Mutual Learning Workshop-Evaluation of Foresight, Brussels
Dursun O, Türe TE, Daim TU (2011) Post-evaluation of Foresight studies: Turkish case. Int J Foresight Innov Policy 7(4):311–337
Eilat H, Golany B, Shtub A (2008) R&D project evaluation: an integrated DEA and balanced scorecard approach. Omega 36:895–912
Georghiou L (2003) Evaluating Foresight and lessons for its future impact. Paper presented at the second international conference on technology Foresight, Tokyo, 27–28 February
Georghiou L, Keenan M (2006) Evaluation of national Foresight activities: assessing rationale, process and impact. Technol Forecast Soc Change 73(7):761–777
Georghiou L, Acheson H, Cassingena Harper J, Clar G, Klusacek K (2004a) Evaluation of the Hungarian Technology Foresight Programme. http://www.nih.gov.hu/english/technology-foresight/evaluation-of-the-080519
Georghiou L, Clarysse B, Steurs G, Bilsen V, Larosse J (2004b) Making the difference: the evaluation of behavioural additionality of R&D subsidies. IWT Studies 48:7–20
Georghiou L, Keenan M, Miles I, Cameron H (2006) An evaluation of the UK Foresight Programme: final report, University of Manchester
German Ministry for Education and Research, BMBF (2002) Futur lead visions complete document, Berlin
Giesecke S (2008) Futur – the German research dialogue. In: Giesecke S, Crehan P, Elkins S (eds) The European Foresight Monitoring Network. Collection of EFMN Briefs – Part 1. Office for Official Publications of the European Communities, Luxembourg. http://ec.europa.eu/research/social-sciences/pdf/efmn-report_en.pdf
Grun RE (2006) Monitoring and evaluating projects: a step-by-step primer on monitoring, Benchmarking and impact evaluation. The World Bank, Washington, DC
Habegger B (2010) Strategic Foresight in public policy: reviewing the experiences of the UK, Singapore, and the Netherlands. Futures 42(1):49–58
Havas A, Schartinger D, Weber A (2010) The impact of Foresight on innovation policy-making: recent experiences and future perspectives. Res Eval 19(2):91–104
IFAD (2009) Evaluation manual: methodology and processes. http://www.ifad.org/evaluation/process_methodology/doc/manual.pdf
JICA (2004) Guideline for project evaluation: practical methods for project evaluation. http://www.jica.go.jp/english/operations/evaluation/tech_and_grant/guides/pdf/guideline01-01.pdf
Johnston R (2012) Developing the capacity to assess the impact of Foresight. Foresight 14(1):56–68
Kappel TA (2001) Perspectives on roadmaps: how organizations talk about the future. J Product Innov Manage 18:39–50
Kováts et al (2000) The Hungarian Technology Foresight Programme, report by the Steering Group, Budapest
Meissner D, Cervantes M (2008) Results and impact of national Foresight-studies. Paper presented at the third international Seville seminar on future-oriented technology analysis: impacts and implications for policy and decision-making, Seville, 16–17 October
Miles I (with contributions from Mike Keenan) (2003) Ten years of Foresight in the UK. Paper presented at NISTEP’s second international conference on technology Foresight, Tokyo
Miles I (2002) Appraisal of alternative methods and procedures for producing regional Foresight, report prepared by CRIC for the European Commission’s DG research funded STRATA – ETAN Expert Group Action, Manchester. http://ec.europa.eu/research/social-sciences/pdf/appraisalof-alternative-methods_en.pdf
Miles I (2012) Dynamic Foresight evaluation. Foresight 14(1):69–81
Phillips JJ, Bothell TW, Snead GL (2002) The project management scorecard: measuring the success of project management solutions. Butterworth-Heinemann, New York
Popper R (2008) Foresight methodology. In: Georghiou L, Cassingena J, Keenan M, Miles I, Popper R (eds) The handbook of technology Foresight. Edward Elgar, Cheltenham
Popper R (2009) Mapping Foresight: revealing how Europe and other world regions navigate into the future. Publications Office of the European Union, European Commission, EFMN, Luxembourg
Popper R, Georghiou L, Keenan M, Miles I (2010) Evaluating Foresight: fully-fledged evaluation of Colombian Technology Foresight Programme. Universidad del Valle, Santiago de Cali
Project Management Institute (PMI) (1996) A guide to the project management body of knowledge (PMBOK Guide). Project Management Institute, Newtown Square
Rader M (2003) Hungary – Hungarian Foresight Programme (TEP) 1997/99. FISTERA report WP 1 – review and analysis of national Foresight D1.1 HU – case study
Remer DS, Nieto AP (1995a) A compendium and comparison of 25 project evaluation techniques. Part 1: net present value and rate of return methods. Int J Prod Econ 42(1):79–96
Remer DS, Nieto AP (1995b) A compendium and comparison of 25 project evaluation techniques. Part 2: ratio, payback and accounting Methods. Int J Prod Econ 42(2):101–129
Ricker KT et al (1998) Water quality project evaluation: a handbook for objectives-based evaluation of water quality projects. http://ohioline.osu.edu/b868
Rijkens-Klomp N, Duin van der P (2011) Evaluating local public Foresight studies from a user perspective. Paper presented at the 4th international Seville conference on future-oriented technology analysis (FTA), Seville, 12–13 May. http://ftp.jrc.es/EURdoc/JRC64647.pdf
Rollwagen I, Hofmann J, Schneider S (2008) Improving the business impact of Foresight. Technol Anal Strateg Manage 20(3):337–349
Saritas O, Taymaz E, Tumer T (2007) Vision 2023: Turkey’s national technology Foresight program: a contextualist analysis and discussion. Technol Forecast Soc Change 74:1374–1393
Treasury HM (2003) The green book: appraisal and evaluation in central government. TSO, London
Westat JF (2002) The 2002 user-friendly handbook for project evaluation, NSF publication. http://www.nsf.gov/pubs/2002/nsf02057/start.htm
Yoda T (2011) Perceptions of domain experts on impact of Foresight on policy making: the case of Japan. Technol Forecast Soc Change 78(3):431–447
Zarinpoush F (2006) Project evaluation guide for nonprofit organizations: fundamental methods and steps for conducting project evaluation. http://library.imaginecanada.ca/files/nonprofitscan/en/csc/projectguide_final.pdf
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Sokolova, A., Makarova, E. (2013). Integrated Framework for Evaluation of National Foresight Studies. In: Meissner, D., Gokhberg, L., Sokolov, A. (eds) Science, Technology and Innovation Policy for the Future. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-31827-6_2
Download citation
DOI: https://doi.org/10.1007/978-3-642-31827-6_2
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-31826-9
Online ISBN: 978-3-642-31827-6
eBook Packages: Business and EconomicsEconomics and Finance (R0)