Surveys in Geophysics

, Volume 36, Issue 2, pp 269–293 | Cite as

Scoring and Testing Procedures Devoted to Probabilistic Seismic Hazard Assessment



This review addresses long-term (tens of years) seismic ground-motion forecasting (seismic hazard assessment) in the presence of alternative computational models (the so-called epistemic uncertainty affecting hazard estimates). We review the different approaches that have been proposed to manage epistemic uncertainty in the context of probabilistic seismic hazard assessment (PSHA). Ex-ante procedures (based on the combination of expert judgments about inherent characteristics of the PSHA model) and ex-post approaches (based on empirical comparison of model outcomes and observations) should not be considered as mutually exclusive alternatives but can be combined in a coherent Bayesian view. Therefore, we propose a procedure that allows a better exploitation of available PSHA models to obtain comprehensive estimates, which account for both epistemic and aleatory uncertainty. We also discuss the respective roles of empirical ex-post scoring and testing of alternative models concurring in the development of comprehensive hazard maps. In order to show how the proposed procedure may work, we also present a tentative application to the Italian area. In particular, four PSHA models are evaluated ex-post against macroseismic effects actually observed in a large set of Italian municipalities during the time span 1957–2006. This analysis shows that, when the whole Italian area is considered, all the models provide estimates that do not agree with the observations. However, two of them provide results that are compatible with observations, when a subregion of Italy (Apulia Region) is considered. By focusing on this area, we computed a comprehensive hazard curve for a single locality in order to show the feasibility of the proposed procedure.


Seismology Seismic hazard PSHA Testing Probability 



This study has benefited from funding provided by the Italian Presidenza del Consiglio dei Ministri -Dipartimento della Protezione Civile (DPC). This paper does not necessarily represent DPC official opinion and policies. Many thanks are also due to Prof. Frederik Tilmann and two anonymous referees whose comments and suggestions helped us in improving the text.


  1. Abrahamson NA, Bommer JJ (2005) Probability and uncertainty in seismic hazard analysis. Earthq Spectra 21(2):603–607CrossRefGoogle Scholar
  2. Abrahamson NA, Birkhauser P, Koller M, Mayer-Rosa D, Smit P, Sprecher C, Tinic S, Graf R (2002) PEGASOS—a comprehensive probabilistic seismic hazard assessment for nuclear power plants in Switzerland. In: Proceedings of the 12th European conference on earthquake engineering, London, Paper, vol 633Google Scholar
  3. Albarello D, D’Amico V (2005) Validation of intensity attenuation relationships. Bull Seismol Soc Am 95(2):719–724CrossRefGoogle Scholar
  4. Albarello D, D’Amico V (2008) Testing probabilistic seismic hazard estimates by comparison with observations: an example in Italy. Geophys J Int 175:1088–1094CrossRefGoogle Scholar
  5. Albarello D, Mucciarelli M (2002) Seismic hazard estimates from ill-defined macroseismic data at a site. Pure Appl Geophys 159(6):1289–1304CrossRefGoogle Scholar
  6. Albarello D, Bramerini F, D’Amico V, Lucantoni A, Naso G (2002) Italian intensity hazard maps: a comparison between results from different methodologies. Boll Geofis Teor Appl 43:249–262Google Scholar
  7. Anooshehpoor A, Brune JN, Zeng Y (2004) Methodology for obtaining constraints on ground motion from precariously balanced rocks. Bull Seismol Soc Am 94(1):285–303CrossRefGoogle Scholar
  8. Azzaro R, Barbano MS, Moroni A, Mucciarelli M, Stucchi M (1999) The seismic history of Catania. J Seismol 3:235–252CrossRefGoogle Scholar
  9. Azzaro R, Barbano MS, D’Amico S, Tuvè T, Albarello D, D’Amico V (2008) Preliminary results of probabilistic seismic hazard assessment in the volcanic region of Mt Etna (Southern Italy). Boll Geofis Teor Appl 49(1):77–91Google Scholar
  10. Beauval C (2011) On the use of observations for constraining probabilistic seismic hazard estimates—brief review of existing methods. In: International conference on applications of statistics and probability in civil engineering, August 1–4, Zurich, p 5Google Scholar
  11. Beauval C, Bard P-Y, Hainzl S, Guéguen P (2008) Can strong-motion observations be used to constrain probabilistic seismic-hazard estimates? Bull Seismol Soc Am 98:509–520CrossRefGoogle Scholar
  12. Beauval C, Bard P-Y, Douglas J (2010) Comment on “Test of seismic hazard map from 500 years of recorded intensity data in Japan” by Masatoshi Miyazawa and Jim Mori. Bull Seismol Soc Am 100(6):3329–3331CrossRefGoogle Scholar
  13. Bender B, Perkins D (1987) SEISRISK III: A computer program for seismic hazard estimation. US Geol Surv Bull 1772:48Google Scholar
  14. Bindi D, Abdrakhmatov K, Parolai S, Mucciarelli M, Grunthal G, Ischuk A, Mikhailova N, Zschau J (2012) Seismic hazard assessment in Central Asia: outcomes from a site approach. Soil Dyn Earthq Eng 37:84–91CrossRefGoogle Scholar
  15. Bommer JJ (2002) Deterministic vs. probabilistic seismic hazard assessment: an exaggerated and obstructive dichotomy. J Earthq Eng 6:43–73Google Scholar
  16. Bommer JJ, Scherbaum F (2008) The use and misuse of logic trees in probabilistic seismic hazard analysis. Earthq Spectra 24(4):997–1009CrossRefGoogle Scholar
  17. Box GEP, Draper NR (1987) Empirical model-building and response surface. Wiley, New YorkGoogle Scholar
  18. Bozkurt SB, Stein RS, Toda S (2007) Forecasting probabilistic seismic shaking for greater Tokyo from 400 years of intensity observations. Earthq Spectra 23:525–546CrossRefGoogle Scholar
  19. Bradley BA, Stirling MW, McVerry GH, Gerstenberger M (2012) Consideration and propagation of epistemic uncertainties in New Zealand probabilistic seismic-hazard analysis. Bull Seismol Soc Am 102(4):1554–1568CrossRefGoogle Scholar
  20. Brier GW (1950) Verification of forecasts expressed in terms of probability. Mon Wea Rev 78(1):1–3CrossRefGoogle Scholar
  21. Brune JN (1996) Precariously balanced rocks and ground motion maps for Southern California. Bull Seismol Soc Am 86:43–54Google Scholar
  22. Budnitz RJ, Apostolakis G, Boore DM, Cluff LS, Coppersmith KJ, Cornell CA, Morris PA (1997) Recommendations for probabilistic seismic hazard analysis: guidance on uncertainty and use of experts. NUREG/CR6372. Nuclear Regulatory Commission, Washington, p 256CrossRefGoogle Scholar
  23. Burnham KP, Anderson DR (2002) The model selection and multimodel inference: a practical information-theoretic approach, 2nd edn. Springer, New York, p 488Google Scholar
  24. Coppersmith KJ, Youngs RR (1986) Capturing uncertainty in probabilistic seismic hazard assessment within intraplate environments. In: Proc 3rd nat conf on earthquake engineering, Vol 1, pp 301–312Google Scholar
  25. Cornell CA (1968) Engineering seismic risk analysis. Bull Seismol Soc Am 58:1583–1606Google Scholar
  26. D’Amico V, Albarello D (2003) The role of data processing and uncertainty management in seismic hazard evaluations: insights from estimates in the Garfagnana–Lunigiana Area (Northern Italy). Nat Hazards 29:77–95CrossRefGoogle Scholar
  27. D’Amico V, Albarello D (2008) SASHA: a computer program to assess seismic hazard from intensity data. Seismol Res Lett 79(5):663–671CrossRefGoogle Scholar
  28. De Finetti B (1974) Theory of probability. Wiley, New YorkGoogle Scholar
  29. Dowrick DJ, Cousins WJ (2003) Historical incidence of Modified Mercalli intensity in New Zealand and comparisons with hazard models. Bull New Zeal Soc Earthq Eng 36:1–24Google Scholar
  30. Edwards AWF (1972) Likelihood. Cambridge University Press, Cambridge, p 235Google Scholar
  31. Faccioli E, Cauzzi C (2006) Macroseismic intensities for seismic scenarios estimated from instrumentally based correlations. In: Proc. of the first European conference on earthquake engineering and seismology, p 569Google Scholar
  32. Faenza L, Michelini A (2010) Regression analysis of MCS intensity and ground motion parameters in Italy and its application in ShakeMap. Geophys J Int 180:1138–1152CrossRefGoogle Scholar
  33. Faenza L, Michelini A (2011) Regression analysis of MCS intensity and ground motion spectral accelerations (SAs) in Italy. Geophys J Int 186(3):1415–1430CrossRefGoogle Scholar
  34. Frankel A (1995) Mapping seismic hazard in the central and Eastern United States. Seismol Res Lett 66(4):8–21CrossRefGoogle Scholar
  35. Fujiwara H, Morikawa N, Ishikawa Y, Okumura T, Miyakoshi J, Nojima N, Fukushima Y (2009) Statistical Comparison of National Probabilistic Seismic Hazard Maps and Frequency of Recorded JMA Seismic Intensities from the K-NET Strong-motion Observation Network in Japan during 1997–2006. Seismol Res Lett 80(3):458–464CrossRefGoogle Scholar
  36. Galea P (2007) Seismic history of the Maltese islands and considerations on seismic risk. Ann Geophys 50(6):725–740Google Scholar
  37. Gelman A, Carlin JB, Stern HS, Rubin DB (1995) Bayesian data analysis. CRC Press, Boca Raton, p 526Google Scholar
  38. Gerstenberger MC, Stirling MW (2011) Ground-motion based tests of the New Zealand national seismic hazard model. In: Proc. of the ninth pacific conference on earthquake engineering building an earthquake-resilient society, Auckland, New Zealand 14–16 Apr, 2011Google Scholar
  39. Gnedenko BV (1976) The theory of probability. Mir Publisher, Moscow, p 392Google Scholar
  40. Gómez Capera AA, D’Amico V, Meletti C, Rovida A, Albarello D (2010) Seismic hazard assessment in terms of macroseismic intensity in Italy: a critical analysis from the comparison of different computational procedures. Bull Seismol Soc Am 100(4):1614–1631CrossRefGoogle Scholar
  41. Grandori G (1993) A methodology for the falsification of local seismic hazard analysis. Ann Geofis 36:191–197Google Scholar
  42. Grandori G, Guagenti E, Tagliani A (1998) A proposal for comparing the reliabilities of alternative seismic hazard models. J Seismol 2:27–35CrossRefGoogle Scholar
  43. Grandori G, Guagenti E, Petrini L (2004) About the statistical validation of probability generators. Boll Geofis Teor Appl 45(4):247–254Google Scholar
  44. Grandori G, Guagenti E, Petrini L (2006) Earthquake catalogues and modelling strategies. A new testing procedure for the comparison between competing models. J Seismol 10:259–269CrossRefGoogle Scholar
  45. Guidoboni E, Ferrari G (1995) Historical cities and earthquakes: florence during the last nine centuries and evaluation of seismic hazard. Ann Geofis 38(5–6):617–648Google Scholar
  46. Humbert N, Viallet E (2008) A method for comparison of recent PSHA on the French territory with experimental feedback. In: Proc. of the 14th world conference on earthquake engineering, Oct 12–17, Beijing, China, p 8Google Scholar
  47. Iervolino I (2013) Probabilities and fallacies: why hazard maps cannot be validated by individual earthquakes. Earthq Spectra 29(3):1125–1136CrossRefGoogle Scholar
  48. Johnstone DJ (2007) The value of a probability forecast from portfolio theory. Theor Decis 63:153–203CrossRefGoogle Scholar
  49. Kagan YY, Jackson DD (1994) Long-term probabilistic forecasting of earthquakes. J Geophys Res 99:13685–13700CrossRefGoogle Scholar
  50. Kass RE, Raftery AE (1995) Bayes factors. J Am Stat Assoc 90:773–795CrossRefGoogle Scholar
  51. Klügel J-U (2005) Problems in the application of the SSHAC probability method for assessing earthquake hazards at Swiss nuclear power plants. Eng Geol 78:285–307CrossRefGoogle Scholar
  52. Krinitzky EL (1993) Earthquake probability in engineering—part I: the use and misuse of expert opinion. Eng Geol 33:257–288CrossRefGoogle Scholar
  53. Krinitzky EL (1995) Deterministic versus probabilistic seismic hazard analysis for critical structures. Eng Geol 40:1–7CrossRefGoogle Scholar
  54. Kulkarni RB, Youngs RR, Coppersmith KJ (1984) Assessment of the confidence intervals for results of seismic hazard analysis. In: Proceedings 8th world conference of earthquake engineering. San Francisco 1:263–270Google Scholar
  55. Lind NC (1996) Validation of probabilistic models. Civ Eng Syst 13:175–183CrossRefGoogle Scholar
  56. Lipton P (2005) Testing hypotheses: prediction and prejudice. Science 307:219–221CrossRefGoogle Scholar
  57. Locati M, Camassi R, Stucchi M (2011) DBMI11, la versione 2011 del Database Macrosismico Italiano. Milano, Bologna, doi: 10.6092/INGV.IT-DBMI11
  58. Luzi L, Hailemikael S, Bindi D, Pacor F, Mele F, Sabetta F (2008) ITACA (ITalian ACcelerometric Archive): a web portal for the dissemination of italian strong-motion data. Seismol Res Lett 79(5):716–722. doi: 10.1785/gssrl.79.5.716 CrossRefGoogle Scholar
  59. Magri L, Mucciarelli M, Albarello D (1994) Estimates of site seismicity rates using ill-defined macroseismic data. Pure Appl Geophys 143:618–632CrossRefGoogle Scholar
  60. Mak S, Clements RA, Schorlemmer D (2014) The statistical power of testing probabilistic seismic-hazard assessments. Seismol Res Lett 85(4):781–783CrossRefGoogle Scholar
  61. Marzocchi W, Zechar JD, Jordan TH (2012) Bayesian forecast evaluation and ensemble earthquake forecasting. Bull Seismol Soc Am 102(6):2574–2584CrossRefGoogle Scholar
  62. McGuire RK (1978) FRISK: computer program for seismic risk analysis using faults as earthquake sources. USGS Open File Report 78-1007Google Scholar
  63. McGuire RK (1979) Adequacy of simple probability models for calculating felt shaking hazard using the Chinese earthquake catalog. Bull Seismol Soc Am 69:877–892Google Scholar
  64. McGuire RK, Barnhard TP (1981) Effects of temporal variations in seismicity on seismic hazard. Bull Seismol Soc Am 71:321–334Google Scholar
  65. Mezcua J, Rueda J, García Blanco RM (2013a) Iberian Peninsula historical seismicity revisited: an intensity data bank. Seismol Res Lett 84:9–18CrossRefGoogle Scholar
  66. Mezcua J, Rueda J, García Blanco RM (2013b) Observed and calculated intensities as a test of a probabilistic seismic-hazard analysis of Spain. Seismol Res Lett 84:772–780CrossRefGoogle Scholar
  67. Miyazawa M, Mori J (2009) Test of seismic hazard map from 500 years of recorded intensity data in Japan. Bull Seismol Soc Am 99:3140–3149CrossRefGoogle Scholar
  68. Miyazawa M, Mori J (2010) Reply to “Comment on ‘Test of seismic hazard map from 500 Years of recorded intensity data in Japan’ by Masatoshi Miyazawa and Jim Mori” by Céline Beauval, Pierre-Yves Bard, and John Douglas. Bull Seismol Soc Am 100(6):3332–3334CrossRefGoogle Scholar
  69. Mucciarelli M, Albarello D, Stucchi M (1996) Sensitivity of seismic hazard estimates to the use of historical site data. In: Shenk VP (ed) Earthquake hazard and risk. Adv in Nat and Tech Hazards, Kluwer, 141–152Google Scholar
  70. Mucciarelli M, Peruzza L, Caroli P (2000) Tuning of seismic hazard estimates by means of observed site intensities. J Earthq Eng 4:141–159Google Scholar
  71. Mucciarelli M, Albarello D, D’Amico V (2006) Comparison between the Italian seismic hazard map (PRSTN04) and alternative PSHA estimates. In: Proc. first european conference on earthquake engineering and seismology, p 595Google Scholar
  72. Mucciarelli M, Albarello D, D’Amico V (2008) Comparison of probabilistic seismic hazard estimates in Italy. Bull Seismol Soc Am 98(6):2652–2664CrossRefGoogle Scholar
  73. Musson RMW (2000) The use of Monte Carlo simulations for seismic hazard assessment in the UK. Ann Geofis 43:1–9Google Scholar
  74. Musson RMW (2004) Objective validation of seismic hazard source models. In: Proc. 13th conference on earthquake engineering, Vancouver, BC, Canada, paper number 2492Google Scholar
  75. Musson RMW (2012) PSHA validated by quasi observational means. Seismol Res Lett 83(1):130–134CrossRefGoogle Scholar
  76. Musson RMW, Toro GR, Coppersmith KJ, Bommer JJ, Deichmann N, Bungum H, Cotton F, Scherbaum F, Slejko D, Abrahamson NA (2005) Evaluating hazard results for Switzerland and how not to do it: a discussion of “Problems in the application of the SSHAC probability method for assessing earthquake hazards at Swiss nuclear power plants” by J-U Kluegel. Eng Geol 82:43–55CrossRefGoogle Scholar
  77. O’Hagan A, Oakley JE (2004) Probability is perfect, but we can’t elicit it perfectly. Reliab Eng Syst Saf 85:239–248CrossRefGoogle Scholar
  78. Ordaz M, Reyes C (1999) Earthquake hazard in Mexico City: observations versus computations. Bull Seismol Soc Am 89:1379–1383Google Scholar
  79. Oreskes N, Shrader-Frechette K, Belitz K (1994) Verification, validation, and confirmation of numerical models in the earth sciences. Science 263:641–646CrossRefGoogle Scholar
  80. Pace B, Peruzza L, Lavecchia G, Boncio P (2006) Layered seismogenic source model and probabilistic seismic-hazard analyses in central Italy. Bull Seismol Soc Am 96:107–132CrossRefGoogle Scholar
  81. Pace B, Albarello D, Boncio P, Dolce M, Galli P, Messina P, Peruzza L, Sabetta F, Sanò T, Visini F (2011) Predicted Ground Motion after the L’Aquila 2009 earthquake (Italy, Mw6.3): input spectra for Seismic Microzoning. Bull Earthq Eng 9:199–230CrossRefGoogle Scholar
  82. Pacor F, Paolucci R, Luzi L, Sabetta F, Spinelli A, Gorini A, Nicoletti M, Marcucci S, Filippi L, Dolce M (2011) Overview of the Italian strong motion database ITACA 1.0. Bull Earthq Eng 9(6):1723–1739. doi: 10.1007/s10518-011-9327-6 CrossRefGoogle Scholar
  83. Page MT, Carlson JM (2006) Methodologies for earthquake hazard assessment: model uncertainty and the WGCEP-2002 forecast. Bull Seismol Soc Am 96:1624–1633CrossRefGoogle Scholar
  84. Rabinowitz N, Steinberg DM (1991) Seismic hazard sensitivity analysis: a multi-parameter approach. Bull Seismol Soc Am 81:796–817Google Scholar
  85. Reiter L (1990) Earthquake hazard analysis. Issues and insights. Columbia University Press, New YorkGoogle Scholar
  86. Rhoades DA, McVerry GA (2001) Joint hazard of earthquake shaking at two or more locations. Earthq Spectra 17(4):697–710CrossRefGoogle Scholar
  87. Rhoades DA, Zhao JX, McVerry GH (2008) A simple test for inhibition of very strong shaking in ground-motion models. Bull Seismol Soc Am 98:448–453CrossRefGoogle Scholar
  88. Rhoades DA, Schorlemmer D, Gerstenberger MC, Christophersen A, Zechar J, Masajiro I (2011) Efficient testing of earthquake forecasting models. Acta Geophys 59(4):728–747CrossRefGoogle Scholar
  89. Scherbaum F, Bommer JJ, Bungum H, Cotton F, Abrahamson NA (2005) Composite ground-motion models and logic trees: methodology, sensitivities and uncertainties. Bull Seismol Soc Am 95:1575–1593CrossRefGoogle Scholar
  90. Schorlemmer D, Gerstenberger MC (2007) RELM testing center. Seismol Res Lett 78(1):30–36CrossRefGoogle Scholar
  91. Schorlemmer D, Gerstenberger MC, Wiemer S, Jackson DD, Rhoades DA (2007) Earthquake likelihood model testing. Seismol Res Lett 78(1):17–29CrossRefGoogle Scholar
  92. Selva J, Sandri L (2013) Probabilistic seismic hazard assessment: combining cornell-like approaches and data at sites through Bayesian inference. Bull Seismol Soc Am 103(3):1709–1722CrossRefGoogle Scholar
  93. Sanders F (1967) The verification of probability forecasts. J Appl Meteor 6:756–761CrossRefGoogle Scholar
  94. Stirling MW, Gerstenberger MC (2010) Ground motion-based testing of seismic hazard models in New Zealand. Bull Seismol Soc Am 100(4):1407–1414CrossRefGoogle Scholar
  95. Stirling MW, Petersen M (2006) Comparison of the historical record of earthquake hazard with seismic hazard models for New Zealand and the Continental United States. Bull Seismol Soc Am 96:1978–1994CrossRefGoogle Scholar
  96. Stucchi M, Camassi R, Rovida A, Locati M, Ercolani E, Meletti C, Migliavacca P, Bernardini F, Azzaro R (2007) DBMI04, il database delle osservazioni macrosismiche dei terremoti italiani utilizzate per la compilazione del catalogo parametrico CPTI04. Quad Geofis 49: 1–38: also Available at
  97. Usami T (2003) Materials for comprehensive list of destructive earthquakes in Japan, (416)-2001. Univ Tokyo Press, Tokyo (in Japanese)Google Scholar
  98. Viallet E, Humbert N, Martin C, Secanell N (2008) On the use of a Bayesian updating technique to get probabilistic seismic hazard assessment more rugged. In: Proc. of the 14th World conference on earthquake engineering, Oct 12–17, Beijing, China, p 10Google Scholar
  99. Ward SN (1995) Area-based tests of long-term seismic hazard predictions. Bull Seismol Soc Am 85(5):1285–1298Google Scholar
  100. Winkler RL (1996) Scoring rules and the evaluation of probabilities. Test 5:1–60CrossRefGoogle Scholar
  101. Woo G (1996) Kernel estimation methods for seismic hazard area source modelling. Bull Seismol Soc Am 86:353–362Google Scholar
  102. Zechar JD, Gerstenberger MC, Rhoades DA (2010) Likelihood-based tests for evaluating space–rate–magnitude earthquake forecasts. Bull Seismol Soc Am 100(3):1184–1195CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 2015

Authors and Affiliations

  1. 1.Dipartimento di Scienze Fisiche, della Terra e dell’AmbienteUniversità degli Studi di SienaSienaItaly
  2. 2.Istituto Nazionale di Geofisica e VulcanologiaPisaItaly

Personalised recommendations