Subjective Beliefs and Statistical Forecasts of Financial Risks: The Chief Risk Officer Project



Information about financial risks comes from many sources. We formally consider how one can elicit and use information from two important sources when making forecasts. One source is a traditional statistical forecast, using familiar econometric methods for extrapolating from the past to the future. The other source is the elicited subjective belief distributions of “experts” in this domain: Chief Risk Officers of major international corporations. We demonstrate how these beliefs can be elicited in a formal, structured and incentivized manner, and critically contain information on the precision of the individual’s belief for each risk. We characterize the manner in which these two sources tell different stories about these risks, arguing that any distributional differences, or similarities, between the two sources are informative for risk managers. Finally, we characterize the degree of consistency among our experts: are they “on the same page” in their beliefs? We argue, again, that consistency or inconsistency of subjective beliefs is in itself informative for risk managers.1


Financial Risk Credit Default Swap Spot Price Concordance Index Subjective Belief 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Andersen S, Fountain J, Harrison GW, Rutström EE. (2014). Estimating subjective probabilities. Journal of Risk & Uncertainty forthcoming.Google Scholar
  2. Bernanke BS, Boivin J, Eliasz P. (2005). Measuring the effects of monetary policy: A factor-augmented vector autoregressive (FAVAR) approach. Quarterly Journal of Economics 120(1): 387–422.Google Scholar
  3. Bruine de Bruin W, Manski CF, Topa G, Van der Klaauw W. (2011). Measuring consumer uncertainty about future inflation. Journal of Applied Econometrics 26: 454–478.CrossRefGoogle Scholar
  4. Bühlmann H, Gisler A. (2005). A Course in Credibility Theory. Springer-Verlag: New York.Google Scholar
  5. Capistrán C, Timmerman A. (2009). Forecast combination with entry and exit of experts. Journal of Business & Economic Statistics 27(4): 428–440.CrossRefGoogle Scholar
  6. Cooke RM. (1991). Experts in Uncertainty: Opinion and Subjective Probability in Science. Oxford University Press: New York.Google Scholar
  7. Delavande A, Giné X, McKenzie D. (2011). Measuring subjective expectations in developing countries: A critical review and new evidence. Journal of Development Economics 94: 151–163.CrossRefGoogle Scholar
  8. Dominitz J, Manski CF. (2011). Measuring and interpreting expectations of equity returns. Journal of Applied Econometrics 26: 352–370.CrossRefGoogle Scholar
  9. Dovern J, Fritsche U, Slacalek J. (2012). Disagreement among forecasters in G7 countries. Review of Economics and Statistics 94(4): 1081–1096.CrossRefGoogle Scholar
  10. Engelberg J, Manski CF, Williams J. (2009). Comparing the point predictions and subjective probability distributions of professional forecasters. Journal of Business & Economic Statistics 27(1): 30–41.CrossRefGoogle Scholar
  11. Engelberg J, Manski CF, Williams J. (2011). Assessing the temporal variation of macroeconomic forecasts by a panel of changing composition. Journal of Applied Econometrics 26: 1059–1078.CrossRefGoogle Scholar
  12. Fehr E, List JA. (2004). The hidden cost and returns of incentives — Trust and trustworthiness among CEOs. Journal of the European Economic Association 2(5): 743–771.CrossRefGoogle Scholar
  13. Galati G, Poelhekke S, Zhou C (2011). Does the crisis affect inflation expectations? International Journal of Central Banking 7: 167–207.Google Scholar
  14. Garthwaite PH, Kadane JB, O’Hagan A. (2005). Statistical methods for eliciting probability distributions. Journal of the American Statistical Association 100(470): 680–700.CrossRefGoogle Scholar
  15. Graham JR, Harvey CR. (2005). The long-run equity risk premium. Finance Research Letters 2: 185–194.CrossRefGoogle Scholar
  16. Hamilton JD. (1994). Time Series Analysis. Princeton University Press: Princeton, NJ.Google Scholar
  17. Harrison G, Lau MI, Rutström EE. (2007). Estimating risk attitudes in Denmark: A field experiment. Scandinavian Journal of Economics 109(2): 341–368.CrossRefGoogle Scholar
  18. Harrison GW (2014). Hypothetical surveys or incentivized scoring rules for eliciting subjective belief distributions? Working Paper 2014–05, Center for the Economic Analysis of Risk, Robinson College of Business, Georgia State University.Google Scholar
  19. Harrison GW, Martínez-Correa J, Swarthout JT. (2014). Eliciting subjective probabilities with binary lotteries. Journal of Economic Behavior & Organization 101: 128–140.CrossRefGoogle Scholar
  20. Harrison GW, Martínez-Correa J, Swarthout JT, Ulm E. (2012). Scoring rules for subjective probability distributions. Working Paper 2012–10, Center for the Economic Analysis of Risk, Robinson College of Business, Georgia State University.Google Scholar
  21. Harrison GW, Phillips R. (2013). Subjective beliefs and statistical forecasts of financial risks: The Chief Risk Officer project. Working Paper 2013–08, Center for the Economic Analysis of Risk, Robinson College of Business, Georgia State University.Google Scholar
  22. Harrison GW, Rutström EE. (2008a). Risk aversion in the laboratory, in Cox JC, Harrison GW. (eds), Risk Aversion in Experiments. Emerald, Research in Experimental Economics, vol. 12. Bingley, UK.Google Scholar
  23. Harrison GW, Rutström EE. (2008b). Experimental evidence on the existence of hypothetical bias in value elicitation methods, in Plott CR, Smith VL. (eds), Handbook of Experimental Economics Results. North-Holland, Amsterdam.Google Scholar
  24. Kadane JB, Winkler RL. (1988). Separating probability elicitation from utilities. Journal of the American Statistical Association 83(402): 357–363.CrossRefGoogle Scholar
  25. Karni E. (2009). A mechanism for eliciting probabilities. Econometrica 77(2): 603–606.CrossRefGoogle Scholar
  26. Karni E, Safra Z. (1995). The impossibility of experimental elicitation of subjective probabilities. Theory and Decision 38: 313–320.CrossRefGoogle Scholar
  27. Kauko K, Palmroos P. (2014). The Delphi method in forecasting financial markets — An experimental study. International Journal of Forecasting 30: 313–327.CrossRefGoogle Scholar
  28. Kim JH. (1999). Asymptotic and bootstrap prediction regions for vector autoregression. International Journal of Forecasting 15(4): 393–403.CrossRefGoogle Scholar
  29. Köszegi B, Rabin M. (2008). Revealed mistakes and revealed preferences. In Caplin A, Schotter A. (eds), The Foundations of Positive and Normative Economics: A Handbook. Oxford University Press: New York.Google Scholar
  30. Lahiri K, Sheng X. (2010). Measuring forecast uncertainty by disagreement: The missing link. Journal of Applied Econometrics 25: 514–538.CrossRefGoogle Scholar
  31. Lichtendahl Jr. KC, Winkler RL. (2007). Probability elicitation, scoring rules, and competition among forecasters. Management Science 53(11): 1745–1755.CrossRefGoogle Scholar
  32. Lin LI. (1989). A concordance correlation coefficient to evaluate reproducibility. Biometrics 45: 255–268.CrossRefGoogle Scholar
  33. Lin LI. (2000). A note on the concordance correlation coefficient. Biometrics 56: 324–325.CrossRefGoogle Scholar
  34. Mankiw NG, Reis R, Wolfers J. (2003). Disagreement on inflation expectations, in Gertler M, Rogoff K. (eds), NBER Macroeconomics Annual 2002. MIT Press: Cambridge, MA.Google Scholar
  35. Manski CF. (2004). Measuring expectations. Econometrica 72(5): 1329–1376.CrossRefGoogle Scholar
  36. Matheson JE, Winkler RL. (1976). Scoring rules for continuous probability distributions. Management Science 22(10): 1087–1096.CrossRefGoogle Scholar
  37. Mitchell J, Wallis KF. (2011). Evaluating density forecasts: Forecast combinations, model mixtures, calibration and sharpness. Journal of Applied Econometrics 26: 1023–1040.CrossRefGoogle Scholar
  38. Nickerson CAE. (1997). A note on “A concordance correlation coefficient to evaluate reproducibility.” Biometrics 53: 1503–1507.CrossRefGoogle Scholar
  39. O’Hagan A, Buck CE, Daneshkhah A, Eiser JR, Garthwaite PH, Jenkinson DJ, Oakley JE, Rakow T. (2006). Uncertain Judgements: Eliciting Experts’ Probabilities. Wiley: Chichester, UK.CrossRefGoogle Scholar
  40. Perozek M. (2008). Using subjective expectations to forecast longevity: Do survey respondents know something we don’t know? Demography 45(1): 95–113.CrossRefGoogle Scholar
  41. Savage LJ. (1971). Elicitation of personal probabilities and expectations. Journal of American Statistical Association 66: 783–801.CrossRefGoogle Scholar
  42. Savage LJ. (1972). The Foundations of Statistics, 2nd ed. Dover Publications: New York .Google Scholar
  43. Shafer G. (1986). The combination of evidence. International Journal of Intelligent Systems 1: 155–179.CrossRefGoogle Scholar
  44. Sims CA. (1980). Macroeconomics and reality. Econometrica 48: 1–48.CrossRefGoogle Scholar
  45. Smith CAB. (1961). Consistency in statistical inference and decision. Journal of the Royal Statistical Society 23: 1–25.Google Scholar
  46. Thombs LA, Schucany WR. (1990). Bootstrap prediction intervals for autoregression. Journal of the American Statistical Association 85: 486–492.CrossRefGoogle Scholar
  47. Vissing-Jorgensen A. (2004). Perspectives on behavioral finance: Does ‘irrationality’ disappear with wealth? Evidence from expectations and actions, in Gertler M, Rogoff K. (eds), NBER Macroeconomics Annual 2003. MIT Press: Boston, MA.Google Scholar
  48. Winkler RL, Murphy AH. (1970). Nonlinear utility and the probability score. Journal of Applied Meteorology 9: 143–148.CrossRefGoogle Scholar
  49. Zarnowitz V, Lambros LA. (1987). Consensus and uncertainty in economic prediction. Journal of Political Economy 95(3): 591–621.CrossRefGoogle Scholar

Copyright information

© Glenn W. Harrison and Richard D. Phillips 2014

Authors and Affiliations

There are no affiliations available

Personalised recommendations