Statistics in Expert Resolution: A Theory of Weights for Combining Expert Opinion

  • Roger M. Cooke
Part of the Boston Studies in the Philosophy of Science book series (BSPS, volume 122)


In many fields, including risk analysis, reliability, policy analysis, forecasting etc., expert opinion in the form of quantitative subjective probability assessments is being increasingly used when objective statistical data is lacking. This raises new problems for statistitians and probabilists, as models for extracting and analysing data from this new source must be developed. It is not surprising that most models to date have proceeded from the Bayesian standpoint, and intend to support the analyst in ‘updating’ his prior opinion with the probabilistic assessments from one or more experts. Many examples from this school can be found in (Clarotti and Lindley, 1988). Although there are countless ad hoc applications of expert opinion, applications of models for using expert opinion are harder to find. Good examples can be found in Apostolakis (1988).


Average Probability Scoring Rule Sample Entropy Response Entropy Uncertain Quantity 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Alpert, M. and Raiffa, H. 1982. ‘A Progress Report on the Training of Probability Assessor’. In D. Kahneman, P. Slovic, and A. Tversky (eds.), Judgment under Uncertainty: Heuristics and Biases, Cambridge University Press, pp. 294–306.Google Scholar
  2. Apostolaris, G. 1988. ‘Expert Judgment in Probabilistic Safety Assessment’. In C. Clarotti and D. Lindley (eds.), Proceedings International School of Physics “Enrico Fermi”, Amsterdam.Google Scholar
  3. Bohla, B., Blauw, H., Cooke, R., and Kok, M. 1988. ‘Expert Resolution in Project Management’. Paper presented at the Conference on Decision Making under Risk, Budapest, June 5–10.Google Scholar
  4. Clarotti, C. and Lindley, D. (eds.). 1988. Accelerated Life Testing and Experts’ Opinions in Reliability, Proceedings of the International School of Physics “Enrico Fermi”. Amsterdam.zbMATHGoogle Scholar
  5. Cooke, R. 1985. ‘Expert Resolution’. Proc. of the IF AC Conference. Varsese Italy, September 1985, Permagon Press.Google Scholar
  6. Cooke, R., Mendel, M., and Thys, W. 1988. ‘Calibration and Information in Expert Resolution: A Classical Approach’. Automatica 24 87–94.CrossRefGoogle Scholar
  7. Cooke, R. 1988. ‘Uncertainty in Risk Assessment’. Reliability Engineering and System Safety 23 277–285.CrossRefGoogle Scholar
  8. Cooke, R. 1989. ‘A Theory of Weights for Combining Expert Opinion’. Report to the Dutch Ministery of Environment. Expert Opinion in Safety Studies, vol. iv, Delft.Google Scholar
  9. Cooke, R., Mendel, M., and Van Steen, J. 1989. ‘Model Description Report, Report to the Dutch Ministery of Environment. Expert Opinion in Safety Studies, vol. iii. Delft.Google Scholar
  10. Cooke, R. 1990. Experts in Uncertainty. Appearing Oxford University Press.Google Scholar
  11. De Finetti, B. 1937. ‘La prevision: ses lois logiques, ses sources subjectives’. Annates de l’Tnstut Henri Poincare 7 1–68. English translation in Kyburg and Smolker (eds.), Studies in Subjective Probability. Wiley.zbMATHGoogle Scholar
  12. De Groot, M. 1974. ‘Reaching Consensus’, J. Amer. Statis. Assoc., vol. 69, 118–121.CrossRefGoogle Scholar
  13. De Groot, M. and Fienberg, S. 1986. ‘Comparing Probability Forecasters: Basic Binary Concepts and Multivariate Extensions’. In P. Goel and A. Zellner (eds.), Bayesian Inference and Decision Techniques, Elsevier.Google Scholar
  14. De Groot M. and Fienberg, S. 1983. ‘The Comparison and Evaluation of Forecasters’. The Statistician 32 12–22.CrossRefGoogle Scholar
  15. De Groot, M. and Bayarri, M. 1987. ‘Gaining Weight: A Bayesian Approach’. Dept. of Stat. Carnegie Mellon University Tech. Report 388, Jan. 1987.Google Scholar
  16. Feller, W. 1971. An Introduction to Probability Theory II, New York: Wiley.zbMATHGoogle Scholar
  17. French, S. 1985. ‘Group Consensus Probability Distributions’, In J. Bernado, M. De Groot, D. Lindley, and A. Smith (eds.), Bayesian Statistics, Elsevier, pp. 188–201.Google Scholar
  18. Friedman, D. 1983. ‘Effective Scoring Rules for Probabilistic Forecasts’. Management Science 29 447–454.zbMATHCrossRefGoogle Scholar
  19. Genest, C. and McConway, K. 1989. ‘Allocating the Weights in the Linear Opinion Pool’. Quebec: Université Laval.Google Scholar
  20. Genest, C. and Zidek, J. 1986. ‘Combining Probability Distributions: A Critique and Annotated Bibliography1. Statistical Science 1 114–148.MathSciNetCrossRefGoogle Scholar
  21. Honglin, W. and Duo, D. 1985. ‘Reliability Calculation of Prestressing Cable System of the PCPV Model’. Trans, of the 8th Intern. Conf. on Structural Mech. in Reactor Techn. Brussels, Aug. 19–23, pp. 41–44.Google Scholar
  22. Hoel, P. 1971. Introduction to Mathematical Statistics. New York: Wiley.Google Scholar
  23. Lichtenstein, S., Fischoff, B., and Phillips, D. 1982. ‘Calibration of Probabilities: The State of the Art of 1980’. In D. Kahneman, P. Slovic, and A. Tversky (eds.), Judgment under Uncertainty: Heuristics and Biases, Cambridge University Press, pp. 306–335.Google Scholar
  24. Matheson, J. and Winkler, R. 1976. ‘Scoring Rules for Continuous Probability Distributions’. Management Science 22 1087–1096.zbMATHCrossRefGoogle Scholar
  25. McCarthy J. 1956. ‘Measures of the Value of Information’. Proc. of the National Academy of Sciences, pp. 654–655.Google Scholar
  26. Morris, P. 1977. ‘Combining Expert Judgments: A Bayesian Approach’, Management Science, vol. 23 nr. 7, 679–693.zbMATHCrossRefGoogle Scholar
  27. Murphy, A. 1973. ‘A New Vector Partition of the Probability Score’. J. of Applied Meteorology 12 595–600.CrossRefGoogle Scholar
  28. Preyssl, C. and Cooke, R. 1989. ‘Expert Judgment; Subjective and Objective Data for Risk Assessment of Spaceflight Systems’, Proceedings PSA ‘89 International Topical Meeting Probability, Reliability and Safety Assessment, Pittsburgh, April 2–7.Google Scholar
  29. Roberts, H. 1965. ‘Probabilistic Prediction’. J. Amer. Statist. Assoc. 60 50–62.MathSciNetzbMATHCrossRefGoogle Scholar
  30. Savage, L. 1971. ‘Elicitation of Personal Probabilities and Expectations’. J. Amer. Statis. Assoc. 66 783–801.MathSciNetzbMATHCrossRefGoogle Scholar
  31. Shuford, E., Albert, A., and Massengil, H. 1966. ‘Admissible Probability Measurement Procedures’. Psychometrika 31 125–145.zbMATHCrossRefGoogle Scholar
  32. Stael von Holstein, C. 1970. ‘Measurement of Subjective Probability’. Acta Psychological 34 146–159.CrossRefGoogle Scholar
  33. Tucker, H. 1967. A Graduate Course in Probability. New York: Wiley.Google Scholar
  34. Wagner, C. and Lehrer, K. 1981. Rational Consensus in Science and Society, Dordrecht: Kluwer Acad. Publ.zbMATHGoogle Scholar
  35. Winkler, R. and Murphy, A. 1968. ‘Good Probability Assessors’. J. of Applied Meteorology 7 751–758.CrossRefGoogle Scholar
  36. Winkler, R. 1968. ‘The Consensus of Subjective Probability Distributions’. Management Science 15 B61-B75.CrossRefGoogle Scholar
  37. Winkler, R. 1969. ‘Scoring Rules and the Evaluation of Probability Assessors’. J. Amer. Statist. Assoc. 64 1073–1078.CrossRefGoogle Scholar
  38. Winkler, R. 1986. ‘On Good Probability Appraisers’. In P. Goel and A. Zellner (eds.), Bayesian Inference and Decision Techniques, Elsevier.Google Scholar

Copyright information

© Kluwer Academic Publishers 1990

Authors and Affiliations

  • Roger M. Cooke

There are no affiliations available

Personalised recommendations