Abstract
Each day a weather forecaster predicts a probability of each type of weather for the next day. After n days, all the predicted probabilities and the real weather data are sent to a test which decides whether to accept the forecaster as possessing predicting power. Consider tests such that forecasters who know the distribution of nature are passed with high probability. Sandroni shows that any such test can be passed by a forecaster who has no prior knowledge of nature [San03], provided that the duration n is known to the forecaster in advance. On the other hand, Fortnow and Vohra [FV06] show that Sandroni’s result may require forecasters with high computational complexity and is thus of little practical relevance in some cases. We consider forecasters who select a deterministic Turing-machine forecaster according to an arbitrary distribution and then use that machine for all future forecasts. We show that forecasters even more powerful than the above ones are required for Sandroni’s result. Also, we show that Sandroni’s result does not apply when the duration n is not known to the forecaster in advance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Al-Najjar, N.I., Weinstein, J.: Comparative testing of experts, Levine’s Working Paper Archive 321307000000000590, Department of Economics, UCLA (2006)
Adams-Watters, F.T.: http://www.research.att.com/~njas/sequences/a002260.txt
Dawid, A.P.: The well calibrated Bayesian. Journal of the American Statistical Association 77(379), 605–613 (1982)
Levine, D., Fudenberg, D.: Conditional universal consistency. Games and Economic Behavior 29, 104–130 (1999)
Dekel, E., Feinberg, Y.: Non-Bayesian testing of a stochastic prediction. Review of Economic Studies 73(4), 893–906 (2006)
Fan, K.: Minimax theorems. Proceedings of the National Academy of Science USA. 39, 42–47 (1953)
Feinberg, Y., Stewart, C.: Testing multiple forecasters. Research Paper Series 1957, Graduate School of Business, Stanford University (2007)
Foster, D.P., Vohra, R.V.: Asymptotic calibration. Biometrika 85(2), 379–390 (1998)
Fortnow, L., Vohra, R.V.: The complexity of forecast testing, Tech. Report TR06-149, Electronic Colloquium on Computational Complexity (2006)
Lehrer, E.: Any inspection rule is manipulable. Econometrica 69(5), 1333–1347 (2001)
Olszewski, W., Sandroni, A.: Counterfactual predictions, Tech. report, Northwestern University, Department of Economics (2006)
Olszewski, W., Sandroni, A.: Strategic manipulation of empirical tests, Tech. report, Northwestern University, Department of Economics (2006)
Sandroni, A.: The reproducible properties of correct forecasts. International Journal of Game Theory 32(1), 151–159 (2003)
Sandroni, A., Smorodinsky, R., Vohra, R.V.: Calibration with many checking rules. Mathematics of Operations Research 28(1), 141–153 (2003)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Chang, CL., Lyuu, YD. (2007). Efficient Testing of Forecasts. In: Lin, G. (eds) Computing and Combinatorics. COCOON 2007. Lecture Notes in Computer Science, vol 4598. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-73545-8_29
Download citation
DOI: https://doi.org/10.1007/978-3-540-73545-8_29
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-73544-1
Online ISBN: 978-3-540-73545-8
eBook Packages: Computer ScienceComputer Science (R0)