# A Practical Approach to Testing Calibration Strategies

Article

First Online:

Accepted:

- 26 Downloads

## Abstract

A calibration strategy tries to match target moments using a model’s parameters. We propose tests for determining whether this is possible. The tests use moments at random parameter draws to assess whether the target moments are similar to the computed ones (evidence of existence) or appear to be outliers (evidence of non-existence). Our experiments show the tests are effective at detecting both existence and non-existence in a non-linear model. Multiple calibration strategies can be quickly tested using just one set of simulated data. Applying our approach to indirect inference allows for the testing of many auxiliary model specifications simultaneously. Code is provided.

### Keywords

Calibration GMM Indirect inference Existence Misspecification Outlier detection Data mining### JEL Classification

C13 C51 C52 C80 F34### References

- Angiulli, F., & Pizzuti, C. (2002).
*Fast outlier detection in high dimensional spaces*(pp. 15–27). Berlin: Springer.Google Scholar - Arellano, C. (2008). Default risk and income fluctuations in emerging economies.
*American Economic Review*,*98*(3), 690–712.CrossRefGoogle Scholar - Ben-Gal, I. (2005). Outlier detection. In O. Maimon & L. Rockach (Eds.),
*Data mining and knowledge discovery handbook*(pp. 131–146). Springer.Google Scholar - Billingsley, P. (1995).
*Probability and measure*(3rd ed.). Hoboken, NJ: Wiley.Google Scholar - Bollen, K. A., & Jackman, R. W. (1990).
*Modern methods of data analysis, chapter regression diagnostics: An expository treatment of outliers and influential cases*. Newbury Park, CA: Sage.Google Scholar - Breunig, M. M., Kriegel, H.-P., Ng, R. T., & Sander, J. (2000). LOF: Identifying density-based local outliers. In
*Proceedings of the ACM SIGMOD 2000 international conference on management of data*(pp. 93–104).Google Scholar - Canova, F., Ferroni, F., & Matthes, C. (2014). Choosing the variables to estimate singular DSGE models.
*Journal of Applied Econometrics*,*29*(7), 1099–1117.CrossRefGoogle Scholar - Canova, F., & Sala, L. (2009). Back to square one: Identification issues in DSGE models.
*Journal of Monetary Economics*,*56*(4), 431–449.CrossRefGoogle Scholar - Chatterjee, S., & Eyigungor, B. (2012). Maturity, indebtedness, and default risk.
*American Economic Review*,*102*(6), 2674–2699.CrossRefGoogle Scholar - Christiano, L. J., & Eichenbaum, M. (1992). Current real-business-cycle theories and aggregate labor-market fluctuations.
*American Economic Review*,*82*(3), 430–450.Google Scholar - Enas, G. G., & Choi, S. C. (1986). Choice of the smoothing parameter and efficiency of the k-nearest neighbor classification.
*Computers & Mathematics with Applications*,*12A*(2), 235–244.CrossRefGoogle Scholar - Feldman, T., & Sun, Y. (2011). Econometrics and computational economics: an exercise in compatibility.
*International Journal of Computational Economics and Econometrics*,*2*(2), 105–114.CrossRefGoogle Scholar - Gallant, A. R., & Tauchen, G. (1996). Which moments to match?
*Econometric Theory*,*12*(4), 657–681.CrossRefGoogle Scholar - Gnanadesikan, R. (1997).
*Methods for statistical data analysis of multivariate observations*. New York, NY: Wiley.CrossRefGoogle Scholar - Gordon, G., & Guerron-Quintana, P. (2017). Dynamics of investment, debt, and default.
*Review of Economic Dynamics*. https://doi.org/10.1016/j.red.2017.07.007. - Gordon, G., & Qiu, S. (2017). A divide and conquer algorithm for exploiting policy function monotonicity.
*Quantitative Economics*. http://qeconomics.org/ojs/forth/640/640-3.pdf. - Gourieroux, C., Monfort, A., & Renault, E. (1993). Indirect inference.
*Journal of Applied Econometrics*,*8*, S85–S118.CrossRefGoogle Scholar - Guerron-Quintana, P. A. (2010). What you match does matter: The effects of data on DSGE estimation.
*Journal of Applied Econometrics*,*25*(5), 774–804.CrossRefGoogle Scholar - Hansen, L. P. (1982). Large sample properties of generalized method of moments estimators.
*Econometrica*,*50*(4), 1029–1054.CrossRefGoogle Scholar - Hansen, L. P. (2008). Generalized method of moments estimation. In S. N. Durlauf & L. E. Blume (Eds.),
*The new Palgrave dictionary of economics*. Basingstoke: Palgrave Macmillan.Google Scholar - Hansen, L. P., & Heckman, J. J. (1996). The empirical foundations of calibration.
*Journal of Economic Perspectives*,*10*(1), 87–104.CrossRefGoogle Scholar - Kleibergen, F. (2005). Testing parameters in GMM without assuming that they are identified.
*Econometrica*,*73*(4), 1103–1123.CrossRefGoogle Scholar - Koopmans, T. C., & Reiersol, O. (1950). The identification of structural characteristics.
*The Annals of Mathematical Statistics*,*21*(2), 165–181.CrossRefGoogle Scholar - Kriegel, H.-P., Kröger, P., Schubert, E., & Zimek, A. (2009). LoOP: Local outlier probabilities. In
*Proceedings of the 18th ACM conference on information and knowledge management*, CIKM ’09 (pp. 1649–1652). New York, NY, USA: ACM. ISBN 978-1-60558-512-3.Google Scholar - Kydland, F. E., & Prescott, E. C. (1982). Time to build and aggregate fluctuations.
*Econometrica*,*50*(6), 1345–70.CrossRefGoogle Scholar - Loftsgaarden, D. O., & Quesenberry, C. P. (1965). A nonparametric estimate of a multivariate density function.
*Annals of Mathematical Statistics*,*36*, 1049–1051.CrossRefGoogle Scholar - Penny, K. I., & Jolliffe, I. T. (2001). A comparison of multivariate outlier detection methods for clinical laboratory safety data.
*The Statistician*,*50*(3), 295–308.Google Scholar - Ramaswamy, S., Rastogi, R., & Shim, K. (2000). Efficient algorithms for mining outliers from large data sets. In
*Proceedings of the 2000 ACM SIGMOD international conference on management of data*, SIGMOD ’00 (pp. 427–438). New York, NY, USA: ACM.Google Scholar - Sargan, J. D. (1958). The estimation of economic relationships using instrumental variables.
*Econometrica*,*26*, 393–415.CrossRefGoogle Scholar - Sargan, J. D. (1959). The estimation of relationships with autocorrelated residuals by the use of instrumental variables.
*Journal of the Royal Statistical Society: Series B (Methodological)*,*21*(1), 91–105.Google Scholar - Smith, A. A, Jr. (1993). Estimating nonlinear time-series models using simulated vector autoregressions.
*Journal of Applied Econometrics*,*8*, S63–S84.CrossRefGoogle Scholar - Stock, J. H., & Wright, J. H. (2000). GMM with weak identification.
*Econometrica*,*68*(5), 1055–1096.CrossRefGoogle Scholar - Stock, J. H., Wright, J. H., & Yogo, M. (2002). A survey of weak instruments and weak identification in generalized method of moments.
*Journal of Business Economics & Statistics*,*20*(4), 518–529.CrossRefGoogle Scholar - Zhang, K., Hutter, M., & Jin, H. (2009).
*A new local distance-based outlier detection approach for scattered real-world data*(pp. 813–822). Berlin: Springer.Google Scholar

## Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018