Skip to main content
SpringerLink
Log in
Book cover

Modeling and Stochastic Learning for Forecasting in High Dimensions pp 55–78Cite as

Constructing Graphical Models via the Focused Information Criterion

  • Conference paper

Part of the book series: Lecture Notes in Statistics ((LNSP,volume 217))

Abstract

A focused information criterion is developed to estimate undirected graphical models where for each node in the graph a generalized linear model is put forward conditioned upon the other nodes in the graph. The proposed method selects a graph with a small estimated mean squared error for a user-specified focus, which is a function of the parameters in the generalized linear models, by selecting an appropriate model at each node. For situations where the number of nodes is large in comparison with the number of cases, the procedure performs penalized estimation with quadratic approximations to several popular penalties. To show the procedure’s applicability and usefulness we have applied it to two datasets involving voting behavior of U.S. senators and to a clinical dataset on psychopathology.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Banerjee, O., El Ghaoui, L., & d’Aspremont, A. (2008). Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data. Journal of Machine Learning Research, 9, 485–516.

    MATH  Google Scholar 

  2. Borsboom, D., Cramer, A. O. J., Schmittmann, V. D., Epskamp, S., & Waldorp, L. J. (2011). The small world of psychopathology. PLoS ONE, 6(11), e27407.

    Article  Google Scholar 

  3. Claeskens, G., & Hjort, N. (2003). The focused information criterion. Journal of the American Statistical Association, 98, 900–916. With discussion and a rejoinder by the authors.

    Google Scholar 

  4. Claeskens, G., & Hjort, N. (2008). Minimising average risk in regression models. Econometric Theory, 24, 493–527.

    Article  MATH  MathSciNet  Google Scholar 

  5. Claeskens, G., & Hjort, N. (2008). Model selection and model averaging. Cambridge: Cambridge University Press.

    Book  MATH  Google Scholar 

  6. Cox, D. R., & Wermuth, N. (1996). Multivariate dependencies: Models, analysis and interpretation. London: Chapman & Hall.

    MATH  Google Scholar 

  7. Dempster, A. (1972). Covariance selection. Biometrics, 28(1), 157–175.

    Article  Google Scholar 

  8. Fan, J., & Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 96(456), 1348–1360.

    Article  MATH  MathSciNet  Google Scholar 

  9. Fiocco, M., & van Zwet, W. (2004). Maximum likelihood estimation for the contact process. Institute of Mathematical Statistics Lecture Notes - Monograph Series, 45, 309–318.

    Google Scholar 

  10. Frank, I. E., & Friedman, J. H. (1993). A statistical view of some chemometrics regression tools. Technometrics, 35(2), 109–135.

    Article  MATH  Google Scholar 

  11. Jalali, A., Ravikumar, P., Vasuki, V., & Sanghavi, S. (2010). On learning discrete graphical models using group-sparse regularization. In Proceedings of the 13th international conference on artificial intelligence and statistics, Chia Laguna Resort, Sardinia, Italy.

    Google Scholar 

  12. Lauritzen, S. (1996). Graphical models. New York: Oxford University Press.

    Google Scholar 

  13. Lee, J., & Hastie, T. (2012). Learning mixed graphical models. Preprint arXiv:1205.5012v3.

    Google Scholar 

  14. Loh, P. L., & Wainwright, M. J. (2013). Structure estimation for discrete graphical models: Generalized covariance matrices and their inverses. The Annals of Statistics, 41(6), 3022–3049.

    Google Scholar 

  15. McCullagh, P., & Nelder, J. (1989). Generalized linear models. London: Chapman & Hall.

    Book  MATH  Google Scholar 

  16. Meinshausen, N., & Bühlmann, P. (2006). High-dimensional graphs and variable selection with the lasso. The Annals of Statistics, 34(3), 1436–1462.

    Article  MATH  MathSciNet  Google Scholar 

  17. Pircalabelu, E., Claeskens, G., Jahfari, S., & Waldorp, L. (2013). Focused information criterion for graphical models. Large p, small n considerations (Technical report). KBI, Faculty of Economics and Business, KU Leuven.

    Google Scholar 

  18. Pircalabelu, E., Claeskens, G., & Waldorp, L. (2012). Structure learning using a focused information criterion in graphical models (Technical report). KBI, Faculty of Economics and Business, KU Leuven.

    Google Scholar 

  19. Schmidt, M., Niculescu-Mizil, A., & Murphy, K. (2007). Learning graphical model structure using ℓ 1-regularization paths. In Proceedings of the 22nd national conference on artificial intelligence, Vancouver, Canada (Vol. 2, pp. 1278–1283).

    Google Scholar 

  20. Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society (SERIES B), 58, 267–288.

    MATH  MathSciNet  Google Scholar 

  21. van Borkulo, C. D., Kamphuis, J. H., & Waldorp, L. J. (2013). Predicting behaviour of networks of mental disorders: The contact process as a model for dynamics of psychopathology (Technical report). University of Amsterdam.

    Google Scholar 

  22. van der Aart, P. J. M., & Smeenk-Enserink, N. (1975). Correlations between distributions of hunting spiders (Lycosidae, Ctenidae) and environmental characteristics in a dune area. Netherlands Journal of Zoology, 25, 1–45.

    Article  Google Scholar 

  23. Wainwright, M. J., & Jordan, M. I. (2008). Graphical models, exponential families, and variational inference. Foundations and Trends in Machine Learning, 1(1–2), 1–305.

    MATH  Google Scholar 

  24. Wainwright, M. J., Ravikumar, P., & Lafferty, J. D. (2007). High-dimensional graphical model selection using ℓ 1-regularized logistic regression. In B. Schölkopf, J. Platt, & T. Hoffman (Eds.), Advances in neural information processing systems, 20 (NIPS 2007), (Vol. 19, pp. 1465–1472). Cambridge: MIT Press.

    Google Scholar 

  25. Watson, D., Clark, L. A., & Tellegen, A. (1988). Development and validation of brief measures of positive and negative affect: The panas scales. Journal of Personality and Social Psychology, 54(6), 1063–1070.

    Article  Google Scholar 

  26. Yang, E., Ravikumar, P. K., Allen, G. I., & Liu, Z. (2012). Graphical models via generalized linear models. In P. Bartlett, F. Pereira, C. Burges, L. Bottou, & K. Weinberger (Eds.), Advances in neural information processing systems, Curran Associates, Inc. (Vol. 25, pp. 1367–1375).

    Google Scholar 

  27. Zou, H. (2006). The adaptive lasso and its oracle properties. Journal of the American Statistical Association, 101(476), 1418–1429.

    Article  MATH  MathSciNet  Google Scholar 

Download references

Acknowledgements

The authors wish to thank Prof. J.-H. Kamphuis for the PANAS data. The authors acknowledge the support of the Fund for Scientific Research Flanders, KU Leuven grant GOA/12/14 and of the IAP Research Network P7/06 of the Belgian Science Policy.

Author information

Authors and Affiliations

  1. ORSTAT and Leuven Statistics Research Center, KU Leuven, Naamsestraat 69, 3000, Leuven, Belgium

    Gerda Claeskens & Eugen Pircalabelu

  2. Department of Psychological Methods, University of Amsterdam, Weesperplein 4, 1018, Amsterdam, The Netherlands

    Lourens Waldorp

Authors
  1. Gerda Claeskens
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Eugen Pircalabelu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lourens Waldorp
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Gerda Claeskens .

Editor information

Editors and Affiliations

  1. Department of Statistics, University Joseph Fourier, Grenoble, France

    Anestis Antoniadis

  2. Laboratoire de Mathématiques, Université Paris-Sud, Orsay Cedex, France

    Jean-Michel Poggi

  3. Electricité de France R & D, OSIRIS, Clamart Cedex, France

    Xavier Brossat

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Claeskens, G., Pircalabelu, E., Waldorp, L. (2015). Constructing Graphical Models via the Focused Information Criterion. In: Antoniadis, A., Poggi, JM., Brossat, X. (eds) Modeling and Stochastic Learning for Forecasting in High Dimensions. Lecture Notes in Statistics(), vol 217. Springer, Cham. https://doi.org/10.1007/978-3-319-18732-7_4

Download citation

Publish with us

Policies and ethics

Search

Navigation