Constructing Graphical Models via the Focused Information Criterion

  • Gerda ClaeskensEmail author
  • Eugen Pircalabelu
  • Lourens Waldorp
Conference paper
Part of the Lecture Notes in Statistics book series (LNS, volume 217)


A focused information criterion is developed to estimate undirected graphical models where for each node in the graph a generalized linear model is put forward conditioned upon the other nodes in the graph. The proposed method selects a graph with a small estimated mean squared error for a user-specified focus, which is a function of the parameters in the generalized linear models, by selecting an appropriate model at each node. For situations where the number of nodes is large in comparison with the number of cases, the procedure performs penalized estimation with quadratic approximations to several popular penalties. To show the procedure’s applicability and usefulness we have applied it to two datasets involving voting behavior of U.S. senators and to a clinical dataset on psychopathology.


Mean Square Error Exponential Family Vote Behavior Basic Reproduction Number Infected Node 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.



The authors wish to thank Prof. J.-H. Kamphuis for the PANAS data. The authors acknowledge the support of the Fund for Scientific Research Flanders, KU Leuven grant GOA/12/14 and of the IAP Research Network P7/06 of the Belgian Science Policy.


  1. 1.
    Banerjee, O., El Ghaoui, L., & d’Aspremont, A. (2008). Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data. Journal of Machine Learning Research, 9, 485–516.zbMATHGoogle Scholar
  2. 2.
    Borsboom, D., Cramer, A. O. J., Schmittmann, V. D., Epskamp, S., & Waldorp, L. J. (2011). The small world of psychopathology. PLoS ONE, 6(11), e27407.CrossRefGoogle Scholar
  3. 3.
    Claeskens, G., & Hjort, N. (2003). The focused information criterion. Journal of the American Statistical Association, 98, 900–916. With discussion and a rejoinder by the authors.Google Scholar
  4. 4.
    Claeskens, G., & Hjort, N. (2008). Minimising average risk in regression models. Econometric Theory, 24, 493–527.zbMATHMathSciNetCrossRefGoogle Scholar
  5. 5.
    Claeskens, G., & Hjort, N. (2008). Model selection and model averaging. Cambridge: Cambridge University Press.zbMATHCrossRefGoogle Scholar
  6. 6.
    Cox, D. R., & Wermuth, N. (1996). Multivariate dependencies: Models, analysis and interpretation. London: Chapman & Hall.zbMATHGoogle Scholar
  7. 7.
    Dempster, A. (1972). Covariance selection. Biometrics, 28(1), 157–175.CrossRefGoogle Scholar
  8. 8.
    Fan, J., & Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 96(456), 1348–1360.zbMATHMathSciNetCrossRefGoogle Scholar
  9. 9.
    Fiocco, M., & van Zwet, W. (2004). Maximum likelihood estimation for the contact process. Institute of Mathematical Statistics Lecture Notes - Monograph Series, 45, 309–318.Google Scholar
  10. 10.
    Frank, I. E., & Friedman, J. H. (1993). A statistical view of some chemometrics regression tools. Technometrics, 35(2), 109–135.zbMATHCrossRefGoogle Scholar
  11. 11.
    Jalali, A., Ravikumar, P., Vasuki, V., & Sanghavi, S. (2010). On learning discrete graphical models using group-sparse regularization. In Proceedings of the 13th international conference on artificial intelligence and statistics, Chia Laguna Resort, Sardinia, Italy.Google Scholar
  12. 12.
    Lauritzen, S. (1996). Graphical models. New York: Oxford University Press.Google Scholar
  13. 13.
    Lee, J., & Hastie, T. (2012). Learning mixed graphical models. Preprint arXiv:1205.5012v3.Google Scholar
  14. 14.
    Loh, P. L., & Wainwright, M. J. (2013). Structure estimation for discrete graphical models: Generalized covariance matrices and their inverses. The Annals of Statistics, 41(6), 3022–3049.Google Scholar
  15. 15.
    McCullagh, P., & Nelder, J. (1989). Generalized linear models. London: Chapman & Hall.zbMATHCrossRefGoogle Scholar
  16. 16.
    Meinshausen, N., & Bühlmann, P. (2006). High-dimensional graphs and variable selection with the lasso. The Annals of Statistics, 34(3), 1436–1462.zbMATHMathSciNetCrossRefGoogle Scholar
  17. 17.
    Pircalabelu, E., Claeskens, G., Jahfari, S., & Waldorp, L. (2013). Focused information criterion for graphical models. Large p, small n considerations (Technical report). KBI, Faculty of Economics and Business, KU Leuven.Google Scholar
  18. 18.
    Pircalabelu, E., Claeskens, G., & Waldorp, L. (2012). Structure learning using a focused information criterion in graphical models (Technical report). KBI, Faculty of Economics and Business, KU Leuven.Google Scholar
  19. 19.
    Schmidt, M., Niculescu-Mizil, A., & Murphy, K. (2007). Learning graphical model structure using 1-regularization paths. In Proceedings of the 22nd national conference on artificial intelligence, Vancouver, Canada (Vol. 2, pp. 1278–1283).Google Scholar
  20. 20.
    Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society (SERIES B), 58, 267–288.zbMATHMathSciNetGoogle Scholar
  21. 21.
    van Borkulo, C. D., Kamphuis, J. H., & Waldorp, L. J. (2013). Predicting behaviour of networks of mental disorders: The contact process as a model for dynamics of psychopathology (Technical report). University of Amsterdam.Google Scholar
  22. 22.
    van der Aart, P. J. M., & Smeenk-Enserink, N. (1975). Correlations between distributions of hunting spiders (Lycosidae, Ctenidae) and environmental characteristics in a dune area. Netherlands Journal of Zoology, 25, 1–45.CrossRefGoogle Scholar
  23. 23.
    Wainwright, M. J., & Jordan, M. I. (2008). Graphical models, exponential families, and variational inference. Foundations and Trends in Machine Learning, 1(1–2), 1–305.zbMATHGoogle Scholar
  24. 24.
    Wainwright, M. J., Ravikumar, P., & Lafferty, J. D. (2007). High-dimensional graphical model selection using 1-regularized logistic regression. In B. Schölkopf, J. Platt, & T. Hoffman (Eds.), Advances in neural information processing systems, 20 (NIPS 2007), (Vol. 19, pp. 1465–1472). Cambridge: MIT Press.Google Scholar
  25. 25.
    Watson, D., Clark, L. A., & Tellegen, A. (1988). Development and validation of brief measures of positive and negative affect: The panas scales. Journal of Personality and Social Psychology, 54(6), 1063–1070.CrossRefGoogle Scholar
  26. 26.
    Yang, E., Ravikumar, P. K., Allen, G. I., & Liu, Z. (2012). Graphical models via generalized linear models. In P. Bartlett, F. Pereira, C. Burges, L. Bottou, & K. Weinberger (Eds.), Advances in neural information processing systems, Curran Associates, Inc. (Vol. 25, pp. 1367–1375).Google Scholar
  27. 27.
    Zou, H. (2006). The adaptive lasso and its oracle properties. Journal of the American Statistical Association, 101(476), 1418–1429.zbMATHMathSciNetCrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2015

Authors and Affiliations

  • Gerda Claeskens
    • 1
    Email author
  • Eugen Pircalabelu
    • 1
  • Lourens Waldorp
    • 2
  1. 1.ORSTAT and Leuven Statistics Research Center, KU LeuvenLeuvenBelgium
  2. 2.Department of Psychological MethodsUniversity of AmsterdamAmsterdamThe Netherlands

Personalised recommendations