Information-Theoretic Measures of Fit for Univariate and Multivariate Linear Regressions

Part of the Advanced Studies in Theoretical and Applied Econometrics book series (ASTA, volume 25)


For the purpose of measuring the relative importance of independent variables in a multiple regression, Kruskal (1987) proposed an averaging procedure over all possible orderings of these variables. The present article uses this suggestion, but it is based on a different measure, from statistical information theory, and it extends the result to systems of equations.

Key Words

Correlation Information theory Multiple regression Systems of Equations 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Anderson, T.W.: 1984, An Introduction to Multivariate Statistical Analysis, 2nd ed., John Wiley, New York.Google Scholar
  2. Chung, C.-F.: 1987, “On Calculating the Information-Theoretic Measure of Fit for Multivariate Linear Regressions,” McKethan-Matherly Discussion Paper 28, University of Florida, College of Business Administration.Google Scholar
  3. Hotelling, H.: 1936, “Relations Between Two Sets of Variates,” Biometrika, 28, 321–377.Google Scholar
  4. Kruskal, W.: 1987, “Relative Importance by Averaging Over Orderings,” The American Statistician, 41, 6–10.Google Scholar
  5. Lawley, D.N.: 1956, “Tests of Significance of the Latent Roots of Covariance and Correlation Matrices,” Biometrika, 43, 128–136.Google Scholar
  6. Lawley, D.N.: 1959, “Tests of Significance in Canonical Analysis,” Biometrika, 46, 59–66.Google Scholar
  7. Theil, H.: 1958, Economic Forecasts and Policy, North-Holland, Amsterdam.Google Scholar
  8. Theil, H.: 1971, Principles of Econometrics, John Wiley, New York.Google Scholar
  9. Theil, H.: 1987, “How Many Bits of Information Does an Independent Variable Yield in a Multiple Regression?” Statistics and Probability Letters, 6, 107–108.CrossRefGoogle Scholar
  10. Theil, H., and Chung, C.-F.: 1988, “Relations Between Two Sets of Variable: The Bits of Information Provided by Each Variate in Each Set,” Statistics and Probability Letters, 6, 137–139.CrossRefGoogle Scholar
  11. Theil, H., and Fiebig, D.G.: 1984, “Exploiting Continuity: Maximum Entropy Estimation of Continuous Distributions,” Ballinger, Cambridge, Massachusetts.Google Scholar
  12. Tinsley, P.A., Spindt, P.A., and Friar, M.E.: 1980, “Indicator and Filter Attributes of Monetary Aggregates: A Nit-Picking Case for Disaggregation,” Journal of Econometrics, 14, 61–91.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media Dordrecht 1992

Authors and Affiliations

  1. 1.University of FloridaGainesvilleUSA

Personalised recommendations