Variance Components in Generalizability Theory

  • Robert L. Brennan
Part of the Perspectives on Individual Differences book series (PIDF)

Abstract

Generalizability theory is described most extensively in a book by Cronbach, Gleser, Nanda, and Rajaratnam (1972) entitled The Dependability of Behavioral Measurements. Brennan (1983) has provided a monograph on generalizability theory that is less comprehensive than Cronbach et al. (1972) but still detailed enough to convey many of the conceptual and statistical issues inherent in generalizability theory. Also, Shavelson and Webb (1991) have provided a primer on generalizability theory. Recent overviews of generalizability theory are provided by Feldt and Brennan (1989) and Shavelson, Webb, and Rowley (1989).

Keywords

Generalizability Theory Variance Component Bootstrap Sample Estimate Variance Component Score Effect 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. American Psychological Association. (1985). Standards for educational and psychological testing. Washington, DC: Author.Google Scholar
  2. Bell, J. F. (1985). Generalizability theory: The software problem. Journal of Educational Statistics, 10, 19–29.CrossRefGoogle Scholar
  3. Bock, R. D. (1972). [Review of The dependability of behavioral measurements]. Science, 178,1275–1276.Google Scholar
  4. Box, G. E. P., & Tiao, G. C. (1973). Bayesian inference in statistical analysis. Reading, MA: Addison-Wesley.Google Scholar
  5. Brennan, R. L. (1983). Elements of generalizability theory. Iowa City, IA: American College Testing.Google Scholar
  6. Brennan, R. L. (1984). Some statistical issues in generalizability theory. (ACT Technical Bulletin No. 46 ). Iowa City: American College Testing.Google Scholar
  7. Brennan, R. L., Harris, D. J., & Hanson, B. A. (1987). The bootstrap and other procedures for examining the variability of estimated variance components in testing contexts. (ACT Research Report No. 87–7 ). Iowa City, IA: American College Testing.Google Scholar
  8. Brennan, R. L., Jarjoura, D., & Deaton, E. L. (1980). Some issues concerning the estimation and interpretation of variance components in generalizability theory. (ACT Technical Bulletin No. 36.) Iowa City, IA: American College Testing.Google Scholar
  9. Cardinet, J., & Allal, L. (1983). Estimation of generalizability parameters. In L. J. Fyans (Ed.), Generalizability theory: Inferences and practical applications (pp. 17–48 ). San Francisco: Jossey-Bass.Google Scholar
  10. Cardinet, J., Tourneur, Y., & Allal, L. (1981). Extension of generalizability theory and its applications in educational measurement. Journal of Educational Measurement, 18, 183–204. Errata, Journal of Educational Measurement, 1982, 19, 331–332.Google Scholar
  11. Chastain, R. L., & Willson, V. L. (1986). Effect on generalizability coefficients when estimating variance components using computer packages. Paper presented at the Annual Meeting of the American Educational Research Association, San Francisco, CA, April.Google Scholar
  12. Collins, J. D. (1970). Jackknifing generalizability. Unpublished doctoral dissertation, University of Colorado, Boulder.Google Scholar
  13. Colton, D. A. (1983). A multivariate generalizability analysis of a test developed according to a table of specifications. Unpublished master’s thesis. The University of Iowa, Iowa City.Google Scholar
  14. Corbeil, R. R., & Searle, S. (1976). Restricted maximum likelihood (REML) estimation of variance components in the mixed model. Technometrics, 15, 819–826.Google Scholar
  15. Cornfield, J., & Tukey, J. W. (1956). Average values of mean squares in factorials. Annals of Mathematical Statistics, 27, 907–949.CrossRefGoogle Scholar
  16. Crick, J. E., & Brennan, R. L. (1983). Manual for GENOVA: A GENeralized analysis Of VAriance system (ACT Technical Bulletin No. 43 ). Iowa City, IA: American College Testing.Google Scholar
  17. Cronbach, L. J., Gleser, G. C., Nanda, H., & Rajaratnam, N. (1972). The dependability of behavioral measurements: Theory of generalizability for scores and profiles. New York: Wiley.Google Scholar
  18. Dixon, W. J. (Ed.) (1985). BMDP statistical software. Los Angeles: University of California Press.Google Scholar
  19. Edelman, D. A. (1974). Three-stage nested designs with composite samples. Technometrics, 16, 409–417.Google Scholar
  20. Efron, B. (1982). The jackknife, the bootstrap, and other resampling plans. Society forIndustrial and Applied Mathematics, CBMS-NSF Monograph 38, Philadelphia. Efron, B., & Tibshirani, R. (1986). Bootstrap methods for standard errors, confidenceGoogle Scholar
  21. intervals, and other measures of statistical accuracy. Statistical Science, 1,54–77.Google Scholar
  22. Feldt, L. S., & Brennan, R. L. (1989). Reliability. In R. L. Linn (Ed.), Educational measurement ( 3rd ed., pp. 105–146 ). New York: Macmillan.Google Scholar
  23. Gates, C. E., & Shiue, C. (1962). The analysis of variance of the S-stage hierarchical classification. Biometrics, 18, 529–536.CrossRefGoogle Scholar
  24. Gower, J. C. (1962). Variance component estimation for unbalanced hierarchical classifications. Biometrics, 18, 537–542.CrossRefGoogle Scholar
  25. Graybill, F. A. (1976). Theory and application of the linear model. North Scituate, MA: Duxbury Press.Google Scholar
  26. Hartley, H. O. (1967). Expectation, variances, and covariances of ANOVA mean squares by `synthesis.’ Biometrics, 23, 105–114, and Corrigenda, 853.Google Scholar
  27. Hartley, H. O., Rao, J. N. K., & LaMotte, L. R. (1978). A simple `synthesis’-based method of variance component estimation. Biometrics, 34, 233–242.CrossRefGoogle Scholar
  28. Hartley, H. O., & Searle, S. R. (1969). A discontinuity in mixed model analyses. Biometrics, 25, 573–576.CrossRefGoogle Scholar
  29. Harville, D. A. (1977). Maximum likelihood approaches to variance component estimation and to related problems. Journal of the American Statistical Association, 72, 320–338.CrossRefGoogle Scholar
  30. Henderson, C. R. (1953). Estimation of variance and covariance components. Biometrics, 9, 226–252.CrossRefGoogle Scholar
  31. Jarjoura, D., & Brennan, R. L. (1981). Three variance components models for some measurement procedures in which unequal numbers of items fall into discrete categories (ACT Technical Bulletin No. 37 ). Iowa City, IA: American College Testing.Google Scholar
  32. Jarjoura, D., & Brennan, R. L. (1982). A variance components model for measurement procedures associated with a table of specifications. Applied Psychological Measurement, 6, 161–171.CrossRefGoogle Scholar
  33. Jarjoura, D., & Brennan, R. L. (1983). Multivariate generalizability models for tests developed from tables of specifications. In L. J. Fyans (Ed.), Generalizability theory: Inferences and practical applications (pp. 83–101 ). San Francisco: Jossey-Bass.Google Scholar
  34. Kirk, R. E. (1982). Experimental design ( 2nd ed. ). Belmont, CA: Wadsworth.Google Scholar
  35. Koch, G. G. (1968). Some further remarks concerning “A general approach to the estimation of variance components.” Technometrics, 10, 551–558.Google Scholar
  36. Loevinger, J. (1965). Person and population as psychometric concepts. Psychological Review, 72, 143–155.PubMedCrossRefGoogle Scholar
  37. Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.Google Scholar
  38. Marcoulides, G. A. (1987). An alternative method for variance component estimation: Applications to generalizability theory. Unpublished doctoral dissertation, University of California, Los Angeles.Google Scholar
  39. Millman, J., & Glass, G. C. (1967). Rules of thumb for writing the ANOVA table. Journal of Educational Measurement, 4, 41–51.CrossRefGoogle Scholar
  40. Mosteller, F., & Tlikey, J. W. (1968). Data analysis, including statistics. In G. Lindzey and E. Aronson (Eds.). The handbook of social psychology, Vol. 2. Research methods ( 2nd ed., pp. 80–203 ). Reading, MA: Addison-Wesley.Google Scholar
  41. Quenouille, M. (1949). Approximate tests of correlation in time series. Journal of the Royal Statistical Society-Series B, 11, 18–84.Google Scholar
  42. Rao, C. R. (1971). Estimation of variance and covariance components-MINQUE theory. Journal of Multivariate Analysis, 1, 257–275.CrossRefGoogle Scholar
  43. Rao, C. R., & Kleffe, J. (1988). Estimation of variance components and applications. New York: Elsevier Science Publishing.Google Scholar
  44. Samuels, M. L., Casella, G., & McCabe, G. P. (1991). Interpreting blocks and random factors. Journal of the American Statistical Association, 86, 798–808.CrossRefGoogle Scholar
  45. SAS Institute. (1989). SASISTAT User’s Guide (version 6, 4th ed. ). Cary, NC: SAS Institute Inc.Google Scholar
  46. Satterthwaite, F. E. (1941). Synthesis of variance. Psychometrika, 6, 309–316. Satterthwaite, F. E. (1946). An approximate distribution of estimates of variance components. Biometrics Bulletin, 2, 110–114.Google Scholar
  47. Scheffé, H. (1959). The analysis of variance. New York: Wiley.Google Scholar
  48. Searle, S. R. (1971). Linear models. New York: Wiley.Google Scholar
  49. Searle, S. R. (1979). Notes on variance component estimation-A detailed account of maximum likelihood and kindred methodologies. Unpublished manuscript, Cornell University, Biometrics Unit, Ithica, NY.Google Scholar
  50. Shavelson, R. J., & Webb, N. M. (1981). Generalizability theory: 1973–1980. British Journal of Mathematical and Statistical Psychology, 34, 133–166.CrossRefGoogle Scholar
  51. Shavelson, R. J., & Webb, N. M. (1991). Generalizability theory: A primer. Newbury Park, CA: Sage.Google Scholar
  52. Shavelson, R. J., Webb, N. M., & Rowley, G. L. (1989). Generalizability theory. American Psychologist, 44, 922–932.CrossRefGoogle Scholar
  53. Sirotnik, K., & Wellington, R. (1977). Incidence sampling: An integrated theory for “matrix sampling.” Journal of Educational Measurement, 14, 343–399.CrossRefGoogle Scholar
  54. Smith, P. L. (1978). Sampling errors of variance components in small sample generalizability studies. Journal of Educational Statistics, 3, 319–346.CrossRefGoogle Scholar
  55. Smith, P. L. (1982). A confidence interval approach for variance component estimates in the context of generalizability theory. Educational and Psychological Measurement, 42, 459–466.CrossRefGoogle Scholar
  56. Tukey, J. (1958). Bias and confidence in not quite large samples. Abstract in Annals of Mathematical Statistics, 29, 614.CrossRefGoogle Scholar
  57. Webb, N. M., Shavelson, R. J., & Maddahian, E. (1983). Multivariate generalizability theory. In L. J. Fyans (Ed.), Generalizability theory: Inferences and practical applications (pp. 67–81 ). San Francisco: Jossey-Bass.Google Scholar
  58. Welch, B. L. (1956). On linear combinations of several variances. American Statistical Association Journal, 51, 132–148.CrossRefGoogle Scholar
  59. Winer, B. J. (1971). Statistical principles in experimental design ( 2nd ed. ). New York: McGraw-Hill.Google Scholar

Copyright information

© Springer Science+Business Media New York 1994

Authors and Affiliations

  • Robert L. Brennan
    • 1
  1. 1.Research DivisionAmerican College TestingIowa CityUSA

Personalised recommendations