A Three-Step Approach to Factor Analysis on Data of Multiple Testlets

  • Soonmook Lee
  • Ahyoung Kim
Conference paper


A testlet consists of a text and several items following it. It is often observed that a test consists of multiple testlets. The items within a testlet are more interrelated than they are with other items in the test. In this note, we attempt to show how to estimate common factors in the data of multiple testlets. We start our argument from the rationale of common factor analysis on each testlet. However, we treat the effect of a testlet on the item scores within the testlet as the method effect. If we are able to remove the method effect from subjects’ response data, then we can apply ordinary factor analysis on the residualized scores that remain after partialing out the method effect. Application of our approach is also demonstrated.


Design Factor Confirmatory Factor Analysis Tacit Knowledge Response Alternative Method Factor 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Gorsuch, R. L. (1983). Factor analysis ( 2nd Ed. ), Hillsdale, NJ: Lawrence Erlbaum.Google Scholar
  2. Harman, H. H. (1976). Modern Factor Analysis. 3rd Ed. Chicago, IL: University of Chicago Press.Google Scholar
  3. Jöreskog, K. G. and Sörbom, D. (1998). LISREL8. Chicago, IL: Scientific Software.Google Scholar
  4. Learner, E. E. (1978). Specification Searches: Ad hoc inference with nonexperimental data, New York: Wiley.Google Scholar
  5. Lee, G. (2000). Estimating conditional standard errors of measurement for tests composed of tests. Applied Measurement in Education, 13, 161–180.CrossRefGoogle Scholar
  6. Lee, G., and Frisbie, D. A. (1999). Estimating reliability under a generalizability theory model for test scores composed of testlets. Applied Measurement in Education, 12, 237–255.CrossRefGoogle Scholar
  7. MacCallum, R. (1986). Specification searches in covariance structure modeling. Psychological Bulletin, 100, 107–120.CrossRefGoogle Scholar
  8. Polanyi, M. (1958). Personal knowledge: Toward a post-critical philosophy. Chicago: University of Chicago PressGoogle Scholar
  9. Sireci, S. G., Thissen, D., and Wainer, H. (1991). On the reliability of testlet-based tests. Journal of Educational Measurement, 28, 237–247.CrossRefGoogle Scholar
  10. Sternberg, R. J., Wagner, R. K. and Okagaki, L. (1993). Practical Intelligence: The nature and role of tacit knowledge in work and at school. In H. Reese and J. Puckett(Eds), Advances in Lifespan Development, 205–227. Hillsdale, NJ: Earlbaum.Google Scholar
  11. Sternberg, R. J., Wagner, R. K., Williams, W. M., and Horvath, J. A. (1995). Testing common sense. American Psychologist. November, 912–926.Google Scholar
  12. Wagner, R. K. and Sternberg, R. J. (1985). Practical Intelligence in Real-World Pursuits: The role of tacit knowledge. Journal of Personality and Social Psychology, 49. 436–458.CrossRefGoogle Scholar
  13. Wagner, R. K. and Sternberg, R. J. (1993). TKIM: The commonsense manager, user manual. New York: Hartcourt Brace Jovanovich, Inc.Google Scholar
  14. Harman, H.H (1976). Modern factor analysis. 3rd Ed. Chicago, ILL: University of Chicago Press.Google Scholar

Copyright information

© Springer Japan 2002

Authors and Affiliations

  • Soonmook Lee
    • 1
  • Ahyoung Kim
    • 1
  1. 1.Sungkyunkwan University, Ewha Womans UniversitySeoulKorea

Personalised recommendations