Abstract
In test score equating, the nonequivalent groups with covariates (NEC) design use covariates with high correlation to the test scores as a substitute for an anchor test when the latter is lacking. However, as the number of covariates increases, the number of observations for each covariate combination decreases. We suggest to use propensity scores instead, which we include in the kernel equating framework using both post-stratification and chained equating. The two approaches are illustrated with data from a large-scale assessment, and the results show an increased precision in comparison with the equivalent groups design and great similarities in comparison with the results when using an anchor test.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
B. Andersson, K. Bränberg, M. Wiberg, Performing the kernel method of test equating using the R package kequate. J. Stat. Softw. 55(6), 1–25 (2013)
K. Bränberg, M. Wiberg, Observed score linear equating with covariates. J. Educ. Meas. 48, 419–440 (2011)
J. González, M. Wiberg, Applying Test Equating Methods Using R (Springer, Berlin, 2017). doi:10.1007/978-3-319-51824-4
S.J. Haberman, Pseudo-equivalent groups and linking. J. Educ. Behav. Stat. 40, 254–273 (2015)
J. Häggström, M. Wiberg, Optimal bandwidth selection in kernel equating. J. Educ. Meas. 51, 201–211 (2014)
T. Hsu, K. Wu, J. Yu, M. Lee, Exploring the feasibility of collateral information test equating. Int. J. Test. 2, 1–14 (2009)
M.J. Kolen, Does matching in equating work? A discussion. Appl. Meas. Educ. 3, 23–39 (1990)
M. Liou, P.E. Cheng, M. Li, Estimating comparable scores using surrogate variables. Appl. Meas. Educ. 25, 197–207 (2001)
S.A. Livingston, N.J. Dorans, N.K. Wright, What combination of sampling and equating methods works best? Appl. Meas. Educ. 3, 73–95 (1990)
N.T. Longford, Equating without an anchor for nonequivalent groups of examinees. J. Educ. Behav. Stat. 40, 227–253 (2015)
P.-E. Lyrén, R.K. Hambleton, Consequences of violated the equating assumptions under the equivalent group design. Int. J. Test. 36, 308–323 (2011)
T. Moses, W. Deng, Y.-L. Zhang, The use of two anchors in the nonequivalent groups with anchor test (NEAT) equating. ETS Research Report RR-10-23, 2010
I. Paek, J. Liu, H.J. Oh, Investigation of Propensity Score Matching on Linear/Nonlinear Equating Method for the P/N/NMSQT (Report SR-2006-55) (ETS, Princeton, NJ, 2006)
S.J. Powers, Impact of matched samples equating methods on equating accuracy and the adequacy of equating assumptions, Ph.D. Dissertation, University of Iowa, 2010., http://ir.uiowa.edu/etd/875
R Core Development Team, R: A Language and Environment for Statistical Computing (R Foundation for statistical computing, Vienna, 2016.) http://www.R-project.org/
P.R. Rosenbaum, D.B. Rubin, The central role of the propensity score in observational studies for causal effects. Biometrika 70, 41–55 (1983)
P.R. Rosenbaum, D.B. Rubin, Reducing bias in observational studies using subclassification on the propensity score. J. Am. Stat. Assoc. 79, 516–524 (1984)
N. Sungworn, An investigation of using collateral information to reduce equating biases of the post-stratification equating method, Ph.D. Thesis, Michigan State University, 2009
A.A. von Davier, P.W. Holland, D.T. Thayer, The Kernel Method of Test Equating (Springer, New York, NY, 2004)
G. Wallin, M. Wiberg, Propensity scores in kernel equating under the non-equivalent groups with covariates design, Manuscript submitted for publication, 2017
M. Wiberg, K. Bränberg, Kernel equating under the non-equivalent groups with covariates design. Appl. Psychol. Meas. 39, 349–361 (2015)
Acknowledgment
The research in this paper was funded by the Swedish Research Council grant: 2014-578.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendix
Appendix
Abbreviations of the Data Designs
- ANCHOR-NEC-PS:
-
Post-stratification under the nonequivalent groups with covariates design using propensity scores
- CE NEAT:
-
Chained equating under the nonequivalent groups with anchor test design
- CE-NEC-PS:
-
Chained equating under the nonequivalent groups with covariates design using propensity scores
- EG:
-
Equivalent groups design
- PSE NEAT:
-
Post-stratification equating under the nonequivalent groups with anchor test design
- PSE-NEC-PS:
-
Post-stratification equating under the nonequivalent groups with covariates design using propensity scores
- PSE-NEC-RAW-COV:
-
Post-stratification equating under the nonequivalent groups with covariates design
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Wallin, G., Wiberg, M. (2017). Nonequivalent Groups with Covariates Design Using Propensity Scores for Kernel Equating. In: van der Ark, L.A., Wiberg, M., Culpepper, S.A., Douglas, J.A., Wang, WC. (eds) Quantitative Psychology. IMPS 2016. Springer Proceedings in Mathematics & Statistics, vol 196. Springer, Cham. https://doi.org/10.1007/978-3-319-56294-0_27
Download citation
DOI: https://doi.org/10.1007/978-3-319-56294-0_27
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-56293-3
Online ISBN: 978-3-319-56294-0
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)