Abstract
Vertical scaling is promising in tracking students’ academic growth over time because it links tests with different difficulty levels that measure the same construct on a common scale. However, its application suffers from the violation of the unidimensionality assumption in practical educational testing and the complex procedure of scale construction. A number of important decisions need to be made during scale construction because the results of vertical scaling are subject to many factors, such as the linking method and the item response theory model used, the ability/difficulty of the estimation method and the design of the data collection. This chapter presents a review of the literature on vertical-scale development and finds that there is no agreement in the literature with regard to which approach generates the “best” vertical scale. A new concurrent-separate approach based on the Rasch model is proposed in this chapter. In this approach, concurrent and separate applications are conducted at different stages. The quality of linking items was investigated intensively, and the impact of underfit individuals was taken into account during the vertical scaling procedure. The properties of the resulting scale, called the Mathematics Competency Vertical Scale (MCVS), make it feasible for tracking Hong Kong students’ development in mathematics over time.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Briggs, D. C., & Weeks, J. P. (2009). The impact of vertical scaling decisions on growth interpretations. Educational Measurement: Issues and Practice Winter, 28(4), 3–14.
Camilli, G. (1999). Measurement error, multidimensionality, and scale shrinkage: A reply to Yen and Burket. Journal of Educational Measurement, 36, 73–78.
Camilli, G., Yamamoto, K., & Wang, M. M. (1993). Scale shrinkage in vertical equating. Applied Psychological Measurement, 17(4), 379–388.
Custer, M., Omar, M. H., & Pomplun, M. (2006). Vertical scaling with the Rasch model utilizing default and tight convergence settings with WINSTEPS and BILOG-MG. Applied Measurement in Education, 19(2), 133–149.
Hanson, B. A., & Béguin, A. A. (2002). Obtaining a common scale for item response theory item parameters using separate versus concurrent estimation in the common-item equating design. Applied Psychological Measurement, 26(1), 3–24.
Harris, D. J. (2007). Practical issues in vertical scaling. In N. J. Dorans, M. Pommerich, & P. W. Holland (Eds.), Linking and aligning scores and scales (pp. 233–251). New York: Springer.
Hendrickson, A., Cao, Y., Chae, S. E., & Li, D. (2006, April). Effect of base year on IRT vertical scaling from the common-item design. Paper presented at the National Council on Measurement in Education, San Francisco, CA.
Hong Kong Education Bureau. (1999). Syllabuses for secondary schools: Mathematics (Secondary 1–5). Retrieved on August 12, 2010, from http://www.edb.gov.hk/index.aspx?nodeID=4905&langno=1
Hong Kong Education Bureau. (2000). Mathematics curriculum guide: P1–P6. Retrieved on August 12, 2010, from http://www.edb.gov.hk/index.aspx?nodeID=4907&langno=1
Ito, K., Sykes, R. C., & Yao, L. (2008). Concurrent and separate grade-groups linking procedures for vertical scaling. Applied Measurement in Education, 21(3), 187–206.
Kim, S., & Cohen, A. S. (1998). A comparison of linking and concurrent calibration under item response theory. Applied Psychological Measurement, 22, 131–143.
Kolen, M. J., & Brennan, R. L. (2004). Test equating, scaling, and linking: Methods and practices (2nd ed.). New York: Springer.
Linacre, J. M. (2011). Winsteps (Version 3.72.3) [Computer software]. Chicago: Winsteps.com.
Lissitz, R. W., & Huynh, H. (2003). Vertical equating for state assessments: Issues and solutions in determination of adequate yearly progress and school accountability. Practical Assessment, Research & Evaluation, 8(10). Retrieved on August 11, 2010, from http://PAREonline.net/getvn.asp?v=8&n=10
Patz, R. J., & Yao, L. (2007). Methods and models for vertical scaling. In N. J. Dorans, M. Pommerich, & P. W. Holland (Eds.), Linking and aligning scores and scales (pp. 253–272). New York: Springer.
Petersen, N. S., Cook, L. L., & Stocking, M. L. (1983). IRT versus conventional equating methods: A comparative study of scale stability. Journal of Educational Statistics, 8(2), 137–156.
Pomplun, M., Omar, M. H., & Custer, M. (2004). A comparison of WINSTEPS and BILOG–MG for vertical scaling with the Rasch model. Educational and Psychological Measurement, 64, 600–616.
Tong, Y., & Kolen, M. J. (2007). Comparisons of methodologies and results in vertical scaling for educational achievement tests. Applied Measurement in Education, 20(2), 227–253.
Wingersky, M. S., Cook, L. L., & Eignor, D. R. (1987). Specifying the characteristics of linking items used for item response theory item calibration (ETS Research Rep. 87–24). Princeton: Educational Testing Service.
Wolfe, E. W., & Chiu, C. W. (1999). Measuring change across multiple occasions using the Rasch rating scale model. Journal of Outcome Measurement, 3(4), 360–381.
Wright, B. D. (1996). Time 1 to time 2 comparison: Racking and stacking. Rasch Measurement Transactions, 10(1), 478–479.
Yen, W. M. (2007). Vertical scaling and no child left behind. In N. J. Dorans, M. Pommerich, & P. W. Holland (Eds.), Linking and aligning scores and scales (pp. 273–282). New York: Springer.
Yen, W. M. (2009). Growth models approved for the NCLB growth model pilot. Unpublished manuscript.
Yon, H. (2006). Multidimensional item response theory (MIRT) approaches to vertical scaling. Unpublished doctoral dissertation, Michigan State University, East Lansing, MI.
Zimowski, M. F., Muraki, E., Mislevy, R. J., & Bock, R. D. (1996). BILOG-MG: Multiple-group IRT analysis and test maintenance for binary items [Computer software]. Chicago: Scientific Software International.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer Science+Business Media Dordrecht
About this chapter
Cite this chapter
Yan, Z., Lau, D.C.H., Mok, M.M.C. (2012). A Concurrent-Separate Approach to Vertical Scaling. In: Mok, M. (eds) Self-directed Learning Oriented Assessments in the Asia-Pacific. Education in the Asia-Pacific Region: Issues, Concerns and Prospects, vol 18. Springer, Dordrecht. https://doi.org/10.1007/978-94-007-4507-0_10
Download citation
DOI: https://doi.org/10.1007/978-94-007-4507-0_10
Published:
Publisher Name: Springer, Dordrecht
Print ISBN: 978-94-007-4506-3
Online ISBN: 978-94-007-4507-0
eBook Packages: Humanities, Social Sciences and LawEducation (R0)