Validating Research-Abstract Writing Assessment Through Latent Regression Modeling and Rater’s Lenses

Abstract

This study validates the measure, research-abstract writing assessment (RAWA), with two rating scales of global move of rhetorical purpose and local pattern of language use in applied linguistics (the scale level/score ranging from 0 to 5). The study adopted the embedded design of mixed-methods research that included both the quantitative latent regression model (LRM) for testing how the examinees’ (30 EFL doctoral students, 30 EFL master’s students) RAWA responses can be explained by examinee-group competence, scale-by-level difficulty of two scales, and rater expertise (5 raters); and the qualitative interviews on five raters’ perceptions. The LRM results revealed the scale-level difficulty effect, namely across the scales level 1 and level 5 of the global move being the easiest and the most difficult. The expert raters rated with lower scores. They also adopted the advanced subscales (i.e., content elements, brevity) as criteria and conducted self-monitoring while rating. The findings reveal the sub-competences of research-abstract writing, namely the global move sub-competence of move and content elements and the local pattern sub-competence of language use and brevity. Pedagogically, EFL graduate students should further develop the sub-competences of content element and brevity, once mastering those of move and language use as the basics.

中文摘要

本文驗證一個論文摘要寫作評量, 此評量使用兩個評分表 (整體性修辭文步、局部性語言構式)於應用語言學學門的論文摘要寫作評量之中 。本文使用鑲嵌式設計的混合研究法, 亦即以質性評分者訪談結果(5位評分者)補充量性潛在迴歸模式分析結果 。潛在迴歸模式用於考驗受試者 (碩士生、博士生各30位)的寫作評量作答反映是否可被受試者知能、評分量尺各等級的難度、評分者的專業度三個變項解釋 。模式考驗結果指出評分量尺各等級的難度對受試者反映有解釋效果(如:修辭文步評量表的一分、五分分別為困難度最低、最高的反映); 評分者的專業度有效果, 即專家評分者給分較低 。專家評分者通常採用進階的評分量尺 (內容要件、簡潔性), 作為評分歷程中自我檢視的標準 。這些研究發現揭示論文摘要寫作的次項知能, 即整體性修辭文部知能包含文步使用、內容要件, 局部性語言構式包含語言使用、簡潔性 。教學意涵如下 : 外語研究生應先發展語言和文步使用等基礎知能, 後續再發展內容要件和簡潔性知能 。