Educational Methods & Psychometrics (EMP)

ISSN: 2943-873X

Abstract


Farshad Effatpanah, Purya Baghaei

Technische Universität Dortmund, Dortmund, Germany

International Association for the Evaluation of Educational Achievement (IEA), Hamburg, Germany

Keywords: L2 writing, linguistic features, dimensionality, rating scales, Rasch model

ABSTRACT

It has been acknowledged that second/foreign language (L2) writing is a complex and multi-dimensional cognitive process, and linguistic knowledge is the foremost predictor of L2 writing. Previous research on developing models and orientations for characterizing L2 writing and its linguistic features are based on methods rooted in classical test theory (CTT) which mostly tend to overlook qualitative differences among writers. The use of item response theory (IRT) and Rasch models has been disregarded in L2 writing research. This study aimed to psychometrically investigate the dimensionality of linguistic features in L2 writing using the Rasch model. To achieve this, 500 Iranian English as a foreign language (EFL) students wrote an essay marked by four experienced raters using an empirically-derived descriptor-based diagnostic checklist. The scores derived from the marking of the essays were subjected to Rasch model analysis. Individual item/descriptor fit, separation and reliability, unidimensionality, and local item dependency (LID) were examined. The results provided evidence for the multidimensionality of linguistic features in L2 writing. The analysis of the positive and negative item loadings on Factor 1, extracted from the Rasch model residuals, revealed the presence of two sets of descriptors that contribute to the definition of two groups of L2 writers. The first set comprises descriptors with positive loadings mostly related to higher-level linguistic features of L2 writing, including content fulfillment (CON) and organizational effectiveness (ORG). However, the second set includes descriptors with negative loadings chiefly related to lower-level linguistic features, such as vocabulary use (VOC), grammatical knowledge (GRM), and mechanics (MCH). Implications and suggestions for further research are discussed.

PUBLISHED

08-02-2024

ISSUE

Vol.2,2024

SECTION

Research Article