Go to content

2026_19_4_1 - XLinguae

Skip menu
Skip menu

2026_19_4_1

Open access Issues > Issue n_4_2026 > Issue n_4_2026
AI, Human, and Hybrid Approaches to Language Assessment: Analyzing EFL Writing across Cognitive and Contextual Dimensions

Sohaib Alam

DOI: 10.18355/XL.2026.19.04.01

Abstract
The increasing integration of artificial intelligence (AI) into language assessment has intensified the need to examine its effectiveness in evaluating both the cognitive and contextual aspects of second language writing. This study compared AI-only, human-only, and hybrid AI-human assessment of English as a Foreign Language (EFL) argumentative writing across three cognitive dimensions (logical organization, argumentative reasoning, and idea development) and three contextual dimensions (contextual appropriateness of language, audience awareness, and register and tone appropriateness). A quantitative, comparative, cross-sectional research design was employed using 167 argumentative essays produced by undergraduate EFL students at a private university in Saudi Arabia. Experienced human examiners and ChatGPT (GPT-4o) independently evaluated the essays using an identical analytic rubric, and hybrid scores were derived by averaging the human and AI evaluations. Descriptive statistics, intraclass correlation coefficients, weighted kappa coefficients, repeated-measures analysis of variance, and Bonferroni-adjusted pairwise comparisons were conducted to compare evaluation outcomes, agreement, and score differences. The findings demonstrated broadly comparable scoring patterns across both cognitive and contextual aspects of writing. The three assessment methods showed moderate to good agreement, though agreement was lower for contextual dimensions requiring more interpretive judgment. Statistically significant differences were identified for only three dimensions, with uniformly small effect sizes indicating limited practical significance. Overall, the findings suggest that AI can effectively complement human judgment under standardized assessment conditions, while hybrid assessment offers a balanced approach that integrates computational consistency with human interpretive expertise for multidimensional EFL writing assessment.

Key words: Artificial intelligence, ChatGPT, EFL writing assessment, hybrid assessment, human assessment, technology-enhanced language assessment, SDG-4, educational technology

Pages: 3-31

Full Text
Back to content