The Crutch or the Ceiling? How Different Generations of LLMs Shape EFL Student Writings

arXiv:2604.15460v1 Announce Type: cross Abstract: The rapid evolution of Large Language Models (LLMs) has made them powerful tools for enhancing student writing. This study explores the extent and limitations of LLMs in assisting secondary-level English as a Foreign Language (EFL) students with their writing tasks. While existing studies focus on output quality, our research examines the developmental shift in LLMs and their impact on EFL students, assessing whether smarter models act as true scaffolds or mere compensatory crutches. To achieve this, we analyse student compositions assisted by LLMs before and after ChatGPT's release, using both expert qualitative scoring and quantitative metrics (readability tests, Pearson's correlation coefficient, MTLD, and others). Our results indicate that advanced LLMs boost assessment scores and lexical diversity for lower-proficiency learners, potentially masking their true ability. Crucially, increased LLM assistance correlated negatively with human expert ratings, suggesting surface fluency without deep coherence. To transform AI-assisted practice into genuine learning, pedagogy must shift from focusing on output quality to verifying the learning process. Educators should align AI functions, specifically differentiating ideational scaffolding from textual production, within the learner's Zone of Proximal Development.

Related

• Generating Multiple-Choice Knowledge Questions with Interpretable Difficulty Estimation using Knowledge Graphs and Large Language Models
• Can We Still Hear the Accent? Investigating the Resilience of Native Language Signals in the LLM Era
• EDUMATH: Generating Standards-aligned Educational Math Word Problems
• Leveraging Machine Learning Techniques to Investigate Media and Information Literacy Competence in Tackling Disinformation

Source: arXiv cs.AI | 2026-04-20