Communications in Humanities Research ISSN: 2753-7072
Home> Volumes> Volume CHR Vol.104> Article
Research Article

Pedagogical Value and Limitations of Automated Writing Evaluation in English as a Second Language Writing Instruction

Ziling HuangNanjing Normal University*

* Corresponding author: [email protected]

Abstract

With the advancement of artificial intelligence and natural language processing technologies, the automated writing evaluation system (AWE) has gradually been integrated into English as a Second Language (ESL) writing instruction and has gained potential value for enhancing feedback efficiency and supporting writing revision. However, there are still some disputes over their instructional effectiveness and the boundaries of their application. This study analyzes the pedagogical value and limitations of AWE in ESL writing instruction through reviews of empirical studies, systematic reviews and meta-analyses published in the past 20 years. The findings show that AWE has a relatively stable positive effect on improving surface-level linguistic features such as grammar, spelling, and punctuation, and can sustain several rounds of modification and to a certain degree, improve learners' motivation and autonomy in writing. However, it is still unknown whether it is valid and stable enough to assess higher-order writing abilities such as writing content development, argumentation logic and appropriateness. This study argues that AWE should be viewed as a supplementary tool to provide formative assessments, functioning effectively in instructional models and human–AI collaboration. The findings of this study may provide pedagogical implications for the rational integration of AWE tools into ESL writing classrooms, offering directions for future research in this field.

Keywords: Automated Writing Evaluation (AWE); ESL writing instruction; formative assessment
Published: March 2, 2026
DOI: 10.54254/2753-7064/2026.HT32003
Volume: CHR Vol.104
pp. 1-10
Download PDF

References

  1. Kurt, G., Atay, D., & Öztürk, H. A. (2022). Student engagement in K12 online education during the pandemic: The case of Turkey. Journal of Research on Technology in Education, 54(sup1), S31–S47.
  2. Evans, S., & Green, C. (2007). Why EAP is necessary: A survey of Hong Kong tertiary students. Journal of English for Academic Purposes, 6(1), 3–17.
  3. Fan, N., & Ma, Y. (2022). The effects of automated writing evaluation (AWE) feedback on students' English writing quality: A systematic literature review. Language Teaching Research Quarterly, 28, 53–73.
  4. Warschauer, M., & Ware, P. (2006). Automated writing evaluation: Defining the classroom research agenda. Language Teaching Research, 10(2), 1–24.
  5. Liu, W. (2024). A systematic review of automated writing evaluation feedback: Validity, effects and students' engagement. Language Teaching Research Quarterly, 45, 86–105.
  6. Barrot, J. S. (2021). Using automated written corrective feedback in the writing classroom: Effects on L2 writing accuracy. Computer-Assisted Language Learning. Advance online publication.
  7. Zhang, X. (2022). Improving Pigai as an automated writing evaluation system: Considerations for refinement. Frontiers in Psychology, 13, Article 795725.
  8. Huang, X., Xu, W., Li, F., & Yu, Z. (2024). A meta-analysis of effects of automated writing evaluation on anxiety, motivation, and second language writing skills. The Asia-Pacific Education Researcher, 33, 957–976.
  9. Shi, H., & Aryadoust, V. (2022). A systematic review of automated writing evaluation systems. Education and Information Technologies. Advance online publication.
  10. Escalante, J., Pack, A., & Barrett, A. (2023). AI-generated feedback on writing: Insights into efficacy and ENL student preference. International Journal of Educational Technology in Higher Education, 20(57).
  11. Shermis, M. D., & Wilson, J. (2024). Introduction to automated essay evaluation. In The Routledge international handbook of automated essay evaluation (pp. 3–22). Routledge.
  12. Tang, J., & Rich, C. S. (2017). Automated writing evaluation in an EFL setting: Lessons from China. JALT CALL Journal, 13(2), 117–143.
  13. Rahman, N. A., Zulkornain, L. H., Che Mat, A., & Kustati, M. (2023). Assessing writing abilities using AI-powered writing evaluations. Journal of Asian Behavioural Studies, 8(24), 1–17.
  14. Wilson, J., Ahrendt, C., Fudge, E. A., Raiche, A., Beard, G., & MacArthur, C. A. (2021). Elementary teachers' perceptions of automated feedback and automated scoring: Transforming the teaching and learning of writing using automated writing evaluation. Computers & Education, 168, 104208.
  15. Zhang, Z. V. (2017). Student engagement with computer-generated feedback: A case study. ELT Journal, 71(3), 317–328.
  16. Roscoe, R. D., Wilson, J., Johnson, A. C., & Mayra, C. R. (2017). Presentation, expectations, and experience: Sources of student perceptions of automated writing evaluation. Computers in Human Behavior, 70, 207–221. https: //doi.org/10.1016/j.chb.2016.12.076
  17. Sung, Y. T., Liao, C. N., Chang, T. H., Chen, C. L., & Chang, K. E. (2016). The effect of online summary assessment and feedback system on the summary writing on 6th graders: The LSA-based technique. Computers & Education, 95, 1–18.
  18. Kim, H., Baghestani, S., Yin, S., Karatay, Y., Kurt, S., Beck, J., & Karatay, L. (2024). ChatGPT for writing evaluation: Examining the accuracy and reliability of AI-generated scores compared to human raters. In C. A. Chapelle, G. H. Beckett, & J. Ranalli (Eds.), Exploring artificial intelligence in applied linguistics (pp. 73–95). Iowa State University Digital Press.
  19. Liu, Z. (2024). Second language writing anxiety and ChatGPT adoption as an automated writing evaluation tool. Journal of Applied Research in Higher Education. Advance online publication.