Reading Between the Lines: Evaluating GenAI's Detection of Attitudinal Meaning in News Discourse
Abstract
This study investigates the potential and limitations of ChatGPT in identifying attitudinal resources in news discourse. Fifteen news articles were analyzed by ChatGPT, and the results generated by it were compared with manual annotations obtained in a previous study. The findings show that ChatGPT is capable of identifying attitudinal resources and generally captures the overall evaluative orientation of the texts. Compared with human annotators, however, ChatGPT tends to identify fewer positive resources but more neutral ones. The analysis also reveals several limitations of ChatGPT in attitude identification. It tends to over mark a particular type of attitudinal resources within a single discourse; moreover, its responses are not always stable, as the identification and interpretation process may be continuously revised when it is challenged through subsequent prompts. These findings suggest that, although ChatGPT has considerable potential as an auxiliary tool for discourse analysis and may improve the efficiency of large-scale attitudinal analysis, its analytical results require careful human validation. At present, it should be regarded only as a complementary tool rather than a substitute for human in discourse research.
References
- Martin, J. R., & White, P. R. R. (2005).The language of evaluation: Appraisal in English. Palgrave Macmillan.
- Liu, B. (2012).Sentiment analysis and opinion mining. Morgan & Claypool Publishers. https://doi.org/10.2200/S00416ED1V01Y201204HLT016
- Wu, H., Bakar, K. A., Jaludin, A., & Mat Awal, N. (2022). Sentiment analysis of China-related news in The Star Online newspaper.GEMA Online Journal of Language Studies, 22(3), 155–175. http://doi.org/10.17576/gema-2022-2203-09
- Thompson, G. (2004).Introducing functional grammar. https://doi.org/10.4324/9780203431474
- Hasbi, N., Ting, S. H., & Jerome, C. (2024). Look beyond the facts: Evaluative language use in Malaysia's six-state election news coverage.Asian Journal of Research in Education and Social Sciences, 6(1), 764–778.
- Grundlingh, L. (2017). Identifying markers of sensationalism in online news reports on crime.Language Matters, 48(2), 11–136.
- Puspita, D., & Pranoto, B. E. (2021). The attitude of Japanese newspapers in narrating disaster events: Appraisal in critical discourse study.Studies in English Language and Education, 8(2), 796-817.
- Kazmi, S., Mohd Noor, S. N. F., & Sanif, S. (2023). Social actors' representational role in political hard news reporting: An approach through attitude analysis.European Chemical Bulletin, 12, 271-285.
- Sobiech-Buzała, M. (2023). Will Chat GPT take our jobs?: Discourse analysis on generative AI from the moral panic perspective.Biuletyn Naukowy Wrocławskiej Wyższej Szkoły Informatyki Stosowanej. Informatyka, 10, 46-59.
- Fan, Y., Jiang, F., Li, P., & Li, H. (2024, May). Uncovering the potential of chatgpt for discourse analysis in dialogue: An empirical study. InProceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), 16998-17010.
- Curry, N., Baker, P., & Brookes, G. (2024). Generative AI for corpus approaches to discourse studies: A critical evaluation of ChatGPT.Applied Corpus Linguistics, 4(1), 100082.
- Gao, Q., & Feng, D. (2025). Deploying large language models for discourse studies: An exploration of automated analysis of media attitudes.PloS One, 20(1), e0313932.
- Wu, H. (2026). China from the lenses of Malaysian news media: A corpus-driven analysis of sentiment and attitude.PhD Thesis. Universiti Kebangsaan Malaysia.