Postgraduate readers used AI critiques in different ways, with original-source checking remaining essential

An AI critique can introduce a useful perspective on a research paper, but it can also become a substitute for checking what the paper actually says. Haoming Lin and colleagues investigated this tension by observing postgraduate students during AI-assisted academic reading. Their focus was the process: which resources students consulted, how they moved between them and why they chose a particular strategy. This offers a different kind of evidence from a study comparing final scores, and it requires a correspondingly careful interpretation.

Fifty-three master's students in English education or applied linguistics at a university in eastern China participated. They were recruited through snowball sampling and already had experience with academic reading and generative AI. Each selected an empirical article, wrote an initial critical review and then revised it with access to a Coze-based tool using DeepSeek R 1. The tool supplied critiques across multiple aspects of the paper and an embedded chatbot, while the original article remained available. Students could decide how to use those resources.

Researchers analyzed screen recordings of the reading and revision activity, then used reflective interviews to explore learners' explanations of their choices. Sequence analysis identified three recurring patterns. One relied mainly on generated critiques with little return to the original article. Another involved repeated comparison between the generated material and the paper. A third included more direct chatbot dialogue, particularly when students were considering whether evidence adequately supported an argument. These patterns describe activities across tasks and should not be treated as fixed categories of learner ability.

Interviews suggested that perceived difficulty shaped the strategy selected. For apparently straightforward aspects, some students focused on recalling content or improving their wording. More challenging issues prompted comparison, verification or dialogue. However, more dialogue did not always mean more direct engagement with the source: students could move between two AI-generated resources while rarely reopening the paper. The result illustrates why a sophisticated-looking discussion can still leave an evidential gap if the model's interpretation is not checked against methods, results and limitations in the original research.

The study was observational and had no non-AI control group or separate experimental comparison of the tool's functions. Screen behavior indicates visible actions rather than directly revealing comprehension, and retrospective accounts may not capture every reason for a choice. The sample came from one discipline cluster and institution and was predominantly female. The study did not establish a causal improvement in critical-reading skill or test whether strategies transferred independently to a new article. Its value is a detailed description of how learners distribute attention and effort among available resources.

For AIEDHK, research-reading activities could require a short evidence trail: the AI's claim, the relevant passage or table in the paper, and the reader's reasoned judgment about the match. A learner could also identify one claim that remains uncertain and explain what additional information would resolve it. Later review of a new paper without the tool would help test transfer. These are proposed extensions beyond the study. The practical lesson is to keep the primary source active in the workflow and evaluate the reader's justification, rather than assume that more generated commentary or a longer chatbot conversation represents deeper critical engagement.