AI-supported lesson-design teams maintained immediate scores after withdrawal, but durable learning was not tested

When an AI system joins a design group, the question is not only whether the group finishes a better plan. It is also whether participants learn from the discussion and continue contributing their own ideas. Daniele Agostini and colleagues explored this tension in collaborative lesson design, asking teachers-in-training to treat ChatGPT as a contributor whose suggestions could be questioned. Their study followed both individual knowledge checks and group-produced artifacts, providing a view of collaboration that extends beyond the final written lesson.

The study involved 102 postgraduate teachers-in-training organized into 21 groups. In a four-hour session, groups completed two connected tasks: first select and justify inclusive educational technologies, then develop a detailed teaching intervention. Groups were allocated to opposite sequences. One used ChatGPT 3.5 in the first task and worked without it in the second; the other began without AI and gained access for the second task. Complete data for all three individual knowledge checks were available for 82 participants, with 38 and 44 in the two sequences.

The first sequence improved on the knowledge check after the AI-supported task, particularly on technology-related items. Its scores did not show a statistically detectable decline in the immediately following task without AI. The other sequence showed its clearer improvement after AI became available. The authors describe this pattern as sustained performance after withdrawal within the study timeframe. That is a narrower conclusion than proving durable retention: the same session continued, and a nonsignificant decline does not establish equivalence or guarantee that knowledge will persist later.

Group artifacts and written reflections added a more complicated picture. Participants described AI as useful for organizing work and producing structured suggestions, but some also reported less spontaneous exchange of ideas among peers. The issue is therefore not whether AI participation is uniformly helpful or harmful. A tool can make drafting easier while changing who proposes, questions and develops ideas. The qualitative evidence comes from participants' accounts of their collaboration, rather than a complete independent record of every conversational contribution.

Several methodological limits deserve attention. The individual test contained thirteen mostly recall or understanding items and had low internal-consistency estimates. Only 82 of 102 participants contributed complete test data. The two design tasks were sequential and different, so task demands, practice and carryover complicate comparisons. There was no delayed follow-up and no final common task in which both sequences worked without AI. The results provide an exploratory account of short-term collaboration and score patterns, not a secure estimate of enduring learning or an isolated causal effect of treating AI as a teammate.

For AIEDHK, a useful classroom experiment would preserve a group's initial ideas and require members to discuss which AI suggestions they accepted, revised or rejected. Individual explanations could reveal whether every participant understood the decisions behind the shared artifact. A later common task without assistance, and a stronger knowledge measure, would help test whether any benefit lasts. These are proposed extensions beyond the reported study. The educational priority is a group that continues to reason together, with efficient document production evaluated separately from individual understanding and the quality of peer participation.