Students preferred AI design feedback in a small study, while improvements in design quality were not established

Feedback can feel encouraging and well organized without necessarily improving the idea it addresses. Tanja Katharina Schmitt-Fumian, Selina Tauscher and Katja Thoring examined how design students received comments generated by GPT-4 compared with comments written by educators. Presented at ICED 25, the study offers a focused look at feedback reception during ideation. Its strongest contribution is understanding what students valued in the comments, while the effect of those comments on subsequent design quality remains a different question.

Twenty-five students developed an idea for a startup intended to reduce academic stress. The task began with rapid idea generation and a short refinement period. Six human educators and GPT-4 then provided textual feedback using a common structure addressing strengths, areas for improvement and further development. Each student received both sources of feedback. Presentation order was randomized, with thirteen seeing human feedback first and twelve seeing AI feedback first. Students completed a survey and a recorded interview about their experience of the comments.

The AI setup was not an untouched default chatbot. Before the main comparison, the researchers iteratively refined its prompts to produce appropriate feedback and make the source less obvious to readers. The intended comment length was 150 words, and the feedback was structured around the same core categories for humans and AI. This preparation is part of the intervention. Any claim about speed or scalability should therefore distinguish rapid generation of an individual response from the earlier work needed to define, calibrate and review a suitable feedback approach.

Students generally gave more favorable ratings to AI comments across the measured dimensions, including supportiveness, inspiration, coherence and organization. Interviews helped explain the appeal of a consistently positive tone and explicit structure. These results suggest features that educators might also adopt in their own feedback. They do not demonstrate that agreement, encouragement or perceived correctness reliably identifies a better critique. A challenging comment may be less comfortable yet reveal a genuine weakness, and an agreeable AI response may be easy to accept without improving the underlying design.

The study was small and conducted outside a normal graded project. Educators and students did not know one another, and the analysis focused on textual feedback and student perceptions. It did not establish changes in independently rated design quality, transfer to a later ideation task or a sustained increase in creative capability. The authors also question how well AI can challenge assumptions rather than soften its response. Those limits make this exploratory evidence about feedback reception, not a basis for replacing the contextual judgment of a design teacher.

For AIEDHK, a useful next step would pair perception measures with revision evidence. Students could identify which comment prompted a change, explain the reason for accepting or rejecting it, and submit the revised idea for independent assessment. A teacher could examine whether the feedback exposed a weak assumption or merely improved presentation. These are proposed extensions beyond the study. The practical opportunity is to combine clear, constructive feedback with substantive critique, and to evaluate whether that combination helps learners make stronger design decisions rather than only feel more supported.