← Volver a noticias de investigación
A diverse computer science class uses a learning dashboard to record self-assessments and discuss algorithms with a teacher and an AI dialogue panel
Artículo de revistaPeer-reviewed study202611 jul 2026· 3 min

An eliciting LLM dashboard prompted more reflective dialogue and showed a trend toward stronger learning-judgment calibration

Laura Graf, Patrick Bassner, Maximilian Anzinger, Felix Dietrich, Stephan Krusche, Oleksandra Poquet

Education and Information Technologies

Resumen de 500 palabras

A diverse computer science class uses a learning dashboard to record self-assessments and discuss algorithms with a teacher and an AI dialogue panel

Graf and colleagues rethink a learning dashboard as an interactive space rather than a static display. Their July 2026 Education and Information Technologies article reports a five-week exploratory case study in an introductory algorithms and data-structures course at a European university. Thirty volunteer computer-science students were compensated and randomly assigned to one of three conditions: a dashboard without a pedagogical agent, an agent that mainly told students information, or an agent designed to elicit reflection through questions. The small sample makes the work exploratory despite the randomized assignment.

The dashboard combined learning analytics with a GPT-4o pedagogical agent and a Judgment of Learning prompt. Students first estimated their own understanding; the interface then revealed a system estimate derived from course activity and performance. The agent could use tools to retrieve exercises, scores, timestamps, competency information and lecture slides. Its ReAct-style process could issue several tool requests for one message. The telling and eliciting conditions therefore differed principally in dialogue strategy, not simply in whether an LLM or analytics data were present.

Reflective messages appeared more often in the eliciting condition than in the telling condition. The reported two-proportion test was z = 2.47, p = .013. This supports a bounded interaction claim: asking learners to explain and inspect their thinking changed the observed dialogue. It does not establish that students acquired more algorithmic knowledge. Static analytics alone were not highly salient to many participants; 70 percent said they did not actively attend to the dashboard visualizations while making their self-rating.

The eliciting condition recorded 104 learning judgments. Those judgments correlated with confidence, progress and system-estimated mastery, with the mastery relationship reaching r = .482, p < .001, in the third interval. The telling condition did not show significant overall relationships on the same measures. Curiosity was also important: 83 percent of participants said wanting to see the system rating motivated them to submit a judgment. These patterns suggest developing calibration, but multiple observations came from a small number of learners and should not be treated as independent proof of improvement.

System quality was imperfect. In an audit of 284 randomly sampled agent responses, 13 percent were rated faulty and 37 percent very useful; during the first two weeks, 37 percent were faulty. The study lasted only five weeks, involved one computer-science course and used a self-selected, paid sample. It did not directly measure changed study behavior, course examination gains, delayed retention or transfer. The findings therefore support more testing of elicitation and calibration, not a claim that an LLM dashboard improves learning outcomes.

For Hong Kong universities and secondary schools, a useful replication could ask learners to explain a mastery judgment before any system score is shown, then test whether that explanation predicts and improves later unaided work. The dashboard should be evaluated in Cantonese, English and Putonghua, with curriculum-grounded retrieval, teacher review and visible correction routes for faulty responses. Engagement, calibration and achievement should be reported separately. The design is promising because it turns analytics into a conversation, but educational value depends on accurate support and evidence beyond the conversation itself.

Artículos relacionados

Illustrative photograph of two educators examining a simple prediction baseline and uneven error bars on a seminar-room display.
Herramienta / conjunto de datos2026
Herramienta / conjunto de datos 245

Preprint: Tutoring logs add little to simple score predictions in one course

Zeyu Dong, Mikael Kwame Kuada, Asa Holley Buchakjian-Tweedy, Ibrahem Aljabea

arXiv preprint

A retrospective analysis compares tutoring-interaction features with constant score predictions. Small overall gains and uneven errors show why a support model needs more than a favorable average accuracy figure.

preprintlearning analyticsprediction baselines
Leer resumen de 500 palabras →
Un estudiante universitario ensaya una presentación sobre el uso del agua mientras un instructor y un compañero comparan las anotaciones de la rúbrica con una grabación en una tableta.
Artículo de congreso2026
Artículo de congreso 230

Demostración de congreso: AICoFe combina evaluación entre pares, autoevaluación y feedback de IA revisado por docentes

Alvaro Becerra, Alejandra Palma, Ruth Cobos, Julian Fierrez

European Conference on Technology Enhanced Learning (EC-TEL 2026), demonstration paper

El documento de demostración de AICoFe describe un flujo de trabajo que conecta grabaciones de presentaciones, valoraciones de rúbricas y comentarios de IA seleccionados por los profesores. En su despliegue universitario participaron 65 estudiantes, con valoraciones positivas de satisfacción. La evidencia reportada se refiere a la utilidad y usabilidad percibidas, lo que deja abiertos a la investigación los avances en el aprendizaje y la carga de trabajo a largo plazo.

evaluación por paresautoevaluaciónsupervisión docente
Leer resumen de 500 palabras →