#AI evaluation
6 논문
Norah Almousa, Shayan Peyghambari Oskoui, Raquel Coelho, Gayle Rogers, Xiang Lorraine Li, Diane Litman
Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Full Papers
FeedType은 대학 글쓰기 강좌, 여섯 언어 모델, 세 가지 프롬프트 전략에서 피드백의 초점을 비교한다. 모델은 여러 범주를 다뤘지만 분포와 조정 양상이 교사와 완전히 일치하지는 않았다. 이 연구는 피드백 행동을 점검할 벤치마크를 제공하며 학생의 글쓰기 향상을 입증하지는 않는다.
Tanya Nazaretsky, Hagit Gabbay, Tanja Käser
Computers and Education: Artificial Intelligence
Researchers compared human and AI feedback on 979 responses from six university STEM courses, then examined ratings from 472 students. Expert analysis found broadly similar pedagogical quality, while students’ judgments were associated with their views of the source’s credibility. The study did not measure feedback uptake or learning gains. For educators, it raises a practical question: how can students justify an evaluation of advice before deciding whether to use it?
Abubakir Siedahmed, Jaclyn Ocumpaugh, Zelda Ferris, Dinesh Kodwani, Neil Heffernan, Eamon Worden
Proceedings of the 18th International Conference on Educational Data Mining
A study of 394 authentic mathematics responses examined how GPT-4o scores language associated with African American Vernacular English. Changing one targeted dialect feature did not significantly alter scores, while changing the grading prompt did. The latter contrast also changed rubric wording, so it cannot cleanly establish that a dialect instruction removed bias. The study makes a useful case for testing content scoring carefully, with language variation and mathematical reasoning evaluated separately.
Manooshree Patel, Rayna Bhattacharyya, Thomas Lu, Arnav Mehta, Niels Voss, Narges Norouzi, Gireeja Ranade
Proceedings of the AAAI Conference on Artificial Intelligence
LeanTutor connects a language model to the Lean theorem prover to support mathematical proof feedback. Its technical evaluation shows promising hints on a small selected benchmark, but also exposes a crucial boundary: a verifier can check the formal proof only after the learner's words have been translated correctly. The study did not test classroom learning, and feedback evaluation excluded failed translations. It is an early system design with explicit limitations, not a validated replacement for mathematics instruction.
Anthropic
AI Product and Learning Report
Anthropic released Claude Sonnet 5.5 on September 28, positioning it as a faster, lower-cost complement to Opus 5.5 for well-defined work. The company reports improvements in coding, documents, and communication, with efficiency gains dependent on the task. OpenAI's recently released GPT-6 Sol and Luna and Gemini's course-connected support provide context. Educational teams should evaluate the work they actually need, including the review required before a draft becomes usable.
Anthropic
AI Product and Learning Report
Anthropic released Claude Opus 5.5 on September 22, emphasizing stronger coding and knowledge work, clearer communication, and lower operating costs. The same day's GPT-6 Sol and Luna launch broadened the choice of affordable models, while Gemini's established Guided Learning tools offered a contrasting educational purpose. For universities, the opportunity is to compare research workflows carefully while keeping model performance, completed artifacts, and student learning as separate outcomes.