Mathematics Assessment

Nonstandard English and the Automated Scoring of Open-Ended Math Problems

Abubakir Siedahmed, Jaclyn Ocumpaugh, Zelda Ferris, Dinesh Kodwani, Neil Heffernan, Eamon Worden

Proceedings of the 18th International Conference on Educational Data Mining

A study of 394 authentic mathematics responses examined how GPT-4o scores language associated with African American Vernacular English. Changing one targeted dialect feature did not significantly alter scores, while changing the grading prompt did. The latter contrast also changed rubric wording, so it cannot cleanly establish that a dialect instruction removed bias. The study makes a useful case for testing content scoring carefully, with language variation and mathematical reasoning evaluated separately.