A museum science program improved measured machine-learning knowledge, while some concepts and access gaps remained unresolved

Learning about machine learning becomes more concrete when young people use it to investigate a scientific question. Gabrielle Rabinowitz and colleagues evaluated a curriculum that connected ML concepts with natural-science data and a research-mentorship pathway at the American Museum of Natural History. The program aimed to make advanced ideas accessible through practical activities while widening participation in a field with uneven access. Its evaluation examined knowledge, confidence and learners' explanations rather than relying only on whether participants enjoyed working with technology.

The four-week curriculum was implemented across two years and the study included 42 high-school-age participants. Recruitment followed the museum mentorship program's existing application routes through prior programs, partner schools and community organizations. Activities combined scientific datasets, interactive tools, unplugged exercises and embodied demonstrations. Learners considered different ML methods, participated in field-science activities and created a decision flowchart for choosing a method appropriate to a research question. Bias and the possible consequences of using a model were also part of the curriculum.

The evaluation combined a 35-item Machine Learning Concept Inventory with measures of self-efficacy and career interest, exit tickets and retrospective interviews. The inventory had undergone preliminary validation, but the authors identified further work needed on its structure and use with different populations. This mix of measures is valuable because operating a tool, explaining a concept and deciding whether ML is appropriate are related but distinct learning outcomes. A single total score can conceal strengths in one area and uncertainty in another.

Overall inventory scores and self-efficacy improved after the program. However, the inventory did not detect significant changes in knowledge of ML bias or the skill of applying ML to scientific research. Exit tickets and interviews contained examples of learners explaining limitations, questioning data quality and considering when a model should be used. The authors interpret these differences as a reason to examine measurement alignment and the difficulty of the targeted concepts. Qualitative examples enrich the picture, but they should not erase nonsignificant results or be treated as proof that every participant mastered those ideas.

The evaluation had no untreated comparison group, so it cannot isolate the curriculum's causal contribution from selection, maturation or other program experiences. The sample was small and drawn from a specific mentorship setting. Subgroup analyses suggested gains among learners from under-represented groups, but some initial knowledge gaps persisted and subgroup sizes were uneven. Long-term effects on understanding, research participation and career interest remained untested. The evidence therefore supports further curriculum development without establishing that the program closes broader educational or demographic opportunity gaps.

For AIEDHK, the program suggests a practical route through locally meaningful scientific data. Learners could classify environmental observations, compare model errors and justify whether the dataset is adequate for a proposed question. Assessment should include explanations of limitations and decisions about when not to use ML, alongside conceptual questions. A later project would help test whether the understanding carries forward. These are proposed extensions beyond the evaluation. The useful contribution is a science-centered curriculum with multiple forms of learning evidence and an explicit recognition that access, confidence and conceptual mastery each require their own assessment.