Computer-assisted learning in the real world: How Khan Academy influences student math learning

Large educational platforms can accumulate extensive usage records, but the scale of a dataset does not by itself establish what caused learning. A PNAS study of Khan Academy mathematics use examines that challenge through approximately 490,000 observations from 214,000 students, 11,000 teachers, and 1,000 schools. It reports positive relationships between use and mathematics progress, while making clear that its identification strategy depends on assumptions that a randomized trial would address differently.

Taryn Eames and colleagues studied three United States school years, from 2020–2021 through 2022–2023, using mathematics assessment and platform data. The setting included grades three to eight in MAP Accelerator and captured Khan Academy activity beyond that product alone. This is evidence about computer-assisted mathematics learning, including structured practice and skill progression. The intervention was not a generative-AI chatbot, and the results cannot be transferred directly to conversational tutoring systems.

Students who voluntarily use a platform more may also differ in motivation, support, or prior skill. To reduce that selection problem, the researchers examined variation in the average use of other students taught by the same teacher, leaving each focal student's own use out of that measure. Models included student, teacher, school, and year-by-grade effects, with additional robustness analyses. The design attempts to capture changes in exposure associated with classroom practice, rather than treating a simple individual usage correlation as causal proof.

Even with these controls, time-varying influences can remain. A teacher might alter mathematics instruction in ways that increase both platform use and achievement, or a school might introduce other support at the same time. The study cannot guarantee that all such changes are removed. Its estimates are therefore best understood as carefully modeled observational evidence, not the result of randomly assigning children to different doses of Khan Academy.

Average annual use was only 6.6 hours, approximately 11 minutes a week, associated in the model with a gain of 0.031 standard deviations in mathematics. Usage was highly uneven: about 24 percent of students recorded no use, and roughly 65 percent averaged under ten minutes a week. Only about one in ten reached the recommended 18 hours a year. These patterns suggest that access to software and sustained classroom implementation are different achievements.

The study also reports variation in estimated benefits. Students with stronger prior achievement appeared to gain more, and skill progression per hour differed across teachers' classes. Those findings invite closer examination of how practice is supported and which learners need additional help. They do not identify a specific teacher behavior as the cause. Similarly, estimates for greater weekly exposure are model projections, not results from a trial assigning that schedule; the data do not show exactly which classroom activities platform time replaced.

For schools, the useful question is how a practice tool fits within teaching, assessment, and support for learners who struggle. The dataset is proprietary, although replication code is available. Author affiliations and a subsequent correction disclose Khan Academy employment or unpaid advisory connections; the correction concerns competing interests rather than numerical results. The paper offers substantial real-world evidence for an established learning platform, with observational uncertainty and implementation differences that should remain visible in decisions about adoption.