For two years, students at 18 middle schools in Tennessee were offered Khanmigo, Khan Academy's generative AI tutor, to use during the school day. Access was close to universal. Engagement was not. Researchers found that students mostly ignored the tool, and that when they did open it the requests frequently had nothing to do with the lesson: write an essay on the history of pizza, tell a joke, or — repeatedly — hand over the correct answer to a system whose system prompt explicitly forbade it from doing that. One finding went the other way. Students who used Khanmigo moved through math faster than a control group with no AI at all.
The design was careful. Khanmigo was instructed not to produce the answer outright but to walk the student toward it in steps, which is the pedagogically defensible version of an AI tutor and the one every vendor in this category describes. The children tested the perimeter of that rule rather than working inside it. Instead of using the tool for help with the remote lessons, they sent it unrelated messages and probed where its limits were, and many of them tried to make it simply produce the answers.
Anyone who has spent time around twelve-year-olds will find none of this surprising, which is part of what makes the study useful. The researchers' own conclusion is that near-universal access coexisted with low engagement, and that human attention may be the decisive condition that lets personalized learning work at all.
Philip Oreopoulos of the University of Toronto, a co-author, told Chalkbeat that if a motivated student wants to use the technology as a real tutor, it is ready for that. The problem is that most students use it differently. Khan Academy chief executive Sal Khan told the same outlet that the study made the importance of motivation unusually visible, though teachers were not surprised by it.
The broader picture around this result is not flattering to the category. Teachers report students struggling to read long documents, to complete homework without assistants, and to sit in-person exams. Early claims that OpenAI's ChatGPT could improve learning outcomes are now being withdrawn in volume. Meanwhile companies and school systems keep embedding the technology into textbooks, curricula, and the rest of school life.
The findings line up with other work. Earlier this month Stanford researchers concluded that AI is not a universal solution and produces no result without human guidance. Effectiveness depends not only on the quality of the software but on how it is placed inside the learning process — a teacher in the classroom, a parent at home. The review of that Stanford work notes that the most visible results so far come from AI tools built for the tutors and teachers, not for the students.
The math gain is the number that will be quoted in every sales deck built from this study, and it is the number that deserves the most scrutiny. A study whose dominant finding is non-use still reports faster math progress against a control group, which means the effect is being generated by whatever fraction of students actually engaged. That is a real result and it is not the result the product is sold on. The pitch is that every student gets a tutor; the evidence here is that the students who would have sought help anyway now get better help, faster. Those are different products with the same price tag.
The question the write-up leaves open is the one a district superintendent should be asking: how the gain distributes. Two years, 18 schools, and the headline deliverable is a single subject-level comparison against a control group, with no account of whether the students who gained were the ones the intervention was meant to reach. If the faster math came from the engaged minority, the intervention is an amplifier of existing motivation, and amplifiers widen gaps rather than close them. Nothing in the reporting rules that out, and nothing rules it in — which is itself a finding about how this field measures itself.
Oreopoulos put the underlying mechanism plainly: the same technology can serve as a good tutor or as a way for a student to make life easier, and less effort means less learning. That is the tension no amount of model improvement resolves. A tutor that refuses to give the answer is working correctly and will always be less attractive to the student than one that does, and the people buying these tools are districts measuring access, not children measuring effort.