How Fragile Knowledge Exposes the Limits of AI-Powered Teaching

Alpha School scales AI-powered teaching toward a billion children
Fragile knowledge forms when instruction skips genuine questioning
Science exposes AI tutoring's sharpest limits under scrutiny

Fifty campaigns, twenty-seven new ones in one fall, tuition fees of up to seventy-five thousand dollars a year and a public statement from the school's founder that the goal is to reach one billion children. These are the figures of the expansion of Alpha School, the network of private schools that assigns most of the morning instruction to adaptive learning software. In the same country, one in five students is considered absent from the classroom for years and reading and math performance has been declining for a decade. The argument in favor of the method sounds logical on this background. The objection, however, is not about whether the technology works at the grade level but whether what it produces is durable understanding or fragile knowledge.

An Experiment at Billion-Child Scale

A senior learning scientist at Alpha School describes the process of training models as something akin to autonomous vehicle training, where the system learns from thousands of incidents to recognize the exception: the cyclist swaying, the child running unexpectedly down the road. In the classroom, the equivalent of unforeseen incidents is each child's misconceptions and learning difficulties. The company has not released the underlying data behind its claims but an education researcher at the University of Washington saw the math teaching model last year and notes that it resembles other established adaptive learning technologies such as IXL Learning, technologies that have already been publicly studied.

In 2025, a randomized study in an introductory physics course at Harvard University compared students who used an adaptive digital assistant with students who followed active teaching in the classroom. The first group saw an average improvement of more than twice as much. A Harvard senior lecturer and co-author of the study characterizes the Alpha School's approach as better than what exists today in public education but adds that when there is no teacher-student relationship, as is the case in campaigns where the adults supervising are not qualified teachers, the question of the meaning of the process remains open.

The Fragile Knowledge Behind the Numbers

The study's lead author identified the problem more accurately. When teaching and assessment are done under similar conditions, students develop what he called fragile knowledge, i.e. performance that seems strong in this context but is not easily transferred elsewhere. This finding does not negate the positive results of the study but it frames them with a caveat that is often lost in marketing announcements around adaptive teaching.

A 2025 review of 28 studies involving nearly 5,000 elementary and middle school students found a generally positive effect of intelligent AI-based teaching systems. Still, the advantage became negligible when compared with simple active learning systems that did not use artificial intelligence at all. An education policy researcher at Pennsylvania State University agrees that Alpha School's basic assumption is not wrong but points out that good teaching and good tutoring support are not the same thing, as there is no literature documenting that a purely automated model can replicate all the multiple benefits of the classroom.

Socratic Dialogue Versus Memorization

The difference between good and mediocre teaching, even with the same material, is found in a second layer of learning created by social interaction. In the Western tradition, this layer often has a Socratic form, with continuous questions that force the student to adapt his logic to new conditions instead of repeating it as it is. When this process is lacking, the alternative is not necessarily failure but another model: memorization without questions, where performance in standardized tests is high but adaptability to new questions is limited.

Today's AI models replicate the second paradigm not because they mimic any particular educational tradition but because their own training architecture works similarly, on a huge volume of question-and-answer pairs where accuracy is optimized. When a real exception appears, something that has never been seen before in this form, the performance breaks down in a way similar to a student who learned by heart without ever being asked why. In both cases, the moment when someone forces the system to justify their answer in other words, in another context, is missing.

The Limits of Bayesian Updating

A principal investigator at the Harvard Graduate School of Education describes the human mind as something that works partly Bayesian but is in many ways better than Bayesian, capable of identifying exceptions to a pattern of covariance and reviewing the entire mental model rather than just gradually adjusting it. A good teacher does just that when he asks a question that isn't in the textbook, designed to test whether understanding survives outside the context in which it was taught. An AI system, by contrast, updates its response on limited information, essentially in the input and output pair of the specific dataset on which it was trained.

The chief executive of the New Zealand research organization HERA recently documented a related problem in a scientific context. An AI tool designed with an explicit mandate to draw solely from peer-reviewed articles ended up returning mostly secondary content from blogs and general websites because most of the reliable literature remains locked behind subscriptions. The finding explains why the inability to transfer knowledge becomes more pronounced precisely in science, where even a slight discrepancy in data or methodology changes the conclusion and where the model simply hasn't seen the sources that would allow it to distinguish the exception to the rule.

Implications for Schools, Teachers and Companies

The strongest argument in favor of models like Alpha School's is not that they outperform the best possible teacher. Hendrick puts it bluntly, noting that the variation in the quality of teaching within the same school is often greater than the variation between different schools, which in other high-responsibility professions would cause a public outcry. However, this argument is weakened by the same finding in the review of the twenty-eight studies, where the advantage of artificial intelligence over simple active teaching without it proved negligible. An education policy researcher at the Educational Testing Service adds that Alpha School's published measurements do not allow for specific conclusions about exactly which element of the method produces results, as they may simply reflect who attends the school and not the teaching experience itself.

For school principals and educational technology companies, the conclusion is not a rejection of adaptive teaching but a need for design that preserves the Socratic element within automation, not just the evaluation of right and wrong. As Alpha School is now testing versions of its model in public schools in Houston and Springfield, Massachusetts, with student populations much more heterogeneous than its private sector, a company spokeswoman describes the school's learning-science team as something that is constantly looking for holes in the system itself to improve. The question that remains open is no longer whether good teaching can be scaled up but whether it can be scaled without losing exactly the element that made it good.

This article reflects the analytical judgment of The EduTimes Editorial Board and does not constitute policy advice or the official position of any affiliated institution.

References

Coyle, T. (2025) The AI Knowledge Crisis: Why Artificial Intelligence Is Struggling to Learn from Science. Auckland: HERA.
Kestin, G., Miller, K., Klales, A., Milbourne, T. and Ponti, G. (2025) AI Tutoring Outperforms In-Class Active Learning. Scientific Reports, 15, 17458.
Létourneau, A., Deslandes Martineau, M., Charland, P., Karran, J.A., Boasen, J. and Léger, P.M. (2025) A Systematic Review of AI-Driven Intelligent Tutoring Systems (ITS) in K-12 Education. npj Science of Learning, 10(1), 29.
Malkus, N. (2026) Return to Learn Tracker: Chronic Absenteeism 2017-2026. Washington, DC: American Enterprise Institute.
Mineo, L. (2025) Is AI Dulling Our Minds? Cambridge, MA: Harvard Gazette.
Randolph, M. (2026) Alpha School Wants AI to Teach a Billion Kids. Should It? New York: Scientific American.