The Access Illusion: Why an AI Tutor Is Not a Tutor Yet
This series examines AI tutoring in real K–12 learning environments, where the central question is not whether a language model can produce a plausible explanation, but whether students use that explanation in ways that improve learning. The practical judgment is deceptively simple: can a school turn scalable access into sustained, productive work and then into knowledge that survives without the tool? The evidence suggests that each transition has its own bottleneck.
The promise of an individual tutor at scale
The appeal of AI tutoring begins with a real constraint in education. A skilled human tutor can notice a misconception, adjust the next question, explain a missed step, and encourage another attempt. That attention is valuable precisely because it is individualized, but individualized attention is expensive and difficult to provide consistently to every student who needs it. Computer-assisted learning has long reduced some of the cost by personalizing pace and practice. Large language models add the possibility of flexible, natural-language feedback rather than only prewritten hints.
That possibility changes the economics of instructional support, but it does not remove the behavioral problem. A school can purchase licenses, create accounts, schedule time, and place a help button beside a problem. None of those actions guarantees that a student will return to the platform, persist through difficulty, ask for help, or think through the answer rather than seek a shortcut. The technology may be capable of tutoring in principle while the student experiences almost no tutoring in practice.
This distinction matters because the students targeted by remedial and supplemental instruction are not a random sample of enthusiastic technology users. In the two elementary literacy trials reported by Robinson, Gormley, Ribeiro, and Loeb, students were given scheduled time and access to an AI literacy platform, yet many never used it and average use was measured in only a few minutes per week. In the two-year Khanmigo school experiment, 96% of activated students messaged the tutor at least once, but the typical student still sent no message on most practice days and on most exercise sessions containing a mistake. Universal first contact and regular, substantive tutoring were very different achievements.
The title’s claim—that engagement, not access, is the binding constraint—should therefore be read as a claim about the learning process, not as a dismissal of access. Access is necessary. A student without a device, account, protected time, or usable connection cannot benefit from an AI tutor. But once basic access is supplied, the next question becomes behavioral: what does the learning environment make easy, expected, salient, and worth the effort?
The missing links between access and learning
It helps to separate five stages that are often collapsed into the single word “access.” First comes availability: the student is technically allowed to use the system. Next comes actual use: they open it and spend time on the task. Then comes productive practice: they attempt problems, read, retrieve, revise, or otherwise do work that could build knowledge. Tutor interaction is a more specific form of engagement: the student invokes help, responds to a question, or explains a step. Finally comes durable learning, shown when the student can perform later, preferably without relying on the same support.
These stages can diverge. A student may log in but do almost no work. They may complete many items by guessing. They may click a hint without processing it. They may ask the tutor for help but provide only a bare answer. Or they may perform well inside a platform and then fail to retain the idea a week later. Conversely, a slower interaction can be educationally useful if it helps a student diagnose an error and make the next attempt independently. The relevant outcome is not maximum activity or minimum time by itself; it is the quality of the work produced under the available constraints.
The three experiments in the source set illuminate different links in this chain. The literacy studies ask whether a human tutor focused on motivation, participation, troubleshooting, and reflection can increase engagement with an AI platform. The tutors did not provide the reading instruction; the platform did. This makes the studies useful for identifying the contribution of relational and organizational support, while their reading-achievement results show whether the added activity reached a meaningful dose.
The Khanmigo study asks a different school-scale question. In 18 Tennessee middle schools, the AI tutor was embedded in existing remedial mathematics blocks, alongside individualized Khan Academy practice. The comparison was ordinary remediation, which could include teacher instruction, worksheets, and other computer-assisted platforms. Assignment to the program increased structured practice and produced math-achievement gains, especially as implementation matured. But the treatment changed several things at once: the content pathway, the protected practice routine, and the availability of Khanmigo. Its results therefore identify the combined program, not a clean, isolated effect of conversation with the AI.
The NUMI experiment moves closer to that value-added question. More than 6,000 middle-school students received the same basic computer-assisted learning environment, while random assignment varied AI support, a mastery progression rule, and the topic practiced. The AI version supplied guard-railed hints and walkthroughs rather than final answers. A delayed assessment one week later included both practiced and unpracticed material. This design makes it possible to ask not only whether students completed more work, but what AI changed when a student encountered an error and whether the change appeared in later performance.
What the evidence can—and cannot—tell us
Taken together, the studies support a disciplined conclusion rather than a universal verdict. They make it difficult to defend the idea that giving students an AI tutor is enough. In the literacy trials, human support increased platform use and content engagement, but the absolute gains remained small and did not improve reading achievement. In the Khanmigo deployment, mandatory supervised time solved much more of the practice problem than voluntary access usually does, while substantive dialogue remained thin. In NUMI, embedding support at the point of difficulty produced a slower but more accurate pattern of recovery after mistakes and a modest, localized signal on a short delayed test.
That pattern also clarifies what “engagement” must mean. It is not simply the percentage of students who ever touched the system. It includes frequency, persistence, time, effort, the kind of help requested, and whether the student performs reasoning rather than merely selecting a prompt or repeating an answer. Nor is engagement the final outcome. A platform can generate impressive activity metrics that do not transfer to independent performance. NUMI’s mastery rule sharply increased the number of students who achieved a short streak of correct answers, but the rule alone did not produce a clear delayed-learning gain. Platform-defined success and durable understanding are related questions, not synonyms.
The evidence also resists a false choice between human and artificial tutors. Human support can supply motivation, accountability, care, and help with the practical friction of participation. AI can provide individualized practice and structured explanations at a scale that human labor cannot easily match. The relevant deployment may sit somewhere between those endpoints. But each design has a cost. Human support requires staffing, training, and coordination. More elaborate AI guidance can consume time that would otherwise be spent on later material. A low-cost license is not a complete substitute for high-dosage tutoring, and a positive usage effect is not proof that a school has achieved enough learning.
For the same reason, randomized evidence must be read with its comparator and denominator attached. A program-level estimate from a two-year remedial deployment is not an AI-only estimate. A message-level pattern among activated students is descriptive, not a randomized effect of a particular interface. A post-mistake comparison is informative about mechanism but can be affected by the fact that AI changes who makes a mistake and who continues to the next question. A one-week test with four items can reveal a promising signal without establishing long-run transfer, course performance, or state-test effects.
The practical question that follows is therefore narrower and more useful than “Does AI tutoring work?” Where does a particular learning environment lose students, and what intervention is supposed to repair that loss? If few students log in, scheduling and human facilitation may matter more than model upgrades. If students practice but never seek help, the interface may need better defaults or more timely prompts. If help is invoked but turns into answer copying, the tutor needs guardrails and tasks that require independent reasoning. If platform completion rises without delayed learning, the school needs a better measure of understanding rather than another completion target.
The first hard test is not whether an AI tutor is available. It is whether students use it often enough, and thoughtfully enough, for its instructional capabilities to become part of their learning. The next question is where that first failure occurs—and whether human support can move students past it without merely creating the appearance of engagement.