Designing for Engagement at School Scale
A school deciding whether to deploy AI tutoring is not choosing a model in isolation. It is designing a learning environment that must make several behaviors occur in sequence: students need to show up, return, do meaningful work, seek or accept help when confused, and later perform without the same support. This chapter turns the evidence into a bounded operating approach. It explains which design levers are supported by the experiments, which remain hypotheses, and how a school can measure the chain before treating a promising pilot as a scalable program.
Start with the behavior the school needs
The first design mistake is to define the objective as access. A district can count licenses, accounts, devices, and scheduled minutes with great precision while learning little about whether students are using the tutor. The elementary literacy trials show why. Students were expected to receive two 30-minute sessions each week, but control-group use averaged only 2.18 minutes per week in one district and 5.23 in the other. In the Khanmigo program, nearly every activated student messaged at least once, but the median student messaged in only 17% of sessions containing a mistake. A deployment plan that says “make the AI available” has not yet specified the behavior that should make learning possible.
A more useful objective is staged. At the first stage, the school needs reliable participation: students enter the platform and return across weeks. At the second, it needs productive work: students read, attempt, revise, and complete practice rather than simply remain logged in. At the third, it needs meaningful support: the tutor is used at moments when the student is stuck, and the student responds with enough reasoning for the support to do more than supply reassurance or another click. At the last stage, the school needs independent performance later. Each stage can fail separately, and a design should say which failure it is trying to repair.
The first lever is protected time. The Khanmigo study solved much of the access-to-practice problem by putting individualized work inside existing remedial mathematics blocks rather than offering the tutor as an optional resource. Assignment generated about 30.8 additional minutes of weekly platform practice, while attendance was essentially unchanged across treatment and control. This is a strong reason to treat scheduling as part of an AI intervention, not as background administration. If a school expects students to use a tutor when every session is optional, it is testing motivation and logistics at the same time as the software.
Protected time is not the same as delivered time. Blocks can be shortened, interrupted, repurposed, or lost to ordinary school events. A pilot should record the time that was planned, the time that actually occurred, and the time students spent on the intended activity. Those quantities answer different questions. Planned time measures opportunity. Attendance measures presence. Platform minutes measure recorded exposure. None, by itself, establishes that the student engaged with the instructional content.
The second lever is a credible routine. The literacy tutors did not provide the platform’s reading instruction; they supported motivation, participation, group norms, troubleshooting, and reflection. Their sessions devoted only part of the period to platform work, yet human support increased usage and stories completed. That result suggests that a person can change the meaning of the session: a task that competes with distractions becomes a shared, expected activity with someone available to help a student remain in it.
But the same experiment shows why routine should not be sold as a guaranteed solution. District A and District B used different settings and tutor types, and treatment effects varied across sites.