A Learning Partner, Not an Answer Engine
The question for student assessment is not whether generative AI is present. It is what role the system plays, what intellectual work remains with the student, and what evidence can show that the intended learning occurred. This chapter compares four roles—direct-answer assistant, pedagogically configured tutor, feedback partner, and assessment-production tool—using the same criteria: student agency, cognitive sequence, credibility of evidence, and the controls required when a score matters.
The role of the tool changes the learning mechanism
A direct-answer chatbot is the simplest arrangement. The student presents a task, the system supplies a solution, and the student submits or adapts the result. This can be useful for overcoming a temporary obstacle, but it is a poor learning design when the obstacle is the capability being assessed. The system has performed the diagnosis, selected a method, or drafted the argument before the student has had to do those things. The final output may improve while the evidence of independent understanding becomes thinner.
The first comparison, then, is not between brands or model sizes. It is between the work the student must do and the work the system does for them. Ask four questions: who forms the initial approach, who evaluates whether it is sound, who adapts it when conditions change, and what remains when the tool is removed? A system can be present in all four cases, yet the answers can differ completely. If it proposes the method, checks its own answer, handles the changed condition, and leaves the student only to format the result, the assessment is largely observing system-assisted production. If the student proposes, challenges, verifies, and adapts, the same technology may be supporting human competence.
A pedagogically configured tutor changes the sequence. Instead of treating the student’s request as an instruction to produce the answer, it can ask a question, offer a hint, require a prediction, or prompt the student to justify a choice. It can support a learner’s next step while leaving the central operation in the learner’s hands. The distinction is not cosmetic. A friendly chatbot that routinely gives complete solutions remains an answer engine; a terse system that consistently elicits reasoning may be a better scaffold.
The evidence reviewed by the OECD is promising for some configured interventions. In a Harvard undergraduate physics trial, a GPT-based tutor was configured to implement active-learning practices online and was compared with in-person active learning. The reported result favoured the AI-tutoring condition in learning, time, motivation, and engagement. In a Nigeria after-school programme, students worked in pairs, received teacher guidance on how to use Copilot, and achieved a positive effect of 0.31 standard deviations. These are not tests of an abstract technology. They are tests of specific combinations of an interface, a teaching method, a social arrangement, and a population.
The configured tutor is therefore not a magic replacement for a teacher. Its behaviour has to remain aligned with the learning goal. A tutoring system that asks a relevant question at the beginning but gives the complete answer after the first sign of difficulty may still train dependence. A system that refuses to answer without adapting its support may frustrate a novice or leave a misconception uncorrected. Good scaffolding is conditional: it provides enough support to keep the learner moving, but not so much that the learner no longer has to perform the target operation. The evidence so far supports that design direction more strongly than it supports any particular commercial label.
That qualification matters because the comparison is not always favourable to GenAI. In the Turkish mathematics experiment discussed earlier, a tutoring version avoided direct answers and performed about as well as self-study on the later closed-book test, while a general-purpose chatbot group performed worse than the no-GenAI group.