Overview

Scaling a conversational assessment is not mainly a question of minutes per student; it is a system problem involving question design, marking interfaces, staffing, technology, policy, accommodations, and fair access. This chapter shows how preparation and role division move work before the live encounter: the reported case used groups of five or six, 30-minute sessions, two pools of 30 questions across difficulty tiers, and separate lead-instructor, co-instructor, and teaching-assistant responsibilities to examine 58 students across ten sessions. It also treats AI boundaries, disclosure, randomisation, public performance, extra-time arrangements, subgroup access, and privacy as validity conditions. The case demonstrates moderate-class feasibility, not automatic scale. Transfer should remain conditional on comparable tasks, transparent rubrics, resources, student experience, marker consistency, subgroup outcomes, workload, and learning evidence.

Learning outcomes

Scaling interactive assessment moves labour into preparation, staffing, calibration, technology checks, accommodations, and debriefing rather than eliminating it.
AI-use policies are valid when tied to the capability being assessed and accompanied by disclosure, verification, and fair conditions.
A conversational exam should transfer only after pilots test comparable questions, marker consistency, subgroup outcomes, workload, and learning evidence.

Keywords

Generative AIScalabilityFairnessAccessibilityPrivacyStaffing

Making Human-Centred Assessment Scale

Interactive assessment does not become scalable merely because each student receives fewer minutes. The real question is how an institution can preserve credible evidence of human understanding while coordinating preparation, staff attention, technology, policy, and fair access. This chapter treats the conversational exam as a system of responsibilities rather than a conversation to copy. That view helps educators estimate its actual cost, identify its failure points, and decide what must change before transferring it to another course or student population.

The work moves; it does not disappear

An oral examination is often described as expensive because an instructor must speak with every student. The conversational-exam case changes the arithmetic by changing the unit of interaction. Five or six students meet together, take turns answering, and observe one another. A session lasts 30 minutes. Across ten such sessions, the authors examined 58 students over two days. This is a meaningful demonstration of feasibility for a moderately sized class, but it is not evidence that the format scales without additional people or preparation.

The time saved during the live encounter is purchased before it begins. The instructors built two question pools, with 30 questions in each difficulty tier, and tied the prompts to course learning objectives. Each question had an expected solution, a rubric, a hint, an elevated concept check, and a simplified fallback. The team also had to prepare the schedule, test screen sharing and notebook access, arrange the physical and digital environment, brief staff, and practise explaining the format to students. This preparation is not administrative decoration. It is what allows the live exchange to generate comparable evidence rather than a different improvised exam for every person.

The marking sheet is the most important interface in this system. It carries the question, expected solution, alternate paths, rubric criteria, and space for quick notes. Without it, the lead examiner would have to question, remember what counts as evidence, notice technical problems, decide on a follow-up, and record the result at the same time. With it, the co-instructor can observe the student’s performance against pre-established criteria while the lead instructor concentrates on the reasoning. The teaching assistant manages the timer, screen display, and technical interruptions. Each role removes a competing demand from the others.

The division of labour also clarifies ownership. The lead instructor owns the intellectual interaction: what to ask, when to probe, and how to interpret a response. The co-instructor owns contemporaneous evidence capture: what the student did, said, corrected, or required. The teaching assistant owns the conditions that let the interaction proceed: time, display, and technical troubleshooting. The institution owns the surrounding guarantees—staffing, training, accessible arrangements, policy, and the consequences of the score. If one person is expected to carry all of these responsibilities, the system’s reliability depends on memory and improvisation at exactly the moment when both are under pressure.

The workflow reduces cognitive load by externalising decisions.

Published by DeepMox

FAQ

Why does scaling interactive assessment not reduce the work of assessment?

It shifts live time into question design, rubrics, staff coordination, technology checks, accommodations, and review rather than eliminating that work.

Why can randomised questions create fairness problems?

They can create fairness problems when prompts elicit unequal capabilities or difficulty, causing question variance to be attributed to students.

What should an institution pilot before transferring a conversational exam?

It should pilot student experience, marker consistency, subgroup outcomes, workload, accommodations, and performance after assistance is removed before transferring the format.

Previous10 minGenAI Conversational Exam

Next10 minGenAI as a Learning Partner

Scaling Human-Centred GenAI Assessment

Making Human-Centred Assessment Scale

The work moves; it does not disappear

FAQ