The Conversational Exam
The conversational exam is a concrete response to a specific assessment problem: how can an instructor see whether a student understands and can use computational ideas when take-home code can be generated, copied, and tuned until it passes? This chapter reconstructs the format developed in an undergraduate engineering-computations course, focusing on the mechanism that makes evidence visible: students code live, explain what they are doing, and respond to structured follow-up questions. The case is useful not because oral examination is automatically superior, but because it shows what an evidence-rich assessment looks like when designed as a complete workflow.
From copy-pasteable homework to performance in context
The course was not built around an abstract debate about technology. It had a practical history. Students worked with Python, numerical methods, linear algebra, differential equations, and a brief introduction to Fourier analysis. For several years, assignments in Jupyter notebooks and an autograding system had provided immediate feedback and multiple attempts. The arrangement had been popular and, by the authors’ account, successful in ordinary conditions.
The arrival of generative AI changed what those assignments observed. Students began copying assignment questions into a chatbot, pasting generated code into their notebooks, and using the autograder through trial and error to obtain high marks without engaging with the underlying work. The authors report that attendance, satisfaction, and student learning slumped. Their interpretation was an illusion of competence: having the answer close at hand made students feel that they were learning even when they were not developing the capability to reproduce or explain the solution.
A rule against AI could state what students should not do, but it could not make an unsupervised assignment reveal what they actually knew. The authors therefore changed the assessment structure rather than trying to win an enforcement contest. Their conversational exam rests on three linked principles.
The first is authenticity. In professional technical work, people consult documentation, search error messages, collaborate, and increasingly use AI tools. An exam that removes every support may test performance under artificial isolation rather than readiness for the work students will later do. The conversational exam keeps students in a familiar notebook environment with course materials, documentation, and an internet connection. Supervised AI use is possible within stated boundaries. The time limit is still an exam constraint, but the broader work setting is recognisable.
The second is validity through performance rather than prohibition. The assessment does not need to prove that a student never looked at an AI tool. It asks the student to perform the target work in real time and to make their reasoning available to an examiner. Typing code while explaining the approach, responding to a changed condition, debugging a small error, and answering why a method works create several observations that a final file cannot provide. A student may have consulted assistance beforehand, but they still have to demonstrate ownership of the relevant concept at the moment of assessment.
The third is scalability through deliberate organisation. An individual oral exam for nearly 60 students would require days of instructor time. The case instead uses small groups of five or six, with students taking turns while peers observe.