Repair, Reflection, and Trust: The Behaviors Confidence Cannot Show
AI literacy becomes easiest to observe when an AI answer stops being easy to accept. This chapter follows the failure, reflection, and trust decisions that occur after a learner detects a mismatch, and uses them to identify evidence that a confidence rating cannot provide. The aim is practical: to recognise a repair loop and design a learning task in which a person must diagnose, justify, revise, or abandon AI use.
A smooth answer can conceal the learner’s reasoning because the system appears to have completed the work. A failed answer creates a visible choice. The learner must decide whether the problem lies in the request, the context, the system, or their own interpretation; determine what evidence can distinguish those possibilities; and choose what to do next. Failure is not automatically educational, but it creates a point at which agency and judgement can be examined.
When a fluent answer breaks down
A breakdown may be obvious: a calculation does not balance, a citation cannot be found, a piece of code does not run, or an explanation contradicts a known condition. It may also be quieter: the answer is generic, the recommendation does not fit the user’s purpose, or a confident paragraph leaves out a factor that changes the conclusion. In every case, the visible symptom is the same—an output that does not adequately serve the task—but the cause is not necessarily the same.
A learner can respond to that symptom in several ways. They may simply ask the system to try again. They may add constraints or more context. They may identify a missing assumption, correct a term, compare the output with a taught method, or start a new conversation. They may decide that the task is not suitable for AI and return to another source or to their own work. These actions are not interchangeable. A more specific prompt can improve the answer, but it can also hide the fact that the learner never checked the underlying claim. A reset can remove a local problem, but it does not by itself show that the learner knows why the problem occurred.
The student–ChatGPT study provides a concrete example. In a chemical-engineering buffer-tank derivation, a student interrogated the system’s reasoning, noticed that it had omitted a density term, and supplied the corrected governing equation. The student used domain knowledge as a test of the output rather than treating the fluent derivation as complete. The exchange then became a form of auditing: the learner examined the system’s work, identified a failure, and directed a correction.
The researchers describe three recurring repair moves. Syntax correction treats the system almost like a compiler: the learner specifies a form or instruction the answer did not follow. Disconfirmation and reset rejects the answer—“that’s wrong”—and re-presents the problem or starts again. Prompt escalation uses a more authoritative instruction in an attempt to make the system comply. Each move can be useful in the right circumstances, but none is automatically proof of understanding.