A Better Measure of AI Literacy
The answer to the central question is straightforward but not simplistic: self-confidence cannot stand alone as a measure of AI literacy. It can tell us whether a learner feels ready to begin, whether a tool feels familiar, or whether they are willing to persist. It cannot show whether they understand the system, choose its role wisely, verify its output, repair a failure, monitor their own thinking, or account for the effects of using it. A better measure keeps confidence as context and looks for evidence in action.
The OECD/European Union AILit Framework supplies the destination: learners should be able to engage with, create with, manage, and shape AI through knowledge, human skills, and responsible attitudes. The student–ChatGPT study supplies a view of practice: learners may use the same system as a fast workhorse, a social interlocutor, a fallible object to repair, a mirror for their own thinking, or a source whose claims must be calibrated. Connecting these perspectives produces a practical rule: judge literacy by the quality of the learner’s relationship with the system and the task, not by the strength of their feeling about the tool.
The measure should follow the work
A useful evidence model has six connected questions. It is not a replacement for the framework, and not every task needs to sample every question equally. It is a way to prevent a single confidence response from hiding the different kinds of judgement that AI literacy requires.
First, can the learner explain the system and its relevant limits? The learner does not need to provide a technical account of every algorithm. They should be able to describe what kind of AI system they are using, what task it is designed to support, why its output may be variable or fallible, and why a fluent answer is not proof of human understanding. They should be able to use language that does not turn the system into a person with beliefs or intentions. This is the knowledge and communication foundation of the framework’s Engage with AI domain.
An explanation is stronger when it is connected to the task. “AI can make mistakes” is true but too general to guide a decision. “This system generates a plausible summary, so I will check names, dates, and interpretations against the assigned sources” connects a system property to an action. Confidence may make the learner willing to explain the tool, but the explanation itself is the evidence.
Second, can the learner choose the system’s role? AI literacy includes deciding whether AI belongs in the task at all. If it does, the learner should be able to say which steps it will support and which steps require human judgement, subject knowledge, voice, or accountability. This is the heart of Manage AI, where learners compare approaches, decompose a problem, allocate work, and monitor progress.
This criterion resists a common but misleading idea that literacy means using AI more often or delegating more work. The framework allows for AI-only, AI-supported, human-led, or human-only parts of a process depending on complexity, context, data, transparency, desired outcomes, and impact. A confident learner who hands over every step may be less literate in that task than a cautious learner who uses AI only for a bounded purpose and can defend the boundary.
Third, can the learner verify and justify what the system produces? Verification is not a ritual request to “check the facts.” It requires a suitable comparison: a taught process for a calculation, a reliable source for a historical claim, execution for code, or the learner’s own stated criteria for a creative result. The learner should be able to identify which parts of an answer are provisional, seek appropriate evidence, and justify whether to accept, revise, or reject them.