Confidence Is a Signal, Not a Score
AI literacy is the subject of this series; the focus here is a narrower question: can a learner’s confidence in using AI tell us how literate they are? The evidence supports a careful answer: not by itself. Confidence can indicate that someone feels ready to try, or that a tool feels familiar and manageable. It cannot, on its own, show whether they understand how the system works, check what it produces, choose an appropriate task, repair a failure, or take responsibility for the result.
That distinction matters because AI systems now sit inside ordinary acts of learning and decision-making. A learner may ask a chatbot to explain a difficult idea, use a recommendation system to find information, or ask a generative tool to help shape a draft. In each case, the interaction can feel successful before the learner has examined whether the output is accurate, relevant, fair, or appropriate for the task. The practical question is therefore not whether a learner feels capable with AI, but what they can demonstrate when the system’s answer needs to be understood and judged.
Feeling capable is not the same as being literate
The OECD/European Union AILit Framework defines AI literacy as the technical knowledge, durable skills, and future-ready attitudes needed to thrive in a world influenced by AI. It describes four kinds of activity: engaging with AI, creating with AI, managing AI, and shaping AI. It also makes an important distinction that is easy to lose in everyday conversation: using an AI tool is not the same as possessing AI literacy. Simply interacting with a system neither develops nor proves the knowledge, skills, and competences in the framework.
Consider a simple example. Two learners use a chatbot to help with a history assignment. Both obtain a fluent paragraph and both may report that they are comfortable using the tool. The first learner copies the paragraph after a quick read. The second asks what claims require checking, compares the answer with a reliable source, notices that one interpretation does not fit the evidence, and rewrites the passage in their own argument. Their confidence might be identical. Their demonstrated literacy is not.
The difference is not a moral judgement about the first learner. It is a measurement problem. Confidence is a report about a person’s perceived readiness or capability. Literacy is a wider relationship among understanding, action, judgement, and responsibility. A person can feel confident because the interface is easy to use, because previous answers happened to work, or because the system responds in a reassuring tone. None of those experiences necessarily supplies knowledge about the system’s limits or the quality of a particular answer.
The framework makes this wider relationship explicit through three supporting layers. Knowledge covers the nature of AI, the way it reflects human choices and perspectives, its capabilities and limitations, and its role in society. Skills apply familiar human capacities—such as critical thinking, communication, problem solving, creativity, and self-awareness—to situations involving AI. Attitudes shape how a learner approaches those situations: reflectively, responsibly, curiously, adaptably, innovatively, and empathetically.
Those layers answer different questions. Knowledge asks, “What is this system likely to do, and what can it not do?” Skills ask, “How will I examine, direct, compare, or revise its contribution?” Attitudes ask, “What obligations and consequences should influence my choice?” A confidence question usually answers none of them with enough specificity. At most, it tells an educator whether a learner believes they are ready to attempt the task.
That does not make confidence irrelevant. The framework’s developmental expectations are intended to help learners progress, and appropriate scaffolding can advance their confidence as their understanding and ability grow. Confidence may support curiosity and persistence, especially when a learner is trying an unfamiliar system. But it belongs inside the learning process, not at the end of it as a substitute for evidence. A learner who is confident and a learner who is uncertain may both be highly literate or poorly prepared. Their confidence is a reason to ask better questions, not a final score.
The central distinction can be stated simply: confidence is evidence about a learner’s starting position; literacy is evidence about what happens next. The latter becomes visible in explanations, checks, choices, revisions, and refusals to rely on an unsuitable answer.
Why the distinction has consequences
The stakes rise because AI can produce a convincing result without providing the conditions that normally help a learner recognise uncertainty. The framework describes AI systems as using algorithms and statistical inferences to process data, detect patterns, and generate outputs such as predictions, content, or recommendations. Generative AI can produce advanced text, images, or other content, but it does not possess authentic human understanding or intent. Its outputs can vary with the input and the system’s design, and their fluency can make fact and fabrication difficult to distinguish.
For a user, this creates an uncomfortable asymmetry. The system can sound finished before the learner’s thinking is finished. A polished explanation may remove the feeling of difficulty that would otherwise prompt a learner to slow down, compare sources, or ask whether they understand the subject. If the answer also uses a warm, conversational style, the interaction may feel collaborative rather than provisional. Ease and social comfort can increase confidence while leaving the learner’s ability to judge the content unchanged.
This is one reason the framework asks learners to describe AI without assigning it human traits. Saying that a system “knows,” “believes,” or “understands” can quietly turn a statistical process into a social authority. More precise language does not solve every reliability problem, but it keeps the user’s mental model aligned with the kind of system they are actually using. The same principle applies to confidence: feeling that an answer is clear is not the same as having grounds to trust it.
The consequences appear in ordinary educational choices. A learner who uses AI to brainstorm may preserve their own ideas, compare suggestions, and reject a proposal that does not fit the task. Another learner may accept the first fluent suggestion and lose the opportunity to formulate a problem independently. A learner who uses AI for research may verify claims and track which evidence supports them. Another may mistake a detailed citation or confident explanation for proof that the underlying source exists. Both learners may say they are comfortable with AI. Only their process reveals how that comfort is being used.
The framework therefore treats critical evaluation as an action. One competence asks learners to decide whether an AI output should be accepted, revised, or rejected. At a basic level, they understand why verification is necessary. At an intermediate level, they check accuracy and relevance against trusted sources and the expectations of the task. At an advanced level, they assess the output and justify the decision. The important evidence is not the learner’s assurance that they can spot an error. It is the quality of the check and the reasoning behind the decision.
The same point extends beyond factual accuracy. AI systems can reflect choices about data, categories, objectives, and deployment. Those choices may amplify social bias, narrow access to information, expose private data, or distribute benefits and harms unevenly. A confident user may still lack the habit of asking who is affected, what information was shared, or whether using AI is appropriate at all. In the AILit Framework, ethics is woven through technical understanding and practical use rather than added as a final warning. Responsibility includes transparency, fairness, privacy, environmental stewardship, and the option not to use AI.
That last option is especially revealing. Tool enthusiasm is easy to observe: the learner reaches for AI, writes a prompt, and receives an output. Literacy also includes restraint. It asks whether the task needs AI, whether human judgement is central, and whether the system’s likely contribution justifies its risks. A person can be highly confident in a tool and still make a poor decision about when to use it. Conversely, a cautious person may need more support to try the tool while already possessing sound judgement about its limits. Measuring confidence alone can miss both kinds of learner.
A better starting set of questions is therefore concrete:
- Can the learner explain relevant features and limits of the system without treating it as human?
- Can they judge whether AI is suitable for this task, and decide what work should remain theirs?
- Can they verify an output against appropriate evidence and explain why they accepted, changed, or rejected it?
- Can they respond constructively when the system fails, including changing the workflow or abandoning AI?
- Can they reflect on how the interaction affected their own thinking and other people?
These questions do not demand suspicion for its own sake. They demand evidence that confidence is calibrated to the task.
A map of what AI literacy includes
The AILit Framework organises those questions into four connected domains. The arrangement is not a checklist of unrelated abilities. It is a progression in agency: from recognising AI’s presence and judging its outputs, to using it creatively, directing its role in a process, and eventually examining how systems themselves can be improved.
Engage with AI is the foundation. Learners identify where AI operates, explain its behaviour without common misconceptions, evaluate outputs, examine recommendations, and consider bias, resources, ethics, and effects on daily life. This domain is where confidence must first meet reality. A learner does not demonstrate engagement literacy merely by opening a chatbot; they demonstrate it by understanding what kind of system they are facing and making a reasoned decision about its output.
Create with AI concerns iterative creative work while maintaining human agency. Learners use AI to explore perspectives, visualise or prototype ideas, elicit feedback, and consider authenticity and intellectual property. The learner’s own idea remains important. A useful question is not simply whether AI generated an attractive result, but how the learner’s idea changed, which suggestions they used, and why they rejected or adapted others.
Manage AI makes the division of labour explicit. Learners decide whether AI fits a task, compare different approaches, decompose a problem, assign steps between human strengths and AI capabilities, and monitor progress against success criteria. This is a direct challenge to the idea that more confident use is always better use. Literacy may lead to greater reliance on AI for one step, less reliance for another, or no reliance at all. The decision depends on the task, the evidence, the need for human judgement, and the consequences of error.
Shape AI extends agency beyond using an existing system. Learners investigate what a system is designed to do, how data and design choices influence behaviour, how performance can be evaluated, and how changes might promote human well-being and societal benefit. This does not require every learner to become an AI engineer. It requires them to see that systems are made through human choices and can be questioned or improved, rather than treated as inevitable authorities.
Each domain includes developmental expectations from Basic through Advanced. They are not fixed age bands and are not presented as a single validated international score. They are a way to make growth more visible: identifying a system is different from explaining its constraints; noticing that an output needs checking is different from verifying it and justifying a decision; using AI in a project is different from monitoring and reallocating its role. These differences are precisely what a confidence-only measure compresses into one impression.
The subject, then, is not whether learners should feel confident around AI. They should have enough support to participate, experiment, and persist. The more important question is whether confidence remains connected to knowledge, judgement, and responsibility as the interaction becomes more consequential. To answer that question, we need to look closely at the system that generates fluent answers and at the moments when users accept, challenge, repair, or abandon them. That is where a feeling of competence either becomes grounded—or is shown to be only a feeling.