The Dashboard Is Not the Intervention
Learning analytics dashboards are often treated as if the difficult part of supporting learners were collecting enough information. This series examines a narrower and more consequential question: what happens between a dashboard showing a learner’s activity and that learner changing how they study? The subject is the learner-facing dashboard—along with the learner, the feedback source, and the tutor or pedagogical agent around it. The practical aim is to help readers tell the difference between a system that reports a state and one that helps a learner understand and respond to it.
That distinction matters because online learning shifts more responsibility onto the student. A learner may need to decide whether to revisit a topic, change a study strategy, ask for help, or continue with confidence. A graph can make evidence available, but availability is not the same as interpretation. The central claim developed here is therefore modest: a dashboard becomes educationally useful when its signals participate in an action loop. The loop connects evidence to a learner’s judgement, interpretation, decision, action, and later re-checking. It is a way to examine the relationship among dashboard design, learner regulation, and feedback—not a claim that either of the studies discussed here tested one complete, universally effective intervention.
What a learner-facing dashboard is supposed to do
A learning analytics dashboard gathers traces from a digital learning environment and presents them as information about progress, engagement, performance, or context. On a student-facing dashboard, the intended user is the learner rather than only an instructor or administrator. The hoped-for result is not simply that a student knows what the system has recorded. It is that the student can reflect on the evidence and make a better decision about studying.
In practice, dashboards have often concentrated on the first half of that job. They may display a line graph of exercise scores, a completion bar, a comparison with a class average, or an indicator of lateness. More recent designs add filtering, zooming, narration, personalisation, and natural-language explanations. These features can make data easier to see and understand, especially for learners who find visualisations difficult. That is useful work. A chart that cannot be read cannot support a decision.
But readable information still leaves a cognitive step to the learner. What does a low score mean in this person’s situation? Is the problem a missing concept, insufficient practice, rushed work, or a misleading measure of progress? Which response is worth trying, and how will the learner know whether it helped? The first study’s review of earlier research describes the evidence for dashboards’ effects on outcomes as mixed and argues that many systems improve exposure to information without requiring learners to connect it to their own experience. It reports that fewer than 1% of reviewed implementations were designed to foster cognitive engagement, although that statistic is part of the study’s literature summary rather than an independently checked finding in the present evidence set.
The issue is not that visualisation is pointless or that every dashboard must become a conversation. It is that a metric is an input to learning, not learning itself. A completion percentage can say that activity occurred; it cannot, by itself, make the learner evaluate the quality of that activity or decide what should happen next. Treating metric quantity as educational value risks building a more elaborate reporting surface while leaving the learner’s central reasoning unchanged.
The missing middle: an action loop
An action loop adds the missing relationship between evidence and behaviour. First, the system collects a trace: an exercise result, a submission time, a viewed resource, or a pattern across tasks. Next, the dashboard represents that trace as a signal. The learner then brings a judgement to it—an estimate of what they understand, how prepared they are, or whether their current approach is working. Feedback or a pedagogical prompt helps the learner interpret agreement or discrepancy between the two. The learner chooses an action, and later activity supplies new evidence for another evaluation.
The ordering matters. If the system immediately announces an interpretation, the learner may only accept, reject, or clarify the system’s statement. If the learner first predicts or explains, the displayed metric becomes something to compare with an existing mental model. A discrepancy can then be useful: it may prompt the learner to inspect their evidence, reconsider a strategy, or seek a different kind of help. The loop does not require disagreement every time. Accurate confirmation can also support a justified decision. What matters is that the learner has a role in making sense of the signal.
This is the distinction between interface interactivity and pedagogical interactivity. Interface interactivity lets a person manipulate a representation—filtering a date range, selecting a competency, or requesting more detail. Pedagogical interactivity involves reciprocal actions that build on one another and require cognitive engagement. In tutoring research, the first study draws on the idea that eliciting a learner’s reasoning can promote explanation, integration with prior knowledge, and monitoring more strongly than simply telling the learner the salient information. Applied to dashboard data, the question becomes: can the learner connect the system’s evidence with their own experience and use that connection to guide a future action?
The action loop is also a safeguard against a common category error. A system may generate a fluent explanation without producing a reliable judgement about the learner. A dashboard may estimate mastery without directly observing understanding. A learner may report changing behaviour without sustaining that change. Each link needs its own evidence. The loop is a model for following those links, not permission to treat one attractive intermediate measure as proof of lasting academic improvement.
Two cases that make the question concrete
The first empirical case, an Interactive Learning Analytics Dashboard in a university programming course, tests what happens when interaction is designed around learner reflection. Thirty volunteer students used the system for five weeks and were assigned to a dashboard with no pedagogical agent, a telling agent, or an eliciting agent. The dashboard included learning-analytics indicators and a Judgement of Learning feature. For a competency, the learner first rated their own mastery on a five-step scale; only then did the system reveal its estimated progress and mastery and offer related material. The design made self-assessment precede system feedback rather than allowing the metric to supply the learner’s first answer.
The reported interaction patterns differed. Students working with the eliciting agent wrote more messages classified as reflection than students using the telling agent. Telling-agent users more often asked for clarification or organisational information. The distinction is not a verdict that questioning is always better: direct explanation can lower access barriers, and the study measured message categories rather than the quality of every learner decision. It does show why “interaction” needs a cognitive definition. Two chat windows can look similar while asking the learner to do very different work.
The same case found promising movement in Judgements of Learning. Across 482 submitted judgements, ratings became more consistent over time. In the eliciting condition, the correlation between learner judgements and the system’s composite mastery estimate increased across three time periods, reaching a reported r = 0.482 in the final interval. The authors explicitly describe these patterns as exploratory; the sample was small, participants were paid volunteers, and the system estimate was not an independent measure of mastery. The finding is best read as evidence that the sequence may support calibration—the learner’s ability to judge their state in closer alignment with available evidence—not as proof of improved exam performance or durable study behaviour.
The second case approaches the loop from the feedback side. In a nine-week online statistics course, researchers analysed the LMS traces of 94 higher-education students and modelled five aspects of self-regulated learning: goal setting, task strategies, time management, help-seeking, and self-evaluation. Clustering produced high- and low-SRL profiles. During the intervention, students received written feedback generated either by GenAI or by a tutor, based on standardised analytics about their profile. The feedback was delayed rather than delivered as a constant stream, allowing time between observation and intervention.
In this study, GenAI feedback was rated much more readable than tutor feedback and generally covered the analytics-defined pillars more consistently. Among low-SRL students, the GenAI group showed significant differences in goal setting and in the composite SRL score relative to the low-SRL tutor group. Other individual pillars did not show significant differences, and no significant differences appeared between GenAI and tutor feedback for high-SRL students. These results make the action-loop question more precise. A message can be clear and aligned with the data used to generate it; that does not establish that it changed every aspect of regulation, helped every learner, or outperformed a well-resourced expert tutor.
Together, the cases point away from a simple technology race. One changes the order of interaction so that learners judge before seeing a system estimate. The other uses a theory-based learner model and timed feedback to address differences in regulation skill. Both treat analytics as something that must be interpreted and acted upon. Both also leave important links open: whether learners actually changed their study routines, whether the changes persisted after incentives or novelty faded, and whether later performance or transfer improved.
The practical question for a dashboard designer, tutor, or institution is therefore not “How many metrics can we show?” It is “What will the learner do with this evidence, and what will we observe next?” Answering that question requires a closer look at how dashboards construct their picture of a learner, how prompts place cognitive responsibility, and how feedback returns to the loop. More metrics may widen the input. Only a well-designed interaction can help determine whether the input becomes learning.