When AI Feedback Helps—and When It Doesn’t
The comparison in this chapter concerns a specific use of generative AI: producing written feedback from learning-analytics evidence about students’ self-regulated learning. The question is not whether a language model can write smoother sentences than a busy tutor. It is what the study actually measured, which learners showed a difference, and when a tutor, GenAI system, or combination of the two is the more defensible choice. That distinction keeps a readable message from being mistaken for a completed learning action.
A comparison of messages, not a contest between professions
The second study compared GenAI-generated and tutor-generated feedback in the same nine-week online statistics course. Students studied the same material and received feedback by email. The messages were based on the learner profiles and standardised scores described earlier, so the two paths had a common evidence base. The tutor used a practical interface that displayed the student’s cluster, proxy scores, and explanations of the associated behavioural variables. The GenAI path inserted the learner’s profile and current scores for goal setting, task strategies, time management, help-seeking, and self-evaluation into a structured prompt.
That setup makes the comparison more informative than asking two unrelated systems to comment on different work. It also limits what can be concluded. The study compared one prompt structure and one tutor working with a support tool under authentic workload conditions. It did not compare GenAI with an ideal tutor who had abundant time, richer contextual information, or a different pedagogical approach. The fair question is therefore not “Which profession is better?” but “For which task, learner, and constraint did each path perform better?”
The first measured dimension was readability. Two external experts rated 94 tutor messages and 94 GenAI messages for grammar, clarity, and coherence without being told which source produced each message. Tutor feedback had a mean readability score of 2.4043; GenAI feedback had a mean of 4.4149. A Mann–Whitney U test reported U = 300.5 and p < .001, favouring GenAI. The distribution also differed: most GenAI messages were rated between four and five, while most tutor messages fell below three and varied more widely.
This result has a straightforward practical meaning. If feedback must consistently describe several dimensions of a learner’s behaviour in clear, non-technical language, a language model can produce a more uniform first draft than a tutor operating under time pressure. The tutor messages in the study often used brief evaluations such as noting a drop in engagement or praising timely submission. GenAI messages more often summarised the learner’s status across the five SRL pillars and included ways to improve or maintain it. Readability can make a feedback signal easier to enter into the learner’s action loop.
It cannot complete that loop. A clear message can still be based on a weak proxy, recommend an unsuitable response, or omit a circumstance that matters. It can also be clear in a way that encourages the learner to accept the system’s framing without examining it. The study’s readability test evaluated language quality, not whether students changed their study routines, learned more statistics, or retained a strategy later.
The second dimension was reliability, but reliability here has a particular reference point. The researchers treated the standardised LA data that triggered the feedback as ground truth and checked whether a message correctly covered the positive or negative polarity of each of the five SRL pillars. Precision asked how much of the feedback’s identified signal was correct; recall asked how much of the relevant signal it captured; F1 combined the two.