Overview
Across two university-course studies, AI grading often aligned with teacher scores, but agreement alone does not prove faithful input, correct judgment, useful feedback, or readiness for high-stakes use. This series traces each evidence layer and connects it to supervised review, deferral, and institutional decisions.
Learning outcomes
- Teacher or TA agreement shows operational fit with a defined course, not absolute grading correctness.
- Validation must check input fidelity, rubrics, reasoning, feedback, human variation, and stakes separately.
- Independent and qualitative review can expose OCR errors and shallow or methodologically wrong feedback that averages miss.
- The evidence supports supervised formative or low-stakes assistance, not autonomous high-stakes grading.
Keywords
Preview every chapter · Unlock full text, audio, and video from chapter 4
PlansFAQ
What does AI–teacher score agreement prove?
It shows that AI matches a defined course’s grading practice; it does not by itself prove correctness, faithful reading, or useful feedback.
Why is teacher agreement not enough to validate AI grading?
A matching score can hide transcription errors, rubric mismatch, flawed reasoning, or weak feedback. Agreement tests operational alignment, while these other claims need separate evidence.
What evidence should an AI grading evaluation include?
It should combine human-reference alignment, input checks, independent review, feedback audits, human-variation baselines, and student response. Each source answers a different validation question.
Is AI grading ready for high-stakes assessments?
The studies do not support autonomous high-stakes grading; they support bounded, human-supervised assistance mainly in formative or low-stakes settings.