Automated graders built on large language models now routinely explain their scores: "the student earns this rubric point because they state that bond breaking absorbs energy." Teachers are asked to trust these explanations, audit them, and pass them on to students as feedback. Work on chain-of-thought faithfulness suggests the trust is misplaced. Chain-of-thought means having the model write out intermediate reasoning before answering. That research shows models often settle on an answer early and then write a plausible justification afterward, a failure called post-hoc rationalization. The deeper cause is that information flows from the input straight to the answer, bypassing the written steps. In grading, this means a model can award a point for the essay's length or vocabulary and then cite a sentence that merely sounds relevant.
We propose a grader that cannot do this by construction. For each rubric item, a retriever first pulls the few sentences from the student's answer that best match the item. A verifier then decides whether the item is satisfied while seeing only those retrieved sentences and the rubric statement, never the full answer. The score is a fixed sum of item verdicts. The quoted evidence is therefore the complete input to every decision, not a commentary on it. We claim two things. First, this evidence bottleneck makes the explanations faithful in a sense that can be checked by deleting and inserting sentences. Second, it improves grading on questions the model has never seen, because it strips away the whole-answer surface cues that do not transfer between questions.
Core Claim and What Is New
The closest existing assessment system is the rubric-entailment formulation of Automated Long Answer Grading with RiceChem Dataset. It treats each rubric item as a hypothesis and the whole student answer as the premise of a natural language inference check, where the model decides whether one text implies another. A fine-tuned RoBERTa-large-mnli reaches 86.8% accuracy and 0.888 F1 on seen questions, but only 60% to 68% on unseen ones. That system decomposes the grade but produces no evidence, and its verifier sees the entire answer. REC-CBM routes grades through a concept bottleneck: an intermediate layer of rubric-concept scores that a teacher can inspect and override. It shows that intervening on those concepts shifts the grade predictably. Its bottleneck, however, sits on latent concept scores computed from the full text, so it offers no textual evidence and no test of whether the concept scores themselves rest on the right sentences.
The rationale-based graders take a different route. AERA and RMTS generate prose rationales, but feed the student model both the rationale and the raw text, leaving the bypass open. AERA even regenerates rationales conditioned on the gold score, which is post-hoc rationalization built into data construction. The mechanism we borrow comes from Markovian Transformers, which hides the question from the answer step so that the reasoning must carry everything. On GSM8K it moved accuracy from 19.6% to 57.1%, while staying only about 3 to 4 points below a model that could still see the question. We apply the same idea with two changes. The bottleneck content is retrieved verbatim spans of student text rather than generated reasoning, which makes it checkable against the source and rules out hidden encodings in the generated text. And the downstream decision is a rubric verdict rather than a math answer.
Retrieval-augmented generation normally fetches external documents to ground a generator. Here the retrieval stage points inward, at the student's own answer, with the rubric item as the query. The known weakness of that design, that the generator cannot recover from a failed retrieval, becomes useful rather than harmful. A missed span yields "not satisfied," which is visible and correctable, instead of a silent shortcut.
Research Program
The system has three parts.
The first is an evidence retriever. It scores every sentence of the answer, and every window of adjacent sentences, against the rubric item and returns the top $k$. It is trained weakly from item labels, meaning that spans leading to correct verdicts are reinforced. A small set of human-marked evidence spans is used only for evaluation.
The second is a verifier. It is an inference model or a small instruction-tuned language model that receives only the rubric item and the $k$ spans and outputs satisfied or not. If it is a language model, it may write short reasoning, but that reasoning also sees only the spans.
The third is a fixed aggregation into the question score. Teachers can override any item verdict, in the spirit of REC-CBM's interventions.
Retrieving external material, such as a reference passage defining the rubric concept for an unseen question, is tested as a single add-on rather than folded into the core design, so that its contribution is measured separately.
The analysis side adapts faithfulness measurement to grading. FRIT swaps a reasoning step for a style-matched distractor and checks whether the answer changes. We make the analogous edit on the student answer. For necessity, we replace the cited sentence with a fluent, on-topic sentence that does not satisfy the item, and check whether credit is withdrawn. For sufficiency, we insert a satisfying sentence taken from another student's answer into a failing response, and check whether credit appears. We report the flip rate on edits that human annotators confirm change the true verdict. We also report the false-flip rate on edits that should not matter, such as removing uncited sentences. For generative chain-of-thought graders we additionally compute the early-answering curve from On the Hardness of Faithful Chain-of-Thought Reasoning, which tracks how the grade probability evolves as reasoning steps are revealed.
The bottleneck grader meets the necessity and sufficiency tests almost by construction when the retriever picks the edited sentence. The interesting question is how unconstrained graders fare on the same tests. Evidence from Analysing Chain of Thought Dynamics suggests they will fail often: instruction-tuned models there changed their initial answers in only 25% of cases and scored 0.00 cue faithfulness on CSQA with a metadata cue.
Baselines
The naive reference point is holistic scoring: a fine-tuned encoder maps the full answer directly to a score. RiceChem reports that rubric decomposition beats this by 9.2% accuracy and over 15% F1, so any method that fails to clear holistic scoring by a similar margin has lost the benefit of decomposition.
The strongest existing method on the primary benchmark is RoBERTa-large-mnli rubric entailment over the full answer, from RiceChem. It reports 86.8% accuracy and 0.888 F1 on seen questions and 60% to 68% on unseen questions. We will retrain it on our exact splits, using the same backbone as our verifier and the same hyperparameter search budget, and report our reproduced numbers alongside the published ones. Beating it means two things. On unseen questions, we must exceed it by a margin outside the variance across held-out question folds. On seen questions, we must lose no more than about 2 points.
The strongest language-model competitor is a per-item chain-of-thought grader in the style of A Chain-of-Thought Prompting Approach with LLMs for Evaluating Students' Formative Assessment Responses in Science. It sees the full answer and rubric, reasons, and is asked to quote evidence. We run it with the same backbone and the same few-shot examples as our verifier. Zero-shot GPT-4 at 70.9% on RiceChem is the published anchor. On Mohler, we compare against REC-CBM and Grade Like a Human. The file reports no absolute Mohler accuracy for either; there is no reported result found on Lacuna for their Mohler accuracy, only REC-CBM's gain of up to 5 points from oracle correction of its top-2 concepts. We will therefore reproduce both where code allows.
The key ablations isolate the bottleneck. The first gives the identical verifier the retrieved spans plus the full answer, which reopens the bypass. The second feeds random spans, the third oracle human spans, and the fourth varies $k$.
Why not just…?
Why not just fine-tune the entailment encoder and read off attention or gradient attributions as evidence? This is the most serious threat, because the encoder is already accurate on seen questions. Attribution maps, however, are post-hoc by nature: nothing forces the verdict to depend on the highlighted tokens. The same necessity and sufficiency edits apply directly to the encoder's attributed sentences. If full-answer RoBERTa matches our flip rates using its attributions and also matches our unseen-question accuracy, the bottleneck adds nothing and we will say so.
Why not just prompt a language model to cite quotes before grading? Citation prompting yields quotes, but the grade can still be computed from the whole answer. Measuring and Mitigating Post-Hoc Rationalization in Reverse Chain-of-Thought Generation found that instructions telling a model to set aside its answer actually increased subtle anchoring on that answer. That result is why we remove the full answer from the verifier's input rather than instructing the verifier to ignore it. The full-context chain-of-thought baseline, scored on the deletion test, directly measures whether prompted quotes are load-bearing.
Why not just retrieve graded exemplars from other questions to fix unseen-question transfer? Nearest-neighbor few-shot retrieval may well recover much of the unseen-question gap, and we include it as a baseline with the same retrieval budget. It does nothing for faithfulness, though. If it wins on accuracy, we will test whether it combines with the bottleneck, rather than claim transfer gains for the bottleneck alone.
Why not just accept that the bottleneck will cost accuracy? Hiding context can break coreference and multi-sentence arguments. The trade-off documented by On the Hardness of Faithful Chain-of-Thought Reasoning, accuracy drops of up to 20% on TruthfulQA, is a real warning. The window retrieval and the $k$ sweep exist to measure this cost, and the oracle-span ablation separates retriever failure from bottleneck failure.
Proposed Experiments
The primary benchmark is RiceChem, with 1,264 responses, 27 rubric items, and 8,392 labels. We add leave-one-question-out splits and roughly 800 double-annotated evidence spans. Mohler serves as the second benchmark. It tests whether short-evidence verification reduces the pedantic errors on simple questions reported in Grade Like a Human. ASAP-SAS is a secondary transfer check against AERA-style distillation. We report the paper's gain of 11% QWK (quadratic weighted kappa, a chance-corrected agreement score with human graders) over ChatGPT as the reference, since the file gives no absolute value.
The metrics are item accuracy and F1, question-level QWK, evidence-span overlap with human annotations, and necessity and sufficiency flip rates with their false-flip controls.
The decisive checks are three. First, the bottleneck grader must beat the reproduced full-answer entailment encoder on unseen questions while staying within 2 points on seen ones. Second, it must achieve substantially higher necessity and sufficiency flip rates than both the chain-of-thought grader and encoder attributions. Third, the spans-plus-full-answer ablation must lose the transfer and faithfulness gains, which shows that removing the bypass, not retrieval alone, is responsible.
If the first check fails but the second passes, the contribution narrows to a faithfulness guarantee with a measured accuracy cost. If the third fails, retrieval rather than the bottleneck is doing the work, and the claim is revised accordingly.
References
- Automated Long Answer Grading with RiceChem Dataset
- REC-CBM: Rubric-Aware Error-Correction Concept Bottleneck Models for Trustworthy Open-Ended Grading
- Distilling ChatGPT for Explainable Automated Student Answer Assessment
- Rationale Behind Essay Scores: Enhancing S-LLM's Multi-Trait Essay Scoring with Rationale Generated by LLMs
- Markovian Transformers for Informative Language Modeling
- FRIT: Using Causal Importance to Improve Chain-of-Thought Faithfulness
- On the Hardness of Faithful Chain-of-Thought Reasoning in Large Language Models
- Analysing Chain of Thought Dynamics: Active Guidance or Unfaithful Post-hoc Rationalisation?
- A Chain-of-Thought Prompting Approach with LLMs for Evaluating Students' Formative Assessment Responses in Science
- Grade Like a Human: Rethinking Automated Assessment with Large Language Models
- Measuring and Mitigating Post-Hoc Rationalization in Reverse Chain-of-Thought Generation