How WBC Turns Local Differences into a Robust Vote

Window-Based Comparison (WBC) is the concrete detector built for the signal pattern described so far: sparse membership evidence is mixed with larger, long-tailed domain effects. The key question is how to turn an aligned target/reference loss sequence into a score without letting one extreme token decide the result. The answer is a sequence of local comparisons, directional votes, and a deliberately broad view of window sizes.

From two loss streams to local votes

WBC begins after both models have scored the same candidate sequence. At every position, there is a target loss and a reference loss, and their difference says how fine-tuning changed the target’s confidence there. WBC does not immediately add all of those differences into one document-level average. It chooses a window size, examines contiguous regions, and asks whether the target wins within each region.

For a window of size w beginning at position i, the method adds the per-token differences across that region. Equivalently, it sums the reference losses and the target losses separately and compares the two totals. The two views are the same because the difference between the sums is the sum of the differences. If the reference’s total loss is greater than the target’s, the target is less surprised over that region, so the window casts a target-favoring vote. If the reference does not exceed the target, the window does not cast that vote.

Sliding the window one position at a time produces a set of overlapping regions. For a sequence of n tokens, a window of size w creates n − w + 1 possible starting positions. WBC counts the target-favoring regions and divides by the total number of regions. The result is a sign score for that window size: the fraction of local comparisons in which the reference loss exceeds the target loss.

The score changes the question being measured. A mean asks, “How much lower is the target’s loss when every token is counted by its magnitude?” A sign score asks, “How often does the target have lower total loss in a local region?” The second question keeps the direction of the comparison but discards the exact size of each win. That is the central robustness move.

Consider a candidate sequence with one unusually large domain-specific improvement and several smaller changes. Under a magnitude-based summary, the large improvement can dominate the score even if the same event occurs in non-members. Under local voting, the extreme event affects the windows that contain it, but its numerical size does not give those windows unlimited influence. Other regions still contribute their own votes. The outlier has not vanished; its ability to overwhelm the entire record has been limited.

Local aggregation also gives several neighboring tokens a chance to contribute together. A single token can be surprising for ordinary linguistic reasons. A short region provides more context for deciding whether the target/reference difference is a meaningful local tendency or an isolated fluctuation. The region is not assumed to be a memorized passage. It is simply a bounded unit in which a direction can be measured.

That distinction matters for interpreting a WBC score. A high score does not identify a particular token as memorized. It means that a larger fraction of the tested regions favor the target than would be expected for non-members under the evaluator’s comparison. The score is evidence about a pattern across windows, not a map of the model’s internal memory.