What the Evidence Actually Shows
This chapter interprets the reported tokenizer attacks as evidence rather than as a verdict. The key questions are what the experiment classified, which shortcuts it tried to rule out, how large the measured signal was, and where the result stops. That distinction matters because an AUC or a low-false-positive true-positive rate can demonstrate distinguishability under a defined setup without proving that a particular person’s text entered a commercial model.
A controlled experiment with a concrete unit
The main evaluation uses web data from the C4 corpus, which was built from cleaned and filtered Common Crawl pages. The study uses 1,681,296 pages across 4,133 websites and treats each website as a dataset. That choice gives the experiment a large number of candidate collections and a clear membership unit: the question is whether a website-level collection was included in tokenizer training.
The target tokenizers are trained on half of the website datasets. The researchers vary the vocabulary size from 80,000 to 200,000 tokens, a range intended to resemble the scale of modern commercial tokenizers. The study also checks bytes per token across general English, code, multilingual content, and mathematics. The trained tokenizers are reported to have utility comparable to the commercial tokenizers used for comparison. This matters because a privacy signal found only in an unusually weak tokenizer would be easier to dismiss as an artifact of a poor setup. The experiment instead tries to keep the representation useful while testing its exposure.
The website-level unit is both a strength and a limit. It makes membership labels available: the researchers know which website collections were used to train a target and which were held out. It also means the main result is not a direct test of whether a named individual’s record was included. A website can contain many authors, copied material, and repeated pages. The classifier is asked to recognize a collection-level contribution, not to identify the person behind every string in that collection.
The design also checks for a common failure mode in membership-inference studies: a classifier may separate members from non-members because their content distributions differ, not because the target model or tokenizer learned a membership signal. For example, if all member pages come from one topical period and all non-member pages come from another, a classifier can exploit vocabulary or style differences without inspecting the target’s training behavior.
To test this possibility, the authors train a random-forest classifier on bag-of-words features from the candidate datasets, without using the target tokenizer. The reported AUC is 0.513. An AUC near 0.5 is close to chance separation, so this check supports the authors’ conclusion that the main evaluation does not offer a useful distribution-shift shortcut in that setting. It does not prove that every possible shortcut has been eliminated, but it makes the central comparison more informative: the stronger results come from examining tokenizer behavior rather than simply recognizing which pages look like members.
The controls are important because the attack is a comparison between hypothetical training histories. If the member and non-member datasets were visibly different before the tokenizer entered the experiment, a high score would not tell us much about tokenizer leakage. The C4 construction, the website-level split, the bag-of-words check, and the utility comparison do not make the setup identical to production. They do make the evidence more specific than a benchmark in which the labels can be recovered from the input text alone.
Utility matching closes one interpretive gap but opens another. It shows that the target tokenizers are not obviously toy systems whose poor compression creates an artificial attack surface.