Defenses Change the Signal—They Do Not All Mean Privacy

The evidence for WBC raises a practical question about fine-tuned LLMs: if local memorization can survive inside a model’s behavior, what kind of defense actually changes the risk? This chapter compares differential privacy, low-rank adaptation, and selective data obfuscation by the part of the system each one changes. Readers will learn to distinguish a formal privacy guarantee from an empirical attack result, weigh utility and computation against protection, and design an audit that does not mistake a quieter signal for proof that privacy has been established.

Differential privacy changes the training guarantee

Differential privacy, or DP, addresses the membership problem at the level of the learning algorithm. The aim is not simply to make a later attack harder on one benchmark. It is to quantify and bound how much the learner’s output can depend on any individual training record under the assumptions used in the privacy analysis. That is a different object from WBC’s score, which measures how well one particular evaluator separates members from non-members.

The distinction is easiest to see in the training step. Ordinary stochastic gradient descent computes an update from a minibatch and uses that update to change the model’s parameters. DP-SGD modifies this process in two important ways. It computes a gradient for each example rather than exposing only an already-averaged batch gradient, clips each per-example gradient to a fixed bound, and adds Gaussian noise to the aggregate before updating the parameters. Random sampling of minibatches is part of the analysis that composes these steps into an overall privacy guarantee.

Clipping limits how much one record can influence an update. Without a bound, one unusually influential example could contribute an arbitrarily large gradient. Noise then makes the output less distinguishable between neighboring training sets—one containing the record and one not containing it—according to the chosen privacy accounting. The mechanism changes the path by which individual information enters the target model, rather than waiting until deployment to hide a signal in the model’s scores.

The protection has costs. For a fixed privacy guarantee, the noise required by DP-SGD depends on how many training steps are taken, so training longer is not a free way to gain utility. Per-example gradient computation can also be more expensive or less compatible with hardware optimizations than ordinary batch averaging. NIST notes a further accuracy problem for heavy-tailed data: if a rare condition has few examples, DP’s restriction on learning patterns that are not broadly represented can make that subgroup harder to model. A smaller privacy budget means a stronger guarantee, but it may also make the model less useful for the task.

DP does not require noise to be added to gradients in every private learning design. NIST describes model-agnostic approaches such as PATE, in which the private dataset is divided into disjoint subsets, separate non-private models are trained as teachers, and their predictions are aggregated with noise before an answer is released. When teachers agree, the noisy vote is less likely to change the result. Here, the privacy boundary is placed around the aggregation of teacher outputs rather than around every gradient update in one model.

These mechanisms illustrate why “privacy” is not one switch. DP-SGD changes parameter updates through clipping and noise. PATE changes how multiple models’ predictions cross the boundary to a released result.