Overview

This chapter separates measured results from proposed mechanisms and unsupported extrapolation. It examines poisoning-to-extraction, extraction-to-distribution-inference, and reconstruction-to-privacy paths under specified datasets, architectures, access models, query budgets, and replacement ratios. The cases include lower extraction metrics for poisoned targets on CIFAR10/CIFAR100, stronger distribution inference from extracted shadows on listed CELEBA/UTKFACE settings, and nonlinear membership results—evidence for conditional audit hypotheses, not causal laws or prevalence estimates.

Learning outcomes

Measured edge direction depends on the baseline and conditions; an attack label alone cannot establish amplification.
In tested CIFAR10/CIFAR100 comparisons, poisoned targets had lower surrogate accuracy and fidelity than no-poison targets.
Extracted shadows improved reported distribution inference, while reconstructed data improved some privacy attacks but reduced membership performance at 75% replacement.

Keywords

evidencebaselinesfidelityextractioninferencereconstruction

What the Case Studies Actually Prove

The case studies in the supplied SoK put colluding adversaries in ML pipelines under a useful constraint: a plausible dependency must survive comparison with a baseline. This chapter examines what the reported experiments show about three relationships—poisoning and model extraction, model extraction and distribution inference, and data reconstruction and downstream privacy attacks. The goal is not to rank attack methods, but to separate measured direction from interpretation, and interpretation from claims the evidence cannot support.

That distinction matters because the framework identifies potential edges by matching factors. An arrow on a conceptual map says that one attack may improve a condition needed by another. An experiment asks whether that improvement appears under a particular dataset, architecture, access model, and evaluation procedure. The answer can be positive, negative, or nonlinear. All three outcomes are informative for threat modeling.

The negative case: a poisoned target is not automatically easier to steal

The first comparison tests the relationship between poisoning and unauthorized model ownership through model extraction. The intuitive assumption would be that a more unusual or compromised target might expose more behavior to a party trying to copy it. The framework predicts the opposite through tail length: poisoning can extend the tail of the training distribution, leaving under-represented records insufficiently learned. If a surrogate learns from the target’s outputs, those irregular or poorly learned regions may be harder to reproduce. The target has become more vulnerable in some respects without becoming a better extraction target.

The experiment compares a no-poison baseline with poisoned targets on CIFAR10 and CIFAR100. The target model is a ResNet and the surrogate is a VGG16. The setup uses a backdoor with a small patch trigger and a knockoff-style extraction procedure in which the surrogate learns from the target’s predictions. Several poisoning levels and query budgets are compared. The measured quantities are surrogate accuracy and fidelity: how well the surrogate performs its task and how often its outputs match the target. The target’s own behavior is also checked to establish that the poisoning took effect.

Across the tested poisoning levels and budgets, the reported surrogate accuracy and fidelity are lower than the no-poison baseline. That pattern supports a narrow conclusion: under this combination of datasets, model families, poisoning method, and extraction procedure, poisoning did not amplify model extraction through the proposed tail-length relationship. It does not mean the target was unharmed, nor does it mean that every kind of poisoning makes extraction more difficult.

The baseline is doing important work here. Without it, a lower surrogate score could be attributed to a poor extraction method, a difficult dataset, or an inadequate query budget. Comparing poisoned and unpoisoned targets under the same general setup makes the direction of change interpretable. Even then, the result remains conditional. The paper’s argument is not that poisoning and extraction can never interact. It is that this particular factor points in the wrong direction for useful collusion in the tested setting.

The case also clarifies what “model theft” means in this research.

Published by DeepMox

FAQ

What did the poisoning-to-extraction experiment show?

On CIFAR10 and CIFAR100, using a ResNet target, VGG16 surrogate, backdoor poisoning, and knockoff extraction, poisoned targets produced lower surrogate accuracy and fidelity than no-poison baselines across tested levels and query budgets.

Why do baselines matter in collusion experiments?

Baselines show whether a downstream metric changed because of the proposed upstream attack rather than a difficult dataset, weak method, or insufficient query budget.

What did reconstruction experiments show about privacy attacks?

In reported settings, reconstructed records improved attribute and distribution inference across tested replacements, while membership inference improved at moderate replacement but declined at 75% at a fixed low false-positive point.

Previous16 minColluding Adversaries in ML Pipelines: Test-Time Collusion

Next15 minColluding Adversaries in ML Pipelines: Partners and Chains

Colluding Adversaries in ML Pipelines: Case Study Evidence

What the Case Studies Actually Prove

The negative case: a poisoned target is not automatically easier to steal

FAQ