From One Task to a Profile: Measuring Prompt and Trait Leakage

The network-level study measures two different kinds of exposure from local research agents: how much of a single task can be reconstructed, and how much a series of sessions can reveal about the person behind the tasks. The results are substantial under the paper's benchmark conditions, but they are not claims of verbatim prompt recovery or universal profiling accuracy. Reading them correctly means keeping the metric, agent, observer view, and dataset attached to every number.

The first result concerns immediate intent. The second concerns persistence. Between them sits a third finding: an observer may be able to recognize agent-generated traffic as a distinctive mode before recovering its topic. That matters because exposure can begin with classification, continue with task inference, and deepen into a profile when sessions can be linked.

Prompt recovery is functional, not verbatim

In the main SESSION14 comparison, five-shot in-context learning with GPT-4o and GPT-Researcher produces a mean SBERT score of 0.492 and an LLM-as-Judge score of 0.415. Under OBELS, the same setting reaches 0.770 for functional equivalence, 0.735 for domain-type equivalence, 0.520 for semantic equivalence, and 0.640 for entity-granularity tolerance. The pattern is more informative than any single number. The observer can often recover what kind of task the agent is performing and what kinds of sources it needs, while losing some precision about the exact entity or level of detail.

This is the basis for the paper's headline claim that the attack recovers more than 73 percent of the functional and domain knowledge in user prompts. The figure should not be read as 73 percent of the original words, nor as a probability that a single observer will reconstruct a complete private request. It summarizes high functional and domain-equivalence scores under the study's evaluation. A paraphrase that preserves the work the agent must do can be a successful recovery even when its wording is substantially different.

The comparison across agents shows why exposure is not a fixed property of “AI research.” On SESSION14, the reported functional and domain OBELS scores are 0.565 and 0.625 for AutoGen, 0.540 and 0.590 for Browser-Use, 0.770 and 0.735 for GPT-Researcher, and 0.680 and 0.720 for the smaller Deep Research comparison. GPT-Researcher exposes more in this comparison, consistent with broader source coverage and richer source diversity. AutoGen and Browser-Use expose less, but their traces remain informative. The result points to orchestration as a security variable: how widely an agent explores and how it organizes retrieval can matter as much as the name of the underlying model.

The numbers also resist a simplistic “more leakage is always worse engineering” interpretation. A research agent that visits many independent sources may produce a more carefully grounded report, while a narrower agent may expose less because it gathers less evidence. The privacy problem arises when the extra exploration is not matched by user value, or when the system has no way to distinguish essential visits from redundant ones.