How an LLM Stream Becomes a Length Signal
The subject here is the application layer between an LLM’s generated response and the TLS records described earlier. The key question is how a passive observer can move from an encrypted stream to a sequence of merged lengths without seeing the response itself. The answer depends on two facts that are easy to blur together: LLM tokens are not the same thing as words, and a network packet is not automatically the same thing as a token or even a TLS record. Keeping those distinctions intact explains both why PromptEcho has a usable signal and why the signal remains ambiguous.
A token is not a word, and a merged length is not a sentence
An LLM generates text through tokens. A token can correspond to a whole word, a fragment of a word, punctuation, or a unit that includes leading whitespace. The tokenizer decides the boundaries. Different model families can split the same visible text differently: a contraction may be kept together by one tokenizer and broken into several pieces by another; a hyphen, a space, or a punctuation mark may be attached to a neighboring fragment or represented separately.
That implementation detail has a direct privacy consequence. If an observer sees a length, they do not see a word of that length. They see a number that could have been produced by many token sequences. A four-character token might be a word, a fragment, or a word-like piece with a leading space. The same visible sentence can produce different token-length patterns under different model tokenizers. A reconstruction system therefore needs the target tokenizer or a compatible approximation, and even an exact tokenizer does not remove the ambiguity.
Consider a sentence such as “I’ll check if it’s a follow-up infection.” One tokenizer may keep parts of the contractions and hyphenated phrase together; another may split the apostrophes, spaces, and hyphen into separate pieces. The visible sentence has not changed, but the sequence of character lengths has. Leading whitespace matters too: a token representing “ check” is not necessarily represented like the token for “check” at the beginning of a string. These differences are small to a reader and decisive to a system trying to fit observed totals.
This is also why access to a tokenizer is useful but not equivalent to access to the answer. Major model providers often publish tokenizer implementations or compatible tools, so an attacker may be able to reproduce the target model’s segmentation rules. That lets them enumerate candidate partitions more accurately. It does not tell them which partition was used in a particular merged observation, and it does not solve the problem of selecting among different strings with the same total length. Tokenizer knowledge defines the space of possibilities; it does not collapse that space to one result.
PromptEcho describes an LLM response as a sequence of tokens, then models each merged observation as the sum of the character lengths of a contiguous span. If the response tokens are r1 through rm, an observed value tj represents the total length of some consecutive subset. In conceptual form:
observed merged length = length(token a) + length(token a+1) + … + length(token b)
The observer does not know a, b, or how many tokens lie between them. The grouping boundaries are hidden. One value can therefore be decomposed in several ways, and different decompositions can lead to different text. This is the source of the attack’s uncertainty, not a minor nuisance to be removed by better arithmetic.
Yet a constraint can still be informative. Language is not a random sequence of fragments. Spaces, punctuation, common openings, grammatical endings, and the relationship between neighboring phrases restrict which decompositions sound plausible. A length that could fit dozens of arbitrary strings may fit only a few natural continuations once the previous context is known. PromptEcho’s later inference stage relies on that narrowing effect, but the application layer must first produce a reliable sequence of observations for it to use.
Aggregation changes the granularity of the leakage. In a single-token stream, an observer may try to associate each transmission event with one token length. In a merged stream, a single event corresponds to the total length of several adjacent tokens. That hides the individual boundaries and makes direct lookup attacks less effective. It does not make the event content-independent. The chunk still carries some amount of generated text, and its size can still vary with the characters in that group.
This is why token aggregation is better understood as a transformation of the side channel than as its elimination. It removes fine detail while preserving a coarser constraint. The coarser signal may be harder to exploit and may degrade more quickly over long responses, but a language model can sometimes compensate for missing boundaries by using syntax and discourse regularities. PromptEcho is designed for precisely that intermediate regime: too much information for the stream to be meaningless, too little information for deterministic token-by-token recovery.
The distinction also matters when evaluating what has been learned. Recovering a plausible text whose token lengths fit an observation does not show that the victim received that exact text. Several candidates may satisfy the same length totals. A candidate can be useful for privacy analysis if it preserves the topic or a sensitive entity, even when it is not the original wording. Conversely, an exact match on a short generic phrase may say little about a sensitive payload. The length sequence is evidence for inference, not a transcript in disguise.
From service framing to a merged-length sequence
The response does not travel from the model directly into a bare TLS length field. A cloud API has to serialize events, add protocol metadata, buffer content, and hand the result to the transport stack. The PromptEcho paper describes real-time API responses commonly using HTTPS with SSE-style streaming or similar mechanisms. To reduce perceived latency, services send incremental chunks rather than waiting for the entire answer. The service may aggregate adjacent output tokens before sending one event or one protected record.
A useful model separates each observed unit into two parts: a relatively stable service-specific overhead and a variable portion carrying response content. The overhead can include framing and metadata around the content event. The variable portion changes with the generated output. If the service’s formatting is stable and it does not add enough padding or encoding to erase the relationship, subtracting the fixed component can yield an estimate of the merged content length.
That subtraction is not something an observer can assume from the TLS standard. It is a calibration problem. The paper describes observing the service through the attacker’s own permitted interactions, including cases where the plaintext or a known response is available to the calibrator, and then correlating those observations with encrypted traces. Repeated responses reveal recurring structures: a preamble before the first answer content, predictable service framing, and a small termination event at the end. Those structures help the observer decide where the response begins and ends and which part of a record is likely content-dependent.
The threat model is important here. Calibration does not mean decrypting the victim’s session. It means using ordinary access to the service to learn how that service packages responses. The observer can then apply a service-specific model to another encrypted flow. If the provider changes its headers, buffering, event format, padding, or aggregation policy, the old model may become inaccurate. Stable overhead is therefore an empirical condition, not a permanent feature of an API.
Transport segmentation adds another boundary. TLS records are carried on a continuous TCP byte stream, while TCP may split a large record into several segments or place multiple records into one segment. A capture that reports a sequence of TCP payload sizes can therefore mix transport effects with TLS record effects. The paper’s extraction procedure accounts for those relationships before deriving its merged-token length sequence. This is why “packet length” is a convenient shorthand, not a complete description of the measurement.
The same caution applies to the response preamble and terminator. The first bytes in a stream may represent session or HTTP metadata and the first streaming event rather than generated content. At the end, a final event and a transfer-termination marker may have a recognizable size that does not correspond to an output token. If those units are fed into reconstruction as if they were text, the model receives a false prefix or suffix. PromptEcho’s extraction stage treats boundary identification as part of the attack rather than as a free observation.
A service can also produce different observations from the same response text. If it flushes after every small event, the content is distributed across more records and the observer receives a finer sequence. If it buffers until several events are ready, the same text appears as fewer, larger groups. The underlying answer has not changed, but the hidden partition has. This is the application-level reason that the paper characterizes merged-token behavior per service rather than claiming that one universal chunk size exists.
Calibration is consequently a form of interface learning. The observer can send known prompts through their own account, compare the visible flow with the response they requested, and estimate recurring offsets or boundary patterns. They still cannot read a victim’s protected stream. They are learning how the provider tends to package content so that an unfamiliar encrypted flow can be interpreted in the same coordinate system. The approach is strongest when those patterns are stable across sessions and weakest when the service deliberately varies them or when connection reuse introduces additional fields.
The paper’s extraction method also has to decide which apparent regularities are content and which are protocol. A repeated first event might mark the transition from response headers to assistant content. A small final event might indicate completion rather than a short final token. A packet that is unusually large might reflect transport segmentation rather than a long generated phrase. Each mistaken classification shifts the derived length sequence, and later inference then has to work with a false constraint. This is why the apparently simple observation “the packet was this many bytes” hides substantial system-level reasoning.
The pipeline can be stated without pretending that any one layer reveals the answer:
The first transformation is semantic generation into model-specific tokens. The second combines neighboring tokens into a service-defined group. The third adds application framing. The fourth protects the result with TLS, adding the cryptographic and optional padding effects described earlier. The observer sees the outside of that stack. PromptEcho attempts to work backward only far enough to estimate the content-dependent component, not to reverse the encryption.
Every arrow introduces a possible mismatch. A tokenizer mismatch changes the candidate lengths. A buffering change changes where one group ends and another begins. Additional metadata changes the offset that must be subtracted. Padding makes the variable component less predictable. TCP loss or capture gaps remove part of the sequence. Multiplexed or otherwise shared traffic can make it harder to identify which records belong to the target response. A successful extraction method must therefore be tied to the service and observation conditions rather than described as a universal property of LLM traffic.
The paper’s measured services supplied an especially favorable combination for inference: recurring framing, merged output that still varied with content, and enough regularity to estimate service-specific offsets. That does not mean the signal is strong in every deployment. It means that the provider’s efficiency and responsiveness choices can create a stable interface between generated language and observable traffic. The same design choice that makes a stream feel immediate can also make its chunks more measurable.
There is a product reason for this behavior. Streaming reduces the time before a user sees the first part of an answer and avoids holding the entire response until generation finishes. Aggregating a few tokens can improve transmission efficiency while retaining a responsive experience. The security cost is that the stream remains fine-grained enough to expose an evolving length pattern. A service can choose larger aggregation or less frequent delivery, but that changes latency and interaction feel. The side channel is therefore connected to a genuine system trade-off, not merely an accidental formatting bug.
At the end of this layer, the observer has a sequence of estimated merged lengths, not a sequence of words. The sequence says how much tokenized content appears to have occupied each group, after the observer has made assumptions about framing and overhead. Its value comes from language structure, and its weakness comes from hidden boundaries and changing interfaces. PromptEcho’s next task is to make that underdetermined sequence usable: segment it, represent its granularity explicitly, and combine its constraints with learned linguistic priors.