The Message Has Two Audiences
This series examines Human-Perceptible Adversarial Attacks (HPAA), a framework for studying harmful meaning that remains recognizable to people while becoming harder for automated content moderation to detect. This opening focuses on the specific boundary that makes HPAA possible: the difference between a message as it is rendered for a human and the representation a detector receives. Understanding that boundary gives readers a way to judge the security of modern moderation systems without confusing this problem with jailbreaks, prompt injection, or ordinary text corruption.
The blind spot is not in the words alone
Content moderation is often described as a language-understanding task. A system receives a post or message, estimates whether it violates a policy, and blocks it, releases it, or sends it to a reviewer. That description is useful, but incomplete. People do not understand written communication from a bare sequence of lexical tokens alone. They also use grouping, spacing, emphasis, position, and other visual cues to decide what belongs together and what deserves attention.
Those cues are usually treated as presentation details. In a moderation pipeline, they can become security-relevant information. If a harmful expression is placed inside otherwise ordinary text and its visual arrangement helps a person reconstruct it, the recipient may understand an intent that is not represented with equal clarity in the detector's input. The detector has not necessarily “misread” the same visual scene. It may never have received that scene in a form it could analyze.
The distinction becomes clearer if we separate three things that are often collapsed into “the text.” First is the submitted material, which may include characters, spacing, and formatting choices. Second is the representation prepared for a detector, which may preserve some properties while reducing or reorganizing others. Third is the presentation a person receives after the platform displays the message. A system can make a sensible decision about the second representation and still fail to account for meaning carried by the third. That is not merely a question of whether the language model is intelligent enough. It is a question of whether the security boundary exposes the relevant information to the component responsible for screening it.
This also explains why a formatting change can be more consequential than a cosmetic change in a screenshot. Human readers routinely group nearby fragments, notice an emphasized region, and use spatial regularity to reconstruct a phrase. A detector that receives only a flattened sequence may not have an equivalent grouping signal. The issue is not that people possess a mysterious extra sense, or that models can never process visual information. The issue is that the moderation architecture may ask a text-oriented detector to make a decision before the system has represented what the platform is actually showing to its users.
That is the central claim behind HPAA. The attack does not primarily try to make an instruction-following model obey a forbidden request. A jailbreak changes model behavior by manipulating the instructions or conversational context. Prompt injection tries to make a model treat untrusted content as a new instruction. Conventional adversarial text attacks commonly alter characters, words, or syntax to induce a classification error while preserving some semantic content. HPAA instead targets the gate that decides whether harmful content reaches people, and it requires a split outcome: the human should still recover the harmful meaning while the automated moderator treats the submission as benign or substantially less suspicious.
This distinction changes what counts as success. A detector-only benchmark can report a low unsafe score and call the sample a successful evasion. HPAA asks a second question at the same time: would a human recipient recognize what the submission communicates? If the answer is no, the manipulation may be machine-resistant but not human-perceptible in the sense the framework studies. If the answer is yes, the moderation result represents a real gap between the two audiences for the message.
The phrase “two audiences” is not meant to suggest two different messages. It points to two processing paths. The same submission can be transformed by a platform into a rendered presentation for a person and a tokenized or otherwise reduced representation for an automated system. The security problem appears when those paths preserve different parts of the message's meaning.
The diagram is deliberately simpler than a production platform. It shows the relationship that matters: a human can receive a visually organized message even when the detector receives a sequential representation that does not carry the same signals with the same strength. HPAA treats that difference as an attack surface rather than as an accidental inconvenience in user-interface design.
There are two distinct ways for this boundary to fail. The first is representational: a formatting or spatial cue survives to the display but is discarded, flattened, or weakened before classification. The second is procedural: the detector may flag only what its policy or confidence threshold recognizes, while the platform's release path makes the unflagged presentation immediately available to a recipient. These failures can reinforce each other. A missing signal makes the classifier less sensitive; a low-confidence or benign decision then prevents the human review that might have supplied the missing interpretation.
The reverse case matters too. A platform may preserve a visual distinction that helps a detector, or it may normalize the message before both humans and models see it. That could reduce the particular HPAA channel, but it may also remove legitimate emphasis, language-specific shaping, or accessibility cues. The existence of a defense therefore cannot be judged only by whether it makes text look uniform. It must be judged by whether the security-relevant meaning remains available to the right component without making ordinary communication unusable.
Why a first-line filter changes the stakes
The USENIX Security paper places HPAA inside a familiar tiered moderation arrangement. A submitted post, comment, or message first encounters an automated moderator. The system may return a binary safe/unsafe decision or a continuous toxicity score. Cases with low confidence or unusual risk can be escalated to human review, but automation remains the scalable first screen.
That ordering matters. Suppose a person would recognize harmful intent in a rendered submission, but the automated gate assigns it a benign decision. The content can reach its intended audience before any reviewer sees it. The eventual human recipient is not a second defensive model in this path; they are the person exposed to the content. If the platform does send the case to a reviewer later, the failure may still increase review workload and delay an effective response. A downstream safeguard cannot automatically compensate for information that the first gate never represented or never flagged.
An automated decision is also not the same thing as ground truth. A score is an operational signal used to choose among actions under constraints: block, allow, hold, or escalate. Raising sensitivity can catch more harmful content, but it can also increase false positives and burden reviewers or legitimate users. Lowering sensitivity can reduce friction while creating more false negatives. HPAA is important because it targets the information available before that trade-off is even made. If the detector never receives the visual relationship that a human uses, changing the threshold may simply move the boundary around an incomplete signal rather than restore the missing evidence.
The paper's threat model makes this consequence concrete without assuming privileged access. The adversary is an ordinary platform user. They do not need the detector's parameters, training data, gradients, or internal moderation logic. They submit content through standard interfaces and observe whatever outcome the platform exposes, such as a safe/unsafe decision or a score. The relevant question is therefore not whether a laboratory attacker can break open a model. It is whether a public formatting environment and a black-box moderation gate leave enough room for the human view and detector view to diverge.
This is why query budget is part of the security story. Repeated attempts may be rate-limited, associated with an account, or routed to human review. A manipulation that succeeds only after extensive experimentation has a different deployment meaning from one that works in a single submission or a few trials. HPAA's later evaluation keeps those cases separate rather than treating all evasions as equally practical.
The same reasoning also explains why moderation robustness is a socio-technical property. The detector is only one component. The outcome depends on the submission interface, the formatting features the platform preserves, the transformations applied before classification, the escalation policy, and the way a recipient sees the released message. A stronger model may improve semantic classification while leaving a presentation channel untouched. Conversely, a platform may sanitize formatting so aggressively that the particular channel studied by HPAA disappears, at the cost of legitimate expressiveness or accessibility.
For an operator, this creates a concrete ownership question: which component is responsible for comparing what was submitted, what was analyzed, and what will be displayed? If those representations are maintained by separate services or libraries, a local guarantee can become a system-wide blind spot. The interface may promise one presentation, the classifier may receive another, and the audit log may retain only a third. HPAA does not require that every platform have this exact arrangement, but it gives a reason to inspect the crossings rather than treating the detector as the whole moderation system.
A boundary, not a universal verdict
The important lesson is not that every use of typography defeats every language model. The paper studies a defined setting: harmful expressions embedded in benign-looking text, ordinary-user black-box interaction, selected platform-compatible typographic cues, human recognition measured through user studies, and automated detection measured across evaluated systems. Its results are evidence about that combined condition. They are not a proof that all text systems, all fonts, all languages, or all rich-text pipelines behave alike.
That discipline matters when thinking about Unicode. Unicode Technical Standard #39 offers established mechanisms for identifier security, including restricted-character profiles, confusable detection, script analysis, and normalization-related checks. Those mechanisms are relevant because they show how a security system can reason about human-visible ambiguity without banning Unicode from ordinary writing. But UTS #39 is not a toxicity classifier, and identifier spoofing is not identical to a harmful phrase hidden in a formatted message. It will be useful later as a bounded defensive comparison, not as a ready-made answer to HPAA.
The paper also contains a scope detail that should remain visible. Its abstract and several evaluation or ethics passages refer to thirteen evaluated moderation systems, while the conclusion says ten and the prominently presented detection table shows ten named rows. The supplied first-party material does not resolve that discrepancy. A careful account therefore attaches each numerical claim to its table, figure, or summary rather than turning “many systems” into an unqualified universal statistic.
The opening conclusion is straightforward: moderation can fail at the boundary between what a platform renders and what its automated gate represents. HPAA gives that failure a name and a way to measure both sides of it. The next question is where, in the actual flow from submission to screening to human receipt, the two representations separate—and what the black-box threat model allows an attacker to learn from that separation.