Unicode's Defensive Lens

Unicode Technical Standard #39 (UTS #39) is not a toxicity detector. It is a security standard for reducing ambiguity in identifiers, and that narrower purpose makes it useful for understanding what a representation-layer defense can and cannot do against HPAA. This chapter explains its profiles, normalization and context rules, confusable and script diagnostics, graduated restrictions, and data lifecycle, then compares those mechanisms with the human-visible moderation problem.

A security profile for identifiers, not a ban on Unicode

An identifier is a string used by an application to refer to a particular entity: a web address, an account name, a software symbol, or another name that is supposed to resolve unambiguously. The security concern is easy to state. Two code-point sequences can be different to a machine yet look similar enough to a person that the person mistakes one entity for another. UTS #39 is designed to reduce that ambiguity.

The standard's current page identifies Version 17.0.0, dated September 4, 2025, as an approved and stable document. Its approach is not to declare Unicode itself unsafe. Instead, it recommends profiles that limit which characters can appear in security-sensitive identifiers. Characters excluded from an identifier profile remain available for ordinary documents and communication. That separation is central: an application can protect a login name or domain label without treating every non-ASCII sentence as suspicious.

The General Security Profile divides character status into broad allowed and restricted outcomes. Restricted characters include, among other cases, unassigned or private-use characters, deprecated characters, default-ignorable characters, characters that do not fit normalization or identifier-continuation rules, and characters whose use is uncommon or specialized. Allowed characters include recommended characters from scripts in widespread everyday use, along with narrowly defined inclusions. The profile is applied together with the application's own identifier syntax, rather than used as a universal replacement for it.

This design makes policy ownership explicit. UTS #39 supplies character properties and recommended security behavior. The application decides which identifier syntax it supports, whether it documents exceptions, and how it handles a flagged value. An implementation may use the allowed set as a filter, add a limited exception for a language or protocol, or reject more than the default profile requires. The standard advises caution with restricted characters, but it does not say that every flagged identifier must be treated as proof of malicious intent.

A profile is consequently a filter layered over an existing contract, not a new definition of language. The application first has to know what counts as a valid identifier: which positions may contain letters, marks, digits, or protocol-specific punctuation. The security profile then narrows that candidate set. This ordering matters because a character can be unsuitable for one identifier syntax yet perfectly legitimate in another context, and a character that is allowed by the general profile may still be excluded by an application's own policy. The result is a documented intersection of rules, not an opaque list that users cannot challenge or understand.

Normalization is part of the same problem. Unicode can represent visually or canonically equivalent text through different sequences. UTS #39 relies on operations and properties such as canonical equivalence, NFKC or NFKD normalization, case folding, and the XID_Start and XID_Continue identifier properties. The purpose is to make security checks compare forms consistently and to keep characters that cannot participate safely in the relevant identifier syntax from silently passing through.

Canonical equivalence illustrates why “one character, one meaning” is too simple.