The Right Job for a Supervisory LLM

This series examines a specific use of artificial intelligence in financial supervision: the BIS procedure that compares capital rules with the prospectuses of Additional Tier 1 (AT1) instruments. The immediate question is not whether a large language model can pronounce a document “compliant.” It is whether the model can help supervisors find and rank wording differences that deserve scarce expert attention, while leaving the legal and regulatory judgment to people with the relevant authority and context. That distinction gives readers a practical way to assess both the promise and the limits of supervisory AI.

The stakes are easy to miss because the input looks like ordinary text. A regulation may be hundreds of pages long; a prospectus may appear to restate it in more accessible contractual language. Yet a small qualification in the contract can change when losses are absorbed, who bears them first, or whether an authority’s intervention activates a write-down. The model’s usefulness therefore depends less on producing an impressive summary than on preserving the relationship between the governing rule, the contractual term, and the expert decision that follows.

A supervisory review is a two-text problem

When a bank proposes an instrument for regulatory capital, supervisors compare two documents that play different roles. The regulatory package states the conditions the instrument must satisfy. The prospectus states the contractual terms on which the instrument is issued: how it pays, how it ranks, and how it absorbs losses. The texts will not normally be identical. A prospectus has to express a rule in a contract, add operational detail, and sometimes reflect discretion that the rule allows. Difference is therefore expected. The supervisory problem is to identify the less common difference that may alter how the rule operates.

AT1 instruments make that problem consequential. In normal conditions, they can resemble subordinated debt, paying discretionary, non-cumulative coupons. Under stress, they can support the issuing bank in two ways: coupons can be cancelled to conserve capital, and the instrument can be written down or converted into common equity after a contractual trigger or a point of non-viability. Four groups of terms matter to that process. Mechanical triggers set a capital threshold. Point-of-non-viability provisions describe when an authority may require loss absorption. Coupon terms govern cancellation without default. Hierarchy determines which form of capital absorbs losses first.

A comparison that misses a qualifying phrase in one of those provisions is not merely missing a stylistic variation. It may be overlooking a condition that becomes important only when a bank is close to failure. Conversely, a difference that looks alarming in isolation may simply be the prospectus supplying detail that the rule leaves open. The reviewer must hold both possibilities at once.

That is why the task cannot be reduced to checking whether the same words appear in both documents. A supervisor has to recover which element of the rule the contractual clause addresses, whether the clause merely makes an implicit condition explicit, and whether it adds a limitation that changes the instrument’s operation. The surrounding provisions may matter as much as the sentence that first attracts attention. The BIS procedure is useful precisely because it can place corresponding passages beside one another and identify the possible relationship. It does not remove the need to read the relevant package as a whole; it makes that reading more focused.

The model’s output should be a candidate, not a verdict

The BIS procedure is designed around that ambiguity. An analyst supplies the applicable regulatory documents, the prospectus, and a structured prompt. The LLM identifies provisions addressing the same requirement, quotes the relevant passages, and flags wording in the prospectus that may alter the requirement’s scope, conditions, or consequences. The output is a set of candidate divergences linked to the texts that produced them.

That output is deliberately weaker than a finding of non-compliance. A candidate may narrow a requirement by adding a condition, expand it beyond the circumstances covered by the rule, omit a trigger or procedural step, leave the relationship ambiguous, or contradict the rule’s conditions or sequencing. Those labels describe how two texts differ. They do not establish that the contractual wording breaches the applicable rule. Whether the variation is permitted, material, or actionable remains for supervisory and legal experts to determine.

The distinction is not a ceremonial disclaimer. A legal conclusion depends on more than semantic similarity. It can depend on the jurisdiction, the rule’s authority, its effective date, exceptions, the facts surrounding issuance, later amendments, and the procedure through which a supervisor or court can act. A language model may help locate the relevant passages, but it does not acquire decision-making authority merely by generating a plausible explanation of them.

Human review is therefore part of the mechanism, not a final stamp placed on a completed machine decision. The reviewer may find that an apparent narrowing is authorised elsewhere, that an omission is immaterial in the circumstances, or that an ambiguity warrants legal escalation rather than a compliance finding. To make those judgments responsibly, they need the source passages, the relevant rule context, and a record of how the candidate was produced. A fluent label without that trail would be difficult to challenge and even harder to defend after an event.

flowchart LR A["Regulatory package"] --> C["LLM comparison"] B["AT1 prospectus"] --> C C --> D["Candidate divergence with paired passages"] D --> E["Ranked review queue"] E --> F["Supervisory and legal expert judgment"]

The diagram captures the central boundary. The model sits between the documents and the queue, not between the queue and the institution’s authority to decide. Its contribution is to make relevant evidence easier to find and compare. The expert’s contribution is to interpret that evidence within the applicable rule and decide what, if anything, should happen next.

Ranking is a different kind of power

It may seem that a system which does not decide the answer is doing only a modest job. In a supervisory setting, that modesty is a source of value. Rules and prospectuses are voluminous, and expert time is limited. Starting with an undifferentiated mass of text forces reviewers to spend attention locating possible issues before they can assess them. A ranked queue changes the starting point: recurrent clause pairs and the passages beside them come first, while the reviewer retains the ability to inspect the wider documents and reject the model’s framing.

This is a form of institutional power, but it is agenda-setting power rather than adjudicative power. What appears near the top of a queue is more likely to receive attention; what is ranked low may wait. That makes the ranking consequential even when it is not a legal conclusion. A responsible design must therefore ask two separate questions. Does the procedure bring useful candidates forward? And does it fail to surface material differences often enough that the remaining review becomes unsafe? A model can be useful on the first question while still requiring safeguards for the second.

The queue can also shape what an institution thinks is worth investigating. If reviewers see only the highest-ranked candidates, recurrent wording patterns may become a proxy for importance, even though a rare clause can matter greatly in an unusual fact pattern. The answer is not to discard ranking; it is to keep ranking visibly provisional. Experts need a route to inspect lower-ranked or independently identified issues, and the institution needs evidence about what the queue tends to miss. Prioritisation is defensible only when it improves attention without quietly defining the whole field of attention.

The BIS evidence makes this separation concrete. In its large-scale exercise, the procedure identified recurrent candidate differences across AT1 prospectuses from European global systemically important banks. In its historical tests, it recovered wording differences associated with the Credit Suisse and Yes Bank cases. Those results show why early attention to contractual language can matter. They do not show that recurrence equals legal significance, that every material difference will be ranked highly, or that the model can settle a contested interpretation.

Why the boundary matters beyond one comparison

The argument also belongs to a wider conversation about responsible AI in finance. The Financial Stability Board’s 2026 consultation report proposes 12 sound practices covering organisation-wide governance, AI lifecycle management, cyber and information-technology risk, and third-party risk. The report presents them as a non-prescriptive, non-exhaustive menu rather than a new international standard or a mandate to adopt a particular technology. It also stresses proportionality: controls should reflect the institution, the criticality of the function, and the materiality and risk of the use case.

That framework sharpens the question posed by the BIS procedure. A supervisory LLM should not be judged only by whether it can compare language. Its use must have a defined purpose, clear accountability, appropriate data controls, performance testing, meaningful human oversight, and a way to respond when the model, the documents, or the surrounding conditions change. Those requirements are not external decoration. They determine whether a ranked queue remains a tool for better judgment or quietly becomes an unexamined substitute for judgment.

The FSB’s distinction between materiality and risk is especially useful here. A screening queue may not itself approve a transaction or impose a loss, yet it can still influence a critical supervisory process, handle confidential documents, or become difficult to replace if a provider fails. Its safeguards should reflect both what the queue affects and how it can fail. A low-autonomy comparison task may need less control than an agent that can alter systems, but it does not follow that the comparison task is consequence-free. Proportionality means matching controls to the actual pathway from model output to institutional action.

The two historical cases explain why this matters over time. A qualifier in a prospectus can remain dormant until a bank receives extraordinary support. A sequencing term can continue to govern a legacy bond after the regulatory framework changes. In both situations, the relevant question is not simply what the model said at the time of screening. It is whether the institution preserved the source passages, understood the limits of the comparison, and knew when the use case needed to be revisited.

The standard this series will build

The rest of this series follows one connected line of reasoning. It first makes the comparison task precise: what counts as a corresponding provision, what the divergence labels mean, and why paired passages matter. It then examines how repeated model runs turn variable outputs into a ranked queue and tests that queue against the AT1 evidence. After the cases establish the practical stakes, the analysis turns to the conditions around the model: governance, materiality, data lineage, explainability, performance, human oversight, cyber risk, and third-party dependence. The final question is when those conditions are strong enough to support bounded adoption and when the safer answer is to narrow, re-test, or stop the use.

The practical conclusion at the opening is therefore simple but demanding. A supervisory LLM earns its place by reducing the cost of finding relevant differences without reducing the visibility of human responsibility. The right test is not whether it sounds like an expert. It is whether an accountable expert can see what the system compared, why the candidate was ranked, what remains uncertain, and what authority is required to act. Understanding how the two texts are compared is the next step in applying that test.

References