Data Is Part of the Supervisory Instrument
This chapter examines the data system behind an AI project run by a financial supervisory authority. The question is not simply whether an authority has enough information to train a model; it is whether the information is fit for a particular supervisory purpose, lawfully obtained, traceable, representative, and reliable enough for people to act on its output. That judgment determines whether an AI system extends supervision or gives an appearance of precision to an evidence gap.
Why regulatory reporting remains the foundation
For market-conduct supervision, CGAP describes regulatory reporting as the usual baseline for market monitoring. Reports submitted by financial-services providers give the supervisor structured fields that can be uploaded, stored, compared, and reviewed over time. They may support indicators such as complaints, product activity, or other signs of consumer risk. Because repeated observations enable benchmarking and trend analysis, the reporting system is not just an input to a dashboard. It is part of how the authority defines what can be seen consistently across providers and periods.
That foundation is easy to undervalue when advanced analytics are available. A new model may promise to identify patterns in a much larger dataset, but it cannot decide whether the underlying fields use consistent definitions, whether missing values cluster around particular providers, or whether a change in a metric reflects market behavior rather than a change in reporting practice. If those questions remain unresolved, the model can make the problem harder to notice. Its output may be numerically tidy while the comparison it implies is not fair.
CGAP therefore advises supervisors to fix quality issues in regulatory reporting before investing in complementary tools, especially when resources are scarce. Suptech can help improve reporting through better interfaces, machine-readable reporting taxonomies, secure channels, and automatic validation and transformation. But the value of that technology depends on the reporting objective and on the capacities of both the authority and the providers submitting data. A tool that reduces manual entry but preserves ambiguous definitions has improved a process without necessarily improving the evidence.
This is also why there is no single best market-monitoring tool. CGAP says the mix and frequency should depend on the supervisor’s objective, budget, risk severity, data quality, and analytical capacity. Regulatory reports may be analyzed frequently, while consumer surveys or mystery shopping may be used less often because they cost more or answer different questions. One source can reveal the scale of a pattern; another can explain how consumers experience it. A market-monitoring system becomes stronger by combining sources that cover different weaknesses, not by declaring one technology universally superior.
The distinction between structured and unstructured information matters at this point. Structured data arrives in predetermined fields and can be checked against formats or rules. Unstructured information includes the words in a complaint, the clauses of a consumer contract, a marketing message, a regulatory filing, or a board meeting record. Those materials may contain the detail needed to understand conduct or governance, but they do not arrive ready for comparison. The authority must first collect, organize, interpret, and sometimes label them.
Suptech can support that work. CGAP describes web scraping and social-media monitoring as ways to collect publicly available material and convert parts of it into structured information. Natural-language processing can scan large volumes of text for terms, topics, patterns, or recurring themes. Such tools can make it possible to examine more than the small samples a manual process can handle. They do not, however, turn every public statement into a representative measure of consumer experience. Social-media users may not reflect all consumer groups, and the data can be noisy or manipulated.
The source type therefore changes the meaning of the output. A missing value in a regulatory report may indicate a reporting failure, a provider with a different process, or the absence of the underlying event. A complaint narrative is evidence of a reported experience, not necessarily a measure of how often that experience occurs across the market. A contract-scanning system can identify language that deserves review, but a keyword match does not by itself establish that a term is unlawful or harmful. The supervisory question must travel with the data through collection and preparation.
The diagram shows interfaces rather than a required software architecture. Information from different sources has to cross a collection boundary, then a governance and quality boundary, before it becomes evidence that can be interpreted. A failure at one boundary can travel downstream. The more polished the final output, the more important it is to preserve visibility of what happened before the model saw the data.
Consider what happens when an authority combines complaints with provider reports to look for a conduct problem. The complaint may provide a detailed account of a consumer’s experience, while the report may provide the number of cases or the provider’s own classification. Those records answer different questions. The first can help explain what happened; the second can help estimate how broadly the issue appears in the provider’s records. If the authority treats them as interchangeable, it may either overstate the prevalence of one vivid complaint or overlook a problem because the provider’s categories do not capture the language consumers use.
Linking the sources does not solve that problem automatically. The authority needs a stable way to identify the provider, product, time period, and relevant event. It needs to record whether a complaint is new, duplicated, or connected to an earlier case. It needs to know whether a provider changed its reporting template, whether a reporting gap occurred during a system outage, and whether a new collection method began halfway through the comparison period. These are not low-level technical details when they can change the apparent trend. They are facts about the evidence itself.
The same interface appears in document analysis. A natural-language system may scan contracts or marketing material and produce a label such as “potentially concerning.” That label can help a supervisor prioritize reading, but its meaning depends on the text selected, the terms used to train or tune the system, and the review standard applied by staff. A phrase that is common in one product or jurisdiction may have a different significance in another. If the authority cannot preserve the document, collection date, relevant version, and reason for the label, it will have difficulty explaining why a provider was selected for further review.
This is why a data catalog or metadata record matters to a supervisory decision. It is not merely a directory of files. It can show the source, owner, time range, definition, transformation, access condition, and known limitation of a field or document. Those records help a reviewer decide whether two observations are comparable and help an auditor reconstruct how an output was produced. The IMF paper recommends inventories, registries, and metadata practices for this broader accountability purpose. The exact design will vary by authority, but the function is stable: preserve enough context that a result can be questioned.
There is also a difference between detecting a gap and proving its consequence. Suppose the reporting system contains fewer records for smaller providers. The gap is evidence that coverage is uneven; it is not, by itself, proof that smaller providers have lower or higher consumer risk. The authority may need to investigate why the records are missing and whether the missing group would change the ranking or escalation decision. Treating a missing group as an ordinary value can hide the very population the authority is meant to protect.
These distinctions keep the chain connected to supervisory purpose. Collection asks what can be observed. Integration asks how observations relate. Governance asks who is responsible and what restrictions apply. Quality and coverage checks ask whether the result can support a comparison. Only then can an AI system help prioritize interpretation. If the authority skips an interface, it may still generate an output, but it will not know how much confidence or authority that output deserves.
Governance determines what the evidence can mean
The IMF paper calls data governance the bedrock of successful AI deployment. It describes data governance as the policies, processes, and activities that keep data secure, private, accurate, available, and usable across its lifecycle—from acquisition and use to disposal. That definition makes ownership and metadata practical rather than administrative. Someone must be able to say where a field came from, which version was used, how it was changed, who may use it, and what limitations could affect the conclusion.
A useful way to see the issue is to follow a single proposed indicator. Suppose an authority wants to identify signs of misleading marketing across financial providers. It first needs a clear definition of the conduct it is looking for and the population it intends to cover. It then needs access to the relevant advertisements or filings, a record of when and where they were collected, and a way to distinguish duplicate, outdated, or altered material. If the authority combines those items with complaints or provider reports, it needs to know how the sources relate. Otherwise, the model may count the same signal several times or treat a change in collection as a change in conduct.
The IMF paper warns that relevant data often exists in silos. A supervisory authority may keep provider information, complaints, inspections, licensing records, and market data in separate systems with different owners and identifiers. Siloed data can produce bias in model outcomes because the records available for one use do not represent the full problem. A unified repository may make access, stewardship, ethics, ownership, and metadata easier to manage, but the source does not claim that centralization alone removes bias. Governance is a set of responsibilities and controls, not a storage location.
Quality has several dimensions. Data may be incomplete, contain typographical or measurement errors, use inconsistent codes, or carry poor metadata. It may be technically accurate but too coarse for the supervisory question. It may be detailed yet unrepresentative because certain providers, products, regions, or consumer groups are missing. It may also be redundant: different reports can describe the same fact through different templates, creating the impression of a larger evidence base than actually exists. Cleaning a dataset can remove obvious errors, but it cannot decide whether the population it represents is the population the authority needs to supervise.
Representativeness becomes especially important when a supervisor uses alternative or public data. CGAP notes that some people do not post on social media, lack convenient digital access, or prefer other channels. A monitoring system based on online activity can therefore overlook consumers whose risks are less visible online. The fact that the system observes a group continuously does not show that it observes the market fairly. An authority must ask who is absent and whether the absence could change the action suggested by the output.
External data also raises questions of authority and privacy. The IMF paper notes that financial supervisory authorities may regularly collect personal data without explicit consumer consent when exercising supervisory powers, but an AI project can create new uses or combine information from non-traditional sources such as devices, smartphones, or web-tracking systems. Any such use still depends on applicable privacy and data-protection rules. People may not expect their information to be used for a particular inference. If a third party develops the model, the authority may also have less ability to verify how data was collected, whether it was used for the agreed purpose, or whether it can be traced back to its source.
Privacy techniques can reduce some exposure but do not erase the underlying responsibility. Anonymization removes direct identifiers; pseudonymization separates identifying information and protects the additional information needed to reconnect it; data masking changes confidential values while trying to preserve useful structure. The IMF paper presents these as techniques that may help authorities comply with data-protection requirements and work with external developers. They are not a blanket permission to collect any data or a guarantee that people cannot be reidentified. The legality and appropriateness of a use remain tied to its purpose, context, and applicable rules.
The same principle applies to data supplied by vendors. Before relying on a third-party dataset or model, an authority needs to understand its sources, ownership, quality, traceability, learning method, possible biases, level of autonomy, and human oversight. A contract can allocate tasks, but it cannot make an unobservable dataset representative or make an unexplained output accountable by itself. If the authority cannot reconstruct enough of the system to challenge its result, its apparent convenience may become a supervisory dependency.
These concerns explain why data preparation is more than tidying a file before modeling. The IMF paper describes selection, cleaning, feature engineering, integration, and formatting as activities that shape the dataset. Decisions about missing values, noisy observations, inconsistent entries, and combined attributes can change what the model learns. Those decisions should be documented because later inconsistencies may otherwise look like model failures when they began as undocumented preparation choices.
A data strategy should consequently answer a sequencing question: what foundation would most improve the authority’s ability to supervise? For one authority, that may be better reporting interfaces and standardized definitions. For another, it may be a controlled way to link complaints with provider or product information. A third may need to improve access controls, metadata, or staff capability before collecting more sources. CGAP’s guidance allows gradual expansion as resources become available, but warns that suptech investment can divert scarce resources from basic needs. The right first investment is the one that closes the most consequential evidence gap, not necessarily the one with the most advanced label.
When the data foundation is credible, an AI model can extend coverage or reduce laborious analysis. When it is not, the model’s output remains bounded by what the authority can establish about the data. The next question is therefore not whether a model can find a pattern, but how the authority should test that pattern, its errors, its fairness, and its explanations before treating it as a supervisory signal.