From Data to Model: The Trade-Offs Hidden in Evaluation
This chapter examines how a financial supervisory authority should evaluate an AI model before treating its output as a supervisory signal. The focus is the gap between technical performance and supervisory usefulness: a model can classify, predict, or summarize well on a test set and still be unsuitable for the decision it is meant to support. Readers should come away able to connect technical indicators to the original supervisory KPI, recognize generalization and bias risks, and judge explainability and human review in proportion to the materiality of the use.
Technical performance is not the supervisory result
A model is a way of converting prepared data into an output. The output might be a classification, a numeric estimate, a grouping, a ranked list, or a summary of text. The IMF paper describes clustering for grouping similar items, regression for predicting numeric outcomes, classification for assigning labels such as suspicious activity, time-series analysis for temporal patterns, and textual analysis for tasks such as document representation or summarization. These are different mechanisms. A model that groups similar providers is doing something different from one that predicts a probability of default or flags a passage in a regulatory filing.
The right choice begins with the supervisory problem. If an authority wants to prioritize documents for human review, a ranking or classification may be appropriate. If it wants to estimate a future risk, it needs a prediction whose error can be understood over time. If it wants to identify themes in complaints, text analysis may help reveal recurring topics, but a topic is not automatically a verified cause of harm. The model family matters because it determines what the output says and what it does not say.
That distinction should shape the evaluation plan from the beginning. The Project Foundation phase described by the IMF paper calls for business or supervisory key performance indicators (KPIs) as well as technical indicators. A KPI asks whether the authority’s work improved. In the paper’s fit-and-proper example, relevant measures include the time required for assessment, the accuracy of the review, the number of issues correctly identified, and the reduction of oversight errors. A technical indicator asks how the model behaves, using measures such as accuracy, precision, recall, or root mean squared error when they fit the task.
The measures answer related but different questions. Accuracy may summarize how often a classifier is correct, but it does not show whether the errors fall on the cases that matter most. Precision asks how many flagged cases are relevant; recall asks how many relevant cases the system finds. A system that produces many false alerts may burden supervisors even if it catches a substantial share of true issues. A system that produces very few alerts may be easy to use while missing risks that the authority intended to detect. The useful balance depends on the supervisory purpose and on the consequences of each error.
A technical score is therefore evidence about the model, not proof of an improved supervisory outcome. The model is a potential cause of a change in workflow; the output is a potential trigger for review; the authority’s judgment and action mediate any effect on a provider or market.