Forecast Architecture Can Soften—or Amplify—the Miss

The Cleveland Fed comparison is not simply a contest between a sophisticated model and a simple one. It asks which representation of auto-loan performance is appropriate for a cumulative stress forecast, and whether a better representation of state movement can compensate for an unstable unemployment relationship. The useful decision is therefore two-dimensional: choose an architecture that follows the portfolio’s actual paths, then test whether the parameters governing those paths still travel to the period being forecast.

A default-only model leaves important paths outside the forecast

The simpler alternative is a binomial discrete-time hazard model. It estimates a one-period probability of default for a loan and a probability of not defaulting, then recursively builds a multi-period default probability from survival. That approach can be reasonable when the state space is simple: exits are rare, cures are minimal, and loans do not move through materially different pre-default conditions.

The Cleveland Fed data do not fit those simplifying conditions especially well. About 40 percent of loans that begin a period delinquent return to current, and more than 12 percent of loans that begin current exit without default. A current loan can also become delinquent before it defaults. Those movements change both the number of loans exposed to later default and the risk relationship that should govern them. A default-survival calculation that omits them is not merely less detailed. It can apply the wrong future state to the wrong population.

Suppose a loan starts current and becomes delinquent in the first forecast period. In a state-transition model, its next-period probability comes from the delinquent row, where the loan can cure, remain delinquent, default, or exit. In a binary survival model, the loan is still generally treated as part of the surviving non-default population and receives the binary default calculation that does not distinguish its recent delinquency. The two methods are now asking different questions about the same loan. One tracks its state; the other tracks only whether the absorbing event has happened yet.

The treatment of exits creates the opposite problem. If a current loan prepays or otherwise leaves without observed default, a Markov model removes it from future transition exposure through the absorbing exit state. A default-only survival model can continue to count that loan as surviving and expose it to later default probabilities. That makes the model’s future at-risk population too large. The error is not guaranteed to point in the same direction in every setting, because the frequency and timing of exits vary, but the omitted pathway is economically meaningful in this evidence.

The Markov model also respects competition among destinations. From current, the probabilities of staying current, becoming delinquent, defaulting, or exiting must sum to one. From delinquent, the probabilities of curing, remaining delinquent, defaulting, or exiting must do the same. A binary model can represent default versus non-default, but it does not allocate the non-default outcome among the state changes that determine what happens next. For a two-year forecast, that allocation affects the second-period risk set.

This is the first comparison criterion: state fidelity. A model should represent the transitions that materially change future exposure. More states do not automatically make a model better. If the additional states are poorly measured or irrelevant to the decision, they can add estimation burden without useful information. Here, however, the observed cure and exit rates make the state distinctions central to cumulative default forecasting. The Markov approach is more complete for these observed dynamics, not universally superior merely because it has more notation.