A risk model in production rarely fails in a way that is visible from one cycle to the next. The PD output for a given portfolio segment moves a few basis points. The LGD assumption holds within reasonable bounds. The macro overlay produces a number that is broadly consistent with prior periods. Nothing in the routine monitoring suggests the model has degraded.
What is much harder to see from cycle-to-cycle monitoring is the cumulative effect of small movements over time. Borrower behaviour in a given segment shifts as the underlying portfolio composition changes. The macroeconomic variables that drove the original calibration weaken in their predictive power as the regional economy evolves. Internal policy changes — new underwriting criteria, revised concentration limits, modified collection practices — accumulate in ways the model was not redesigned to absorb. Each adjustment is small. The aggregate effect, two or three years in, is a model that no longer represents the portfolio it is meant to measure.
This is model drift. It is not a failure of model design and it is not a failure of model monitoring as conventionally practised. It is a structural feature of running quantitative models against a dynamic portfolio in a dynamic economy. The question is not whether drift will occur. The question is whether the institution has the governance to detect it before the supervisor does.
Why developer-led validation does not address drift
The conventional response to model risk has been ongoing monitoring conducted by the team that built or maintains the model. This monitoring is necessary. It is also structurally limited in what it can detect.
The maintenance team’s interpretive frame is the part of the system most likely to drift in parallel with the model itself — gradually accepting outputs that an outside reviewer would flag, gradually rationalising movements that should have triggered recalibration.
This is not a failing of the maintenance team. It is the predictable consequence of asking the people closest to a model to be its primary critics. The supervisory frameworks that govern model risk — the CBUAE Model Management Standards prominently among them, but also SR 11-7 in the US context and the EBA guidelines in Europe — uniformly require independent validation precisely because developer-led monitoring cannot, by design, identify the assumptions the developer team has stopped questioning.
Article 4.6.5 adds that the majority of the Model Oversight Committee must not be drawn from business lines that benefit from the models being overseen. Article 4.7 sets governance around third-party providers, including the requirement for knowledge transfer to internal staff. The structural intent is consistent: model risk governance requires perspectives that the development and maintenance functions cannot, on their own, provide.
What an independent validation should actually cover
A validation exercise that genuinely tests model performance rather than confirming its existence tends to cover four areas in depth, each with both quantitative and qualitative components.
Conceptual soundness is the first. The reviewer asks whether the model’s underlying theory is appropriate for the portfolio it is being applied to, whether the variable selection is defensible, and whether the methodology choices made during development still make sense given how the portfolio has evolved. This is the part of validation that pure backtesting does not reach. A model can backtest within tolerance and still be conceptually unsuited to the current portfolio.
Data quality and representativeness is the second. The reviewer tests whether the data the model was calibrated on remains representative of the data the model is being applied to. For IFRS 9 models, this includes default rate stability across segments, vintage analysis, and assessment of whether the calibration window includes a sufficient range of economic conditions. For ICAAP and stress testing models, it includes coverage of the institution’s actual exposure profile against the data used to fit the model. The CBUAE MMS framework specifies six data quality dimensions in Article 5.4.1 — accuracy, completeness, consistency, timeliness, uniqueness, validity — each of which should be assessed against the actual portfolio.
Performance testing is the third. This is the area most often equated with validation as a whole, and it is necessary, but it is not sufficient. Backtesting, sensitivity analysis, benchmark comparison, and out-of-sample testing each surface different aspects of performance. A model can pass backtesting while failing sensitivity analysis, indicating that it produces correct outputs for the wrong reasons. A model can pass sensitivity analysis while failing benchmark comparison, indicating that its absolute level is questionable even if its relative behaviour is sound.
Governance is the fourth. The reviewer assesses whether the model’s documentation, version control, override governance, and reporting to oversight bodies meet the standards the institution and the supervisor require. Article 8.4 of the CBUAE MMS deals specifically with model overrides — these are permitted, but each must be documented, justified, time-bounded, and reviewed. A validation that does not examine the override register has missed one of the most operationally important parts of the framework.
The independence question, practically
Independence in model validation is sometimes treated as a binary — either the validator is internal independent (in a separate function from development) or external. The substantive question is less binary and more about the validator’s freedom to disagree.
A validator who reports through the same chain as the development team is unlikely to surface a finding that materially questions a major model the institution has built. A validator who reports independently but lacks the technical depth to challenge the development team’s assumptions will accept those assumptions by default. A validator who has the technical depth and the reporting independence but a long-standing professional relationship with the development team will, predictably, see what the relationship makes it harder not to see.
Each of these is a degree of independence. The CBUAE MMS framework is explicit that the validator must be functionally independent and technically capable, and that the third-party providers used for validation must transfer knowledge in a way that builds internal capacity rather than creating ongoing dependency. The supervisory intent is that the institution should be able to demonstrate, on a multi-year horizon, that its model validation function has both the structural independence and the technical depth to challenge its own model framework.
What this means in practice
The institutions whose model risk frameworks hold up under thematic review and supervisory dialogue tend to share several characteristics. The validation function reports through a chain that is independent of model development. The validation work is documented in a way that allows a third reviewer — typically the audit function or the supervisor — to retrace the validator’s logic. The findings of validation are tracked through to remediation, with explicit governance on closure. The validation cycle covers all material models on a defined frequency, with the frequency itself defensible against the model’s tier and the materiality of its outputs.
What is rarely the issue is the existence of validation. What more often is the issue is whether the validation that has been conducted would survive critical examination by an outside reviewer. The question worth asking, periodically, is whether the institution’s own validation function would identify drift in a model it has been validating for several cycles. If the answer is uncertain, the case for periodic external validation is structurally a strong one.
A closing observation
These are not findings that a maintenance cycle surfaces. They are findings that an independent perspective is structurally positioned to see. That is the case for validation as a discipline, conducted by people whose interpretive frame is not the same as the developer’s.