Case Study 15.2: Cornerstone's IFRS 9 Model Validation Crisis — When PSI Tells You the Model Is Gone

Background: The Comfortable Complacency of 2019

Cornerstone Financial Group is a composite multi-jurisdictional institution — fictional but drawn from the experience of several large European banks — with operations across the UK, Germany, the Netherlands, and Ireland. By total assets, Cornerstone ranks in the upper tier of European banks not classified as globally systemic. Its IFRS 9 implementation programme, completed in 2018, had been expensive, complex, and ultimately regarded as a success. A project that had occupied 200+ staff and consumed £45 million in implementation spend had delivered, in the view of the board risk committee, a provision framework that was "forward-looking, appropriately conservative, and fully compliant."

The IFRS 9 models at the centre of this case study were built for Cornerstone's retail consumer finance portfolios: personal loans, credit cards, and overdrafts — approximately 4.2 million accounts across four jurisdictions, with a combined gross lending balance of £28 billion.

The models had been built by an internal analytics team with assistance from a specialist external firm. At go-live, the validation team had documented:

• Gini coefficients of 0.52–0.61 across the four retail segments
• PSI at 0.03–0.06 (population very stable, as expected — models were built on recent data)
• Calibration was within 10% of observed default rates for all segments
• SICR staging models had been separately validated

Between 2018 and the end of 2019, the models performed well. Quarterly monitoring reports showed stable metrics. Annual validations were signed off with minor findings. The board received model risk reports noting that "credit models remain within acceptable performance parameters."

The Unasked Question

Rafael Torres, who was at this time VP Compliance Technology at Meridian Capital rather than the independent consultant he would later become, had a conversation at a banking industry conference in January 2020 that he would later describe as "the moment I understood what model governance actually meant."

He was speaking with a chief model validation officer from a large European bank about IFRS 9 implementation. The officer mentioned that his bank had begun building refreshed IFRS 9 models in 2019 — two years after go-live — because data had continued to accumulate and the economic environment had shifted enough to warrant a rebuild.

"How often does your bank schedule full model redevelopments?" Rafael had asked.

"For the IFRS 9 ECL models? We treat it like any major production system. Ongoing maintenance, recalibration annually, full rebuild every three to four years."

Rafael had returned to his desk and looked at Meridian's model governance calendar. The IFRS 9 models were scheduled for "validation" in 2021, and "redevelopment, if required," at some unspecified future date. No one had asked the question: when the economic environment changes materially, what happens to models built on 2015–2018 data?

The answer arrived, as these answers sometimes do, through events rather than foresight.

March 2020: The Distribution Shift Nobody Had Modelled

When COVID-19 lockdowns were announced across Europe in March 2020, Cornerstone's credit risk team was focused on a different problem: government payment holiday schemes. IFRS 9 guidance was evolving in real time. The EBA, FCA, and ECB all issued rapid guidance on whether COVID payment holidays triggered SICR and Stage 2 migration. Cornerstone's IFRS 9 models became almost secondary to the policy question of how to classify 340,000 accounts on government-mandated payment deferrals.

What the credit risk team did not do — in fairness, what very few banks did systematically in the crisis period — was run a comprehensive PSI analysis on the input variables driving their ECL models.

Had they done so, they would have seen what the external validator found eighteen months later:

• Unemployment rate (a macro variable in the staging model): shifted dramatically — UK unemployment peaked at 5.2% in Q4 2020, from 3.8% at model calibration date
• Debt-to-income ratio distribution across the personal loan portfolio: shifted significantly as government support payments temporarily inflated household income measures
• Bureau utilisation rates: fell as credit card spending contracted and balances were paid down (some borrowers with high utilisation — normally a strong default predictor — appeared low-utilisation during lockdown due to government support)
• SICR migration rates: payment holiday accounts created an artificial pattern break — accounts were technically current but economically stressed

The models had been built on 2015–2018 data. They were being applied to a 2020–2021 population that, on multiple key dimensions, looked nothing like the development population.

February 2022: The External Validation

Cornerstone's audit committee had approved an external model validation programme in late 2021, covering the IFRS 9 retail ECL models as the highest-priority engagement. The validation was conducted by a specialist credit risk model consulting firm over four months.

The lead validator's report, delivered in February 2022, contained a section titled: "Population Stability Assessment — Summary of Findings." Its opening paragraph read:

"PSI values computed across all four retail portfolio segments (personal loans, credit cards, overdrafts — UK and ROI) indicate that the scoring population as of Q3 2021 is materially different from the development population (2015–2018) on multiple key dimensions. Across all retail segments, PSI values for the composite scorecard output exceed 0.25. PSI values for individual input variables range from 0.09 to 0.41. On the basis of these findings, the validation team cannot confirm that the current models are producing reliable estimates of credit risk for the current population."

PSI results by segment:

The validation report's overall conclusion:

"The external validation team recommends formal withdrawal of all four retail IFRS 9 ECL models from production use pending full redevelopment on current, representative data. Provisions calculated using the current models are likely to be materially unreliable. The models should not be relied upon without substantial expert judgment overlay and board disclosure of model limitations."

The Management Response: Resistance and Reality

The external validator's recommendation arrived in the IFRS 9 Working Group — a committee of the CFO, CRO, Chief Accounting Officer, and Group Head of Credit Risk — on 14 February 2022.

The response was not what the validator had expected.

"Model withdrawal is not operationally achievable," said Cornerstone's Chief Accounting Officer, Annelise Vogel. "We publish half-year accounts in August. Full redevelopment cannot be completed before then. Our provision calculation system is architected around these specific model outputs. Replacing the models requires a system change that goes through a six-month change management cycle."

"The PRA and DNB will ask what provisions are based on if the models are withdrawn," said the Group CRO. "Saying 'expert judgment' without any systematic framework invites more questions than it answers."

Rafael, who was by this point a board-commissioned independent adviser rather than an employee, understood both positions. The validator was technically correct: the models were not fit for purpose by any statistical standard. Management was practically correct: withdrawing the models without a replacement created a different kind of problem — a provision framework with no systematic quantitative underpinning, which was neither auditable nor regulatorily defensible in its own right.

This was the model validation crisis in its purest form: the model is wrong, but stopping using it creates a different problem.

The Resolution: Overlay Architecture

After three weeks of negotiation between the working group, the external validator, the external auditor (who had been briefed on the PSI findings and whose sign-off on the annual accounts was non-negotiable), and informal engagement with the PRA through the bank's relationship supervisors, Cornerstone adopted an overlay approach.

The overlay approach had two components:

Component 1: Model Output Recalibration Overlays

Rather than withdrawing the models entirely, Cornerstone's credit risk team developed a set of calibration overlays — adjustments applied to the model's raw ECL output — designed to compensate for the known distributional shifts.

For each segment, the overlay was developed by:

1. Computing the model's predicted default rate for the current population
2. Estimating the "true" default rate for the current population using post-pandemic actual default data (available for 2020–2021 cohorts) and expert judgment benchmarks from industry data
3. Computing a scalar adjustment: Overlay Multiplier = Estimated True Rate / Model Predicted Rate

For the UK Personal Loans segment: - Model predicted 12-month PD: 1.8% (for the Stage 1 portfolio) - Estimated actual 12-month PD (based on emerging post-pandemic data): 2.6% - Overlay multiplier: 2.6 / 1.8 = 1.44

All ECL outputs for this segment were multiplied by 1.44 pending model redevelopment.

Component 2: Qualitative Overlays (Post-Model Adjustment)

For variables where the distributional shift was most pronounced (particularly bureau utilisation, which had artificially fallen during lockdown), a qualitative overlay was applied: the model's utilisation-related contribution was replaced with an expert judgment adjustment informed by more recent utilisation data.

This overlay was documented as a formal model assumption, reviewed and approved by the Model Validation Committee, disclosed in the notes to the financial statements as a material model limitation, and subject to quarterly review.

Documentation Requirements for Overlay Approach

The working group agreed that the overlay approach required, as a minimum:

• A formal Overlay Governance Policy approved by the board
• Disclosure in the annual report under IFRS 7 that ECL models had been supplemented by expert judgment overlays pending redevelopment, with the reason and approximate financial impact (material provision movements)
• Quarterly review by the Model Validation Committee against emerging actual default data
• Automatic sunset: the overlays would expire when redeveloped models went live
• An updated model risk log entry for each affected model: "Classification: Model Impairment — In-Use Pending Redevelopment"

The Redevelopment Programme

Cornerstone committed to a full IFRS 9 model redevelopment programme, with a target of new models in production by Q1 2023. The programme was led by the Group Head of Credit Risk with external specialist support, and included:

Data infrastructure: Incorporating 2020–2022 default data — the first time post-pandemic credit behavior was available at scale. Development samples were constructed to include at least two years of post-lockdown data.

Variable review: Bureau utilisation was retained but supplemented with a utilisation trend variable (capturing change in utilisation over six months, not just the point-in-time level). Employment status proxies were replaced with more granular income source classifications available from updated bureau feeds.

Macro satellite models: A separate macroeconomic scenario model was developed, updated quarterly using Bank of England, ECB, and Central Bank of Ireland published forecasts, replacing the models' original built-in macro assumptions (which reflected 2018 conditions).

SICR model refresh: The staging model thresholds were recalibrated using post-pandemic migration data — the COVID period had revealed that some staging triggers were too sensitive (firing on payment holidays that were not indicative of genuine credit deterioration) and others were insufficiently sensitive (missing early warning signs in specific sectors).

New models were delivered to production in February 2023, approximately twelve months after the external validation finding.

What the PSI Data Had Really Been Saying (And When)

One of the most uncomfortable insights from the post-mortem was the timeline of PSI evolution. When the analytics team reconstructed PSI calculations quarterly from 2018 onwards, the picture was clear:

The PSI had crossed the "investigate" threshold (0.10) by mid-2020. Had monthly or quarterly PSI monitoring been in place for the IFRS 9 models — with the governance trigger to escalate PSI above 0.10 to model risk management — Cornerstone would have had a head start of eighteen months on identifying the problem.

Instead, the quarterly monitoring reports produced during 2020 and 2021 had focused on Gini coefficient (which had declined modestly — from 0.58 to 0.51 for UK Personal Loans — but remained above the minimum threshold). PSI had not been part of the standard monitoring pack.

The external validator's report contained a blunt observation: "A monitoring framework that tracks discrimination metrics but not population stability metrics is like a driver who checks their speedometer but not their fuel gauge. Both are necessary. The car stops without fuel regardless of its speed."

Regulatory Outcome and Lessons

Cornerstone's PRA relationship supervisors were briefed informally on the situation in March 2022 and formally in the subsequent supervisory dialogue meeting. The PRA's response, documented in a supervisory letter, noted:

• Cornerstone had self-identified the issue through appropriate external validation governance
• The overlay approach was an acceptable interim measure provided the disclosure and governance requirements were met
• The PRA expected to see evidence of updated ICAAP Pillar 2A capital add-ons to reflect model risk while models were in overlay
• The bank was expected to demonstrate that the revised monitoring framework — including quarterly PSI for all material models — was operational within 90 days

No formal enforcement action was taken. However, the incident was referenced in the PRA's 2023 thematic review of IFRS 9 model governance, which noted that several UK banks had "not maintained adequate population stability monitoring for their IFRS 9 models during the post-pandemic transition period."

Key Lessons for Practitioners

1. IFRS 9 models require active, ongoing governance — not just initial validation. The 2018 implementation generation of IFRS 9 models at many UK banks was treated as a project with a delivery date, not as a long-term model management obligation. Models built on pre-pandemic data were applied, largely unreflected upon, through the most significant economic disruption since the 2008 financial crisis.

2. PSI is not optional monitoring — it is the early warning system. PSI above 0.10 is not a crisis indicator; it is an early warning requiring investigation. Had Cornerstone's monitoring framework included PSI as a standard quarterly metric in 2020, the external validator's findings in 2022 would have been anticipated at least 18 months earlier.

3. The overlay approach is legitimate — but it has conditions. Using expert judgment overlays to compensate for model limitations is not a sign of failure; it is a well-established practice in model risk management. The conditions are: (a) the overlay is formally documented and governed, (b) it is disclosed appropriately in financial statements, (c) it has a defined sunset date, and (d) model redevelopment proceeds in parallel.

4. System constraints are a model risk issue. If a bank cannot replace a model within a reasonable timeframe because of system dependencies, those constraints are themselves a model risk management concern. The inability to respond quickly to model impairment is a governance vulnerability that should be reflected in the model risk appetite and the ICAAP.

5. Redevelopment on current data produces better models. Cornerstone's 2023 models — built on data that included the pandemic period, post-lockdown recovery, and the 2022–2023 rate rise environment — were materially more predictive than the 2018 models had been by 2021. The data lag that had caused the problem, once corrected, also provided the richest development dataset the bank had ever had.

6. Disclosure of model limitations is not regulatory weakness — it is expected governance. Cornerstone's transparent disclosure of the overlay approach and its drivers, in the notes to the financial statements, was received constructively by both the regulator and the external auditor. The alternative — presenting model-based provisions without disclosure of known material limitations — would have been a more serious failure.

Discussion Questions

1. Cornerstone's quarterly monitoring reports in 2020–2021 tracked Gini but not PSI. If you were designing a minimum monitoring pack for IFRS 9 ECL models today, what metrics would you include, at what frequency, and with what escalation triggers?

2. The overlay multiplier approach (multiplying model output by a scalar adjustment) is relatively simple. Can you identify any risks or limitations of this approach compared to a more granular variable-level overlay? Under what circumstances might the scalar approach be insufficient?

3. The PRA required Cornerstone to include a Pillar 2A capital add-on reflecting model risk while the overlay was in place. How would you approach quantifying the appropriate capital buffer for a model classified as "impaired — in use pending redevelopment"?

4. Rafael's role in this scenario was as a board-commissioned independent adviser rather than an executive with line responsibility. What are the advantages and disadvantages of using an external independent adviser for model risk governance decisions, compared to relying on the internal model validation function?

5. The case implies that some banks in 2020 did not run PSI analyses on their IFRS 9 models at the height of the pandemic. Given the operational pressures of that period, how would you design a governance framework that ensures critical model monitoring is performed even during crisis conditions?

Chapter 15 of Regulatory Technology (RegTech): A Practitioner's Guide