Case Study 25.1: The Model That Learned to Miss Fraud — Cornerstone's Feedback Loop Failure

The Situation

Organization: Cornerstone Financial Group (fictional composite institution) Division: Retail Banking — Card Fraud Detection Challenge: A statistically sound fraud detection model experiencing degrading performance despite a stable fraud environment — root cause traced to a corrupted training feedback loop Timeline: January–September 2024

Background

Cornerstone's retail banking division issues 4.2 million credit and debit cards. Its card fraud detection system — deployed eighteen months earlier — was a gradient-boosted ensemble (LightGBM) trained on twelve months of labeled historical transaction data. At deployment, the model had demonstrated strong performance: precision 61%, recall 87%, F1 score 0.72. These numbers were considered competitive with industry benchmarks.

By April 2024, the fraud risk team noticed something troubling. Fraud losses were increasing month-over-month despite no reported increase in the external fraud environment. The card base was not growing unusually fast. The mix of channels was stable. Yet confirmed fraud losses had risen 23% over six months.

The first hypothesis: a new fraud attack vector. The fraud operations team reviewed recent confirmed fraud cases. The pattern was recognizable — "friendly fraud" / synthetic identity combinations, a known attack type that the model had historically detected reasonably well.

The second hypothesis: threshold drift. The model's score distribution was checked. No meaningful shift.

The third hypothesis: the model itself was degrading. A data scientist, Laura Chen, was tasked with a root cause investigation.

The Investigation

Laura's first step was to audit the training label quality for the most recent model refresh. Cornerstone retrains its fraud model quarterly, incorporating four quarters of labeled transaction data. The most recent retraining had used labels generated by the fraud investigation team over the prior 12 months.

Laura sampled 500 alerts from the previous 12 months that had been dispositioned as "Customer Confirmed Legitimate" (CCL) — meaning the customer had called in and confirmed the transaction was theirs. She then cross-referenced those transactions against Cornerstone's chargeback records.

The finding was striking: 71 of the 500 CCL-dispositioned transactions (14.2%) had subsequently generated chargebacks — meaning the customer had later disputed those same transactions as fraudulent.

This meant 14.2% of the transactions that the investigation team had labeled as "legitimate" in the training data were actually fraudulent. The model had been retrained on corrupted labels.

Further investigation revealed the mechanism. Cornerstone's fraud investigation process relied heavily on inbound customer calls to confirm or deny suspicious transactions. When a customer called in about a flagged transaction, the agent would verify identity (knowledge-based authentication: date of birth, address, last 4 digits of card) and ask whether they recognized the transaction. If the customer confirmed, the alert was resolved as CCL.

The problem: a fraudster who had gained access to a customer's account (through phishing or data breach) could also call the contact centre, pass knowledge-based authentication (they had the stolen data), and "confirm" the fraudulent transaction. The alert was then resolved as legitimate — and the label went into the training data.

This was not a new problem. Social engineering of contact centres is well-documented. What had changed: the specific attack group targeting Cornerstone had, apparently, identified that confirming flagged transactions through the contact centre was a reliable way to reduce friction. A third-party fraud intelligence service subsequently identified a fraud ring in Eastern Europe systematically doing exactly this across several UK card issuers.

The Impact on Model Performance

Laura constructed a clean holdout dataset — transactions she independently labeled using chargeback records (not investigation dispositions) — and scored the current model against it.

The results told the story. Compared to the model's performance at deployment:

Recall had dropped 16 percentage points. For every 100 fraudulent transactions, the model was now missing 29 instead of 13. The precision decline indicated that the model was also generating more false positives — flagging legitimate transactions the fraudster's calls had "taught" it to consider normal.

The mechanism was clear in the feature importances: the model had learned to give lower weight to the behavioral features that characterized the fraudsters' behavior, because those features had been associated with "legitimate" labels in the training data.

The Remediation Program

Cornerstone implemented a three-stream remediation over 90 days.

Stream 1: Investigation process reform. The "Customer Confirmed Legitimate" disposition was suspended as a standalone resolution path. Going forward: - Any alert above score threshold 0.60 required verification by two methods (something you know + something you are — callback to a pre-registered mobile number, not just knowledge-based authentication) - Alerts with matching fraud typology profiles (the synthetic identity + account takeover combination) required supervisor review before CCL disposition - All CCL dispositions were tagged for a 60-day follow-up check: if the transaction subsequently generated a chargeback, the label was automatically updated to "Confirmed Fraud" in the training set

Stream 2: Training label remediation. The corrupted training labels were identified and replaced: - All CCL dispositions from the previous 18 months were reviewed against chargeback records - 891 transactions with subsequent chargebacks were re-labeled as "Fraud" in the training dataset - An emergency model retrain was performed using the cleaned labels, with the results validated against a clean holdout set before deployment

Stream 3: Chargeback-first labeling. A policy change: chargeback records now supersede investigation dispositions in the training label hierarchy. If a transaction was labeled "CCL" but subsequently generated a chargeback, the chargeback label wins. The model training pipeline was updated to apply this priority automatically.

Emergency model retrain performance (using cleaned labels):

The retrained model slightly outperformed the original deployment — partly because the cleaned training data now contained accurate labels for the specific attack type that had been corrupted.

The Regulatory Dimension

Cornerstone's Head of Compliance, working with the fraud risk team, assessed the incident against its regulatory obligations.

FCA Consumer Duty. The rising fraud losses and the degraded model detection rate had caused direct customer harm: 16 percentage points more fraud was getting through, resulting in customer accounts being compromised. The Consumer Duty required Cornerstone to assess this harm and demonstrate remediation. The incident was documented and included in the firm's Consumer Duty board report.

GDPR. The corrupted training data had included transactions that were subsequently confirmed as fraudulent — personal data processed in a way that the data subjects had not anticipated (their fraudulent transactions were labeled as "their" confirmed transactions, creating a false record). The DPO reviewed whether this constituted a GDPR data integrity issue. Conclusion: the labeling error was an operational process failure, not a data breach, and did not require breach notification — but the training data remediation was documented as a data quality correction.

Model Governance. Under Cornerstone's model risk policy (aligned to SR 11-7 principles), the performance degradation and its root cause were required to be reported to the Model Risk Committee. The incident report noted: the monitoring process had detected performance degradation (satisfactory); the root cause was correctly identified (satisfactory); the model retrain and validation were completed within 90 days (satisfactory). The Model Risk Committee concluded: the monitoring framework worked as designed; the process failure was in the investigation workflow rather than the model development process.

One finding for improvement: the quarterly model monitoring report should include a label quality audit (comparison of investigation dispositions against subsequent chargeback records) as a standing metric, not just as an ad hoc exercise. This was implemented.

Discussion Questions

1. The root cause of Cornerstone's fraud model degradation was not a data science failure but an investigation process failure. What does this reveal about the interdependence between model development and operational processes in a live machine learning system?

2. Cornerstone's fraud investigation team used knowledge-based authentication (date of birth, address, last 4 digits of card) to verify customer identity during inbound calls. This authentication method was defeated by social engineering. What alternative or additional authentication methods could have prevented this? What are the tradeoffs in terms of customer friction and operational cost?

3. The remediation involved re-labeling 891 historical training records from "legitimate" to "fraud" and retraining the model. What risks are involved in retroactively changing training labels? How should the validation of the retrained model be designed to ensure the relabeled data is itself accurate?

4. Cornerstone's monitoring process detected the performance degradation (rising fraud losses) but did not identify the root cause (label corruption) until a specific investigation was triggered. What leading indicators — detectable earlier than fraud loss trends — could have provided earlier warning of the label corruption problem?

5. The incident was reported to Cornerstone's Model Risk Committee as an operational finding rather than a model development failure. From a governance perspective, who bears ultimate accountability for a fraud model's performance: the data science team (who built the model), the fraud operations team (who generated the corrupted labels), or the model risk function (who monitors performance)? How should accountability be structured in the model governance framework to ensure this question has a clear answer?