Case Study 1: Federal Benefits Administration's Modernization Assessment and Strategy

Background

Federal Benefits Administration (FBA) administers benefits for 22 million Americans. The system runs on a 40-year-old COBOL/IMS codebase — 15.2 million lines of COBOL across 4,847 programs, 47 IMS databases, 23 DB2 subsystems, and 8,200 batch jobs defined in the scheduler. The mainframe runs on an IBM z16 with 8,000 MIPS, and annual operating costs total approximately $28 million.

Sandra Chen — modernization lead, PhD in CS, fifteen years of government service — was hired in 2022 to design and execute a modernization strategy. Marcus Whitfield — the legacy SME who wrote much of the original benefits calculation engine — was two years from retirement. His departure created the ticking clock that drove the project's urgency.

The mandate from the deputy director was blunt: "Make this system work for the next twenty years. I don't care if the answer is 'keep the mainframe' — I care that we can still run this system when Marcus leaves, when mobile is the primary access channel, and when Congress changes the benefits formula again."

The Vendor Slide That Started Everything

In March 2023, a major systems integrator presented to FBA leadership. Their pitch: a full migration from COBOL/IMS to Java microservices on AWS GovCloud. Timeline: 18 months. Cost: $340 million. Projected savings: $15 million per year in reduced mainframe costs.

Sandra was in the room. She later described the experience as "watching someone try to sell a perpetual motion machine to the physics department."

Her concerns were immediate and specific:

1. The 18-month timeline was physically impossible. 15.2 million lines of COBOL encoding 40 years of federal benefits law, regulation, and edge cases could not be replicated in any language in 18 months. The requirements discovery alone — understanding what the code actually does — would take longer than that.

2. **The $340 million estimate had no contingency.** Historical data on projects of this scale shows cost overruns of 200-500%. A realistic budget range was $600M-$1.5B.

3. The projected savings were based on a false comparison. The vendor compared FBA's current $28M/year mainframe cost against projected cloud costs of $13M/year. But the cloud estimate didn't include: the cost of running both systems during the multi-year migration (the mainframe can't be turned off until the new system is validated), the cost of regression testing 40 years of business rules, the cost of retraining 200+ FBA staff, or the cost of re-certifying the system with the Office of Inspector General.

4. The risk was unacceptable for a system paying benefits to 22 million Americans. If the migration failed or introduced errors, real people would not receive their benefits. The political and human cost of failure was not on the vendor's slide.

Sandra asked the vendor one question: "What happens to the 22 million beneficiaries if your project is six months behind schedule?"

The vendor's answer — "We'll manage the timeline with agile methodology" — confirmed Sandra's assessment. She recommended that FBA decline the proposal.

The Portfolio Assessment

Sandra spent the first six months (April-September 2023) conducting a rigorous portfolio assessment. Her team of four — two mainframe specialists and two cloud architects — inventoried every program, database, batch job, and interface in the FBA portfolio.

Methodology

The assessment used three tools and one irreplaceable human:

1. IBM Application Discovery and Delivery Intelligence (ADDI): Automated analysis of COBOL source code, copybook usage, CALL chains, and SQL dependencies. Generated the initial dependency graph and complexity metrics.

2. JCL Parser (custom Python script): Analyzed 8,200 batch job definitions to map job dependencies, identify dataset usage patterns, and flag jobs whose output datasets were never referenced by downstream jobs.

3. CICS CSD Analysis: Extracted the CICS System Definition to map every installed transaction, program, mapset, and file — revealing which CICS resources were actively used versus dormant.

4. Marcus Whitfield: Reviewed every finding. Corrected the automated analysis. Identified dynamic CALL targets that ADDI couldn't detect. Explained why programs that looked dead were actually called through obscure scheduling triggers. And — critically — identified programs that ADDI flagged as "active" but that produced output nobody used.

"The tools find about 80% of the truth," Sandra observed. "Marcus found the other 20%. That 20% included every landmine that would have blown up a migration."

Key Findings

The Dead Code Problem: - 1,847 programs (38%) were dead — never executed in production, or executed but producing output nobody consumed - 1,200 batch jobs ran nightly but their output files hadn't been accessed in 2+ years - 312 copybooks were duplicates with minor variations (created when developers copied and modified rather than updating the shared version) - 2.3 million lines of code were provably unreachable (dead paragraphs, unreachable EVALUATE WHEN branches, disabled PERFORM statements)

The Dependency Web: - The benefits calculation engine (FBACALC, 47,000 LOC) was called by 347 other programs - 23 programs had undocumented dependencies on Marcus's personal knowledge - The IMS database schema had evolved through 312 schema changes over 40 years, creating a layer-cake of segment types where some segments hadn't been written to since 2004 but couldn't be removed because downstream programs still had the DBD definitions - 14 programs used dynamic CALL with program names constructed at runtime from data values — making static dependency analysis impossible for those call chains

The Business Rule Archaeology: - The benefits calculation engine contained 2,847 distinct business rules (identified by systematic code review, not automated analysis) - 340 of those rules implemented benefits formulas that had been superseded by subsequent legislation but were retained to handle grandfathered beneficiaries — people whose benefits were calculated under the old formula - 89 rules implemented "special handling" for edge cases that Marcus described as "Congress passed a law, then passed an exception to the law, then passed an exception to the exception" - None of these rules were documented anywhere except in the COBOL code and in Marcus's memory

The Three-Axis Scoring

Sandra scored every program on Business Criticality, Technical Complexity, and Technical Debt. Here is the summary for FBA's top-tier applications:

The Strategy

Sandra's recommendation — approved by FBA leadership in October 2023 — was a five-year incremental modernization plan:

Portfolio Strategy Distribution

The Non-Negotiables

Sandra established four non-negotiable principles for the modernization:

1. Production stability comes first. No modernization activity may degrade the system's ability to pay benefits accurately and on time. Every change goes through the existing change management process. No shortcuts.

2. Knowledge capture is parallel to everything else. Marcus's retirement is a hard deadline. Knowledge capture for the 23 programs dependent on his expertise begins immediately and continues alongside all other modernization activities.

3. Incremental delivery, not big-bang cutover. Every modernization initiative must deliver independently. If the project were canceled at any point, the system would be in a better state than when the project started.

4. Data-driven decisions. Every strategy recommendation has a TCO analysis, a risk assessment, and a rollback plan. "It feels like the right thing to do" is not an acceptable justification.

Phase Execution

Phase 1 (Year 1): Foundation and Quick Wins

Month 1-3: Retired 1,200 batch jobs. MIPS consumption dropped 12%. Annual savings: $2.1M.

Month 2-4: Marcus began documenting FBACALC's business rules. Sandra paired him with two junior developers who recorded his explanations and built regression test cases from his edge-case knowledge. By Month 4, they had documented 1,200 of the 2,847 business rules.

Month 3-6: First API endpoint — FBAELIG (eligibility determination) wrapped with z/OS Connect. The web portal team replaced their screen-scraping integration. Response time improved from 8 seconds to 200 milliseconds.

Month 4-6: Deployed Zowe CLI and VS Code with COBOL extensions to the development team. Git repository established for all COBOL source (migrated from SCLM). Marcus initially resisted but converted within two weeks.

Month 5-8: Automated regression testing framework deployed for FBACALC, FBAELIG, and FBAPAY. First test run found three dormant bugs (incorrect rounding in a grandfathered benefits formula that had been overpaying 847 beneficiaries by $0.03/month since 2019).

Phase 2 (Year 2-3): Strategic Modernization

The phase 2 plan (in progress at time of writing): - API exposure for all 50 highest-value transactions - IMS-to-DB2 migration for 3 selected databases (FBAMASTR deferred due to complexity) - CI/CD pipeline with automated testing for all Refactor-category programs - AI-assisted documentation (Chapter 35 tools) for the remaining 1,647 undocumented business rules - Reporting applications replatformed to cloud-based analytics (Replatform category)

Phase 3 (Year 4-5): Optimization and Steady State

• Complete remaining refactoring
• Hybrid architecture operational (Chapter 37)
• Next-generation workforce fully trained
• Annual mainframe cost reduced from $28M to approximately $17M (MIPS reduction + optimization)

Results After Year 1

Lessons Learned

Lesson 1: The 38% retirement number stunned leadership. Nobody believed that more than a third of the portfolio was dead — until Sandra showed them the evidence. This underscores the importance of the portfolio assessment: you don't know what you have until you look.

Lesson 2: Marcus's knowledge was even more critical than expected. During the documentation effort, Marcus identified 17 programs where the automated analysis was wrong — either flagging active programs as dead or missing critical runtime dependencies. Without his review, the retirement phase would have broken production.

Lesson 3: Quick wins created political capital. The $2.1M/year savings from Phase 1 retirements gave Sandra credibility with the deputy director. When Phase 2 required $6.8M in funding, the argument was simple: "Phase 1 already paid for Phase 2 with savings."

Lesson 4: The first API endpoint changed the conversation. When the web portal team saw their integration go from 8-second screen scraping to 200-millisecond API calls, they became the modernization project's most enthusiastic supporters. They brought their own budget to fund additional API endpoints.

**Lesson 5: The vendor's $340M proposal would have been a disaster.** Sandra's team estimated that following the vendor's approach — full replacement in 18 months — would have taken 5-7 years, cost $800M-$1.2B, and carried a high probability of failure that would have affected 22 million beneficiaries. Her $22.3M incremental approach is delivering better results with 1/15th the risk.

Discussion Questions

1. Sandra's assessment found that 38% of FBA's portfolio was dead code. If you were conducting a similar assessment at your organization, what methods would you use to distinguish "truly dead" from "apparently dead but actually used through dynamic calls or obscure scheduling triggers"?

2. The benefits calculation engine (FBACALC) was scored as Business Criticality 5, Technical Complexity 5, Technical Debt 4. Given these scores, Sandra recommended "Refactor (API + doc)" rather than any form of replacement. Do you agree? Under what circumstances might replacement be justified for a program with these scores?

3. Marcus's knowledge proved essential for validating the portfolio assessment. What would Sandra's team have done if Marcus had retired before the assessment? How should organizations prepare for the loss of key SMEs?

4. The vendor proposed $340M for an 18-month replacement. Sandra estimated the real cost at $800M-$1.2B over 5-7 years. What specific factors account for this 3-4x cost increase? Are Sandra's estimates reasonable?

5. Sandra's Phase 1 achieved a 12% MIPS reduction simply by retiring dead applications. This suggests that 12% of FBA's mainframe cost was waste. Why wasn't this waste identified and eliminated earlier? What organizational incentives prevent dead code retirement?

6. The first API endpoint (eligibility determination) reduced response time from 8 seconds to 200 milliseconds. The 8-second time was from screen scraping. Does this mean the original CICS transaction was slow, or that the access method (screen scraping) was the bottleneck? What are the implications for other modernization approaches?