Chapter 35 Key Takeaways: AI-Assisted COBOL

Core Principles

  1. AI tools are force multipliers, not replacements. They make veteran COBOL developers more productive by handling the tedious parts (comprehension, documentation, test generation) while humans provide the judgment, business context, and system-level understanding that AI lacks.

  2. COBOL's verbosity is an AI advantage. The English-like syntax, explicit data definitions, paragraph names, and rigid structure give AI models more semantic context than terse modern languages, making COBOL more amenable to AI comprehension than you might expect.

  3. The 10% error rate matters. AI output is typically 80-90% accurate for COBOL comprehension and documentation. In financial systems processing billions of dollars, that 10-20% error rate is unacceptable without human review. Every AI artifact must be verified.

  4. Context is everything. AI output quality depends heavily on prompt quality. Always provide copybook definitions, system context, execution context, and business glossaries. Without context, the AI is guessing.

  5. Start read-only, earn trust, then expand. Begin with comprehension and documentation (no risk to production). Track accuracy. Build confidence. Only then move to AI-assisted refactoring and code generation.

Key Technical Points

  1. Copybooks are essential input. COBOL programs are meaningless without their copybooks. Always include relevant data definitions in AI prompts. The AI cannot reason about field behavior without knowing types, sizes, and valid values.

  2. COPY/REPLACING is an AI blind spot. When copybooks are included with REPLACING clauses, the AI may analyze the base copybook text rather than the replaced version that actually compiles. Always provide the post-replacement text.

  3. COMP-3 and numeric handling trip up AI consistently. Truncation vs. rounding, sign nibble handling, intermediate precision in multi-step COMPUTE statements, and field-size-dependent results are areas where AI-generated output requires extra scrutiny.

  4. Dead code detection is the safest refactoring entry point. Identifying never-executed paragraphs, unreachable branches, and unused variables is high-value and relatively low-risk — but always verify against the complete calling chain for dynamic CALLs.

  5. The test oracle problem is real. AI-generated test cases may have incorrect expected results because the AI misunderstands COBOL's numeric behavior. Always verify expected results independently.

Process and Workflow

  1. The documentation pipeline has five stages. Inventory, Context Gathering, AI Generation, Expert Review, Publication. Skipping any stage degrades the result.

  2. Three-tier review catches different error types. Tier 1 (automated) catches syntax and compilation errors. Tier 2 (technical review) catches logic and COBOL-specific errors. Tier 3 (business review) catches business rule and domain errors.

  3. Hybrid human-AI code review outperforms either alone. SecureFirst's experiment showed 85% bug detection with hybrid review vs. 65% human-only and 60% AI-only. The two approaches catch different types of issues.

  4. AI-generated code needs audit trail documentation. In regulated industries, every AI-generated artifact must be tagged with: model/version, generation date, reviewer, review date, and approver.

  5. Maintain the documentation pipeline or lose the investment. AI-generated documentation decays as fast as manually-written documentation. Automate regeneration on code changes to keep documentation current.

Strategic Insights

  1. AI cannot understand system-level context. JCL dependencies, batch chain sequencing, DB2 plan binds, CICS resource management, and cross-program interactions are beyond current AI capabilities unless explicitly provided.

  2. Business rules encoded in data values are invisible to AI. Conventions like '99' meaning "no end date" or status code 'X' meaning "experimental" exist only as institutional knowledge. Capture these conventions as AI input.

  3. Security policies must address AI tool usage. Determine what source code can be sent to cloud services, how to sanitize sensitive code, and how AI-generated artifacts fit into your audit and compliance framework.

  4. Measure everything. Track AI accuracy by category, review time with and without AI, defect rates before and after AI introduction, and documentation coverage. Hard numbers overcome soft skepticism and justify continued investment.

  5. The retirement crisis makes this urgent. The knowledge of veteran COBOL developers is irreplaceable. AI tools are the bridge for capturing that knowledge before it walks out the door. The time to start is now.