Chapter 35 Exercises: AI-Assisted COBOL
Section 35.2 — Code Understanding
Exercise 35.1: Basic Paragraph Summarization
Feed the following COBOL paragraph to an LLM and evaluate the accuracy of the response. Identify at least two things the AI gets right and two potential areas of concern:
4500-CALC-INTEREST.
IF ACCT-IS-SAVINGS
COMPUTE WS-DAILY-RATE =
WS-ANNUAL-RATE / 365
COMPUTE WS-INT-AMOUNT ROUNDED =
ACCT-LEDGER-BAL * WS-DAILY-RATE
IF WS-INT-AMOUNT > 0
ADD WS-INT-AMOUNT TO WS-ACCUM-INT
ADD 1 TO WS-INT-CALC-CTR
END-IF
END-IF.
Exercise 35.2: Context-First Prompting
Write a context-first prompt for the following scenario: You have a CICS online program that handles ATM balance inquiry transactions. The program reads from the ACCOUNT-MASTER VSAM file and the HOLD-FILE. Write the system context, program context, and task instruction that would produce the most accurate AI summary.
Exercise 35.3: Copybook-Inclusive Prompting
Given a transaction record copybook with fields TXN-TYPE (PIC X(2)), TXN-AMOUNT (PIC S9(9)V99 COMP-3), TXN-DATE (PIC 9(8)), and TXN-STATUS (PIC X(1)), write a prompt that includes the copybook definition and asks the AI to explain how a processing paragraph uses these fields. Include at least three 88-level conditions for TXN-TYPE.
Exercise 35.4: Data Flow Tracing
Write a prompt that asks an AI to trace the field WS-TOTAL-DEBITS through a complete batch processing program. Specify what information you want for each reference (read/write, paragraph, business significance). Explain why you would also need to include the copybooks and any called subprograms in the prompt.
Exercise 35.5: Cross-Program Interface Analysis
You have two COBOL programs: Program A writes a sequential file, and Program B reads that file in the next job step. Write a prompt that asks the AI to document the interface contract between these programs. What specific artifacts would you need to include in the prompt besides the source code? List at least five.
Exercise 35.6: Identifying AI Comprehension Errors
The following AI-generated summary contains three errors. Identify them:
"Paragraph 3000-PROCESS-WITHDRAWAL subtracts the withdrawal amount from the account balance using SUBTRACT. If the result is negative, it sets the overdraft flag and calls the overdraft-fee subroutine. The paragraph uses COMP-3 arithmetic for all calculations, which ensures rounding to two decimal places."
Hints: Consider what COMP-3 actually does regarding rounding, whether SUBTRACT necessarily uses COMP-3, and what determines the precision of the result.
Exercise 35.7: Chain-of-Paragraphs Analysis
Given a main processing paragraph that PERFORMs five sub-paragraphs in sequence (VALIDATE, LOOKUP, CALCULATE, UPDATE, LOG), write a prompt that asks the AI to trace the data flow across all five paragraphs. How would you handle the case where one of the paragraphs uses a GO TO to skip subsequent processing?
Section 35.3 — Documentation Generation
Exercise 35.8: Program Documentation Template
Design a documentation prompt template for a CICS online transaction processing program. Your template should include placeholders for: system context, transaction ID, BMS map names, file resources, DB2 tables, and any DL/I databases. The template should produce output covering program flow, error handling, and user interaction.
Exercise 35.9: Copybook Annotation
Write an AI prompt that generates annotated comments for the following copybook. Then manually review the output and identify any annotations that would be wrong or misleading:
01 WS-CTRL-FLAGS.
05 WS-EOF-FLAG PIC X(1).
88 WS-EOF VALUE 'Y'.
88 WS-NOT-EOF VALUE 'N'.
05 WS-ERR-FLAG PIC X(1).
88 WS-ERROR VALUE 'Y'.
88 WS-NO-ERROR VALUE 'N'.
05 WS-UPD-FLAG PIC X(1) VALUE 'N'.
88 WS-UPDATED VALUE 'Y'.
88 WS-NOT-UPDATED VALUE 'N'.
05 WS-FIRST-TIME PIC X(1) VALUE 'Y'.
88 WS-IS-FIRST-TIME VALUE 'Y'.
88 WS-NOT-FIRST-TIME VALUE 'N'.
Exercise 35.10: JCL Documentation
Write an AI prompt that produces a plain-English runbook entry for a batch job with three steps: step 1 sorts a file, step 2 runs a COBOL program against the sorted file, step 3 runs a DFSORT to reformat the output. Include in your prompt what specific JCL details the AI should explain (COND parameters, DD names, DISP settings, return codes).
Exercise 35.11: Documentation Pipeline Design
Design a documentation pipeline for a system with 50 COBOL programs, 30 copybooks, and 25 JCL procedures. Specify: prioritization criteria, batch sizes for AI processing, review assignment strategy, and quality metrics. How would you handle the case where the subject matter expert disagrees with the AI-generated documentation?
Exercise 35.12: Maintaining Documentation Currency
Propose a process for keeping AI-generated documentation current as programs change. Address: trigger mechanisms (when does documentation get regenerated?), differential analysis (what changed?), review requirements (does every change need full review?), and version tracking.
Section 35.4 — Assisted Refactoring
Exercise 35.13: Dead Code Detection
Analyze the following code and identify all dead code. Then write an AI prompt that would detect the same dead code, and evaluate whether the AI's detection would be complete:
PROCEDURE DIVISION.
0000-MAIN.
PERFORM 1000-INIT
PERFORM 2000-PROCESS
PERFORM 3000-CLEANUP
STOP RUN.
1000-INIT.
OPEN INPUT IN-FILE
MOVE 'N' TO WS-EOF-FLAG.
1500-UNUSED-PARA.
DISPLAY 'THIS NEVER RUNS'.
2000-PROCESS.
PERFORM UNTIL WS-EOF
READ IN-FILE
AT END MOVE 'Y' TO WS-EOF-FLAG
NOT AT END PERFORM 2100-HANDLE-REC
END-READ
END-PERFORM.
2100-HANDLE-REC.
IF WS-REC-TYPE = 'A'
PERFORM 2200-TYPE-A
ELSE IF WS-REC-TYPE = 'B'
PERFORM 2300-TYPE-B
END-IF.
2200-TYPE-A.
ADD 1 TO WS-TYPE-A-CTR.
2300-TYPE-B.
ADD 1 TO WS-TYPE-B-CTR.
2400-TYPE-C.
ADD 1 TO WS-TYPE-C-CTR.
3000-CLEANUP.
CLOSE IN-FILE.
Exercise 35.14: GOTO Elimination
Rewrite the following paragraph to eliminate all GO TO statements while preserving exact semantic equivalence. Then write an AI prompt that would produce the same refactoring, and identify where the AI might introduce subtle behavioral changes:
5000-VALIDATE-TXN.
IF TXN-AMT = ZERO
GO TO 5000-EXIT
END-IF
IF TXN-AMT < ZERO
MOVE 'NEG AMT' TO WS-ERR-MSG
GO TO 5000-ERROR
END-IF
IF TXN-AMT > WS-MAX-AMT
MOVE 'OVER MAX' TO WS-ERR-MSG
GO TO 5000-ERROR
END-IF
IF ACCT-IS-FROZEN
MOVE 'FROZEN' TO WS-ERR-MSG
GO TO 5000-ERROR
END-IF
MOVE 'Y' TO WS-TXN-VALID.
GO TO 5000-EXIT.
5000-ERROR.
MOVE 'N' TO WS-TXN-VALID
ADD 1 TO WS-ERR-CTR
PERFORM 5100-LOG-ERROR.
5000-EXIT.
EXIT.
Exercise 35.15: Naming Improvement Analysis
Given the following working storage, use an AI to suggest more descriptive names. For each suggestion, rate the AI's recommendation as: Excellent (use as-is), Good (minor adjustment needed), or Wrong (would mislead):
01 WS-WORK-FIELDS.
05 WS-W1 PIC S9(9)V99 COMP-3.
05 WS-W2 PIC S9(9)V99 COMP-3.
05 WS-W3 PIC S9(9)V99 COMP-3.
05 WS-F1 PIC X(1).
05 WS-F2 PIC X(1).
05 WS-CTR1 PIC S9(7) COMP.
05 WS-CTR2 PIC S9(7) COMP.
05 WS-SAVE-X PIC X(80).
Note: Without seeing how these fields are used, the AI cannot suggest accurate names. Explain why program context is essential for naming improvement.
Exercise 35.16: Refactoring Review Checklist Application
Apply the six-point refactoring review checklist from Section 35.4 to the following AI-suggested refactoring. Identify which checklist items pass and which fail:
Original:
MOVE WS-INPUT-AMT TO WS-WORK-AMT
COMPUTE WS-RESULT = WS-WORK-AMT * 1.05
AI Refactoring:
COMPUTE WS-RESULT = WS-INPUT-AMT * 1.05
Consider: What if WS-WORK-AMT is referenced later in the program? What if WS-WORK-AMT and WS-INPUT-AMT have different PIC clauses?
Exercise 35.17: Structure Modernization Evaluation
An AI suggests converting the following nested IF structure to an EVALUATE. Write both versions and identify where the AI might change the execution semantics:
IF WS-CODE = 'A'
PERFORM 100-HANDLE-A
ELSE
IF WS-CODE = 'B'
IF WS-SUB-CODE = '1'
PERFORM 200-HANDLE-B1
ELSE
PERFORM 210-HANDLE-B-OTHER
END-IF
ELSE
PERFORM 300-HANDLE-DEFAULT
END-IF
END-IF
Section 35.5 — AI for Testing
Exercise 35.18: Test Case Generation
Generate a complete set of test cases for the 4500-CALC-INTEREST paragraph from Exercise 35.1. Include positive tests, negative tests, boundary tests, and COBOL-specific tests (truncation, sign handling, leap year considerations for the 365-day divisor).
Exercise 35.19: Test Data Specification
Design test data for the following copybook record. Specify at least 15 test records covering normal cases, boundary values, and error conditions:
01 TRANSACTION-REC.
05 TXN-ACCT-NUM PIC 9(10).
05 TXN-TYPE PIC X(2).
05 TXN-AMOUNT PIC S9(9)V99 COMP-3.
05 TXN-DATE PIC 9(8).
05 TXN-TIME PIC 9(6).
05 TXN-BRANCH PIC 9(4).
05 TXN-TELLER PIC X(8).
Exercise 35.20: Test Oracle Verification
An AI generates the following test case for a COBOL COMPUTE statement. Identify the error in the expected result:
Test: COMPUTE WS-RESULT = 100 / 3
WS-RESULT is PIC S9(5)V99 COMP-3
Expected: 33.33
Explain why the expected result might be wrong and what the correct expected result depends on.
Exercise 35.21: Regression Test Suite Design
Design a regression test suite for the HA banking system's deposit processing module. Specify: how many test cases per paragraph, how to handle DB2 dependencies during testing, how to verify test results automatically, and how to maintain the test suite as the code evolves.
Section 35.6 — Limitations and Risks
Exercise 35.22: Hallucination Detection
Write a prompt that asks an AI to generate code for a COBOL paragraph that reads a VSAM file, processes each record, and writes a report. Review the output for: invented syntax, wrong compiler options, incorrect file handling, and any other hallucinations. Document each error you find.
Exercise 35.23: Security Policy for AI-Assisted Development
Draft a security policy for using AI tools with COBOL source code. Address: what can be sent to cloud AI services, what must stay on-premises, how to sanitize code before sending it to AI services, and how to handle AI-generated code in audit trails. Consider SOX, PCI-DSS, and HIPAA requirements.
Exercise 35.24: COBOL Gotcha Catalog
Create a catalog of at least 10 COBOL-specific behaviors that AI tools are likely to misunderstand. For each behavior, provide: the COBOL feature, how the AI is likely to interpret it, the correct interpretation, and a test that would reveal the AI's error.
Exercise 35.25: Vendor Evaluation Criteria
You are evaluating three AI-assisted COBOL tools for your organization. Design an evaluation rubric with at least 10 criteria, including: COBOL dialect support, accuracy on your code patterns, security compliance, integration with your toolchain, and total cost of ownership.
Section 35.7 — Human-AI Workflow
Exercise 35.26: Review Process Design
Design a review process for AI-generated documentation at an organization with 200 COBOL programs. Specify: who reviews what, turnaround time targets, escalation procedures for disagreements, and metrics for tracking quality over time.
Exercise 35.27: Trust-Building Rollout Plan
Create a 6-month rollout plan for introducing AI-assisted COBOL tools to a team of 15 mainframe developers, of whom 10 are veterans with 20+ years of experience and 5 are new hires with less than 3 years. Address resistance management, training, pilot selection, and success metrics.
Exercise 35.28: AI Accuracy Scorecard
Design a scorecard for tracking AI tool accuracy over time. Include: categories of errors (syntax, logic, semantic, omission), severity levels, tracking by program type (batch, CICS, DB2), and monthly trend reporting. Provide a sample scorecard with hypothetical data for three months.
Exercise 35.29: HA Banking System Documentation Plan
Create a detailed plan for using AI tools to document the entire HA banking system. Include: inventory of programs to document, prioritization by risk and criticality, resource requirements (AI tool time, reviewer time, SME time), timeline, and deliverables. Use the Documentation Pipeline from Section 35.3.
Exercise 35.30: Ethical Considerations
Write a position paper (500-750 words) on the ethical implications of AI-assisted COBOL development. Address: impact on veteran developers' job security, the risk of over-reliance on AI tools, accountability when AI-generated code causes a production failure, and the responsibility of organizations to their mainframe workforce.
Challenge Exercises
Challenge 35.A: End-to-End AI Workflow
Design a complete end-to-end workflow for using AI to add a new feature to the HA banking system (specifically, adding a new transaction type). Cover every stage from requirements through production deployment, specifying where AI assists and where human judgment is required. Estimate time savings compared to a fully manual process.
Challenge 35.B: AI Tool Comparison
If you have access to multiple AI tools (ChatGPT, Claude, IBM watsonx, Copilot, etc.), feed the same COBOL paragraph to each and compare the results. Create a comparison matrix showing accuracy, completeness, COBOL-specific awareness, and usefulness for each tool.
Challenge 35.C: Prompt Engineering Lab
Create a library of 10 optimized prompt templates for different COBOL-related tasks: program summary, copybook annotation, data flow analysis, dead code detection, test case generation, JCL documentation, interface documentation, refactoring suggestion, naming improvement, and performance analysis. Test each template against real or realistic COBOL code and refine based on results.