Chapter 35 Quiz: AI-Assisted COBOL
Multiple Choice (Questions 1-15)
1. What is the primary reason COBOL's verbose syntax can be advantageous for AI comprehension compared to more terse modern languages?
A) COBOL programs are shorter than equivalent programs in modern languages B) COBOL's English-like syntax and explicit data definitions provide more context for AI models C) AI models have more COBOL in their training data than other languages D) COBOL's compiler catches more errors, so the AI doesn't need to check for them
Answer: B — COBOL's verbose, English-like syntax, paragraph names, PERFORM structures, and explicit DATA DIVISION definitions give AI models more semantic context to work with than terse languages with implicit types.
2. When using context-first prompting for COBOL comprehension, which piece of context is MOST critical for accurate AI output?
A) The compiler version used to compile the program B) The relevant copybook definitions for fields referenced in the code C) The name of the developer who wrote the program D) The date the program was last modified
Answer: B — Without copybook definitions, the AI cannot know the types, sizes, or valid values of fields, making its analysis largely guesswork. Copybooks are where business meaning lives in COBOL systems.
3. An AI tool analyzes a COBOL program and reports that paragraph 5000-LEGACY-CALC is dead code (never executed). Before removing it, you should:
A) Remove it immediately since the AI confirmed it's dead B) Check the JCL and all calling programs for dynamic CALL statements that might invoke it C) Keep it as a comment block for documentation purposes D) Replace it with a DISPLAY statement to verify at runtime
Answer: B — A paragraph that appears unused in one program might be dynamically CALLed from another via a computed entry point. The AI cannot see cross-program relationships unless explicitly provided. Always verify against the complete calling chain.
4. Which of the following is a common AI hallucination when generating COBOL code?
A) Generating valid PERFORM UNTIL loops B) Using EVALUATE with WHEN clauses correctly C) Referencing COBOL features that exist in one dialect but not the target compiler D) Using MOVE statements with correct receiving field behavior
Answer: C — AI tools frequently reference features that exist in Micro Focus COBOL but not Enterprise COBOL for z/OS, or vice versa. They may also suggest features from newer compiler versions than the shop uses.
5. What is the "test oracle problem" in the context of AI-generated test cases for COBOL?
A) The AI cannot generate enough test cases to achieve full coverage B) The AI may generate expected results that are wrong because it misunderstands COBOL's numeric behavior C) Oracle databases are incompatible with COBOL testing frameworks D) The AI cannot connect to z/OS to run the tests
Answer: B — The test oracle problem means the AI might misunderstand COBOL truncation, rounding, or sign handling rules and generate test cases with incorrect expected results, which would pass review because the expected values "look right."
6. In Sandra's three-tier review system at Federal Benefits, what does Tier 1 (Automated Validation) check?
A) Business accuracy and regulatory compliance B) Whether the code compiles, tests run, references are real, and standards are met C) Whether the original developer approves the AI output D) Whether the AI model version is approved for production use
Answer: B — Tier 1 automated validation checks that COBOL code compiles, test cases run without abends, documentation references real program/field names (no hallucinations), and code follows shop standards.
7. Why is COBOL's COPY/REPLACING mechanism particularly problematic for AI tools?
A) REPLACING is not a standard COBOL feature B) The replaced text creates code that doesn't exist in any source file, so the AI analyzes the base copybook text instead of what actually compiles C) REPLACING changes the file layout, causing compilation errors D) AI tools cannot parse COPY statements
Answer: B — Copybooks included with REPLACING clauses effectively create code that doesn't exist in any source file. The AI sees the base copybook, but the actual compiled code uses the replaced text, causing comprehension errors.
8. When using AI for refactoring, the MOST dangerous type of error is:
A) Syntax errors that prevent compilation B) Logic changes that alter behavior but compile and pass most tests C) Missing comments in the refactored code D) Inconsistent indentation in the output
Answer: B — Syntax errors are caught by the compiler. Logic changes that alter behavior subtly — such as changing truncation to rounding, or reordering operations that have side effects — are the most dangerous because they compile and may pass most tests while failing in production edge cases.
9. CNB's policy requires AI-generated code blocks to carry comment headers identifying: the AI model/version, generation date, reviewer, review date, and approver. This is primarily required for:
A) Performance optimization B) Audit trail compliance in regulated financial systems C) AI model training improvement D) Version control tagging
Answer: B — Financial regulators require knowing who wrote and reviewed every line of production code. "The AI wrote it" is not an acceptable audit trail. The comment block provides traceability for audit and compliance purposes.
10. An AI suggests eliminating a GO TO by restructuring code into an EVALUATE statement. Before accepting, you should verify all of the following EXCEPT:
A) The refactored code produces exactly the same outputs for all inputs B) All file I/O and CALL statements execute in the same order C) The AI model used is on the organization's approved tool list D) COMP-3/COMP data handling is preserved
Answer: C — While tool approval is an organizational policy matter, it's not part of the semantic equivalence verification needed for a specific refactoring. The review checklist focuses on: semantic equivalence, data handling preservation, condition codes, side effects, performance, and abend behavior.
11. What is the recommended first step when introducing AI tools to a veteran mainframe team?
A) Start with AI-generated code in production to demonstrate value quickly B) Start with read-only applications (comprehension and documentation) before code generation C) Replace manual testing with AI-generated tests immediately D) Have the AI refactor the most complex programs to show its capabilities
Answer: B — Starting with read-only applications lets the team see that the AI can accurately describe their systems before asking them to trust it to modify those systems. This builds trust incrementally.
12. Which COBOL feature do AI tools most consistently struggle with when performing data flow analysis?
A) MOVE statements between alphanumeric fields B) PERFORM UNTIL loops C) REDEFINES clauses where different field names occupy the same storage D) DISPLAY statements
Answer: C — REDEFINES means two differently-named fields share the same storage location. AI tools frequently miss this, leading to incorrect data flow analysis where changes to one field's value don't register as affecting the redefined field.
13. The documentation pipeline at Federal Benefits has five stages. In what order do they occur?
A) AI Generation, Inventory, Context Gathering, Expert Review, Publication B) Inventory, Context Gathering, AI Generation, Expert Review, Publication C) Context Gathering, Inventory, AI Generation, Publication, Expert Review D) Inventory, AI Generation, Expert Review, Context Gathering, Publication
Answer: B — The correct order is: (1) Inventory — catalog all programs, (2) Context Gathering — collect source code, copybooks, JCL, (3) AI Generation — run through prompt templates, (4) Expert Review — SME corrects and adds context, (5) Publication — integrate into documentation system.
14. Why should you NOT ask an AI to generate raw hex values for COMP-3 test data?
A) COMP-3 is no longer used in modern COBOL B) AI models cannot process hexadecimal notation C) The sign nibble handling and packed decimal encoding are error-prone for AI, so logical values should be generated and converted by utility tools D) COMP-3 test data must be created manually
Answer: C — COMP-3 encoding involves packed decimal format with a sign nibble (C/F for positive, D for negative) that AI tools handle inconsistently. The safer approach is to have the AI generate logical values and use a utility to convert them to the physical format.
15. In the context of AI-assisted COBOL development, the statement "AI tools augment expert knowledge; they don't replace it" means:
A) AI tools are not yet capable enough to help with COBOL B) The developer's role evolves to directing and reviewing AI output while providing the human judgment the AI lacks C) Only experts should be allowed to use AI tools D) AI tools should only be used for documentation, never for code
Answer: B — The veteran developer's deep knowledge of the system, business rules, and operational context is irreplaceable. AI tools handle the tedious parts (comprehension, documentation, test generation) while experts provide judgment about design, business rules, and production operations.
True/False (Questions 16-20)
16. AI tools can reliably understand the interactions between COBOL programs in a batch chain without being provided the JCL that chains them.
Answer: FALSE — AI tools cannot understand system-level context like JCL dependencies, batch chain sequencing, or inter-program file contracts unless this information is explicitly provided in the prompt.
17. When an AI suggests a refactoring that changes a nested IF structure to an EVALUATE statement, semantic equivalence is guaranteed because EVALUATE is functionally equivalent to nested IF.
Answer: FALSE — While EVALUATE can express the same logic as nested IF, the conversion can introduce subtle semantic changes, especially with compound conditions, ALSO clauses, or nested IF/ELSE pairing. Each conversion must be verified for exact behavioral equivalence.
18. At SecureFirst, Carlos's team found that AI-generated regression tests caught three bugs in the first month that would have reached production under manual testing.
Answer: TRUE — After generating 4,200 AI test cases, reducing to 3,100 through expert review, the automated nightly test runs caught three regression bugs in the first month.
19. The implicit period in legacy COBOL code (pre-COBOL-85) terminates only the current statement, similar to a semicolon in C.
Answer: FALSE — In legacy COBOL, a period terminates ALL open scopes (IF, EVALUATE, PERFORM). Moving a period changes the logic completely because it closes every scope currently open. This is fundamentally different from a C semicolon.
20. Sandra's Documentation Pipeline at Federal Benefits produced reviewed, published documentation at approximately five times the rate of purely manual documentation efforts.
Answer: TRUE — The AI-assisted pipeline with expert review produced documentation at approximately 5x the rate of purely manual documentation, because the AI generates drafts that experts edit rather than experts writing from scratch.