Chapter 40 Quiz: Testing, Quality Assurance, and Deployment
Test your knowledge of COBOL testing practices, quality assurance processes, and mainframe deployment strategies. Answers are hidden behind expandable sections.
Question 1 — Multiple Choice
Which type of testing verifies that individual COBOL paragraphs or sections produce correct output for given input?
A. Integration testing B. Unit testing C. System testing D. Regression testing
Answer
**B. Unit testing** Unit testing focuses on the smallest testable components of a program, such as individual paragraphs or sections. In COBOL, this typically involves testing a specific calculation paragraph, validation routine, or data transformation in isolation from the rest of the program.Question 2 — True/False
Regression testing is only necessary when a program's source code has been modified.
Answer
**False.** Regression testing may also be necessary when dependencies change, such as when a shared copybook is modified, a DB2 table structure is altered, a VSAM file layout changes, or when the operating environment is updated (e.g., a COBOL compiler upgrade, DB2 version migration, or z/OS release change). Any change that could affect program behavior warrants regression testing.Question 3 — Fill in the Blank
A __ is a simplified program that calls the module under test with controlled inputs and verifies the outputs, isolating the module from its normal runtime dependencies.
Answer
**test harness** (also acceptable: test driver, test stub, test wrapper) A test harness provides a controlled execution environment for the code under test. It supplies predetermined inputs, invokes the target code, captures the outputs, and compares them against expected results. In COBOL, test harnesses are typically standalone programs that CALL or PERFORM the paragraph being tested.Question 4 — Multiple Choice
Which of the following is the correct order of environments in a typical mainframe promotion path?
A. Development -> Production -> QA -> UAT B. Development -> QA -> UAT -> Production C. QA -> Development -> UAT -> Production D. Development -> UAT -> QA -> Production
Answer
**B. Development -> QA -> UAT -> Production** The standard promotion path moves code from the development environment (where developers build and unit test), to the QA environment (where the testing team performs functional and regression testing), to the UAT environment (where business users validate the changes), and finally to the production environment. Each promotion requires passing the exit criteria of the current phase.Question 5 — Short Answer
Explain why test data masking is important for COBOL banking applications, and name three fields that should always be masked when copying production data to a test environment.
Answer
Test data masking is important because banking production data contains personally identifiable information (PII) and financial data protected by regulations such as GLBA (Gramm-Leach-Bliley Act), PCI-DSS (for card data), and GDPR (for European customers). Using unmasked production data in test environments exposes the organization to regulatory penalties, security risks, and privacy violations. Test environments typically have weaker access controls than production, increasing the risk of data exposure. Three fields that should always be masked: 1. **Social Security Numbers** (or national ID numbers) 2. **Account numbers** (checking, savings, credit card) 3. **Customer names and addresses** (PII that could identify individuals) Other candidates include dates of birth, phone numbers, email addresses, and any financial balances that could be linked to identifiable customers.Question 6 — Multiple Choice
What does a return code of 4 from the COBOL compiler typically indicate?
A. Successful compilation with no issues. B. Successful compilation with warning messages. C. Compilation failed due to severe errors. D. Compilation was not attempted.
Answer
**B. Successful compilation with warning messages.** In IBM Enterprise COBOL, return codes follow this convention: 0 = success with no diagnostics, 4 = informational or warning messages (compilation succeeded but there are advisory conditions), 8 = error messages (compilation completed but the resulting code may not execute correctly), 12 = severe errors (compilation completed but the code will likely fail), 16 = unrecoverable errors (compilation terminated).Question 7 — True/False
A smoke test is a comprehensive test that exercises all features of a deployed application.
Answer
**False.** A smoke test is a quick, lightweight test that verifies the most basic functionality of a deployed application. It confirms that the program loads, runs, connects to its databases, reads its input files, and produces some output without errors. It is not comprehensive; it checks only critical paths to confirm that the deployment is viable before investing in full testing. The term comes from hardware testing: "turn it on and see if smoke comes out."Question 8 — Fill in the Blank
The metric that measures the percentage of executable code statements exercised by a test suite is called __ coverage.
Answer
**statement** (also acceptable: line) Statement coverage measures what percentage of the program's executable statements were executed during testing. It is the most basic code coverage metric. More rigorous metrics include branch coverage (percentage of decision paths taken), path coverage (percentage of complete execution paths), and condition coverage (percentage of Boolean sub-expressions evaluated to both true and false).Question 9 — Multiple Choice
Which tool is commonly used on the mainframe to compare two files and identify differences, often used in regression testing to compare actual output against expected output?
A. IEBGENER B. SUPERC (SuperC) C. IEBCOPY D. IDCAMS
Answer
**B. SUPERC (SuperC)** SUPERC (also known as SuperC or ISRSUPC) is a file and member comparison utility available on z/OS. It compares two sequential files or PDS members and produces a listing showing the differences. In regression testing, it is used to compare the actual output of a test run against a known-good expected output file to detect any changes in program behavior. IEBGENER copies files, IEBCOPY manages PDS libraries, and IDCAMS manages VSAM datasets.Question 10 — Short Answer
A COBOL batch program processes 2 million records and the business requires it to complete within a 3-hour batch window. Describe three performance testing metrics you would capture and the tools you would use to measure them.
Answer
Three performance testing metrics: 1. **Elapsed time (wall clock time):** The total time from job start to job end. Measured via the JES job log (start/end timestamps) or SMF type 30 records. The target is under 3 hours. 2. **CPU time:** The amount of processor time consumed by the program, distinguishing between TCB (task) time and SRB (system service) time. Measured via SMF type 30 records or the JES job statistics. High CPU time relative to elapsed time may indicate compute-bound processing; low CPU time may indicate I/O waits. 3. **I/O counts and wait time:** The number of EXCP (Execute Channel Program) operations and the time spent waiting for I/O completion, especially for DB2 buffer pool reads and VSAM file accesses. Measured via SMF type 30 records, RMF reports, or DB2 accounting trace (IFCID 3). High I/O wait times may indicate insufficient buffering or inefficient access patterns. Additional useful metrics: DB2 lock wait time, DB2 getpage counts, virtual storage high-water mark, and paging rates (if virtual storage is constrained).Question 11 — Multiple Choice
In a change management process, what is the role of a Change Advisory Board (CAB)?
A. To write the code changes. B. To execute the deployment scripts. C. To review and approve proposed changes before implementation. D. To perform user acceptance testing.
Answer
**C. To review and approve proposed changes before implementation.** The Change Advisory Board is a cross-functional group (typically including IT management, operations, security, and business stakeholders) that reviews proposed changes to production systems. The CAB evaluates the risk, impact, and readiness of each change request and approves, rejects, or defers changes. This process ensures that changes are properly assessed before they reach production.Question 12 — True/False
Static analysis tools can detect runtime errors such as division by zero in COBOL programs.
Answer
**Partially True.** Static analysis tools can identify *potential* division by zero conditions by analyzing code paths where a divisor variable might be zero (e.g., when no prior validation check exists). However, static analysis cannot determine with certainty whether the condition will actually occur at runtime, because the variable's value depends on input data. Static analysis flags the risk; dynamic testing (with appropriate test data) confirms whether the error actually occurs. Static analysis is better at detecting structural issues like dead code, uninitialized variables, and unreachable paragraphs.Question 13 — Fill in the Blank
The practice of running the existing test suite against a modified program to ensure that previously working functionality has not been broken is called __ testing.
Answer
**regression** Regression testing verifies that changes (bug fixes, enhancements, or refactoring) have not introduced new defects in previously working functionality. It is one of the most important testing practices for maintaining COBOL systems, where programs often have decades of accumulated business logic and changes can have subtle, far-reaching effects.Question 14 — Multiple Choice
What is the primary purpose of a deployment rollback plan?
A. To document what was deployed. B. To provide a procedure for reverting to the previous version if the deployment causes problems. C. To test the deployment before it reaches production. D. To notify stakeholders about the deployment schedule.
Answer
**B. To provide a procedure for reverting to the previous version if the deployment causes problems.** A rollback plan defines the specific steps, responsibilities, and time estimates for reverting each deployed component (load modules, DB2 packages, JCL, VSAM files) to its previous state. It should include decision criteria for triggering a rollback, the maximum time window within which rollback is feasible, and any data recovery procedures needed if transactions have been processed using the new code.Question 15 — Short Answer
Describe three advantages of using automated testing over manual testing for COBOL batch programs in a banking environment.
Answer
1. **Repeatability and consistency:** Automated tests execute the same steps with the same data every time, eliminating human error in test execution. This is critical in banking where even small calculation differences can result in financial discrepancies across millions of accounts. 2. **Speed and efficiency:** Automated regression suites can run hundreds of test cases in minutes, whereas manual execution and verification could take days. This enables frequent testing during development and faster feedback loops, which is essential when working within tight batch processing windows. 3. **Continuous integration support:** Automated tests can be triggered automatically by code changes, providing immediate feedback to developers about the impact of their modifications. This catches defects early, when they are cheapest to fix, and enables the team to maintain confidence in the codebase through continuous validation. Additional advantages include: auditability (automated test results provide a documented trail for regulators), scalability (easy to add new test cases), and overnight/unattended execution.Question 16 — Multiple Choice
Which of the following is NOT typically considered a component of a COBOL production deployment package?
A. Compiled load module B. DB2 DBRM and bind cards C. Updated JCL procedures D. Developer's personal test data
Answer
**D. Developer's personal test data** A production deployment package includes all artifacts needed to install and run the changed application: compiled load modules, DB2 DBRMs with bind JCL, updated JCL procedures and cataloged procedures, VSAM file definitions (if new), and deployment instructions. Developer test data is used during development and testing but is never promoted to production, where the application runs against real business data.Question 17 — True/False
Code coverage of 100% guarantees that a COBOL program is free of defects.
Answer
**False.** 100% code coverage means every statement (or branch) was executed at least once during testing, but it does not guarantee correctness. A program can have 100% coverage and still contain defects because: (1) the test cases may not check all possible input combinations, (2) the expected results in the test cases themselves may be wrong, (3) timing-dependent or environment- dependent issues may not manifest during testing, (4) certain defects only appear with specific data combinations that were not tested, and (5) coverage does not verify that the program meets all business requirements. Code coverage is a measure of testing thoroughness, not of program correctness.Question 18 — Fill in the Blank
In mainframe COBOL development, the utility __ is used to copy load modules between partitioned data sets (PDS), often as part of promoting a program from one environment to another.
Answer
**IEBCOPY** IEBCOPY is a z/OS utility that copies, compresses, and manages members of partitioned data sets. In deployment pipelines, it is used to copy compiled load modules from a development or QA load library to the production load library. It can selectively copy specific members, replace existing members, and report on the copy operation.Question 19 — Short Answer
Explain the concept of "shift-left testing" and how it applies to COBOL mainframe development.
Answer
"Shift-left testing" refers to the practice of performing testing activities earlier in the software development lifecycle — shifting them "left" on the timeline. Instead of waiting until code is complete and deployed to a test environment, testing begins during or even before development. In COBOL mainframe development, shift-left testing applies in several ways: 1. **Early unit testing:** Developers write and run test harnesses on their own workstations or in personal development environments before submitting code for review, rather than relying on QA to find bugs later. 2. **Static analysis during coding:** Running static analysis tools (such as IBM Application Discovery or SonarQube with COBOL plugins) during the development phase to catch coding standard violations, dead code, and potential defects before code review. 3. **Code review as testing:** Conducting peer reviews immediately after coding, treating the review as a testing activity that catches logic errors, missing error handling, and design issues. 4. **Test-driven development:** Writing test cases before writing the COBOL paragraph, defining expected behavior first and then implementing to satisfy the tests. The benefit is catching defects when they are cheapest to fix — a defect found in development costs a fraction of one found in production, where it may also carry regulatory and financial penalties.Question 20 — Multiple Choice
When a COBOL program abends with system completion code 0C7, what type of error has occurred?
A. Protection exception (invalid memory access) B. Data exception (invalid decimal data) C. Operation exception (invalid instruction) D. Addressing exception (invalid address)
Answer
**B. Data exception (invalid decimal data)** System completion code 0C7 indicates a data exception, which in COBOL typically occurs when a packed decimal (COMP-3) or zoned decimal field contains invalid data (non-numeric characters in a field defined as numeric). Common causes include uninitialized WORKING-STORAGE fields, reading a file record that does not match the expected layout, and moving alphanumeric data to a numeric field without validation. This is one of the most common COBOL abend codes.Question 21 — True/False
A well-designed deployment pipeline should allow any deployment to be rolled back within minutes, regardless of the complexity of the changes.
Answer
**False.** While the goal of a rollback plan is to restore the previous state as quickly as possible, some changes cannot be rolled back in minutes. For example, DB2 schema changes that involve adding columns and populating them with data, or changes that alter file formats after records have been written in the new format, require careful data recovery procedures that may take considerably longer. A well-designed pipeline should assess rollback complexity during planning and set realistic time expectations. Some changes may require a "roll-forward" fix rather than a true rollback.Question 22 — Fill in the Blank
The testing approach where business users verify that the system meets their requirements and is ready for production use is called _ _ testing.
Answer
**user acceptance** (UAT — User Acceptance Testing) User acceptance testing is the final testing phase before production deployment. Business users execute scenarios that reflect real-world usage to verify that the system meets functional requirements, business rules are correctly implemented, and the user experience is acceptable. UAT sign-off is typically a mandatory gate before production deployment in regulated industries such as banking.Question 23 — Multiple Choice
Which approach to test data management provides the most realistic test scenarios while complying with data privacy regulations?
A. Using production data directly in the test environment. B. Creating entirely synthetic test data from scratch. C. Copying production data and masking sensitive fields while preserving data relationships. D. Using empty tables and testing only with manually entered records.
Answer
**C. Copying production data and masking sensitive fields while preserving data relationships.** Masked production data provides the most realistic test scenarios because it preserves the volume, variety, and distribution of real data while replacing sensitive information (names, SSNs, account numbers) with fictitious values. The key requirement is that masking must be consistent across related tables (e.g., the same masked account number must appear in both the ACCOUNTS and TRANSACTIONS tables) to maintain referential integrity. Purely synthetic data (option B) can miss real-world data patterns, while using raw production data (option A) violates privacy regulations.Question 24 — Short Answer
Describe the purpose and typical contents of a deployment checklist for a COBOL mainframe application release.
Answer
A deployment checklist is a sequential, step-by-step document that guides the operations team through every action required to deploy a release to production. It ensures that nothing is missed and that steps are performed in the correct order, which is critical during high-pressure overnight deployments. Typical contents include: 1. **Pre-deployment steps:** Verify all approvals are obtained, confirm the deployment window, notify affected teams, take backups of current load modules and DB2 packages, verify that rollback materials are in place. 2. **Deployment steps** (in order): - Stop affected CICS regions or batch job streams. - Copy new load modules to production libraries (IEBCOPY). - Copy DBRMs and execute BIND PACKAGE/PLAN jobs. - Apply DB2 schema changes (ALTER TABLE, CREATE INDEX). - Deploy new or updated JCL procedures. - Load new VSAM reference data files. - Refresh CICS program definitions (NEWCOPY/PHASEIN). - Restart affected regions. 3. **Post-deployment verification:** Execute smoke tests, verify program execution with sample transactions, check DB2 binds were successful, confirm JCL changes are active. 4. **Rollback triggers:** Defined criteria for when to execute the rollback plan (e.g., smoke test failure, abend on first production run). 5. **Sign-off:** Confirmation checkboxes for each step, deployer's name, and timestamps.Question 25 — True/False
Cyclomatic complexity of a COBOL paragraph directly correlates with the minimum number of test cases required to achieve 100% branch coverage.
Answer
**True.** Cyclomatic complexity, defined as the number of linearly independent paths through a program's control flow graph, establishes the minimum number of test cases needed to cover every branch. For example, a COBOL paragraph with a cyclomatic complexity of 5 requires at least 5 test cases to exercise all decision paths. In practice, more test cases may be needed to cover boundary conditions and data variations, but the cyclomatic complexity sets the theoretical minimum for branch coverage.Question 26 — Multiple Choice
In a COBOL-DB2 deployment, what must happen if the DB2 BIND step produces a return code of 8?
A. Proceed with deployment; return code 8 is acceptable. B. Investigate the bind messages, resolve any issues, and rebind before deploying. C. Skip the bind step and use the existing package. D. Restart the DB2 subsystem and retry the bind.
Answer
**B. Investigate the bind messages, resolve any issues, and rebind before deploying.** A return code of 8 from the BIND utility indicates errors that may result in incorrect SQL execution or runtime failures (e.g., SQLCODE -818 for timestamp mismatches, -805 for missing packages). The bind output messages must be reviewed to identify the cause (such as missing table authorizations, invalid SQL, or package version conflicts). The issues must be resolved and the bind rerun successfully (return code 0 or 4) before the deployment proceeds.Question 27 — Fill in the Blank
The practice of deploying a change to a small subset of production workload before rolling it out fully is called a _ deployment (or _ release).
Answer
**canary** deployment (or **canary** release) A canary deployment routes a small percentage of production traffic or workload to the new version while the majority continues to use the old version. If the new version performs correctly, the rollout is gradually expanded. If problems are detected, only the small canary group is affected, and rollback is simpler. In mainframe COBOL environments, this can be implemented by routing specific transaction codes or batch job streams to use the new load modules while others continue using the previous version.Question 28 — Short Answer
A developer claims that testing is unnecessary for a one-line change that corrects a misspelled text literal in a DISPLAY statement. Provide three reasons why even this trivial change should go through the standard testing process.
Answer
1. **Compilation verification:** Even a one-line change requires recompilation. The compilation process could fail or produce warnings if the change inadvertently introduces a syntax error (e.g., an unclosed quote). Running the standard test process confirms that the program compiles cleanly and the new load module is functional. 2. **Unintended side effects:** Recompilation with a newer compiler version or different compiler options could change program behavior. Additionally, the developer might have inadvertently modified adjacent code (stray keystrokes, accidental deletions) that would only be detected by running tests. Regression testing catches these unintended changes. 3. **Process discipline and auditability:** In a regulated banking environment, auditors require evidence that every production change has been tested, regardless of size. Bypassing the testing process for "small" changes creates a precedent that erodes discipline and can lead to larger, riskier changes also skipping testing. Consistent process adherence is essential for regulatory compliance (SOX, OCC guidelines) and operational risk management.Question 29 — Multiple Choice
Which of the following best describes the purpose of a test stub in COBOL testing?
A. A program that generates random test data. B. A simplified replacement for a called program or external dependency that returns predetermined responses. C. A report that summarizes test execution results. D. A copy of the production program used for debugging.
Answer
**B. A simplified replacement for a called program or external dependency that returns predetermined responses.** A test stub replaces a real dependency (such as a called subprogram, a DB2 database, or a CICS service) with a controlled substitute that returns predictable results. This allows the program under test to be executed in isolation without requiring the full production infrastructure. For example, a stub for a DB2 call might return a predetermined account balance, allowing the calling program's logic to be tested without an actual database connection.Question 30 — True/False
In mainframe COBOL shops, the same JCL used for production batch processing should be used for testing to ensure environmental consistency.