Case Study 1: Resilient Batch Processing with Comprehensive Error Handling

Background

Midwest Federal Credit Union processes over 400,000 member transactions every night during its end-of-day batch cycle. The core posting program, MFCU-POST, reads transactions from three separate input files -- branch transactions, ATM transactions, and ACH (Automated Clearing House) transactions -- and applies them to the member account master file. A fourth file, the general ledger summary file, receives debit and credit totals by account category for downstream reconciliation.

For years the program ran without incident, but a series of production failures over six months exposed fundamental weaknesses in the program's error handling. In January, a corrupted ATM transaction record containing non-numeric data in the amount field caused an S0C7 abend at 2:14 AM, halting the entire batch cycle. In March, the general ledger output disk filled up (file status 34) after processing 280,000 records, and the program crashed without closing the master file, requiring a VSAM recovery. In May, a network glitch caused the ACH file to arrive empty, and the program treated zero transactions as a successful run, failing to raise an alarm that the file was missing.

The credit union's IT director mandated a complete rewrite of the error-handling infrastructure. The new program, MFCUPOST2, must be resilient: it must handle errors at every level -- file open failures, record-level I/O errors, data validation failures, and arithmetic overflows -- without crashing. When errors are unrecoverable, the program must shut down gracefully, closing all files, writing a processing summary, and setting an appropriate return code for the job scheduler.


Design Requirements

The resilient batch processing program must satisfy these requirements:

  1. DECLARATIVES for every file: Each of the five files (three input, one I-O master, one output GL summary) has its own declarative error handler that captures the file status, increments an error counter, and sets a file-specific error flag.

  2. Error classification: Errors are classified into three severity levels: - Warning (severity 4): Informational conditions that do not affect processing, such as a short record that can still be parsed. - Error (severity 8): Record-level failures that cause individual records to be rejected but do not stop the batch. Examples: account not found, invalid amount. - Critical (severity 16): File-level or batch-level failures that require the program to stop. Examples: master file open failure, error threshold exceeded.

  3. Graceful degradation: If one of the three input files cannot be opened, the program processes the other two and reports which file was skipped. If the error count for any single input file exceeds a configurable threshold (default 100), the program stops processing that file but continues with the others.

  4. Processing statistics: At the end of the run, the program writes a detailed summary showing records read, posted, rejected, and error counts by file and by error type.


Data Design

File Definitions

       ENVIRONMENT DIVISION.
       INPUT-OUTPUT SECTION.
       FILE-CONTROL.
           SELECT BRANCH-TRANS-FILE ASSIGN TO BRNTRANS
               FILE STATUS IS WS-BRANCH-STATUS.
           SELECT ATM-TRANS-FILE    ASSIGN TO ATMTRANS
               FILE STATUS IS WS-ATM-STATUS.
           SELECT ACH-TRANS-FILE    ASSIGN TO ACHTRANS
               FILE STATUS IS WS-ACH-STATUS.
           SELECT MEMBER-MASTER     ASSIGN TO MBRMAST
               ORGANIZATION IS INDEXED
               ACCESS MODE IS RANDOM
               RECORD KEY IS MM-MEMBER-ID
               FILE STATUS IS WS-MASTER-STATUS.
           SELECT GL-SUMMARY-FILE   ASSIGN TO GLSUMM
               FILE STATUS IS WS-GL-STATUS.

Transaction Record Layout

All three input files share a common 120-byte transaction record format:

       FD  BRANCH-TRANS-FILE
           RECORD CONTAINS 120 CHARACTERS.
       01  BRANCH-TRANS-RECORD      PIC X(120).

       FD  ATM-TRANS-FILE
           RECORD CONTAINS 120 CHARACTERS.
       01  ATM-TRANS-RECORD         PIC X(120).

       FD  ACH-TRANS-FILE
           RECORD CONTAINS 120 CHARACTERS.
       01  ACH-TRANS-RECORD         PIC X(120).

Working-Storage Structures

       WORKING-STORAGE SECTION.

       01  WS-FILE-STATUSES.
           05  WS-BRANCH-STATUS     PIC XX.
           05  WS-ATM-STATUS        PIC XX.
           05  WS-ACH-STATUS        PIC XX.
           05  WS-MASTER-STATUS     PIC XX.
           05  WS-GL-STATUS         PIC XX.

       01  WS-FILE-FLAGS.
           05  WS-BRANCH-FLAG       PIC X VALUE 'N'.
               88  FL-BRANCH-OPEN         VALUE 'Y'.
               88  FL-BRANCH-CLOSED       VALUE 'N'.
               88  FL-BRANCH-ERROR        VALUE 'E'.
           05  WS-ATM-FLAG          PIC X VALUE 'N'.
               88  FL-ATM-OPEN            VALUE 'Y'.
               88  FL-ATM-CLOSED          VALUE 'N'.
               88  FL-ATM-ERROR           VALUE 'E'.
           05  WS-ACH-FLAG          PIC X VALUE 'N'.
               88  FL-ACH-OPEN            VALUE 'Y'.
               88  FL-ACH-CLOSED          VALUE 'N'.
               88  FL-ACH-ERROR           VALUE 'E'.
           05  WS-MASTER-FLAG       PIC X VALUE 'N'.
               88  FL-MASTER-OPEN         VALUE 'Y'.
               88  FL-MASTER-CLOSED       VALUE 'N'.
               88  FL-MASTER-ERROR        VALUE 'E'.
           05  WS-GL-FLAG           PIC X VALUE 'N'.
               88  FL-GL-OPEN             VALUE 'Y'.
               88  FL-GL-CLOSED           VALUE 'N'.
               88  FL-GL-ERROR            VALUE 'E'.

       01  WS-EOF-FLAGS.
           05  WS-EOF-BRANCH        PIC X VALUE 'N'.
               88  FL-EOF-BRANCH          VALUE 'Y'.
           05  WS-EOF-ATM           PIC X VALUE 'N'.
               88  FL-EOF-ATM             VALUE 'Y'.
           05  WS-EOF-ACH           PIC X VALUE 'N'.
               88  FL-EOF-ACH             VALUE 'Y'.

       01  WS-TRANSACTION-REC.
           05  WS-TR-MEMBER-ID      PIC X(10).
           05  WS-TR-TRANS-TYPE     PIC X(2).
               88  TR-DEPOSIT             VALUE 'DP'.
               88  TR-WITHDRAWAL          VALUE 'WD'.
               88  TR-TRANSFER            VALUE 'TF'.
               88  TR-PAYMENT             VALUE 'PY'.
               88  TR-FEE                 VALUE 'FE'.
               88  TR-INTEREST            VALUE 'IN'.
               88  TR-VALID-TYPE          VALUE 'DP' 'WD'
                                                'TF' 'PY'
                                                'FE' 'IN'.
           05  WS-TR-AMOUNT         PIC X(11).
           05  WS-TR-AMOUNT-N REDEFINES WS-TR-AMOUNT
                                    PIC S9(9)V99.
           05  WS-TR-ACCOUNT-TYPE   PIC X(2).
           05  WS-TR-EFFECTIVE-DATE PIC X(8).
           05  WS-TR-BRANCH-ID      PIC X(4).
           05  WS-TR-SEQUENCE-NUM   PIC X(8).
           05  WS-TR-DESCRIPTION    PIC X(30).
           05  FILLER               PIC X(45).

       01  WS-ERROR-THRESHOLD       PIC 9(5) VALUE 100.
       01  WS-MAX-RETURN-CODE       PIC 99   VALUE 0.

       01  WS-COUNTERS.
           05  WS-CTR-GROUP OCCURS 3 TIMES.
               10  CT-RECORDS-READ  PIC 9(9) VALUE 0.
               10  CT-RECORDS-POST  PIC 9(9) VALUE 0.
               10  CT-RECORDS-REJ   PIC 9(9) VALUE 0.
               10  CT-IO-ERRORS     PIC 9(5) VALUE 0.
               10  CT-DATA-ERRORS   PIC 9(5) VALUE 0.
           05  CT-MASTER-READS      PIC 9(9) VALUE 0.
           05  CT-MASTER-REWRITES   PIC 9(9) VALUE 0.
           05  CT-MASTER-ERRORS     PIC 9(5) VALUE 0.
           05  CT-GL-WRITES         PIC 9(9) VALUE 0.
           05  CT-GL-ERRORS         PIC 9(5) VALUE 0.

       01  WS-FILE-INDEX            PIC 9    VALUE 0.
       01  WS-CURRENT-SOURCE        PIC X(8) VALUE SPACES.

Master Record Layout

       FD  MEMBER-MASTER
           RECORD CONTAINS 250 CHARACTERS.
       01  MASTER-RECORD.
           05  MM-MEMBER-ID         PIC X(10).
           05  MM-MEMBER-NAME       PIC X(30).
           05  MM-SHARE-BALANCE     PIC S9(11)V99.
           05  MM-CHECK-BALANCE     PIC S9(11)V99.
           05  MM-LOAN-BALANCE      PIC S9(11)V99.
           05  MM-LAST-TRANS-DATE   PIC X(8).
           05  MM-ACCOUNT-STATUS    PIC X(1).
               88  MM-ACTIVE              VALUE 'A'.
               88  MM-FROZEN              VALUE 'F'.
               88  MM-CLOSED              VALUE 'C'.
           05  MM-OPEN-DATE         PIC X(8).
           05  FILLER               PIC X(166).

The DECLARATIVES Section

The heart of the resilient design is the DECLARATIVES section, which provides a centralized handler for every file in the program:

       PROCEDURE DIVISION.
       DECLARATIVES.

       BRANCH-FILE-ERROR SECTION.
           USE AFTER STANDARD ERROR PROCEDURE
               ON BRANCH-TRANS-FILE.
       BRANCH-FILE-ERROR-PARA.
           ADD 1 TO CT-IO-ERRORS(1)
           DISPLAY 'DECLARATIVE: BRANCH FILE ERROR'
           DISPLAY '  FILE STATUS: ' WS-BRANCH-STATUS
           DISPLAY '  RECORDS READ SO FAR: '
               CT-RECORDS-READ(1)
           IF WS-BRANCH-STATUS = '30' OR '34' OR '35'
               SET FL-BRANCH-ERROR TO TRUE
               DISPLAY '  SEVERITY: CRITICAL'
               IF WS-MAX-RETURN-CODE < 16
                   MOVE 16 TO WS-MAX-RETURN-CODE
               END-IF
           ELSE
               DISPLAY '  SEVERITY: ERROR'
               IF WS-MAX-RETURN-CODE < 8
                   MOVE 8 TO WS-MAX-RETURN-CODE
               END-IF
           END-IF
           .

       ATM-FILE-ERROR SECTION.
           USE AFTER STANDARD ERROR PROCEDURE
               ON ATM-TRANS-FILE.
       ATM-FILE-ERROR-PARA.
           ADD 1 TO CT-IO-ERRORS(2)
           DISPLAY 'DECLARATIVE: ATM FILE ERROR'
           DISPLAY '  FILE STATUS: ' WS-ATM-STATUS
           DISPLAY '  RECORDS READ SO FAR: '
               CT-RECORDS-READ(2)
           IF WS-ATM-STATUS = '30' OR '34' OR '35'
               SET FL-ATM-ERROR TO TRUE
               DISPLAY '  SEVERITY: CRITICAL'
               IF WS-MAX-RETURN-CODE < 16
                   MOVE 16 TO WS-MAX-RETURN-CODE
               END-IF
           ELSE
               DISPLAY '  SEVERITY: ERROR'
               IF WS-MAX-RETURN-CODE < 8
                   MOVE 8 TO WS-MAX-RETURN-CODE
               END-IF
           END-IF
           .

       ACH-FILE-ERROR SECTION.
           USE AFTER STANDARD ERROR PROCEDURE
               ON ACH-TRANS-FILE.
       ACH-FILE-ERROR-PARA.
           ADD 1 TO CT-IO-ERRORS(3)
           DISPLAY 'DECLARATIVE: ACH FILE ERROR'
           DISPLAY '  FILE STATUS: ' WS-ACH-STATUS
           DISPLAY '  RECORDS READ SO FAR: '
               CT-RECORDS-READ(3)
           IF WS-ACH-STATUS = '30' OR '34' OR '35'
               SET FL-ACH-ERROR TO TRUE
               DISPLAY '  SEVERITY: CRITICAL'
               IF WS-MAX-RETURN-CODE < 16
                   MOVE 16 TO WS-MAX-RETURN-CODE
               END-IF
           ELSE
               DISPLAY '  SEVERITY: ERROR'
               IF WS-MAX-RETURN-CODE < 8
                   MOVE 8 TO WS-MAX-RETURN-CODE
               END-IF
           END-IF
           .

       MASTER-FILE-ERROR SECTION.
           USE AFTER STANDARD ERROR PROCEDURE
               ON MEMBER-MASTER.
       MASTER-FILE-ERROR-PARA.
           ADD 1 TO CT-MASTER-ERRORS
           DISPLAY 'DECLARATIVE: MASTER FILE ERROR'
           DISPLAY '  FILE STATUS: ' WS-MASTER-STATUS
           DISPLAY '  MEMBER ID: ' WS-TR-MEMBER-ID
           IF WS-MASTER-STATUS = '30' OR '35' OR '93'
               SET FL-MASTER-ERROR TO TRUE
               DISPLAY '  SEVERITY: CRITICAL'
               MOVE 16 TO WS-MAX-RETURN-CODE
           ELSE
               DISPLAY '  SEVERITY: ERROR'
               IF WS-MAX-RETURN-CODE < 8
                   MOVE 8 TO WS-MAX-RETURN-CODE
               END-IF
           END-IF
           .

       GL-FILE-ERROR SECTION.
           USE AFTER STANDARD ERROR PROCEDURE
               ON GL-SUMMARY-FILE.
       GL-FILE-ERROR-PARA.
           ADD 1 TO CT-GL-ERRORS
           DISPLAY 'DECLARATIVE: GL SUMMARY FILE ERROR'
           DISPLAY '  FILE STATUS: ' WS-GL-STATUS
           IF WS-GL-STATUS = '34'
               SET FL-GL-ERROR TO TRUE
               DISPLAY '  SEVERITY: CRITICAL - DISK FULL'
               MOVE 16 TO WS-MAX-RETURN-CODE
           ELSE
               DISPLAY '  SEVERITY: ERROR'
               IF WS-MAX-RETURN-CODE < 8
                   MOVE 8 TO WS-MAX-RETURN-CODE
               END-IF
           END-IF
           .

       END DECLARATIVES.

Notice that the declarative handlers do not call STOP RUN. They set flags and severity levels, allowing the main program logic to decide when to stop. This is a key design principle: declaratives should capture information and set state, not make flow-control decisions.


Main Processing Logic

       MAIN-PROCESSING SECTION.
       0000-MAIN.
           PERFORM 1000-INITIALIZE
           IF FL-MASTER-ERROR
               DISPLAY 'CANNOT CONTINUE WITHOUT MASTER FILE'
               PERFORM 9000-TERMINATE
               STOP RUN
           END-IF

           IF FL-BRANCH-OPEN
               MOVE 'BRANCH' TO WS-CURRENT-SOURCE
               MOVE 1 TO WS-FILE-INDEX
               PERFORM 2000-PROCESS-BRANCH
           END-IF

           IF FL-ATM-OPEN
               MOVE 'ATM' TO WS-CURRENT-SOURCE
               MOVE 2 TO WS-FILE-INDEX
               PERFORM 3000-PROCESS-ATM
           END-IF

           IF FL-ACH-OPEN
               MOVE 'ACH' TO WS-CURRENT-SOURCE
               MOVE 3 TO WS-FILE-INDEX
               PERFORM 4000-PROCESS-ACH
           END-IF

           PERFORM 9000-TERMINATE
           MOVE WS-MAX-RETURN-CODE TO RETURN-CODE
           STOP RUN
           .

File Opening with Graceful Degradation

       1000-INITIALIZE.
      *---------------------------------------------------------------
      * Open master file first -- this is mandatory
      *---------------------------------------------------------------
           OPEN I-O MEMBER-MASTER
           IF WS-MASTER-STATUS = '00'
               SET FL-MASTER-OPEN TO TRUE
               DISPLAY 'MASTER FILE OPENED SUCCESSFULLY'
           ELSE
               SET FL-MASTER-ERROR TO TRUE
               DISPLAY 'FATAL: MASTER FILE OPEN FAILED'
               DISPLAY 'FILE STATUS: ' WS-MASTER-STATUS
               MOVE 16 TO WS-MAX-RETURN-CODE
           END-IF

      *---------------------------------------------------------------
      * Open GL summary file -- important but not fatal if missing
      *---------------------------------------------------------------
           OPEN OUTPUT GL-SUMMARY-FILE
           IF WS-GL-STATUS = '00'
               SET FL-GL-OPEN TO TRUE
               DISPLAY 'GL SUMMARY FILE OPENED SUCCESSFULLY'
           ELSE
               SET FL-GL-ERROR TO TRUE
               DISPLAY 'WARNING: GL FILE OPEN FAILED'
               DISPLAY 'GL ENTRIES WILL BE WRITTEN TO SYSOUT'
               IF WS-MAX-RETURN-CODE < 4
                   MOVE 4 TO WS-MAX-RETURN-CODE
               END-IF
           END-IF

      *---------------------------------------------------------------
      * Open input files -- each is optional for degraded operation
      *---------------------------------------------------------------
           OPEN INPUT BRANCH-TRANS-FILE
           IF WS-BRANCH-STATUS = '00'
               SET FL-BRANCH-OPEN TO TRUE
               DISPLAY 'BRANCH TRANS FILE OPENED'
           ELSE
               SET FL-BRANCH-ERROR TO TRUE
               DISPLAY 'WARNING: BRANCH FILE NOT AVAILABLE'
               DISPLAY 'FILE STATUS: ' WS-BRANCH-STATUS
               IF WS-MAX-RETURN-CODE < 4
                   MOVE 4 TO WS-MAX-RETURN-CODE
               END-IF
           END-IF

           OPEN INPUT ATM-TRANS-FILE
           IF WS-ATM-STATUS = '00'
               SET FL-ATM-OPEN TO TRUE
               DISPLAY 'ATM TRANS FILE OPENED'
           ELSE
               SET FL-ATM-ERROR TO TRUE
               DISPLAY 'WARNING: ATM FILE NOT AVAILABLE'
               DISPLAY 'FILE STATUS: ' WS-ATM-STATUS
               IF WS-MAX-RETURN-CODE < 4
                   MOVE 4 TO WS-MAX-RETURN-CODE
               END-IF
           END-IF

           OPEN INPUT ACH-TRANS-FILE
           IF WS-ACH-STATUS = '00'
               SET FL-ACH-OPEN TO TRUE
               DISPLAY 'ACH TRANS FILE OPENED'
           ELSE
               SET FL-ACH-ERROR TO TRUE
               DISPLAY 'WARNING: ACH FILE NOT AVAILABLE'
               DISPLAY 'FILE STATUS: ' WS-ACH-STATUS
               IF WS-MAX-RETURN-CODE < 4
                   MOVE 4 TO WS-MAX-RETURN-CODE
               END-IF
           END-IF

      *--- Check if at least one input file is available
           IF FL-BRANCH-ERROR AND FL-ATM-ERROR AND FL-ACH-ERROR
               DISPLAY 'FATAL: NO INPUT FILES AVAILABLE'
               MOVE 16 TO WS-MAX-RETURN-CODE
               SET FL-MASTER-ERROR TO TRUE
           END-IF
           .

Transaction Processing with Error Threshold

The processing paragraphs for each input file follow the same pattern. Here is the branch file processor as a representative example:

       2000-PROCESS-BRANCH.
           PERFORM 2100-READ-BRANCH
           PERFORM UNTIL FL-EOF-BRANCH OR FL-BRANCH-ERROR
                          OR FL-MASTER-ERROR
               PERFORM 5000-VALIDATE-TRANSACTION
               IF WS-VALID-FLAG = 'Y'
                   PERFORM 6000-POST-TRANSACTION
               ELSE
                   ADD 1 TO CT-RECORDS-REJ(1)
               END-IF
      *------- Check error threshold
               IF CT-IO-ERRORS(1) > WS-ERROR-THRESHOLD
                   DISPLAY 'ERROR THRESHOLD EXCEEDED FOR '
                       'BRANCH FILE'
                   DISPLAY 'ERRORS: ' CT-IO-ERRORS(1)
                   DISPLAY 'THRESHOLD: ' WS-ERROR-THRESHOLD
                   SET FL-BRANCH-ERROR TO TRUE
                   MOVE 12 TO WS-MAX-RETURN-CODE
               ELSE
                   PERFORM 2100-READ-BRANCH
               END-IF
           END-PERFORM
           .

       2100-READ-BRANCH.
           READ BRANCH-TRANS-FILE
               INTO WS-TRANSACTION-REC
               AT END
                   SET FL-EOF-BRANCH TO TRUE
               NOT AT END
                   ADD 1 TO CT-RECORDS-READ(1)
           END-READ
      *--- Check for non-EOF, non-success status
           IF WS-BRANCH-STATUS NOT = '00'
              AND WS-BRANCH-STATUS NOT = '10'
               ADD 1 TO CT-IO-ERRORS(1)
               DISPLAY 'READ ERROR ON BRANCH FILE'
               DISPLAY 'STATUS: ' WS-BRANCH-STATUS
               DISPLAY 'RECORD: ' CT-RECORDS-READ(1)
           END-IF
           .

Data Validation

       5000-VALIDATE-TRANSACTION.
           MOVE 'Y' TO WS-VALID-FLAG

      *--- Validate member ID is not spaces
           IF WS-TR-MEMBER-ID = SPACES
               MOVE 'N' TO WS-VALID-FLAG
               ADD 1 TO CT-DATA-ERRORS(WS-FILE-INDEX)
               DISPLAY 'REJECT: BLANK MEMBER ID'
               DISPLAY '  SOURCE: ' WS-CURRENT-SOURCE
               DISPLAY '  RECORD: '
                   CT-RECORDS-READ(WS-FILE-INDEX)
           END-IF

      *--- Validate transaction type
           IF WS-VALID-FLAG = 'Y'
              AND NOT TR-VALID-TYPE
               MOVE 'N' TO WS-VALID-FLAG
               ADD 1 TO CT-DATA-ERRORS(WS-FILE-INDEX)
               DISPLAY 'REJECT: INVALID TRANS TYPE '
                   WS-TR-TRANS-TYPE
               DISPLAY '  MEMBER: ' WS-TR-MEMBER-ID
           END-IF

      *--- Validate amount is numeric
           IF WS-VALID-FLAG = 'Y'
               INSPECT WS-TR-AMOUNT
                   TALLYING WS-TALLY-COUNT
                   FOR ALL SPACES
               IF WS-TALLY-COUNT > 0
                   MOVE 'N' TO WS-VALID-FLAG
                   ADD 1 TO CT-DATA-ERRORS(WS-FILE-INDEX)
                   DISPLAY 'REJECT: NON-NUMERIC AMOUNT'
                   DISPLAY '  MEMBER: ' WS-TR-MEMBER-ID
                   DISPLAY '  AMOUNT FIELD: ' WS-TR-AMOUNT
               END-IF
           END-IF
           .

Posting with Arithmetic Protection

       6000-POST-TRANSACTION.
      *--- Read the master record
           MOVE WS-TR-MEMBER-ID TO MM-MEMBER-ID
           READ MEMBER-MASTER
               INVALID KEY
                   ADD 1 TO CT-RECORDS-REJ(WS-FILE-INDEX)
                   DISPLAY 'REJECT: MEMBER NOT FOUND '
                       WS-TR-MEMBER-ID
                   DISPLAY '  SOURCE: ' WS-CURRENT-SOURCE
               NOT INVALID KEY
                   ADD 1 TO CT-MASTER-READS
                   PERFORM 6100-APPLY-TRANSACTION
           END-READ
           .

       6100-APPLY-TRANSACTION.
      *--- Check account status
           IF MM-FROZEN
               ADD 1 TO CT-RECORDS-REJ(WS-FILE-INDEX)
               DISPLAY 'REJECT: ACCOUNT FROZEN '
                   WS-TR-MEMBER-ID
           ELSE IF MM-CLOSED
               ADD 1 TO CT-RECORDS-REJ(WS-FILE-INDEX)
               DISPLAY 'REJECT: ACCOUNT CLOSED '
                   WS-TR-MEMBER-ID
           ELSE
               EVALUATE TRUE
                   WHEN TR-DEPOSIT
                       ADD WS-TR-AMOUNT-N
                           TO MM-SHARE-BALANCE
                           ON SIZE ERROR
                               ADD 1 TO
                                 CT-RECORDS-REJ(WS-FILE-INDEX)
                               DISPLAY 'SIZE ERROR ON DEPOSIT'
                               DISPLAY '  MEMBER: '
                                   WS-TR-MEMBER-ID
                               DISPLAY '  AMOUNT: '
                                   WS-TR-AMOUNT-N
                               DISPLAY '  BALANCE: '
                                   MM-SHARE-BALANCE
                           NOT ON SIZE ERROR
                               PERFORM 6200-REWRITE-MASTER
                       END-ADD
                   WHEN TR-WITHDRAWAL
                       SUBTRACT WS-TR-AMOUNT-N
                           FROM MM-SHARE-BALANCE
                           ON SIZE ERROR
                               ADD 1 TO
                                 CT-RECORDS-REJ(WS-FILE-INDEX)
                               DISPLAY 'SIZE ERROR ON '
                                   'WITHDRAWAL'
                               DISPLAY '  MEMBER: '
                                   WS-TR-MEMBER-ID
                           NOT ON SIZE ERROR
                               PERFORM 6200-REWRITE-MASTER
                       END-SUBTRACT
                   WHEN TR-PAYMENT
                       SUBTRACT WS-TR-AMOUNT-N
                           FROM MM-LOAN-BALANCE
                           ON SIZE ERROR
                               ADD 1 TO
                                 CT-RECORDS-REJ(WS-FILE-INDEX)
                               DISPLAY 'SIZE ERROR ON PAYMENT'
                               DISPLAY '  MEMBER: '
                                   WS-TR-MEMBER-ID
                           NOT ON SIZE ERROR
                               PERFORM 6200-REWRITE-MASTER
                       END-SUBTRACT
                   WHEN OTHER
                       PERFORM 6200-REWRITE-MASTER
               END-EVALUATE
           END-IF
           .

       6200-REWRITE-MASTER.
           MOVE WS-TR-EFFECTIVE-DATE TO MM-LAST-TRANS-DATE
           REWRITE MASTER-RECORD
           IF WS-MASTER-STATUS = '00'
               ADD 1 TO CT-RECORDS-POST(WS-FILE-INDEX)
               ADD 1 TO CT-MASTER-REWRITES
           ELSE
               ADD 1 TO CT-RECORDS-REJ(WS-FILE-INDEX)
               ADD 1 TO CT-MASTER-ERRORS
               DISPLAY 'REWRITE FAILED FOR MEMBER '
                   WS-TR-MEMBER-ID
               DISPLAY 'FILE STATUS: ' WS-MASTER-STATUS
           END-IF
           .

Termination and Reporting

       9000-TERMINATE.
           DISPLAY '============================================'
           DISPLAY '  MFCUPOST2 PROCESSING SUMMARY'
           DISPLAY '============================================'

           DISPLAY ' '
           DISPLAY 'BRANCH TRANSACTIONS:'
           IF FL-BRANCH-ERROR
               DISPLAY '  STATUS: NOT PROCESSED (FILE ERROR)'
           ELSE
               DISPLAY '  RECORDS READ:     '
                   CT-RECORDS-READ(1)
               DISPLAY '  RECORDS POSTED:   '
                   CT-RECORDS-POST(1)
               DISPLAY '  RECORDS REJECTED: '
                   CT-RECORDS-REJ(1)
               DISPLAY '  I/O ERRORS:       '
                   CT-IO-ERRORS(1)
               DISPLAY '  DATA ERRORS:      '
                   CT-DATA-ERRORS(1)
           END-IF

           DISPLAY ' '
           DISPLAY 'ATM TRANSACTIONS:'
           IF FL-ATM-ERROR
               DISPLAY '  STATUS: NOT PROCESSED (FILE ERROR)'
           ELSE
               DISPLAY '  RECORDS READ:     '
                   CT-RECORDS-READ(2)
               DISPLAY '  RECORDS POSTED:   '
                   CT-RECORDS-POST(2)
               DISPLAY '  RECORDS REJECTED: '
                   CT-RECORDS-REJ(2)
               DISPLAY '  I/O ERRORS:       '
                   CT-IO-ERRORS(2)
               DISPLAY '  DATA ERRORS:      '
                   CT-DATA-ERRORS(2)
           END-IF

           DISPLAY ' '
           DISPLAY 'ACH TRANSACTIONS:'
           IF FL-ACH-ERROR
               DISPLAY '  STATUS: NOT PROCESSED (FILE ERROR)'
           ELSE
               DISPLAY '  RECORDS READ:     '
                   CT-RECORDS-READ(3)
               DISPLAY '  RECORDS POSTED:   '
                   CT-RECORDS-POST(3)
               DISPLAY '  RECORDS REJECTED: '
                   CT-RECORDS-REJ(3)
               DISPLAY '  I/O ERRORS:       '
                   CT-IO-ERRORS(3)
               DISPLAY '  DATA ERRORS:      '
                   CT-DATA-ERRORS(3)
           END-IF

           DISPLAY ' '
           DISPLAY 'MASTER FILE ACTIVITY:'
           DISPLAY '  READS:     ' CT-MASTER-READS
           DISPLAY '  REWRITES:  ' CT-MASTER-REWRITES
           DISPLAY '  ERRORS:    ' CT-MASTER-ERRORS

           DISPLAY ' '
           DISPLAY 'GL SUMMARY FILE:'
           DISPLAY '  WRITES:    ' CT-GL-WRITES
           DISPLAY '  ERRORS:    ' CT-GL-ERRORS

           DISPLAY ' '
           DISPLAY 'OVERALL RETURN CODE: ' WS-MAX-RETURN-CODE
           DISPLAY '============================================'

      *--- Close all open files
           IF FL-BRANCH-OPEN
               CLOSE BRANCH-TRANS-FILE
           END-IF
           IF FL-ATM-OPEN
               CLOSE ATM-TRANS-FILE
           END-IF
           IF FL-ACH-OPEN
               CLOSE ACH-TRANS-FILE
           END-IF
           IF FL-MASTER-OPEN
               CLOSE MEMBER-MASTER
           END-IF
           IF FL-GL-OPEN
               CLOSE GL-SUMMARY-FILE
           END-IF
           .

JCL for Execution

//MFCUPOST JOB (ACCT),'DAILY POSTING',CLASS=A,
//         MSGCLASS=X,MSGLEVEL=(1,1)
//POST     EXEC PGM=MFCUPOST2
//STEPLIB  DD DSN=MFCU.PROD.LOADLIB,DISP=SHR
//BRNTRANS DD DSN=MFCU.DAILY.BRANCH.TRANS,DISP=SHR
//ATMTRANS DD DSN=MFCU.DAILY.ATM.TRANS,DISP=SHR
//ACHTRANS DD DSN=MFCU.DAILY.ACH.TRANS,DISP=SHR
//MBRMAST  DD DSN=MFCU.MEMBER.MASTER,DISP=SHR
//GLSUMM   DD DSN=MFCU.DAILY.GL.SUMMARY,
//         DISP=(NEW,CATLG,DELETE),
//         SPACE=(CYL,(10,5),RLSE),
//         DCB=(RECFM=FB,LRECL=100,BLKSIZE=0)
//SYSOUT   DD SYSOUT=*

Lessons and Design Principles

This case study illustrates several enterprise error-handling principles:

Declaratives as observers, not controllers. The declarative handlers capture error information and set flags but do not make flow-control decisions. The main program logic reads the flags and decides whether to continue, skip, or terminate. This separation of concerns makes the error-handling behavior predictable and testable.

Graceful degradation over fail-fast. In a financial batch environment, processing three out of four files is better than processing none. The program adapts to missing inputs and clearly reports what was and was not processed. Operations staff can then schedule a supplemental run for the missing file when it becomes available.

Error thresholds as circuit breakers. Rather than blindly processing a file that is producing thousands of errors (which might indicate a format change or corruption), the program stops processing that file after a configurable threshold. This prevents cascading errors from consuming resources and polluting the master file.

Statistics-driven postmortem. The processing summary provides operations staff with everything they need to assess the run: how many records were processed from each source, how many succeeded, how many failed, and why. This eliminates the need to search through thousands of lines of log output.

No STOP RUN in subroutines. The program has exactly one STOP RUN, at the end of the main paragraph. Every other path sets flags and return codes that flow back to the main paragraph, which then goes through the standard termination routine. This ensures files are always closed properly and statistics are always printed, regardless of how the program ends.