Case Study 2: Master File Update from Transaction File at Consolidated Trust Bank
Background
Consolidated Trust Bank maintains a customer account master file containing 2.4 million records. Every night, the batch cycle applies the day's transactions to update account balances, add new accounts, and mark accounts for closure. This master-transaction update is the most critical batch job in the bank's processing chain -- if it fails, no customer sees the correct balance the next morning.
The classic sequential master-transaction update is one of the oldest and most important patterns in COBOL batch processing. Two sorted sequential files are merged together: the existing master file and a sorted transaction file. The output is a new master file that reflects all applied transactions, plus an audit trail file that records every change for regulatory compliance.
Sarah Kim, a mainframe architect at Consolidated Trust, designed the master file update program (MSTRUPD) to demonstrate the canonical update algorithm with all its production-grade features: three-file merge logic, add/change/delete transaction handling, sequence checking, audit trail generation, and comprehensive error handling.
The Problem
Input Files
Old Master File (OLDMAST): 2.4 million records, sorted by account number.
| Field | Picture | Description |
|---|---|---|
| Account Number | 9(10) | Primary key |
| Customer Name | X(30) | Account holder |
| Account Type | X(1) | C=Checking, S=Savings, L=Loan |
| Current Balance | S9(11)V99 COMP-3 | Account balance |
| Last Activity Date | 9(8) | YYYYMMDD |
| Status | X(1) | A=Active, I=Inactive, X=Closed |
| Filler | X(20) | Reserved |
Transaction File (TRANSACT): 50,000-200,000 records per day, sorted by account number and then by transaction sequence within account.
| Field | Picture | Description |
|---|---|---|
| Account Number | 9(10) | Must match master or be a new account |
| Transaction Code | X(1) | A=Add, C=Change, D=Delete |
| Transaction Amount | S9(11)V99 | Signed amount (+ or -) |
| Customer Name | X(30) | Used only for Add transactions |
| Account Type | X(1) | Used only for Add transactions |
| Transaction Date | 9(8) | YYYYMMDD |
| Filler | X(10) | Reserved |
Output Files
New Master File (NEWMAST): Updated master file with all transactions applied.
Audit Trail File (AUDTFILE): One record for every change applied, recording the before-image, after-image, and transaction details.
Update Rules
- Add (A): Create a new account. The transaction account number must NOT exist in the old master. Initial balance comes from the transaction amount.
- Change (C): Apply the transaction amount to the existing balance. The account must exist and be active.
- Delete (D): Mark the account as closed (status = 'X'). The account must exist and have a zero balance.
- Multiple transactions may exist for the same account. They must be applied in sequence order.
- Master records with no matching transactions are carried forward unchanged.
- Both files must be in ascending order by account number. A sequence error in either file is a fatal error.
The Solution
The COBOL Program
IDENTIFICATION DIVISION.
PROGRAM-ID. MSTRUPD.
AUTHOR. SARAH KIM.
DATE-WRITTEN. 2024-12-01.
*================================================================
* PROGRAM: MSTRUPD - MASTER FILE UPDATE
* PURPOSE: Classic sequential master-transaction update.
* Reads sorted old master and transaction files,
* applies adds/changes/deletes, writes new
* master and audit trail. Demonstrates the
* three-file merge algorithm with comprehensive
* error handling.
*================================================================
ENVIRONMENT DIVISION.
INPUT-OUTPUT SECTION.
FILE-CONTROL.
SELECT OLD-MASTER-FILE
ASSIGN TO OLDMAST
ORGANIZATION IS SEQUENTIAL
FILE STATUS IS WS-OLD-STATUS.
SELECT TRANSACTION-FILE
ASSIGN TO TRANSACT
ORGANIZATION IS SEQUENTIAL
FILE STATUS IS WS-TRN-STATUS.
SELECT NEW-MASTER-FILE
ASSIGN TO NEWMAST
ORGANIZATION IS SEQUENTIAL
FILE STATUS IS WS-NEW-STATUS.
SELECT AUDIT-TRAIL-FILE
ASSIGN TO AUDTFILE
ORGANIZATION IS SEQUENTIAL
FILE STATUS IS WS-AUD-STATUS.
DATA DIVISION.
FILE SECTION.
FD OLD-MASTER-FILE
RECORDING MODE IS F
RECORD CONTAINS 80 CHARACTERS
BLOCK CONTAINS 0 RECORDS.
01 FS-OLD-MASTER-REC.
05 FS-OM-ACCOUNT-NO PIC 9(10).
05 FS-OM-CUST-NAME PIC X(30).
05 FS-OM-ACCT-TYPE PIC X(1).
05 FS-OM-BALANCE PIC S9(11)V99 COMP-3.
05 FS-OM-LAST-ACTIVITY PIC 9(8).
05 FS-OM-STATUS PIC X(1).
88 OM-ACTIVE VALUE 'A'.
88 OM-INACTIVE VALUE 'I'.
88 OM-CLOSED VALUE 'X'.
05 FILLER PIC X(22).
FD TRANSACTION-FILE
RECORDING MODE IS F
RECORD CONTAINS 80 CHARACTERS
BLOCK CONTAINS 0 RECORDS.
01 FS-TRANS-REC.
05 FS-TR-ACCOUNT-NO PIC 9(10).
05 FS-TR-TRANS-CODE PIC X(1).
88 TR-ADD VALUE 'A'.
88 TR-CHANGE VALUE 'C'.
88 TR-DELETE VALUE 'D'.
05 FS-TR-AMOUNT PIC S9(11)V99.
05 FS-TR-CUST-NAME PIC X(30).
05 FS-TR-ACCT-TYPE PIC X(1).
05 FS-TR-TRANS-DATE PIC 9(8).
05 FILLER PIC X(16).
FD NEW-MASTER-FILE
RECORDING MODE IS F
RECORD CONTAINS 80 CHARACTERS
BLOCK CONTAINS 0 RECORDS.
01 FS-NEW-MASTER-REC PIC X(80).
FD AUDIT-TRAIL-FILE
RECORDING MODE IS F
RECORD CONTAINS 200 CHARACTERS
BLOCK CONTAINS 0 RECORDS.
01 FS-AUDIT-REC.
05 FS-AU-TIMESTAMP PIC 9(14).
05 FS-AU-ACCOUNT-NO PIC 9(10).
05 FS-AU-ACTION PIC X(6).
05 FS-AU-BEFORE-BAL PIC S9(11)V99.
05 FS-AU-AFTER-BAL PIC S9(11)V99.
05 FS-AU-TRANS-AMOUNT PIC S9(11)V99.
05 FS-AU-TRANS-CODE PIC X(1).
05 FS-AU-STATUS PIC X(1).
05 FS-AU-RESULT PIC X(10).
05 FILLER PIC X(107).
WORKING-STORAGE SECTION.
*----------------------------------------------------------------
* FILE STATUS FIELDS
*----------------------------------------------------------------
01 WS-OLD-STATUS PIC X(2).
88 OLD-OK VALUE "00".
88 OLD-EOF VALUE "10".
01 WS-TRN-STATUS PIC X(2).
88 TRN-OK VALUE "00".
88 TRN-EOF VALUE "10".
01 WS-NEW-STATUS PIC X(2).
88 NEW-OK VALUE "00".
01 WS-AUD-STATUS PIC X(2).
88 AUD-OK VALUE "00".
*----------------------------------------------------------------
* EOF FLAGS
*----------------------------------------------------------------
01 WS-OLD-EOF-FLAG PIC X(1) VALUE 'N'.
88 OLD-AT-EOF VALUE 'Y'.
88 OLD-NOT-EOF VALUE 'N'.
01 WS-TRN-EOF-FLAG PIC X(1) VALUE 'N'.
88 TRN-AT-EOF VALUE 'Y'.
88 TRN-NOT-EOF VALUE 'N'.
*----------------------------------------------------------------
* CURRENT RECORD KEYS (for merge comparison)
* HIGH-VALUES used as sentinel for exhausted file.
*----------------------------------------------------------------
01 WS-OLD-KEY PIC 9(10) VALUE ZERO.
01 WS-TRN-KEY PIC 9(10) VALUE ZERO.
01 WS-PREV-OLD-KEY PIC 9(10) VALUE ZERO.
01 WS-PREV-TRN-KEY PIC 9(10) VALUE ZERO.
01 WS-HIGH-KEY PIC 9(10) VALUE 9999999999.
*----------------------------------------------------------------
* WORK RECORD (new master under construction)
*----------------------------------------------------------------
01 WS-WORK-MASTER.
05 WS-WM-ACCOUNT-NO PIC 9(10).
05 WS-WM-CUST-NAME PIC X(30).
05 WS-WM-ACCT-TYPE PIC X(1).
05 WS-WM-BALANCE PIC S9(11)V99 COMP-3.
05 WS-WM-LAST-ACTIVITY PIC 9(8).
05 WS-WM-STATUS PIC X(1).
88 WM-ACTIVE VALUE 'A'.
88 WM-CLOSED VALUE 'X'.
05 FILLER PIC X(22).
*----------------------------------------------------------------
* BEFORE-IMAGE FOR AUDIT
*----------------------------------------------------------------
01 WS-BEFORE-BALANCE PIC S9(11)V99 COMP-3.
*----------------------------------------------------------------
* COUNTERS
*----------------------------------------------------------------
01 WS-COUNTERS.
05 WS-OLD-READ PIC S9(7) COMP-3 VALUE 0.
05 WS-TRN-READ PIC S9(7) COMP-3 VALUE 0.
05 WS-NEW-WRITTEN PIC S9(7) COMP-3 VALUE 0.
05 WS-ADDS-APPLIED PIC S9(7) COMP-3 VALUE 0.
05 WS-CHANGES-APPLIED PIC S9(7) COMP-3 VALUE 0.
05 WS-DELETES-APPLIED PIC S9(7) COMP-3 VALUE 0.
05 WS-ERRORS-FOUND PIC S9(7) COMP-3 VALUE 0.
05 WS-CARRIED-FWD PIC S9(7) COMP-3 VALUE 0.
05 WS-AUDIT-WRITTEN PIC S9(7) COMP-3 VALUE 0.
01 WS-DISP-COUNT PIC Z,ZZZ,ZZ9.
01 WS-RETURN-CODE PIC S9(4) COMP VALUE 0.
01 WS-AUDIT-TIMESTAMP PIC 9(14).
PROCEDURE DIVISION.
0000-MAIN-CONTROL.
PERFORM 1000-INITIALIZE
PERFORM 2000-UPDATE-PROCESS
UNTIL OLD-AT-EOF AND TRN-AT-EOF
PERFORM 8000-DISPLAY-STATISTICS
PERFORM 9000-FINALIZE
MOVE WS-RETURN-CODE TO RETURN-CODE
STOP RUN
.
1000-INITIALIZE.
DISPLAY "MSTRUPD: Starting master file update"
OPEN INPUT OLD-MASTER-FILE
IF NOT OLD-OK
DISPLAY "MSTRUPD: FATAL - OLDMAST open failed."
" Status: " WS-OLD-STATUS
MOVE 16 TO WS-RETURN-CODE
STOP RUN
END-IF
OPEN INPUT TRANSACTION-FILE
IF NOT TRN-OK
DISPLAY "MSTRUPD: FATAL - TRANSACT open failed."
" Status: " WS-TRN-STATUS
MOVE 16 TO WS-RETURN-CODE
STOP RUN
END-IF
OPEN OUTPUT NEW-MASTER-FILE
IF NOT NEW-OK
DISPLAY "MSTRUPD: FATAL - NEWMAST open failed."
" Status: " WS-NEW-STATUS
MOVE 16 TO WS-RETURN-CODE
STOP RUN
END-IF
OPEN OUTPUT AUDIT-TRAIL-FILE
IF NOT AUD-OK
DISPLAY "MSTRUPD: FATAL - AUDTFILE open failed."
" Status: " WS-AUD-STATUS
MOVE 16 TO WS-RETURN-CODE
STOP RUN
END-IF
* Get current timestamp for audit records
MOVE FUNCTION CURRENT-DATE(1:14)
TO WS-AUDIT-TIMESTAMP
* Prime both files
PERFORM 2100-READ-OLD-MASTER
PERFORM 2200-READ-TRANSACTION
.
2000-UPDATE-PROCESS.
* -------------------------------------------------------
* The three-way merge algorithm:
* Compare the current old master key to the current
* transaction key and take the appropriate action.
*
* Case 1: OLD-KEY < TRN-KEY
* Master record has no transactions. Carry forward.
* Case 2: OLD-KEY = TRN-KEY
* Apply transaction(s) to the master record.
* Case 3: OLD-KEY > TRN-KEY
* Transaction has no matching master. Must be an Add
* or an error.
* -------------------------------------------------------
EVALUATE TRUE
WHEN WS-OLD-KEY < WS-TRN-KEY
* No transactions for this master record
PERFORM 3000-CARRY-FORWARD
WHEN WS-OLD-KEY = WS-TRN-KEY
* Transaction(s) match this master record
PERFORM 4000-APPLY-TRANSACTIONS
WHEN WS-OLD-KEY > WS-TRN-KEY
* Transaction without matching master
PERFORM 5000-PROCESS-UNMATCHED-TRANS
END-EVALUATE
.
2100-READ-OLD-MASTER.
READ OLD-MASTER-FILE
AT END
SET OLD-AT-EOF TO TRUE
MOVE WS-HIGH-KEY TO WS-OLD-KEY
NOT AT END
ADD 1 TO WS-OLD-READ
MOVE FS-OM-ACCOUNT-NO TO WS-OLD-KEY
* Sequence check
IF WS-OLD-KEY < WS-PREV-OLD-KEY
DISPLAY "MSTRUPD: FATAL - Old master "
"out of sequence at record "
WS-OLD-READ
DISPLAY " Key: " WS-OLD-KEY
" Prev: " WS-PREV-OLD-KEY
MOVE 16 TO WS-RETURN-CODE
STOP RUN
END-IF
MOVE WS-OLD-KEY TO WS-PREV-OLD-KEY
END-READ
.
2200-READ-TRANSACTION.
READ TRANSACTION-FILE
AT END
SET TRN-AT-EOF TO TRUE
MOVE WS-HIGH-KEY TO WS-TRN-KEY
NOT AT END
ADD 1 TO WS-TRN-READ
MOVE FS-TR-ACCOUNT-NO TO WS-TRN-KEY
* Sequence check
IF WS-TRN-KEY < WS-PREV-TRN-KEY
DISPLAY "MSTRUPD: FATAL - Transaction "
"file out of sequence at "
"record " WS-TRN-READ
DISPLAY " Key: " WS-TRN-KEY
" Prev: " WS-PREV-TRN-KEY
MOVE 16 TO WS-RETURN-CODE
STOP RUN
END-IF
MOVE WS-TRN-KEY TO WS-PREV-TRN-KEY
END-READ
.
3000-CARRY-FORWARD.
* -------------------------------------------------------
* No transactions for this master record.
* Write it unchanged to the new master file.
* -------------------------------------------------------
MOVE FS-OLD-MASTER-REC TO FS-NEW-MASTER-REC
WRITE FS-NEW-MASTER-REC
IF NOT NEW-OK
DISPLAY "MSTRUPD: Write error on NEWMAST. "
"Status: " WS-NEW-STATUS
ADD 1 TO WS-ERRORS-FOUND
ELSE
ADD 1 TO WS-NEW-WRITTEN
ADD 1 TO WS-CARRIED-FWD
END-IF
PERFORM 2100-READ-OLD-MASTER
.
4000-APPLY-TRANSACTIONS.
* -------------------------------------------------------
* One or more transactions match this master record.
* Copy the master to the work area, then apply each
* matching transaction in sequence.
* -------------------------------------------------------
MOVE FS-OLD-MASTER-REC TO WS-WORK-MASTER
MOVE WS-WM-BALANCE TO WS-BEFORE-BALANCE
* Apply all transactions for this account
PERFORM UNTIL WS-TRN-KEY NOT = WS-OLD-KEY
OR TRN-AT-EOF
EVALUATE TRUE
WHEN TR-ADD
* Error: cannot add an account that exists
DISPLAY "MSTRUPD: ERROR - Add for "
"existing account "
FS-TR-ACCOUNT-NO
ADD 1 TO WS-ERRORS-FOUND
PERFORM 6100-WRITE-AUDIT-ERROR
WHEN TR-CHANGE
IF WS-WM-STATUS = 'A'
ADD FS-TR-AMOUNT TO WS-WM-BALANCE
MOVE FS-TR-TRANS-DATE
TO WS-WM-LAST-ACTIVITY
ADD 1 TO WS-CHANGES-APPLIED
PERFORM 6000-WRITE-AUDIT-SUCCESS
ELSE
DISPLAY "MSTRUPD: ERROR - Change "
"to non-active account "
FS-TR-ACCOUNT-NO
ADD 1 TO WS-ERRORS-FOUND
PERFORM 6100-WRITE-AUDIT-ERROR
END-IF
WHEN TR-DELETE
IF WS-WM-BALANCE = ZERO
SET WM-CLOSED TO TRUE
MOVE FS-TR-TRANS-DATE
TO WS-WM-LAST-ACTIVITY
ADD 1 TO WS-DELETES-APPLIED
PERFORM 6000-WRITE-AUDIT-SUCCESS
ELSE
DISPLAY "MSTRUPD: ERROR - Delete "
"with non-zero balance. "
"Account: "
FS-TR-ACCOUNT-NO
ADD 1 TO WS-ERRORS-FOUND
PERFORM 6100-WRITE-AUDIT-ERROR
END-IF
WHEN OTHER
DISPLAY "MSTRUPD: ERROR - Unknown "
"trans code '"
FS-TR-TRANS-CODE
"' for account "
FS-TR-ACCOUNT-NO
ADD 1 TO WS-ERRORS-FOUND
PERFORM 6100-WRITE-AUDIT-ERROR
END-EVALUATE
PERFORM 2200-READ-TRANSACTION
END-PERFORM
* Write the updated master record
MOVE WS-WORK-MASTER TO FS-NEW-MASTER-REC
WRITE FS-NEW-MASTER-REC
IF NOT NEW-OK
DISPLAY "MSTRUPD: Write error on NEWMAST. "
"Status: " WS-NEW-STATUS
ADD 1 TO WS-ERRORS-FOUND
ELSE
ADD 1 TO WS-NEW-WRITTEN
END-IF
PERFORM 2100-READ-OLD-MASTER
.
5000-PROCESS-UNMATCHED-TRANS.
* -------------------------------------------------------
* Transaction has no matching master record.
* If it is an Add, create a new master record.
* If it is anything else, it is an error.
* -------------------------------------------------------
IF TR-ADD
* Create new account
INITIALIZE WS-WORK-MASTER
MOVE FS-TR-ACCOUNT-NO TO WS-WM-ACCOUNT-NO
MOVE FS-TR-CUST-NAME TO WS-WM-CUST-NAME
MOVE FS-TR-ACCT-TYPE TO WS-WM-ACCT-TYPE
MOVE FS-TR-AMOUNT TO WS-WM-BALANCE
MOVE FS-TR-TRANS-DATE TO WS-WM-LAST-ACTIVITY
SET WM-ACTIVE TO TRUE
MOVE ZERO TO WS-BEFORE-BALANCE
* Write the new master record
MOVE WS-WORK-MASTER TO FS-NEW-MASTER-REC
WRITE FS-NEW-MASTER-REC
IF NOT NEW-OK
DISPLAY "MSTRUPD: Write error on NEWMAST."
" Status: " WS-NEW-STATUS
ADD 1 TO WS-ERRORS-FOUND
ELSE
ADD 1 TO WS-NEW-WRITTEN
ADD 1 TO WS-ADDS-APPLIED
PERFORM 6000-WRITE-AUDIT-SUCCESS
END-IF
ELSE
* Error: Change or Delete without matching master
DISPLAY "MSTRUPD: ERROR - Trans code '"
FS-TR-TRANS-CODE
"' for non-existent account "
FS-TR-ACCOUNT-NO
ADD 1 TO WS-ERRORS-FOUND
PERFORM 6100-WRITE-AUDIT-ERROR
END-IF
PERFORM 2200-READ-TRANSACTION
.
6000-WRITE-AUDIT-SUCCESS.
INITIALIZE FS-AUDIT-REC
MOVE WS-AUDIT-TIMESTAMP TO FS-AU-TIMESTAMP
MOVE FS-TR-ACCOUNT-NO TO FS-AU-ACCOUNT-NO
MOVE FS-TR-TRANS-CODE TO FS-AU-TRANS-CODE
MOVE WS-BEFORE-BALANCE TO FS-AU-BEFORE-BAL
MOVE WS-WM-BALANCE TO FS-AU-AFTER-BAL
MOVE FS-TR-AMOUNT TO FS-AU-TRANS-AMOUNT
MOVE WS-WM-STATUS TO FS-AU-STATUS
EVALUATE TRUE
WHEN TR-ADD
MOVE "ADD " TO FS-AU-ACTION
WHEN TR-CHANGE
MOVE "CHANGE" TO FS-AU-ACTION
WHEN TR-DELETE
MOVE "DELETE" TO FS-AU-ACTION
END-EVALUATE
MOVE "SUCCESS " TO FS-AU-RESULT
WRITE FS-AUDIT-REC
IF AUD-OK
ADD 1 TO WS-AUDIT-WRITTEN
END-IF
.
6100-WRITE-AUDIT-ERROR.
INITIALIZE FS-AUDIT-REC
MOVE WS-AUDIT-TIMESTAMP TO FS-AU-TIMESTAMP
MOVE FS-TR-ACCOUNT-NO TO FS-AU-ACCOUNT-NO
MOVE FS-TR-TRANS-CODE TO FS-AU-TRANS-CODE
MOVE FS-TR-AMOUNT TO FS-AU-TRANS-AMOUNT
MOVE "REJECTED " TO FS-AU-RESULT
WRITE FS-AUDIT-REC
IF AUD-OK
ADD 1 TO WS-AUDIT-WRITTEN
END-IF
.
8000-DISPLAY-STATISTICS.
DISPLAY " "
DISPLAY "MSTRUPD: ===== PROCESSING STATISTICS ====="
MOVE WS-OLD-READ TO WS-DISP-COUNT
DISPLAY " Old master records read: " WS-DISP-COUNT
MOVE WS-TRN-READ TO WS-DISP-COUNT
DISPLAY " Transactions read: " WS-DISP-COUNT
MOVE WS-NEW-WRITTEN TO WS-DISP-COUNT
DISPLAY " New master records: " WS-DISP-COUNT
DISPLAY " "
MOVE WS-CARRIED-FWD TO WS-DISP-COUNT
DISPLAY " Carried forward: " WS-DISP-COUNT
MOVE WS-ADDS-APPLIED TO WS-DISP-COUNT
DISPLAY " Accounts added: " WS-DISP-COUNT
MOVE WS-CHANGES-APPLIED TO WS-DISP-COUNT
DISPLAY " Changes applied: " WS-DISP-COUNT
MOVE WS-DELETES-APPLIED TO WS-DISP-COUNT
DISPLAY " Accounts deleted: " WS-DISP-COUNT
MOVE WS-ERRORS-FOUND TO WS-DISP-COUNT
DISPLAY " Errors: " WS-DISP-COUNT
MOVE WS-AUDIT-WRITTEN TO WS-DISP-COUNT
DISPLAY " Audit records written: " WS-DISP-COUNT
DISPLAY "MSTRUPD: =================================="
* Verification: new master count should equal
* old master + adds - deletes
DISPLAY " "
DISPLAY "MSTRUPD: Verification:"
DISPLAY " Old + Adds - Deletes = "
"Expected New Master Count"
* Set return code
IF WS-ERRORS-FOUND > ZERO
MOVE 4 TO WS-RETURN-CODE
DISPLAY "MSTRUPD: Completed with errors. RC=4"
ELSE
MOVE 0 TO WS-RETURN-CODE
DISPLAY "MSTRUPD: Completed successfully. RC=0"
END-IF
.
9000-FINALIZE.
CLOSE OLD-MASTER-FILE
TRANSACTION-FILE
NEW-MASTER-FILE
AUDIT-TRAIL-FILE
.
The Companion JCL
//MSTRUPJ JOB (ACCT),'MASTER FILE UPDATE',
// CLASS=A,MSGCLASS=X,
// MSGLEVEL=(1,1),
// NOTIFY=&SYSUID
//*================================================================
//* JOB: MSTRUPJ
//* PURPOSE: NIGHTLY MASTER FILE UPDATE FOR CONSOLIDATED TRUST
//* STEP 1: SORT TRANSACTIONS BY ACCOUNT NUMBER
//* STEP 2: APPLY TRANSACTIONS TO MASTER FILE
//* STEP 3: VERIFY NEW MASTER RECORD COUNT
//*================================================================
//*
//*-------- STEP 1: SORT TRANSACTIONS BY ACCOUNT NUMBER -----------
//*
//SORT01 EXEC PGM=SORT
//SORTIN DD DSN=CONTRUST.DAILY.TRANS.D&LYYMMDD,
// DISP=SHR
//SORTOUT DD DSN=CONTRUST.DAILY.TRANS.SORTED,
// DISP=(NEW,CATLG,DELETE),
// SPACE=(CYL,(5,2),RLSE),
// DCB=(RECFM=FB,LRECL=80,BLKSIZE=27920)
//SYSOUT DD SYSOUT=*
//SYSIN DD *
SORT FIELDS=(1,10,CH,A)
/*
//*
//*-------- STEP 2: APPLY TRANSACTIONS TO MASTER -----------------
//*
//UPDATE EXEC PGM=MSTRUPD,
// COND=(0,NE,SORT01)
//STEPLIB DD DSN=CONTRUST.PROD.LOADLIB,DISP=SHR
//*
//* INPUT: OLD (CURRENT) MASTER FILE
//*
//OLDMAST DD DSN=CONTRUST.ACCOUNT.MASTER,
// DISP=SHR
//*
//* INPUT: SORTED TRANSACTIONS
//*
//TRANSACT DD DSN=CONTRUST.DAILY.TRANS.SORTED,
// DISP=SHR
//*
//* OUTPUT: NEW (UPDATED) MASTER FILE
//*
//NEWMAST DD DSN=CONTRUST.ACCOUNT.MASTER.NEW,
// DISP=(NEW,CATLG,DELETE),
// SPACE=(CYL,(100,20),RLSE),
// DCB=(RECFM=FB,LRECL=80,BLKSIZE=27920)
//*
//* OUTPUT: AUDIT TRAIL
//*
//AUDTFILE DD DSN=CONTRUST.AUDIT.TRAIL.D&LYYMMDD,
// DISP=(NEW,CATLG,DELETE),
// SPACE=(CYL,(5,2),RLSE),
// DCB=(RECFM=FB,LRECL=200,BLKSIZE=27800)
//*
//SYSOUT DD SYSOUT=*
//*
//*-------- STEP 3: RENAME NEW MASTER TO PRODUCTION NAME ----------
//* Only if Step 2 completed successfully (RC=0)
//*
//RENAME EXEC PGM=IDCAMS,
// COND=(0,NE,UPDATE)
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
ALTER CONTRUST.ACCOUNT.MASTER -
NEWNAME(CONTRUST.ACCOUNT.MASTER.BACKUP)
ALTER CONTRUST.ACCOUNT.MASTER.NEW -
NEWNAME(CONTRUST.ACCOUNT.MASTER)
/*
Solution Walkthrough
The Three-Way Merge Algorithm
The core of the program is the EVALUATE in paragraph 2000-UPDATE-PROCESS. It compares the current key from each file:
- OLD-KEY < TRN-KEY: The current master record has no transactions. It is written to the new master file unchanged ("carried forward").
- OLD-KEY = TRN-KEY: One or more transactions apply to this master record. All matching transactions are applied before writing the updated record.
- OLD-KEY > TRN-KEY: A transaction exists for an account number that is not in the master file. This must be an Add transaction (or an error).
The HIGH-VALUES Sentinel
When a file reaches EOF, its key is set to WS-HIGH-KEY (9999999999). This ensures that the exhausted file never "wins" the comparison:
AT END
SET OLD-AT-EOF TO TRUE
MOVE WS-HIGH-KEY TO WS-OLD-KEY
This eliminates the need for separate logic to handle "master exhausted but transactions remain" and "transactions exhausted but master records remain." The sentinel value makes both cases fall naturally out of the normal comparison logic.
Sequence Checking
Both read paragraphs verify that keys are in ascending order:
IF WS-OLD-KEY < WS-PREV-OLD-KEY
DISPLAY "MSTRUPD: FATAL - Old master "
"out of sequence"
MOVE 16 TO WS-RETURN-CODE
STOP RUN
END-IF
A sequence error in either file would produce incorrect results silently -- some records would be missed, others double-processed. The sequence check transforms a silent data corruption into a loud, immediate failure. This is essential in production.
Multiple Transactions Per Account
The inner loop in 4000-APPLY-TRANSACTIONS handles multiple transactions for the same account:
PERFORM UNTIL WS-TRN-KEY NOT = WS-OLD-KEY
OR TRN-AT-EOF
This loop continues reading transactions as long as they match the current master key. Each transaction is applied to the work copy of the master record, and only after all matching transactions have been processed is the updated master written.
The JCL Job Stream
The JCL demonstrates a three-step job: 1. SORT: Sorts the day's transactions by account number, which is a prerequisite for the merge algorithm. 2. UPDATE: Runs the COBOL program, conditional on successful sort completion. 3. RENAME: Uses IDCAMS to rename the new master to the production name, conditional on successful update. The old master is renamed as a backup.
The COND parameter (0,NE,UPDATE) means "skip this step if UPDATE's return code is NOT equal to 0" -- in other words, only rename if the update completed without errors.
Lessons Learned
1. Both Files Must Be Sorted on the Same Key
The merge algorithm fundamentally depends on both files being in ascending order by the same key. The JCL sorts the transaction file to guarantee this. The master file, produced by the previous night's update, is already sorted.
2. HIGH-VALUES Eliminates Special-Case Logic
Setting the exhausted file's key to a sentinel value that is higher than any real key is an elegant technique that reduces the merge logic from five cases (both active, master only, trans only, both exhausted, mixed) to three cases (less, equal, greater).
3. The Audit Trail Is Not Optional
Every change to the master file is recorded in the audit trail with before-image, after-image, and the transaction that caused the change. This is a regulatory requirement for banking systems and an essential debugging tool when balances are questioned.
4. Write the New Master, Do Not Update In Place
The program reads the old master and writes a completely new master file. It never updates the old file in place. If the program fails midway, the old master is intact and the job can be restarted. This is the fundamental safety principle of sequential file updates.
5. Carry-Forward Records Are the Majority
In a typical run, 2.4 million master records are read but only 50,000-200,000 have matching transactions. The vast majority are simply carried forward. The program must handle this efficiently -- and it does, since carrying forward requires only a read and a write with no computation.
Discussion Questions
-
The program aborts immediately on a sequence error. An alternative would be to log the error and continue, skipping the out-of-sequence record. What are the risks of each approach? Which is safer for a banking master file?
-
The delete transaction requires a zero balance. What would happen if a customer had a $0.01 balance and a pending delete transaction? How would you handle the business requirement of "close the account after applying all pending transactions"?
-
The JCL renames the new master to the production name only if the update succeeds. What happens if the rename step fails? How would you design a more robust cutover procedure?
-
The program processes all transactions for the same account in a single batch. What happens if two conflicting transactions exist for the same account (for example, a Delete followed by a Change)? How does the order of transactions affect the result?
-
The audit trail file grows without bound -- one run can add 200,000 records. How would you manage the audit trail over time? Consider archival, compression, and the regulatory requirement to retain records for seven years.
-
What would change if this program needed to handle a master file with 100 million records instead of 2.4 million? Consider I/O optimization, buffer sizes, and the JCL SPACE parameters.
-
The program uses
BLOCK CONTAINS 0 RECORDSto let the system determine the optimal block size. What determines this block size on z/OS, and how does it affect I/O performance for sequential files?