Case Study 2: GDG Management for Daily Batch Processing
Background
Commonwealth National Bank (CNB) runs a batch processing cycle every night that touches nearly every dataset in the core banking system. The cycle begins at 10:00 PM when the online system quiesces and ends by 5:30 AM when the branches reopen. Within this 7.5-hour window, the batch cycle must complete transaction posting, interest accrual, fee assessment, regulatory reporting, account statement generation, and backup operations.
A central challenge in this batch architecture is managing the daily files that accumulate over time. Every night, the system produces a new daily transaction file, a new account snapshot, a new set of backup files, and new report archives. These files must be retained for defined periods -- the transaction file for 90 days (regulatory requirement), the account snapshot for 30 days (operational recovery), and the report archives for 7 years (audit requirement).
For the first 15 years of the system's life, the operations team managed daily files by appending the date to the dataset name: CNB.CORE.TRAN.D20240115, CNB.CORE.TRAN.D20240116, and so on. This approach created several persistent problems:
- JCL maintenance: Every job that referenced daily files required date-variable substitution using symbolic parameters or PROC overrides. When a job needed to reference "yesterday's file," the JCL had to compute the previous date, accounting for weekends, holidays, month boundaries, and leap years.
- Housekeeping complexity: A separate housekeeping job ran weekly to delete expired files, using date arithmetic to identify candidates. This job failed periodically on month-end boundaries and during the February 28/29 transition.
- Restart difficulty: When a batch job abended and needed to be restarted, the operations team had to manually verify which date-stamped files had been created and which needed to be recreated.
The bank migrated to Generation Data Groups (GDGs) in 2010, and the improvement in operational reliability was immediate and dramatic. This case study documents the GDG architecture, the JCL patterns that reference generations, and the housekeeping procedures that keep the system running cleanly.
GDG Architecture Design
The team defined four GDG bases, each with retention characteristics matched to the data it contains:
| GDG Base Name | Purpose | Generations | Scratch/NoScratch |
|---|---|---|---|
| CNB.CORE.TRAN.DAILY | Daily transaction files | 90 | SCRATCH |
| CNB.CORE.SNAP.ACCT | Account master snapshots | 30 | SCRATCH |
| CNB.CORE.BKUP.MASTER | Master file backups | 14 | NOSCRATCH |
| CNB.CORE.REPT.ARCHIVE | Report archives | 255 | NOSCRATCH |
Defining the GDG Bases
The GDG bases are defined using IDCAMS DEFINE GDG, followed by a model DSCB (Data Set Control Block) that provides the DCB attributes for each generation.
//CNBGDGDF JOB (ACCT),'CNB GDG DEFINE',
// CLASS=A,MSGCLASS=X,MSGLEVEL=(1,1),
// NOTIFY=&SYSUID
//*
//*================================================================*
//* JOB: CNBGDGDF - DEFINE GDG BASES AND MODEL DSCBS
//* DATE: 2024-01-15
//*================================================================*
//*
//*------------------------------------------------------------*
//* STEP 1: DELETE EXISTING GDG BASES (IF RERUN)
//*------------------------------------------------------------*
//DELGDG EXEC PGM=IDCAMS
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
DELETE CNB.CORE.TRAN.DAILY -
GDG -
FORCE -
PURGE
IF LASTCC <= 8 THEN -
SET MAXCC = 0
DELETE CNB.CORE.SNAP.ACCT -
GDG -
FORCE -
PURGE
IF LASTCC <= 8 THEN -
SET MAXCC = 0
DELETE CNB.CORE.BKUP.MASTER -
GDG -
FORCE -
PURGE
IF LASTCC <= 8 THEN -
SET MAXCC = 0
DELETE CNB.CORE.REPT.ARCHIVE -
GDG -
FORCE -
PURGE
IF LASTCC <= 8 THEN -
SET MAXCC = 0
/*
//*
//*------------------------------------------------------------*
//* STEP 2: DEFINE GDG BASES
//*------------------------------------------------------------*
//DEFGDG EXEC PGM=IDCAMS
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
DEFINE GDG -
(NAME(CNB.CORE.TRAN.DAILY) -
LIMIT(90) -
NOEMPTY -
SCRATCH)
DEFINE GDG -
(NAME(CNB.CORE.SNAP.ACCT) -
LIMIT(30) -
NOEMPTY -
SCRATCH)
DEFINE GDG -
(NAME(CNB.CORE.BKUP.MASTER) -
LIMIT(14) -
NOEMPTY -
NOSCRATCH)
DEFINE GDG -
(NAME(CNB.CORE.REPT.ARCHIVE) -
LIMIT(255) -
NOEMPTY -
NOSCRATCH)
/*
//*
//*------------------------------------------------------------*
//* STEP 3: CREATE MODEL DSCBS FOR EACH GDG BASE
//* MODEL DSCB PROVIDES DEFAULT DCB ATTRIBUTES
//*------------------------------------------------------------*
//MODLTRAN EXEC PGM=IEFBR14
//MODELDCB DD DSN=CNB.CORE.TRAN.DAILY.MODEL,
// DISP=(NEW,CATLG,DELETE),
// UNIT=SYSDA,
// SPACE=(TRK,0),
// DCB=(RECFM=FB,LRECL=250,BLKSIZE=27750)
//*
//MODLSNAP EXEC PGM=IEFBR14
//MODELDCB DD DSN=CNB.CORE.SNAP.ACCT.MODEL,
// DISP=(NEW,CATLG,DELETE),
// UNIT=SYSDA,
// SPACE=(TRK,0),
// DCB=(RECFM=FB,LRECL=500,BLKSIZE=27500)
//*
//MODLBKUP EXEC PGM=IEFBR14
//MODELDCB DD DSN=CNB.CORE.BKUP.MASTER.MODEL,
// DISP=(NEW,CATLG,DELETE),
// UNIT=SYSDA,
// SPACE=(TRK,0),
// DCB=(RECFM=FB,LRECL=500,BLKSIZE=27500)
//*
//MODLREPT EXEC PGM=IEFBR14
//MODELDCB DD DSN=CNB.CORE.REPT.ARCHIVE.MODEL,
// DISP=(NEW,CATLG,DELETE),
// UNIT=SYSDA,
// SPACE=(TRK,0),
// DCB=(RECFM=VBA,LRECL=137,BLKSIZE=27998)
Critical Design Decisions
NOEMPTY vs. EMPTY: The NOEMPTY option means that when the generation limit is reached, only the oldest generation is uncataloged (and optionally scratched). With EMPTY, all existing generations would be uncataloged when the limit is exceeded -- a catastrophic behavior for a banking system that would delete 89 days of retained data when the 91st generation is created.
SCRATCH vs. NOSCRATCH: SCRATCH means that when a generation is uncataloged (rolls off the GDG), the physical dataset is also deleted. This is appropriate for transaction files and snapshots where the data has no value after the retention period. NOSCRATCH means the dataset is uncataloged but not deleted -- it remains on the volume and can be accessed by its absolute name (G0000V00 format). This is required for backup files and report archives that may need to be recalled for audit purposes long after they roll off the GDG.
LIMIT(255) for report archives: The maximum GDG limit on z/OS is 255 generations. For 7-year retention of daily reports, this is insufficient (7 x 365 = 2,555 generations needed). The team addresses this by archiving older generations to tape through the HSM (Hierarchical Storage Manager) migration policy, and the NOSCRATCH attribute ensures that migrated generations are not deleted when they roll off the GDG catalog.
GDG References in the Batch Job Stream
The power of GDGs becomes apparent in the JCL for the nightly batch cycle. Relative generation references (+1, 0, -1) eliminate all date arithmetic and make the JCL completely date-independent.
The End-of-Day Batch Job Stream
//CNBEOD01 JOB (ACCT),'CNB EOD STEP 1',
// CLASS=A,MSGCLASS=X,MSGLEVEL=(1,1),
// NOTIFY=&SYSUID
//*
//*================================================================*
//* END-OF-DAY PROCESSING - STEP 1: TRANSACTION POSTING
//* READS TODAY'S TRANSACTIONS, POSTS TO ACCOUNT MASTER,
//* CREATES NEW TRANSACTION ARCHIVE GENERATION
//*================================================================*
//*
//POSTING EXEC PGM=CNBPOST1
//STEPLIB DD DSN=CNB.CORE.PROD.LOADLIB,DISP=SHR
//*
//* TODAY'S ONLINE TRANSACTION FILE (NOT A GDG)
//TRANIN DD DSN=CNB.CORE.TRAN.ONLINE,DISP=SHR
//*
//* ACCOUNT MASTER - UPDATED IN PLACE
//ACCTMAST DD DSN=CNB.CORE.MAST.ACCT,DISP=SHR
//*
//* CREATE NEW GENERATION OF TRANSACTION ARCHIVE (+1)
//TRANARCH DD DSN=CNB.CORE.TRAN.DAILY(+1),
// DISP=(NEW,CATLG,DELETE),
// UNIT=SYSDA,
// SPACE=(CYL,(120,24),RLSE),
// DCB=(RECFM=FB,LRECL=250,BLKSIZE=27750)
//*
//* REFERENCE YESTERDAY'S ARCHIVE FOR COMPARISON (-1 RELATIVE)
//* NOTE: AT THIS POINT IN THE JOB, (+1) IS NOT YET CATALOGED
//* SO (0) STILL REFERS TO YESTERDAY'S FILE
//YESTARCH DD DSN=CNB.CORE.TRAN.DAILY(0),
// DISP=SHR
//*
//SYSOUT DD SYSOUT=*
//*
//*================================================================*
//* END-OF-DAY PROCESSING - STEP 2: ACCOUNT SNAPSHOT
//* CREATES A POINT-IN-TIME COPY OF THE ACCOUNT MASTER
//* AFTER POSTING IS COMPLETE
//*================================================================*
//*
//SNAPSHOT EXEC PGM=CNBSNAP1,COND=(0,NE,POSTING)
//STEPLIB DD DSN=CNB.CORE.PROD.LOADLIB,DISP=SHR
//*
//* ACCOUNT MASTER (JUST UPDATED BY POSTING STEP)
//ACCTMAST DD DSN=CNB.CORE.MAST.ACCT,DISP=SHR
//*
//* CREATE NEW GENERATION OF ACCOUNT SNAPSHOT (+1)
//ACCTSNAP DD DSN=CNB.CORE.SNAP.ACCT(+1),
// DISP=(NEW,CATLG,DELETE),
// UNIT=SYSDA,
// SPACE=(CYL,(950,190),RLSE),
// DCB=(RECFM=FB,LRECL=500,BLKSIZE=27500)
//*
//SYSOUT DD SYSOUT=*
//*
//*================================================================*
//* END-OF-DAY PROCESSING - STEP 3: BACKUP MASTER FILES
//* CREATES BACKUP COPIES OF ALL MASTER FILES
//*================================================================*
//*
//BACKUP EXEC PGM=CNBBKUP1,COND=(0,NE,SNAPSHOT)
//STEPLIB DD DSN=CNB.CORE.PROD.LOADLIB,DISP=SHR
//*
//* INPUT: CURRENT MASTER FILES
//ACCTMAST DD DSN=CNB.CORE.MAST.ACCT,DISP=SHR
//CUSTMAST DD DSN=CNB.CORE.MAST.CUST,DISP=SHR
//*
//* OUTPUT: NEW BACKUP GENERATION
//BKUPOUT DD DSN=CNB.CORE.BKUP.MASTER(+1),
// DISP=(NEW,CATLG,DELETE),
// UNIT=SYSDA,
// SPACE=(CYL,(1200,240),RLSE),
// DCB=(RECFM=FB,LRECL=500,BLKSIZE=27500)
//*
//SYSOUT DD SYSOUT=*
//*
//*================================================================*
//* END-OF-DAY PROCESSING - STEP 4: RECONCILIATION
//* COMPARES TODAY'S SNAPSHOT TO YESTERDAY'S SNAPSHOT
//* TO VERIFY POSTING ACCURACY
//*================================================================*
//*
//RECON EXEC PGM=CNBRECON,COND=(0,NE,BACKUP)
//STEPLIB DD DSN=CNB.CORE.PROD.LOADLIB,DISP=SHR
//*
//* TODAY'S SNAPSHOT - THE ONE WE JUST CREATED
//* NOW IT IS THE CURRENT (0) GENERATION
//TODAYSNP DD DSN=CNB.CORE.SNAP.ACCT(0),
// DISP=SHR
//*
//* YESTERDAY'S SNAPSHOT - ONE GENERATION BACK
//YESTSNAP DD DSN=CNB.CORE.SNAP.ACCT(-1),
// DISP=SHR
//*
//* TODAY'S POSTED TRANSACTIONS
//TODAYTXN DD DSN=CNB.CORE.TRAN.DAILY(0),
// DISP=SHR
//*
//RECONRPT DD SYSOUT=*
//SYSOUT DD SYSOUT=*
Understanding Relative Generation References
The relative generation numbering system is the key to GDG usability:
-
(+1) -- The next generation to be created. This reference creates a new generation dataset. Within a single job, each (+1) reference to the same GDG base creates the same new generation. The generation is cataloged at end-of-step.
-
(0) -- The current (most recent) generation. Before any (+1) is cataloged in the current job, (0) refers to the most recently cataloged generation from a previous job. After a (+1) is cataloged (at end-of-step), the reference shifts: (0) now refers to the generation just created.
-
(-1) -- One generation back from current. After (+1) is cataloged, (-1) refers to what was previously (0).
This shifting reference is the most common source of confusion in GDG processing. Consider the reconciliation step above:
- The POSTING step creates
CNB.CORE.TRAN.DAILY(+1), which is cataloged at end-of-step - The SNAPSHOT step creates
CNB.CORE.SNAP.ACCT(+1), cataloged at end-of-step - In the RECON step,
CNB.CORE.SNAP.ACCT(0)now refers to the snapshot just created by SNAPSHOT, andCNB.CORE.SNAP.ACCT(-1)refers to yesterday's snapshot - Similarly,
CNB.CORE.TRAN.DAILY(0)now refers to the transaction archive just created by POSTING
This behavior is correct and intentional. The reconciliation step compares today's data (the generations just created) against yesterday's data (the previous generations).
COBOL Program for GDG-Aware Reconciliation
The reconciliation program reads two generations of the account snapshot and the current transaction archive, then verifies that the balance changes match the posted transactions:
IDENTIFICATION DIVISION.
PROGRAM-ID. CNBRECON.
*================================================================*
* PROGRAM: CNBRECON - DAILY RECONCILIATION
* PURPOSE: VERIFY THAT ACCOUNT BALANCE CHANGES BETWEEN
* YESTERDAY'S AND TODAY'S SNAPSHOTS MATCH THE
* POSTED TRANSACTIONS FOR THE DAY.
*================================================================*
ENVIRONMENT DIVISION.
INPUT-OUTPUT SECTION.
FILE-CONTROL.
SELECT TODAY-SNAPSHOT
ASSIGN TO TODAYSNP
ORGANIZATION IS SEQUENTIAL
FILE STATUS IS WS-TODAY-STATUS.
SELECT YESTERDAY-SNAPSHOT
ASSIGN TO YESTSNAP
ORGANIZATION IS SEQUENTIAL
FILE STATUS IS WS-YEST-STATUS.
SELECT TRANSACTION-FILE
ASSIGN TO TODAYTXN
ORGANIZATION IS SEQUENTIAL
FILE STATUS IS WS-TRAN-STATUS.
SELECT RECON-REPORT
ASSIGN TO RECONRPT
ORGANIZATION IS SEQUENTIAL
FILE STATUS IS WS-REPT-STATUS.
DATA DIVISION.
FILE SECTION.
FD TODAY-SNAPSHOT
RECORDING MODE IS F
RECORD CONTAINS 500 CHARACTERS.
01 TODAY-SNAP-REC.
05 TS-ACCOUNT-NUMBER PIC X(10).
05 TS-CUSTOMER-ID PIC X(10).
05 TS-ACCOUNT-TYPE PIC X(2).
05 TS-CURRENT-BALANCE PIC S9(13)V99 COMP-3.
05 TS-AVAILABLE-BALANCE PIC S9(13)V99 COMP-3.
05 FILLER PIC X(462).
FD YESTERDAY-SNAPSHOT
RECORDING MODE IS F
RECORD CONTAINS 500 CHARACTERS.
01 YEST-SNAP-REC.
05 YS-ACCOUNT-NUMBER PIC X(10).
05 YS-CUSTOMER-ID PIC X(10).
05 YS-ACCOUNT-TYPE PIC X(2).
05 YS-CURRENT-BALANCE PIC S9(13)V99 COMP-3.
05 YS-AVAILABLE-BALANCE PIC S9(13)V99 COMP-3.
05 FILLER PIC X(462).
FD TRANSACTION-FILE
RECORDING MODE IS F
RECORD CONTAINS 250 CHARACTERS.
01 TRANS-REC.
05 TX-ACCOUNT-NUMBER PIC X(10).
05 TX-BRANCH-CODE PIC X(4).
05 TX-REGION-CODE PIC X(2).
05 TX-TRANS-DATE PIC 9(8).
05 TX-TRANS-TIME PIC 9(6).
05 TX-TRANS-TYPE PIC X(2).
05 TX-AMOUNT PIC S9(11)V99.
05 FILLER PIC X(205).
FD RECON-REPORT
RECORDING MODE IS F
RECORD CONTAINS 133 CHARACTERS.
01 REPORT-LINE PIC X(133).
WORKING-STORAGE SECTION.
01 WS-FILE-STATUSES.
05 WS-TODAY-STATUS PIC X(2).
05 WS-YEST-STATUS PIC X(2).
05 WS-TRAN-STATUS PIC X(2).
05 WS-REPT-STATUS PIC X(2).
01 WS-FLAGS.
05 WS-TODAY-EOF PIC X(1) VALUE 'N'.
88 TODAY-AT-END VALUE 'Y'.
05 WS-YEST-EOF PIC X(1) VALUE 'N'.
88 YEST-AT-END VALUE 'Y'.
05 WS-TRAN-EOF PIC X(1) VALUE 'N'.
88 TRAN-AT-END VALUE 'Y'.
01 WS-COUNTERS.
05 WS-ACCOUNTS-CHECKED PIC 9(9) VALUE ZERO.
05 WS-ACCOUNTS-MATCHED PIC 9(9) VALUE ZERO.
05 WS-ACCOUNTS-MISMATCHED PIC 9(9) VALUE ZERO.
05 WS-NEW-ACCOUNTS PIC 9(9) VALUE ZERO.
05 WS-CLOSED-ACCOUNTS PIC 9(9) VALUE ZERO.
05 WS-TRANS-PROCESSED PIC 9(9) VALUE ZERO.
01 WS-ACCUMULATORS.
05 WS-TRAN-NET-AMOUNT PIC S9(13)V99 COMP-3
VALUE ZERO.
05 WS-BALANCE-CHANGE PIC S9(13)V99 COMP-3
VALUE ZERO.
05 WS-TOTAL-VARIANCE PIC S9(13)V99 COMP-3
VALUE ZERO.
05 WS-CURRENT-ACCT-NET PIC S9(13)V99 COMP-3
VALUE ZERO.
01 WS-HOLD-ACCOUNT PIC X(10) VALUE SPACES.
01 WS-DETAIL-LINE.
05 FILLER PIC X(1) VALUE SPACE.
05 DL-ACCOUNT PIC X(10).
05 FILLER PIC X(2) VALUE SPACES.
05 DL-YEST-BALANCE PIC -(13)9.99.
05 FILLER PIC X(2) VALUE SPACES.
05 DL-TODAY-BALANCE PIC -(13)9.99.
05 FILLER PIC X(2) VALUE SPACES.
05 DL-TRAN-NET PIC -(13)9.99.
05 FILLER PIC X(2) VALUE SPACES.
05 DL-VARIANCE PIC -(13)9.99.
05 FILLER PIC X(2) VALUE SPACES.
05 DL-STATUS PIC X(12).
01 WS-SUMMARY-LINE.
05 FILLER PIC X(1) VALUE SPACE.
05 SL-LABEL PIC X(30).
05 SL-COUNT PIC ZZZ,ZZZ,ZZ9.
05 FILLER PIC X(80) VALUE SPACES.
PROCEDURE DIVISION.
0000-MAIN.
PERFORM 1000-INITIALIZE
PERFORM 2000-PROCESS-RECONCILIATION
PERFORM 3000-PRINT-SUMMARY
PERFORM 9000-TERMINATE
STOP RUN.
1000-INITIALIZE.
OPEN INPUT TODAY-SNAPSHOT
OPEN INPUT YESTERDAY-SNAPSHOT
OPEN INPUT TRANSACTION-FILE
OPEN OUTPUT RECON-REPORT
PERFORM 8100-READ-TODAY
PERFORM 8200-READ-YESTERDAY
PERFORM 8300-READ-TRANSACTION.
2000-PROCESS-RECONCILIATION.
PERFORM UNTIL TODAY-AT-END AND YEST-AT-END
ADD 1 TO WS-ACCOUNTS-CHECKED
EVALUATE TRUE
WHEN TODAY-AT-END
PERFORM 2300-CLOSED-ACCOUNT
PERFORM 8200-READ-YESTERDAY
WHEN YEST-AT-END
PERFORM 2200-NEW-ACCOUNT
PERFORM 8100-READ-TODAY
WHEN TS-ACCOUNT-NUMBER <
YS-ACCOUNT-NUMBER
PERFORM 2200-NEW-ACCOUNT
PERFORM 8100-READ-TODAY
WHEN TS-ACCOUNT-NUMBER >
YS-ACCOUNT-NUMBER
PERFORM 2300-CLOSED-ACCOUNT
PERFORM 8200-READ-YESTERDAY
WHEN OTHER
PERFORM 2100-COMPARE-ACCOUNT
PERFORM 8100-READ-TODAY
PERFORM 8200-READ-YESTERDAY
END-EVALUATE
END-PERFORM.
2100-COMPARE-ACCOUNT.
MOVE ZERO TO WS-CURRENT-ACCT-NET
PERFORM 2110-SUM-TRANSACTIONS
COMPUTE WS-BALANCE-CHANGE =
TS-CURRENT-BALANCE - YS-CURRENT-BALANCE
IF WS-BALANCE-CHANGE = WS-CURRENT-ACCT-NET
ADD 1 TO WS-ACCOUNTS-MATCHED
ELSE
ADD 1 TO WS-ACCOUNTS-MISMATCHED
COMPUTE WS-TOTAL-VARIANCE =
WS-TOTAL-VARIANCE +
(WS-BALANCE-CHANGE - WS-CURRENT-ACCT-NET)
PERFORM 2400-WRITE-MISMATCH
END-IF.
2110-SUM-TRANSACTIONS.
PERFORM UNTIL TRAN-AT-END
OR TX-ACCOUNT-NUMBER >
TS-ACCOUNT-NUMBER
IF TX-ACCOUNT-NUMBER = TS-ACCOUNT-NUMBER
ADD TX-AMOUNT TO WS-CURRENT-ACCT-NET
ADD 1 TO WS-TRANS-PROCESSED
END-IF
PERFORM 8300-READ-TRANSACTION
END-PERFORM.
2200-NEW-ACCOUNT.
ADD 1 TO WS-NEW-ACCOUNTS.
2300-CLOSED-ACCOUNT.
ADD 1 TO WS-CLOSED-ACCOUNTS.
2400-WRITE-MISMATCH.
MOVE TS-ACCOUNT-NUMBER TO DL-ACCOUNT
MOVE YS-CURRENT-BALANCE TO DL-YEST-BALANCE
MOVE TS-CURRENT-BALANCE TO DL-TODAY-BALANCE
MOVE WS-CURRENT-ACCT-NET TO DL-TRAN-NET
COMPUTE WS-BALANCE-CHANGE =
WS-BALANCE-CHANGE - WS-CURRENT-ACCT-NET
MOVE WS-BALANCE-CHANGE TO DL-VARIANCE
MOVE 'MISMATCH' TO DL-STATUS
WRITE REPORT-LINE FROM WS-DETAIL-LINE
AFTER ADVANCING 1 LINE.
3000-PRINT-SUMMARY.
MOVE SPACES TO REPORT-LINE
WRITE REPORT-LINE AFTER ADVANCING 2 LINES
MOVE 'ACCOUNTS CHECKED:' TO SL-LABEL
MOVE WS-ACCOUNTS-CHECKED TO SL-COUNT
WRITE REPORT-LINE FROM WS-SUMMARY-LINE
AFTER ADVANCING 1 LINE
MOVE 'ACCOUNTS MATCHED:' TO SL-LABEL
MOVE WS-ACCOUNTS-MATCHED TO SL-COUNT
WRITE REPORT-LINE FROM WS-SUMMARY-LINE
AFTER ADVANCING 1 LINE
MOVE 'MISMATCHES FOUND:' TO SL-LABEL
MOVE WS-ACCOUNTS-MISMATCHED TO SL-COUNT
WRITE REPORT-LINE FROM WS-SUMMARY-LINE
AFTER ADVANCING 1 LINE
MOVE 'NEW ACCOUNTS:' TO SL-LABEL
MOVE WS-NEW-ACCOUNTS TO SL-COUNT
WRITE REPORT-LINE FROM WS-SUMMARY-LINE
AFTER ADVANCING 1 LINE
MOVE 'CLOSED ACCOUNTS:' TO SL-LABEL
MOVE WS-CLOSED-ACCOUNTS TO SL-COUNT
WRITE REPORT-LINE FROM WS-SUMMARY-LINE
AFTER ADVANCING 1 LINE.
8100-READ-TODAY.
READ TODAY-SNAPSHOT
AT END SET TODAY-AT-END TO TRUE
END-READ.
8200-READ-YESTERDAY.
READ YESTERDAY-SNAPSHOT
AT END SET YEST-AT-END TO TRUE
END-READ.
8300-READ-TRANSACTION.
READ TRANSACTION-FILE
AT END SET TRAN-AT-END TO TRUE
END-READ.
9000-TERMINATE.
CLOSE TODAY-SNAPSHOT
CLOSE YESTERDAY-SNAPSHOT
CLOSE TRANSACTION-FILE
CLOSE RECON-REPORT
IF WS-ACCOUNTS-MISMATCHED > ZERO
DISPLAY 'RECONCILIATION COMPLETED WITH '
WS-ACCOUNTS-MISMATCHED
' MISMATCHES - REVIEW REQUIRED'
MOVE 4 TO RETURN-CODE
ELSE
DISPLAY 'RECONCILIATION COMPLETED - ALL'
' ACCOUNTS BALANCED'
MOVE 0 TO RETURN-CODE
END-IF.
The program does not know or care which specific generation numbers it is processing. It simply opens TODAYSNP and YESTSNAP -- the JCL maps these to the appropriate GDG generations using (0) and (-1) relative references. If the bank needs to rerun reconciliation for a prior date, the JCL can be modified to reference specific absolute generations (e.g., G0087V00 and G0086V00) without changing the COBOL program.
GDG Housekeeping: Managing the Lifecycle
Even with GDG automatic uncatalog-on-overflow, active management is required to maintain system health.
Monitoring GDG Status
//CNBGDGMN JOB (ACCT),'CNB GDG MONITOR',
// CLASS=A,MSGCLASS=X,MSGLEVEL=(1,1),
// NOTIFY=&SYSUID
//*
//*================================================================*
//* GDG MONITORING AND HEALTH CHECK
//*================================================================*
//*
//MONITOR EXEC PGM=IDCAMS
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
LISTCAT GDG -
ENTRIES( -
CNB.CORE.TRAN.DAILY -
CNB.CORE.SNAP.ACCT -
CNB.CORE.BKUP.MASTER -
CNB.CORE.REPT.ARCHIVE ) -
ALL
LISTCAT ENTRIES(CNB.CORE.TRAN.DAILY.*) -
VOLUME
/*
The LISTCAT with the GDG keyword shows the GDG base definition, including the current generation count and limit. The second LISTCAT with the wildcard pattern lists all active generation datasets with their volume assignments, enabling capacity planning.
Recovering from GDG Problems
When a batch job abends after creating a (+1) generation, the incomplete generation may need to be deleted and the GDG base reset. The following recovery procedure handles this situation:
//CNBGDGRV JOB (ACCT),'CNB GDG RECOVERY',
// CLASS=A,MSGCLASS=X,MSGLEVEL=(1,1),
// NOTIFY=&SYSUID
//*
//*================================================================*
//* GDG RECOVERY: DELETE INCOMPLETE GENERATION AND VERIFY
//*================================================================*
//*
//RECOVER EXEC PGM=IDCAMS
//SYSPRINT DD SYSOUT=*
//SYSIN DD *
* DELETE THE MOST RECENT (INCOMPLETE) GENERATION
DELETE CNB.CORE.TRAN.DAILY(0) -
PURGE
* VERIFY THE GDG BASE IS CONSISTENT
LISTCAT GDG -
ENTRIES(CNB.CORE.TRAN.DAILY) -
ALL
/*
Lessons Learned
1. Relative References Eliminate Date Arithmetic
The most significant benefit of GDGs is the elimination of date-dependent JCL. The expression (+1) always means "the next generation," (0) always means "the current generation," and (-1) always means "one back." No date calculations, no day-of-week logic, no month-end boundary handling. This eliminated the most common category of JCL errors in the bank's batch schedule.
2. SCRATCH vs. NOSCRATCH Must Match the Retention Requirement
For data that must be retained beyond the GDG limit (audit archives, regulatory records), NOSCRATCH is mandatory. The team initially defined the report archive GDG with SCRATCH and lost 3 months of archived reports before the error was discovered. The recovery required restoring data from backup tapes -- a process that took 4 days and required an incident report to the compliance department.
3. The Generation Shift Within a Job Is a Source of Subtle Bugs
Within a single job, creating a (+1) generation causes all relative references to shift at end-of-step. If a later step references (0) expecting yesterday's file but the generation has already shifted to today's, the wrong data will be processed. The team maintains a reference chart for every job, documenting which generation each DD name resolves to at each step boundary.
4. GDG Limits Must Account for Operational Peaks
A 90-generation limit for daily transaction files seems adequate for 90-day retention. But the operations team occasionally needs to create extra generations for reruns, testing, or special processing. A job that creates generation 91 when 90 already exist triggers the GDG overflow behavior. The team pads the limit by 10% (using LIMIT(99) instead of LIMIT(90)) to provide operational headroom.
5. Model DSCBs Prevent DCB Mismatches
Without a model DSCB, every JCL DD statement that creates a new GDG generation must specify the complete DCB attributes. If one job specifies LRECL=250 and another specifies LRECL=252, the generations have incompatible record formats. The model DSCB provides default attributes that are used whenever the JCL omits DCB parameters, ensuring consistency across all generations.
Discussion Questions
-
The CNBRECON program uses a sequential merge approach to compare two generations of the account snapshot. What assumption about the data order is required for this approach to work, and how would a GDG-based system guarantee this assumption is met?
-
The report archive GDG has a 255-generation limit but needs 7-year retention. Design a complete archival strategy that combines GDG management with HSM migration to meet the full retention requirement.
-
If the POSTING step abends after writing 2 million of 3.5 million records to the (+1) transaction archive, what state is the GDG in? Describe the complete recovery procedure, including how to determine whether the incomplete generation was cataloged.
-
The team pads the GDG limit by 10% to allow for reruns and special processing. What alternative approaches could handle this requirement without over-sizing the GDG limit?
-
Consider a scenario where two independent batch jobs both need to create a (+1) generation of the same GDG. What happens if they run concurrently? How would you design the job schedule to prevent this conflict?
Connection to Chapter Concepts
This case study builds on several key concepts from Chapter 30:
-
Generation Data Groups (Section: GDG Concepts and Management): The GDG base definition, model DSCB creation, and relative generation references demonstrate the complete GDG lifecycle.
-
Dataset naming conventions (Section: Dataset Naming Rules and Standards): The CNB naming convention integrates GDG base names into the enterprise naming hierarchy.
-
IDCAMS for dataset management (Section: Catalog Management with IDCAMS): DEFINE GDG, LISTCAT, and DELETE commands for GDG administration are demonstrated in production-ready JCL.
-
Batch job design patterns (Section: Batch Processing Dataset Patterns): The end-of-day job stream illustrates how multiple steps within a job interact with GDG relative references, including the generation shift behavior at step boundaries.