Case Study 2: Financial Data Validation Framework
Background
Great Plains National Bank receives daily electronic feeds from twelve correspondent banks, three regulatory agencies, and two payment networks. Each feed contains financial transaction data -- wire transfers, ACH batches, check clearings, and loan payments -- that must be validated, normalized, and loaded into the bank's core processing system. The daily volume is approximately 1.2 million records across all feeds, with each feed arriving in a slightly different format but all ultimately conforming to a standard internal layout after transformation.
For years, the validation logic was embedded directly in each feed-processing program. When the bank's examiners audited the system, they discovered that validation rules were inconsistent across programs: one program rejected transactions with future dates while another accepted them; one program allowed negative amounts for reversals while another rejected them outright; one program checked for duplicate transaction IDs while three others did not. The inconsistency had allowed approximately $2.3 million in erroneous transactions to enter the core system over a two-year period.
The IT steering committee authorized the development of a centralized Financial Data Validation Framework. The framework would be implemented as a reusable COBOL program called FINVALID that all feed-processing programs would call. FINVALID would apply a standard set of validation rules to every transaction, classify each record as ACCEPTED, REJECTED, or FLAGGED (accepted with warnings), write rejected records to a standardized reject file, and maintain processing statistics that would be written to a control file for end-of-day reconciliation.
Architecture
The validation framework uses a pipeline architecture with three stages:
-
Stage 1 -- Format Validation: Ensures all fields contain syntactically valid data (numeric fields are numeric, date fields contain valid dates, required fields are not blank).
-
Stage 2 -- Business Rule Validation: Applies financial business rules (amount ranges, valid account types, effective date ranges, duplicate detection).
-
Stage 3 -- Cross-Field Validation: Checks relationships between fields (debit/credit indicator matches amount sign, settlement date is on or after transaction date, currency code matches amount precision).
Records that fail Stage 1 are immediately rejected. Records that fail Stage 2 or Stage 3 may be rejected or flagged depending on the severity of the rule violation.
Data Structures
Standard Transaction Layout
01 WS-TRANSACTION-RECORD.
05 WS-TXN-HEADER.
10 WS-TXN-ID PIC X(16).
10 WS-TXN-SOURCE PIC X(4).
10 WS-TXN-TYPE PIC X(3).
88 TXN-WIRE VALUE 'WIR'.
88 TXN-ACH VALUE 'ACH'.
88 TXN-CHECK VALUE 'CHK'.
88 TXN-LOAN-PMT VALUE 'LPM'.
88 TXN-FEE VALUE 'FEE'.
88 TXN-REVERSAL VALUE 'REV'.
88 TXN-VALID-TYPE VALUE 'WIR'
'ACH' 'CHK' 'LPM' 'FEE' 'REV'.
10 WS-TXN-DATE PIC X(8).
10 WS-TXN-TIME PIC X(6).
05 WS-TXN-ACCOUNTS.
10 WS-TXN-DEBIT-ACCT PIC X(12).
10 WS-TXN-CREDIT-ACCT PIC X(12).
10 WS-TXN-ACCT-TYPE PIC X(2).
88 ACCT-CHECKING VALUE 'CK'.
88 ACCT-SAVINGS VALUE 'SV'.
88 ACCT-MONEY-MKT VALUE 'MM'.
88 ACCT-LOAN VALUE 'LN'.
88 ACCT-GL VALUE 'GL'.
88 ACCT-VALID-TYPE VALUE 'CK'
'SV' 'MM' 'LN' 'GL'.
05 WS-TXN-FINANCIALS.
10 WS-TXN-AMOUNT PIC S9(11)V99.
10 WS-TXN-CURRENCY PIC X(3).
10 WS-TXN-DC-IND PIC X(1).
88 TXN-DEBIT VALUE 'D'.
88 TXN-CREDIT VALUE 'C'.
05 WS-TXN-SETTLEMENT.
10 WS-TXN-SETTLE-DATE PIC X(8).
10 WS-TXN-VALUE-DATE PIC X(8).
05 WS-TXN-REFERENCE.
10 WS-TXN-ORIGINATOR PIC X(30).
10 WS-TXN-BENEFICIARY PIC X(30).
10 WS-TXN-MEMO PIC X(50).
05 FILLER PIC X(13).
Validation Result Structure
01 WS-VALIDATION-RESULT.
05 WS-VAL-STATUS PIC X(1).
88 VAL-ACCEPTED VALUE 'A'.
88 VAL-REJECTED VALUE 'R'.
88 VAL-FLAGGED VALUE 'F'.
05 WS-VAL-ERROR-COUNT PIC 9(2) VALUE 0.
05 WS-VAL-WARNING-COUNT PIC 9(2) VALUE 0.
05 WS-VAL-ERRORS OCCURS 10 TIMES.
10 WS-VAL-ERR-CODE PIC X(4).
10 WS-VAL-ERR-FIELD PIC X(20).
10 WS-VAL-ERR-VALUE PIC X(20).
10 WS-VAL-ERR-DESC PIC X(50).
10 WS-VAL-ERR-SEV PIC X(1).
88 ERR-REJECT VALUE 'R'.
88 ERR-WARNING VALUE 'W'.
Reject File Layout
ENVIRONMENT DIVISION.
INPUT-OUTPUT SECTION.
FILE-CONTROL.
SELECT INPUT-TRANS-FILE ASSIGN TO TRANSIN
FILE STATUS IS WS-INPUT-STATUS.
SELECT VALID-TRANS-FILE ASSIGN TO TRANSOUT
FILE STATUS IS WS-OUTPUT-STATUS.
SELECT REJECT-FILE ASSIGN TO REJECTS
FILE STATUS IS WS-REJECT-STATUS.
SELECT CONTROL-FILE ASSIGN TO CTRLFILE
FILE STATUS IS WS-CTRL-STATUS.
SELECT DUP-CHECK-FILE ASSIGN TO DUPCHECK
ORGANIZATION IS INDEXED
ACCESS MODE IS RANDOM
RECORD KEY IS DC-TXN-ID
FILE STATUS IS WS-DUP-STATUS.
DATA DIVISION.
FILE SECTION.
FD INPUT-TRANS-FILE
RECORD CONTAINS 250 CHARACTERS.
01 INPUT-TRANS-RECORD PIC X(250).
FD VALID-TRANS-FILE
RECORD CONTAINS 250 CHARACTERS.
01 VALID-TRANS-RECORD PIC X(250).
FD REJECT-FILE
RECORD CONTAINS 400 CHARACTERS.
01 REJECT-RECORD.
05 RJ-ORIGINAL-RECORD PIC X(250).
05 RJ-REJECT-CODE PIC X(4).
05 RJ-REJECT-REASON PIC X(50).
05 RJ-SOURCE-FILE PIC X(4).
05 RJ-RECORD-NUMBER PIC 9(9).
05 RJ-TIMESTAMP PIC X(26).
05 RJ-ERROR-COUNT PIC 9(2).
05 FILLER PIC X(55).
FD CONTROL-FILE
RECORD CONTAINS 200 CHARACTERS.
01 CONTROL-RECORD.
05 CR-RUN-DATE PIC X(8).
05 CR-RUN-TIME PIC X(6).
05 CR-SOURCE-ID PIC X(4).
05 CR-RECORDS-READ PIC 9(9).
05 CR-RECORDS-ACCEPTED PIC 9(9).
05 CR-RECORDS-REJECTED PIC 9(9).
05 CR-RECORDS-FLAGGED PIC 9(9).
05 CR-TOTAL-DEBITS PIC S9(15)V99.
05 CR-TOTAL-CREDITS PIC S9(15)V99.
05 CR-FORMAT-ERRORS PIC 9(7).
05 CR-RULE-ERRORS PIC 9(7).
05 CR-CROSS-ERRORS PIC 9(7).
05 CR-DUPLICATE-COUNT PIC 9(7).
05 CR-RETURN-CODE PIC 9(2).
05 FILLER PIC X(82).
FD DUP-CHECK-FILE
RECORD CONTAINS 26 CHARACTERS.
01 DUP-CHECK-RECORD.
05 DC-TXN-ID PIC X(16).
05 DC-TXN-DATE PIC X(8).
05 DC-STATUS PIC X(2).
Processing Counters and Accumulators
WORKING-STORAGE SECTION.
01 WS-FILE-STATUSES.
05 WS-INPUT-STATUS PIC XX.
05 WS-OUTPUT-STATUS PIC XX.
05 WS-REJECT-STATUS PIC XX.
05 WS-CTRL-STATUS PIC XX.
05 WS-DUP-STATUS PIC XX.
01 WS-PROCESSING-FLAGS.
05 WS-EOF-INPUT PIC X VALUE 'N'.
88 FL-EOF-INPUT VALUE 'Y'.
05 WS-CRITICAL-ERROR PIC X VALUE 'N'.
88 FL-CRITICAL-ERROR VALUE 'Y'.
05 WS-VALID-FLAG PIC X VALUE 'Y'.
88 FL-RECORD-VALID VALUE 'Y'.
88 FL-RECORD-INVALID VALUE 'N'.
01 WS-STATISTICS.
05 CT-INPUT-READ PIC 9(9) VALUE 0.
05 CT-ACCEPTED PIC 9(9) VALUE 0.
05 CT-REJECTED PIC 9(9) VALUE 0.
05 CT-FLAGGED PIC 9(9) VALUE 0.
05 CT-FORMAT-ERRORS PIC 9(7) VALUE 0.
05 CT-RULE-ERRORS PIC 9(7) VALUE 0.
05 CT-CROSS-ERRORS PIC 9(7) VALUE 0.
05 CT-DUPLICATES PIC 9(7) VALUE 0.
05 AC-TOTAL-DEBITS PIC S9(15)V99 VALUE 0.
05 AC-TOTAL-CREDITS PIC S9(15)V99 VALUE 0.
01 WS-DATE-WORK-FIELDS.
05 WS-CURRENT-DATE-INT PIC 9(8).
05 WS-TXN-DATE-INT PIC 9(8).
05 WS-SETTLE-DATE-INT PIC 9(8).
05 WS-DATE-YEAR PIC 9(4).
05 WS-DATE-MONTH PIC 9(2).
05 WS-DATE-DAY PIC 9(2).
05 WS-MAX-FUTURE-DAYS PIC 9(3) VALUE 5.
05 WS-MAX-PAST-DAYS PIC 9(3) VALUE 90.
01 WS-AMOUNT-LIMITS.
05 WS-MAX-WIRE-AMOUNT PIC 9(11)V99
VALUE 99999999999.99.
05 WS-MAX-ACH-AMOUNT PIC 9(11)V99
VALUE 99999999.99.
05 WS-MAX-CHECK-AMOUNT PIC 9(11)V99
VALUE 9999999.99.
05 WS-MIN-AMOUNT PIC 9(3)V99
VALUE 0.01.
01 WS-RETURN-CODE PIC 99 VALUE 0.
01 WS-CURRENT-TIMESTAMP PIC X(26).
01 WS-SOURCE-ID PIC X(4).
DECLARATIVES and Main Processing
PROCEDURE DIVISION.
DECLARATIVES.
INPUT-ERR SECTION.
USE AFTER STANDARD ERROR PROCEDURE
ON INPUT-TRANS-FILE.
INPUT-ERR-PARA.
DISPLAY 'DECLARATIVE: INPUT FILE I/O ERROR'
DISPLAY ' STATUS: ' WS-INPUT-STATUS
DISPLAY ' RECORD: ' CT-INPUT-READ
IF WS-INPUT-STATUS = '30' OR '34' OR '35'
SET FL-CRITICAL-ERROR TO TRUE
MOVE 16 TO WS-RETURN-CODE
END-IF
.
OUTPUT-ERR SECTION.
USE AFTER STANDARD ERROR PROCEDURE
ON VALID-TRANS-FILE.
OUTPUT-ERR-PARA.
DISPLAY 'DECLARATIVE: OUTPUT FILE I/O ERROR'
DISPLAY ' STATUS: ' WS-OUTPUT-STATUS
IF WS-OUTPUT-STATUS = '34'
SET FL-CRITICAL-ERROR TO TRUE
DISPLAY ' OUTPUT DISK FULL - STOPPING'
MOVE 16 TO WS-RETURN-CODE
END-IF
.
REJECT-ERR SECTION.
USE AFTER STANDARD ERROR PROCEDURE
ON REJECT-FILE.
REJECT-ERR-PARA.
DISPLAY 'DECLARATIVE: REJECT FILE I/O ERROR'
DISPLAY ' STATUS: ' WS-REJECT-STATUS
IF WS-REJECT-STATUS = '34'
SET FL-CRITICAL-ERROR TO TRUE
DISPLAY ' REJECT FILE DISK FULL - STOPPING'
MOVE 16 TO WS-RETURN-CODE
END-IF
.
DUPCHK-ERR SECTION.
USE AFTER STANDARD ERROR PROCEDURE
ON DUP-CHECK-FILE.
DUPCHK-ERR-PARA.
DISPLAY 'DECLARATIVE: DUP-CHECK FILE ERROR'
DISPLAY ' STATUS: ' WS-DUP-STATUS
IF WS-DUP-STATUS = '30' OR '35' OR '93'
SET FL-CRITICAL-ERROR TO TRUE
MOVE 16 TO WS-RETURN-CODE
END-IF
.
END DECLARATIVES.
MAIN-PROCESSING SECTION.
0000-MAIN.
PERFORM 1000-INITIALIZE
IF NOT FL-CRITICAL-ERROR
PERFORM 2000-PROCESS-PIPELINE
END-IF
PERFORM 8000-WRITE-CONTROL-RECORD
PERFORM 9000-TERMINATE
MOVE WS-RETURN-CODE TO RETURN-CODE
STOP RUN
.
Initialization with Comprehensive File Status Checking
1000-INITIALIZE.
MOVE FUNCTION CURRENT-DATE TO WS-CURRENT-TIMESTAMP
MOVE WS-CURRENT-TIMESTAMP(1:8)
TO WS-CURRENT-DATE-INT
DISPLAY '============================================'
DISPLAY ' FINVALID - FINANCIAL DATA VALIDATION'
DISPLAY ' RUN DATE: ' WS-CURRENT-DATE-INT
DISPLAY ' RUN TIME: ' WS-CURRENT-TIMESTAMP(9:6)
DISPLAY '============================================'
OPEN INPUT INPUT-TRANS-FILE
IF WS-INPUT-STATUS NOT = '00'
DISPLAY 'FATAL: INPUT FILE OPEN FAILED'
DISPLAY 'STATUS: ' WS-INPUT-STATUS
SET FL-CRITICAL-ERROR TO TRUE
MOVE 16 TO WS-RETURN-CODE
END-IF
IF NOT FL-CRITICAL-ERROR
OPEN OUTPUT VALID-TRANS-FILE
IF WS-OUTPUT-STATUS NOT = '00'
DISPLAY 'FATAL: OUTPUT FILE OPEN FAILED'
DISPLAY 'STATUS: ' WS-OUTPUT-STATUS
SET FL-CRITICAL-ERROR TO TRUE
MOVE 16 TO WS-RETURN-CODE
END-IF
END-IF
IF NOT FL-CRITICAL-ERROR
OPEN OUTPUT REJECT-FILE
IF WS-REJECT-STATUS NOT = '00'
DISPLAY 'FATAL: REJECT FILE OPEN FAILED'
DISPLAY 'STATUS: ' WS-REJECT-STATUS
SET FL-CRITICAL-ERROR TO TRUE
MOVE 16 TO WS-RETURN-CODE
END-IF
END-IF
IF NOT FL-CRITICAL-ERROR
OPEN I-O DUP-CHECK-FILE
IF WS-DUP-STATUS NOT = '00'
DISPLAY 'WARNING: DUP CHECK FILE OPEN FAILED'
DISPLAY 'STATUS: ' WS-DUP-STATUS
DISPLAY 'DUPLICATE CHECKING DISABLED'
IF WS-RETURN-CODE < 4
MOVE 4 TO WS-RETURN-CODE
END-IF
END-IF
END-IF
.
The Validation Pipeline
2000-PROCESS-PIPELINE.
PERFORM 2100-READ-INPUT
PERFORM UNTIL FL-EOF-INPUT OR FL-CRITICAL-ERROR
INITIALIZE WS-VALIDATION-RESULT
SET VAL-ACCEPTED TO TRUE
MOVE 0 TO WS-VAL-ERROR-COUNT
MOVE 0 TO WS-VAL-WARNING-COUNT
*------- Stage 1: Format Validation
PERFORM 3000-VALIDATE-FORMAT
*------- Stage 2: Business Rules (only if format OK)
IF NOT VAL-REJECTED
PERFORM 4000-VALIDATE-BUSINESS-RULES
END-IF
*------- Stage 3: Cross-Field (only if still valid)
IF NOT VAL-REJECTED
PERFORM 5000-VALIDATE-CROSS-FIELD
END-IF
*------- Route the record
PERFORM 6000-ROUTE-RECORD
PERFORM 2100-READ-INPUT
END-PERFORM
.
2100-READ-INPUT.
READ INPUT-TRANS-FILE
INTO WS-TRANSACTION-RECORD
AT END
SET FL-EOF-INPUT TO TRUE
NOT AT END
ADD 1 TO CT-INPUT-READ
END-READ
.
Stage 1: Format Validation
3000-VALIDATE-FORMAT.
*--- Check transaction ID is not blank
IF WS-TXN-ID = SPACES
PERFORM 3900-ADD-REJECT-ERROR
MOVE 'F001' TO WS-VAL-ERR-CODE
(WS-VAL-ERROR-COUNT)
MOVE 'TXN-ID'
TO WS-VAL-ERR-FIELD
(WS-VAL-ERROR-COUNT)
MOVE SPACES TO WS-VAL-ERR-VALUE
(WS-VAL-ERROR-COUNT)
MOVE 'TRANSACTION ID IS BLANK'
TO WS-VAL-ERR-DESC
(WS-VAL-ERROR-COUNT)
ADD 1 TO CT-FORMAT-ERRORS
END-IF
*--- Check transaction type
IF NOT TXN-VALID-TYPE
PERFORM 3900-ADD-REJECT-ERROR
MOVE 'F002' TO WS-VAL-ERR-CODE
(WS-VAL-ERROR-COUNT)
MOVE 'TXN-TYPE'
TO WS-VAL-ERR-FIELD
(WS-VAL-ERROR-COUNT)
MOVE WS-TXN-TYPE TO WS-VAL-ERR-VALUE
(WS-VAL-ERROR-COUNT)
MOVE 'INVALID TRANSACTION TYPE'
TO WS-VAL-ERR-DESC
(WS-VAL-ERROR-COUNT)
ADD 1 TO CT-FORMAT-ERRORS
END-IF
*--- Validate transaction date is numeric
IF WS-TXN-DATE IS NOT NUMERIC
PERFORM 3900-ADD-REJECT-ERROR
MOVE 'F003' TO WS-VAL-ERR-CODE
(WS-VAL-ERROR-COUNT)
MOVE 'TXN-DATE'
TO WS-VAL-ERR-FIELD
(WS-VAL-ERROR-COUNT)
MOVE WS-TXN-DATE TO WS-VAL-ERR-VALUE
(WS-VAL-ERROR-COUNT)
MOVE 'DATE FIELD NOT NUMERIC'
TO WS-VAL-ERR-DESC
(WS-VAL-ERROR-COUNT)
ADD 1 TO CT-FORMAT-ERRORS
ELSE
*------- Validate date components
MOVE WS-TXN-DATE(1:4) TO WS-DATE-YEAR
MOVE WS-TXN-DATE(5:2) TO WS-DATE-MONTH
MOVE WS-TXN-DATE(7:2) TO WS-DATE-DAY
IF WS-DATE-MONTH < 01 OR > 12
OR WS-DATE-DAY < 01 OR > 31
PERFORM 3900-ADD-REJECT-ERROR
MOVE 'F004' TO WS-VAL-ERR-CODE
(WS-VAL-ERROR-COUNT)
MOVE 'TXN-DATE'
TO WS-VAL-ERR-FIELD
(WS-VAL-ERROR-COUNT)
MOVE WS-TXN-DATE TO WS-VAL-ERR-VALUE
(WS-VAL-ERROR-COUNT)
MOVE 'INVALID DATE VALUE'
TO WS-VAL-ERR-DESC
(WS-VAL-ERROR-COUNT)
ADD 1 TO CT-FORMAT-ERRORS
END-IF
END-IF
*--- Validate account type
IF NOT ACCT-VALID-TYPE
PERFORM 3900-ADD-REJECT-ERROR
MOVE 'F005' TO WS-VAL-ERR-CODE
(WS-VAL-ERROR-COUNT)
MOVE 'ACCT-TYPE'
TO WS-VAL-ERR-FIELD
(WS-VAL-ERROR-COUNT)
MOVE WS-TXN-ACCT-TYPE TO WS-VAL-ERR-VALUE
(WS-VAL-ERROR-COUNT)
MOVE 'INVALID ACCOUNT TYPE CODE'
TO WS-VAL-ERR-DESC
(WS-VAL-ERROR-COUNT)
ADD 1 TO CT-FORMAT-ERRORS
END-IF
*--- Validate debit/credit indicator
IF NOT TXN-DEBIT AND NOT TXN-CREDIT
PERFORM 3900-ADD-REJECT-ERROR
MOVE 'F006' TO WS-VAL-ERR-CODE
(WS-VAL-ERROR-COUNT)
MOVE 'DC-IND'
TO WS-VAL-ERR-FIELD
(WS-VAL-ERROR-COUNT)
MOVE WS-TXN-DC-IND TO WS-VAL-ERR-VALUE
(WS-VAL-ERROR-COUNT)
MOVE 'INVALID DEBIT/CREDIT INDICATOR'
TO WS-VAL-ERR-DESC
(WS-VAL-ERROR-COUNT)
ADD 1 TO CT-FORMAT-ERRORS
END-IF
*--- Validate currency code
IF WS-TXN-CURRENCY NOT = 'USD'
AND WS-TXN-CURRENCY NOT = 'CAD'
AND WS-TXN-CURRENCY NOT = 'EUR'
AND WS-TXN-CURRENCY NOT = 'GBP'
PERFORM 3900-ADD-REJECT-ERROR
MOVE 'F007' TO WS-VAL-ERR-CODE
(WS-VAL-ERROR-COUNT)
MOVE 'CURRENCY'
TO WS-VAL-ERR-FIELD
(WS-VAL-ERROR-COUNT)
MOVE WS-TXN-CURRENCY TO WS-VAL-ERR-VALUE
(WS-VAL-ERROR-COUNT)
MOVE 'UNSUPPORTED CURRENCY CODE'
TO WS-VAL-ERR-DESC
(WS-VAL-ERROR-COUNT)
ADD 1 TO CT-FORMAT-ERRORS
END-IF
.
3900-ADD-REJECT-ERROR.
IF WS-VAL-ERROR-COUNT < 10
ADD 1 TO WS-VAL-ERROR-COUNT
SET ERR-REJECT (WS-VAL-ERROR-COUNT) TO TRUE
END-IF
SET VAL-REJECTED TO TRUE
.
Stage 2: Business Rule Validation
4000-VALIDATE-BUSINESS-RULES.
*--- Check amount is positive (reversals handled separately)
IF WS-TXN-AMOUNT < WS-MIN-AMOUNT
AND NOT TXN-REVERSAL
PERFORM 4900-ADD-RULE-ERROR
MOVE 'R001' TO WS-VAL-ERR-CODE
(WS-VAL-ERROR-COUNT)
MOVE 'AMOUNT'
TO WS-VAL-ERR-FIELD
(WS-VAL-ERROR-COUNT)
MOVE SPACES TO WS-VAL-ERR-VALUE
(WS-VAL-ERROR-COUNT)
MOVE 'AMOUNT BELOW MINIMUM THRESHOLD'
TO WS-VAL-ERR-DESC
(WS-VAL-ERROR-COUNT)
ADD 1 TO CT-RULE-ERRORS
END-IF
*--- Check amount does not exceed type-specific limit
EVALUATE TRUE
WHEN TXN-WIRE
IF WS-TXN-AMOUNT > WS-MAX-WIRE-AMOUNT
PERFORM 4900-ADD-RULE-ERROR
MOVE 'R002' TO WS-VAL-ERR-CODE
(WS-VAL-ERROR-COUNT)
MOVE 'AMOUNT'
TO WS-VAL-ERR-FIELD
(WS-VAL-ERROR-COUNT)
MOVE SPACES TO WS-VAL-ERR-VALUE
(WS-VAL-ERROR-COUNT)
MOVE 'WIRE EXCEEDS MAXIMUM AMOUNT'
TO WS-VAL-ERR-DESC
(WS-VAL-ERROR-COUNT)
ADD 1 TO CT-RULE-ERRORS
END-IF
WHEN TXN-ACH
IF WS-TXN-AMOUNT > WS-MAX-ACH-AMOUNT
PERFORM 4900-ADD-RULE-ERROR
MOVE 'R003' TO WS-VAL-ERR-CODE
(WS-VAL-ERROR-COUNT)
MOVE 'AMOUNT'
TO WS-VAL-ERR-FIELD
(WS-VAL-ERROR-COUNT)
MOVE SPACES TO WS-VAL-ERR-VALUE
(WS-VAL-ERROR-COUNT)
MOVE 'ACH EXCEEDS MAXIMUM AMOUNT'
TO WS-VAL-ERR-DESC
(WS-VAL-ERROR-COUNT)
ADD 1 TO CT-RULE-ERRORS
END-IF
WHEN TXN-CHECK
IF WS-TXN-AMOUNT > WS-MAX-CHECK-AMOUNT
PERFORM 4900-ADD-RULE-ERROR
MOVE 'R004' TO WS-VAL-ERR-CODE
(WS-VAL-ERROR-COUNT)
MOVE 'AMOUNT'
TO WS-VAL-ERR-FIELD
(WS-VAL-ERROR-COUNT)
MOVE SPACES TO WS-VAL-ERR-VALUE
(WS-VAL-ERROR-COUNT)
MOVE 'CHECK EXCEEDS MAXIMUM AMOUNT'
TO WS-VAL-ERR-DESC
(WS-VAL-ERROR-COUNT)
ADD 1 TO CT-RULE-ERRORS
END-IF
END-EVALUATE
*--- Check for duplicate transaction ID
IF WS-DUP-STATUS = '00'
MOVE WS-TXN-ID TO DC-TXN-ID
READ DUP-CHECK-FILE
INVALID KEY
CONTINUE
NOT INVALID KEY
ADD 1 TO CT-DUPLICATES
IF WS-VAL-WARNING-COUNT < 10
ADD 1 TO WS-VAL-WARNING-COUNT
END-IF
SET VAL-FLAGGED TO TRUE
DISPLAY 'WARNING: DUPLICATE TXN '
WS-TXN-ID
END-READ
END-IF
.
4900-ADD-RULE-ERROR.
IF WS-VAL-ERROR-COUNT < 10
ADD 1 TO WS-VAL-ERROR-COUNT
SET ERR-REJECT (WS-VAL-ERROR-COUNT) TO TRUE
END-IF
SET VAL-REJECTED TO TRUE
.
Stage 3: Cross-Field Validation
5000-VALIDATE-CROSS-FIELD.
*--- Debit indicator should match positive amount
IF TXN-DEBIT AND WS-TXN-AMOUNT < 0
AND NOT TXN-REVERSAL
PERFORM 5900-ADD-CROSS-ERROR
MOVE 'X001' TO WS-VAL-ERR-CODE
(WS-VAL-ERROR-COUNT)
MOVE 'DC-IND/AMOUNT'
TO WS-VAL-ERR-FIELD
(WS-VAL-ERROR-COUNT)
MOVE SPACES TO WS-VAL-ERR-VALUE
(WS-VAL-ERROR-COUNT)
MOVE 'DEBIT IND WITH NEGATIVE AMOUNT'
TO WS-VAL-ERR-DESC
(WS-VAL-ERROR-COUNT)
ADD 1 TO CT-CROSS-ERRORS
END-IF
*--- Settlement date must be >= transaction date
IF WS-TXN-SETTLE-DATE IS NUMERIC
AND WS-TXN-DATE IS NUMERIC
MOVE WS-TXN-DATE TO WS-TXN-DATE-INT
MOVE WS-TXN-SETTLE-DATE TO WS-SETTLE-DATE-INT
IF WS-SETTLE-DATE-INT < WS-TXN-DATE-INT
PERFORM 5900-ADD-CROSS-ERROR
MOVE 'X002' TO WS-VAL-ERR-CODE
(WS-VAL-ERROR-COUNT)
MOVE 'SETTLE/TXN DATE'
TO WS-VAL-ERR-FIELD
(WS-VAL-ERROR-COUNT)
MOVE SPACES TO WS-VAL-ERR-VALUE
(WS-VAL-ERROR-COUNT)
MOVE 'SETTLE DATE BEFORE TXN DATE'
TO WS-VAL-ERR-DESC
(WS-VAL-ERROR-COUNT)
ADD 1 TO CT-CROSS-ERRORS
END-IF
END-IF
*--- Loan payment must be to a loan account
IF TXN-LOAN-PMT AND NOT ACCT-LOAN
PERFORM 5900-ADD-CROSS-ERROR
MOVE 'X003' TO WS-VAL-ERR-CODE
(WS-VAL-ERROR-COUNT)
MOVE 'TXN-TYPE/ACCT-TYPE'
TO WS-VAL-ERR-FIELD
(WS-VAL-ERROR-COUNT)
MOVE WS-TXN-ACCT-TYPE TO WS-VAL-ERR-VALUE
(WS-VAL-ERROR-COUNT)
MOVE 'LOAN PMT TO NON-LOAN ACCOUNT'
TO WS-VAL-ERR-DESC
(WS-VAL-ERROR-COUNT)
ADD 1 TO CT-CROSS-ERRORS
END-IF
*--- Debit account and credit account must differ
IF WS-TXN-DEBIT-ACCT = WS-TXN-CREDIT-ACCT
AND WS-TXN-DEBIT-ACCT NOT = SPACES
PERFORM 5900-ADD-CROSS-ERROR
MOVE 'X004' TO WS-VAL-ERR-CODE
(WS-VAL-ERROR-COUNT)
MOVE 'DEBIT/CREDIT ACCT'
TO WS-VAL-ERR-FIELD
(WS-VAL-ERROR-COUNT)
MOVE WS-TXN-DEBIT-ACCT TO WS-VAL-ERR-VALUE
(WS-VAL-ERROR-COUNT)
MOVE 'DEBIT AND CREDIT ACCTS MATCH'
TO WS-VAL-ERR-DESC
(WS-VAL-ERROR-COUNT)
ADD 1 TO CT-CROSS-ERRORS
END-IF
.
5900-ADD-CROSS-ERROR.
IF WS-VAL-ERROR-COUNT < 10
ADD 1 TO WS-VAL-ERROR-COUNT
SET ERR-REJECT (WS-VAL-ERROR-COUNT) TO TRUE
END-IF
SET VAL-REJECTED TO TRUE
.
Record Routing and Reject File Writing
6000-ROUTE-RECORD.
EVALUATE TRUE
WHEN VAL-ACCEPTED
PERFORM 6100-WRITE-ACCEPTED
WHEN VAL-FLAGGED
PERFORM 6100-WRITE-ACCEPTED
ADD 1 TO CT-FLAGGED
WHEN VAL-REJECTED
PERFORM 6200-WRITE-REJECTED
END-EVALUATE
.
6100-WRITE-ACCEPTED.
WRITE VALID-TRANS-RECORD
FROM WS-TRANSACTION-RECORD
IF WS-OUTPUT-STATUS = '00'
ADD 1 TO CT-ACCEPTED
*------- Accumulate debit/credit totals
IF TXN-DEBIT
ADD WS-TXN-AMOUNT TO AC-TOTAL-DEBITS
ON SIZE ERROR
DISPLAY 'SIZE ERROR: DEBIT TOTAL'
DISPLAY 'AMOUNT: ' WS-TXN-AMOUNT
IF WS-RETURN-CODE < 4
MOVE 4 TO WS-RETURN-CODE
END-IF
END-ADD
ELSE
ADD WS-TXN-AMOUNT TO AC-TOTAL-CREDITS
ON SIZE ERROR
DISPLAY 'SIZE ERROR: CREDIT TOTAL'
DISPLAY 'AMOUNT: ' WS-TXN-AMOUNT
IF WS-RETURN-CODE < 4
MOVE 4 TO WS-RETURN-CODE
END-IF
END-ADD
END-IF
*------- Register in duplicate check file
IF WS-DUP-STATUS = '00'
MOVE WS-TXN-ID TO DC-TXN-ID
MOVE WS-TXN-DATE TO DC-TXN-DATE
MOVE 'OK' TO DC-STATUS
WRITE DUP-CHECK-RECORD
INVALID KEY
CONTINUE
END-WRITE
END-IF
ELSE
DISPLAY 'WRITE ERROR ON OUTPUT FILE'
DISPLAY 'STATUS: ' WS-OUTPUT-STATUS
IF WS-RETURN-CODE < 8
MOVE 8 TO WS-RETURN-CODE
END-IF
END-IF
.
6200-WRITE-REJECTED.
MOVE WS-TRANSACTION-RECORD
TO RJ-ORIGINAL-RECORD
MOVE WS-VAL-ERR-CODE(1) TO RJ-REJECT-CODE
MOVE WS-VAL-ERR-DESC(1) TO RJ-REJECT-REASON
MOVE WS-TXN-SOURCE TO RJ-SOURCE-FILE
MOVE CT-INPUT-READ TO RJ-RECORD-NUMBER
MOVE WS-CURRENT-TIMESTAMP TO RJ-TIMESTAMP
MOVE WS-VAL-ERROR-COUNT TO RJ-ERROR-COUNT
WRITE REJECT-RECORD
IF WS-REJECT-STATUS = '00'
ADD 1 TO CT-REJECTED
ELSE
DISPLAY 'WRITE ERROR ON REJECT FILE'
DISPLAY 'STATUS: ' WS-REJECT-STATUS
DISPLAY 'TXN ID: ' WS-TXN-ID
IF WS-RETURN-CODE < 8
MOVE 8 TO WS-RETURN-CODE
END-IF
END-IF
.
Control Record and Termination
8000-WRITE-CONTROL-RECORD.
OPEN OUTPUT CONTROL-FILE
IF WS-CTRL-STATUS NOT = '00'
DISPLAY 'WARNING: CONTROL FILE OPEN FAILED'
DISPLAY 'STATUS: ' WS-CTRL-STATUS
DISPLAY 'CONTROL RECORD WRITTEN TO SYSOUT'
PERFORM 8100-DISPLAY-CONTROL-INFO
ELSE
MOVE WS-CURRENT-DATE-INT TO CR-RUN-DATE
MOVE WS-CURRENT-TIMESTAMP(9:6)
TO CR-RUN-TIME
MOVE WS-SOURCE-ID TO CR-SOURCE-ID
MOVE CT-INPUT-READ TO CR-RECORDS-READ
MOVE CT-ACCEPTED TO CR-RECORDS-ACCEPTED
MOVE CT-REJECTED TO CR-RECORDS-REJECTED
MOVE CT-FLAGGED TO CR-RECORDS-FLAGGED
MOVE AC-TOTAL-DEBITS TO CR-TOTAL-DEBITS
MOVE AC-TOTAL-CREDITS TO CR-TOTAL-CREDITS
MOVE CT-FORMAT-ERRORS TO CR-FORMAT-ERRORS
MOVE CT-RULE-ERRORS TO CR-RULE-ERRORS
MOVE CT-CROSS-ERRORS TO CR-CROSS-ERRORS
MOVE CT-DUPLICATES TO CR-DUPLICATE-COUNT
MOVE WS-RETURN-CODE TO CR-RETURN-CODE
WRITE CONTROL-RECORD
IF WS-CTRL-STATUS NOT = '00'
DISPLAY 'CONTROL RECORD WRITE FAILED'
PERFORM 8100-DISPLAY-CONTROL-INFO
END-IF
CLOSE CONTROL-FILE
END-IF
.
8100-DISPLAY-CONTROL-INFO.
DISPLAY 'CONTROL: READ=' CT-INPUT-READ
' ACCEPTED=' CT-ACCEPTED
' REJECTED=' CT-REJECTED
' FLAGGED=' CT-FLAGGED
DISPLAY 'CONTROL: DEBITS=' AC-TOTAL-DEBITS
' CREDITS=' AC-TOTAL-CREDITS
DISPLAY 'CONTROL: FMT-ERR=' CT-FORMAT-ERRORS
' RULE-ERR=' CT-RULE-ERRORS
' CROSS-ERR=' CT-CROSS-ERRORS
' DUPS=' CT-DUPLICATES
.
9000-TERMINATE.
DISPLAY ' '
DISPLAY '============================================'
DISPLAY ' FINVALID PROCESSING SUMMARY'
DISPLAY '============================================'
DISPLAY ' RECORDS READ: ' CT-INPUT-READ
DISPLAY ' RECORDS ACCEPTED: ' CT-ACCEPTED
DISPLAY ' RECORDS FLAGGED: ' CT-FLAGGED
DISPLAY ' RECORDS REJECTED: ' CT-REJECTED
DISPLAY ' -----------------------------------------'
DISPLAY ' FORMAT ERRORS: ' CT-FORMAT-ERRORS
DISPLAY ' RULE ERRORS: ' CT-RULE-ERRORS
DISPLAY ' CROSS-FIELD ERRORS:' CT-CROSS-ERRORS
DISPLAY ' DUPLICATES FOUND: ' CT-DUPLICATES
DISPLAY ' -----------------------------------------'
DISPLAY ' TOTAL DEBITS: ' AC-TOTAL-DEBITS
DISPLAY ' TOTAL CREDITS: ' AC-TOTAL-CREDITS
DISPLAY ' -----------------------------------------'
DISPLAY ' RETURN CODE: ' WS-RETURN-CODE
DISPLAY '============================================'
CLOSE INPUT-TRANS-FILE
CLOSE VALID-TRANS-FILE
CLOSE REJECT-FILE
IF WS-DUP-STATUS = '00'
CLOSE DUP-CHECK-FILE
END-IF
.
JCL for Execution
//FINVALID JOB (ACCT),'DATA VALIDATION',CLASS=A,
// MSGCLASS=X,MSGLEVEL=(1,1)
//VALIDATE EXEC PGM=FINVALID
//STEPLIB DD DSN=GPNB.PROD.LOADLIB,DISP=SHR
//TRANSIN DD DSN=GPNB.DAILY.WIRE.TRANS,DISP=SHR
//TRANSOUT DD DSN=GPNB.DAILY.WIRE.VALID,
// DISP=(NEW,CATLG,DELETE),
// SPACE=(CYL,(20,10),RLSE),
// DCB=(RECFM=FB,LRECL=250,BLKSIZE=0)
//REJECTS DD DSN=GPNB.DAILY.WIRE.REJECTS,
// DISP=(NEW,CATLG,DELETE),
// SPACE=(CYL,(5,2),RLSE),
// DCB=(RECFM=FB,LRECL=400,BLKSIZE=0)
//CTRLFILE DD DSN=GPNB.DAILY.WIRE.CONTROL,
// DISP=(NEW,CATLG,DELETE),
// SPACE=(TRK,(1,1),RLSE),
// DCB=(RECFM=FB,LRECL=200,BLKSIZE=0)
//DUPCHECK DD DSN=GPNB.TXN.DUPLICATE.KSDS,DISP=SHR
//SYSOUT DD SYSOUT=*
Lessons and Design Principles
This case study demonstrates a validation framework built on several important patterns:
Pipeline architecture with early exit. The three validation stages execute in order, and a record rejected in Stage 1 (format) skips Stages 2 and 3. This saves processing time and avoids cascading errors: if the amount field is not numeric, there is no point checking whether it exceeds the maximum wire amount.
Error accumulation. Rather than stopping at the first error in a record, the framework captures up to ten errors per record. This gives operations staff complete diagnostic information for each rejected record, reducing the number of correction-and-resubmission cycles.
Reject file as audit trail. Every rejected record is written to a structured reject file that includes the original record, the primary reject code, the reject reason, the source file identifier, and a timestamp. This file can be loaded into a database for analysis or fed into a correction workflow.
Control file for reconciliation. The control record provides the downstream system with everything it needs to verify that the validation run completed correctly: record counts, totals, error counts, and a return code. If the accepted record count plus the rejected record count does not equal the input record count, something went wrong during validation.
Graceful handling of support file failures. The duplicate-check file is important but not critical. If it cannot be opened (perhaps it is being reorganized), the program continues without duplicate checking and sets a warning return code. The program degrades gracefully rather than failing entirely.
Declaratives as a safety net. The declarative handlers catch any I/O error that slips past the inline file status checking. They serve as a last line of defense, ensuring that unexpected I/O errors are logged and that the program sets an appropriate return code rather than continuing as if nothing happened.