Case Study 2: SOX Compliance Audit Trail System
Background
Pinnacle Financial Group (PFG) is a publicly traded bank holding company with $42 billion in consolidated assets. As a public company, PFG is subject to the Sarbanes-Oxley Act of 2002 (SOX), which mandates rigorous internal controls over financial reporting. Section 404 of SOX requires management to assess the effectiveness of internal controls, and the external auditors must attest to that assessment.
During the most recent annual audit, PFG's external auditors issued a material weakness finding related to the mainframe core banking system. The finding stated: "The company lacks a comprehensive, tamper-evident audit trail for changes to financial master data. While individual application logs exist, they are fragmented across multiple files, use inconsistent formats, and do not capture before-and-after images of modified records. The absence of a unified audit trail prevents management and auditors from independently verifying the completeness and accuracy of master data changes."
The remediation required PFG to design and implement a centralized audit logging system that captures every modification to financial master data, records both the before-image and after-image of each changed record, is tamper-evident (cannot be modified after the fact), and provides queryable access for auditors.
This case study presents the complete design and implementation of PFG's SOX-compliant audit trail system, including the COBOL audit logging module, the tamper-evident file design, and the audit query facility.
Audit Trail Design Requirements
The SOX audit trail must satisfy five requirements defined by PFG's compliance team in consultation with the external auditors:
-
Completeness: Every change to a financial master record must produce an audit entry. No path through the application can modify data without triggering the audit module.
-
Before/After Images: Each audit entry must contain the complete record as it existed before the change (before-image) and the complete record after the change (after-image). This allows auditors to reconstruct the exact state of any record at any point in time.
-
Attribution: Each audit entry must identify who made the change (user ID), when it was made (timestamp with microsecond precision), what program made it, from which terminal or batch job, and why (transaction type and reference).
-
Tamper-Evidence: Once written, audit records must not be modifiable. The system must detect if any audit records have been altered, deleted, or inserted out of sequence.
-
Queryability: Auditors must be able to retrieve the complete change history for any account, any user, or any time period without requiring programmer assistance.
Audit Record Design
The audit trail record is designed as a fixed-length 2000-byte record that contains the audit metadata, the before-image, and the after-image in a single self-contained entry:
*================================================================*
* COPYBOOK: AUDREC01
* PURPOSE: SOX AUDIT TRAIL RECORD LAYOUT
* USED BY: ALL PROGRAMS THAT MODIFY FINANCIAL MASTER DATA
*================================================================*
01 AUDIT-TRAIL-RECORD.
* --- AUDIT HEADER (200 BYTES) ---
05 AUD-HEADER.
10 AUD-RECORD-TYPE PIC X(4).
88 AUD-IS-CHANGE VALUE 'CHNG'.
88 AUD-IS-INSERT VALUE 'INSR'.
88 AUD-IS-DELETE VALUE 'DELE'.
88 AUD-IS-CONTROL VALUE 'CTRL'.
10 AUD-SEQUENCE-NUM PIC 9(12).
10 AUD-TIMESTAMP PIC X(26).
10 AUD-USER-ID PIC X(8).
10 AUD-PROGRAM-NAME PIC X(8).
10 AUD-TRANSACTION-ID PIC X(4).
10 AUD-TERMINAL-ID PIC X(8).
10 AUD-JOB-NAME PIC X(8).
10 AUD-STEP-NAME PIC X(8).
10 AUD-MASTER-FILE-ID PIC X(8).
10 AUD-RECORD-KEY PIC X(30).
10 AUD-CHANGE-REASON PIC X(40).
10 AUD-HASH-VALUE PIC X(32).
10 AUD-PREV-HASH PIC X(32).
10 FILLER PIC X(4).
* --- BEFORE IMAGE (900 BYTES) ---
05 AUD-BEFORE-IMAGE.
10 AUD-BEFORE-LENGTH PIC 9(4).
10 AUD-BEFORE-DATA PIC X(896).
* --- AFTER IMAGE (900 BYTES) ---
05 AUD-AFTER-IMAGE.
10 AUD-AFTER-LENGTH PIC 9(4).
10 AUD-AFTER-DATA PIC X(896).
Tamper-Evidence Through Hash Chaining
The AUD-HASH-VALUE and AUD-PREV-HASH fields implement a hash chain -- a simplified blockchain concept applied to the audit trail. Each audit record contains two hash values:
- AUD-PREV-HASH: The hash value from the immediately preceding audit record
- AUD-HASH-VALUE: A hash computed over the current record's contents (excluding AUD-HASH-VALUE itself) concatenated with AUD-PREV-HASH
This creates a chain where each record's integrity depends on all preceding records. If any record in the chain is modified, deleted, or a spurious record is inserted, the hash chain breaks at that point and every subsequent record's hash will fail verification.
The Audit Logging Module
The core of the system is a reusable COBOL module (PFGAUD01) that any application program can CALL to write an audit record. This centralized approach ensures consistent audit record creation regardless of which program initiates the data change.
IDENTIFICATION DIVISION.
PROGRAM-ID. PFGAUD01.
*================================================================*
* PROGRAM: PFGAUD01 - SOX AUDIT TRAIL LOGGING MODULE
* PURPOSE: CENTRALIZED AUDIT RECORD CREATION FOR ALL
* FINANCIAL MASTER DATA CHANGES.
* CALLED BY: ANY PROGRAM THAT MODIFIES MASTER DATA
* INTERFACE: CALLED WITH AUDIT-INTERFACE-AREA
*================================================================*
ENVIRONMENT DIVISION.
INPUT-OUTPUT SECTION.
FILE-CONTROL.
SELECT AUDIT-FILE
ASSIGN TO AUDITOUT
ORGANIZATION IS SEQUENTIAL
ACCESS MODE IS SEQUENTIAL
FILE STATUS IS WS-AUDIT-FILE-STATUS.
DATA DIVISION.
FILE SECTION.
FD AUDIT-FILE
RECORDING MODE IS F
RECORD CONTAINS 2000 CHARACTERS.
01 AUDIT-FILE-RECORD PIC X(2000).
WORKING-STORAGE SECTION.
01 WS-AUDIT-FILE-STATUS PIC X(2).
01 WS-FILE-OPEN-FLAG PIC X(1) VALUE 'N'.
88 AUDIT-FILE-IS-OPEN VALUE 'Y'.
88 AUDIT-FILE-IS-CLOSED VALUE 'N'.
01 WS-SEQUENCE-COUNTER PIC 9(12) VALUE ZERO.
01 WS-PREVIOUS-HASH PIC X(32)
VALUE ALL '0'.
01 WS-HASH-WORK-AREA.
05 WS-HASH-INPUT PIC X(2000).
05 WS-HASH-INPUT-LEN PIC 9(8).
05 WS-HASH-OUTPUT PIC X(32).
01 WS-CURRENT-TIMESTAMP PIC X(26).
01 WS-ABSTIME PIC S9(15) COMP-3.
01 WS-RETURN-CODE PIC 9(4) VALUE ZERO.
COPY AUDREC01.
LINKAGE SECTION.
01 LS-AUDIT-INTERFACE.
05 LS-FUNCTION-CODE PIC X(4).
88 LS-FUNC-OPEN VALUE 'OPEN'.
88 LS-FUNC-WRITE VALUE 'WRIT'.
88 LS-FUNC-CLOSE VALUE 'CLOS'.
05 LS-RETURN-CODE PIC 9(4).
05 LS-RECORD-TYPE PIC X(4).
05 LS-USER-ID PIC X(8).
05 LS-PROGRAM-NAME PIC X(8).
05 LS-TRANSACTION-ID PIC X(4).
05 LS-TERMINAL-ID PIC X(8).
05 LS-JOB-NAME PIC X(8).
05 LS-STEP-NAME PIC X(8).
05 LS-MASTER-FILE-ID PIC X(8).
05 LS-RECORD-KEY PIC X(30).
05 LS-CHANGE-REASON PIC X(40).
05 LS-BEFORE-LENGTH PIC 9(4).
05 LS-BEFORE-DATA PIC X(896).
05 LS-AFTER-LENGTH PIC 9(4).
05 LS-AFTER-DATA PIC X(896).
PROCEDURE DIVISION USING LS-AUDIT-INTERFACE.
0000-MAIN.
MOVE ZERO TO LS-RETURN-CODE
EVALUATE TRUE
WHEN LS-FUNC-OPEN
PERFORM 1000-OPEN-AUDIT-FILE
WHEN LS-FUNC-WRITE
PERFORM 2000-WRITE-AUDIT-RECORD
WHEN LS-FUNC-CLOSE
PERFORM 3000-CLOSE-AUDIT-FILE
WHEN OTHER
MOVE 9999 TO LS-RETURN-CODE
END-EVALUATE
GOBACK.
1000-OPEN-AUDIT-FILE.
IF AUDIT-FILE-IS-OPEN
MOVE 0 TO LS-RETURN-CODE
ELSE
OPEN EXTEND AUDIT-FILE
IF WS-AUDIT-FILE-STATUS = '00'
SET AUDIT-FILE-IS-OPEN TO TRUE
PERFORM 1100-WRITE-CONTROL-RECORD
ELSE
MOVE 1001 TO LS-RETURN-CODE
END-IF
END-IF.
1100-WRITE-CONTROL-RECORD.
*--------------------------------------------------------------*
* WRITE A CONTROL RECORD AT THE START OF EACH SESSION
* THIS MARKS THE BEGINNING OF A NEW BATCH OR CICS SESSION
*--------------------------------------------------------------*
INITIALIZE AUDIT-TRAIL-RECORD
SET AUD-IS-CONTROL TO TRUE
PERFORM 2100-GET-TIMESTAMP
MOVE WS-CURRENT-TIMESTAMP TO AUD-TIMESTAMP
MOVE LS-PROGRAM-NAME TO AUD-PROGRAM-NAME
MOVE LS-JOB-NAME TO AUD-JOB-NAME
MOVE 'SESSION OPEN' TO AUD-CHANGE-REASON
ADD 1 TO WS-SEQUENCE-COUNTER
MOVE WS-SEQUENCE-COUNTER TO AUD-SEQUENCE-NUM
MOVE WS-PREVIOUS-HASH TO AUD-PREV-HASH
PERFORM 2200-COMPUTE-HASH
MOVE WS-HASH-OUTPUT TO AUD-HASH-VALUE
MOVE WS-HASH-OUTPUT TO WS-PREVIOUS-HASH
WRITE AUDIT-FILE-RECORD FROM AUDIT-TRAIL-RECORD.
2000-WRITE-AUDIT-RECORD.
IF AUDIT-FILE-IS-CLOSED
MOVE 2001 TO LS-RETURN-CODE
ELSE
PERFORM 2000-BUILD-AUDIT-RECORD
WRITE AUDIT-FILE-RECORD
FROM AUDIT-TRAIL-RECORD
IF WS-AUDIT-FILE-STATUS = '00'
MOVE 0 TO LS-RETURN-CODE
ELSE
MOVE 2002 TO LS-RETURN-CODE
END-IF
END-IF.
2000-BUILD-AUDIT-RECORD.
INITIALIZE AUDIT-TRAIL-RECORD
MOVE LS-RECORD-TYPE TO AUD-RECORD-TYPE
MOVE LS-USER-ID TO AUD-USER-ID
MOVE LS-PROGRAM-NAME TO AUD-PROGRAM-NAME
MOVE LS-TRANSACTION-ID TO AUD-TRANSACTION-ID
MOVE LS-TERMINAL-ID TO AUD-TERMINAL-ID
MOVE LS-JOB-NAME TO AUD-JOB-NAME
MOVE LS-STEP-NAME TO AUD-STEP-NAME
MOVE LS-MASTER-FILE-ID TO AUD-MASTER-FILE-ID
MOVE LS-RECORD-KEY TO AUD-RECORD-KEY
MOVE LS-CHANGE-REASON TO AUD-CHANGE-REASON
PERFORM 2100-GET-TIMESTAMP
MOVE WS-CURRENT-TIMESTAMP TO AUD-TIMESTAMP
MOVE LS-BEFORE-LENGTH TO AUD-BEFORE-LENGTH
MOVE LS-BEFORE-DATA TO AUD-BEFORE-DATA
MOVE LS-AFTER-LENGTH TO AUD-AFTER-LENGTH
MOVE LS-AFTER-DATA TO AUD-AFTER-DATA
ADD 1 TO WS-SEQUENCE-COUNTER
MOVE WS-SEQUENCE-COUNTER TO AUD-SEQUENCE-NUM
MOVE WS-PREVIOUS-HASH TO AUD-PREV-HASH
PERFORM 2200-COMPUTE-HASH
MOVE WS-HASH-OUTPUT TO AUD-HASH-VALUE
MOVE WS-HASH-OUTPUT TO WS-PREVIOUS-HASH.
2100-GET-TIMESTAMP.
*--------------------------------------------------------------*
* GET CURRENT TIMESTAMP WITH MICROSECOND PRECISION
* FORMAT: YYYY-MM-DD-HH.MM.SS.MMMMMM
*--------------------------------------------------------------*
ACCEPT WS-CURRENT-TIMESTAMP
FROM DATE YYYYMMDD
ACCEPT WS-CURRENT-TIMESTAMP(12:15)
FROM TIME.
* IN A CICS ENVIRONMENT, USE:
* EXEC CICS ASKTIME ABSTIME(WS-ABSTIME) END-EXEC
* EXEC CICS FORMATTIME ABSTIME(WS-ABSTIME)
* YYYYMMDD(WS-CURRENT-TIMESTAMP)
* TIME(WS-CURRENT-TIMESTAMP(12:8))
* TIMESEP('.') END-EXEC
2200-COMPUTE-HASH.
*--------------------------------------------------------------*
* COMPUTE A HASH VALUE OVER THE AUDIT RECORD CONTENTS
* USES A SIMPLIFIED HASH ALGORITHM FOR DEMONSTRATION
* PRODUCTION WOULD USE ICSF (INTEGRATED CRYPTOGRAPHIC
* SERVICE FACILITY) FOR SHA-256 OR EQUIVALENT
*--------------------------------------------------------------*
* PREPARE INPUT: RECORD CONTENT EXCLUDING HASH FIELD
* CONCATENATED WITH PREVIOUS HASH
MOVE AUDIT-TRAIL-RECORD TO WS-HASH-INPUT
* ZERO OUT THE HASH FIELD POSITION IN THE WORK AREA
* SO IT DOES NOT PARTICIPATE IN ITS OWN HASH
MOVE SPACES TO WS-HASH-INPUT(165:32)
MOVE 2000 TO WS-HASH-INPUT-LEN
* CALL ICSF FOR CRYPTOGRAPHIC HASH
* IN PRODUCTION:
* CALL 'CSNBOWH' USING
* WS-ICSF-RC
* WS-ICSF-REASON
* WS-ICSF-EXIT-LEN
* WS-ICSF-EXIT-DATA
* WS-ICSF-RULE-COUNT
* WS-ICSF-RULE-ARRAY ('SHA-256')
* WS-HASH-INPUT-LEN
* WS-HASH-INPUT
* WS-ICSF-CHAIN-LEN
* WS-ICSF-CHAIN-DATA
* WS-HASH-OUTPUT
* SIMPLIFIED HASH FOR DEMONSTRATION
* (NOT CRYPTOGRAPHICALLY SECURE)
PERFORM 2210-SIMPLE-HASH.
2210-SIMPLE-HASH.
*--------------------------------------------------------------*
* DEMONSTRATION HASH: COMPUTES A 32-CHARACTER HEX STRING
* FROM THE INPUT DATA. IN PRODUCTION, REPLACE WITH ICSF
* SHA-256 CALL SHOWN IN COMMENTS ABOVE.
*--------------------------------------------------------------*
MOVE ZERO TO WS-HASH-ACCUM-1
MOVE ZERO TO WS-HASH-ACCUM-2
MOVE ZERO TO WS-HASH-ACCUM-3
MOVE ZERO TO WS-HASH-ACCUM-4
PERFORM VARYING WS-HASH-IDX FROM 1 BY 1
UNTIL WS-HASH-IDX > WS-HASH-INPUT-LEN
COMPUTE WS-HASH-BYTE =
FUNCTION ORD(WS-HASH-INPUT(
WS-HASH-IDX:1))
COMPUTE WS-HASH-ACCUM-1 =
FUNCTION MOD(
WS-HASH-ACCUM-1 * 31 +
WS-HASH-BYTE, 99999999)
COMPUTE WS-HASH-ACCUM-2 =
FUNCTION MOD(
WS-HASH-ACCUM-2 * 37 +
WS-HASH-BYTE, 99999999)
COMPUTE WS-HASH-ACCUM-3 =
FUNCTION MOD(
WS-HASH-ACCUM-3 * 41 +
WS-HASH-BYTE, 99999999)
COMPUTE WS-HASH-ACCUM-4 =
FUNCTION MOD(
WS-HASH-ACCUM-4 * 43 +
WS-HASH-BYTE, 99999999)
END-PERFORM
STRING WS-HASH-ACCUM-1
WS-HASH-ACCUM-2
WS-HASH-ACCUM-3
WS-HASH-ACCUM-4
DELIMITED BY SIZE
INTO WS-HASH-OUTPUT.
3000-CLOSE-AUDIT-FILE.
IF AUDIT-FILE-IS-OPEN
PERFORM 3100-WRITE-CLOSE-CONTROL
CLOSE AUDIT-FILE
SET AUDIT-FILE-IS-CLOSED TO TRUE
END-IF.
3100-WRITE-CLOSE-CONTROL.
INITIALIZE AUDIT-TRAIL-RECORD
SET AUD-IS-CONTROL TO TRUE
PERFORM 2100-GET-TIMESTAMP
MOVE WS-CURRENT-TIMESTAMP TO AUD-TIMESTAMP
MOVE LS-PROGRAM-NAME TO AUD-PROGRAM-NAME
MOVE LS-JOB-NAME TO AUD-JOB-NAME
MOVE 'SESSION CLOSE' TO AUD-CHANGE-REASON
STRING 'RECORDS WRITTEN: '
WS-SEQUENCE-COUNTER
DELIMITED BY SIZE
INTO AUD-BEFORE-DATA
ADD 1 TO WS-SEQUENCE-COUNTER
MOVE WS-SEQUENCE-COUNTER TO AUD-SEQUENCE-NUM
MOVE WS-PREVIOUS-HASH TO AUD-PREV-HASH
PERFORM 2200-COMPUTE-HASH
MOVE WS-HASH-OUTPUT TO AUD-HASH-VALUE
WRITE AUDIT-FILE-RECORD FROM AUDIT-TRAIL-RECORD.
Calling the Audit Module from Application Programs
Every program that modifies financial master data calls PFGAUD01. Here is how the account posting program integrates with the audit module:
IDENTIFICATION DIVISION.
PROGRAM-ID. PFGPOST1.
*================================================================*
* PROGRAM: PFGPOST1 - ACCOUNT POSTING WITH SOX AUDIT
* PURPOSE: POST TRANSACTIONS TO ACCOUNT MASTER WITH
* MANDATORY AUDIT TRAIL LOGGING
*================================================================*
DATA DIVISION.
WORKING-STORAGE SECTION.
01 WS-AUDIT-INTERFACE.
05 WS-AI-FUNCTION PIC X(4).
05 WS-AI-RETURN-CODE PIC 9(4).
05 WS-AI-RECORD-TYPE PIC X(4).
05 WS-AI-USER-ID PIC X(8).
05 WS-AI-PROGRAM-NAME PIC X(8).
05 WS-AI-TRANSACTION-ID PIC X(4).
05 WS-AI-TERMINAL-ID PIC X(8).
05 WS-AI-JOB-NAME PIC X(8).
05 WS-AI-STEP-NAME PIC X(8).
05 WS-AI-MASTER-FILE-ID PIC X(8).
05 WS-AI-RECORD-KEY PIC X(30).
05 WS-AI-CHANGE-REASON PIC X(40).
05 WS-AI-BEFORE-LENGTH PIC 9(4).
05 WS-AI-BEFORE-DATA PIC X(896).
05 WS-AI-AFTER-LENGTH PIC 9(4).
05 WS-AI-AFTER-DATA PIC X(896).
01 WS-ACCOUNT-BEFORE PIC X(500).
01 WS-ACCOUNT-AFTER PIC X(500).
01 WS-ACCOUNT-RECORD.
05 WS-ACCT-NUMBER PIC X(10).
05 WS-ACCT-NAME PIC X(30).
05 WS-ACCT-BALANCE PIC S9(13)V99 COMP-3.
05 WS-ACCT-AVAIL-BAL PIC S9(13)V99 COMP-3.
05 WS-ACCT-LAST-ACTIVITY PIC 9(8).
05 FILLER PIC X(434).
PROCEDURE DIVISION.
0000-MAIN.
PERFORM 0100-OPEN-AUDIT
PERFORM 1000-PROCESS-TRANSACTIONS
PERFORM 0900-CLOSE-AUDIT
STOP RUN.
0100-OPEN-AUDIT.
INITIALIZE WS-AUDIT-INTERFACE
MOVE 'OPEN' TO WS-AI-FUNCTION
MOVE 'PFGPOST1' TO WS-AI-PROGRAM-NAME
MOVE 'PFGEOD01' TO WS-AI-JOB-NAME
MOVE 'POSTING' TO WS-AI-STEP-NAME
CALL 'PFGAUD01' USING WS-AUDIT-INTERFACE
IF WS-AI-RETURN-CODE NOT = ZERO
DISPLAY 'FATAL: CANNOT OPEN AUDIT FILE RC='
WS-AI-RETURN-CODE
MOVE 16 TO RETURN-CODE
STOP RUN
END-IF.
1000-PROCESS-TRANSACTIONS.
* (READ TRANSACTION FILE, PROCESS EACH RECORD)
PERFORM UNTIL END-OF-TRANSACTIONS
PERFORM 2000-POST-ONE-TRANSACTION
READ TRANSACTION-FILE
AT END SET END-OF-TRANSACTIONS TO TRUE
END-READ
END-PERFORM.
2000-POST-ONE-TRANSACTION.
* READ THE ACCOUNT RECORD
READ ACCOUNT-MASTER INTO WS-ACCOUNT-RECORD
KEY IS WS-TRANS-ACCOUNT
INVALID KEY
PERFORM 8000-HANDLE-MISSING-ACCOUNT
END-READ
* CAPTURE BEFORE-IMAGE
MOVE WS-ACCOUNT-RECORD TO WS-ACCOUNT-BEFORE
* APPLY THE TRANSACTION
EVALUATE WS-TRANS-TYPE
WHEN 'DP'
ADD WS-TRANS-AMOUNT TO WS-ACCT-BALANCE
WHEN 'WD'
SUBTRACT WS-TRANS-AMOUNT
FROM WS-ACCT-BALANCE
WHEN 'FE'
SUBTRACT WS-TRANS-AMOUNT
FROM WS-ACCT-BALANCE
WHEN 'IN'
ADD WS-TRANS-AMOUNT TO WS-ACCT-BALANCE
END-EVALUATE
* UPDATE LAST ACTIVITY DATE
MOVE WS-CURRENT-DATE TO WS-ACCT-LAST-ACTIVITY
* CAPTURE AFTER-IMAGE
MOVE WS-ACCOUNT-RECORD TO WS-ACCOUNT-AFTER
* WRITE THE UPDATED RECORD
REWRITE ACCOUNT-MASTER-REC FROM WS-ACCOUNT-RECORD
INVALID KEY
PERFORM 8100-HANDLE-REWRITE-ERROR
END-REWRITE
* LOG THE AUDIT RECORD
PERFORM 2100-WRITE-AUDIT-ENTRY.
2100-WRITE-AUDIT-ENTRY.
MOVE 'WRIT' TO WS-AI-FUNCTION
MOVE 'CHNG' TO WS-AI-RECORD-TYPE
MOVE WS-BATCH-USER TO WS-AI-USER-ID
MOVE 'PFGPOST1' TO WS-AI-PROGRAM-NAME
MOVE SPACES TO WS-AI-TRANSACTION-ID
MOVE SPACES TO WS-AI-TERMINAL-ID
MOVE 'PFGEOD01' TO WS-AI-JOB-NAME
MOVE 'POSTING' TO WS-AI-STEP-NAME
MOVE 'ACCTMAST' TO WS-AI-MASTER-FILE-ID
MOVE WS-TRANS-ACCOUNT
TO WS-AI-RECORD-KEY
STRING 'POSTING TYPE=' WS-TRANS-TYPE
' AMT=' WS-TRANS-AMOUNT
DELIMITED BY SIZE
INTO WS-AI-CHANGE-REASON
MOVE 500 TO WS-AI-BEFORE-LENGTH
MOVE WS-ACCOUNT-BEFORE
TO WS-AI-BEFORE-DATA
MOVE 500 TO WS-AI-AFTER-LENGTH
MOVE WS-ACCOUNT-AFTER
TO WS-AI-AFTER-DATA
CALL 'PFGAUD01' USING WS-AUDIT-INTERFACE
IF WS-AI-RETURN-CODE NOT = ZERO
DISPLAY 'AUDIT WRITE FAILED FOR ACCOUNT '
WS-TRANS-ACCOUNT
' RC=' WS-AI-RETURN-CODE
PERFORM 8200-HANDLE-AUDIT-FAILURE
END-IF.
0900-CLOSE-AUDIT.
MOVE 'CLOS' TO WS-AI-FUNCTION
CALL 'PFGAUD01' USING WS-AUDIT-INTERFACE.
Critical Design Pattern: Audit Before Commit
Notice the order of operations in paragraph 2000-POST-ONE-TRANSACTION:
- Read the account record
- Capture the before-image
- Apply the business logic
- Capture the after-image
- Rewrite the account record
- Write the audit record
In a CICS environment, both the REWRITE and the audit WRITE would occur within the same logical unit of work (LUW). If either operation fails, the CICS syncpoint mechanism rolls back both changes. This guarantees that the master data and the audit trail remain synchronized -- it is impossible for a data change to exist without a corresponding audit record.
The audit failure handler (paragraph 8200) is intentionally severe: if the audit write fails, the entire transaction is aborted. The business rule is explicit -- "no audit, no change." This policy was mandated by the compliance team and accepted by the business despite the potential for transaction failures due to audit system problems.
Audit Trail Verification Program
The hash chain verification program reads the entire audit trail and verifies that no records have been tampered with:
IDENTIFICATION DIVISION.
PROGRAM-ID. PFGAVFY1.
*================================================================*
* PROGRAM: PFGAVFY1 - AUDIT TRAIL HASH CHAIN VERIFIER
* PURPOSE: READ THE COMPLETE AUDIT TRAIL AND VERIFY THAT
* THE HASH CHAIN IS UNBROKEN, DETECTING ANY
* TAMPERING, DELETION, OR INSERTION OF RECORDS.
*================================================================*
ENVIRONMENT DIVISION.
INPUT-OUTPUT SECTION.
FILE-CONTROL.
SELECT AUDIT-FILE
ASSIGN TO AUDITIN
ORGANIZATION IS SEQUENTIAL
FILE STATUS IS WS-AUDIT-STATUS.
SELECT VERIFY-REPORT
ASSIGN TO VERIFYRPT
ORGANIZATION IS SEQUENTIAL
FILE STATUS IS WS-REPORT-STATUS.
DATA DIVISION.
FILE SECTION.
FD AUDIT-FILE
RECORDING MODE IS F
RECORD CONTAINS 2000 CHARACTERS.
01 AUDIT-FILE-RECORD PIC X(2000).
FD VERIFY-REPORT
RECORDING MODE IS F
RECORD CONTAINS 133 CHARACTERS.
01 REPORT-LINE PIC X(133).
WORKING-STORAGE SECTION.
01 WS-AUDIT-STATUS PIC X(2).
01 WS-REPORT-STATUS PIC X(2).
01 WS-EOF-FLAG PIC X(1) VALUE 'N'.
88 END-OF-FILE VALUE 'Y'.
COPY AUDREC01.
01 WS-EXPECTED-HASH PIC X(32) VALUE ALL '0'.
01 WS-COMPUTED-HASH PIC X(32).
01 WS-SAVED-HASH PIC X(32).
01 WS-EXPECTED-SEQUENCE PIC 9(12) VALUE 1.
01 WS-COUNTERS.
05 WS-RECORDS-READ PIC 9(12) VALUE ZERO.
05 WS-HASH-ERRORS PIC 9(8) VALUE ZERO.
05 WS-SEQUENCE-ERRORS PIC 9(8) VALUE ZERO.
05 WS-CHAIN-ERRORS PIC 9(8) VALUE ZERO.
05 WS-TOTAL-ERRORS PIC 9(8) VALUE ZERO.
01 WS-HASH-WORK-AREA.
05 WS-HASH-INPUT PIC X(2000).
05 WS-HASH-INPUT-LEN PIC 9(8).
05 WS-HASH-OUTPUT PIC X(32).
05 WS-HASH-ACCUM-1 PIC 9(8).
05 WS-HASH-ACCUM-2 PIC 9(8).
05 WS-HASH-ACCUM-3 PIC 9(8).
05 WS-HASH-ACCUM-4 PIC 9(8).
05 WS-HASH-IDX PIC 9(8).
05 WS-HASH-BYTE PIC 9(3).
01 WS-DETAIL-LINE.
05 FILLER PIC X(1) VALUE SPACE.
05 DL-SEQ-NUM PIC Z(11)9.
05 FILLER PIC X(1) VALUE SPACE.
05 DL-TIMESTAMP PIC X(26).
05 FILLER PIC X(1) VALUE SPACE.
05 DL-ERROR-TYPE PIC X(20).
05 FILLER PIC X(1) VALUE SPACE.
05 DL-EXPECTED PIC X(32).
05 FILLER PIC X(1) VALUE SPACE.
05 DL-ACTUAL PIC X(32).
PROCEDURE DIVISION.
0000-MAIN.
PERFORM 1000-INITIALIZE
PERFORM 2000-VERIFY-RECORDS
UNTIL END-OF-FILE
PERFORM 3000-PRINT-SUMMARY
PERFORM 9000-TERMINATE
STOP RUN.
1000-INITIALIZE.
OPEN INPUT AUDIT-FILE
OPEN OUTPUT VERIFY-REPORT
PERFORM 8000-READ-AUDIT.
2000-VERIFY-RECORDS.
ADD 1 TO WS-RECORDS-READ
MOVE AUDIT-FILE-RECORD TO AUDIT-TRAIL-RECORD
* CHECK 1: VERIFY SEQUENCE NUMBER
IF AUD-SEQUENCE-NUM NOT = WS-EXPECTED-SEQUENCE
ADD 1 TO WS-SEQUENCE-ERRORS
ADD 1 TO WS-TOTAL-ERRORS
MOVE AUD-SEQUENCE-NUM TO DL-SEQ-NUM
MOVE AUD-TIMESTAMP TO DL-TIMESTAMP
MOVE 'SEQUENCE GAP' TO DL-ERROR-TYPE
WRITE REPORT-LINE FROM WS-DETAIL-LINE
AFTER ADVANCING 1 LINE
END-IF
* CHECK 2: VERIFY PREVIOUS-HASH CHAIN LINK
IF AUD-PREV-HASH NOT = WS-EXPECTED-HASH
ADD 1 TO WS-CHAIN-ERRORS
ADD 1 TO WS-TOTAL-ERRORS
MOVE AUD-SEQUENCE-NUM TO DL-SEQ-NUM
MOVE AUD-TIMESTAMP TO DL-TIMESTAMP
MOVE 'CHAIN BREAK' TO DL-ERROR-TYPE
MOVE WS-EXPECTED-HASH TO DL-EXPECTED
MOVE AUD-PREV-HASH TO DL-ACTUAL
WRITE REPORT-LINE FROM WS-DETAIL-LINE
AFTER ADVANCING 1 LINE
END-IF
* CHECK 3: RECOMPUTE AND VERIFY RECORD HASH
MOVE AUD-HASH-VALUE TO WS-SAVED-HASH
MOVE AUDIT-TRAIL-RECORD TO WS-HASH-INPUT
MOVE SPACES TO WS-HASH-INPUT(165:32)
MOVE 2000 TO WS-HASH-INPUT-LEN
PERFORM 2200-COMPUTE-HASH
MOVE WS-HASH-OUTPUT TO WS-COMPUTED-HASH
IF WS-COMPUTED-HASH NOT = WS-SAVED-HASH
ADD 1 TO WS-HASH-ERRORS
ADD 1 TO WS-TOTAL-ERRORS
MOVE AUD-SEQUENCE-NUM TO DL-SEQ-NUM
MOVE AUD-TIMESTAMP TO DL-TIMESTAMP
MOVE 'HASH MISMATCH' TO DL-ERROR-TYPE
MOVE WS-SAVED-HASH TO DL-EXPECTED
MOVE WS-COMPUTED-HASH TO DL-ACTUAL
WRITE REPORT-LINE FROM WS-DETAIL-LINE
AFTER ADVANCING 1 LINE
END-IF
* ADVANCE EXPECTED VALUES FOR NEXT RECORD
MOVE AUD-HASH-VALUE TO WS-EXPECTED-HASH
ADD 1 TO WS-EXPECTED-SEQUENCE
PERFORM 8000-READ-AUDIT.
2200-COMPUTE-HASH.
* (SAME HASH ALGORITHM AS PFGAUD01)
MOVE ZERO TO WS-HASH-ACCUM-1
MOVE ZERO TO WS-HASH-ACCUM-2
MOVE ZERO TO WS-HASH-ACCUM-3
MOVE ZERO TO WS-HASH-ACCUM-4
PERFORM VARYING WS-HASH-IDX FROM 1 BY 1
UNTIL WS-HASH-IDX > WS-HASH-INPUT-LEN
COMPUTE WS-HASH-BYTE =
FUNCTION ORD(WS-HASH-INPUT(
WS-HASH-IDX:1))
COMPUTE WS-HASH-ACCUM-1 =
FUNCTION MOD(
WS-HASH-ACCUM-1 * 31 +
WS-HASH-BYTE, 99999999)
COMPUTE WS-HASH-ACCUM-2 =
FUNCTION MOD(
WS-HASH-ACCUM-2 * 37 +
WS-HASH-BYTE, 99999999)
COMPUTE WS-HASH-ACCUM-3 =
FUNCTION MOD(
WS-HASH-ACCUM-3 * 41 +
WS-HASH-BYTE, 99999999)
COMPUTE WS-HASH-ACCUM-4 =
FUNCTION MOD(
WS-HASH-ACCUM-4 * 43 +
WS-HASH-BYTE, 99999999)
END-PERFORM
STRING WS-HASH-ACCUM-1
WS-HASH-ACCUM-2
WS-HASH-ACCUM-3
WS-HASH-ACCUM-4
DELIMITED BY SIZE
INTO WS-HASH-OUTPUT.
3000-PRINT-SUMMARY.
DISPLAY '======================================'
DISPLAY 'AUDIT TRAIL VERIFICATION COMPLETE'
DISPLAY 'RECORDS VERIFIED: ' WS-RECORDS-READ
DISPLAY 'HASH ERRORS: ' WS-HASH-ERRORS
DISPLAY 'SEQUENCE ERRORS: ' WS-SEQUENCE-ERRORS
DISPLAY 'CHAIN ERRORS: ' WS-CHAIN-ERRORS
DISPLAY 'TOTAL ERRORS: ' WS-TOTAL-ERRORS
DISPLAY '======================================'
IF WS-TOTAL-ERRORS = ZERO
DISPLAY 'RESULT: AUDIT TRAIL INTEGRITY VERIFIED'
ELSE
DISPLAY 'RESULT: INTEGRITY VIOLATIONS DETECTED'
DISPLAY '*** IMMEDIATE INVESTIGATION REQUIRED ***'
END-IF.
8000-READ-AUDIT.
READ AUDIT-FILE INTO AUDIT-TRAIL-RECORD
AT END SET END-OF-FILE TO TRUE
END-READ.
9000-TERMINATE.
CLOSE AUDIT-FILE
CLOSE VERIFY-REPORT
IF WS-TOTAL-ERRORS > ZERO
MOVE 8 TO RETURN-CODE
ELSE
MOVE 0 TO RETURN-CODE
END-IF.
The verification program is run nightly as part of the batch cycle and monthly by the internal audit team using an independent copy of the verification program compiled from source they control. The dual-run approach provides separation of duties: IT operations runs the nightly check, and internal audit independently verifies the results.
The JCL for the Complete Audit Trail Processing
//PFGAUDIT JOB (ACCT),'PFG AUDIT VERIFY',
// CLASS=A,MSGCLASS=X,MSGLEVEL=(1,1),
// NOTIFY=&SYSUID
//*
//*================================================================*
//* NIGHTLY AUDIT TRAIL VERIFICATION
//*================================================================*
//*
//VERIFY EXEC PGM=PFGAVFY1
//STEPLIB DD DSN=PFG.CORE.PROD.LOADLIB,DISP=SHR
//AUDITIN DD DSN=PFG.CORE.PROD.LOG.AUDIT.DAILY(0),
// DISP=SHR
//VERIFYRPT DD SYSOUT=*,
// DCB=(RECFM=FBA,LRECL=133,BLKSIZE=26600)
//SYSOUT DD SYSOUT=*
Lessons Learned
1. "No Audit, No Change" Must Be an Inviolable Rule
The most contentious design decision was the policy that a failed audit write aborts the entire transaction. Business stakeholders initially resisted this, arguing that operational availability should take precedence over audit logging. The compliance team overruled them: SOX requires demonstrable controls, and a system that can modify financial data without creating an audit record is, by definition, not in compliance. In two years of production operation, the audit system has had zero outages, validating the engineering team's reliability design.
2. Hash Chaining Provides Tamper Detection, Not Tamper Prevention
The hash chain can detect that records have been altered, but it cannot prevent a sufficiently privileged administrator from rewriting the entire audit file with a consistent new hash chain. The compensating control is RACF protection: no human user has write access to the audit file. Only the CICS and batch service accounts can append records, and only the audit team can read them. The combination of hash chaining (detection) and RACF (prevention) provides defense in depth.
3. Before/After Images Double the Storage Requirement but Eliminate Ambiguity
Storing complete before and after images of every changed record is expensive -- the audit file grows by approximately 6 GB per day. The team initially considered storing only the changed fields (a delta approach). The auditors rejected this: reconstructing a record from a chain of deltas introduces complexity and potential for error. With complete images, an auditor can examine any audit record and see exactly what the record looked like before and after the change without any reconstruction logic.
4. The Centralized Audit Module Prevents Inconsistency
By funneling all audit writes through a single COBOL module (PFGAUD01), the team ensures that every audit record has the same format, the same timestamp precision, and the same hash chain logic. If each application program implemented its own audit logging, format inconsistencies and hash chain breaks would be inevitable.
5. Verification Must Be Independent
The nightly verification job runs the same PFGAVFY1 program that IT developed. The internal audit team compiles their own copy from reviewed source code and runs it independently. If IT modified the verification program to hide tampering, the audit team's independent copy would detect the discrepancy. This separation of duties is a core SOX requirement.
Discussion Questions
-
The hash chain starts with AUD-PREV-HASH set to all zeros for the first record. An attacker who could delete the entire audit file and rewrite it from scratch could create a valid hash chain. What additional controls beyond RACF could prevent this attack?
-
The simplified hash algorithm used in this case study is not cryptographically secure. In production, ICSF provides SHA-256. What properties does SHA-256 provide that the simplified hash does not, and why do those properties matter for audit integrity?
-
The audit record stores the complete before and after images. For a 500-byte account record, the audit record is 2000 bytes. If the bank processes 3 million transactions per day, calculate the daily and annual storage requirements for the audit trail. How would you manage this data volume?
-
The "no audit, no change" policy means that an audit system failure halts all financial processing. Design a fallback mechanism that maintains SOX compliance while providing some level of business continuity during an audit system outage.
-
The current design writes audit records to a sequential file. Propose an alternative architecture that writes audit records to a DB2 table. What are the advantages for queryability, and what are the risks for tamper-evidence?
Connection to Chapter Concepts
This case study builds on several key concepts from Chapter 31:
-
RACF protection for audit files (Section: RACF -- Resource Access Control Facility): The write-only access model for the audit file demonstrates how RACF profiles enforce append-only semantics at the operating system level.
-
Separation of duties (Section: Security Principles for z/OS): The independent verification by IT operations and internal audit demonstrates the SOX-mandated separation of duties principle.
-
Secure COBOL coding (Section: Secure Coding Practices for z/OS): The centralized audit module, mandatory audit-before-commit pattern, and hash chaining demonstrate application-level security controls built directly into COBOL programs.
-
Regulatory compliance (Section: Compliance Frameworks and z/OS Security): The SOX Section 404 requirements for internal controls over financial reporting drive every design decision in this case study, illustrating how regulatory mandates shape mainframe security architecture.