Case Study 2: CNB's Inter-Bank SOAP Services

Case Study 2: CNB's Inter-Bank SOAP Services

Connecting to the Clearing Network with WSDL Contracts and WS-Security

Background

Continental National Bank's (CNB) core banking system processes 500 million transactions per day. A significant portion of those transactions involve communication with external entities: other banks in the ACH network, the Federal Reserve's wire transfer system (Fedwire), credit card networks, and international payment systems (SWIFT).

These inter-bank interfaces are governed by strict standards. The ACH network uses NACHA file formats. Fedwire uses proprietary message formats. But the growing ISO 20022 migration — which replaces legacy message formats with structured XML — and the rise of real-time payment networks (RTP, FedNow) have pushed CNB toward SOAP-based web services for an increasing share of inter-bank communication.

This case study examines how Kwame Mensah's architecture team implemented SOAP web services for CNB's participation in a regional clearing network that mandates ISO 20022 XML messaging with WS-Security.

The Mandate

In early 2022, the Regional Payment Clearing Network (RPCN) — a consortium of 35 banks that facilitates same-day settlement for domestic payments — announced a mandatory migration from flat-file batch exchange to real-time SOAP web services. The specification:

Protocol: SOAP 1.2 over HTTPS with mutual TLS
Message format: ISO 20022 pain.001.001.03 (customer credit transfer initiation) and pacs.008.001.02 (interbank credit transfer)
Security: WS-Security with X.509 certificate-based message signing
SLA: 99.95% availability, maximum 2-second response time for acknowledgment
Volume: CNB's share approximately 120,000 messages per day (60,000 inbound, 60,000 outbound)
Deadline: Q4 2023

CNB needed to implement both provider mode (receiving inbound payment messages from other banks) and requester mode (sending outbound payment messages to other banks through the RPCN gateway).

Architecture

Kwame's team designed a dedicated SOAP services layer within CNB's existing CICS topology:

                          RPCN Network
                              │
                    ┌─────────┴─────────┐
                    │  RPCN Gateway      │
                    │  (Network Hub)     │
                    └─────┬───────┬─────┘
                          │       │
            Inbound       │       │      Outbound
            (Provider)    ▼       │      (Requester)
                    ┌─────────────┤
                    │             │
              ┌─────▼──────┐  ┌──▼───────────┐
              │ CNBTORD1   │  │ CNBAORD3     │
              │ TOR/WOR    │  │ AOR          │
              │ (SYSD)     │  │ (SYSD)       │
              │ Port 8443  │  │ Outbound     │
              │ SOAP In    │  │ SOAP calls   │
              └─────┬──────┘  └──────────────┘
                    │ MRO
              ┌─────▼──────┐
              │ CNBAORD1   │
              │ AOR        │
              │ Payment    │
              │ Processing │
              └─────┬──────┘
                    │ DB2
              ┌─────▼──────┐
              │ DB2 (DSND) │
              │ Payment    │
              │ Tables     │
              └────────────┘

Key design decisions:

Dedicated LPAR (SYSD) for inter-bank traffic. RPCN messages are processed on SYSD, isolated from customer-facing channels (SYSA/SYSB for 3270/ATM, SYSC for web). This prevents inter-bank processing issues from affecting customer service and vice versa.
Separate TOR (CNBTORD1) for SOAP inbound. The TCPIPSERVICE on port 8443 receives SOAP messages from the RPCN gateway. This TOR handles only RPCN traffic and applies mutual TLS authentication using RPCN's certificate chain.
AOR (CNBAORD1) for payment processing. The existing COBOL payment processing programs run here. They were not modified for SOAP — a wrapper program handles the XML-to-COMMAREA mapping.
AOR (CNBAORD3) for outbound calls. Outbound SOAP messages (requester mode) originate from this AOR. Separating outbound from inbound prevents slow external responses from blocking inbound message processing.

Implementation: Inbound (Provider Mode)

WSDL Contract

RPCN published a WSDL for inbound payment messages. The key operation:

<operation name="processPaymentInstruction">
  <input message="tns:PaymentInstructionRequest"/>
  <output message="tns:PaymentInstructionAcknowledgment"/>
  <fault name="PaymentInstructionFault"
         message="tns:ProcessingError"/>
</operation>

The PaymentInstructionRequest wraps an ISO 20022 pain.001 XML document. The structure includes nested complex types:

<CstmrCdtTrfInitn>
  <GrpHdr>
    <MsgId>MSG-2023-10-15-CNB-000001</MsgId>
    <CreDtTm>2023-10-15T14:30:00Z</CreDtTm>
    <NbOfTxs>1</NbOfTxs>
    <CtrlSum>5000.00</CtrlSum>
    <InitgPty>
      <Nm>ACME Corporation</Nm>
      <Id>
        <OrgId>
          <Othr>
            <Id>ACME-CORP-001</Id>
          </Othr>
        </OrgId>
      </Id>
    </InitgPty>
  </GrpHdr>
  <PmtInf>
    <PmtInfId>PMT-2023-10-15-001</PmtInfId>
    <PmtMtd>TRF</PmtMtd>
    <CdtTrfTxInf>
      <Amt>
        <InstdAmt Ccy="USD">5000.00</InstdAmt>
      </Amt>
      <CdtrAgt>
        <FinInstnId>
          <BICFI>CNBKUS33</BICFI>
        </FinInstnId>
      </CdtrAgt>
      <Cdtr>
        <Nm>John Smith</Nm>
      </Cdtr>
      <CdtrAcct>
        <Id>
          <Othr>
            <Id>9876543210</Id>
          </Othr>
        </Id>
      </CdtrAcct>
    </CdtTrfTxInf>
  </PmtInf>
</CstmrCdtTrfInitn>

The DFHWS2LS Challenge

Kwame's team ran DFHWS2LS against the RPCN WSDL. The generated COBOL copybook was 847 lines long. The ISO 20022 XML schema's deeply nested structure produced a COBOL copybook with 8 levels of nesting and numerous OCCURS DEPENDING ON constructs.

       01  PMTINST-REQUEST.
           05  GRP-HDR.
               10  MSG-ID             PIC X(35).
               10  CRE-DT-TM         PIC X(25).
               10  NB-OF-TXS         PIC S9(09) COMP.
               10  CTRL-SUM           PIC S9(14)V99 COMP-3.
               10  INITG-PTY.
                   15  INITG-NM       PIC X(140).
                   15  INITG-ORG-ID   PIC X(35).
           05  PMT-INF-COUNT          PIC S9(04) COMP.
           05  PMT-INF OCCURS 100 TIMES
               DEPENDING ON PMT-INF-COUNT.
               10  PMT-INF-ID         PIC X(35).
               10  PMT-MTD            PIC X(03).
               10  CDT-TRF-COUNT      PIC S9(04) COMP.
               10  CDT-TRF-TX-INF OCCURS 999 TIMES
                   DEPENDING ON CDT-TRF-COUNT.
                   15  INSTD-AMT       PIC S9(14)V99 COMP-3.
                   15  INSTD-CCY       PIC X(03).
                   15  CDTR-AGT-BIC    PIC X(11).
                   15  CDTR-NM         PIC X(140).
                   15  CDTR-ACCT-ID    PIC X(34).
      *        ... (47 additional fields per transaction)

The problem: the generated copybook with OCCURS DEPENDING ON for 100 payment information groups, each containing up to 999 transactions, produced a potential COMMAREA size exceeding 32KB — far beyond the COMMAREA limit.

Solution: Channel/Container interface. Instead of COMMAREA, the team used channels and containers (Chapter 15). The pipeline places the transformed XML data into containers, and the COBOL program retrieves them individually. This eliminates the size constraint.

      *    Retrieve group header from container
           EXEC CICS GET CONTAINER('GRP-HDR')
               CHANNEL('RPCN-INBOUND')
               INTO(WS-GRP-HDR)
               FLENGTH(WS-GRP-LEN)
           END-EXEC

      *    Process each payment instruction
           PERFORM VARYING WS-PMT-IDX FROM 1 BY 1
               UNTIL WS-PMT-IDX > WS-PMT-COUNT

               STRING 'PMT-INF-' DELIMITED SIZE
                      WS-PMT-IDX-CHAR DELIMITED SIZE
                      INTO WS-CONTAINER-NAME
               END-STRING

               EXEC CICS GET CONTAINER(WS-CONTAINER-NAME)
                   CHANNEL('RPCN-INBOUND')
                   INTO(WS-PMT-INF)
                   FLENGTH(WS-PMT-LEN)
               END-EXEC

               PERFORM PROCESS-PAYMENT-INSTRUCTION
           END-PERFORM

WS-Security Integration

RPCN requires WS-Security with X.509 certificate-based message signing. Each inbound SOAP message includes a digital signature in the SOAP header:

<wsse:Security>
  <ds:Signature>
    <ds:SignedInfo>
      <ds:Reference URI="#Body">
        <ds:DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256"/>
        <ds:DigestValue>abc123...</ds:DigestValue>
      </ds:Reference>
    </ds:SignedInfo>
    <ds:SignatureValue>xyz789...</ds:SignatureValue>
    <ds:KeyInfo>
      <wsse:SecurityTokenReference>
        <wsse:Reference URI="#X509Token"/>
      </wsse:SecurityTokenReference>
    </ds:KeyInfo>
  </ds:Signature>
</wsse:Security>

CICS TS 5.6 supports WS-Security validation through the SOAP pipeline's security handler. The configuration:

<provider_pipeline>
  <service>
    <terminal_handler>
      <cics_soap_1.2_handler/>
    </terminal_handler>
    <service_handler_list>
      <handler>
        <program name="DFHPIWSSH"/>
        <handler_parameter_list>
          <handler_parameter name="validate-signature"
                             value="yes"/>
          <handler_parameter name="keystore"
                             value="safkeyring:///CNBKeyring"/>
        </handler_parameter_list>
      </handler>
      <handler>
        <program name="DFHPITP"/>
      </handler>
    </service_handler_list>
  </service>
  <apphandler>DFHPITP</apphandler>
</provider_pipeline>

The DFHPIWSSH program validates the digital signature using the sender's public certificate stored in the RACF keyring. If validation fails, CICS returns a SOAP fault before the COBOL program is invoked.

Implementation: Outbound (Requester Mode)

For outbound messages, CNB sends payment instructions to the RPCN gateway. The COBOL program builds the payment data, and CICS handles SOAP envelope construction, WS-Security signing, and HTTP transmission.

The Outbound Flow

       SEND-RPCN-PAYMENT.
      *    Build payment data in containers
           EXEC CICS PUT CONTAINER('GRP-HDR')
               CHANNEL('RPCN-OUTBOUND')
               FROM(WS-OUT-GRP-HDR)
               FLENGTH(LENGTH OF WS-OUT-GRP-HDR)
               CHAR
           END-EXEC

           PERFORM VARYING WS-PMT-IDX FROM 1 BY 1
               UNTIL WS-PMT-IDX > WS-OUT-PMT-COUNT

               STRING 'PMT-INF-' DELIMITED SIZE
                      WS-PMT-IDX-CHAR DELIMITED SIZE
                      INTO WS-CONTAINER-NAME
               END-STRING

               EXEC CICS PUT CONTAINER(WS-CONTAINER-NAME)
                   CHANNEL('RPCN-OUTBOUND')
                   FROM(WS-OUT-PMT-INF(WS-PMT-IDX))
                   FLENGTH(LENGTH OF WS-OUT-PMT-INF(1))
                   CHAR
               END-EXEC
           END-PERFORM

      *    Invoke the outbound web service
           EXEC CICS INVOKE WEBSERVICE('RPCNOUT')
               CHANNEL('RPCN-OUTBOUND')
               OPERATION('processPaymentInstruction')
               RESP(WS-CICS-RESP)
               RESP2(WS-CICS-RESP2)
           END-EXEC

           EVALUATE WS-CICS-RESP
               WHEN DFHRESP(NORMAL)
                   PERFORM PROCESS-RPCN-ACK
               WHEN DFHRESP(INVREQ)
                   PERFORM HANDLE-RPCN-REJECTION
               WHEN DFHRESP(TIMEDOUT)
                   PERFORM HANDLE-RPCN-TIMEOUT
               WHEN OTHER
                   PERFORM HANDLE-RPCN-ERROR
           END-EVALUATE

The requester pipeline configuration includes WS-Security signing:

<requester_pipeline>
  <service>
    <service_handler_list>
      <handler>
        <program name="DFHPIWSSH"/>
        <handler_parameter_list>
          <handler_parameter name="sign-body" value="yes"/>
          <handler_parameter name="signing-key-label"
                             value="CNB-RPCN-SIGN"/>
          <handler_parameter name="keystore"
                             value="safkeyring:///CNBKeyring"/>
        </handler_parameter_list>
      </handler>
      <handler>
        <program name="DFHPITP"/>
      </handler>
    </service_handler_list>
    <terminal_handler>
      <cics_soap_1.2_handler/>
    </terminal_handler>
  </service>
  <apphandler>DFHPITP</apphandler>
</requester_pipeline>

Outbound Timeout and Retry Logic

The RPCN gateway SLA guarantees a response within 2 seconds. CNB's COBOL program sets a 3-second timeout with retry logic:

       SEND-WITH-RETRY.
           MOVE 0 TO WS-RETRY-COUNT
           SET WS-SEND-PENDING TO TRUE

           PERFORM UNTIL WS-SEND-COMPLETE OR
                         WS-SEND-FAILED

               ADD 1 TO WS-RETRY-COUNT

               PERFORM INVOKE-RPCN-SERVICE

               EVALUATE TRUE
                   WHEN WS-INVOKE-SUCCESS
                       SET WS-SEND-COMPLETE TO TRUE
                   WHEN WS-INVOKE-TIMEOUT
                       IF WS-RETRY-COUNT < 3
                           CONTINUE
                       ELSE
                           SET WS-SEND-FAILED TO TRUE
                           MOVE 'TIMEOUT AFTER 3 RETRIES'
                               TO WS-FAILURE-REASON
                       END-IF
                   WHEN WS-INVOKE-REJECTED
                       SET WS-SEND-FAILED TO TRUE
                       MOVE 'RPCN REJECTED MESSAGE'
                           TO WS-FAILURE-REASON
                   WHEN OTHER
                       IF WS-RETRY-COUNT < 3
                           CONTINUE
                       ELSE
                           SET WS-SEND-FAILED TO TRUE
                           MOVE 'SEND FAILED AFTER 3 RETRIES'
                               TO WS-FAILURE-REASON
                       END-IF
               END-EVALUATE
           END-PERFORM

           IF WS-SEND-FAILED
               PERFORM QUEUE-FOR-MANUAL-REVIEW
           END-IF

Production Results

The RPCN SOAP services went live on October 2, 2023. Results after 6 months:

Volume and Performance

Metric	Target	Actual
Inbound messages/day	60,000	58,000–72,000
Outbound messages/day	60,000	55,000–68,000
Inbound response time (P95)	2,000ms	340ms
Outbound response time (P95)	2,000ms	510ms
Availability	99.95%	99.993%

Error Analysis

Error Type	Count (6 months)	Resolution
WS-Security signature invalid	47	All from one bank with misconfigured certificate; resolved in 2 days
Schema validation failure	312	Various banks sending non-conforming XML; flagged to RPCN for enforcement
CICS internal errors	3	All during a planned DB2 maintenance window; scheduled processing avoided these hours
Timeout (3-second)	89	86 during RPCN planned maintenance; 3 genuine network issues
Retry success	71	80% of retryable errors resolved on first retry

The XML Transformation Cost

Unlike SecureFirst's lightweight JSON transformations, the ISO 20022 XML transformation was significantly more expensive:

Message Size	XML Transformation Time
Simple (1 transaction, 3KB XML)	4.2ms
Medium (10 transactions, 25KB XML)	18.5ms
Large (100 transactions, 200KB XML)	142ms
Maximum (999 transactions, 1.8MB XML)	1,240ms

For large batch payment messages with hundreds of transactions, the XML transformation consumed more time than the actual payment processing. Lisa Tran noted: "The COBOL program processes each transaction in 2ms. The XML parser spends 1.2 seconds just unpacking them."

Kwame's solution: for messages with more than 50 transactions, the inbound handler splits them into individual transaction containers and processes them in parallel across multiple AOR tasks using EXEC CICS START. This reduced effective processing time for a 999-transaction message from 3.2 seconds (sequential) to 890ms (parallel across 4 AOR tasks).

Key Architectural Decisions and Their Rationale

Decision 1: SOAP Over REST

The RPCN network mandated SOAP. But even if they hadn't, Kwame would have recommended SOAP for this use case:

WS-Security provides message-level security (the message is signed, not just the transport). If the message passes through an intermediary gateway, the signature remains valid. TLS only secures the transport — once the TLS session terminates, the message is unprotected.
WSDL contracts provide machine-readable, unambiguous interface definitions. When 35 banks must implement the same interface, a WSDL is more precise than an OpenAPI spec.
SOAP faults provide structured error reporting. When a payment is rejected, the SOAP fault contains a machine-readable error code, a human-readable description, and a detail element with the specific validation failure. REST error responses vary widely between implementations.

Decision 2: Separate Outbound AOR

Separating inbound (provider) and outbound (requester) processing into different AORs prevents a cascading failure pattern:

If the RPCN gateway slows down, outbound calls accumulate in the AOR, consuming MAXTASK slots. If outbound and inbound share an AOR, the MAXTASK exhaustion blocks inbound message processing — CNB stops receiving payments because it's stuck waiting to send them. With separate AORs, the outbound AOR hits MAXTASK while the inbound AOR continues processing normally.

Decision 3: Channel/Container Over COMMAREA

ISO 20022 messages routinely exceed the 32KB COMMAREA limit. Channels and containers have no practical size limit (limited by region storage). This decision was forced by the message format, but Kwame notes: "We would have chosen channels regardless. COMMAREA is a legacy interface. Every new CICS program we write uses channels."

Lessons Learned

XML transformation is expensive. Budget CPU and elapsed time for XML parsing, especially for large ISO 20022 messages. Paginate or parallelize where possible.
WS-Security adds significant overhead. Digital signature generation and validation adds 5–15ms per message. For high-volume services, ensure the CICS region has sufficient cryptographic hardware (CP Assist for Cryptographic Functions — CPACF, or a Crypto Express adapter) offloaded from general-purpose CPs.
Certificate management is an ongoing operation. RPCN certificates rotate annually. CNB certificates rotate every two years. The RACF keyring must be updated before expiration, or services fail with signature validation errors. Kwame's team maintains a certificate expiration calendar and tests renewal 30 days before expiration.
SOAP 1.2 and SOAP 1.1 are not interchangeable. Three partner banks initially connected using SOAP 1.1 clients against CNB's SOAP 1.2 endpoint. The namespace differences caused silent failures. The solution: explicit SOAP version checking in the pipeline, with a clear error message when the wrong version is detected.
ISO 20022 compliance testing is non-trivial. The RPCN certification process required processing 500 test messages covering edge cases: multi-currency transactions, rejected payments, partial batch failures, and duplicate detection. The COBOL programs handled the business logic correctly, but several test cases failed due to XML formatting issues in the response — specifically, the DFHLS2WS-generated WSDL produced slightly different namespace prefixes than the RPCN conformance checker expected.

Discussion Questions

If RPCN migrated from SOAP to REST in the future, what components of CNB's architecture would change and what would remain unchanged?
The XML transformation for a 999-transaction message takes 1.2 seconds. Propose two alternative architectures that could reduce this without changing the SOAP protocol.
How does message-level security (WS-Security signing) differ from transport-level security (TLS) in terms of what is protected? When does the difference matter?
CNB processes outbound payment messages from a separate AOR. If the RPCN gateway is down for 30 minutes, describe what happens to the outbound message queue and how the system recovers when the gateway comes back.
Compare CNB's SOAP implementation with SecureFirst's REST implementation from Case Study 1. Identify three architectural decisions that differ and explain why each difference is justified by the different use cases.