Case Study 1: Eleven Hours — The Meridian Ransomware Incident

"We didn't beat them because we were smarter that morning. We beat them because we'd already had that morning, on paper, four times." — Priya Nair, IR lead, Meridian Regional Bank (constructed)

Executive Summary

This case study is the real-time, end-to-end account of the ransomware incident that opens Chapter 24 — the one the tabletop in §24.5 rehearsed in advance. Over eleven hours on a Saturday, an attacker who had quietly compromised an over-privileged service account attempted to deploy ransomware across Meridian Regional Bank's loan-origination infrastructure, deleting backups to remove the bank's recovery path and threatening to leak exfiltrated customer data. Meridian detected it in nine minutes, declared a SEV-1, appointed an incident commander, contained the active encryption to two servers, eradicated the attacker, recovered from offline backups without paying the ransom, and notified its regulator inside the 36-hour window. We follow the response through all four phases of the NIST lifecycle as the team actually experienced them — which, per Figure 24.1, is never the clean linear order the diagram suggests. The scenario and all figures are constructed for teaching (Tier 3), built on the well-documented pattern of real critical-infrastructure ransomware such as the 2021 Colonial Pipeline incident.

Skills applied: triage and severity classification; incident declaration and the incident-commander role; scoping a privileged-account compromise; the containment decision under time pressure; eradication of a domain-admin-level compromise; recovery from offline/immutable backups; multi-audience crisis communications under a regulatory clock; the blameless postmortem and its action items.

Background

Meridian Regional Bank — ~1,800 employees, ~120 branches, ~2.5 million customers, hybrid on-prem/AWS infrastructure (see the bank's profile threaded through this book) — had spent the prior year maturing its security program after a phishing near-miss (Chapter 1). By the time of this incident it had a SIEM (Chapter 21), detection engineering and threat hunting (Chapter 22), a privileged-access program (Chapter 19), and — crucially — an incident-response plan with playbooks, runbooks, a severity matrix, an incident-commander model, an out-of-band communications channel, and a quarterly tabletop. Most of those artifacts were less than a year old. One gap remained that no one had prioritized: a legacy backup service account, svc_backup, had been granted domain-admin rights years earlier "to make backups work" and never walked back. The bank's most recent tabletop had, in fact, flagged exactly this kind of over-privileged service account as a worry — a foreshadowing the team would think about ruefully before the day was out.

The attacker's path into Meridian, reconstructed later: svc_backup used a password that was weak, reused, and present in a credential dump from an unrelated breach; and a misconfigured remote-management interface had left a path that could be reached and authenticated against. The attacker had logged in days earlier, moved quietly, and chosen a Saturday morning — low staffing, slow response — to act. This is the textbook ransomware setup: a single weak, over-privileged credential on an exposed surface, and an attacker who waits for the worst possible moment.

🔗 Connection: Notice that the initial access here is not exotic. It is the same class of failure as Chapter 1's near-miss (a credential problem) and Chapter 19's central warning (an over-privileged account). The lesson the book keeps making: most serious incidents begin with boring, preventable identity and access failures — which is also why most of the program's value compounds. The sophisticated part of this incident is not how the attacker got in; it is how Meridian responded.

The Response

We follow the timeline as the scribe recorded it, annotated with the phase and decision at each step.

Phase: Detect & Analyze — T+0 to T+9 minutes

06:42 (T-9). A backup-integrity monitoring job emailed the on-call engineer: the previous night's snapshot of the loan-origination file server failed verification. Routine-looking. Almost snoozed.

06:49 (T-2). Two more backup-integrity failures arrived. The on-call engineer, now awake, started looking.

06:51 (T+0). Meridian's EDR fired four near-simultaneous alerts: a process named svc_host32.exe (a deliberate look-alike for the legitimate svchost.exe) was executing vssadmin.exe delete shadows /all on four servers. The on-call analyst pulled the alerts together with the backup failures and recognized the pattern instantly, because the team had drilled it: shadow-copy deletion is a ransomware precursor — the MITRE ATT&CK technique "Inhibit System Recovery," the attacker destroying local recovery before encrypting (Chapter 22's mapping discipline paying off in seconds).

Triage, the four questions (06:51–06:53):

• Is it real? Four corroborating EDR alerts + three failed backups + a masquerading process name. Yes. Not a false positive.
• How bad? Recovery-inhibition on production servers — by the SEV matrix this is unambiguously SEV-1.
• How far? Unknown yet — but at least four servers, and the technique implies imminent encryption.
• What now? Declare.

06:53 (T+2). The analyst declared an incident — did not wait, did not "watch it a bit longer" — and paged Priya Nair per the IR plan. This single decision, made by a junior analyst with the authority and the training to make it, compressed the whole incident. A program where the analyst hesitates for an hour "to be sure" loses the morning.

🛡️ Defender's Lens: The detection here was behavioral, not signature-based. There was no known-bad file hash to match — the attacker's tooling was novel. What tripped the alarm was the behavior (a process deleting shadow copies), which is exactly the kind of high-value, behavior-based detection Chapter 22 argues for over brittle indicator-matching. Behavioral detections cost more to build and tune, but they catch the attacker you have never seen before. This one bought Meridian its nine-minute detection time.

Phase: Detect & Analyze + Contain (iterating) — T+2 to T+25 minutes

06:55 (T+4). Priya joined the out-of-band Signal channel and the conference bridge, took the role of incident commander, and opened the war room. Her first three actions were structural, not technical: assign a scribe (a second SOC analyst, who began the timeline that this case study is built from), summon Sam Whitfield (technical lead) and put Dana Okafor (CISO) and Elena Vasquez (GRC) on notice, and state the cadence: "Status every 30 minutes, even if nothing changes. Nobody takes a containment action without telling me first." That last sentence is the lesson the War Story in §24.2 teaches — coordinate before you act, or twelve smart people become a stampede.

06:59 (T+8). Scoping produced the gut-punch. All four servers had been reached from a single account: svc_backup — a service account with domain-admin rights. Sam pulled the privileged-account inventory (Chapter 19) and confirmed it: this account could, in principle, reach the entire domain. By the scoping discipline of §24.3, a compromised domain admin is presumed to have reached everything it could reach until proven otherwise. Scope was, worst case, the whole bank.

The containment decision (07:00, T+9). This is the worked example from §24.4, made for real. Priya walked the four competing concerns aloud, fast, because she had practiced the framework:

• Stop the damage? Active/imminent encryption; the recovery path (backups) is under active attack. Every minute is irreversible. This dominates.
• Preserve evidence? Matters — but not at the price of letting encryption run. Isolate, keep powered; do not power off; do not wait.
• Maintain business operations? The affected servers are already failing; it is Saturday; loan origination can pause. Yields.
• Avoid tipping off the attacker? Moot — they are loud and mid-objective.

Decision: immediate, aggressive containment. Priya authorized three actions in parallel (she could, because the authority was assigned in preparation), each a pre-written runbook:

1. Runbook IR-07 (host isolation): isolate all four servers at the network layer via EDR — powered on, to preserve memory for the forensics of Chapter 25. ~30 seconds per host.
2. Runbook IR-03 (account containment): disable svc_backup, force-revoke its active sessions, and reset/invalidate its Kerberos tickets domain-wide.
3. Block the command-and-control domains the EDR had surfaced, at the DNS resolver and the proxy.

07:11 (T+20). Containment held: no new shadow-copy deletions, no new encryption starts since the isolation. Two of the four servers had begun encrypting before isolation and were partially encrypted; two were caught clean. The blast radius was four servers, not four hundred — because detection was fast, declaration was immediate, and containment was a rehearsed reflex rather than an improvisation.

⚠️ Common Pitfall (avoided): A less-prepared team, finding the encryption on the four servers, might have cleaned those four and declared victory — the classic under-scoping error. Meridian instead treated the domain-admin compromise as a domain-wide event and kept scoping aggressively after containment: hunting for other logons by svc_backup, other masquerading processes, new accounts, new scheduled tasks, and additional persistence across the environment. They found two dormant scheduled tasks the attacker had planted on other hosts for re-entry — which, uncontained, would have brought the attacker back days later. Scoping after containment is what turns "we stopped it" into "we evicted them."

How the scoping actually ran (07:11–08:51). Scoping is not a single query; it is disciplined pivoting, and it is worth seeing the moves Meridian made because they are the same moves any responder makes. The scribe's notes captured the pivots:

PIVOT 1 (indicator): hash of svc_host32.exe -> search every endpoint via EDR
  -> found on the 4 known servers only. Good: tooling did not spread widely.
PIVOT 2 (identity): every authentication by svc_backup in the last 14 days
  -> 4 target servers + DC01 (a domain controller logon at 02:31 three nights ago!)
  -> the DC logon re-scopes the incident UPWARD: the attacker reached a domain controller.
PIVOT 3 (persistence): scheduled tasks / services / run keys created in the window
  -> 2 dormant scheduled tasks on OTHER hosts (re-entry mechanisms) -> removed.
PIVOT 4 (new accounts): accounts created/modified in AD in the window
  -> none. The attacker used svc_backup rather than creating an account (stealthier).
PIVOT 5 (egress): large outbound transfers in the window (DLP + proxy + flow)
  -> one ~400MB transfer to an unknown host; NOT the 2.5M-record claim in the note.

Pivot 2 is the one that mattered most and the one a tired team most easily misses: searching all authentications by the compromised account, not just the ones on the servers that alerted, revealed a domain-controller logon from three nights earlier. That single line re-scoped the incident upward — the attacker had touched a domain controller, which is why the eradication later required the full krbtgt reset rather than a lighter credential rotation. Scope is not what alerted; scope is everywhere the attacker's account, tools, and techniques can be shown to have reached — and you only find it if you go looking.

🔗 Connection: Every one of those five pivots is a capability built in an earlier chapter. The hash and indicator search is Chapter 22's ioc_match. The identity pivot leans on the centralized, normalized authentication logs of Chapter 21 and the access context of Chapters 18–19. The egress analysis is the network-monitoring and exfiltration-detection work of Chapter 10 feeding the SIEM. Scoping is where the whole monitoring program is finally spent — and a program that skimped on any of those layers would have a blind spot exactly where the incident needed sight.

Phase: Detect & Analyze + Communications — T+25 minutes to T+2 hours

07:16 (T+25). The ransom note appeared on the two encrypted servers. Files now carried a .MERIDIANLOCK extension. The note demanded 75 Bitcoin within 72 hours and threatened to publicly leak exfiltrated customer data ("double extortion") if unpaid. The 72-hour countdown was designed to melt the team's judgment. Because the team had rehearsed this exact inject, it did not.

Now several workstreams ran at once — which is the entire reason roles exist:

• Technical (Sam + analysts): Continue scoping. Did data actually leave? Pull egress and proxy logs, check the data-loss-prevention alerts, hunt for large outbound transfers in the incident window. Confirm containment is holding. Begin identifying the initial access vector.
• Legal + insurer (Dana + Elena): Dana called outside counsel and the cyber-insurer's breach hotline — both reachable in minutes because the comms plan listed them with after-hours numbers. Counsel formally directed the investigation (establishing privilege over the work product); the insurer's breach coach joined the bridge and pre-authorized a retained forensics firm if needed.
• Regulatory clock (Elena + counsel): Elena flagged that a determination of a qualifying "computer-security incident" was now plausibly forming, starting the 36-hour federal banking notification clock, and that if customer data was exfiltrated, state breach-notification duties to customers would attach. She and counsel began the determination analysis on a deadline — not waiting for perfect certainty.
• The pay-or-not question (escalated, not decided in the room): Per Meridian's pre-written ransom policy, the default was do not pay; recover from backups. The decision to even consider paying would rise to the CEO with counsel and the insurer, and only if recovery proved impossible. No responder in the war room owned that call, by design.

⚖️ Authorization & Ethics: The temptation, with a 72-hour clock and a leak threat, is to quietly pay and make it disappear before customers or regulators learn anything. Meridian's pre-decided posture resisted this on three grounds: paying certain sanctioned ransomware groups can itself be unlawful; payment guarantees neither a working decryptor nor genuine deletion of stolen data; and the bank's customers have a right to know if their data is at risk. The ethics were settled in a calm boardroom months earlier precisely so they would not have to be re-litigated under duress. That is what a ransom policy is for.

08:51 (T+2h). Scoping found the initial access: svc_backup's weak, reused, breach-exposed password, authenticated through the misconfigured management interface. Critically, the egress analysis found no evidence of bulk customer-data exfiltration — the only sizable outbound transfer in the window was ~400 MB to an unknown host, consistent with the attacker pulling tools in and some staging out, but not the 2.5-million-record exfiltration the note claimed. The team treated the note's claim as unproven, neither confirmed nor dismissed — the right posture, because attackers routinely bluff about exfiltration to pressure payment, yet sometimes tell the truth.

Phase: Eradicate & Recover — T+2 hours to T+11 hours

With containment holding and scope understood, the response moved into eradication and recovery — still iterating with analysis, because forensics (Chapter 25) would keep refining the picture for days.

Eradication (the domain-admin compromise forced heavy actions):

• Rebuild, do not disinfect. The four isolated servers would be wiped and reimaged from known-good media, not "cleaned" — Meridian could not prove it had found every implant on machines an attacker had reached with domain-admin rights.
• Remove persistence. The two dormant scheduled tasks found during scoping were removed; the environment was swept for the same and for rogue accounts, services, and run keys.
• Rotate everything privileged. Because a domain admin was compromised, the team executed the heavy procedure: reset all privileged credentials and perform the krbtgt double-reset — necessary because the attacker may have forged Kerberos golden tickets, and a single reset leaves a window. This is disruptive and is exactly the runbook the previous tabletop had discovered was missing and the team had written the following week. Without that runbook, this step would have been a frightening improvisation.
• Close the door. Remove the management-interface exposure; and the root cause — strip svc_backup of domain-admin rights entirely, moving it to a least-privilege, tiered, JIT model (Chapter 19).

Recovery (deliberate, not rushed):

• Restore from offline, immutable backups. Here the attacker's strategy backfired. The vssadmin delete shadows command destroyed the online volume shadow copies, but Meridian's true backups were offline and immutable — a separate, air-gapped tier the attacker's domain-admin access could not reach or delete. (This control was a direct descendant of Chapter 1's CRITICAL "untested backups" risk, which the program had since remediated into tested, offline backups.) The loan-origination data was restored from the last clean offline backup.
• Validate and stage. Each restored system was validated against the known-good baseline (the harden.py audit logic of Chapter 11) before returning to service; critical functions were restored first.
• Heightened monitoring. The affected systems and all privileged identities were placed under intensified monitoring for a defined two-week window; the incident stayed formally open until that watch passed clean.

The recovery sequencing itself was a decision, not a default. Meridian did not restore everything at once or in the order systems happened to be ready. Sam worked from a pre-agreed restoration priority derived from a business-impact analysis: identity and authentication infrastructure first (because everything depends on it and it had to be trustworthy before anything else came back), then the loan-origination data the attacker had targeted, then secondary systems. Each tier was validated against the known-good baseline and watched before the next came online. The discipline here is counterintuitive under pressure: the fastest safe recovery is often the most deliberate one, because a rushed restore that reintroduces a compromised system or an unpatched vector simply restarts the incident.

17:40 (T+11h). Priya declared the incident contained and recovered: initial vector closed, persistence removed, credentials rotated, systems rebuilt and restored, services back. The incident remained open for the monitoring window and the forensic and regulatory follow-through, but the emergency was over. Loan origination resumed Monday. No ransom was paid. The regulator was notified inside 36 hours of the determination. The customer-data-leak threat was, after forensic confirmation that no bulk exfiltration occurred, handled through careful disclosure consistent with what the evidence showed.

The counterfactual: the same attack at an unprepared bank

It is worth holding Meridian's eleven hours against the incident that would have unfolded at a bank with the same intrusion and no IR program — because the contrast is the whole argument of this chapter, and it is not exaggerated. At the unprepared bank:

• The backup-failure email is snoozed. No one connects three boring backup failures to anything. The EDR alerts (if EDR exists and is tuned for behavior, which at an unprepared shop it often is not) sit unacknowledged in a queue no one watches on a Saturday.
• There is no behavioral detection, so detection is the ransom note itself — discovered Monday morning when employees cannot open their files. MTTD is not nine minutes; it is two days, during which the attacker encrypted not four servers but every server the domain-admin account could reach.
• No one is sure who is in charge. Engineers take uncoordinated containment actions; one powers off a server (destroying memory evidence), another starts re-imaging before anyone has scoped, and the attacker's hidden footholds are never found.
• The backups are online and were deleted along with the shadow copies, because no one had built the offline/immutable tier. Recovery from backup is impossible, which is precisely the situation that forces the ransom question into the only-remaining-option corner.
• The regulatory clock is blown because no one knew it existed or when it started, compounding the breach with a compliance failure and the fines that follow.
• The ransom gets paid — under duress, possibly to a sanctioned entity, with no guarantee — because it is framed as the only path back to operations.

Same attacker. Same initial access. Opposite outcome: a lost weekend versus a lost quarter, a contained four-server incident versus a domain-wide catastrophe, "no ransom paid" versus a six-figure payment to criminals and a regulatory enforcement action on top. The difference is not intelligence or luck. It is preparation, exercised in advance. That is Theme 4 — defense in depth assumes each layer fails — made concrete: prevention failed (the weak credential got in), but the response layer held.

🔄 Check Your Understanding: The attacker's first action was to delete shadow copies — to destroy recovery before encrypting. Meridian recovered anyway. Identify the single control that most directly defeated the attacker's recovery-destruction strategy, and trace it back to the chapter where the underlying risk was first flagged. (Hint: it is not the EDR detection — that limited the damage; this control is what made the damage reversible.)

Phase: Post-Incident Activity — The Blameless Postmortem (T+9 days)

Nine days later, with the monitoring window clean and the forensic report (Chapter 25) in hand, Priya facilitated the blameless postmortem. Ground rule, stated first: we are here to fix systems, not to blame people. The whole svc_backup problem made this tempting to violate — someone, years ago, granted that account domain admin — but the blameless frame held: the question was not "who did this?" but "what about our processes let an over-privileged service account persist for years, and what makes that impossible going forward?"

The structured output:

1. Timeline. Reconstructed from logs and the scribe's record (the timeline above). Facts first.
2. What went well. The analyst declared in two minutes. Containment was a rehearsed reflex. The out-of-band channel worked. The offline backups saved the bank. Legal and the insurer were reachable. Name the wins so you keep them.
3. What went poorly / got lucky. svc_backup should never have had domain admin. The management-interface exposure existed. Some scoping was slower than ideal because not everyone knew the saved hunting queries.
4. Root-cause analysis (five whys). Why did the attacker get domain-wide reach? Because a service account had domain admin. Why? Because it was granted years ago and never reviewed. Why? Because there was no recurring privileged-access review for service accounts. Why? Because the PAM program (Chapter 19) had focused on human admins first. Root cause: a gap in the access-review process — a system problem, not a person.
5. Action items (owned, deadlined, tracked):

📟 War Story: The action items are the entire point, and Meridian had learned that the hard way once before. An earlier postmortem at the bank had produced a beautiful report and no owned action items — and three months later a nearly identical near-miss recurred because nothing had actually changed. After that, the rule became ironclad: every item has a name and a date, and the list is reviewed in the SOC's weekly meeting until every item is closed. A lesson is not "learned" when it is written; it is learned when the system has changed so the failure cannot recur the same way. (This discipline, and the MTTD/MTTR tracking that goes with it, is developed fully in Chapter 36.)

The honest metrics, captured for trending (Chapter 36): MTTD ≈ 9 minutes (backup failure at T-9 to the EDR behavioral alert at T+0, with declaration at T+2). Time to containment ≈ 20 minutes. Time to recovery ≈ 11 hours. These numbers were not luck. They were the cashing-in of a year of preparation — and the postmortem's job was to make the next incident's numbers even better.

Discussion Questions

1. The junior on-call analyst declared a SEV-1 in two minutes. What in Meridian's preparation made it safe and correct for a junior to make that call, rather than escalating and waiting? What would have happened to the incident if the bank's culture punished "crying wolf"?
2. Priya's first three actions as incident commander were structural (assign scribe, summon team, set cadence and the "tell me first" rule), not technical. Argue for why a commander who instead dove straight into the keyboard would have produced a worse outcome.
3. The ransom note claimed 2.5 million records were exfiltrated; the evidence showed only a 400 MB transfer. The team treated the claim as "unproven — neither confirmed nor dismissed." Defend that posture against two alternatives: (a) believe it and notify all customers immediately; (b) dismiss it and tell no one. What are the costs of each error?
4. The single control that most made the damage reversible was the offline, immutable backup — descended from a risk first flagged in Chapter 1. Trace two or three other controls in this incident back to the earlier chapters that built them. What does that tell you about how a security program's value compounds?
5. The blameless postmortem resisted blaming whoever granted svc_backup domain admin years ago. Some would argue that is letting someone off the hook. Make the empirical case that blamelessness produces better security outcomes than blame — and state where accountability still legitimately lives.

Your Turn

Take the ransomware tabletop scenario from Exercise 20 (or this case) and produce two artifacts as if you were on Meridian's team. First, write the incident timeline as a scribe would, from first alert to "contained," with at least eight time-stamped entries, each tagged with the lifecycle phase. Second, write the blameless postmortem's action-item table: at least four items, each with a specific description, a named owner, and a due date, traceable to a root cause you identify (not a person). Then write one paragraph: which earlier chapter's control, had it been stronger, would most have changed this incident's outcome — and why?

Key Takeaways

• Detection speed compounds everything. A behavioral detection (shadow-copy deletion) and an analyst empowered to declare in two minutes turned a potential domain-wide catastrophe into a four-server incident. The blast radius is set in the first ten minutes.
• The incident commander's first job is structure, not keystrokes: assign a scribe, summon the team, set the cadence, and require "tell me before you act." Coordination prevents the stampede.
• A compromised domain-admin account is a domain-wide incident until proven otherwise; scope aggressively, especially after containing, to find the persistence the attacker left for re-entry.
• The incident type sets the containment posture. Active ransomware → immediate aggressive containment (stop the irreversible damage), isolate-but-don't-destroy to preserve evidence; tipping off is moot.
• Eradicating a domain-admin compromise is heavy (rebuild from known-good, rotate all privileged creds, krbtgt double-reset, close the vector) — which is why you write the runbooks before you need them.
• Offline, immutable, tested backups defeat ransomware's recovery-destruction strategy. This single control — a direct descendant of Chapter 1's "untested backups" risk — is what made the damage reversible without paying.
• Communications and the regulatory clock run in parallel with the technical response, not after it; legal, the insurer, and the 36-hour determination must be at the table early. The ransom decision is pre-decided by policy and escalated, never improvised in the war room.
• The blameless postmortem turns a bad day into permanent improvement — but only through owned, deadlined, tracked action items that change the system. A written report that changes nothing guarantees the incident recurs.