Chapter 25: Digital Forensics for Defenders: Preserving Evidence, Timeline Analysis, and Post-Breach Investigation

"In God we trust. All others must bring data." — W. Edwards Deming

Overview

The ransomware at Meridian Regional Bank was contained by Wednesday afternoon. The encrypted file server was off the network, the compromised account was disabled, and the core-banking system — segmented off, never touched — had kept the branches open the whole time. From the incident-response seat (Chapter 24), the fire was out. And that is exactly the moment a different and harder set of questions arrives, the ones a CISO cannot answer with "we contained it." How did the attacker get in? When did they actually get in — was it Tuesday, or had they been resident for three weeks? What did they touch before they detonated the ransomware? Did they take any customer data on the way out, which would convert a disruptive outage into a reportable breach under the GLBA Safeguards Rule and a dozen state laws? And — the question every regulator, every cyber-insurer, and possibly every plaintiff's attorney will eventually ask — can you prove it?

Those questions are the domain of digital forensics, and they are not answered by guessing. They are answered by going back to the machines the attacker touched and reading the evidence those machines recorded, whether the attacker meant to leave it or not. A Windows system narrates its own history in a dozen places the average administrator never looks: a registry hive that remembers which USB device was plugged in and when, a master file table that timestamps the creation of the attacker's tools, event logs that record the exact moment a service was installed, prefetch files that prove a program ran. A defender who knows how to preserve that evidence soundly, read it correctly, and assemble it into a single coherent timeline can reconstruct an intrusion almost minute by minute — and can stand behind that reconstruction when it counts.

This chapter is where incident response becomes investigation. It is the most exacting work in security operations, because here the order of your actions and the integrity of your evidence matter as much as your conclusions. Rebooting the wrong machine, copying a file the wrong way, or skipping a hash can destroy the very proof you were sent to collect — and unlike most security work, you usually get exactly one chance. We will preserve the Meridian evidence properly, image a disk and capture memory without altering them, analyze the artifacts that reveal what the attacker did, build the timeline that ties it all together, scope how far the compromise reached, and find the root cause — all while keeping the chain of custody intact so the findings survive scrutiny.

In this chapter, you will learn to:

• Explain what digital forensics (DFIR) contributes to defense, and how it differs from — and depends on — incident response.
• Apply the order of volatility to decide what to collect first, and acquire disk and memory images soundly with a write blocker and cryptographic hashing.
• Maintain an unbroken chain of custody and understand the legal soundness that makes evidence admissible and findings defensible.
• Read the forensic artifacts that matter — Windows registry, the $MFT, event logs; Linux logs, shell history, and filesystem timestamps — to reconstruct attacker activity.
• Build a unified timeline across sources, scope the breach by pivoting on indicators, perform root-cause analysis, and detect the signs of anti-forensics.

Learning Paths

This is an advanced Security Operations chapter; it is the natural home of the SOC and IR analyst, and it builds directly on the incident-response lifecycle of Chapter 24.

🛡️ SOC Analyst: This entire chapter is yours. §25.2 (acquisition order) and §25.4 (artifacts) are the hands-on core you will use on real cases; §25.5 (timeline) is the skill that separates an analyst who thinks they know what happened from one who can show it. Internalize the order of volatility (§25.2) before anything else — getting it wrong is the most common, most irreversible mistake in the field. 🏗️ Security Engineer: Read §25.1 and §25.6 for forensic readiness — the architectural decisions (centralized logging, retention, host visibility, immutable storage) that determine whether an investigation is possible. You build the conditions that make forensics work or fail. 📋 GRC: Focus on §25.3 (chain of custody, legal soundness) and the breach-scoping conclusions in §25.6 that feed regulatory notification decisions. Forensics produces the facts your breach-notification and legal-hold obligations depend on. 📜 Certification Prep: Order of volatility, chain of custody, write blockers, and the acquisition process appear on both Security+ and CISSP. The key-takeaways.md file maps each to its exam domain.

25.1 Forensics in defense: what DFIR is, and what it is not

Picture two ways the Meridian investigation could go. In the first, the team is relieved the ransomware is contained, they rebuild the encrypted file server from a clean image to get it back into production fast, and they move on. Three weeks later a cyber-insurance adjuster asks for proof of whether customer data was accessed before the encryption. The only machine that held the answer has been wiped and rebuilt. There is no answer. The bank must now assume the worst, notify regulators and millions of customers about a possible data breach it cannot actually confirm, and absorb the cost and reputational damage of a maximal disclosure — all because the evidence was destroyed in the rush to recover.

In the second version, before that server is rebuilt, someone takes a forensic image of its disk and a capture of its memory, hashes both, logs the handoff, and then recovery proceeds on a copy. When the adjuster asks the question, the team analyzes the preserved image and finds that the attacker's access was limited to the file server and never reached the database holding customer records — and they can demonstrate it. The notification is narrow, accurate, and defensible. The difference between these two outcomes is not luck or skill at containment. It is whether anyone did forensics, and whether they did it in the right order, before the evidence was gone.

Digital forensics (often bundled with incident response into the single discipline DFIR, for "digital forensics and incident response") is the practice of identifying, preserving, acquiring, analyzing, and reporting on digital evidence in a manner that is methodical, repeatable, and defensible — typically to reconstruct what happened during a security incident and to support decisions, including legal ones. The four verbs are the whole job: preserve the evidence before it changes, acquire a sound copy, analyze that copy to answer questions, and report in a way that holds up.

Notice how forensics relates to the incident response you learned in Chapter 24. IR is about stopping the bleeding — detect, contain, eradicate, recover. Forensics is about understanding the wound — what happened, when, how, and how far. They overlap and they sometimes pull in opposite directions: containment may want to pull a machine off the network or shut it down now, while forensics wants that machine left in exactly the state the attacker left it. A mature team holds both goals at once, and the order-of-volatility discipline in §25.2 is precisely the protocol that lets you preserve evidence while you respond, rather than choosing one or the other.

🚪 Threshold Concept: In ordinary IT, the goal of touching a sick machine is to fix it — reboot it, clear the cache, reinstall, get it healthy. In forensics, the goal is the opposite: to change it as little as humanly possible, because every action you take overwrites some of the evidence you were sent to collect. The forensic mindset is to treat a compromised system as a crime scene, not a broken appliance. Once you adopt it, you will instinctively pause before every click and ask: "what evidence does this action destroy?" That pause is the whole discipline.

Why does a defender need this? Three reasons, each of which the rest of the chapter develops:

• Root cause. You cannot prevent a recurrence of an intrusion whose root cause you never established. If you rebuild the file server but never learn the attacker entered through a stolen VPN credential with no multi-factor authentication, you will be breached the same way next month. Forensics turns "we got hit" into "we got hit because of this specific gap," which is the only kind of finding that drives a real fix.
• Scope. Containment tells you where the fire was; forensics tells you how far it actually spread. The attacker who detonated ransomware on one server may have touched ten others first. Scoping (§25.6) is what lets you contain the whole incident rather than the part you happened to notice.
• Proof. Breach notification, insurance claims, regulatory examinations, and litigation all turn on demonstrable facts about what was and was not accessed. "We think it was fine" is not a defensible position in front of an FFIEC examiner or a court. Forensics, done with chain of custody intact, produces facts that survive that scrutiny.

🛡️ Defender's Lens: The attacker's goal during and after an intrusion includes destroying your ability to investigate — clearing logs, deleting tools, tampering with timestamps (the anti-forensics of §25.6). From the blue-team seat, this means two things. First, the absence of evidence is itself evidence: a Windows Security log that suddenly has a gap, or an event ID 1102 ("audit log cleared"), is a loud signal that something happened there. Second, forensic readiness (§25.1's architectural cousin) is a race against deletion — evidence shipped off the host to a central, tamper-resistant store before the attacker arrives is evidence they cannot erase. We design for that race in the Project Checkpoint.

25.2 Order of volatility and sound acquisition

The single most important rule in digital forensics is also the easiest to violate by accident, and it follows from one physical fact: digital evidence is fragile, and some of it is far more fragile than the rest. The contents of a running computer's memory vanish the instant you cut the power. Network connections close. Temporary files get cleaned up. Meanwhile, data written to a disk is comparatively durable, and data archived to a backup tape or a write-once log store is more durable still. If you collect evidence in the wrong order — say, you pull the power to "freeze" a machine — you have just destroyed everything that lived only in memory, which is often where the most valuable evidence of an active intrusion sits (decrypted malware, network connections, injected code, credentials in cleartext).

The discipline that prevents this is the order of volatility: the principle that you must collect digital evidence from the most volatile (most quickly lost) source to the least volatile, so that the act of collecting durable evidence does not destroy fragile evidence first. This ordering is enshrined in the foundational standard for this work, the IETF's RFC 3227 ("Guidelines for Evidence Collection and Archiving"), and echoed in NIST SP 800-86 ("Guide to Integrating Forensic Techniques into Incident Response"). Here it is as the defender applies it:

                ORDER OF VOLATILITY — collect from TOP to BOTTOM
                (most fragile / shortest-lived first)

   MORE VOLATILE   ┌─────────────────────────────────────────────┐
   (seconds)       │ 1. CPU registers, cache                     │  gone on power loss
        ▲          │ 2. RAM: running processes, network          │  gone on power loss /
        │          │    connections, open files, injected code   │     reboot
        │          │ 3. Network state: ARP cache, routing,        │  changes continuously
        │          │    live connections, listening sockets      │
        │          │ 4. Running system / kernel state, temp data │
        │          ├─────────────────────────────────────────────┤
        │          │ 5. DISK: filesystem, $MFT, registry, logs,  │  survives reboot;
        │          │    user files, slack/unallocated space      │     can be wiped
        │          │ 6. Remote logging / SIEM (off-host copies)  │  survives the host
        ▼          │ 7. Physical config, network topology        │  stable
   LESS VOLATILE   │ 8. Backups, archival / write-once storage   │  most durable
   (months+)       └─────────────────────────────────────────────┘

   RULE: capture memory (2) BEFORE imaging disk (5);
         NEVER pull power on a live system you have not memory-captured.

Figure 25.1 — The order of volatility. The arrow of fragility runs top to bottom; collect in that order. The classic, irreversible mistake is jumping to "pull the power and image the disk" (item 5) while skipping the memory capture (item 2), permanently destroying the most valuable evidence of an active intrusion.

This is why the very first forensic decision at a live compromised machine is live capture versus power-off. If the machine is running and you have authorization and tooling, you capture its memory first — using a small, trusted tool that reads RAM to an external drive — and only then decide whether to leave it running, isolate it on the network, or shut it down for disk imaging. Pulling the plug is sometimes the right call (for certain ransomware that is actively encrypting, or to stop active exfiltration when you cannot memory-capture in time), but it is a deliberate trade, made knowing you are sacrificing all of items 1–4 to preserve the disk in a frozen state. It is never the thoughtless default.

Acquiring a disk image soundly

When you do image a disk, the goal is a disk image: a bit-for-bit, sector-by-sector copy of the entire storage device — not a file copy, but a complete duplicate including deleted files, slack space, and unallocated areas where evidence frequently hides. (Copying files misses everything the filesystem no longer references; copying bits misses nothing.) Two non-negotiable requirements make a disk image forensically sound:

1. Don't change the original. You acquire through a write blocker — a hardware device or a strictly enforced software mode that sits between the investigator's system and the evidence drive, permitting read commands but physically or logically blocking every write command. Without it, the mere act of connecting an evidence disk to a Windows analysis machine can alter the disk (Windows likes to write recovery files, update timestamps, mount the volume). The write blocker guarantees the original is read, never written — which is the foundation of everything you will later claim about it.

2. Prove you didn't change it. Before and after imaging, you compute a cryptographic hash of the source (building on the hashing you learned in Chapter 4). A hash function reduces an arbitrary amount of data to a fixed-size fingerprint such that any change to the input — even a single bit — produces a wildly different output. You hash the original drive, hash the image you created, and confirm the two hashes are identical. That match is your mathematical proof that the image is a perfect copy and that nothing was altered in the process. You record those hashes; from that moment on, anyone can re-hash the image and confirm it has not been tampered with.

⚠️ Common Pitfall: Treating MD5 as "broken, therefore unusable" — or, conversely, trusting it for integrity in an adversarial setting. MD5 and SHA-1 are cryptographically broken for collision resistance, meaning a motivated adversary can craft two different inputs with the same hash. For verifying that you did not accidentally alter an image, an MD5 match is still a useful integrity check and is still widely seen in tooling and case files. But when the integrity of evidence might be contested by an adversary, use SHA-256, which has no practical collision attack. The defensible modern practice is to record a SHA-256 hash (many tools record both). Knowing why you chose the algorithm is itself part of a defensible process.

Worked example: verifying an evidence hash

Here is the integrity check at the center of every acquisition, traced by hand. The hashes below are illustrative placeholders — real SHA-256 digests are 64 hex characters; we truncate and label them so the logic is clear without pretending to compute a real digest.

ACQUISITION LOG — Meridian file server FS-01 (constructed)
-----------------------------------------------------------
Source device : /dev/sdb  (500 GB, SATA)  via hardware write blocker
Imaging tool  : dd-style bitstream copy -> FS-01-disk.img

Step 1  Hash the SOURCE before imaging:
        SHA-256(/dev/sdb)      = a1b2c3...EXAMPLE   (illustrative)

Step 2  Create the image (read-only through the write blocker).

Step 3  Hash the IMAGE after imaging:
        SHA-256(FS-01-disk.img)= a1b2c3...EXAMPLE   (illustrative)

Step 4  Compare:
        source hash  == image hash  ->  MATCH  ->  image is SOUND  ✓

   If Step 4 had NOT matched, the image is unusable: re-image, and
   investigate whether the write blocker failed or the drive is failing.

The logic, not the digits, is the lesson: a match certifies a sound copy you can analyze freely without ever touching the original again; a mismatch invalidates the image. We encode exactly this evidence_hash step in the Project Checkpoint's forensics.py.

🧩 Try It in the Lab: On a file you own, compute a SHA-256 hash with your platform's tool (sha256sum file on Linux/macOS, Get-FileHash file in PowerShell). Now change a single character in the file and hash it again. Watch the entire digest change. Then copy the file and hash the copy — identical to the original. You have just performed, in miniature, the integrity check that underpins every forensic acquisition. (Do this only on your own files, in your own environment — the authorization rule from Chapter 1 always holds.)

🔄 Check Your Understanding: 1. You arrive at a running, compromised workstation. List, in order, the first two categories of evidence you should collect and explain why that order matters. 2. Why is a bit-for-bit disk image forensically superior to copying the suspicious files off the machine? 3. What two things does hashing the source and the image prove?

Answers

1. (1) Volatile memory (RAM) — running processes, network connections, injected code — because it is destroyed by reboot or power-off; (2) then the disk, which survives. Capturing memory first preserves the most fragile evidence before any action that could lose it. 2. A file copy misses deleted files, slack space, and unallocated areas where evidence often hides, and it can alter timestamps; a bit-for-bit image captures the entire device exactly, including those regions. 3. That the image is a perfect copy of the source (the hashes match), and — by re-hashing later — that the image has not been altered since acquisition (integrity over time).

25.3 Chain of custody and legal soundness

A perfect, hash-verified disk image is forensically worthless if you cannot account for where it has been every moment since you created it. Evidence that might have been altered, accessed by unknown people, or swapped is evidence a regulator can dismiss and a court can exclude. The protocol that prevents this is the chain of custody: the documented, unbroken record of who collected each piece of evidence, when, from where, and everyone who has handled, accessed, transferred, or stored it since — establishing that the evidence presented is the same evidence collected and that it was not tampered with along the way.

Chain of custody is mostly paperwork and discipline, not technology, which is exactly why teams under pressure neglect it and exactly why that neglect is so damaging. The unglamorous truth is that the most sophisticated forensic analysis in the world can be thrown out because someone left an evidence drive on a desk overnight with no record of who had access to the room. A defensible chain of custody answers, for every item, an auditor's questions:

CHAIN-OF-CUSTODY RECORD — Evidence item E-2025-0417-01 (constructed)
---------------------------------------------------------------------
Item        : HDD, Seagate, S/N ABC123EXAMPLE, from server FS-01
Description : 500 GB system disk, Meridian file server, ransomware case
Acquired by : T. Brandt (SOC)      Date/Time : 2025-04-17 15:20 UTC
Location    : Meridian DC, rack B12, removed under camera
Hash (SHA-256): a1b2c3...EXAMPLE   Write blocker: Tableau T35u (S/N ...)

  HANDOFF LOG
  | When (UTC)        | From       | To          | Reason            | Sig |
  |-------------------|------------|-------------|-------------------|-----|
  | 2025-04-17 15:20  | (scene)    | T. Brandt   | acquisition       |  ✓  |
  | 2025-04-17 16:05  | T. Brandt  | Evidence safe | secure storage  |  ✓  |
  | 2025-04-18 09:10  | Evidence safe | P. Nair  | analysis (image)  |  ✓  |
  | 2025-04-18 17:30  | P. Nair    | Evidence safe | return            |  ✓  |

  Every access logged. Original never analyzed directly — work on the
  verified IMAGE; the original disk stays sealed in the safe.

Figure 25.2 — A chain-of-custody record. The handoff log is the heart of it: every transfer is dated, signed, and reasoned. Note the cardinal rule — analysis is performed on the verified image, never on the original, which stays sealed.

Several principles make evidence legally sound — admissible and persuasive — and they are good practice even when you never expect a courtroom, because the same rigor is what makes findings trustworthy for a regulator, an insurer, or your own board:

• Authorization and legal authority. You must have the right to collect the evidence in the first place — an employer's authority over its own systems, a documented incident-response mandate, and where personal or third-party data is involved, an awareness of privacy law. A legal hold (a directive to preserve all potentially relevant data once litigation is anticipated) may attach early; once it does, deleting relevant data — even routine log rotation — can become "spoliation," a serious legal problem.
• Integrity. The hashing and write-blocking of §25.2, plus the custody record, together demonstrate the evidence is unaltered.
• Repeatability and documentation. Another competent examiner, given your notes and the image, should be able to reproduce your findings. This is why forensic analysts keep contemporaneous notes of every command and action — the analysis must be a transparent, repeatable process, not a magic trick.
• Competence and tooling. Findings are stronger when produced with validated tools and methods by someone qualified to interpret them. Courts have well-known standards for the admissibility of technical evidence and expert testimony; you do not need to be a lawyer, but you must know when to involve one.

⚖️ Authorization & Ethics: Forensics sits closer to the law than almost any other security work, and the boundaries are real. Collecting evidence from systems you are authorized to investigate (your employer's) is routine; reaching onto a system you do not control — even to "follow the attacker" — can itself be a crime, and can taint your evidence. The moment an investigation might involve law enforcement, litigation, or the personal data of employees or customers, the right move is to pause and involve legal counsel and, where relevant, HR, and to follow a defined process rather than improvising. A defender's instinct to chase the attacker must yield to the discipline of staying within authority. When in doubt, preserve (image and hash) and ask before you act further.

🔗 Connection: The decision that ultimately rests on this work — do we have to notify? — lives in governance and incident handling. Forensic scoping (§25.6) produces the factual answer ("customer data was / was not accessed"), and that answer drives obligations under regimes like GLBA, PCI-DSS, and state breach-notification laws. Chapter 24 framed incident response as a lifecycle whose final phases (recovery, lessons learned) assume you actually know what happened; this chapter is how you earn that knowledge defensibly.

🔄 Check Your Understanding: 1. Why is analysis always performed on the image rather than the original evidence drive? 2. A team has a perfect, hash-verified image but no record of where the original drive was for two days. What is the risk, and which principle was violated?

Answers

1. To preserve the original in its unaltered, sealed state as the authoritative source; all analysis (which can inadvertently change data) happens on a verified copy, and the original can be re-hashed at any time to prove it is untouched. 2. The two-day gap breaks the chain of custody: an opposing party can argue the drive could have been altered or swapped, which can render the evidence inadmissible or unpersuasive regardless of how good the image is. The chain-of-custody/continuous-accountability principle was violated.

25.4 Key artifacts: reading what the system recorded

With a sound, custody-tracked image in hand, the investigation becomes a matter of reading the right things. Every operating system leaves a trail of forensic artifacts — data left behind by normal system and application activity that, though not created for investigators, records evidence of what users and programs did: which programs ran, when files were created or modified, which accounts logged in, which devices were attached. Knowing where these artifacts live, and what each one proves, is the core craft of forensic analysis. We will tour the highest-value ones on Windows (Meridian's endpoints and file servers run Windows) and Linux (its web and application tier).

Windows artifacts

Windows is unusually talkative; an experienced examiner can reconstruct remarkable detail from a single disk image.

• The registry. The Windows registry is a hierarchical database of configuration that, as a side effect, records a great deal of history. Specific keys answer specific forensic questions: which USB storage devices were ever connected (and when) live under USBSTOR; recently run programs and user activity surface in keys like the UserAssist and ShimCache/AppCompatCache structures; persistence mechanisms (programs set to auto-run) live under the Run and RunOnce keys — a classic place to find an attacker's foothold. Registry hives are files on disk (SYSTEM, SOFTWARE, NTUSER.DAT), so they are captured in your image and analyzed offline.
• The $MFT` (Master File Table).** On an NTFS volume, the **`$MFT is the index of every file and directory, holding for each one a set of timestamps — created, modified, accessed, and MFT-entry-modified (collectively the "MACB" times). The $MFT` is forensic gold: it can reveal files that have been *deleted* (their entries may persist), and the timestamps let you place file creation in time — for instance, the exact moment the attacker's malicious executable first appeared on disk. (Timestamps can be tampered with — see anti-forensics in §25.6 — but the `$MFT records two sets of times, and discrepancies between them are themselves a red flag.)
• Windows Event Logs. The event logs are the system's own narration in structured records, each with an Event ID. A handful are workhorses of intrusion analysis: 4624 (successful logon) and 4625 (failed logon) with their logon type (type 3 = network, type 10 = remote interactive/RDP) reveal how and from where accounts authenticated; 4688 records process creation (what ran); 7045 records a new service installed (a favorite attacker persistence and lateral-movement technique); and 1102 records that the Security log itself was cleared — an unmistakable anti-forensic signal. These live in .evtx files in your image.
• Prefetch, Amcache, and execution evidence. Windows keeps Prefetch files to speed up program launches; as a side effect, a .pf file is strong evidence that a given executable ran, and records how many times and when last. Amcache and ShimCache similarly retain evidence of program execution and presence. These answer the crucial question artifacts like the registry only hint at: not "was this file here," but "was this file executed."

Linux and Unix artifacts

Linux is quieter but no less informative once you know where to look (this builds on the host hardening of Chapter 11, where the same logs serve detection):

• System and authentication logs. /var/log/auth.log (Debian/Ubuntu) or /var/log/secure (RHEL/CentOS) record authentications, sudo use, and SSH sessions — the Linux equivalent of the Windows logon events. /var/log/syslog and the systemd journal (journalctl) carry general system events. These are where you see the attacker's logins, privilege escalations, and service manipulations.
• Shell history and timestamps. A user's ~/.bash_history can record the literal commands an attacker typed (when not disabled — its absence or truncation is itself suspicious). Filesystem timestamps (atime/mtime/ctime, read with stat) place file activity in time, the Linux analog of $MFT times.
• Cron, services, and persistence. Attacker persistence on Linux hides in crontab entries, new or modified systemd units, and startup scripts — the places you check to answer "how did they arrange to survive a reboot?"

🛡️ Defender's Lens: Notice that almost every artifact above does double duty: the same Windows event ID 7045 (service installed) or Linux auth.log sudo entry that you read after an incident in forensics is also a detection you should be alerting on during one (Chapters 21–22). Forensics and detection are the same data viewed at different times — which is the strongest possible argument for collecting and centralizing this telemetry before you need it. The artifact you forgot to log is the question you will not be able to answer.

Worked example: an artifact tells a story

Suppose the Meridian investigation pulls these records from the file server's image. Read them the way an analyst does — each line is a fact; together they are a narrative:

ARTIFACT EXCERPTS — file server FS-01 (constructed, times UTC)

  [Security.evtx]
  2025-04-15 02:14:07  EventID 4624  Logon  type=3 (network)
                       Account=svc_backup  Source=10.20.0.55
  2025-04-15 02:15:31  EventID 7045  Service installed
                       Name="SysHelper"  ImagePath=C:\Windows\Temp\sh.exe

  [$MFT]
  2025-04-15 02:15:02  C:\Windows\Temp\sh.exe   CREATED   (Std-Info time)
                       (FileName-attribute CREATED time = 2019-... )  <-- mismatch!

  [Prefetch]
  SH.EXE-9F3A1B2C.pf   run count=3   last run=2025-04-15 02:16:44

  [Security.evtx]
  2025-04-15 03:02:10  EventID 1102  "The audit log was cleared"
                       Account=svc_backup

The story assembles itself: at 02:14 the svc_backup service account logged on over the network from an internal host — already odd for a backup account. One minute later a service named SysHelper was installed, pointing at sh.exe in a temp directory (services do not normally live in C:\Windows\Temp). The $MFT shows sh.exe was created at 02:15 — but its two timestamp sets disagree, one claiming a 2019 date, a tell-tale sign of timestomping (anti-forensics, §25.6). Prefetch proves sh.exe actually ran, three times. And at 03:02 the same account cleared the audit log — a deliberate attempt to erase the very trail we just read. Each artifact alone is a hint; merged into a timeline, they are a confession. That merge is the next section.

🔄 Check Your Understanding: 1. Which Windows artifact best answers "did this executable actually run (not just exist on disk)?" 2. You find Windows Event ID 1102 and a ~/.bash_history that is empty on an account the attacker used. What do both suggest, and why are they still useful to you?

Answers

1. Prefetch (a .pf file), which is created as a side effect of execution and records run count and last-run time; Amcache/ShimCache also retain execution evidence. (The registry or $MFT may show the file existed, but Prefetch shows it ran.) 2. Both are anti-forensic signs — 1102 is the Security log being cleared, an empty/truncated history suggests deliberate clearing. They are still useful because the absence or clearing of evidence is itself evidence: it marks where and when the attacker acted and confirms intent to hide, narrowing your investigation.

25.5 Building the timeline

Individual artifacts are facts in isolation; an investigation needs a narrative, and the instrument that produces one is the timeline. Timeline analysis is the technique of merging timestamped events from many artifact sources — event logs, $MFT MACB times, registry key write times, log files, network records — into a single chronological sequence, so that the activity of an intrusion can be reconstructed and understood in order. A timeline is what turns "we found a malicious service and a cleared log and a weird logon" into "here is exactly what the attacker did, minute by minute, from initial access to impact."

The power of a timeline comes from correlation across sources. No single artifact tells the whole story, and an attacker may have tampered with one source (clearing a log) but missed another (the $MFT timestamp, the off-host SIEM record). When you lay all sources on one time axis, three things emerge that no single source reveals:

• Sequence and causation. A logon at 02:14, a service install at 02:15, a file creation at 02:15, an execution at 02:16 — placed in order, these stop being coincidences and become a chain of attacker actions.
• The initial access point. Walking the timeline backward from the impact (the ransomware) to its earliest related event leads you to patient zero — the first foothold — which is the root cause you are hunting.
• Gaps and tampering. When one source goes silent (the cleared Security log at 03:02) but others keep recording, the timeline makes the gap visible and lets the surviving sources fill it in. Tampering that defeats one artifact rarely defeats all of them at once.

The mechanics of building a "super-timeline" by hand are tedious, which is why tools exist to do it; conceptually, though, the operation is simple — normalize every source to a common timestamp format and time zone, then merge and sort. The two traps that ruin timelines are both about time itself: time-zone confusion (one log in local time, another in UTC, silently off by hours) and clock skew (a machine whose clock was wrong). The discipline is to convert everything to UTC, record the source of each event, and note any known clock offsets — exactly the normalization our merge_timeline function performs.

Worked example: merging two sources into one timeline

Take two of the Meridian sources from §25.4 — the Windows Security log and the $MFT — plus a VPN log, each in its own format, and merge them. The operation that matters is normalize, then sort by time:

SOURCE A (Security.evtx, UTC)        SOURCE B ($MFT, UTC)        SOURCE C (VPN log, UTC)
  02:14:07  logon svc_backup           02:15:02  sh.exe created     02:02:50  VPN connect
  02:15:31  service "SysHelper"                                              user=jlopez
  03:02:10  audit log cleared                                               src=203.0.113.9

                         │  merge_timeline()  → normalize + sort by timestamp
                         ▼

  UNIFIED TIMELINE (sorted, source-tagged)
  ┌──────────┬─────────┬───────────────────────────────────────────────┐
  │ Time UTC │ Source  │ Event                                          │
  ├──────────┼─────────┼───────────────────────────────────────────────┤
  │ 02:02:50 │ VPN     │ VPN connect — user=jlopez from 203.0.113.9     │ ← initial access
  │ 02:14:07 │ WinEvt  │ Network logon — account=svc_backup             │   (pivot to svc acct)
  │ 02:15:02 │ MFT     │ File created — C:\Windows\Temp\sh.exe          │   (tool staged)
  │ 02:15:31 │ WinEvt  │ Service installed — "SysHelper" -> sh.exe      │   (persistence)
  │ 03:02:10 │ WinEvt  │ Audit log cleared (account=svc_backup)        │   (anti-forensics)
  └──────────┴─────────┴───────────────────────────────────────────────┘

Read top to bottom, the merged timeline is the intrusion narrative: a VPN login by jlopez at 02:02 (the entry), a pivot to the svc_backup service account at 02:14, the malicious tool staged at 02:15, persistence installed seconds later, and an attempt to cover tracks at 03:02. The VPN row — which lives in a different system entirely from the Windows artifacts — is the one that reveals the true initial access, and it is only by merging sources that it lands in sequence with the rest. This merge is the heart of the Project Checkpoint's merge_timeline.

⚠️ Common Pitfall: Trusting a single source — especially one the attacker could reach. A team that built its timeline only from the Windows Security log would have hit the 03:02 "audit log cleared" event and gone blind, possibly concluding the activity started later than it did. The off-host VPN log and the on-disk $MFT (which the attacker did not, or could not, fully scrub) are what rescue the timeline. Always corroborate across independent sources, and weight off-host, append-only sources most heavily, because they are the hardest for an attacker to tamper with.

🔄 Check Your Understanding: 1. What two normalization steps must you perform before merging events from different sources, and what goes wrong if you skip them? 2. An attacker cleared the Windows Security log at 03:02. Explain how a multi-source timeline can still reconstruct activity that occurred before 03:02.

Answers

1. Convert all timestamps to a common time zone (UTC) and account for any clock skew, recording each event's source. Skipping this causes events to sort in the wrong order — time-zone differences can shift events by hours, and a skewed clock can make a later event appear earlier — corrupting the sequence and any causal conclusions. 2. Because other sources independent of the Security log — the $MFT file-creation times, registry write times, and especially the off-host VPN/SIEM records — were not cleared; merged onto one axis they fill the gap and reveal the pre-03:02 activity the log clearing was meant to hide.

25.6 Scoping the breach, root cause, and anti-forensics

A timeline tells you what happened on the machines you looked at. The final and most consequential forensic question is broader: how far did this go? Indicator scoping (often just "scoping") is the process of taking the indicators you have found — a malicious file hash, an attacker IP, a compromised account, a registry key, a service name — and searching for those same indicators across the entire environment, to determine the full extent of the compromise: every host touched, every account used, every channel of data movement. Containment (Chapter 24) addresses the part of the incident you can see; scoping is how you discover the part you cannot yet see, so that you contain and eradicate the whole thing rather than playing whack-a-mole while the attacker persists elsewhere.

The mechanism of scoping is the pivot: each confirmed indicator becomes a search term that reveals new indicators, which you then pivot on in turn.

                 SCOPING BY PIVOTING ON INDICATORS

   START: one confirmed fact (from the FS-01 timeline)
   ┌─────────────────────────────┐
   │ malicious file: sh.exe       │
   │ SHA-256: a1b2c3...EXAMPLE    │
   └──────────────┬──────────────┘
                  │ search this hash fleet-wide (EDR/SIEM)
                  ▼
   ┌─────────────────────────────┐    found on 3 MORE hosts
   │ sh.exe also on WS-14, WS-22, │ ──► widen scope
   │ DB-07                        │
   └──────────────┬──────────────┘
                  │ what account placed it? -> svc_backup
                  ▼
   ┌─────────────────────────────┐    svc_backup logons reveal
   │ account: svc_backup          │ ──► 5 hosts, and 1 VPN entry
   │ pivot: where did it log on?  │     by user jlopez
   └──────────────┬──────────────┘
                  │ pivot on jlopez VPN src IP 203.0.113.9
                  ▼
   ┌─────────────────────────────┐
   │ ROOT CAUSE: jlopez VPN cred  │  stolen creds, VPN had NO MFA
   │ used from attacker IP; no MFA│  ──► the gap to fix
   └─────────────────────────────┘
   FULL SCOPE: FS-01, WS-14, WS-22, DB-07 + svc_backup + jlopez VPN

Figure 25.3 — Scoping by pivoting. Each indicator (file hash → account → credential → entry IP) is searched across the environment to reveal the next, expanding from one known-bad host to the full footprint and converging on the root cause. DB-07 in scope is the finding that determines whether customer data was exposed.

This pivoting is precisely where the earlier-built detection tooling pays off: searching a file hash or account name across the fleet is exactly the ioc_match-style operation from Chapter 22 and the centralized SIEM queries of Chapter 21, now run with a forensic purpose. Scoping is only as good as your visibility — which is why forensic readiness (the Project Checkpoint) is not a "nice to have" but the precondition for ever answering "how far did it go."

Root-cause analysis

Scoping that walks the chain to its origin delivers the deliverable a security program actually needs: root-cause analysis (RCA), the determination of the underlying reason an incident was possible — not merely the proximate symptom, but the failure that, if fixed, prevents recurrence of the whole class of incident. In the Meridian case the proximate event was "ransomware ran on FS-01." The root cause, found by pivoting back to the entry point, was "a VPN account protected by a password alone, with no multi-factor authentication, allowed an attacker with a stolen credential straight in." Patching FS-01 fixes nothing; requiring phishing-resistant MFA on the VPN fixes the actual hole — and connects directly back to the authentication lessons of Chapter 16 and the very control that saved Meridian from the Chapter 1 phishing attempt. A good RCA names a fix at the level of the root cause, and it is the single most valuable product of the entire investigation, because it is what turns an expensive incident into a permanent improvement.

🔗 Connection: Root cause is also the seam between forensics and the rest of the program. The RCA finding feeds the "lessons learned" phase of incident response (Chapter 24), updates the risk register (Chapter 1's first artifact), and may reshape controls anywhere in the program. An investigation that ends at "we cleaned it up" and never reaches root cause has failed at its most important job — it has spent the cost of an incident without buying the lesson.

Anti-forensics: detecting the attacker's counter-play

Throughout this chapter the attacker has been fighting back against the investigation, and that effort has a name. Anti-forensics is the set of techniques attackers use to hinder investigation — destroying, hiding, or falsifying evidence — including log clearing, timestomping (altering file timestamps to disguise when files were created or modified), secure deletion of tools, encryption, and living-off-the-land (using built-in system tools so nothing unusual is installed to find).

The defender's stance toward anti-forensics is the theme of this whole chapter restated: you cannot prevent every anti-forensic act, so you (a) detect its traces and (b) deny it a complete victory by putting evidence beyond the attacker's reach. We have seen both halves in action:

• Detect the traces. Anti-forensics is rarely perfect, and its attempts leave their own evidence. Event ID 1102 is the log clearing — the clearing is logged. Timestomping leaves the $MFT's two timestamp sets disagreeing. A .bash_history that is empty on an active account, a gap in otherwise continuous logs, a file present in Prefetch (it ran) but mysteriously absent from disk (it was deleted) — each is a signature of an attempt to hide, and each tells you exactly where to look harder.
• Deny the victory. The single most effective counter to anti-forensics is architectural, not investigative: ship evidence off the host to append-only, tamper-resistant storage before the attacker arrives. An attacker who clears the local Security log cannot reach the copy already forwarded to the SIEM; an attacker who wipes a disk cannot un-write the flow records your network sensors already stored. This is the entire rationale for forensic readiness, and it is why the most important forensic work happens long before the incident, in the logging and retention architecture you build now.

📟 War Story (constructed, illustrative). A mid-size firm suffered an intrusion in which the attacker, on the way out, ran a tool to clear local event logs on every machine they touched — confident they had erased their trail. They had, locally. What they did not know was that the firm had quietly configured Windows Event Forwarding months earlier, streaming a copy of every security event to a central collector the attacker never identified or reached. The local logs were gone; the forwarded copies were complete. The entire intrusion was reconstructed from the off-host evidence, and the "cleared" local logs became a damning indicator of intent rather than a dead end. The lesson is the chapter in one sentence: forensics is won or lost before the incident, in the decision to put evidence somewhere the attacker cannot follow.

🔄 Check Your Understanding: 1. Define indicator scoping and explain how a single confirmed indicator (a file hash) leads to discovering the full extent of a compromise. 2. An attacker cleared local logs on every host. Name one way you might still reconstruct their activity, and one anti-forensic signature the clearing itself left behind.

Answers

1. Indicator scoping is searching confirmed indicators (file hash, account, IP, registry key) across the whole environment to find every affected host/account. A file hash is searched fleet-wide and turns up on additional hosts; identifying the account that placed it reveals more hosts; pivoting on that account's logons reveals the entry point — each indicator surfaces the next, expanding to full scope. 2. Off-host evidence the attacker could not reach — forwarded events in the SIEM, network flow records, the $MFT timestamps they missed — can reconstruct activity; the signature left behind is the log-clearing event itself (Windows Event ID 1102) and the conspicuous gap in otherwise continuous logs.

Project Checkpoint

Meridian's program now has an incident-response plan and a tested ransomware playbook (Chapter 24). This chapter adds forensic readiness — the architectural and procedural conditions that make a future investigation possible and defensible — and the forensics.py module of bluekit.

Program increment — forensics readiness. After the (constructed) ransomware incident, Priya Nair leads a forensic readiness review. The deliverable is a short standard with four parts, each a direct lesson from this chapter: (1) Evidence-handling procedure — order-of-volatility acquisition, write-blocker and hashing requirements, and a chain-of-custody form (Figure 25.2) every responder uses. (2) Forensic logging and retention — centralize the high-value artifacts (Windows Security events including 4624/4625/4688/7045/1102, Linux auth/journal) into the SIEM with retention long enough to investigate a dwell time of months, not days, and store them append-only so an attacker cannot scrub them (the anti-forensics counter, §25.6). (3) Host visibility — endpoint tooling capable of memory capture and fleet-wide indicator search, so scoping (§25.6) is actually possible. (4) Legal-hold and escalation — a defined point at which legal counsel and HR are engaged (§25.3), so the team does not improvise the parts of forensics that are really about the law. This standard, plus the IR plan from Chapter 24, is what lets Meridian answer "what happened, how far, and can you prove it" the next time.

bluekit increment — forensics.py. Two functions formalize the chapter's two core operations: verifying an evidence hash, and merging multi-source events into one timeline. As always, the code is illustrative and is never executed during authoring — every output is hand-traced in an # Expected output: comment, and the hashes are labeled placeholders, not real digests.

# bluekit/forensics.py  — Chapter 25 increment
"""Two core DFIR operations: evidence-integrity verification and timeline merge.

evidence_hash(): confirm an image matches its source (integrity, ch.25 sec.2).
merge_timeline(): normalize multi-source events to one sorted timeline (sec.5).

NOT EXECUTED during authoring — outputs below are hand-traced. Hashes are
illustrative placeholders (a real SHA-256 digest is 64 hex chars).
"""
import hashlib

def evidence_hash(data: bytes, expected: str | None = None) -> dict:
    """Hash evidence bytes (SHA-256) and, if given, verify against expected."""
    digest = hashlib.sha256(data).hexdigest()
    verified = (expected is not None and digest == expected)
    return {"sha256": digest, "verified": verified}

def merge_timeline(sources: list[tuple[str, list[tuple[str, str]]]]) -> list[tuple]:
    """Merge (source_name, [(utc_timestamp, event), ...]) into one sorted list.

    Each input timestamp is assumed already normalized to UTC 'YYYY-MM-DD HH:MM:SS'.
    Returns rows (timestamp, source, event) sorted chronologically.
    """
    rows = [(ts, name, event)
            for name, events in sources
            for ts, event in events]
    return sorted(rows, key=lambda r: r[0])      # sort by UTC timestamp string

if __name__ == "__main__":
    h = evidence_hash(b"forensic image bytes", expected=hashlib.sha256(b"forensic image bytes").hexdigest())
    print(f"verified={h['verified']}")
    tl = merge_timeline([
        ("VPN",    [("2025-04-15 02:02:50", "VPN connect user=jlopez")]),
        ("WinEvt", [("2025-04-15 02:14:07", "logon svc_backup"),
                    ("2025-04-15 03:02:10", "audit log cleared")]),
        ("MFT",    [("2025-04-15 02:15:02", "sh.exe created")]),
    ])
    for ts, src, event in tl:
        print(f"{ts}  {src:7s} {event}")

# Expected output:
# verified=True
# 2025-04-15 02:02:50  VPN     VPN connect user=jlopez
# 2025-04-15 02:14:07  WinEvt  logon svc_backup
# 2025-04-15 02:15:02  MFT     sh.exe created
# 2025-04-15 03:02:10  WinEvt  audit log cleared

Trace it by hand and the two lessons land. evidence_hash returns verified=True only because the digest of the bytes matches the expected digest — the §25.2 integrity check in code; feed it a tampered image and verified flips to False. merge_timeline flattens three differently-shaped sources into one list and sorts by the UTC timestamp, reproducing the §25.5 unified timeline — and notice the VPN row sorts to the top, surfacing the true initial access exactly as it did in the worked example. You have written the tools that turn raw evidence into an answer.

Summary

This chapter turned the contained incident of Chapter 24 into a defensible investigation.

• Digital forensics (DFIR) is the methodical, repeatable, defensible practice of preserving, acquiring, analyzing, and reporting on digital evidence. It answers what happened, when, how, and how far — and produces the proof that breach notification, insurance, and litigation require. Its mindset is the inverse of IT's: change the evidence as little as possible.
• The order of volatility dictates collection from most fragile to least: registers/cache → RAM → network state → running system → disk → off-host logs → backups. Capture memory before imaging disk; never pull power on a live, un-captured system. (RFC 3227; NIST SP 800-86.)
• Sound acquisition requires a write blocker (read the original, never write it) and cryptographic hashing (SHA-256) of source and image to prove the copy is perfect and unaltered. A disk image is bit-for-bit, capturing deleted files and slack space a file copy misses.
• Chain of custody — the unbroken, documented record of who handled evidence when — plus authorization, integrity, repeatability, and competent tooling, make findings legally sound. Analyze the image, never the original; involve legal counsel when the law is in play (legal hold, privacy, litigation).
• High-value forensic artifacts: Windows registry (USB, persistence Run keys, execution), $MFT (MACB timestamps, deleted files), Event Logs (4624/4625 logon, 4688 process, 7045 service install, 1102 log cleared), Prefetch/Amcache (proof of execution); Linux auth.log/secure, journal, .bash_history, stat timestamps, cron/systemd persistence.
• Timeline analysis merges timestamped events from all sources into one chronological narrative — normalize to UTC, account for clock skew, tag each source. It reveals sequence/causation, the initial-access point, and gaps where one source was tampered with — corroborate across independent, off-host sources.
• Indicator scoping pivots each confirmed indicator (hash → account → credential → entry IP) across the environment to find the full footprint; root-cause analysis walks that chain to the underlying gap whose fix prevents recurrence (here: a VPN with no MFA). Anti-forensics (log clearing, timestomping, secure deletion, living-off-the-land) is countered by detecting its traces and putting evidence off-host beyond the attacker's reach.

Spaced Review

Retrieval practice across this chapter and earlier ones. Answer before checking.

1. (Ch. 24) In the incident-response lifecycle, which phases most directly depend on forensic findings, and why does containment sometimes pull against forensic preservation?
2. (Ch. 11) Name two host artifacts on a Linux server — relevant to both hardening/detection and forensics — and say what each would reveal about an attacker's activity.
3. (This chapter) State the order of volatility for the first four categories of evidence, and give the one acquisition mistake it most exists to prevent.
4. (This chapter) Why is an off-host, append-only copy of logs the strongest single defense against anti-forensics?

What's Next

You have now completed the operational core of defense — Part V took you from centralized logging (Chapter 21) through detection and hunting, vulnerability management, incident response, and, here, the forensic investigation that closes the loop on an incident with provable facts. But operations do not run themselves: they require governance — the policies, risk processes, frameworks, and human programs that turn a collection of capabilities into a managed, auditable security program. Part VI begins there. The forensic findings you learned to produce here become inputs to that program: root causes update the risk register, breach-scoping conclusions drive compliance and notification obligations, and the forensic-readiness standard you wrote becomes one more documented control in a governance structure. We will return to forensics' lessons directly when we build the security team and SOC (Chapter 37) and when we analyze the landmark breaches that taught the industry its hardest lessons (Chapter 40) — but first, we learn how security becomes a program that an organization can govern, fund, and defend to its board.