Appendix A — File Signatures Reference

Purpose. A keyboard-side lookup of file signatures (magic numbers) — headers, footers, offsets, extensions, and carving notes — for identifying file types by content and for recovering files when the file system is gone. This is the reference table promised in Chapter 7 — File Carving; read that chapter for the why, keep this open for the what.

A file signature (also magic number) is a short, fixed byte sequence at a known position — almost always the first few bytes — that identifies a file's format independent of its name or extension. Two disciplines lean on this constantly:

• Recovery and carving (💾) — when the metadata that says "this run of sectors is a file" has been wiped (a reformat, an overwritten MFT, a dead partition table), the only way to find a file is to recognize its bytes. Carvers scan raw sectors for a known header, then read or compute where the file ends. See Chapter 6 — Logical Recovery for when metadata survives and carving is not needed.
• Forensic typing and masquerade detection (🔍) — the extension is a label a user (or an adversary) chose; the magic number is ground truth. A file named vacation.jpg whose first bytes are 4D 5A is a Windows executable wearing a costume. Extension-vs-signature mismatch is a classic anti-forensic tell — see Chapter 30 — Anti-Forensics and the masquerade workflow at the end of this appendix.

Recovery vs. Forensics. The same three bytes FF D8 FF serve both masters. To the recovery tech they say "a JPEG starts here — carve it and give the client their photo back." To the examiner they say "this object is a JPEG regardless of what the file table claims — and if the file table claims otherwise, that discrepancy is itself evidence." Recognize the bytes once; choose your purpose second.

A.1 How to read this appendix

• Hex notation. Bytes are shown space-separated in uppercase hex: FF D8 FF. In code and tool configs they appear escaped: \xFF\xD8\xFF (C/foremost/scalpel/YARA) or \xff\xd8\xff. Printable bytes are sometimes shown as ASCII in quotes: %PDF, "PK", "ftyp".
• Offset. Unless a @N annotation appears, the signature is at byte offset 0 (the very start of the file). @4 means "starting at byte 4," @0x8001 means byte 33,793. Offsets matter: an ftyp box and a ustar marker live deep inside their files, not at the front.
• Endianness. Some formats exist in both little-endian (Intel) and big-endian (Motorola/network) flavors, and the magic number itself tells you which (TIFF II vs MM; PCAP D4 C3 B2 A1 vs A1 B2 C3 D4). Byte order is reviewed in Chapter 2 — How Data Is Stored.
• Footer. Many formats have no reliable terminal marker — their length lives in a header field or in internal structures. "—" in the footer column means no fixed footer; size from header or max-carve (see A.7 Carving Strategy).
• "It identifies the type, not the truth." A valid signature proves a byte run claims to be a format. It does not prove the file is intact, uncorrupted, or contiguous. Always validate (parse/decode) before you trust a carve.

A.2 Images

A.3 Documents and e-books

A.4 Archives and compression

A.5 Audio, video, and containers

A.6 Databases, email, and system artifacts

A.7 Executables, code, and fonts

A.8 Disk, partition, and forensic-image formats

These matter at the container layer — what you mount, image, or carve inside of. See Chapter 14 — Forensic Acquisition.

A.9 Network captures, crypto, and miscellany

A.10 Container disambiguation

A single magic number frequently maps to many real formats. These are the four families that cause the most confusion in the field.

A.10.1 ZIP-based formats (OOXML, ODF, EPUB, JAR/APK)

All begin with the ZIP local-file-header magic 50 4B 03 04. They differ only in their contents. Peek inside (unzip the central directory, or 7z l, or read the first entry name at offset 30):

Tool Tip. ODF and EPUB authors deliberately put an uncompressed mimetype entry first so that file(1) can identify the type at a fixed offset without decompressing anything. That is why the EPUB string mimetypeapplication/epub+zip sits predictably at offset 30 — a gift to both libmagic and your carver.

A.10.2 ISO BMFF "ftyp" brands (MP4 family)

The ISO Base Media File Format puts a 4-byte box size first, then 66 74 79 70 (ftyp) @4, then a 4-byte major brand @8 that tells you what it really is:

Tool Tip. Carve ISO BMFF on the ftyp @4 marker, not on the leading size bytes (which vary). A robust carver reads each top-level box's 32-bit size field and walks ftyp → moov → mdat … to find the true end — there is no footer.

A.10.3 RIFF form types

RIFF starts with 52 49 46 46 (RIFF), a 4-byte little-endian total-size-minus-8 @4, then a 4-byte form type @8:

Because the RIFF header carries its own size, RIFF files are a textbook case for structure-aware carving: read the size at offset 4, carve exactly that many bytes plus 8.

A.10.4 OLE2 / CFBF (legacy Office)

D0 CF 11 E0 A1 B1 1A E1 is the Compound File Binary Format — a tiny FAT-like file system inside a file. Word, Excel, PowerPoint, Outlook .msg, .msi, and more all share it. Identify the real type by the named streams inside:

A.10.5 TIFF-based RAW photo formats

Many camera RAW formats are TIFF under the hood, so they share 49 49 2A 00 (II*) or 4D 4D 00 2A (MM). Disambiguate by the maker note / second magic:

A.11 Footers and end-of-file markers

Footers are precious — when present and reliable, they let you carve a file at its exact end. But most binary formats do not have one; they store length in a header or in internal structures. Memorize which is which.

Limitation. A footer signature is not always the file's last byte. ZIP's End-Of-Central-Directory record is followed by 18+ bytes plus a variable comment; PDF's %%EOF may be followed by whitespace and is repeated in incrementally-updated files; PNG's IEND may be followed by ancillary data appended by some tools. Carve to the footer, then read the format's length fields to trim or extend precisely. Naïve "stop at first footer" carving truncates incrementally-saved PDFs and ZIPs.

A.12 Reverse lookup — "I see these bytes, what is it?"

You are staring at a hex dump and need to name it fast. Sorted by first byte. (For full disambiguation of shared headers, jump to A.10.)

A.13 Carving strategy — header/footer, max-size, and fragmentation

A signature tells you where a file starts. The hard part is knowing where it ends and whether it is in one piece. Four strategies, in rough order of reliability:

1. Header-footer carving. The format has both a known header and a reliable footer (JPEG FF D8 FF…FF D9; PDF %PDF…%%EOF; GIF…00 3B; ZIP…50 4B 05 06). Carve from header to footer. Best case. Cautions: footers can repeat (carve to the last %%EOF) and may be followed by trailing bytes (read length fields to trim).

2. Header + embedded length (structure-aware). No footer, but the format stores its own size: BMP (@2), RIFF (@4), ISO BMFF box sizes, SQLite (page_size × page_count), ELF/PE section headers. Read the length and carve exactly. Most robust when available — immune to false footers and tolerant of trailing data.

3. Header + maximum-size carving. No footer and no usable length (raw MP3, many video streams, plain text). Carve from the header up to a configured maximum (e.g., foremost's per-type cap). You will over-carve — trailing garbage rides along — and downstream validation/parsing trims it. Pick the cap carefully: too small truncates large files; too large wastes disk and time and swallows the next file.

4. Validation / semantic carving. Don't trust the bytes — parse them. A carved JPEG that fully decodes is almost certainly intact; one that decodes halfway then errors flags a fragment boundary or corruption. PhotoRec, Scalpel's validation mode, and bulk_extractor lean on this to set correct ends and to reject false positives. Always validate the carve before reporting it.

Fragmentation — the real enemy

Carving's core assumption is that a file occupies one contiguous run of sectors. File systems break that promise: appends, deletion-and-reuse, copy-on-write, and SSD wear-leveling all scatter a file across the volume. (Recoverability on SSDs is further crushed by TRIM — see Chapter 9 — SSD and Flash Recovery.)

Contiguous (easy):
  [ HDR ........................... FTR ]      one run; carve HDR → FTR

Bifragmented (two pieces, a gap between):
  [ HDR ...... frag1 ]   [ other data ]   [ frag2 ...... FTR ]
   \____ file part ___/   \____ gap ___/    \___ file part ___/
   Bifragment Gap Carving: try gap = 1,2,3,… clusters,
   reassemble, VALIDATE (decode) at each — keep the gap that decodes.

Highly fragmented (3+ pieces):
  [HDR.f1]  [x]  [f2]  [y]  [f3..FTR]
   → SmartCarving / content-based reassembly; far harder, often partial.

Practical consequences and tactics:

• Cluster alignment. Fragments begin on cluster boundaries (commonly 4 KiB / 8 sectors). Search candidate fragment starts on aligned offsets, not every byte — fewer false positives, far faster.
• Bifragment gap carving (BGC). For two-fragment files with a known footer, brute-force the gap size between header-fragment and footer-fragment, validating each reconstruction. Works well for JPEG/PDF where a decoder gives a clean pass/fail signal.
• SmartCarving. For 3+ fragments, statistical/content models reassemble pieces (Pal & Memon). Expensive and imperfect; expect partial recovery.
• Validation is what makes fragmentation survivable. Without a decode/parse step, a fragmented carve yields a header followed by the wrong data — a file that opens to garbage. The validator is what tells truth from coincidence.
• Know when to stop (Theme #5). A heavily fragmented file on a busy, partly-overwritten volume may be unrecoverable. "Partial JPEG recovered; lower two-thirds corrupt due to fragmentation across an overwritten region" is an honest, professional result — see the recovery limits discussion in Chapter 7.

False positives and short signatures

Signature length is everything. A 2-byte magic like BMP's 42 4D or MP3's FF FB collides constantly with random and compressed data — a gigabyte of entropy contains thousands of accidental FF FB pairs. Longer signatures (PNG's 8 bytes, SQLite's 16) are nearly false-positive-free. Mitigations: prefer the longest reliable header; require correct offset (a real ftyp is at byte 4, not scattered mid-file); and validate every hit.

Embedded and nested files

Files contain files. A JPEG carries a smaller JPEG thumbnail inside its EXIF; an OOXML/ODF document is a ZIP full of XML and embedded media; a PST holds thousands of messages and attachments; an installer wraps a CAB. A naïve scanner will surface the inner objects as independent carves — sometimes useful (an embedded image you wanted), sometimes noise (ten thumbnail fragments of one photo). Tools like binwalk exist specifically to recursively find and extract nested signatures. Decide per case whether nested extraction helps or clutters your evidence set.

A.14 Tool configuration and recipes

foremost / scalpel configuration

foremost and scalpel (a foremost fork) share a config grammar: extension case-sensitive max-size header [footer] [flag]. Bytes are C-escaped (\xNN); ? is a single-byte wildcard (foremost); REVERSE searches for the footer backward from the max-size window (ideal for PDF/ZIP, whose footers can repeat).

# ext  case  max-size    header (hex)                          footer (hex)            flag
  jpg   y    20000000    \xff\xd8\xff\xe0                      \xff\xd9
  jpg   y    20000000    \xff\xd8\xff\xe1                      \xff\xd9
  png   y    20000000    \x89PNG\x0d\x0a\x1a\x0a               \x49\x45\x4e\x44\xae\x42\x60\x82
  gif   y    20000000    \x47\x49\x46\x38\x39\x61              \x00\x3b
  gif   y    20000000    \x47\x49\x46\x38\x37\x61              \x00\x3b
  pdf   y    50000000    %PDF                                  %%EOF                   REVERSE
  doc   y    25000000    \xd0\xcf\x11\xe0\xa1\xb1\x1a\xe1
  zip   y    50000000    PK\x03\x04                            PK\x05\x06              REVERSE
  rar   y    50000000    Rar!\x1a\x07\x00
  mov   y    100000000   ????ftyp
  sqlite y   50000000    SQLite\x20format\x203\x00
  htm   n    500000      <html                                 </html>

# Run foremost over a forensic image, into ./carved/, using your config:
foremost -v -T -c ./mycarve.conf -i suspect.dd -o ./carved

# Scalpel (same config grammar), with its preview/audit pass:
scalpel -c ./scalpel.conf -o ./carved suspect.dd

Limitation. The png footer \x49\x45\x4e\x44... and the ? wildcard make these carvers good but not structure-aware: foremost will happily carve to the first IEND even if ancillary chunks follow, and ????ftyp matches the marker but ignores box-length fields. For length-bearing formats (RIFF, ISO BMFF, SQLite), a parser-driven tool (PhotoRec, custom script) beats a flat config.

PhotoRec (signature carver, no config)

photorec            # interactive: pick disk → partition → file types → dest
photorec /d ./out /cmd suspect.dd search    # scripted run

PhotoRec ships ~480 built-in signatures and validates many of them (it parses, not just pattern-matches), which is why it recovers cleanly where flat carvers leave corrupt tails. It ignores the file system entirely — perfect after a reformat.

binwalk (firmware / nested signatures)

binwalk firmware.bin          # list every embedded signature + offset
binwalk -e firmware.bin       # extract them (recursively with -M)
binwalk -E firmware.bin       # entropy map: spot encrypted/compressed regions

file / libmagic (single-object typing)

file mystery.bin                       # human-readable type
file -i mystery.bin                    # MIME type
file -m ./custom.magic mystery.bin     # use a custom magic database

A custom magic rule (offset, type, test value, message):

0    string    \x89PNG\x0d\x0a\x1a\x0a    PNG image data
0    beshort   0xFFD8                     JPEG image data
0    string    SQLite\ format\ 3          SQLite 3.x database

YARA (hunting a signature across many files)

rule jpeg_at_start {
    meta:
        author = "DataField Project"
        desc   = "JPEG SOI marker at file start"
    strings:
        $soi = { FF D8 FF }
    condition:
        $soi at 0
}

rule masquerade_pe_in_image {
    strings:
        $mz  = { 4D 5A }
    condition:
        $mz at 0          // an .jpg/.png that is really a PE — see A.15
}

Python — identify a file and scan for carving offsets

import mmap

# (description, magic, offset, footer-or-None, extension)
SIGNATURES = [
    ("JPEG",            b"\xFF\xD8\xFF",                       0, b"\xFF\xD9", "jpg"),
    ("PNG",             b"\x89PNG\r\n\x1a\n",                  0, b"IEND\xAE\x42\x60\x82", "png"),
    ("GIF",             b"GIF8",                               0, b"\x00\x3B", "gif"),
    ("PDF",             b"%PDF-",                              0, b"%%EOF",  "pdf"),
    ("ZIP/OOXML/JAR",   b"PK\x03\x04",                         0, b"PK\x05\x06", "zip"),
    ("RAR v1.5-4",      b"Rar!\x1a\x07\x00",                   0, None,      "rar"),
    ("RAR v5",          b"Rar!\x1a\x07\x01\x00",               0, None,      "rar"),
    ("7-Zip",           b"7z\xBC\xAF\x27\x1C",                 0, None,      "7z"),
    ("GZIP",            b"\x1F\x8B\x08",                       0, None,      "gz"),
    ("BMP",             b"BM",                                 0, None,      "bmp"),
    ("TIFF (LE)",       b"II*\x00",                            0, None,      "tif"),
    ("TIFF (BE)",       b"MM\x00*",                            0, None,      "tif"),
    ("SQLite 3",        b"SQLite format 3\x00",               0, None,      "sqlite"),
    ("MS Office (OLE)", b"\xD0\xCF\x11\xE0\xA1\xB1\x1A\xE1",   0, None,     "doc/xls/ppt"),
    ("PE/EXE",          b"MZ",                                 0, None,      "exe"),
    ("ELF",             b"\x7FELF",                            0, None,      "elf"),
    ("ISO BMFF",        b"ftyp",                               4, None,      "mp4/mov/heic"),
    ("RIFF",            b"RIFF",                               0, None,      "avi/wav/webp"),
    ("MP3 (ID3)",       b"ID3",                                0, None,      "mp3"),
]

def identify(path):
    """Return (description, extension) by content, or (None, None)."""
    with open(path, "rb") as f:
        head = f.read(64)
    for desc, magic, off, _footer, ext in SIGNATURES:
        if head[off:off + len(magic)] == magic:
            return desc, ext
    return None, None

def find_offsets(image_path, magic):
    """Yield every byte offset where `magic` appears (carving seed scan)."""
    with open(image_path, "rb") as f:
        mm = mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ)
        i = mm.find(magic)
        while i != -1:
            yield i
            i = mm.find(magic, i + 1)
        mm.close()

# Example: locate every JPEG start in a raw image
# for off in find_offsets("suspect.dd", b"\xFF\xD8\xFF"):
#     print(f"JPEG SOI candidate at offset {off} (0x{off:X})")

The reusable, validated version of this scanner lives in Appendix B — Python Forensics Toolkit.

bash — find header offsets with grep/xxd

# First 16 bytes of a file, hex + ASCII:
xxd -l 16 mystery.bin

# Byte offsets of every JPEG SOI in a raw image (-b = byte offset, -a = treat as text):
grep -abo -P '\xFF\xD8\xFF' suspect.dd | head

# Hunt the ZIP local-file-header magic:
grep -abo -P 'PK\x03\x04' suspect.dd | head

PowerShell — read the first bytes on Windows

# Windows PowerShell 5.1:
$bytes = Get-Content -Path .\mystery.bin -Encoding Byte -TotalCount 16
# PowerShell 7+:
# $bytes = Get-Content -Path .\mystery.bin -AsByteStream -TotalCount 16
($bytes | ForEach-Object { '{0:X2}' -f $_ }) -join ' '

The full command-line cheat sheet for these tools (with flags and chain-of-custody notes) is Appendix H — Command-Line Reference.

A.15 Forensic use — extension / signature mismatch detection

The most common forensic application of this appendix is masquerade detection: comparing what a file is named against what its bytes say. A renamed or double-extension file is a deliberate hide — and detecting it is a quiet win for the examiner (Anchor Case #2, the employee covering their tracks).

import os

EXT_TO_MAGIC = {
    ".jpg":  [b"\xFF\xD8\xFF"],
    ".jpeg": [b"\xFF\xD8\xFF"],
    ".png":  [b"\x89PNG\r\n\x1a\n"],
    ".pdf":  [b"%PDF-"],
    ".docx": [b"PK\x03\x04"],
    ".zip":  [b"PK\x03\x04"],
    ".exe":  [b"MZ"],
}

def mismatch(path):
    ext = os.path.splitext(path)[1].lower()
    expected = EXT_TO_MAGIC.get(ext)
    if not expected:
        return None                      # unknown extension; no opinion
    with open(path, "rb") as f:
        head = f.read(8)
    return not any(head.startswith(m) for m in expected)   # True = suspicious

# A 'family_photo.jpg' that returns True is a file whose contents are NOT a JPEG.

Ethics Note. Signature analysis routinely surfaces files that have nothing to do with your authorized scope, and occasionally surfaces material you are legally obligated to report. Stay inside the warrant or engagement scope, and follow the mandatory-reporting and well-being guidance in Chapter 28 — Ethics. A magic-number scan that wanders outside scope is a finding you may not be allowed to keep.

Chain of Custody. Run signature analysis and carving on your verified working copy, never the original (Theme #2). Carving reads raw sectors; do it against the forensic image whose hash you recorded at acquisition (Chapter 14). Document the tool, version, config/signatures used, and the offsets recovered so the carve is reproducible and defensible.

A.16 Caveats and accuracy

• Signatures evolve. Formats add brands and versions (note RAR4 vs RAR5, EWF vs EWF2, the growing ftyp brand list). When a header doesn't match anything here, dump 64–128 bytes with xxd/Format-Hex, search the bytes, and consult Gary Kessler's File Signatures Table and the libmagic source — both cited in the Bibliography.
• Position matters. ftyp @4, ustar @257, CD001 @0x8001, ESE @4 — a byte string found at the wrong offset is probably coincidence, not a file.
• Type ≠ integrity ≠ relevance. A matching signature means the bytes claim a format. It says nothing about whether the file is complete, uncorrupted, contiguous, or pertinent to your case. Validate, then judge.
• This appendix pairs with Appendix G — File System Reference (where files live when metadata survives), Appendix B — Python Forensics Toolkit (working scanner/carver code), and Appendix H — Command-Line Reference (the carving tools). Terms used here are defined in the Glossary.

Reference companion to Chapter 7 — File Carving. Keep it next to the keyboard.