Further Reading: AI and Machine Learning in Security

Curated, annotated resources to deepen this chapter. Each entry notes which learning path it serves most (🛡️ SOC, 🏗️ Engineer, 📋 GRC, 📜 Cert) and its citation tier. Start with the suggested order; you do not need to read everything before Chapter 35.

Suggested order

  1. Read the OWASP Top 10 for LLM Applications — short, concrete, and immediately useful before you let any LLM near your data or tools.
  2. Skim MITRE ATLAS to see adversarial-ML attacks (poisoning, evasion) organized the way ATT&CK organizes intrusions.
  3. Read the NIST AI Risk Management Framework overview for the governance lens your GRC team will need.
  4. For the statistics, revisit any solid treatment of anomaly detection and the base-rate fallacy — the two ideas that make or break a deployment.

Standards & primary documents (Tier 1)

  • OWASP, Top 10 for Large Language Model (LLM) Applications (owasp.org). 🛡️🏗️📋 The community reference for LLM risks — prompt injection, insecure output handling, training-data poisoning, excessive agency, sensitive-information disclosure. Read this before deploying any LLM assistant in the SOC; it directly informs §34.5.
  • MITRE, ATLAS (Adversarial Threat Landscape for Artificial-Intelligence Systems) (atlas.mitre.org). 🛡️🏗️ The ML-specific cousin of ATT&CK: a public knowledge base of real-world attacks on ML systems (poisoning, evasion, model theft) with case studies. The map for §34.4's threat model.
  • NIST, AI Risk Management Framework (AI RMF 1.0) (NIST AI 100-1, 2023). 📋🏗️ The authoritative governance framework for trustworthy AI — Govern, Map, Measure, Manage. The board-level lens for AI in security, including the security-of-AI and AI-for-security distinction.
  • NIST, Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations (NIST AI 100-2). 🏗️ A rigorous taxonomy of evasion, poisoning, and privacy attacks and their defenses; the precise-vocabulary companion to §34.4.
  • NIST SP 800-94, Guide to Intrusion Detection and Prevention Systems (IDPS). 🛡️ Frames signature- based versus anomaly-based detection at the standard level — the foundation this chapter generalizes to learned models.
  • MITRE ATT&CK (attack.mitre.org). 🛡️ Revisit from Chapter 22: the behavioral techniques your anomaly detector and UEBA use cases should map to, so "unusual" connects to "which adversary technique."

Books & long-form (Tier 1 / Tier 2)

  • Chio, C., & Freeman, D., Machine Learning and Security (O'Reilly). 🏗️🛡️ The practitioner's book on exactly this chapter's subject — anomaly detection, classifier evaluation, adversarial ML, and the operational realities (including the base-rate problem) of security ML. The single best deeper read.
  • Sommer, R., & Paxson, V., "Outside the Closed World: On Using Machine Learning for Network Intrusion Detection" (IEEE S&P, 2010). 🛡️🏗️ The classic, still-relevant paper on why anomaly-based intrusion detection is so much harder than it looks — high cost of errors, the semantic gap, evaluation difficulties. Read it as the antidote to vendor hype. (Tier 2: a specific paper; confirm the venue if citing formally.)
  • Goodfellow, I., Bengio, Y., & Courville, A., Deep Learning (MIT Press). 🏗️ Background on the models themselves for engineers who want to understand what is under the hood; dip into the relevant chapters rather than reading cover to cover.
  • Anderson, R., Security Engineering (3rd ed.). 🏗️📋 The chapters on detection, fraud, and reasoning-about-attackers frame why a control's operational economics (false positives, who acts on alerts) matter more than its theoretical power — the §34.3 lesson in a broader setting.

Free online & talks (Tier 1 / Tier 2)

  • CISA / NCSC joint guidance on AI and on Secure AI System Development. 📋🏗️ Government guidance on building and deploying AI securely, including secure design, data provenance, and the supply chain of models — the policy framing for §34.4's defenses. (Tier 2: title/issuer pattern; verify current document before citing.)
  • Reporting on real deepfake-fraud incidents (2023–2024). 📋🛡️ Well-sourced accounts of finance employees tricked by deepfaked voices/video calls into authorizing multi-million-dollar transfers — the real pattern behind Case Study 2. (Tier 2: read reputable, well-sourced coverage; specifics vary by retelling.)
  • The base-rate fallacy, any rigorous explainer. 📜🛡️ Understanding why a rare-event detector produces mostly false positives is the most important quantitative idea in this chapter; internalize it from any solid probability source. (Tier 2.)

Tools to explore (in your own authorized lab only)

  • A hand-built z-score / UEBA notebook. 🛡️🏗️ Extend this chapter's mlsec.py: collect a per-entity baseline from your own lab logs, compute z-scores, then add a second feature and fuse them — feel the jump from a single-feature score to a UEBA-style risk score (and the explainability cost).
  • An adversarial-examples toolkit (defensive study). 🏗️ Libraries exist to generate evasion examples against your own models so you can test robustness. Use them only on models you own or are authorized to test (§34.4 ethics note).
  • A local/sandboxed LLM with deliberate prompt-injection test cases. 🛡️🏗️ Build a toy "summarize this email" assistant and try to hijack it with hidden instructions in the email body — the fastest way to internalize why least privilege and human-in-the-loop are non-negotiable.

⚖️ Authorization & Ethics reminder: Several resources describe how to attack ML systems (evasion, poisoning) and LLMs (prompt injection). Study them to defend; apply them only to systems you own or are explicitly authorized to test (Chapter 39). Crafting adversarial inputs against a third party's deployed model can cause real harm and is very likely illegal.