Exercises: AI and Machine Learning in Security

These exercises move from the arithmetic of anomaly scoring to the judgment of deploying ML against an adversary. Difficulty is marked ⭐ (recall/application), ⭐⭐ (analysis), and ⭐⭐⭐ (synthesis/open-ended). A dagger (†) marks problems with a full worked solution in Appendix: Answers to Selected Exercises — try every problem before you read one.

Do all arithmetic by hand (a calculator is fine; running code is not the point). Where a problem asks you to "tune" or "design," there is rarely one right answer — the reasoning and the honesty about limits matter more than the number. All scenarios are illustrative (Tier 3) unless a real standard is named.

Part A — Core concepts: what ML can and can't do ⭐

1.† Define in one sentence each: supervised detection, unsupervised detection, anomaly detection, and UEBA. Then state the single sentence that the chapter calls a threshold concept about the relationship between "anomalous" and "malicious."

2. For each detection task, state whether you would use a deterministic rule, supervised ML, or unsupervised/anomaly detection, and why: (a) alert when any account logs in from a country the bank does not operate in; (b) classify an inbound email as phishing or legitimate; (c) flag a service account whose nightly failed-login count is unusual for that account; (d) block a file whose hash is on a known-malware list; (e) surface the one user, out of 1,800, whose data-download behavior departed most from their own past.

3. Give one strength and one weakness of supervised detection, and one strength and one weakness of unsupervised detection. For each weakness, name an attack type that exploits it.

4.† Explain explainability and why an unexplainable alert is a problem specifically in (a) an incident review and (b) a conversation with a regulator. Why do simpler models (like a z-score) have an explainability advantage over complex ones?

5. The chapter argues the default answer to "should this detection use ML?" is "write the rule." Give two concrete advantages a deterministic rule has over an ML model for a problem you can state exactly, and one situation where a rule genuinely cannot do the job.

Part B — Compute an anomaly score ⭐⭐

6.† A privileged user's count of distinct source IPs per day over a baseline of eight days is $[2, 1, 2, 3, 2, 1, 2, 3]$. Today the count is $9$. Compute $\mu$, the population standard deviation $\sigma$, and the z-score of today's value. Does it cross a threshold of $3$? Show every step.

7. Using the same baseline as Exercise 6, suppose today's value were $4$ instead of $9$. Compute the z-score. Is it anomalous at threshold 3? At threshold 1.5? What does the choice of threshold do to your answer, and who should make that choice?

8.† A host's hourly count of outbound DNS queries to never-before-seen domains over six hours is $[5, 6, 4, 5, 5, 5]$. (a) Compute $\mu$ and $\sigma$. (b) You observe a new hour with $5$ queries — compute its z-score. (c) The standard deviation you computed is very small; explain what would happen to this detector if one hour in the baseline had been $50$ instead of one of the $5$s, and which robust statistic would help.

9. Explain, with the arithmetic, why a zero-variance baseline (e.g., $[4, 4, 4, 4]$ then a test value of $7$) makes the z-score undefined, and why a well-built zscore_anomaly function should raise an error rather than return a number in that case.

10.† A service account's nightly failed-login baseline is $[3, 4, 4, 3, 4, 5, 4, 3]$ and tonight is $12$. Compute the z-score. Then suppose an attacker had been present during the last three nights of the baseline, inflating those counts (the trailing $5, 4, 3$) to $7, 8, 9$ instead. Recompute $\mu$, $\sigma$, and tonight's z-score with the contaminated baseline $[3, 4, 4, 3, 4, 7, 8, 9]$. By how much did the contamination lower tonight's score, does it still cross a threshold of 3, and what attack does this illustrate?

Part C — Tune the false-positive tradeoff ⭐⭐

11.† Meridian processes $500{,}000$ authentication events per day, of which $40$ are genuinely malicious. A detector catches $95\%$ of attacks and has a false-positive rate of $0.8\%$. Compute: (a) true positives per day; (b) false positives per day; (c) total alert volume; (d) precision (the probability that a given alert is a real attack). Is this queue workable for a five-person SOC?

12. Take the detector from Exercise 11 and improve its false-positive rate to $0.1\%$ (catching the same $95\%$). Recompute false positives, total volume, and precision. Did precision become "good"? What does this tell you about chasing accuracy improvements versus other levers?

13.† Restate the base-rate problem in your own words and explain why a detector that flags nothing at all can still report "99.99% accuracy" on a rare-event problem. Why is accuracy therefore the wrong headline metric, and what two metrics should replace it?

14. Your SOC can investigate $30$ alerts per shift, but the anomaly detector is producing $450$. List three different levers from §34.3 you could pull to fit the queue to capacity, and for each state exactly what you give up (the cost). Which lever would you pull first at a bank, and why?

15. A vendor claims their model is "99.9% accurate at detecting account takeover." Write three questions you would ask before believing the queue will be workable. For each question, explain what a bad answer would reveal.

Part D — Spot the adversarial attack ⭐⭐

16.† For each scenario, name the adversarial-ML attack (data poisoning — and which flavor — or model evasion), the kind of model attacked, and one defense: - (a) An attacker repeatedly submits malware samples to your public scanning API, tweaking each one slightly until a variant scores "clean," then uses that variant. - (b) An attacker who is already inside the network slowly ramps up data exfiltration by a few percent a week; the anomaly detector's baseline drifts and never fires. - (c) A contributor to a shared threat-intel dataset injects many mislabeled "benign" samples that resemble a specific malware family, so a model trained on it later misses that family. - (d) A model is trained to be perfectly accurate except on inputs containing a specific rare byte sequence, which it always labels benign.

17. Explain why query access to a model is dangerous in the context of model evasion. What defensive measures reduce the risk, and how does limiting/logging queries connect to the anomaly detection ideas earlier in the chapter?

18.† The chapter says the unifying defense against both poisoning and evasion is "assume the layer fails" (Theme 4). Sketch a layered detection architecture for a malware-scoring model that follows this principle: what sits in front of the model, what sits behind it, and why each placement blunts a different adversarial attack.

19. Provenance as a control. Explain how the supply-chain discipline of data provenance (knowing where training data came from) defends against data poisoning. Why is an anomaly detector that auto-updates its baseline from live, attacker-reachable traffic especially exposed, and what changes would you make to its baseline policy to reduce that exposure?

Part E — Design a UEBA use case (design it) ⭐⭐–⭐⭐⭐

20.† Design it. Design a UEBA use case to detect a compromised privileged administrator account at Meridian. Specify all four design elements from §34.2: the entity, at least four features (and what each would reveal about a takeover), the baseline (per-entity? peer group? time windows?), and the scoring and threshold approach. Then name two benign events that would generate false positives and how you would enrich the alert to suppress them.

21. Design a UEBA use case to detect a departing insider exfiltrating customer data (the War Story pattern from §34.2). How does this differ from the account-takeover design in Exercise 20 — especially in the features and in why a single-night threshold fails? Which signal type (point anomaly vs. trend) matters most here?

22. ⭐⭐⭐ Pick a system you know well (your school, employer, or a public service). Design a UEBA use case for one realistic threat to it: name the entity, features, baseline, and threshold, then write a paragraph honestly assessing what the detector would miss and how an attacker who knew it existed would evade it.

Part F — Write the policy / rule ⭐⭐

23. Write a short SOC policy snippet (4–8 bullet points) governing the safe use of an LLM assistant in the SOC. It must address least privilege/tool access, treatment of untrusted input and unverified output, human-in-the-loop for consequential actions, and secrets in prompts. Map each bullet to the security principle (and chapter) it generalizes.

24.† Write a control requirement (policy language) mandating out-of-band verification for high-value or unusual money-movement requests at Meridian, designed specifically to defeat deepfake-voice and deepfake-video authorization fraud. Specify the threshold, the second channel, and the dual-authorization rule, and state explicitly why a deepfake cannot defeat it.

25. Draft a one-paragraph addition to Meridian's security-awareness training (Chapter 30) for the AI era. Retire the "spot the typo" guidance and replace it with the durable signature of social engineering, including a line on voice/video pretexts.

Part G — Respond to this incident (tabletop) ⭐⭐

26.† A finance clerk reports that they joined a video call with "the CFO and two colleagues" who urgently directed a $2.3M wire, which the clerk initiated but has not yet released. You suspect a deepfake. Write the first five response steps in order (think Chapter 24 lifecycle), the single most important immediate action, and the two process controls whose absence allowed this to get as far as it did.

27. Your SOC's new LLM triage assistant suddenly starts marking a cluster of obvious phishing emails as "benign — no action." You suspect indirect prompt injection. Describe how you would (a) confirm the hypothesis, (b) contain the assistant safely, and (c) prevent recurrence. What capability should the assistant not have had?

Part H — CTF-style challenge ⭐⭐⭐

28.† The detector that learned to fail. A team proudly reports that their six-month-old anomaly detector "has had almost no alerts for the last two months — the network must be clean." Given everything in this chapter, list at least four distinct, plausible explanations for the silence (some benign, some alarming), the one you would investigate first, and exactly how you would test whether the detector is still working at all. (Hint: consider drift, poisoning, a muted feed, a contaminated baseline, and a broken pipeline.)

Part I — Interleaved & forward-looking ⭐⭐

29. (Interleave Ch. 21–22.) Explain how this chapter's anomaly detector should feed the SIEM and detection-engineering workflow rather than stand alone. Where in the pipeline (Figure 34.2) do the Chapter 22 enrichment signals (threat intel, IoC context) attach, and why does adding them raise the true-positive yield the SOC can afford to surface?

30. (Interleave Ch. 30 + forward to Ch. 35.) AI-enabled phishing makes the human firewall harder to maintain. Name one Chapter 30 program change you would make in response, and one emerging threat from Chapter 35's preview (deepfakes/synthetic identity, ransomware-as-a-service, post-quantum) that you expect anomaly detection to be poorly suited to catch — and say why.

31. ⭐⭐⭐ Open reflection. The chapter claims "AI does not obsolete security fundamentals — it makes them matter more." Choose three fundamentals from earlier chapters (e.g., least privilege, input validation, out-of-band verification, defense in depth, provenance) and write half a page showing how each one is the actual defense behind a flashy AI-era threat in this chapter.

Solutions to daggered (†) problems are in the Answers appendix. The remaining problems are deliberately open — bring them to a study group or your instructor. Remember: hand-trace the math, and always pair an ML capability with its limit.