Chapter 4 Quiz

Technology Foundations: AI, ML, NLP, and Automation in Compliance

20 questions.

1. "Precision" in a machine learning context measures:

a) How accurate the model is overall on the test dataset b) Of all transactions the model flagged as suspicious, what proportion are genuinely suspicious c) Of all genuinely suspicious transactions, what proportion did the model flag d) The statistical confidence interval of the model's predictions

2. "Recall" in a machine learning context measures:

a) The proportion of correct predictions among all predictions b) How quickly the model processes new transactions c) Of all genuinely suspicious transactions, what proportion did the model flag d) The F1 score minus the precision score

3. A compliance officer says: "Our new ML model has 99% accuracy." Why might this claim be misleading in an AML context?

a) AML models are not measured by accuracy b) If only 1% of transactions are suspicious, a model that flags nothing achieves 99% accuracy while missing all suspicious activity c) Accuracy is not a meaningful metric for gradient boosting models d) 99% accuracy is statistically impossible in AML detection

4. Which compliance problem type is BEST addressed by a graph analytics approach?

a) Verifying that a customer's passport is genuine b) Calculating a bank's risk-weighted assets under Basel III c) Detecting money mule networks where multiple accounts pass funds in a chain d) Generating a SAR narrative from structured alert data

5. The primary advantage of a rule-based transaction monitoring system over ML-based approaches is:

a) Higher accuracy in detecting suspicious transactions b) Ability to adapt to new typologies without programmer intervention c) Transparency and determinism — you can always explain exactly why a transaction was flagged d) Lower false positive rates

6. "Class imbalance" in AML machine learning refers to:

a) The imbalance between analyst review capacity and alert volume b) The fact that suspicious transactions are very rare compared to legitimate ones, making it difficult for models to learn the suspicious class c) The imbalance between precision and recall in production models d) The uneven distribution of transaction amounts in the training data

7. Named entity recognition (NER) in NLP refers to:

a) The classification of entire documents into predefined categories b) The identification and extraction of specific types of information (people, organizations, places) from text c) The detection of sentiment (positive/negative) in communications d) The generation of summaries from long documents

8. Robotic Process Automation (RPA) is MOST appropriately used for:

a) Detecting complex money laundering patterns across thousands of transactions b) Learning new compliance requirements from regulatory publications c) Automating repetitive manual tasks like populating regulatory report templates d) Generating real-time risk scores for incoming transactions

9. Setting the alert threshold in an ML-based AML system (the score above which a transaction is flagged) is:

a) A technical decision that should be made by the data science team b) A compliance and legal decision requiring documentation and management approval c) A regulatory decision that must be approved by the supervisor before deployment d) Automatically set by the model based on the optimal F1 score

10. Which of the following is NOT a risk of using large language models (LLMs) for compliance documentation?

a) Hallucination — generating plausible but incorrect information b) Temporal limitation — models may not know recently enacted regulations c) Governance requirements under model risk management frameworks d) LLMs cannot process text faster than human analysts

11. In graph analytics for AML, "nodes" and "edges" represent:

a) Servers and network connections in the monitoring infrastructure b) Entities (customers, accounts) and relationships (payments, ownership) respectively c) Individual transactions and their associated risk scores d) Regulatory rules and the transactions they apply to

12. The AUC-ROC score of 0.5 for an ML model indicates:

a) The model has excellent discrimination ability b) The model performs similarly to random chance c) The model misclassifies 50% of all transactions d) The model meets the minimum regulatory standard for AML models

13. Which of the following is the BEST description of supervised machine learning?

a) The model learns without any human-provided labels, finding patterns independently b) A human supervisor reviews all model outputs before they are acted upon c) The model learns from labeled examples (e.g., confirmed suspicious and confirmed legitimate transactions) d) The model is programmed with explicit rules by a human supervisor

14. The "AI readiness" dimension that is most commonly underdeveloped when organizations attempt to deploy ML in compliance is:

a) Data readiness b) Technology readiness c) Governance readiness d) People readiness

15. "Semantic search" in a regulatory intelligence context refers to:

a) Searching for exact keyword matches in regulatory documents b) Finding regulatory requirements that are semantically similar to a query, even without exact keyword matches c) Categorizing regulatory publications by their legal basis d) Automatically translating regulations from multiple languages

16–20. Short answer / applied questions

16. Maya is reviewing a vendor's ML-based KYC document verification system. The vendor claims 99.2% accuracy in document authentication. What additional metrics should Maya request, and why?

17. Explain in plain English why graph analytics is useful for detecting "layering" — the second stage of money laundering where criminal funds are moved through multiple accounts to disguise their source.

18. Rafael's compliance team is considering using an LLM to generate first drafts of SAR narratives from structured alert data. What governance controls should be in place before deploying this capability?

19. A rules-based transaction monitoring system flags 400 alerts per day with a 97% false positive rate. An ML enhancement reduces the false positive rate to 80%. If each alert review takes 15 minutes, how many analyst hours per day are saved? What is the annual savings at a loaded analyst cost of £60,000/year?

20. Priya's client has asked her why their ML-based AML model performs well on historical data but seems to be generating more false negatives (missing suspicious transactions) than expected in production. What is the most likely explanation for this discrepancy, and what should be done?