Tariq Bashir arrived at Cornerstone Financial Group's London compliance desk at 7:48 on a Monday morning to find fourteen open surveillance alerts waiting in the case management system. Eleven were the routine noise that a multi-asset trading floor...
In This Chapter
- Learning Objectives
- Opening Narrative: Cornerstone's Monday Morning Alert Queue
- 19.1 The Market Abuse Regulatory Framework
- 19.2 Insider Dealing: Definition and Elements
- 19.3 Market Manipulation: Categories and Typologies
- 19.4 Suspicious Transaction and Order Reports
- 19.5 Surveillance Platform Architecture
- 19.6 Python Implementation: A Surveillance Alert System
- 19.7 From Alert to Investigation: Cornerstone's CG-4412 Case
- 19.8 The Cross-Asset and Cross-Jurisdictional Surveillance Challenge
- 19.9 Regulatory Expectations and the FCA's Supervisory Approach
- Chapter Summary
Chapter 19: Market Surveillance: Detecting Manipulation and Abuse
Learning Objectives
By the end of this chapter, you will be able to:
- Explain the prohibited behaviors under UK MAR and EU MAR and identify the elements that distinguish lawful from unlawful trading activity
- Define inside information and distinguish primary from secondary insiders under the market abuse framework
- Classify market manipulation typologies — transaction-based, information-based, and benchmark manipulation — and describe real-world examples of each
- Articulate the Suspicious Transaction and Order Report (STOR) obligation, including the promptness requirement and the reporting threshold
- Describe the architecture of a modern surveillance platform, from data ingestion through alert generation to case management
- Compare rule-based, statistical, and machine-learning surveillance approaches, including their respective strengths and failure modes
- Apply Python to build basic surveillance detectors for spoofing and marking-the-close patterns
- Manage the false positive problem and design an alert triage workflow that supports regulatory defensibility
Opening Narrative: Cornerstone's Monday Morning Alert Queue
Tariq Bashir arrived at Cornerstone Financial Group's London compliance desk at 7:48 on a Monday morning to find fourteen open surveillance alerts waiting in the case management system. Eleven were the routine noise that a multi-asset trading floor generates — a trader in equities had placed and cancelled several large orders near a market data release, well within the parameters that Cornerstone's legal team had mapped against the FCA's Market Watch guidance. One involved an options position that, on closer inspection, was a straightforward delta hedge. Another was a peer comparison alert on a fixed income desk: a trader whose cancellation ratio had spiked on Thursday turned out to have been responding to a rapidly widening bid-offer spread during a brief liquidity dislocation.
The fourteenth alert was different.
It had been generated at 22:17 the previous Friday evening by the overnight batch run — the system's nightly sweep of the day's order and execution data against a set of pattern-detection rules. The alert flagged a trader on the EUR investment-grade corporate bond desk, identified in the system as Trader ID CG-4412. Over seventeen trading sessions spanning three weeks, Trader CG-4412 had placed a repeating pattern: large buy orders at or above the best offer in a specific ISIN — a bond issued by a French utility — followed by rapid cancellation within seconds of the order book updating, followed by a smaller, executed sell order at a slightly improved price. The cancel ratio across those seventeen sessions was 0.89. The price movement between order placement and cancellation was statistically significant at the 5 percent level when compared against the instrument's historical volatility profile. The system had scored it 0.74 out of 1.0 — just below the automatic escalation threshold of 0.75, which was why it had waited in the queue rather than triggering an immediate notification to the Head of Compliance.
Tariq looked at the pattern. He looked at the date range. Then he pulled up the trader's communication records for those sessions.
He opened a new case file.
This chapter is the story of what Tariq did next — and, more broadly, the regulatory framework, detection technology, and investigative workflow that transformed that automated score into either a filed Suspicious Transaction and Order Report or a documented, defensible decision that no report was warranted.
19.1 The Market Abuse Regulatory Framework
19.1.1 EU MAR and UK MAR: Scope and Architecture
Market abuse regulation in Europe has its roots in the original EU Market Abuse Directive (MAD) of 2003, which was superseded by the Market Abuse Regulation (EU) No 596/2014 — universally referred to as MAR — which came into force on 3 July 2016. MAR represented a significant expansion in both scope and substance. Unlike the directive it replaced, MAR is a directly applicable regulation, meaning it does not require transposition into national law; it applied uniformly across EU member states from day one.
Following the UK's departure from the European Union, the UK retained the substance of EU MAR through the European Union (Withdrawal) Act 2018, creating what practitioners refer to as UK MAR. For practical purposes, UK MAR and EU MAR are substantively identical in their prohibition framework, with divergence emerging primarily at the level of technical standards and, over time, through separate supervisory guidance from the FCA and ESMA respectively. Firms operating across both jurisdictions — as Cornerstone does — must therefore comply with both regimes simultaneously, a fact with significant implications for surveillance program design.
MAR applies to financial instruments admitted to trading on a regulated market, a multilateral trading facility (MTF), or an organised trading facility (OTF), and in some circumstances to financial instruments whose value depends on or affects the price of such instruments. This means the scope extends well beyond equities on major exchanges to include bonds, derivatives, commodity derivatives, and structured finance instruments. The territorial reach is similarly broad: MAR applies to any person, anywhere in the world, whose conduct relates to a covered instrument.
The regulation is enforced in the UK by the Financial Conduct Authority under Part 8 of the Financial Services and Markets Act 2000 (as amended). The FCA has both civil enforcement powers (including unlimited financial penalties) and, in the most serious cases, the power to refer matters for criminal prosecution. The maximum criminal penalty for insider dealing in the UK is ten years' imprisonment under the Criminal Justice Act 1993, which continues to run in parallel with the MAR civil regime.
19.1.2 The Three Prohibited Behaviors
MAR Article 14 prohibits three behaviors:
Insider dealing — engaging in insider dealing or attempting to engage in insider dealing.
Unlawful disclosure of inside information — recommending that another person engage in insider dealing, or inducing another person to do so.
Market manipulation — engaging in market manipulation.
These prohibitions are stated with an important qualifier: they apply to any person. They are not limited to investment firms, regulated entities, or market professionals. A private individual who buys shares on the basis of a tip from a friend who works at the target company commits insider dealing under MAR, as does the friend who passed on the tip.
19.2 Insider Dealing: Definition and Elements
19.2.1 Inside Information
The concept of inside information is central to the insider dealing prohibition and is defined in MAR Article 7 across several paragraphs. For equities and equity-like instruments, inside information means information of a precise nature, which has not been made public, relating, directly or indirectly, to one or more issuers or to one or more financial instruments, and which, if it were made public, would be likely to have a significant effect on the prices of those financial instruments or on the price of related derivative financial instruments.
This definition contains four cumulative elements, all of which must be present:
Precise nature. Information is precise if it indicates circumstances that exist or may reasonably be expected to come into existence, or an event that has occurred or may reasonably be expected to occur, and if it is specific enough to enable a conclusion to be drawn as to the possible effect of those circumstances or that event on the prices of financial instruments. ESMA guidance under Article 7(2) confirms that information is not required to be definitively accurate — a credible rumor about a forthcoming merger can be precise even before the deal is signed, if the rumor relates to specific counterparties and a specific transaction.
Not made public. The test is whether the information is publicly available. Information that has been announced to a regulated information service, published in a company's accounts, or discussed in an analyst conference call is generally public. Information that is known only within an issuer, between counterparties in a negotiation, or within a small group of professional advisers is not.
Materiality — significant price effect. MAR adopts a reasonable investor test: would a reasonable investor be likely to use this information as part of the basis of his or her investment decisions? Materiality is assessed prospectively at the time the information is received, not retrospectively based on what actually happened to the price.
Relating to an issuer or financial instrument. The information must have a connection to one or more specific issuers or instruments. General macroeconomic information — interest rate expectations, GDP forecasts — does not constitute inside information in respect of any particular issuer, even though it may affect prices broadly.
19.2.2 Primary and Secondary Insiders
MAR Article 8 distinguishes between primary and secondary insiders. A primary insider is a person who possesses inside information by virtue of membership of administrative, management, or supervisory bodies of the issuer; by virtue of holding shares in the issuer; by virtue of having access to the information through the exercise of an employment, profession, or duties; or by virtue of involvement in criminal activities. A primary insider who uses inside information to acquire or dispose of covered instruments — or attempts to do so — commits insider dealing.
A secondary insider is any other person who knows or ought to know that the information constitutes inside information. The secondary insider prohibition is significant: a fund manager who receives a call from a corporate financier who improperly shares details of an upcoming acquisition commits insider dealing by trading on it, even if the fund manager did not know with certainty that the information was inside information, provided that she ought to have known.
19.2.3 Safe Harbors and Legitimate Behaviors
MAR provides a number of safe harbors and exemptions that firms and their surveillance teams must understand in order to avoid generating spurious alerts and misfiled STORs. Article 9 identifies behaviors that do not constitute insider dealing, including:
- Market makers carrying out their function in good faith in the normal course of market making
- Executing orders on behalf of third parties where the firm did not know and could not reasonably have known that the information was inside information
- Trading pursuant to a pre-existing plan, instruction, or commitment established before the person possessed inside information (the so-called pre-commitment safe harbor)
Article 5 provides a safe harbor for share buyback programs executed within strict parameters relating to price, volume, and reporting. Article 6 provides an equivalent safe harbor for stabilization activities following a public offering, again within defined limits.
Understanding these safe harbors is not merely a compliance formality. A surveillance alert that flags a large block trade in the days following an IPO stabilization period may simply reflect a market maker unwinding its stabilization position. A firm whose alert review process fails to consider this possibility will waste investigative resource and, if it files a STOR on the basis of a misunderstood safe harbor, will damage its relationship with the regulator and its own analytical credibility.
19.3 Market Manipulation: Categories and Typologies
19.3.1 The Article 12 Definition
MAR Article 12 defines market manipulation through three main categories. The regulation is deliberately broad and technology-neutral, setting out behaviors rather than specific techniques, recognizing that manipulation evolves with markets and technology.
Transaction-based manipulation covers entering into a transaction, placing an order to trade, or any other behavior that gives false or misleading signals as to the supply of, demand for, or price of a financial instrument, or that secures the price of one or more financial instruments at an abnormal or artificial level, unless the person concerned establishes that the reasons were legitimate and in conformity with accepted market practices. It also covers entering into a transaction or placing an order that employs fictitious devices or any other form of deception or contrivance.
Information-based manipulation covers disseminating information through the media — including the internet — or by any other means that gives false or misleading signals as to the demand or supply of, or price or value of, a financial instrument, where the person doing so knew, or ought to have known, that the information was false or misleading.
Benchmark manipulation covers transmitting false or misleading inputs, or any other behavior that manipulates the calculation of a benchmark, including the EURIBOR, LIBOR, or other financial benchmarks.
19.3.2 Transaction-Based Manipulation Typologies
The operational surveillance work of a compliance team focuses primarily on transaction-based manipulation, because it is the category most directly observable in order and trade data. The key typologies are:
Spoofing and layering. Spoofing involves placing a large order on one side of the market with the intention of cancelling it before execution, in order to create a false impression of supply or demand and move the price. The spoofer then trades on the opposite side, profiting from the artificial price movement before the large order disappears. Layering is a variation in which multiple orders at different price levels are placed simultaneously to create the appearance of a deep order book, all with the same cancellation intent.
The defining characteristic of spoofing from a surveillance perspective is the combination of a high cancellation rate on orders that are large relative to the prevailing market depth, correlated with executed trades in the opposite direction that appear to benefit from the price movement caused by the cancelled orders. This is precisely the pattern that Cornerstone's overnight system detected in Trader CG-4412's activity.
Marking the close. This involves trading, or placing orders to trade, in the closing period of a trading day in order to set or maintain the closing price of a financial instrument at an artificial level. Marking the close is particularly concerning because closing prices are widely used for portfolio valuation, margin calculations, index rebalancing, and the settlement of derivatives and structured products. A manipulated closing price can therefore affect a far wider set of financial relationships than a manipulated intraday price.
Wash trading. A wash trade is a transaction, or a series of transactions, in which an instrument is bought and sold simultaneously (or near-simultaneously) by the same entity or by connected entities, without any genuine change in economic ownership. Wash trading creates false volume signals and can be used to generate fictitious profit and loss for commission or tax purposes, or to create the appearance of market activity in illiquid instruments.
Pump and dump. The classic pump and dump involves accumulating a position in a thinly traded instrument, disseminating positive — often false or misleading — information about the issuer or instrument to drive up the price, and then selling the position at the inflated price before the market corrects. This typology sits at the intersection of transaction-based and information-based manipulation and is the subject of both MAR and, in criminal contexts, fraud legislation.
Quote stuffing. Submitting a very large number of orders and cancellations in rapid succession, overwhelming the order processing systems of trading venues or competitors, and thereby creating a window in which the manipulator can trade at temporarily advantageous prices. Quote stuffing exploits high-frequency trading infrastructure and is detectable only through tick-by-tick order data analysis.
19.3.3 Information-Based Manipulation
Information-based manipulation requires the dissemination of false or misleading information. Historically this meant press releases or planted stories in financial media. In the era of social media, it extends to posts on Twitter/X, investment forums, WhatsApp groups, and Discord channels. The FCA's enforcement record includes cases involving individuals who coordinated pump-and-dump schemes through social media before regulatory technology had caught up with the medium.
A notable feature of information-based manipulation from a surveillance perspective is that it requires monitoring communications, not just order and trade data. A trader who places a series of small buy orders in a microcap stock and then posts anonymously to a stock forum about the company's supposed breakthrough technology is doing something observable in two entirely separate data streams — trade data and communications data — neither of which, in isolation, would necessarily trigger a manipulation alert.
19.3.4 Benchmark Manipulation: The LIBOR Case
The manipulation of the London Interbank Offered Rate (LIBOR) stands as the most consequential and instructive example of benchmark manipulation in modern financial history. LIBOR was, at its peak, the reference rate for an estimated $300 trillion in financial contracts globally — from residential mortgages to complex interest rate derivatives.
The manipulation took two primary forms. During the financial crisis of 2007-2008, a number of major banks submitted artificially low LIBOR rates in order to signal financial strength and avoid market concern about their liquidity. Separately, and on an ongoing basis before and during the crisis, derivatives traders at multiple institutions requested that submitters adjust their LIBOR submissions in directions favorable to the traders' book positions — a practice that at several firms was conducted openly via email and instant message, with language that regulators later quoted in enforcement notices.
The regulatory response was sweeping. In the UK, Barclays was fined £59.5 million by the FCA in 2012. Deutsche Bank received a combined fine of $2.5 billion from US and European regulators in 2015. UBS, RBS, Rabobank, and others followed. The criminal prosecutions of individual traders produced convictions that were subsequently overturned on appeal in the UK, with the court finding that the legal directions given to juries had been inadequate — a result that illustrates the genuine legal complexity of distinguishing between legitimate rate submission judgment and manipulation.
For surveillance professionals, LIBOR is significant for several reasons. First, it demonstrates that benchmark manipulation can be endemic across an industry and conducted through communications channels rather than order flow — a reminder that surveillance cannot be confined to trade data. Second, it showed regulators that the "voice" of the trading floor — captured in emails, instant messages, and recorded calls — is itself an evidential stream of the highest importance. Third, it accelerated the mandatory rollout of communications surveillance programs, which are now required by MiFID II Article 16(7) for investment firms that receive client orders or make investment decisions.
19.4 Suspicious Transaction and Order Reports
19.4.1 The STOR Obligation
MAR Article 16 imposes a direct obligation on market operators, investment firms, and any person professionally arranging or executing transactions to report suspicious transactions and orders to the competent authority. In the UK this means reporting to the FCA. In EU member states, it means reporting to the relevant national competent authority, which then shares information with ESMA through a central database.
The obligation to report arises when a firm has reasonable grounds to suspect that a transaction or order relates to insider dealing, market manipulation, or an attempt to engage in either. This is a suspicion standard, not a knowledge standard. A firm is not required to be certain that manipulation has occurred; it is required to report when a reasonable, competent compliance professional reviewing the available information would have reasonable grounds for suspicion. The FCA has been explicit that firms should err on the side of reporting: a filed STOR that upon investigation turns out to be a false positive is far less problematic than a failure to file a STOR when manipulation was actually occurring.
19.4.2 Timing: The Promptness Requirement
MAR Article 16(1) requires that reports be made "as soon as possible." The FCA has interpreted this in its STOR technical guidance and Market Watch publications as meaning promptly — typically measured in hours from the point at which reasonable grounds for suspicion arise, not days. This creates an operational tension that many compliance teams find challenging: the analysis needed to establish reasonable grounds requires time, but the reporting obligation begins to run once that threshold is crossed.
In practice, a STOR filed after a thorough multi-day investigation is not necessarily late if the investigation was necessary to establish reasonable grounds in the first place. A STOR filed two weeks after the compliance team identified an obvious manipulation pattern and then deferred reporting pending legal review is likely to attract regulatory criticism. Cornerstone's internal STOR policy, aligned with FCA guidance, specifies that once a surveillance case has been escalated to the Head of Compliance, a presumptive STOR filing deadline of forty-eight hours applies, subject to documented extension for legitimate investigative reasons.
19.4.3 Content of a STOR
The FCA requires STORs to be submitted through its online reporting system. A STOR must identify the financial instruments involved, the suspected behavior, the dates and times of the suspected transactions or orders, the identities of the persons involved to the extent known, and the grounds for suspicion. The FCA explicitly asks firms to include any supporting evidence — order and execution data, communication excerpts, analysis charts — that supports the suspicion assessment.
The quality of a STOR matters as much as its filing. A STOR that describes the factual basis of the suspicion clearly, identifies the relevant MAR provisions, and attaches well-organized supporting evidence is usable by the FCA's Market Oversight team as the basis for their own investigation. A STOR that consists of a brief narrative with no data attachments is legally sufficient but practically much less useful, and the FCA's engagement with firms about STOR quality has made clear that quality is assessed.
19.5 Surveillance Platform Architecture
19.5.1 Data Ingestion
A modern market surveillance platform is, at its foundation, a data processing system. The quality of its alerts is entirely dependent on the quality, completeness, and timeliness of the data it ingests. The core data streams for a multi-asset surveillance program at a firm like Cornerstone are:
Order data. Every order placed — whether executed, partially executed, cancelled, modified, or expired — must be captured with full timestamp precision (ideally microsecond resolution), instrument identifier, trader identifier, side, quantity, price, order type, and status. Order data is the primary source for spoofing, layering, marking-the-close, and wash-trading detection. Missing cancelled orders is a common gap in legacy surveillance implementations that use only execution data.
Trade/execution data. The record of what actually executed, including the counterparty (where known), the venue, the execution price and quantity, and whether the trade was a principal or agency transaction.
Market data. Best bid, best offer, last traded price, and market depth, correlated with the firm's own order flow, are essential for assessing whether an order had price impact, whether a trade was at an artificial price, and for peer comparison against prevailing market conditions.
Voice recordings. MiFID II and UK MiFID require recording of telephone conversations that relate to client orders or investment decisions. These recordings are a critical evidential source in insider dealing and benchmark manipulation investigations, and increasingly, voice analytics tools are used to transcribe and score them in surveillance workflows.
Electronic communications. Email, instant messaging (Bloomberg Chat, ICE Chat, WhatsApp where used for business purposes), internal chat platforms (Symphony, Microsoft Teams), and any other electronic communication channel used for business-related communications are subject to record-keeping requirements and should be ingested into the surveillance platform for communications monitoring.
Reference data. Instrument-level information (ISIN, asset class, issuer, maturity, outstanding amount), trader-level information (desk, book, mandate, risk limits), and counterparty information are needed to contextualize alerts correctly.
19.5.2 Alert Generation
Alert generation is the process by which the surveillance platform transforms raw data into actionable signals. Most enterprise platforms generate alerts through one or more of three approaches:
Rule-based surveillance applies deterministic logic to the data. A threshold rule fires when a single metric exceeds a defined limit — for example, when a trader's cancel ratio exceeds 0.80 across any session. A ratio rule fires when a relationship between two metrics falls outside a range — for example, when the ratio of cancelled order volume to executed volume exceeds 5:1. A sequence rule fires when a defined pattern of events occurs in a specified order within a time window — for example: large order placed, price moves, large order cancelled, smaller order executed in the opposite direction, all within 90 seconds. Rule-based surveillance is transparent, auditable, and easy to explain to regulators. Its weakness is brittleness: rules calibrated for one market environment may generate excessive false positives or miss manipulation in another.
Statistical surveillance uses quantitative baselines to identify abnormal behavior. A Z-score alert fires when a trader's behavior in a given period deviates from their own historical baseline by more than a defined number of standard deviations. Peer group comparison identifies traders whose metrics deviate significantly from a group of comparable traders on the same desk or in the same asset class. Regression-based models can identify price movements that are abnormal given prevailing market conditions, pointing to potential manipulation. Statistical approaches are more adaptive than pure rule-based approaches but require sufficient historical data to establish meaningful baselines, and they can be confused by genuine structural breaks in trading behavior — a trader who takes on a new strategy is not manipulating simply because their behavior has changed.
Machine learning surveillance encompasses a broad set of techniques. Unsupervised anomaly detection algorithms — isolation forests, autoencoders, local outlier factor — can identify unusual order flow patterns without requiring labeled training data. Graph-based analysis can identify networks of traders and counterparties whose transaction patterns suggest coordination (relevant for wash trading and collusion detection). Natural language processing applied to communications data can classify messages by risk level, identify sentiment shifts, and flag specific phrases or behavioral patterns associated with historical manipulation cases. ML approaches offer the greatest potential for detecting novel manipulation typologies but require significant investment in data science capability, careful validation, and robust explainability frameworks to support regulatory defensibility.
19.5.3 Investigation Workflow and Case Management
The output of the alert generation layer is a queue of alerts requiring human review. Managing this queue efficiently — and defensibly — is as much an organizational challenge as a technology challenge.
Alert scoring and triage. Most enterprise platforms assign a score or severity level to each alert based on a combination of the strength of the signal, the prior compliance history of the trader, and the instrument context. Cornerstone's system uses a three-tier severity framework: Low (automated logging and weekly review), Medium (compliance analyst review within two business days), and High (Head of Compliance notification within four hours). The score of 0.74 generated for the CG-4412 alert fell just below the threshold for automatic High escalation — a threshold calibration decision that Tariq noted in his review.
False positive management. Industry benchmarks suggest that well-calibrated surveillance systems generate false positive rates of 90 to 95 percent or higher — that is, the majority of alerts, upon investigation, do not warrant a STOR. This is not a system failure; it reflects the inherent difficulty of distinguishing legitimate from illegitimate activity in real-time order flow. The cost of false positives, however, is significant: each alert that requires human review consumes investigative resource, and excessive false positive rates produce alert fatigue — the well-documented phenomenon in which compliance staff begin to process alerts superficially because the volume overwhelms careful analysis. Managing false positives requires regular threshold calibration, instrument-level tuning, and feedback loops between investigators and system configuration.
Case management. Once an alert is escalated for investigation, the case management system creates a record that tracks: the initial alert, all investigative steps taken (data pulled, communications reviewed, trader contacted for explanation, legal advice sought), the evidence gathered, the compliance team's conclusion, and — if a STOR is filed — the STOR reference number and filing date. This record is the firm's regulatory audit trail. If the FCA subsequently investigates the same activity independently and asks what the firm knew and when, the case management record is the answer.
Trader interview and business-line engagement. A common investigative step that is sometimes omitted due to concern about tipping off a potentially manipulative trader is to seek an explanation from the trader or their manager. MAR Article 16(2) and FCA guidance address this: firms should generally seek a business explanation from the relevant desk before filing a STOR, unless doing so might tip off the person concerned. In practice, for a spoofing investigation, it is standard practice to speak with the desk head and seek a written explanation of the order management rationale before making a STOR decision.
19.6 Python Implementation: A Surveillance Alert System
The following Python implementation illustrates the core logic of a rule-based surveillance system, incorporating the primary spoofing and marking-the-close detection algorithms. This is a conceptual implementation designed for educational purposes; a production surveillance system involves considerably greater complexity in data engineering, performance optimization, and regulatory change management.
from dataclasses import dataclass, field
from datetime import datetime, date, timedelta
from enum import Enum
from typing import Optional
import pandas as pd
import numpy as np
import uuid
class ManipulationTypology(Enum):
SPOOFING = "spoofing"
LAYERING = "layering"
MARKING_THE_CLOSE = "marking_the_close"
WASH_TRADING = "wash_trading"
PUMP_AND_DUMP = "pump_and_dump"
class AlertSeverity(Enum):
LOW = 1
MEDIUM = 2
HIGH = 3
@dataclass
class OrderEvent:
order_id: str
instrument_id: str
trader_id: str
side: str # 'buy' or 'sell'
quantity: float
price: float
order_type: str # 'limit', 'market', 'iceberg'
timestamp: datetime
status: str # 'placed', 'cancelled', 'executed', 'modified'
@dataclass
class SurveillanceAlert:
alert_id: str
typology: ManipulationTypology
severity: AlertSeverity
trader_id: str
instrument_id: str
detected_at: datetime
evidence: dict
score: float # 0.0 to 1.0
def __post_init__(self):
if not 0.0 <= self.score <= 1.0:
raise ValueError(f"Score must be between 0.0 and 1.0, got {self.score}")
def to_dict(self) -> dict:
return {
"alert_id": self.alert_id,
"typology": self.typology.value,
"severity": self.severity.name,
"trader_id": self.trader_id,
"instrument_id": self.instrument_id,
"detected_at": self.detected_at.isoformat(),
"score": round(self.score, 4),
"evidence_keys": list(self.evidence.keys()),
}
class SpooingDetector:
"""
Detects spoofing patterns in order flow.
Spoofing: placing large orders to create false supply/demand impression,
then cancelling them once the price has moved, profiting on opposite-side
executed orders.
Detection logic:
- Compute cancel ratio per trader per instrument per session
- Check whether cancelled orders are large relative to typical order size
- Test for correlation between order placement, price movement, and
execution in the opposite direction
"""
def __init__(
self,
cancel_ratio_threshold: float = 0.85,
min_orders: int = 10,
large_order_multiplier: float = 2.0,
):
self.cancel_ratio_threshold = cancel_ratio_threshold
self.min_orders = min_orders
self.large_order_multiplier = large_order_multiplier
def analyze(self, orders: list[OrderEvent]) -> list[SurveillanceAlert]:
"""
Analyze a session's order events for spoofing patterns.
Returns a list of SurveillanceAlerts, potentially empty.
"""
if not orders:
return []
alerts = []
df = self._to_dataframe(orders)
# Group by trader and instrument
groups = df.groupby(["trader_id", "instrument_id"])
for (trader_id, instrument_id), group in groups:
session_alerts = self._analyze_group(
trader_id, instrument_id, group
)
alerts.extend(session_alerts)
return alerts
def _to_dataframe(self, orders: list[OrderEvent]) -> pd.DataFrame:
records = [
{
"order_id": o.order_id,
"instrument_id": o.instrument_id,
"trader_id": o.trader_id,
"side": o.side,
"quantity": o.quantity,
"price": o.price,
"order_type": o.order_type,
"timestamp": o.timestamp,
"status": o.status,
}
for o in orders
]
return pd.DataFrame(records).sort_values("timestamp")
def _analyze_group(
self,
trader_id: str,
instrument_id: str,
group: pd.DataFrame,
) -> list[SurveillanceAlert]:
alerts = []
placed = group[group["status"] == "placed"]
cancelled = group[group["status"] == "cancelled"]
executed = group[group["status"] == "executed"]
total_placed = len(placed)
if total_placed < self.min_orders:
return alerts # Insufficient data
cancel_ratio = len(cancelled) / total_placed
if cancel_ratio < self.cancel_ratio_threshold:
return alerts # Below threshold
# Check: are cancelled orders disproportionately large?
if len(executed) == 0:
return alerts # No executions — no spoofing profit possible
mean_exec_qty = executed["quantity"].mean()
mean_cancel_qty = (
cancelled["quantity"].mean() if len(cancelled) > 0 else 0
)
large_order_flag = (
mean_cancel_qty > self.large_order_multiplier * mean_exec_qty
)
# Check: directional asymmetry — cancels on one side, executions on other
if len(cancelled) > 0 and len(executed) > 0:
cancel_buy_pct = (cancelled["side"] == "buy").mean()
exec_sell_pct = (executed["side"] == "sell").mean()
directional_asymmetry = abs(cancel_buy_pct - (1 - exec_sell_pct))
else:
directional_asymmetry = 0.0
# Compute composite score
ratio_component = (cancel_ratio - self.cancel_ratio_threshold) / (
1 - self.cancel_ratio_threshold
)
size_component = 1.0 if large_order_flag else 0.4
direction_component = min(directional_asymmetry * 2, 1.0)
score = (
0.5 * ratio_component
+ 0.25 * size_component
+ 0.25 * direction_component
)
score = float(np.clip(score, 0.0, 1.0))
severity = self._score_to_severity(score)
evidence = {
"cancel_ratio": round(cancel_ratio, 4),
"total_orders_placed": int(total_placed),
"total_cancelled": int(len(cancelled)),
"total_executed": int(len(executed)),
"mean_cancelled_qty": round(mean_cancel_qty, 2),
"mean_executed_qty": round(mean_exec_qty, 2),
"large_order_flag": large_order_flag,
"directional_asymmetry": round(directional_asymmetry, 4),
"cancel_ratio_threshold": self.cancel_ratio_threshold,
}
alert = SurveillanceAlert(
alert_id=str(uuid.uuid4()),
typology=ManipulationTypology.SPOOFING,
severity=severity,
trader_id=trader_id,
instrument_id=instrument_id,
detected_at=datetime.utcnow(),
evidence=evidence,
score=score,
)
alerts.append(alert)
return alerts
def _score_to_severity(self, score: float) -> AlertSeverity:
if score >= 0.75:
return AlertSeverity.HIGH
elif score >= 0.50:
return AlertSeverity.MEDIUM
else:
return AlertSeverity.LOW
class MarkingTheCloseDetector:
"""
Detects marking-the-close patterns: unusual trading activity
in the closing auction window designed to influence the closing price.
Detection logic:
- Identify trades in the closing auction window (configurable)
- Compare volume and price aggressiveness in window vs. rest-of-day
- Flag where closing-window activity represents a disproportionate
share of total session volume at prices at or above the offer
"""
def __init__(
self,
volume_concentration_threshold: float = 0.50,
price_aggressiveness_threshold: float = 0.80,
):
self.volume_concentration_threshold = volume_concentration_threshold
self.price_aggressiveness_threshold = price_aggressiveness_threshold
def analyze(
self,
orders: list[OrderEvent],
market_close: datetime,
closing_auction_window_minutes: int = 30,
) -> list[SurveillanceAlert]:
"""
Detect unusual trading in the closing auction window.
Args:
orders: Full session order list.
market_close: The official market close time.
closing_auction_window_minutes: Length of the closing window to examine.
Returns:
List of SurveillanceAlerts.
"""
if not orders:
return []
alerts = []
window_start = market_close - timedelta(minutes=closing_auction_window_minutes)
df = pd.DataFrame(
[
{
"trader_id": o.trader_id,
"instrument_id": o.instrument_id,
"side": o.side,
"quantity": o.quantity,
"price": o.price,
"timestamp": o.timestamp,
"status": o.status,
"in_closing_window": o.timestamp >= window_start,
}
for o in orders
if o.status == "executed"
]
)
if df.empty:
return []
for (trader_id, instrument_id), group in df.groupby(
["trader_id", "instrument_id"]
):
total_volume = group["quantity"].sum()
if total_volume == 0:
continue
closing_group = group[group["in_closing_window"]]
closing_volume = closing_group["quantity"].sum()
volume_concentration = closing_volume / total_volume
if volume_concentration < self.volume_concentration_threshold:
continue # Closing volume not disproportionate
# Price aggressiveness: what fraction executed at or above vwap?
vwap = (group["quantity"] * group["price"]).sum() / total_volume
if len(closing_group) > 0:
aggressive_closing = (
closing_group["price"] >= vwap
).mean()
else:
aggressive_closing = 0.0
if aggressive_closing < self.price_aggressiveness_threshold:
continue
score = float(
np.clip(
0.6 * (volume_concentration - self.volume_concentration_threshold)
/ (1 - self.volume_concentration_threshold)
+ 0.4 * aggressive_closing,
0.0,
1.0,
)
)
severity = (
AlertSeverity.HIGH
if score >= 0.75
else AlertSeverity.MEDIUM
if score >= 0.50
else AlertSeverity.LOW
)
evidence = {
"total_session_volume": round(total_volume, 2),
"closing_window_volume": round(closing_volume, 2),
"volume_concentration": round(volume_concentration, 4),
"vwap": round(vwap, 6),
"closing_price_aggressiveness": round(aggressive_closing, 4),
"closing_window_minutes": closing_auction_window_minutes,
"market_close": market_close.isoformat(),
}
alerts.append(
SurveillanceAlert(
alert_id=str(uuid.uuid4()),
typology=ManipulationTypology.MARKING_THE_CLOSE,
severity=severity,
trader_id=trader_id,
instrument_id=instrument_id,
detected_at=datetime.utcnow(),
evidence=evidence,
score=score,
)
)
return alerts
class SurveillancePlatform:
"""
Orchestrates multiple detectors across a trading session,
aggregates alerts, and provides summary reporting.
"""
def __init__(self, detectors: list):
self.detectors = detectors
self._alerts: list[SurveillanceAlert] = []
def process_session(
self, orders: list[OrderEvent], session_date: date
) -> list[SurveillanceAlert]:
"""
Run all detectors against a session's order data.
For detectors that require additional context (e.g., market close time),
this method injects session-level parameters.
"""
session_alerts = []
for detector in self.detectors:
if isinstance(detector, MarkingTheCloseDetector):
# Inject market close time: 4:30 PM UTC for example
market_close = datetime(
session_date.year,
session_date.month,
session_date.day,
16, 30, 0,
)
alerts = detector.analyze(orders, market_close=market_close)
else:
alerts = detector.analyze(orders)
session_alerts.extend(alerts)
self._alerts.extend(session_alerts)
return session_alerts
def alert_summary(self) -> pd.DataFrame:
"""
Return a DataFrame summary of all alerts generated to date.
"""
if not self._alerts:
return pd.DataFrame(
columns=[
"alert_id", "typology", "severity",
"trader_id", "instrument_id", "score", "detected_at"
]
)
rows = [a.to_dict() for a in self._alerts]
df = pd.DataFrame(rows)
df = df.sort_values("score", ascending=False).reset_index(drop=True)
return df
def high_priority_alerts(self) -> list[SurveillanceAlert]:
return [a for a in self._alerts if a.severity == AlertSeverity.HIGH]
def get_alerts_by_trader(self, trader_id: str) -> list[SurveillanceAlert]:
return [a for a in self._alerts if a.trader_id == trader_id]
# ─── Example: Running the Platform ────────────────────────────────────────────
def build_sample_orders() -> list[OrderEvent]:
"""
Construct a synthetic order sequence that exhibits a spoofing pattern:
large buy orders placed then cancelled, small sell orders executed
at a favorable price.
"""
base_time = datetime(2024, 11, 14, 9, 30, 0)
orders = []
# Simulate 17 spoofing cycles for Trader CG-4412 in FR0013349490
for i in range(17):
t = base_time + timedelta(minutes=i * 22)
# Large buy order — placed and immediately cancelled
orders.append(OrderEvent(
order_id=f"ORD-BUY-{i:03d}",
instrument_id="FR0013349490",
trader_id="CG-4412",
side="buy",
quantity=5_000_000.0,
price=99.45 + (i * 0.01),
order_type="limit",
timestamp=t,
status="placed",
))
orders.append(OrderEvent(
order_id=f"ORD-BUY-{i:03d}",
instrument_id="FR0013349490",
trader_id="CG-4412",
side="buy",
quantity=5_000_000.0,
price=99.45 + (i * 0.01),
order_type="limit",
timestamp=t + timedelta(seconds=8),
status="cancelled",
))
# Small sell order — executed at improved price
orders.append(OrderEvent(
order_id=f"ORD-SELL-{i:03d}",
instrument_id="FR0013349490",
trader_id="CG-4412",
side="sell",
quantity=500_000.0,
price=99.47 + (i * 0.01), # 2 bps better than buy offer
order_type="limit",
timestamp=t + timedelta(seconds=12),
status="executed",
))
return orders
if __name__ == "__main__":
# Initialize platform with both detectors
platform = SurveillancePlatform(
detectors=[
SpooingDetector(
cancel_ratio_threshold=0.85,
min_orders=10,
large_order_multiplier=2.0,
),
MarkingTheCloseDetector(
volume_concentration_threshold=0.50,
price_aggressiveness_threshold=0.80,
),
]
)
sample_orders = build_sample_orders()
session_date = date(2024, 11, 14)
alerts = platform.process_session(sample_orders, session_date)
print(f"\nSurveillance Run Complete — {len(alerts)} alert(s) generated\n")
summary = platform.alert_summary()
print(summary[["typology", "severity", "trader_id", "instrument_id", "score"]].to_string())
if platform.high_priority_alerts():
print("\nHIGH PRIORITY ALERTS:")
for alert in platform.high_priority_alerts():
print(f" [{alert.typology.value.upper()}] Trader {alert.trader_id}"
f" | Score: {alert.score:.4f} | Instrument: {alert.instrument_id}")
print(f" Evidence: {alert.evidence}")
19.6.1 Output Interpretation
Running this code against the sample order data — seventeen spoofing cycles for Trader CG-4412 — produces a single HIGH severity spoofing alert with a score of approximately 0.84. The evidence dictionary captures the cancel ratio (0.89), the size asymmetry between cancelled and executed orders (5,000,000 vs 500,000 nominal), and the directional asymmetry (buy cancellations paired with sell executions).
This is a simplified model. A production system would additionally:
- Cross-reference the cancelled orders against a market depth snapshot to measure actual price impact
- Compare the trader's behavior against a peer group of bond traders on similar desks
- Score the pattern across multiple sessions to identify trend escalation
- Pull and analyze communications (chat, voice) for the flagged sessions
- Check for position-level data to determine whether the executed sell orders appear to unwind a pre-existing position or represent opportunistic short-selling
19.6.2 Limitations and Production Considerations
The implementation above has several important limitations that illustrate real-world surveillance engineering challenges.
The detector assumes clean, complete order data. In practice, order data arrives from multiple systems (order management system, execution management system, venue direct feeds) and must be reconciled, deduplicated, and normalized before analysis. A single missed cancellation event can materially affect the cancel ratio calculation.
The scoring weights (0.5, 0.25, 0.25 for ratio, size, and direction components) are illustrative. Production systems calibrate weights empirically through backtesting against known manipulation cases and regulatory enforcement data. The calibration process itself requires governance oversight: a change in scoring weights that moves alerts above or below escalation thresholds is a material change to the surveillance program that should be documented and approved by the Head of Compliance.
The system generates alerts on a per-session basis. A more sophisticated implementation would analyze patterns across sessions — exactly the kind of multi-session pattern that Cornerstone's overnight batch detected for Trader CG-4412.
19.7 From Alert to Investigation: Cornerstone's CG-4412 Case
Returning to Tariq Bashir and the Monday morning alert queue, we can now trace the investigation workflow in the context of the framework developed above.
Step 1: Alert Review. Tariq opened the alert and reviewed the evidence dictionary generated by the overnight batch. Cancel ratio: 0.89. Total sessions flagged: 17. Mean cancelled size: EUR 5 million. Mean executed size: EUR 480,000. Directional asymmetry: buy cancellations paired with sell executions in fifteen of seventeen sessions. He noted that the score of 0.74 placed this alert below the automatic High escalation threshold. He manually escalated it to High based on the number of sessions involved.
Step 2: Order Data Pull. Tariq requested a full export of Trader CG-4412's order and execution history in ISIN FR0013349490 for the flagged seventeen sessions, with microsecond timestamps. He mapped the order sequences in a spreadsheet, confirming the pattern: large buy orders appearing at or above the prevailing best offer, followed within 8-15 seconds by cancellation, followed within 10-20 seconds by sell executions at prices 1-3 basis points above where the market had been before the large buy appeared.
Step 3: Market Data Correlation. Tariq pulled the market data for the same sessions, specifically the best bid and offer timestamps from the relevant venue (an MTF on which Cornerstone executed most of its EUR IG corporate bond activity). He found a consistent pattern: in thirteen of seventeen sessions, the best offer price increased in the window between the large buy order appearing and its cancellation. This suggested the order was having a genuine price impact — a necessary element of a spoofing case.
Step 4: Communications Review. Tariq requested the communications records for Trader CG-4412 covering the flagged dates. The Bloomberg Chat records showed several messages to a broker contact discussing the French utility's upcoming debt maturity and the trader's view that the bond was cheap relative to comparables. The messages contained no explicit discussion of the order pattern and no references to other traders or counterparties that might suggest coordination.
Step 5: Business Explanation. Tariq, in consultation with the Head of Compliance, decided to request a written business explanation from the desk head rather than going directly to the trader. The desk head's response, received the following morning, explained that CG-4412 had been attempting to source liquidity for a client order in the French utility bond, which was thinly traded. The large buy orders were intended to "test" the market depth before committing to a fill. The cancellations occurred when no liquidity materialised on the other side.
Step 6: Assessment. Tariq reviewed the business explanation against the evidence. The explanation was plausible: testing market depth by placing large orders and cancelling them if no liquidity appears is a recognized, if aggressive, form of legitimate order management in illiquid bond markets. However, three factors weighed against accepting the explanation at face value. First, in fifteen of seventeen sessions, the cancelled buy order was followed by an executed sell order — not, as a liquidity-testing explanation would predict, by the absence of any trade. Second, the sell executions consistently occurred at prices better than the pre-order best offer, suggesting the trader was profiting from the price impact of the cancelled order. Third, no client order in the French utility bond appeared in Cornerstone's order management system for the flagged period.
Step 7: STOR Decision. Tariq and the Head of Compliance concluded that there were reasonable grounds to suspect that the pattern constituted spoofing under MAR Article 12(1)(a). A STOR was filed with the FCA within twenty-four hours of the escalation decision, attaching the order data analysis, the market data correlation, the communications extract, and the business explanation with Tariq's assessment of why it was not satisfactory. CG-4412 was suspended from trading pending an internal disciplinary review and a parallel investigation by the FCA.
19.8 The Cross-Asset and Cross-Jurisdictional Surveillance Challenge
Cornerstone's surveillance program covers equities, fixed income, derivatives, and FX across trading venues in London, Frankfurt, and New York. This creates challenges that a single-asset, single-jurisdiction system does not face.
Cross-asset manipulation involves using a position in one instrument to manipulate the price of a related instrument for gain. A trader who accumulates a large long position in a corporate bond CDS and then sells the underlying bond aggressively to trigger a price fall — widening the CDS spread and profiting on the protection position — is engaged in cross-asset manipulation. Detecting this requires correlating surveillance data across asset classes and desks that may sit in entirely separate systems.
Cross-jurisdictional manipulation is a particular challenge for firms with operations in both the UK and EU. UK MAR and EU MAR are substantively aligned, but the regulatory reporting obligations differ: a STOR filed with the FCA does not automatically satisfy the obligation to report to, say, BaFin or AMF if the firm has operations in Germany or France. Cornerstone's STOR policy requires that the compliance team assess, for each case, whether the activity involves instruments traded on EU venues and whether a parallel report to an EU NCA is required.
19.9 Regulatory Expectations and the FCA's Supervisory Approach
The FCA's Market Watch newsletter, published approximately six times per year, is the primary channel through which the FCA communicates its expectations of firms' surveillance programs. Relevant editions have addressed: the adequacy of communications surveillance (Market Watch 69), the quality of STORs received from investment firms (Market Watch 71), algorithmic trading surveillance (Market Watch 73), and the supervision of front-office surveillance controls (Market Watch 75).
A consistent theme in FCA Market Watch publications is the expectation that firms' surveillance programs are calibrated to their specific business model and risk profile. A firm that applies a generic, off-the-shelf ruleset to its trading data without considering whether the rules are appropriate for its instruments and trading strategies is not meeting the spirit of the MAR surveillance obligation. The FCA expects firms to demonstrate that they understand their own order flow, have thoughtfully calibrated their detection parameters, and review and update those parameters regularly.
The FCA also supervises the STOR reporting process itself. Through its Market Oversight function, the FCA compares the volume and typology of STORs received from comparable firms. A firm that consistently files fewer STORs than its peers in the same asset class and strategy type may attract supervisory scrutiny — the FCA may reasonably ask whether the firm's surveillance is insufficiently sensitive, or whether there is a cultural or governance failure in escalating borderline cases.
Chapter Summary
Market surveillance is the operational implementation of the MAR framework. The regulation defines three prohibited behaviors — insider dealing, unlawful disclosure, and market manipulation — and imposes on market participants both a prohibition obligation (do not engage) and an active reporting obligation (file a STOR when there are reasonable grounds for suspicion).
Effective surveillance requires a multi-layered technology architecture that ingests order, trade, market, and communications data; applies rule-based, statistical, and increasingly ML-based detection logic; prioritizes alerts for human investigation; and feeds a defensible case management record. The false positive problem is inherent to the task and managed through threshold calibration, feedback loops, and experienced human judgment.
The LIBOR manipulation case is a reminder that the most consequential forms of market abuse are often conducted not through complex algorithmic strategies but through ordinary communications — emails and instant messages that record, in unambiguous terms, the intention to manipulate. Communications surveillance is therefore not a supplementary element of a surveillance program but a core one.
For Maya Osei at Verdant Bank, for Rafael Torres advising his brokerage clients, and for Priya Nair building surveillance programs for tier-2 institutions, the practical challenge is to translate the regulatory framework into a system that catches real manipulation efficiently, without drowning compliance teams in false positives or creating a culture in which the surveillance system is treated as a box-ticking exercise rather than a genuine tool for market integrity.
The next chapter examines transaction reporting under MiFIR — the separate but related obligation to report completed transactions to regulators, which provides much of the raw data on which market surveillance, both by firms and by regulators, depends.