Case Study 1: Automated Offside Detection -- How VAR Technology Works

Background

The offside rule is among the most consequential and contentious in soccer. A player is in an offside position if any part of their body (excluding hands and arms) that can legally play the ball is nearer to the opponents' goal line than both the ball and the second-to-last opponent at the moment the ball is played by a teammate. Historically, this call was made by assistant referees running along the sideline, judging a three-dimensional spatial relationship in real time from a single vantage point. The error rate was substantial.

The introduction of Video Assistant Referee (VAR) technology, and subsequently Semi-Automated Offside Technology (SAOT), has transformed offside decision-making from a human judgment call into a computer vision problem. This case study examines the technical architecture behind automated offside detection, as deployed at the 2022 FIFA World Cup and subsequently adopted by major European leagues.

The Technical Challenge

Automated offside detection requires solving several interrelated CV problems simultaneously:

  1. Precise temporal synchronization: Identifying the exact frame when the ball is played (the "trigger moment").
  2. Player limb detection: Determining the position of the furthest forward playable body part for the potentially offside attacker.
  3. Defensive line detection: Determining the position of the second-to-last defender's furthest back playable body part.
  4. Calibration: Mapping all detected positions to a common, geometrically accurate pitch coordinate system.

The margin of error is extraordinarily small. In elite soccer, offside decisions can hinge on distances of 1--5 centimeters. This demands sub-pixel accuracy in detection and sub-frame accuracy in temporal synchronization.

System Architecture

Hardware Setup

The SAOT system used at the 2022 World Cup employed:

  • 12 dedicated tracking cameras mounted beneath the stadium roof, providing full-pitch coverage at high frame rates (50 fps).
  • A sensor embedded in the match ball (the Adidas Al Rihla) containing an inertial measurement unit (IMU) transmitting data at 500 Hz, used to detect the precise moment of a pass or kick.
  • Limb-tracking technology that identifies 29 skeletal points per player at 50 times per second.

The ball sensor is critical because it resolves the temporal ambiguity: the ball's IMU data can pinpoint the moment of foot-to-ball contact with sub-millisecond accuracy, far exceeding what frame-based analysis alone could achieve.

The Pipeline

Ball IMU Sensor (500 Hz)
    |
    v
[1] Kick Detection
    |-- Detect sudden acceleration (ball struck)
    |-- Identify exact timestamp of ball release
    |
    v
[2] Frame Selection
    |-- Select the tracking frame closest to the kick timestamp
    |-- Interpolate player positions if kick falls between frames
    |
    v
[3] Player Limb Detection (per selected frame)
    |-- For each player, detect 29 body points
    |-- Identify the furthest-forward playable body part
    |-- Exclude hands and arms from consideration
    |
    v
[4] Defensive Line Computation
    |-- Identify the second-to-last defender
    |-- Compute the offside line from their furthest-back body part
    |
    v
[5] Offside Determination
    |-- Compare attacker's forward-most point to the offside line
    |-- Apply a tolerance margin (if used)
    |-- Generate visual output for review
    |
    v
[6] Visualization
    |-- 3D animation showing player positions
    |-- Offside line overlay on broadcast footage
    |-- Alert sent to VAR operator

Mathematical Formulation

Let $t^*$ be the moment the ball is played (detected by the ball sensor). At time $t^*$, for each player $i$, the system determines the set of body point positions:

$$\mathcal{B}_i = \{(x_{i,k}, y_{i,k}) : k \in \text{playable body parts}\}$$

The furthest-forward position of player $i$ toward the opponent's goal line is:

$$x_i^{\text{forward}} = \max_{k} \, x_{i,k}$$

(assuming the opponent's goal is in the positive $x$ direction).

The offside line is defined by the second-to-last defender's rearmost playable body part:

$$x_{\text{offside}} = x_{d_2}^{\text{back}} = \min_{k} \, x_{d_2, k}$$

where $d_2$ is the second-to-last defender (sorted by $x$ position).

A player $a$ is offside if:

$$x_a^{\text{forward}} > x_{\text{offside}} \quad \text{and} \quad x_a^{\text{forward}} > x_{\text{ball}}(t^*)$$

Handling Uncertainty

No measurement system is perfect. The SAOT system accounts for uncertainty through:

Temporal uncertainty: The ball sensor provides kick timing to within approximately 2ms. At 50 fps, frames are 20ms apart. Linear interpolation of player positions between frames introduces positional uncertainty of:

$$\delta x_{\text{temporal}} \approx v_{\text{player}} \times \delta t \approx 8 \, \text{m/s} \times 0.01 \, \text{s} = 0.08 \, \text{m}$$

Spatial uncertainty: Limb detection has an accuracy of approximately 0.05--0.10m, depending on occlusion and camera angle.

Combined uncertainty: The total uncertainty is:

$$\delta x_{\text{total}} = \sqrt{\delta x_{\text{temporal}}^2 + \delta x_{\text{spatial, attacker}}^2 + \delta x_{\text{spatial, defender}}^2}$$

$$\delta x_{\text{total}} \approx \sqrt{0.08^2 + 0.07^2 + 0.07^2} \approx 0.13 \, \text{m}$$

This uncertainty bound informs the "tolerance" or "margin of error" that some competitions apply. If the computed offside distance is within $\delta x_{\text{total}}$, the original on-field decision stands, as the technology cannot determine the outcome with sufficient confidence.

Implementation Details

Kick Detection from Ball Sensor Data

The ball's IMU provides tri-axial acceleration data. A kick is detected as a sudden spike in the acceleration magnitude:

import numpy as np

def detect_kicks(
    acceleration: np.ndarray,
    timestamps: np.ndarray,
    threshold_g: float = 12.0,
    min_interval_ms: float = 200.0
) -> list[float]:
    """Detect kick events from ball IMU acceleration data.

    Args:
        acceleration: Tri-axial acceleration, shape (N, 3), in g units.
        timestamps: Corresponding timestamps in milliseconds, shape (N,).
        threshold_g: Minimum acceleration magnitude to register as kick.
        min_interval_ms: Minimum time between consecutive kicks.

    Returns:
        List of kick timestamps in milliseconds.
    """
    # Compute acceleration magnitude
    acc_magnitude = np.linalg.norm(acceleration, axis=1)

    # Find peaks above threshold
    kick_candidates = []
    for i in range(1, len(acc_magnitude) - 1):
        if (acc_magnitude[i] > threshold_g
                and acc_magnitude[i] > acc_magnitude[i - 1]
                and acc_magnitude[i] > acc_magnitude[i + 1]):
            kick_candidates.append(timestamps[i])

    # Apply minimum interval filter
    kicks = []
    for t in kick_candidates:
        if not kicks or (t - kicks[-1]) > min_interval_ms:
            kicks.append(t)

    return kicks

Offside Line Computation

import numpy as np

def compute_offside_line(
    defender_positions: dict[str, list[tuple[float, float]]],
    ball_position: tuple[float, float],
    attacking_direction: int = 1
) -> tuple[float, str]:
    """Compute the offside line from defender limb positions.

    Args:
        defender_positions: Dict mapping player ID to list of
            (x, y) positions for each playable body part.
        ball_position: (x, y) position of the ball.
        attacking_direction: 1 if attacking toward positive x,
            -1 if toward negative x.

    Returns:
        Tuple of (offside line x-coordinate, ID of the defining defender).
    """
    # Find the rearmost position for each defender
    defender_rear = {}
    for player_id, limb_positions in defender_positions.items():
        if attacking_direction == 1:
            # Rearmost = smallest x (furthest from attacking goal)
            rear_x = min(pos[0] for pos in limb_positions)
        else:
            rear_x = max(pos[0] for pos in limb_positions)
        defender_rear[player_id] = rear_x

    # Sort defenders by rearmost position
    sorted_defenders = sorted(
        defender_rear.items(),
        key=lambda item: item[1],
        reverse=(attacking_direction == -1)
    )

    # The offside line is defined by the second-to-last defender
    if len(sorted_defenders) >= 2:
        offside_defender_id, offside_x = sorted_defenders[1]
    else:
        # Fewer than 2 defenders -- offside line at ball position
        offside_x = ball_position[0]
        offside_defender_id = "ball"

    return offside_x, offside_defender_id


def is_offside(
    attacker_limb_positions: list[tuple[float, float]],
    offside_x: float,
    ball_x: float,
    attacking_direction: int = 1,
    tolerance: float = 0.0
) -> tuple[bool, float]:
    """Determine if an attacker is in an offside position.

    Args:
        attacker_limb_positions: List of (x, y) for each playable body part.
        offside_x: X-coordinate of the offside line.
        ball_x: X-coordinate of the ball.
        attacking_direction: 1 if attacking toward positive x.
        tolerance: Tolerance margin in meters.

    Returns:
        Tuple of (is_offside boolean, margin in meters).
    """
    if attacking_direction == 1:
        attacker_forward = max(pos[0] for pos in attacker_limb_positions)
        margin = attacker_forward - offside_x
        beyond_ball = attacker_forward > ball_x
    else:
        attacker_forward = min(pos[0] for pos in attacker_limb_positions)
        margin = offside_x - attacker_forward
        beyond_ball = attacker_forward < ball_x

    offside = margin > tolerance and beyond_ball
    return offside, margin

Real-World Performance

At the 2022 FIFA World Cup, the SAOT system:

  • Processed all 64 matches in the tournament.
  • Provided offside decisions with an average review time of approximately 25 seconds from incident to final call (compared to 70+ seconds for manual VAR offside reviews).
  • Detected limb positions accurate to approximately 5 centimeters.
  • Generated 3D visualizations that were broadcast to television audiences, significantly improving transparency.

Several notable incidents tested the system:

  1. Marginal decisions: Cases where the offside margin was under 10cm were handled consistently, with the visual output clearly showing the relevant body parts and the offside line.

  2. Clustered players: During corner kicks and free kicks with many players in close proximity, the system maintained accurate tracking despite significant occlusion.

  3. Edge cases: Situations where players were lying on the ground or in unusual body positions required the limb detection algorithm to handle non-standard poses.

Limitations and Controversies

Despite its technical sophistication, the system has faced criticism:

  1. The "tolerance" debate. Some argue that any tolerance margin is arbitrary and that the system should make binary decisions. Others argue that the inherent uncertainty demands a margin. The choice of tolerance (or absence thereof) is ultimately a policy decision, not a technical one.

  2. Defining "playable body parts." The system must correctly identify which body parts can legally play the ball (excluding hands and arms). When a player's arm extends beyond the offside line but their shoulder does not, the distinction between "arm" and "shoulder" becomes technically challenging.

  3. The moment of the pass. While the ball sensor detects foot-to-ball contact, there is a philosophical question about whether offside should be judged at the moment of contact, the moment the ball leaves the foot, or the moment the passing decision is made.

  4. Fan experience. The delay required for SAOT review, while shorter than manual VAR, still interrupts the flow of the game and the spontaneous celebration of goals.

Discussion Questions

  1. The SAOT system introduces a "tolerance margin" to account for measurement uncertainty. How should this tolerance be determined? Should it be based purely on technical measurement error, or should it also consider the spirit of the offside law?

  2. If the system's spatial accuracy improves to 1cm, should the tolerance margin be reduced accordingly? What are the arguments for maintaining a larger tolerance even with improved accuracy?

  3. How does the transparency of the 3D visualization affect public acceptance of controversial decisions? Compare this to the opaque nature of traditional VAR offside reviews using static images with manually drawn lines.

  4. Could a similar system be deployed at lower levels of professional soccer? What would need to change in terms of hardware cost, installation requirements, and operational complexity?

Connection to Chapter Concepts

This case study integrates several concepts from the chapter:

  • Object detection (Section 23.4): Player and ball detection is the foundation of the system.
  • Pose estimation (Section 23.5): Limb-level detection (29 body points) goes beyond standard bounding-box tracking to enable the precise geometric comparisons required for offside decisions.
  • Homography and calibration (Section 23.3): Accurate mapping from camera coordinates to pitch coordinates is essential for measuring centimeter-level distances.
  • Sensor fusion (Section 23.7): The combination of optical tracking (cameras) with inertial measurement (ball sensor) exemplifies the multi-modal approach discussed as a future direction.

The automated offside system represents one of the most demanding applications of CV in soccer, where the accuracy requirements are at their most stringent and the consequences of errors are immediately visible to a global audience.