Case Study: Defensive Programming in Critical Systems

The Stakes

In some software domains, a crash doesn't just inconvenience a user — it can kill someone. Healthcare systems that calculate drug dosages, aviation software that manages flight controls, banking systems that process millions of transactions per second — these systems cannot afford to crash, and they cannot afford to produce wrong answers silently.

This case study examines how three industries approach error handling and what you, as a CS1 student, can learn from their practices — even if your current project is a task manager rather than an autopilot system.

Healthcare: The Therac-25 Radiation Machine

What Happened

Between 1985 and 1987, the Therac-25 — a computer-controlled radiation therapy machine — delivered massive radiation overdoses to six patients, causing serious injuries and three deaths. The root causes included race conditions, inadequate error handling, and over-reliance on software without hardware safety interlocks.

The Error Handling Failures

The Therac-25 software had error codes, but they were cryptic numbers like "Malfunction 54" that operators couldn't interpret. Operators became accustomed to seeing error messages, dismissing them as glitches and pressing "P" to proceed. The system allowed them to override safety warnings without understanding what the warnings meant.

In Python terms, the system's error handling looked something like this:

# ANTI-PATTERN: Therac-25-style error handling (simplified)
def deliver_radiation(dose, patient_position):
    status = calibrate_beam(dose)
    if status != 0:
        # Display cryptic error code — operator usually ignores it
        display_message(f"Malfunction {status}")
        # Allow override — THIS IS THE FATAL FLAW
        if operator_presses_proceed():
            deliver_beam(dose)  # Delivers radiation despite error

What Good Error Handling Would Look Like

# Proper approach: errors in critical operations cannot be overridden
class RadiationSafetyError(Exception):
    """Critical safety error — radiation delivery must not proceed."""
    pass

def deliver_radiation(dose, patient_position):
    """Deliver radiation only if ALL safety checks pass."""
    try:
        verify_beam_calibration(dose)
        verify_patient_position(patient_position)
        verify_dose_within_limits(dose)
    except RadiationSafetyError as e:
        log_safety_event(e)
        alert_medical_physicist(e)
        raise  # Re-raise — do NOT allow override

    # Only reached if all checks pass
    deliver_beam(dose)
    log_successful_delivery(dose, patient_position)

The Lesson

Error messages must be clear, actionable, and not overridable when safety is at stake. "Malfunction 54" is useless. RadiationSafetyError("Beam calibration failed: expected 200 rads, measured 25,000 rads") tells the operator exactly what's wrong and why the machine stopped.

Aviation: Fly-by-Wire Error Handling

How Modern Aircraft Handle Errors

Modern commercial aircraft like the Airbus A320 family use "fly-by-wire" systems — the pilot's inputs are processed by computers that then control the flight surfaces. These systems handle errors using principles that map directly to what you learned in this chapter.

Redundancy and Voting

Flight-critical systems run on multiple independent computers. If three computers calculate the same flight input and two agree but one disagrees, the system uses the majority result and flags the disagreeing computer for maintenance. In Python terms:

def get_reliable_reading(sensors):
    """Get a reliable reading from redundant sensors using majority vote."""
    readings = []
    for sensor in sensors:
        try:
            readings.append(sensor.read())
        except SensorError as e:
            log_sensor_failure(sensor.id, e)
            # Continue — don't crash because one sensor failed

    if len(readings) < 2:
        raise CriticalSystemError(
            f"Insufficient sensor data: {len(readings)} of "
            f"{len(sensors)} sensors responding"
        )

    # Use median to reject outliers
    readings.sort()
    return readings[len(readings) // 2]

Graceful Degradation

When errors occur, aviation software doesn't crash — it degrades to a simpler mode of operation. An aircraft might go from "full autopilot" to "assisted manual control" to "direct manual control" as systems fail. Each degradation level is less capable but still safe.

# Aviation-style graceful degradation (simplified)
def control_aircraft(autopilot, flight_computer, manual_input):
    """Attempt highest automation level; degrade gracefully on failure."""
    try:
        return autopilot.compute_control(flight_data)
    except AutopilotError as e:
        log_degradation("autopilot", e)
        alert_crew("Autopilot disengaged. Switching to flight computer assist.")

    try:
        return flight_computer.assist(manual_input, flight_data)
    except FlightComputerError as e:
        log_degradation("flight_computer", e)
        alert_crew("Flight computer assist unavailable. Direct control active.")

    # Final fallback — direct mechanical control
    return manual_input

The Lesson

Don't design your programs with a single mode that either "works" or "crashes." Design multiple levels of capability so that when something goes wrong, the program degrades gracefully instead of failing completely.

TaskFlow v1.0 does this in a small way: if the save file is corrupted, it doesn't crash — it starts with an empty task list. That's graceful degradation.

Banking: Transaction Safety with try/finally

The Problem

When a banking system transfers money from Account A to Account B, it performs two operations: subtract from A, add to B. What if the system crashes after subtracting from A but before adding to B? The money vanishes. This is the "atomicity" problem — either both operations should happen, or neither should.

Error Handling for Atomicity

Database systems solve this with "transactions" — a pattern that uses try/except/finally logic:

def transfer_funds(from_account, to_account, amount):
    """Transfer funds atomically — all or nothing."""
    if amount <= 0:
        raise ValueError(f"Transfer amount must be positive, got {amount}")

    if from_account.balance < amount:
        raise InsufficientFundsError(
            f"Cannot transfer ${amount:.2f}: "
            f"account {from_account.id} has ${from_account.balance:.2f}"
        )

    # Begin transaction
    original_from = from_account.balance
    original_to = to_account.balance

    try:
        from_account.balance -= amount
        to_account.balance += amount
        log_transaction(from_account.id, to_account.id, amount)
    except Exception as e:
        # ROLLBACK: restore original balances
        from_account.balance = original_from
        to_account.balance = original_to
        log_failed_transaction(from_account.id, to_account.id, amount, e)
        raise TransactionError(
            f"Transfer failed and was rolled back: {e}"
        ) from e

The Lesson

When operations must happen as a group (all-or-nothing), save the original state before starting, and restore it if anything goes wrong. This is the same pattern you'll use if TaskFlow ever needs to update multiple files atomically — save the old state, attempt the update, roll back on failure.

Common Principles Across All Three Industries

Discussion Questions

1. The Therac-25 allowed operators to override safety errors. In what situations, if any, should a program allow users to override error conditions? How do you design a safe override mechanism?

2. Aviation uses redundancy (multiple computers doing the same calculation) to handle errors. Where might redundancy be useful in everyday software? What are the costs?

3. The banking example saves original state before attempting a change (the "rollback" pattern). In TaskFlow, what operations could benefit from this pattern? What state would you save?

4. All three industries require extensive logging. Your programs currently use print() for error messages. What limitations does print() have for error logging? (Hint: think about where print() output goes and how permanent it is.)

5. These are all examples of error handling in critical systems. Does error handling matter in non-critical software like a to-do app? Why or why not?

Mini-Project

Build a simple banking simulation that demonstrates the atomicity pattern:

1. Create a BankAccount class with account_id, owner, and balance attributes
2. Write a transfer(from_acct, to_acct, amount) function that: - Validates all inputs (raises appropriate exceptions) - Saves original balances before the transfer - Rolls back if anything goes wrong - Returns a transaction receipt dict on success
3. Write a batch_transfers(accounts, transfer_list) function that processes a list of transfers, handling each one's errors independently (one failed transfer shouldn't stop the rest)
4. Test with: valid transfers, insufficient funds, invalid accounts, negative amounts, and a mix of good and bad transfers in a batch
5. Print a summary: N successful, M failed, with details of each failure