Case Study 18-1: The Uber Self-Driving Car Fatality — Accountability in Autonomous Systems

Overview

On the night of March 18, 2018, a Volvo XC90 modified by Uber's Advanced Technologies Group (ATG) was traveling northbound on North Mill Avenue in Tempe, Arizona. The vehicle was operating in autonomous mode, with a human safety driver — Rafaela Vasquez — in the driver's seat as required by Arizona's testing permit. At approximately 9:58 PM, Elaine Herzberg, 49, was crossing the road at a location that was not a marked crosswalk, pushing a bicycle laden with bags.

The vehicle's automated systems detected an object in the road approximately 5.6 seconds before impact. The system went through a series of misclassifications, ultimately projecting that the object would not be in the vehicle's path. No emergency braking was applied. The vehicle struck Herzberg at approximately 43 miles per hour. She died of her injuries at a local hospital.

What followed was one of the most consequential accident investigations in autonomous vehicle history — a process that revealed systemic safety failures far more extensive than the single tragic event.

The NTSB Investigation: A Catalogue of Failures

The National Transportation Safety Board published its final report on the Herzberg fatality in November 2019. The report is notable not just for what it found, but for how many distinct failures it documented at how many distinct levels.

The Automated System's Failures

The Uber ATG system — called Artemis — used LIDAR (light detection and ranging), cameras, and RADAR to detect and classify objects in the environment. The investigation found that the system detected Herzberg approximately six seconds before impact but cycled through multiple object classifications during that interval: unknown object, vehicle, bicycle, unknown object again. The system's classification algorithm was not designed to handle ambiguous objects that changed classification over time.

More critically, the investigation found that Uber's engineers had disabled the automatic emergency braking (AEB) functionality in the modified Volvo. The Volvo XC90's production systems included Volvo's City Safety AEB, which was capable of automatically stopping the vehicle for pedestrians in its path. Uber had disabled this system to prevent it from interfering with the autonomous vehicle software. The Artemis system had its own emergency braking capability, but this had also been suppressed during testing because frequent emergency braking was creating problematic "false positives" that degraded the riding experience and, Uber engineers argued, the data quality of testing.

The NTSB found that if either the Volvo OEM system or the Artemis emergency braking system had been enabled, the collision would likely have been averted or its severity significantly reduced. The decision to disable emergency braking to improve testing metrics — a decision that traded pedestrian safety for data quality — is among the most damning findings in the report.

The Artemis system also lacked a reliable capability to classify an object as a person or pedestrian crossing the road in an unexpected location. The system had been trained primarily on scenarios in which pedestrians were in marked crosswalks or on sidewalks. Herzberg was crossing mid-block, which was an edge case the system was not designed to handle reliably.

The Safety Driver Failure

Rafaela Vasquez was present in the vehicle as the safety driver — the human backup required to monitor the road and intervene if the autonomous system failed. The NTSB found that she was looking at her mobile phone, which was streaming video from the Hulu app, during the forty-three seconds before impact. She was looking down at the phone at the moment of impact and did not apply the brakes before the collision.

Vasquez's distraction was an individual failure. She failed to do what she was employed to do: monitor the road and intervene when the autonomous system encountered a hazard. But the investigation revealed that her distraction was not unusual — it was part of a systemic pattern.

Uber's monitoring of safety driver behavior included a camera system that tracked driver eye movements and could detect inattention. Data from this system showed that Vasquez had been looking away from the road for a significant portion of the drive preceding the collision. But Uber's internal protocols for safety driver monitoring had not generated any response to this pattern — not during that night's testing, and not across the fleet.

Uber's safety driver training and protocols did not adequately address mobile phone use. The company's testing program guidelines discouraged safety driver phone use, but the investigation found that phone use was widespread among safety drivers and that Uber's monitoring and enforcement systems were inadequate to detect or deter it. This made Vasquez's distraction not an outlier, but a predictable product of the organization's safety culture.

The Organizational Failures

The investigation revealed that Uber's safety culture in its ATG division was deeply inadequate. Several findings are particularly striking:

Safety reporting failures. The NTSB found that Uber had experienced 37 disengagements in the six months before the Herzberg collision — instances in which safety drivers had to take manual control of the vehicle. These data were being tracked internally, but the company lacked adequate systems for analyzing safety trends and escalating concerns.

Competitive pressure overriding safety. Internal communications and testimony reviewed by the NTSB indicated that Uber's ATG division was operating under significant competitive pressure from Waymo (Google's autonomous vehicle project), which had made rapid progress. This pressure contributed to an organizational culture that prioritized deployment milestones over safety validation.

Safety plan deficiencies. The NTSB reviewed Uber's ATG Safety Plan — the company's documentation of how it managed safety in its autonomous vehicle program. The investigation found that the plan was inadequate in multiple respects: it lacked clear criteria for deployment decisions, did not establish robust processes for investigating safety incidents, and did not create adequate accountability structures for safety-critical decisions.

Emergency braking suppression. The decision to disable emergency braking — which the NTSB identified as a critical contributing factor to the fatality — was made by Uber engineers and approved within the ATG division without adequate safety review. No evidence emerged that the decision was reviewed by an independent safety function, a safety committee, or senior leadership with a safety mandate.

The Regulatory Failures

Arizona had actively competed to attract autonomous vehicle testing companies, promoting itself as a regulatory-light environment for AV testing. The NTSB found significant failures in Arizona's oversight framework:

Inadequate permit requirements. Arizona's framework for issuing autonomous vehicle testing permits did not require companies to demonstrate that their systems met any minimum safety standard before testing on public roads. The permit Uber received was administrative, not a safety certification.

No testing protocol requirements. Arizona did not require autonomous vehicle operators to file or comply with any testing protocols, training requirements for safety drivers, or system performance standards. Companies were essentially self-regulating on safety during public road testing.

Limited post-incident oversight. Arizona's response to the Herzberg collision — suspending Uber's permit and later reinstating it — was made without access to the full NTSB investigation findings, because the investigation was ongoing. The state lacked the technical expertise to independently evaluate Uber's safety practices.

At the federal level, the NTSB found that NHTSA — the National Highway Traffic Safety Administration — had not established mandatory safety standards or testing protocols for autonomous vehicle testing on public roads. NHTSA had been developing voluntary guidelines for autonomous vehicles, but these were advisory only. The agency had explicitly declined to exercise its regulatory authority to establish mandatory AV testing standards, in part out of concern about stifling innovation.

The result was a situation in which Uber's autonomous vehicle testing occurred on public roads — with real pedestrians, cyclists, and other drivers as involuntary participants in the test — without any external validation that the system met minimum safety standards, any mandatory reporting of safety incidents, or any mechanism for regulators to identify safety problems before a fatality occurred.

Civil Liability

Uber settled the civil wrongful death lawsuit brought by Herzberg's family for an undisclosed amount, generally reported to be in the millions of dollars, before any judgment on the merits. The settlement included no admission of liability. This outcome is typical in high-profile wrongful death cases: the financial exposure is large enough that settlement is rational for both parties, and the absence of an admission of liability means that the settlement does not create a legal precedent that could be used in future cases.

The settlement meant that no court ever ruled on whether Uber was liable for Herzberg's death, what legal theories would have applied, or what the relevant standard of care for autonomous vehicle operators should be. From an accountability perspective, the settlement resolved the immediate financial dispute without producing legal clarity about corporate responsibility for autonomous vehicle fatalities.

Criminal Proceedings Against the Safety Driver

In September 2020, Yavapai County prosecutors charged Rafaela Vasquez with negligent homicide — the criminal charge most directly applicable to a situation of fatal negligence. The case proceeded slowly, partly due to COVID-19 court delays. In July 2023, Vasquez reached a plea agreement. She pleaded guilty to endangerment, a lesser charge, and was sentenced to three years of probation. She did not face prison time.

This outcome illustrates both the appropriateness and the limitations of individual criminal accountability in systemic failures. Vasquez was genuinely at fault: she failed to monitor the road, which was the essential function of her presence in the vehicle. Her prosecution was legally appropriate.

But the outcome also reflects the "few bad apples" problem. The systemic failures — disabled emergency braking, inadequate safety protocols, insufficient safety driver training and monitoring, regulatory permissiveness — were not subjects of criminal prosecution. The safety engineers who disabled the emergency braking system, the managers who approved deployment with known system limitations, the executives who set the competitive pressure that distorted safety culture — none faced criminal charges. The criminal justice system reached the individual at the bottom of the organizational hierarchy and stopped there.

Absence of Corporate Criminal Accountability

No Uber employee other than Vasquez faced criminal prosecution. This is legally explicable — it is difficult to identify specific individuals whose specific decisions constituted criminal negligence in ways that support prosecution — but it is ethically troubling. The organizational decisions that contributed to Herzberg's death were made by identifiable people with identifiable responsibilities. The decision to disable emergency braking was made by specific engineers and approved by specific managers. The decision to deploy a system with known limitations was made by specific executives. The failure to enforce safety driver protocols was a choice made by specific safety managers.

Corporate criminal liability for organizational negligence exists in law — it is used regularly in financial fraud cases — but it has rarely been applied to technology companies for product safety failures. The absence of any attempt to hold Uber as a corporation criminally liable, or to prosecute the specific individuals whose decisions created the dangerous conditions, reflects both legal culture and political economy: technology companies are politically powerful, safety regulation of AI systems is nascent, and prosecutors face high burdens of proof.

NTSB Safety Recommendations

The NTSB issued a series of safety recommendations following its investigation:

  • To NHTSA: develop mandatory requirements for testing autonomous vehicles on public roads.
  • To NHTSA: require companies to submit safety self-assessments before autonomous vehicle testing.
  • To Uber: improve safety management systems for autonomous vehicle testing.
  • To Arizona: develop more robust requirements for AV testing permits.

NTSB recommendations are advisory only — the NTSB has no enforcement authority and cannot compel compliance. NHTSA did not implement mandatory AV testing standards following the recommendation, a decision the NTSB later rated as "Open — Unacceptable Response."

What Structural Changes Followed

The Herzberg fatality had several structural effects on the autonomous vehicle industry and its regulation, though fewer than the severity of the failure would have warranted.

Uber ATG. Uber suspended all autonomous vehicle testing for several months following the collision. When testing resumed, it was with enhanced safety protocols, mandatory safety driver training, and — eventually — re-enabling of emergency braking. In 2020, Uber sold its ATG division to Aurora, a self-driving startup, for approximately $4 billion.

Industry practice. Several other autonomous vehicle companies — Waymo, Cruise, GM — adopted more conservative testing protocols in the aftermath of the Herzberg case. The industry's formal commitment to safety improved, though whether this reflected genuine cultural change or reputation management is difficult to determine from the outside.

Arizona regulation. Arizona enacted modest new requirements for autonomous vehicle testing permits in 2019, including requirements for companies to submit safety plans and to report accidents involving autonomous vehicles. These requirements were more substantive than what existed before the Herzberg case but fell short of what the NTSB had recommended.

NHTSA. The federal government did not implement mandatory pre-deployment testing standards for autonomous vehicles following the Herzberg case. NHTSA's approach remained voluntary guidance rather than mandatory standards through at least 2024. Congress failed to pass legislation that would have established a federal regulatory framework for autonomous vehicles, largely due to disagreements about the appropriate scope of federal preemption of state vehicle safety law.

Public Trust. The Herzberg case significantly damaged public trust in autonomous vehicles, contributing to caution about the technology that has persisted. Public opposition to AV testing on public roads increased substantially following the case, creating political pressure for more careful rollout even in the absence of mandatory standards.

Analysis: What This Case Reveals About AI Accountability

The Herzberg case illustrates several of the accountability problems discussed in Chapter 18.

The many hands problem is clearly visible: the fatality resulted from failures at the level of the AI system design, the safety driver, the organizational safety culture, the regulatory framework, and the federal government's choice not to establish mandatory standards. No single party was sufficient cause of Herzberg's death; every party contributed. The legal system, not designed for this kind of distributed causation, produced modest accountability for the individual at the bottom of the hierarchy and minimal accountability for the organizational and systemic failures higher up.

The innovation bias in regulation is clearly visible: Arizona's explicit strategy was to attract autonomous vehicle companies by minimizing regulatory friction. The costs of this strategy — borne by Elaine Herzberg — were concentrated on individuals, while the economic benefits were diffuse (jobs, tax revenue, technology sector investment). This asymmetry between concentrated harms and diffuse benefits is a classic condition for under-regulation.

The Collingridge dilemma is visible: by the time Uber's safety failures were apparent, the company had invested billions in its ATG division and had substantial political and economic incentives to continue. Early regulation — before deployment, when harms were speculative — would have been contested on the grounds that it was restricting innovation based on hypothetical risks. After the fatality, the political economy of the situation made even modest regulation contentious.

The ethics washing problem is visible in Uber's safety documentation. The company had a Safety Plan — the formal document that described how it managed safety in its AV program. The NTSB found this plan to be substantively inadequate. But the existence of the plan allowed Uber to claim a safety commitment that its actual practices did not support. Ethics washing through documentation — creating governance artifacts that provide the appearance of accountability without the substance — is a recurring pattern in AI governance.

Questions for Discussion

  1. The NTSB found that Uber had disabled emergency braking to reduce "false positives." This was an engineering tradeoff: reducing unnecessary stops to improve testing quality. How should engineers navigate tradeoffs between testing efficiency and safety when testing is conducted with involuntary participants (members of the public)?

  2. Rafaela Vasquez was the only person charged with a crime related to Herzberg's death. Do you find this outcome appropriate? If not, who else should have faced criminal accountability, and under what legal theory?

  3. Arizona competed with other states to attract autonomous vehicle companies by minimizing regulatory requirements. What arguments would you make to state legislators considering whether to adopt a similar strategy today?

  4. The NTSB's safety recommendations had no legal force. What institutional changes would be needed to give investigative recommendations more authority? What would be the costs and risks of granting the NTSB enforcement authority?

  5. Uber settled civil claims without admitting liability, preventing any court from ruling on the applicable legal standard. What are the systemic implications of this pattern — high-profile AI liability cases settled before judgment — for the development of AI liability law?