Case Study 2: When Hospital Ratings Kill Patients

The Paradox of Quality Measurement

Hospital quality rating systems were created with the best of intentions: to give patients information, hold providers accountable, and drive improvement. The logic seemed unassailable: measure quality, publish the results, and competition will drive everyone to improve.

The reality has been more complex — and in some cases, the measurement systems designed to save lives may be costing them.

The Cardiac Surgery Scorecard

In the early 1990s, New York State began publicly reporting risk-adjusted mortality rates for cardiac surgeons and hospitals. The initiative was a landmark in healthcare transparency. For the first time, patients could see how their surgeon compared to peers.

The published mortality rates declined impressively. The system appeared to be working.

But closer examination revealed troubling patterns:

Risk selection increased. Surgeons and hospitals began declining high-risk patients — patients who needed surgery the most but who threatened to worsen the institution's mortality statistics. Studies documented that the most critically ill cardiac patients in New York were increasingly referred to out-of-state facilities or denied surgery altogether.

Coding practices shifted. Hospitals learned to code patients as sicker than they were at admission (increasing the "expected" mortality and making actual mortality look better by comparison). This "upcoding" improved risk-adjusted scores without improving actual care.

Access decreased for the sickest patients. The patients most in need of cardiac surgery were the ones most likely to be turned away — because they were the ones who posed the greatest risk to institutional statistics.

The Construct-Proxy Gap

Construct Proxy Gap
Quality of cardiac care Risk-adjusted mortality rate Mortality rate incentivizes patient selection, not quality improvement
Patient wellbeing Patient satisfaction scores Satisfaction correlates with unnecessary prescriptions
Hospital safety Hospital-acquired infection rates Incentivizes not testing for infections
Timely care Emergency department wait times Incentivizes undertriage and hallway care

In each case, the proxy captures a real dimension of quality. And in each case, making the proxy a high-stakes target distorts behavior in ways that can harm the very patients the system is designed to protect.

A Specific Case (Tier 3 — Composite Illustration)

Consider Dr. Elena Vasquez (composite case, based on patterns documented in the literature). She is a cardiac surgeon at a major teaching hospital. She specializes in the most complex cases — patients with multiple comorbidities, failed prior surgeries, or unusual anatomies. Her outcomes, given the difficulty of her cases, are excellent. She is one of the most skilled surgeons in the region.

But her publicly reported mortality rate is higher than average — because she takes the cases no one else will take. In a system that publishes mortality rates, Dr. Vasquez faces a choice: continue taking the hardest cases (and appear worse than her peers on the scorecard) or decline them (and appear better). The measurement system punishes her for doing the most difficult, most important work.

Some variant of this dilemma affects every provider in a publicly rated system. And the rational response — declining high-risk patients — is a form of metric optimization that directly harms the patients who most need help.

Structural Lessons

  1. Quality metrics can reduce quality. This is not a paradox but a predictable consequence of Goodhart's Law applied to healthcare.

  2. Risk adjustment doesn't solve the problem. Even with sophisticated risk adjustment, the incentive to select low-risk patients persists — because no risk model perfectly captures patient complexity.

  3. The most vulnerable patients bear the cost. When metrics drive patient selection, the sickest patients — who are disproportionately poor, elderly, and members of minority groups — are the ones most likely to be turned away.

  4. The measurement system's success is self-reported. When mortality rates decline under a public reporting system, the decline may reflect genuine improvement, risk selection, coding changes, or a combination. The measurement system cannot distinguish between these explanations.

Discussion Questions

  1. Is hospital quality measurement net positive or net negative? Can the benefits be preserved while reducing the harms?
  2. Design a hospital quality measurement system that minimizes the risk selection problem. What trade-offs does your system involve?
  3. Compare Dr. Vasquez's dilemma to the school principal's dilemma in section 4.6. What structural features do they share?
  4. Should patients have access to individual surgeon mortality rates? Argue both sides.

References

  • Dranove, D. et al. (2003). "Is More Information Better? The Effects of 'Report Cards' on Health Care Providers." Journal of Political Economy, 111(3), 555–588. (Tier 1)
  • Research on the effects of public reporting on risk selection in cardiac surgery has been published in multiple cardiology and health policy journals. (Tier 2)
  • Muller, J. Z. (2018). The Tyranny of Metrics. Princeton University Press. (Tier 1 — includes healthcare chapters)
  • The New York State cardiac surgery reporting system is documented in annual reports from the New York State Department of Health. (Tier 2)