Case Study 1 — Beta-Carotene: The Twenty-Year Wrong Answer

A scientific case. The trials, findings, and reversal described here are real; the framing conversation with Samir Ostrowski is illustrative.

Setup

Samir keeps a printout on the wall of his office. It's not a graph or an award. It's a single paragraph from a review article published in the late 1980s, and it says — in the confident, unhedged prose of a field that thinks it has something — that the evidence linking carotenoid intake to reduced cancer risk is consistent, strong, and biologically coherent.

He points at it when students get too confident. "That's the smartest people in the field," he says, "being completely wrong, in public, with excellent reasoning."

I asked him once why he keeps it rather than something more encouraging.

"Because everyone in this building can tell you that correlation isn't causation," he said. "And almost nobody can tell you what it feels like from the inside when it isn't. It feels like this. It feels like being right."

The evidence, as it stood

Reconstruct what a well-informed nutrition scientist knew in about 1985.

Ecological data. Populations eating more fruits and vegetables had less cancer, especially lung cancer. Observed across many countries and cultures.

Case-control studies. People with lung cancer reported eating fewer carotenoid-rich foods than matched controls. Replicated repeatedly.

Prospective cohorts. Better still — people were surveyed before getting sick, then followed. Those with higher carotenoid intake developed less lung cancer.

Biomarker data. This was the clincher, and it's worth dwelling on. Rather than relying on what people said they ate, researchers measured serum beta-carotene — an objective laboratory value, immune to the food-frequency-questionnaire problems from Chapter 1. People with higher blood beta-carotene got less lung cancer. And the association persisted after adjustment for smoking.

Dose-response. More carotenoid, less cancer, in a graded fashion. One of the features §2.10 lists as a marker of good evidence.

Mechanism. Oxidative damage to DNA contributes to carcinogenesis. Beta-carotene is an antioxidant. It quenches singlet oxygen. It's a vitamin A precursor, and vitamin A regulates cell differentiation. The story was not merely plausible — it was elegant.

Consistency. Different countries, different research groups, different funding, different methods. Convergent.

Look at that list against the §2.10 criteria for convincing evidence. Dose-response: yes. Consistency across populations: yes. Mechanism: yes. Objective biomarker rather than self-report: yes. Multiple designs: yes.

It hits nearly every criterion — and it was wrong.

That is the most uncomfortable sentence in this chapter, and I'm not going to soften it.

What happened next

Two large randomized trials were launched, because the field did the right thing: before recommending supplementation to millions, confirm it and quantify the benefit.

ATBC — the Alpha-Tocopherol, Beta-Carotene Cancer Prevention Study — enrolled male smokers in Finland, testing beta-carotene and vitamin E.

CARET — the Beta-Carotene and Retinol Efficacy Trial — enrolled smokers and people with asbestos exposure in the United States, testing beta-carotene combined with retinol.

Both were designed to detect a benefit. Both were run by serious investigators. Both enrolled exactly the high-risk populations where a protective effect should have been easiest to demonstrate.

Both found more lung cancer in the supplemented group.

CARET was stopped early. Trials get stopped early for two reasons: overwhelming benefit, or harm. This was the second one.

Analysis: what went wrong, mechanism by mechanism

1. Healthy-user bias, in its purest form

Serum beta-carotene is a good biomarker of vegetable intake. Vegetable intake, in the populations studied, is a superb marker of a whole cluster of behaviors: not smoking, or smoking less; exercising; drinking less; having money; having education; having a doctor; living somewhere with clean air.

The biomarker felt like it solved the measurement problem — and it did. It did not touch the confounding problem at all. Measuring the exposure objectively tells you nothing about whether the exposure is causal.

This is the single most instructive detail in the whole story, and it's the one people miss. The field had fixed the problem it could see (self-report error) and this made everyone more confident, while the problem that actually mattered (confounding) was untouched.

The lesson stated generally: an objective measurement of a confounded exposure is a precise measurement of a confounded exposure.

2. The whole-food fallacy

Beta-carotene is one of hundreds of compounds in a carrot. Isolating it and delivering it at supraphysiological doses is not "eating vegetables." It is a completely different intervention that happens to share a molecule.

We now understand — vaguely, incompletely — that carotenoids in food come as a family, that they may compete for absorption, and that high-dose single-carotenoid supplementation can suppress levels of other carotenoids. The trials weren't testing the observational finding. They were testing a different thing entirely, which is a criticism sometimes raised in defense of the observational data, and which is fair as far as it goes.

But note where it leaves you: if isolating the compound doesn't test the food, then the observational finding about the compound was never actionable in the first place. Either way, the supplement was never justified.

3. Adjustment for smoking wasn't enough

The studies adjusted for smoking, and everyone found that reassuring.

But smoking was measured coarsely — packs per day, years, current or former. It does not capture depth of inhalation, tar content, decades of variation, or the difference between a man who quit at 40 and one who quit at 55. The residual confounding from imperfectly-measured smoking, in a population where smoking dominates lung cancer risk, was almost certainly larger than any dietary effect.

This is §2.3, exactly: you adjust as well as you measured, and your correction was coarser than the thing you were trying to detect.

4. A surrogate assumption nobody tested

"Antioxidant capacity" was treated as though it were a health outcome. It isn't. The chain from "raises measured antioxidant status" to "prevents cancer" was assumed, not demonstrated — and when tested, it broke. In smokers' lungs, under conditions of high oxidative stress, beta-carotene appears to behave differently than the simple model predicted.

We still don't fully understand the mechanism of the harm. That's worth saying plainly.

What this case does and doesn't prove

It does prove: that a large, coherent, mechanistically supported, dose-responsive, biomarker- confirmed, multiply-replicated body of observational evidence can be causally wrong, and that supplement extrapolation from food-based observational findings is a specific and repeated failure mode.

It does not prove: that observational evidence is worthless. Note what didn't fall. "People who eat more vegetables get less cancer" survived — it is still true, still replicated, and still part of the evidence base for dietary patterns. What fell was a much narrower claim: that a specific isolated molecule, at a supraphysiological dose, in a pill, was the active ingredient.

That distinction is the whole lesson, and it recurs so often it has a shape:

Observed: people who eat food F have less disease D. Inferred: compound C in food F prevents disease D. Tested: compound C in a pill. Result: nothing, or harm.

Fill in the blanks with beta-carotene and lung cancer. Then with vitamin E and cardiovascular disease. Then with the antioxidant supplement being advertised to you this week.

Discussion Questions

1. The biomarker data was the strongest part of the observational case, and it made the field more confident. Explain precisely why an objective biomarker fixes one problem and not the other. Why is this so easy to miss?

2. Given the state of the evidence in 1985, would it have been wrong to recommend beta-carotene supplements? Consider that the trials hadn't been run yet and people were dying of lung cancer in the meantime. What is the correct standard of evidence for a public recommendation?

3. One defense of the observational data is that the trials tested a different intervention (isolated high-dose compound) than the one observed (whole foods). Is this a good defense? What does accepting it imply about whether the observational finding was ever actionable?

4. Samir keeps the printout because "everyone can tell you correlation isn't causation, and almost nobody can tell you what it feels like from the inside when it isn't." What would it take for you to notice that feeling in yourself, about a nutrition belief you currently hold?

5. Apply the template above to a current supplement claim you've encountered. Where in the four steps is it? Has anyone run step 3 yet?

Your Turn

Pick a supplement currently marketed on the basis that a nutrient is associated with good health outcomes — there are dozens; open any pharmacy website.

Work out, as far as you can:

• What is the observational evidence that the nutrient is associated with the outcome?
• Has anyone run a randomized trial of the isolated compound?
• If yes, what did it find? If no, why not — and how is the product being sold in the meantime?
• Does the marketing distinguish between "people who eat foods containing this are healthier" and "taking this makes you healthier"?

Write a paragraph. Most readers find that the fourth question answers itself, and that the marketing depends entirely on the reader not making the distinction the whole of Chapter 2 was about.