Exercises — Chapter 15: AI in Healthcare

Exercise 15.1: Evaluating a Healthcare AI Claim (Analyze)

Find a recent news article (from the past two years) that claims an AI system has achieved some medical breakthrough — diagnosing a disease, discovering a drug, predicting patient outcomes, or similar.

Apply the following analysis:

  1. Describe the claim. What does the article say the AI can do?
  2. Apply the three-question test from Section 15.2: (a) What specific task and what specific metric? (b) Under what conditions — lab or real world? (c) For which patients?
  3. Evaluate the evidence. Is the claim based on a peer-reviewed study? A press release? A company announcement? What is the source's credibility?
  4. Identify what is missing. What information would you need to fully evaluate this claim? Did the article report performance by demographic subgroup? Was the study prospective or retrospective?
  5. Write a revised headline that more accurately reflects the evidence.

Target length: 400–500 words.

Exercise 15.2: The Obermeyer Case — Proxy Problem Analysis (Evaluate)

The Obermeyer study found that using healthcare costs as a proxy for healthcare needs systematically disadvantaged Black patients because of existing disparities in healthcare spending.

  1. Explain the proxy problem in your own words. Why did using cost as a proxy for need produce biased outcomes? Use a concrete example.
  2. Identify two other contexts (inside or outside healthcare) where a proxy variable might encode existing inequalities. For each, explain what disparity the proxy might capture and who would be harmed.
  3. Propose an alternative. If you were redesigning the algorithm from the Obermeyer study, what variable(s) would you use instead of healthcare costs to predict healthcare needs? What are the potential limitations of your alternative?
  4. Address the harder question. Even if you find a better proxy, can an algorithm trained on historically biased data ever be truly fair? Why or why not? Connect your answer to the threshold concept from Chapter 9: "Fairness is not a single metric."

Target length: 500–600 words.

Exercise 15.3: Stakeholder Analysis — MedAssist AI Deployment (Evaluate)

Imagine you are on the hospital committee deciding whether to continue, modify, or discontinue MedAssist AI after the problems described in Section 15.6. Conduct a stakeholder analysis.

For each of the following stakeholders, describe: (a) their primary interests, (b) their likely position on continuing/modifying/discontinuing, and (c) the strongest argument from their perspective.

  1. Patients in demographic groups for whom MedAssist works well
  2. Patients in demographic groups for whom MedAssist performs poorly
  3. Radiologists who use MedAssist daily
  4. The hospital's chief medical officer
  5. The hospital's legal counsel
  6. The company that manufactures MedAssist
  7. The hospital's patient advocacy group
  8. A health equity researcher

After completing the stakeholder analysis, write a 200-word recommendation to the committee. What should the hospital do, and why?

Exercise 15.4: Regulation Comparison (Analyze)

Research the regulatory framework for medical AI in two different jurisdictions (choose from: United States/FDA, European Union/MDR + AI Act, United Kingdom/MHRA, Japan/PMDA, or another country of your choice).

Create a comparison table addressing:

Dimension Jurisdiction 1 Jurisdiction 2
Regulatory body
Pathway for AI medical devices
Pre-market requirements
Post-market surveillance requirements
Requirements for demographic performance reporting
Treatment of algorithms that update/learn
Transparency/explainability requirements

Write a 300-word analysis: Which framework do you think better protects patients? Why? What gaps exist in both?

Exercise 15.5: The Transparency Debate (Evaluate/Create)

Read the arguments for and against disclosing AI involvement in patient care from Section 15.4. Then:

  1. Write a 200-word argument in favor of mandatory disclosure — the position that patients should always be told when AI plays a significant role in their diagnosis or treatment.

  2. Write a 200-word argument against mandatory disclosure — the position that disclosure may cause more harm than good (patient anxiety, refusal of beneficial treatment, confusion about the physician's role).

  3. Write your own 200-word position. Where do you come down, and why? What conditions or qualifications would you add?

  4. Design a disclosure statement. Write a brief, patient-friendly statement (50–100 words) that a hospital could use to inform patients about AI involvement in their care. Test it on a non-expert friend or family member. Does it inform without alarming? Does it respect patient autonomy without overwhelming with technical detail?

Exercise 15.6: Global Health Equity and AI (Analyze)

AI diagnostic tools are increasingly being proposed for deployment in low- and middle-income countries where specialist physicians are scarce. Research one specific example of this (such as Google Health's diabetic retinopathy screening in India, or AI-assisted tuberculosis detection in sub-Saharan Africa).

Address the following questions:

  1. What problem is the AI system trying to solve?
  2. Who developed the system, and where?
  3. Was the system validated on the population where it is being deployed?
  4. What infrastructure (internet, electricity, equipment) is required to run it?
  5. What happens when the system fails or is unavailable?
  6. Who benefits from this deployment, and who bears the risks?
  7. Could this be considered a form of what some scholars call "data colonialism" — where technology developed by wealthy nations is tested on populations in poorer nations? Why or why not?

Target length: 500–600 words.

Exercise 15.7: Design a Healthcare AI Safeguard (Create)

You have been hired as a consultant by a hospital that is about to deploy a new AI system for detecting diabetic retinopathy from retinal photographs. The system has FDA clearance and performed well in clinical trials. Your job is to design safeguards for real-world deployment.

Write a 500-word deployment plan that addresses:

  1. Pre-deployment: What should the hospital verify before deploying the system? (Consider: patient demographics, image quality, clinician training, workflow integration.)
  2. During deployment: What monitoring should be in place? (Consider: performance metrics, demographic breakdowns, clinician feedback, patient complaints.)
  3. Failure response: What should happen if the system's performance falls below acceptable thresholds? (Consider: triggers, notification, suspension, communication to patients.)
  4. Equity audit: How will the hospital ensure the system performs equitably across patient demographics?
  5. Patient communication: How will patients be informed about AI involvement in their care?