Chapter 29 Exercises: A/B Testing Your Mind

Reflection Exercises

Exercise 1 (Reflection) Think about a time when you noticed that a social media interface had changed — a button moved, a color changed, a feature appeared or disappeared. Did you know that change was likely the result of an A/B test? How does knowing that you are constantly being experimented on change your relationship to the platforms you use? Does it matter to you?

Exercise 2 (Reflection) The chapter describes the distinction between what users say they want and what their behavior reveals. Can you identify a case in your own social media use where your behavior diverges from your stated preferences? For example, do you click on content you wouldn't say you want to see? What does this divergence tell you about self-knowledge and behavioral prediction?

Exercise 3 (Reflection) The Facebook emotional contagion experiment manipulated users' emotional environments without their knowledge. Imagine you were one of the 689,000 users who were unknowingly assigned to see more negative content. How would you feel about having been a research subject without consent? Does the scale of the manipulation (subtle, brief) change your reaction?

Exercise 4 (Reflection) Consider the notification timing experiments described in the chapter — tests that determine the precise moment of day when you are most susceptible to a notification compelling you to open an app. Reflect on your own notification behavior. Do you notice patterns in when you are more or less able to resist checking notifications? What factors seem to affect this susceptibility?

Exercise 5 (Reflection) The chapter describes the "optimization target problem" — that what gets measured is what gets optimized, and easily measured proxies may diverge from what actually matters. Reflect on an area of your own life where you have relied on a proxy metric (GPA, social media followers, weight, salary) rather than the underlying value it was supposed to represent. How did optimizing for the proxy affect your pursuit of the underlying value?

Exercise 6 (Reflection) The chapter ends with Velocity Media CEO Sarah Chen implementing an imperfect internal ethics review process after Dr. Aisha Johnson's advocacy. Reflect on the political dynamics of this institutional outcome — neither Webb nor Johnson got exactly what they wanted, and the result was a compromise. Do you think this kind of institutional negotiation is the appropriate way to develop ethical norms in commercial organizations? What are its limitations?

Exercise 7 (Reflection) Facebook's "meaningful social interactions" algorithm change was framed as a user wellbeing initiative but may have amplified misinformation. This is an example of a good-faith effort to improve an outcome that produced unintended negative consequences. How should organizations respond when well-intentioned interventions backfire? What structures help detect and respond to such failures?

Research Exercises

Exercise 8 (Research) Find and read the full text of the Facebook emotional contagion paper: Kramer, A. D. I., Guillory, J. E., & Hancock, J. T. (2014). "Experimental Evidence of Massive-Scale Emotional Contagion Through Social Networks." PNAS, 111(24), 8788–8790. What are the exact findings? What limitations do the authors acknowledge? What is the effect size? How do you evaluate the scientific significance of the findings given the ethical controversy?

Exercise 9 (Research) Research the history of the Belmont Report (1979) and the three principles it established (respect for persons, beneficence, justice). How have these principles been applied in landmark cases of research ethics violations? How do each of the three principles apply to the Facebook emotional contagion experiment?

Exercise 10 (Research) Look up Christian Rudder's 2014 blog post "We Experiment on Human Beings!" on the OkCupid blog. What specific experiments does Rudder describe? What is his argument for why platform experimentation is acceptable? How did the public react? How does his argument compare to the ethical framework established in this chapter?

Exercise 11 (Research) Research the EU's Digital Services Act (DSA) and its provisions related to algorithmic transparency and risk assessment for "very large online platforms." What does the DSA require platforms to disclose about their algorithmic systems? How does it address A/B testing and experimental manipulation? What are the enforcement mechanisms?

Exercise 12 (Research) Research the multi-armed bandit problem in machine learning and probability theory. Explain the difference between Thompson Sampling, Upper Confidence Bound (UCB), and epsilon-greedy approaches. How does each approach balance exploration and exploitation? Which approach is most likely to be used by major social media platforms, and why?

Exercise 13 (Research) Find at least three peer-reviewed studies that examine the relationship between specific social media engagement patterns and user wellbeing. What do these studies find? What engagement patterns are associated with better versus worse wellbeing outcomes? What does the evidence suggest about the relationship between engagement metrics and wellbeing metrics?

Exercise 14 (Research) Research the history of Institutional Review Board (IRB) review in the United States, including the specific abuses that motivated its creation (the Tuskegee syphilis study, the Milgram obedience experiments). How have IRB requirements evolved since the Belmont Report? What categories of research are exempt from full IRB review? Does the commercial A/B testing of social media platforms fall within any category of IRB coverage?

Analysis Exercises

Exercise 15 (Analysis) Apply the Belmont Report's three principles — respect for persons, beneficence, justice — to the Facebook emotional contagion experiment. For each principle, analyze whether the experiment complied with the principle, violated it, or occupied an ambiguous middle ground. Conclude with an overall ethical assessment.

Exercise 16 (Analysis) Compare the Facebook emotional contagion experiment to the OkCupid experiments disclosed by Christian Rudder. What are the relevant similarities and differences? Does the difference in the platforms' responses to public scrutiny (Facebook expressing regret; Rudder expressing defiance) affect the ethical analysis?

Exercise 17 (Analysis) Analyze the "meaningful social interactions" algorithm change using the optimization target framework. What was the stated optimization target? How was it operationalized? Where did the operationalization diverge from the underlying value? What design changes might have produced an operationalization that better tracked the stated goal?

Exercise 18 (Analysis) The chapter describes clickbait as emerging from A/B testing optimization that selected for click-through rates. Apply the same analysis to at least two other internet phenomena — viral misinformation, outrage-based political content, or another specific content pattern you observe — and argue that they can be understood as products of optimization pressures rather than deliberate creation.

Exercise 19 (Analysis) Analyze the multi-armed bandit problem as a continuous optimization mechanism. How does it differ ethically from a discrete A/B test with a defined start and end? Does the absence of a discrete experimental period make the consent question easier or harder to address? Does it make harm detection easier or harder?

Exercise 20 (Analysis) The chapter identifies dark patterns as emergent products of optimization rather than deliberate design choices. Does this origin story change the ethical analysis? Compare a dark pattern that was deliberately designed to deceive with one that emerged from A/B testing optimization. Are they ethically equivalent? Should they be treated equivalently by regulators?

Exercise 21 (Analysis) Evaluate the four proposed remedies for the optimization target problem: expanding optimization targets, transparency requirements, consent-based experimentation, and independent oversight. For each remedy, analyze: (a) its potential effectiveness, (b) its implementation challenges, and (c) its potential unintended consequences. Which remedy or combination of remedies do you think would be most effective?

Creative Exercises

Exercise 22 (Creative) Design an "Informed Consent for Platform Experimentation" document that you believe would constitute genuine, meaningful consent for the kind of A/B testing described in this chapter. What information would it need to convey? At what level of detail? How would you present it in a way that users would actually read and understand? How would you handle consent for continuous experimentation rather than discrete experiments?

Exercise 23 (Creative) Write a response to Christian Rudder's "We Experiment on Human Beings!" blog post from the perspective of a bioethicist who applies research ethics principles to platform experimentation. Make the argument specific and evidence-based, drawing on the principles and cases discussed in the chapter.

Exercise 24 (Creative) Design an internal Experimental Ethics Review process for a hypothetical social media platform. What experiments would require review? What would the review process look like? Who would be on the review committee? What criteria would be used to evaluate proposed experiments? What would happen if a review determined that an experiment should not proceed?

Exercise 25 (Creative) Write a short story (500-800 words) from the perspective of a platform data scientist who runs an A/B test that produces excellent engagement metrics but later realizes has been causing psychological harm to a specific subset of users. How do they discover the harm? What do they do with the information? What institutional barriers do they face?

Exercise 26 (Creative) Imagine you are a regulator designing a comprehensive framework for governing behavioral experimentation at social media platforms. Write a one-page policy brief describing your proposed framework: what it requires, what it prohibits, how it is enforced, and how it balances the benefits of product experimentation with user protection.

Group Discussion Exercises

Exercise 27 (Group Discussion) Divide the group into two sides: one arguing that platform A/B testing is a legitimate business activity that does not require the oversight requirements of academic research, the other arguing that the scale and potential for harm of platform experimentation requires the same ethical framework as academic research. After the debate, discuss what distinctions between the two positions are most significant.

Exercise 28 (Group Discussion) The chapter notes that A/B testing correctly identified that users click on clickbait at higher rates than on honest headlines, even though users report disliking clickbait. This raises a philosophical question: should platforms optimize for what users do or for what users say they want? Is there a defensible distinction between "what users actually want" and "what users do"?

Exercise 29 (Group Discussion) The Facebook emotional contagion experiment generated enormous public controversy while the routine A/B testing that Facebook conducts every day generates essentially no public scrutiny. What accounts for this asymmetry? Is it justified by meaningful differences between the emotional contagion study and routine product A/B testing? Or is the public attention distribution distorted by factors unrelated to ethical significance?

Exercise 30 (Group Discussion) The chapter describes red notification badges as exploiting attentional systems that evolved to respond to threat cues. Is designing an interface to exploit attentional systems always unethical, or only sometimes? Where do you draw the line between legitimate attention capture (advertising, news headlines) and manipulative attention capture (social media notification design)?

Exercise 31 (Group Discussion) The chapter proposes that dark patterns can emerge from optimization processes without any individual designer deliberately choosing deception. Does this diffusion of responsibility change the ethical analysis? Who should be held accountable for harm that emerges from optimization processes rather than individual choices? What institutional structures would help assign accountability more clearly?

Exercise 32 (Group Discussion) Consider the regulatory options for platform A/B testing: expanding optimization targets, transparency requirements, consent-based experimentation, and independent oversight. Which of these remedies would you prioritize? What are the tradeoffs? What would happen to social media product development if the most restrictive version of these remedies were implemented?

Exercise 33 (Group Discussion) The Velocity Media Experimental Ethics Review was described as an imperfect compromise. Is this kind of voluntary internal process an adequate response to the ethical challenges of behavioral experimentation at scale? Or does it represent a way for companies to appear to take ethical concerns seriously without making the fundamental changes that would address those concerns? Under what conditions would voluntary industry ethics processes be adequate, and when are they inadequate?

Exercise 34 (Group Discussion) The chapter argues that the structural conflict between engagement optimization and user wellbeing is a property of the advertising-supported business model, not a product of individual ethical failures. Does this structural argument suggest that only changing the business model will adequately address the problem? Are there modifications to the advertising-supported model that would reduce the structural conflict? What would those modifications look like?

Exercise 35 (Group Discussion) Consider the consent problem from a practical perspective: Facebook has over 3 billion users. How do you obtain meaningful, informed consent from 3 billion people for behavioral experimentation in a way that is operationally feasible? Does the practical difficulty of consent at scale constitute a legitimate reason to exempt platform experimentation from consent requirements? Or does it require us to rethink how consent works in the digital age?