Chapter 29 Quiz: A/B Testing Your Mind

Instructions: Select the best answer for each question. Answer key appears at the end.

Question 1. In an A/B test, what is the crucial element that allows causal inference?

A) A large sample size B) Random assignment of users to conditions C) Double-blind methodology with researchers unaware of group assignments D) The use of validated psychometric instruments

Question 2. Approximately how many A/B tests per year does Facebook reportedly run?

A) Hundreds B) Thousands C) More than 10,000 D) More than 100,000

Question 3. What advantage does A/B testing have over user self-report as a method for understanding behavior?

A) It is less expensive to conduct B) It measures actual behavior rather than predicted behavior C) It provides qualitative insight into user motivations D) It is more legally defensible than survey research

Question 4. The "optimization target problem" refers to:

A) The technical challenge of running multiple A/B tests simultaneously B) The risk that A/B tests will produce false positive results C) The divergence between easily measured proxy metrics and the outcomes that actually matter for users D) The difficulty of obtaining regulatory approval for platform experimentation

Question 5. The Facebook emotional contagion experiment was published in June 2014 in which journal?

A) Nature B) Science C) Proceedings of the National Academy of Sciences D) Journal of Experimental Psychology

Question 6. How many users were involved in the Facebook emotional contagion experiment?

A) 6,890 B) 68,900 C) 689,000 D) 6,890,000

Question 7. What was the main finding of the Facebook emotional contagion experiment?

A) Seeing positive content made users more likely to post negative content B) Emotional states expressed in social networks are contagious through content exposure C) Users who saw more content were more emotionally reactive than those who saw less D) Social media use generally reduces emotional expression compared to face-to-face interaction

Question 8. The Belmont Report (1979) established which three principles for ethical human subjects research?

A) Transparency, accountability, and proportionality B) Voluntary participation, privacy, and fairness C) Respect for persons, beneficence, and justice D) Autonomy, non-maleficence, and informed consent

Question 9. What primary justification did Facebook use to defend the emotional contagion experiment's lack of individual informed consent?

A) The research was conducted outside the United States and US regulations did not apply B) Facebook's data use policy mentioned "research" and users had accepted it C) The effect size was so small that the research posed no meaningful risk to users D) The researchers had obtained informed consent from a representative sample

Question 10. The multi-armed bandit problem in platform optimization involves balancing:

A) Multiple A/B tests running simultaneously without statistical interference B) Exploration (learning which variants perform better) and exploitation (deploying the best-known variant) C) The tradeoff between user engagement and user privacy D) Competing optimization targets such as engagement and wellbeing

Question 11. A "contextual bandit" algorithm extends the basic multi-armed bandit approach by:

A) Adding multiple simultaneous treatment conditions instead of just two B) Incorporating information about individual users to personalize which variant is shown C) Applying bandit optimization to multiple platforms simultaneously D) Using bandit methods for content moderation rather than content ranking

Question 12. Facebook's "meaningful social interactions" algorithm change in 2018 operationalized "meaningful" primarily as:

A) Content shared by close friends rather than distant acquaintances B) Content with verified factual accuracy C) Content from friends and family that generated comments D) Content that users subsequently reported as valuable in surveys

Question 13. According to the chapter, dark patterns often emerge from platform optimization processes because:

A) Platform designers deliberately build deceptive elements into their products B) Regulatory requirements create perverse incentives for deceptive design C) Optimization processes select for high-engagement variants regardless of whether they are manipulative D) Users prefer deceptive designs when tested against honest alternatives

Question 14. OkCupid's 2014 disclosure of its A/B testing practices was made by:

A) A whistleblower who leaked the information to journalists B) The company's CEO in a public blog post C) Co-founder Christian Rudder in a blog post titled "We Experiment on Human Beings!" D) The Federal Trade Commission following an investigation

Question 15. Why does A/B testing's advantage in measuring actual behavior not automatically translate into an advantage for measuring user wellbeing?

A) Behavioral measurement technology is not sophisticated enough to capture wellbeing-relevant behaviors B) A/B tests measure what users do, but not whether what they do is good for them C) Wellbeing-relevant behaviors are too infrequent to detect with statistical significance D) Users behave differently when they know they are being observed

Question 16. The chapter states that the regulatory gap for platform behavioral experimentation exists because:

A) Regulators are unaware of the scale of platform A/B testing B) Platform companies have lobbied successfully against regulatory oversight C) Commercial product testing is excluded from the research ethics frameworks designed for academic research D) International jurisdictional complexity prevents any single regulatory body from having authority

Question 17. In the Velocity Media case, Dr. Aisha Johnson proposed an Experimental Ethics Review process. What triggered her proposal?

A) A public controversy about a Velocity Media experiment that became public B) A request from regulators for information about Velocity Media's experimentation practices C) She discovered that a product experiment affecting 3.4 million users had been running for six weeks without her awareness D) Marcus Webb proposed an experiment that she considered clearly unethical

Question 18. Red notification badge colors are described in the chapter as exploiting which psychological mechanism?

A) Cognitive dissonance B) Social proof C) Attentional systems that evolved to respond to threat cues D) The peak-end rule

Question 19. Which of the following best describes the "inform and debrief" requirement in academic human subjects research?

A) Researchers must publish all findings in open-access journals B) Researchers must tell subjects they are in a study before participation, and explain the study's purpose afterward C) Researchers must report any negative outcomes to their institution's board of governors D) Researchers must provide subjects with compensation for their participation

Question 20. The clickbait emergence phenomenon illustrates the optimization target problem because:

A) Headlines optimized for click-through rates consistently underperformed compared to honest headlines B) A/B testing correctly identified that users click on clickbait while users report disliking it C) Clickbait emerged because designers deliberately wanted to deceive users D) Click-through rate optimization produced content that was both more clicked on and more valued by users

Question 21. According to the chapter, which EU legislation provides the most direct tools for addressing platform algorithmic transparency through transparency requirements and risk assessments?

A) The General Data Protection Regulation (GDPR) B) The Digital Services Act (DSA) and Digital Markets Act (DMA) C) The EU AI Act D) The ePrivacy Directive

Question 22. The chapter argues that the structural conflict between engagement optimization and user wellbeing in advertising-supported social media is:

A) Unique to Facebook and a consequence of that company's specific corporate culture B) Temporary and will be resolved as platforms develop better wellbeing measurement tools C) A structural feature of the advertising-supported business model that affects every platform operating within it D) Primarily a problem in developing countries where regulatory oversight is weaker

Answer Key

1. B
2. C
3. B
4. C
5. C
6. C
7. B
8. C
9. B
10. B
11. B
12. C
13. C
14. C
15. B
16. C
17. C
18. C
19. B
20. B
21. B
22. C