Chapter 16 Exercises
How to use these exercises: Work through the parts in order. Part A builds recognition skills, Part B develops analysis, Part C applies concepts to your own domain, Part D requires synthesis across multiple ideas, Part E stretches into advanced territory, and Part M provides interleaved practice that mixes skills from all levels.
For self-study, aim to complete at least Parts A and B. For a course, your instructor will assign specific sections. For the Deep Dive path, do everything.
Part A: Pattern Recognition
These exercises develop the fundamental skill of recognizing the legibility-vitality tradeoff across domains.
A1. For each of the following scenarios, identify (i) the complex system, (ii) the central authority demanding legibility, (iii) what is being made legible, (iv) what vital complexity is being destroyed, and (v) the likely long-term consequence.
a) A hospital requires all nurses to document their patient interactions using a standardized electronic health record system, spending 30 minutes per hour on data entry rather than with patients.
b) A national government replaces diverse local farming practices with a single "scientifically optimal" crop rotation mandated for all regions.
c) A tech company replaces informal team communication with a project management tool that requires all work to be broken into measurable tasks with estimated completion times.
d) A city replaces a neighborhood of mixed small businesses, apartments, and workshops with a zoned development: commercial buildings on one block, residential towers on the next, parking structures between them.
e) A school district requires all teachers to follow the same lesson plans, delivered on the same schedule, to ensure "consistency" across classrooms.
f) A social media platform redesigns its feed to show users only content that its algorithm predicts they will "engage with," filtering out content that does not match the model of the user's preferences.
g) A parent installs a GPS tracker on their teenager's phone and requires hourly check-ins when the teenager is out with friends.
h) A government requires all traditional herbal medicine practitioners to pass a standardized licensing exam based on Western pharmacological categories.
A2. Classify each of the following as primarily a legibility-as-observation project (thermometer) or a legibility-as-intervention project (reshaping the system). Explain your reasoning.
a) A national census that counts population by region, age, and occupation.
b) A standardized curriculum that requires all schools to teach the same subjects in the same order.
c) A wildlife survey that tracks animal populations across a national park.
d) A corporate restructuring that reorganizes departments by function rather than by product line to make reporting clearer.
e) A health department that requires restaurants to post their inspection grades in the window.
f) A public transit system that replaces a network of informal minibus routes (used by drivers who know where passengers want to go) with a planned bus system of fixed routes.
A3. For each of the following, identify the metis that is at risk and explain why it cannot be captured by the proposed legibility system.
a) A chain restaurant replaces experienced chefs with standardized recipes and pre-measured ingredient kits.
b) A policing department replaces beat cops who walk a neighborhood daily with patrol cars dispatched based on crime statistics.
c) A university replaces tenure decisions based on faculty committee judgment with a formula based on publication count, citation metrics, and grant dollars.
d) A farmer's market is replaced by a supermarket that sources all produce through a centralized supply chain with standardized quality grades.
e) An apprenticeship system for training electricians is replaced by a classroom-based certification program.
A4. The chapter describes four elements of "authoritarian high modernism." For each of the following historical or contemporary examples, identify which of the four elements are present and which are absent. Based on your analysis, predict whether the legibility project succeeds or fails.
a) The United States Interstate Highway System (built in the 1950s-60s)
b) Mao's Great Leap Forward (1958-62)
c) The European Union's Common Agricultural Policy
d) A school district's adoption of standardized testing under No Child Left Behind
e) Wikipedia's system of collaborative knowledge production
A5. The chapter argues that legibility projects often produce impressive "first-generation results" followed by "second-generation failures." Identify the first-generation success and second-generation failure in each of the following:
a) Scientific forestry in Germany
b) High-stakes standardized testing in education
c) Helicopter parenting
d) Algorithmic content curation on social media
Part B: Analysis
These exercises require deeper analysis of the legibility-vitality tradeoff.
B1. The Legibility Audit. Choose one of the following systems and perform a complete legibility audit:
- A hospital or healthcare system
- A university or school system
- A corporation or nonprofit you have worked for
- A government agency or public service
- An online platform you use regularly
For your chosen system:
a) Identify all the legibility projects currently in operation (metrics, standardizations, reporting requirements, dashboards, assessments).
b) For each legibility project, identify what is being made legible and what complexity is being sacrificed.
c) Identify the metis in the system -- the local, practical, experiential knowledge that practitioners rely on but that the legibility systems do not capture.
d) Assess whether each legibility project is functioning as observation (thermometer) or intervention (reshaping the system).
e) Identify one legibility project that is clearly necessary and one that is clearly destructive. Explain the difference using the chapter's diagnostic framework.
f) Propose modifications that would preserve the benefits of legibility while reducing the destruction of vital complexity.
B2. The Brasilia Problem. Brasilia and Greenwich Village represent two models of urban order: planned (legible) and organic (illegible).
a) List five specific features of Greenwich Village (as described by Jane Jacobs) that make it vital and five specific features that make it illegible.
b) For each illegible feature, explain what a planner trying to make the neighborhood legible would do to it, and what would be lost.
c) Could a planner design a neighborhood that has the vitality of Greenwich Village? Why or why not? What does your answer reveal about the nature of emergent order?
d) Some modern urban planners practice "tactical urbanism" -- small, temporary interventions (pop-up parks, temporary pedestrian zones, food truck clusters) designed to allow organic order to develop. Analyze this approach using the chapter's framework. Is tactical urbanism a solution to the legibility problem, or is it something else?
B3. Algorithmic Legibility vs. Human Illegibility. The chapter argues that algorithmic legibility is uniquely dangerous because the algorithm's model can reshape the person being modeled.
a) Explain this claim in detail, using a specific example. How does a recommendation algorithm's model of a person become self-fulfilling?
b) Is there a meaningful difference between an algorithm that predicts your preferences and one that creates your preferences? Where is the line, and how would you know if it had been crossed?
c) Some researchers propose "serendipity engines" -- algorithms designed to occasionally show users content that does not match their profile, to introduce randomness and prevent preference narrowing. Analyze this proposal using the chapter's framework. Is it a genuine solution or a cosmetic fix?
d) Compare algorithmic legibility to the legibility of a standardized test. In what ways are they structurally similar? In what ways does the algorithmic case introduce genuinely new dynamics?
B4. The Metis Problem. The chapter introduces metis as practical, local, embodied knowledge that resists formalization.
a) Give a detailed example of metis from your own experience -- knowledge you possess that you could not fully teach to someone through a textbook or manual.
b) How did you acquire this metis? How long did it take? Could the acquisition process have been shortened by making it more "efficient" (i.e., more legible)?
c) Some organizations try to capture metis through "knowledge management" systems -- databases of best practices, standard operating procedures, and lessons learned. Analyze the strengths and limitations of this approach using the chapter's framework. Under what conditions can knowledge management capture metis? Under what conditions does it merely produce a legible simulacrum of metis?
d) If metis cannot be made legible, how should organizations that depend on it protect and transmit it? Propose at least three specific practices.
Part C: Application to Your Own Domain
These exercises connect the legibility-vitality tradeoff to your area of expertise.
C1. Identify a legibility project in your field of study or professional domain.
a) What complex system is being made legible?
b) What is the simplification, and who benefits from it?
c) What vital complexity is being destroyed?
d) What metis is being overridden or devalued?
e) Has the legibility project produced "first-generation success" followed by signs of "second-generation failure"? If so, describe both.
f) Using the chapter's diagnostic framework, evaluate whether the legibility project is justified, destructive, or a mix of both. Propose modifications.
C2. Design a system for your domain that balances legibility and vitality.
a) Identify the minimum legibility necessary for coordination, accountability, or public safety.
b) Identify the vital complexity that must be preserved -- the illegible elements that the system depends on for its functioning.
c) Design metrics and reporting requirements that satisfy the minimum legibility need without destroying the vital complexity.
d) Include a mechanism for protecting metis -- for ensuring that practitioner knowledge is valued, transmitted, and not overridden by centralized mandates.
e) Include a feedback mechanism that detects when legibility projects are crossing the line from useful to destructive.
C3. Interview a practitioner in your domain -- someone with deep metis gained through years of experience. Ask them:
a) What do you know about your work that you could not teach someone by writing it in a manual?
b) How did you learn it?
c) Are there metrics, assessments, or reporting requirements in your work that miss what actually matters? What do they miss?
d) If you could design the measurement system for your domain, what would you measure? What would you deliberately not measure?
Write a one-page summary connecting their responses to the concepts in this chapter.
Part D: Synthesis
These exercises require integrating ideas across multiple chapters.
D1. Legibility and Goodhart's Law. Chapter 15 introduced Goodhart's Law -- when a measure becomes a target, it ceases to be a good measure. Chapter 16 introduces the deeper legibility problem.
a) Explain how every instance of Goodhart's Law is also a legibility problem. What is the legibility demand, and how does it create the conditions for metric gaming?
b) Is every legibility problem also an instance of Goodhart's Law? Or are there legibility failures that do not involve metric gaming? Give examples.
c) The chapter argues that the legibility problem is "deeper" than Goodhart's Law. Explain what this means. How does the legibility framework add explanatory power beyond what Goodhart's Law alone provides?
d) Scott identifies four elements of "authoritarian high modernism." Which of these elements is addressed by the solutions to Goodhart's Law proposed in Chapter 15 (multi-metric approaches, qualitative assessment, rotating metrics, gaming detection, polycentric governance)? Which elements are not addressed?
D2. Legibility and Emergence. Chapter 3 introduced emergence -- the phenomenon of complex collective properties arising from simple individual interactions.
a) Explain why legibility projects systematically destroy emergent order. What is it about emergence that makes it illegible?
b) Can emergent order be preserved in a legible system? Or are legibility and emergence fundamentally incompatible? Defend your answer with examples.
c) The chapter suggests that "you cannot design emergence; you can only create the conditions from which it arises." What would these conditions look like in a specific domain (city, school, organization, ecosystem)? How would you know if emergence was occurring?
d) Chapter 3 discussed downward causation -- how emergent properties constrain the components that produce them. How does legibility-driven simplification disrupt downward causation?
D3. Legibility and Satisficing. Chapter 12 introduced satisficing -- accepting "good enough" rather than optimizing.
a) The chapter argues that satisficing is "a strategy for preserving vital complexity." Explain this claim in detail. How does satisficing protect the "understory"?
b) High modernism can be understood as a refusal to satisfice -- a demand for optimal, planned, legible systems. Explain this connection.
c) Is there a level of optimization pressure beyond which satisficing becomes impossible? If a school district, corporation, or government agency faces intense competitive pressure, can it choose to satisfice, or will it be forced to pursue legibility and optimization? What does this suggest about the structural conditions that enable or prevent satisficing?
D4. Legibility and Overfitting. Chapter 14 introduced overfitting -- the error of fitting a model too closely to specific data, capturing noise rather than signal.
a) The chapter describes legibility-driven curriculum narrowing as "overfitting in institutional form." Explain this analogy in detail. What is the model? What is the training data? What is the overfitting?
b) Chapter 14's solution to overfitting is regularization -- constraining the model's complexity to prevent it from capturing noise. What would "institutional regularization" look like in the context of a legibility project? Propose a specific example.
c) Overfitting is worst when the training data is small and the model is complex. Under what conditions is the legibility-vitality tradeoff worst? Is there an analogy to the size of the training data?
d) Cross-validation (Chapter 14) tests a model on data it was not trained on. What would "cross-validation" look like for a legibility project? How would you test whether a simplification is capturing essential features or merely imposing a convenient fiction?
Part E: Advanced Challenges
These exercises push beyond the chapter's material into deeper or more speculative territory.
E1. Scott argues that legibility projects fail most catastrophically when four elements combine: administrative ordering, high-modernist ideology, authoritarian power, and a prostrate civil society. Research a contemporary legibility project (China's Social Credit System, predictive policing algorithms in American cities, standardized global education metrics like PISA, or corporate algorithmic management of warehouse workers). Analyze it using Scott's four-element framework. Which elements are present? What outcome do you predict? Write a 500-word analysis.
E2. The chapter presents metis and formal knowledge as complementary but in tension. Some philosophers argue that all knowledge is ultimately tacit -- that even "formal" knowledge depends on a substrate of embodied, contextual understanding to be applied. If this is true, what does it imply for the possibility of legibility? Can any system be made fully legible? Write a 500-word philosophical analysis.
E3. Artificial intelligence models are trained by making human knowledge legible -- converting text, images, and human judgments into datasets that machine learning algorithms can process. Analyze AI training as a legibility project using the chapter's framework. What metis is captured by the training process? What metis is lost? What are the potential consequences of the legibility-vitality tradeoff for AI systems?
E4. Design a thought experiment: a world where legibility has no cost -- where making a system measurable and controllable never destroys any of its vital complexity. What structural features would this world need? Is such a world logically possible? What would the consequences be? Would they all be positive?
Part M: Mixed Practice (Interleaved Review)
These exercises mix concepts from Chapters 12-16 to build integrated understanding.
M1. A technology company implements Objectives and Key Results (OKRs) -- a metric system designed to align employee effort with company goals (legibility, Ch. 16). An engineer satisfices (Ch. 12) by doing enough to meet her OKRs rather than pursuing a risky but potentially transformative project. Her manager, observing that OKR scores correlate with promotion, optimizes aggressively for OKR metrics (Goodhart's Law, Ch. 15). The team's innovation output declines. Analyze this system using concepts from all three chapters. Where did the failure occur? What would you change?
M2. A hospital introduces a clinical decision support system that provides algorithmic treatment recommendations based on patient data (legibility, Ch. 16). The algorithm was trained on historical data from a different hospital system (overfitting risk, Ch. 14). Experienced nurses report that the algorithm's recommendations sometimes conflict with their clinical judgment (metis, Ch. 16). The hospital's administration ties compliance with algorithmic recommendations to performance reviews (Goodhart's Law, Ch. 15). Analyze this system using concepts from Chapters 14, 15, and 16. What will happen? What should the hospital do differently?
M3. A city government, concerned about neighborhood decline, commissions a data dashboard that tracks property values, crime statistics, building permits, and business openings (legibility, Ch. 16). The dashboard shows that neighborhoods with more diverse, mixed-use development have better outcomes on all metrics. The city then designs a new mixed-use neighborhood from scratch, hoping to replicate the pattern (high modernism, Ch. 16). The new neighborhood struggles. Explain why, using concepts from Chapters 3 (emergence), 9 (distributed vs. centralized), and 16 (legibility).
M4. A parent reads about the importance of "grit" in child development and begins measuring their child's persistence on difficult tasks, praising high persistence and expressing concern about low persistence (making resilience legible, Ch. 16). The child learns to perform persistence rather than genuinely developing it -- continuing to work on a task while actually daydreaming, to satisfy the parent's metric (Goodhart's Law, Ch. 15). When the child faces a genuinely difficult challenge in college, they lack actual resilience. Analyze this situation using concepts from Chapters 15 and 16. What went wrong? What would a metis-respecting approach to developing resilience look like?
M5. An environmental agency needs to protect biodiversity in a national forest. They consider two approaches: (A) a standardized management plan with measurable targets for species counts, canopy density, and deadwood volume, applied uniformly across the entire forest; (B) a flexible approach that gives local rangers discretion to manage their sections based on their knowledge of local conditions, with only loose guidelines and qualitative reporting. Analyze the tradeoffs using concepts from Chapters 9 (distributed vs. centralized), 12 (satisficing), 14 (overfitting), 15 (Goodhart's Law), and 16 (legibility). Which approach better serves biodiversity, and why? Under what conditions might approach A be preferable?