Chapter 39 Exercises

How to use these exercises: Work through the parts in order. Part A builds recognition skills, Part B develops analysis, Part C applies concepts to your own domain, Part D requires synthesis across multiple ideas, Part E stretches into advanced territory, and Part M provides interleaved practice that mixes skills from all levels.

For self-study, aim to complete at least Parts A and B. For a course, your instructor will assign specific sections. For the Deep Dive path, do everything.

Part A: Pattern Recognition

These exercises develop the fundamental skill of recognizing information patterns across domains.

A1. For each of the following scenarios, identify (a) what information is being stored, transmitted, or processed, (b) what the "channel" is, (c) what constitutes "noise" in that channel, and (d) where Shannon's constraints might be operating.

a) A parent teaching a child to ride a bicycle.

b) A bee returning to the hive and performing a waggle dance to indicate the location of flowers.

c) A stock market crash triggered by a false rumor.

d) A doctor interpreting a blood test result that has a known false-positive rate.

e) A gene being copied during cell division.

f) A central bank adjusting interest rates in response to inflation data.

g) A species of moth evolving camouflage coloration over many generations.

h) A restaurant menu communicating value to potential diners.

A2. Classify each of the following as primarily an example of (i) information storage, (ii) information transmission, (iii) information processing, or (iv) information asymmetry. Some may fit more than one category -- justify your classification.

a) A library catalog.

b) The price of wheat on a commodity exchange.

c) An immune cell recognizing a pathogen it has encountered before.

d) A seller who knows their used car has a faulty transmission.

e) A fossilized tree ring recording ancient climate data.

f) A neuron firing in response to a visual stimulus.

g) A musical score being sight-read by an orchestra.

h) A job candidate with an impressive resume but poor actual skills.

A3. For each of the following pairs, explain which situation contains more information in Shannon's sense (more uncertainty resolved) and why:

a) Being told "it will rain tomorrow" in London in November vs. being told "it will rain tomorrow" in the Sahara Desert.

b) Receiving a message in English vs. receiving a message in a language you do not understand.

c) A coin that lands heads 50% of the time vs. a coin that lands heads 99% of the time.

d) Learning the winner of a two-person election vs. learning the winner of a ten-person election.

e) A DNA sequence composed entirely of the letter A vs. a DNA sequence with equal proportions of A, T, G, and C.

f) Being told the price of a stock that you already knew vs. being told the price of a stock that you had no prior information about.

A4. Identify the "Maxwell's demon" in each of the following scenarios -- the entity that appears to create order from disorder, and explain what hidden information cost prevents it from violating the "second law" of its domain.

a) A financial advisor who claims to consistently beat the market.

b) A sorting algorithm that arranges a random list into alphabetical order.

c) A refrigerator that moves heat from a cold interior to a warm room.

d) A manager who claims to identify top talent without any formal evaluation process.

e) An organism that maintains its internal order despite the tendency of all things toward disorder.

A5. For each scenario, determine whether information asymmetry is present, identify which party has more information, and predict the likely market outcome:

a) A homeowner selling a house in a neighborhood they know is about to be rezoned for industrial use.

b) An employee negotiating a salary at a company where all salaries are publicly posted.

c) A patient consulting a doctor about a medical condition.

d) A buyer purchasing organic produce at a farmers' market where they can watch the growing process.

e) An investor buying stock based on a recommendation from the company's CEO.

f) A student enrolling in an online course with no reviews or reputation data.

Part B: Analysis and Explanation

These exercises develop the ability to analyze the structural mechanisms of information across domains.

B1. Shannon defined information as "the resolution of uncertainty." Explain why this definition was revolutionary compared to the everyday meaning of "information." Provide two examples where the everyday meaning and Shannon's definition diverge -- situations where a message is informative in the everyday sense but contains little Shannon information, or vice versa.

B2. Explain Landauer's principle in your own words and trace its logical chain: a) Why does Maxwell's demon need to store information? b) Why does the demon's memory eventually fill up? c) Why does erasing information produce heat? d) Why does this resolve the apparent violation of the second law? e) What is the broader implication for the relationship between information and physics?

B3. Compare and contrast the following three "codes": a) The genetic code (DNA nucleotides to amino acids) b) The Morse code (dots and dashes to letters) c) The price system (supply/demand to price numbers)

For each, identify: the alphabet, the encoding rule, the channel, the noise sources, and the error-correction mechanisms (if any). What structural features do all three share?

B4. Akerlof's Market for Lemons describes a market death spiral driven by information asymmetry. Trace the causal chain step by step: a) Start with a market containing equal proportions of good cars and lemons. b) Explain why buyers offer an "average" price. c) Explain why good-car owners withdraw. d) Explain how this changes the composition of remaining cars. e) Show how the process feeds back on itself. f) Identify the equilibrium (or lack thereof). g) What interventions could break the spiral? (Consider warranties, inspections, certifications, regulations.)

B5. The chapter argues that Shannon entropy and thermodynamic entropy are "the same quantity, measured in different units." Develop this claim: a) What does Shannon entropy measure? b) What does thermodynamic entropy measure? c) Why does Jaynes's maximum entropy principle unify them? d) What does it mean to say that "disorder is missing information"? e) What would change about your understanding of the second law of thermodynamics if you took this claim seriously?

B6. Explain why the efficient market hypothesis is an information theory claim. What would it mean, in Shannon's terms, for a market to be "efficient"? What forms of market inefficiency correspond to specific information-theoretic failures (noise, limited bandwidth, channel capacity constraints, information asymmetry)?

Part C: Application to Your Own Domain

These exercises ask you to apply information concepts to fields you know personally.

C1. Identify a communication channel in your field or daily life. Characterize it in Shannon's terms: a) What is the source of messages? b) What is the channel (the physical medium)? c) What is the noise (what interferes with accurate transmission)? d) What is the receiver? e) Estimate whether the channel is operating near its capacity or well below it. What evidence supports your estimate? f) What would it take to increase the channel's capacity?

C2. Identify an information asymmetry in your field or organization. For the situation you identify: a) Who has more information? What specific information do they have that the other party lacks? b) What are the consequences of this asymmetry? Does it lead to adverse selection, market failure, exploitation, or misallocated resources? c) What mechanisms exist (or could exist) to reduce the asymmetry? Consider transparency, reputation systems, certification, regulation, and incentive alignment. d) Is the asymmetry inevitable, or could it be eliminated? What would be the costs and benefits of eliminating it?

C3. Identify a system in your domain that stores information. Assess it using the following criteria: a) How much information does it store (estimate in bits or bytes if possible)? b) How durable is the storage? What could cause information loss? c) What error-correction or redundancy mechanisms protect against data loss? d) How efficiently is the information compressed? Is there redundancy that could be removed, or is the redundancy serving a protective function? e) Compare your system to DNA as an information storage medium. What does DNA do better? What does your system do better?

C4. Describe a situation in your professional or personal life where you were Maxwell's demon -- where you sorted or organized something, apparently creating order from disorder. What was the hidden information cost? Where did the entropy go?

Part D: Synthesis and Cross-Chapter Integration

These exercises require integrating information concepts with ideas from earlier chapters.

D1. Information + Signal and Noise (Ch. 6) Chapter 6 discussed the universal challenge of separating signal from noise. Now that you understand Shannon's channel capacity theorem, revisit the signal-and-noise pattern. Explain why the signal-to-noise ratio is not just a practical nuisance but a fundamental physical constraint. For two of the domains discussed in Chapter 6 (astronomy, medicine, spam filtering, detective work, central banking), identify the channel, the signal, the noise, and estimate whether the domain is operating near or far from its channel capacity limit.

D2. Information + Distributed vs. Centralized (Ch. 9) Hayek argued that the price system outperforms central planning because it distributes information processing. Using Shannon's framework, explain why distributed processing has a higher aggregate channel capacity than centralized processing. What are the conditions under which centralization would be more information-efficient? (Hint: consider noise, coordination costs, and the bandwidth of the aggregation channel.)

D3. Information + Feedback Loops (Ch. 2) Every feedback loop is an information-processing circuit. Choose three examples of feedback loops from Chapter 2 and analyze each in terms of information flow: What information does the sensor collect? Through what channel is it transmitted to the controller? What noise corrupts the signal? How does the controller process the information? What information does the controller transmit back to the system? Where are Shannon's constraints binding?

D4. Information + Survivorship Bias (Ch. 37) Survivorship bias is, at bottom, an information problem: the observer has information about the survivors but is missing information about the non-survivors. Develop this connection. How does the missing information (the non-survivors) distort the observer's conclusions? In Shannon's terms, what is the entropy of the full population vs. the entropy of the visible (surviving) sample? How does this connect to Akerlof's Market for Lemons, where the visible population of cars for sale is also a biased sample?

D5. Information + The Streetlight Effect (Ch. 35) The streetlight effect is the tendency to search where information is available rather than where information is needed. Reframe the streetlight effect as an information-processing constraint: the observer faces a choice between processing high-bandwidth, low-relevance information (under the streetlight) and processing low-bandwidth, high-relevance information (in the dark). Under what conditions is it rational to search under the streetlight? Under what conditions does the streetlight effect become a systematic error?

D6. Information Across All Domains Choose a single system that you understand well (an organization, an ecosystem, a technology, a market, a biological process). Analyze it as an information-processing system by identifying: a) What information is stored and where. b) What information is transmitted and through what channels. c) What noise corrupts the signals. d) Where information asymmetry creates problems. e) Where Shannon's channel capacity is a binding constraint. f) Where entropy (disorder, information loss) is increasing. g) What error-correction mechanisms exist.

Part E: Advanced and Extended Problems

These exercises push beyond the chapter's explicit content into deeper territory.

E1. The Information Cost of Complexity The chapter argues that DNA stores about 800 megabytes of raw information. The human brain contains roughly 86 billion neurons with approximately 100 trillion synaptic connections. Estimate the information storage capacity of the brain (making and stating your assumptions). Compare this to the genome. What does the vast difference suggest about the relationship between genetic information and the information that emerges during development and learning? Is the brain's complexity "in" the genome, or does it emerge from a smaller informational specification?

E2. Information and the Arrow of Time The second law of thermodynamics gives time a direction: entropy increases toward the future. If entropy is missing information, then the second law says that information is lost over time. But Shannon's channel coding theorem says that information can be transmitted reliably over noisy channels. Reconcile these two statements. Under what conditions can information be preserved against the general tendency toward loss? What does this tell you about the relationship between life (which preserves and transmits information) and the second law (which says information should degrade)?

E3. The Limits of the Information Lens The chapter argues that viewing complex systems through the lens of information makes cross-domain patterns "snap into focus." Construct a counterargument: identify at least two important phenomena or patterns that the information lens does not illuminate well. What is missing from the information perspective? What additional frameworks (from this book or elsewhere) are needed to capture what information theory leaves out?

E4. Designing Information Markets If information asymmetry causes market failure, then designing mechanisms to reduce information asymmetry should improve market function. Choose a specific market that suffers from information asymmetry (healthcare, real estate, employment, financial markets) and design a mechanism to reduce the asymmetry. Address: (a) what information is asymmetric, (b) why the asymmetry exists, (c) what your mechanism does to reduce it, (d) what new costs or problems your mechanism creates, and (e) whether the benefits outweigh the costs.

E5. Wheeler's "It from Bit" -- Strengths and Weaknesses Wheeler's thesis that "every it derives its existence from bits" is radical and controversial. Construct the strongest argument in favor of the thesis, drawing on evidence from this chapter (Landauer's principle, black hole entropy, the holographic principle, Verlinde's entropic gravity). Then construct the strongest argument against the thesis. Consider: is the thesis testable? Is it falsifiable? Is it a scientific claim or a philosophical one? What would it take to settle the question?

Part M: Mixed Practice (Interleaved)

These exercises deliberately mix concepts from different sections and chapters to build flexible retrieval.

M1. A pharmaceutical company has developed a new drug. In clinical trials, the drug showed a statistically significant effect, but the trials were conducted only on a specific demographic group (recall the WEIRD problem from Chapter 35). The company now faces information asymmetry: it knows the drug's efficacy data, but the FDA and the public do not have access to the raw data. An independent analyst argues that the drug's true effect size is exaggerated due to survivorship bias (Ch. 37) -- only positive trials were reported. Analyze this scenario using information theory (Shannon entropy of the trial results, information asymmetry between company and regulators), the streetlight effect (are the trials measuring what matters?), and survivorship bias (what information is missing?).

M2. A social media platform's recommendation algorithm processes user behavior data (clicks, likes, shares, time spent) to predict what content each user wants to see. Analyze this system using: a) Shannon's channel capacity: what is the bandwidth of user behavior as a signal of user preferences? b) Signal and noise (Ch. 6): what is signal and what is noise in click data? c) Feedback loops (Ch. 2): how does the algorithm's output become its input? d) Information asymmetry: who knows more -- the platform or the user? About what? e) The streetlight effect (Ch. 35): is the algorithm optimizing for what is measurable or what is important?

M3. A small island nation faces a decision: should it switch from its traditional agricultural practices (diverse polyculture, knowledge transmitted orally across generations) to modern monoculture farming (using published, scientifically validated techniques)? Analyze using: a) Information storage: where is the information about each farming system stored? Which storage medium is more robust? b) Channel capacity: which system has more bandwidth for transmitting agricultural knowledge to the next generation? c) Dark knowledge (Ch. 28) and Chesterton's fence (Ch. 38): what information might be embedded in the traditional practices that is not captured in the scientific literature? d) Information asymmetry: do the agricultural advisors know more, or do the traditional farmers know more? About what?

M4. Consider the 2008 financial crisis through the lens of information theory: a) Where was information asymmetry operating? (Consider mortgage originators, securitizers, rating agencies, investors, and homebuyers.) b) Where was the channel capacity of the price system insufficient to convey the true risk? c) Where was noise mistaken for signal, or signal lost in noise? d) How does this connect to Hayek's argument that the price system is an information-processing network? e) What information was missing, and who was missing it?

M5. You are designing a new communication system for a hospital that currently relies on verbal handoffs between nurses at shift change. Using Shannon's framework: a) What is the current channel capacity of a verbal handoff? What limits it? b) What noise sources corrupt the signal? c) What information asymmetry exists between outgoing and incoming nurses? d) Design an improved system that increases channel capacity, reduces noise, and reduces information asymmetry. Justify your design choices using concepts from this chapter. e) What Chesterton's fences (Ch. 38) might exist in the current verbal handoff system that your new design could accidentally destroy?

M6. A government is deciding whether to mandate public disclosure of all corporate environmental data (emissions, waste, water usage). Supporters argue this will reduce information asymmetry and allow markets and citizens to make better decisions. Opponents argue that the data will be misinterpreted (noise), that the cost of reporting will burden businesses, and that competitors will exploit the disclosed data. Analyze using information theory, information asymmetry, channel capacity (can the public actually process the disclosed data?), and the streetlight effect (will people focus on the measurable metrics and ignore the unmeasurable ones?).