Chapter 39 Quiz: Self-Assessment

Instructions: Answer each question without looking back at the chapter. After completing all questions, check your answers against the key at the bottom. If you score below 70%, revisit the relevant sections before moving on to Chapter 40.

Multiple Choice

Q1. Shannon defined information as:

a) Facts or knowledge that a person possesses b) The resolution of uncertainty -- the amount by which a message reduces the receiver's uncertainty about the state of the world c) Any data stored in a computer d) Meaningful communication between two intelligent beings

Q2. A bit is:

a) The smallest unit of computer memory b) The amount of information gained from resolving a single binary uncertainty -- a fair coin flip, a yes-or-no question c) One-eighth of a byte d) A unit of data transfer speed

Q3. Shannon entropy is highest when:

a) All outcomes are certain b) One outcome is much more likely than all others c) All outcomes are equally likely -- maximum surprise per observation d) The source produces no output at all

Q4. Landauer's principle states that:

a) Information can be created from nothing b) Erasing one bit of information produces a minimum amount of heat equal to kT ln 2, establishing that information processing has an unavoidable physical cost c) Information travels at the speed of light d) More complex information requires more energy to transmit

Q5. Maxwell's demon does not violate the second law of thermodynamics because:

a) The demon does not actually exist b) The door requires energy to open and close c) The demon must store and eventually erase information about the molecules, and the energy cost of erasure (Landauer's principle) compensates for the entropy reduction in the gas d) The demon eventually gets tired

Q6. DNA stores information at a density of approximately:

a) One bit per nucleotide (A, T, G, or C) b) Two bits per nucleotide, because there are four possible nucleotides and log base 2 of 4 equals 2 c) Four bits per nucleotide, one for each type d) Eight bits per nucleotide, like a computer byte

Q7. Eigen's "error catastrophe" in molecular biology is the equivalent of:

a) A computer virus destroying data b) Shannon's channel capacity theorem -- if the genome exceeds a maximum length relative to the copying fidelity, genetic information degrades faster than selection can repair it c) A mutation that kills an organism d) The extinction of a species due to environmental change

Q8. Hayek's argument about the price system is best understood as:

a) An argument that prices should be set by government b) An argument that the price system is a distributed information-processing network that compresses vast amounts of knowledge about supply, demand, and costs into a single number -- a price c) An argument that markets are always perfectly efficient d) An argument against international trade

Q9. Akerlof's "Market for Lemons" demonstrates that:

a) Used cars are always poor quality b) Buyers always have more information than sellers c) When sellers have more information about product quality than buyers (information asymmetry), a market death spiral can occur -- good products withdraw, average quality drops, prices fall, more good products withdraw d) Government regulation always improves market outcomes

Q10. The connection between Shannon entropy and thermodynamic entropy is:

a) They share a name but are completely unrelated quantities b) Shannon entropy is a metaphor inspired by thermodynamic entropy c) They are mathematically identical in form -- both are sums of probabilities times logarithms of probabilities -- because thermodynamic entropy is, at bottom, missing information about the microscopic state of a system d) Thermodynamic entropy is always greater than Shannon entropy

Q11. Wheeler's "it from bit" thesis proposes that:

a) Information is useful for describing the physical world b) Every physical entity -- every particle, every field, even spacetime -- derives its existence from information; the universe is fundamentally made of bits rather than matter or energy c) Computers will eventually simulate the entire universe d) Biological systems process more information than physical systems

Q12. Shannon's channel capacity theorem states that:

a) Any amount of information can be transmitted over any channel if you use the right technology b) Every communication channel has a maximum rate of reliable information transmission, determined by the channel's bandwidth and signal-to-noise ratio; transmitting above this rate inevitably introduces errors c) Digital channels are always superior to analog channels d) The faster you transmit, the more information you can send without limit

Q13. The efficient market hypothesis, stated in information-theoretic terms, claims that:

a) Markets are always fair b) Market prices fully reflect all available information -- the market's communication channel is operating at capacity, with every bit of available information already incorporated into prices c) No one ever makes money in the stock market d) Government regulation is unnecessary in financial markets

Q14. Information asymmetry in health insurance leads to:

a) Lower prices for everyone b) Adverse selection -- sicker people are more likely to buy insurance, driving up costs, driving away healthier people, and creating a death spiral analogous to Akerlof's Market for Lemons c) Perfect allocation of healthcare resources d) Insurance companies always profit

Q15. The holographic principle implies that:

a) The universe is literally a hologram projected from a screen b) The maximum information content of a region of space is proportional to the area of the region's boundary, not its volume -- suggesting a deep connection between information and the geometry of spacetime c) Holograms contain more information than ordinary photographs d) Three-dimensional objects are illusions

Short Answer

Q16. In one sentence, explain why a message telling you "the sun rose this morning" contains almost no information in Shannon's sense, even though it is factually true and describes a significant event.

Q17. Explain the analogy between DNA replication and Shannon's communication channel. What plays the role of signal, what plays the role of noise, and what plays the role of error correction?

Q18. In one to two sentences, explain why the Market for Lemons is not just about cars but is a universal pattern that appears whenever information is distributed asymmetrically.

Q19. Explain in your own words what it means to say "disorder is missing information." How does this connect Shannon entropy to thermodynamic entropy?

Q20. State the threshold concept -- Information Is Physical -- and explain what it means for information to be physical rather than abstract, giving one example from physics and one from biology.

True or False

Q21. Shannon's definition of information depends on whether a human mind reads and understands the message.

Q22. Landauer's principle has been experimentally verified -- physicists have measured the heat produced by erasing a single bit.

Q23. The genetic code is a communication channel in Shannon's sense, subject to the same constraints (noise, channel capacity, error rates) that apply to telephone lines.

Q24. Hayek published his essay on the price system three years after Shannon published his information theory.

Q25. The second law of thermodynamics, interpreted through information theory, states that information about a system's microscopic state is always lost over time in a closed system.

Answer Key

Q1. b) -- Shannon defined information as the resolution of uncertainty -- how much a message reduces the receiver's uncertainty. This is an objective, measurable quantity independent of meaning or importance. (Section 39.1)

Q2. b) -- A bit is the information gained from resolving a single binary uncertainty, such as a fair coin flip. It is the fundamental unit of information. (Section 39.2)

Q3. c) -- Shannon entropy is maximized when all outcomes are equally likely, because each observation is maximally surprising. A fair coin has higher entropy than a biased coin. (Section 39.2)

Q4. b) -- Landauer proved that erasing one bit of information requires a minimum energy expenditure of kT ln 2, dissipated as heat. This establishes information processing as a physical process with unavoidable costs. (Section 39.4)

Q5. c) -- The demon must store information about each molecule and eventually erase it. The energy cost of erasure (per Landauer's principle) ensures that the total entropy of the system (gas plus demon) increases, preserving the second law. (Section 39.4)

Q6. b) -- Each nucleotide position has four possible values (A, T, G, C), and log base 2 of 4 = 2, so each position stores two bits. (Section 39.6)

Q7. b) -- Eigen's error catastrophe is the biological equivalent of Shannon's channel capacity theorem: if the genome is too long for the fidelity of the copying machinery, errors accumulate faster than selection can correct them. (Section 39.6)

Q8. b) -- Hayek argued that the price system compresses vast distributed knowledge into price signals, functioning as a distributed information-processing network that outperforms any centralized alternative. (Section 39.7)

Q9. c) -- Information asymmetry causes a market death spiral: sellers of good products withdraw because the average price does not reflect their product's quality, reducing average quality, reducing prices further, and so on until mostly lemons remain. (Section 39.10)

Q10. c) -- Shannon and Boltzmann entropy have identical mathematical form because thermodynamic entropy is fundamentally about missing information -- how much the observer does not know about the system's microscopic state. Jaynes showed this rigorously. (Section 39.8)

Q11. b) -- Wheeler proposed that information is the most fundamental entity in physics: every particle, field, and spacetime itself derives its existence from bits. (Section 39.9)

Q12. b) -- Shannon proved that every channel has a maximum reliable transmission rate (channel capacity) determined by bandwidth and signal-to-noise ratio. Above this rate, errors are inevitable regardless of technology. (Section 39.3)

Q13. b) -- The EMH claims prices fully incorporate all available information, meaning the market channel is operating at capacity. In practice, this is debated. (Section 39.7)

Q14. b) -- Adverse selection in health insurance is the Market for Lemons applied to insurance: sicker people buy more insurance, raising costs and driving out healthier participants. (Section 39.10)

Q15. b) -- The holographic principle states that maximum information scales with boundary area, not volume -- a profound constraint on the relationship between information and spatial geometry. (Section 39.9)

Q16. Sample answer: The message resolves almost no uncertainty because you already knew the sun would rise -- the event has near-certain probability, so learning it occurred provides negligible surprise and thus negligible Shannon information. (Section 39.1)

Q17. Sample answer: The functional DNA sequence is the signal; mutations, replication errors, and environmental damage are the noise; proofreading enzymes and mismatch repair systems are the error-correction codes. DNA replication is a noisy channel, and cells have evolved error-correcting codes that Shannon's theory predicts must exist for reliable transmission. (Section 39.6)

Q18. Sample answer: Whenever one party to a transaction has more relevant information than the other, the same death spiral can occur -- the better-informed party exploits the gap, the less-informed party reduces participation, and quality or trust deteriorates. This pattern appears in insurance, labor markets, financial markets, and any setting where quality is difficult to observe. (Section 39.10)

Q19. Sample answer: A system in a state of high disorder has many possible microscopic arrangements consistent with its macroscopic properties; "disorder" is the observer's lack of information about which specific arrangement obtains. Thermodynamic entropy measures this missing information, and Shannon entropy measures missing information about information sources -- the two are mathematically identical because both quantify how much the observer does not know. (Section 39.8)

Q20. Sample answer: Information Is Physical means that information is not merely an abstract description of the world but a physical quantity with measurable consequences. In physics, erasing a bit produces measurable heat (Landauer's principle). In biology, DNA stores information in physical molecules, and errors in copying have physical consequences (mutations). Information is embedded in the world, not just in our descriptions of it. (Section 39.11)

Q21. False. Shannon defined information as a property of the world -- the statistical structure of the source -- not a property of minds. The number of bits in a message does not depend on who reads it or whether anyone reads it at all. (Section 39.2)

Q22. True. In 2012, physicists at the Ecole Normale Superieure in Lyon measured the heat produced by erasing a single bit stored in the position of a silica bead, confirming Landauer's prediction. (Section 39.4)

Q23. True. DNA replication is a noisy channel with measurable error rates, channel capacity constraints (Eigen's error catastrophe), and error-correction codes -- functionally identical to the constraints Shannon identified for engineered communication channels. (Section 39.6)

Q24. False. Hayek published "The Use of Knowledge in Society" in 1945, three years before Shannon published "A Mathematical Theory of Communication" in 1948. (Section 39.7)

Q25. True. The second law, through the information-theoretic lens, states that information about a closed system's microscopic state is irreversibly lost over time -- correlations wash out, details become irrecoverable, and entropy (missing information) increases. (Section 39.8)