Chapter 18 Key Takeaways: Information Theory & Music

Shannon's Framework

Information = surprise. Claude Shannon (1948) defined the information content of a message with probability P as I = -log₂(P) bits. Rare events carry more information than common ones.

Entropy H = -Σ P(m)log₂P(m) is the average information content of a source — the expected surprise per message.

• Maximum entropy (all outcomes equally likely): H = log₂(N) bits
• Minimum entropy (one outcome certain): H = 0
• Music occupies the interesting middle: structured but not monotonous

Aiko's Key Insight

Aiko Tanaka's entropy experiment revealed the critical distinction:

High Shannon entropy ≠ structural sophistication

• Her composition: higher conditional entropy (more statistically unpredictable)
• Bach's chorale: lower conditional entropy (more predictable given context)
• But Bach's low entropy reflects sophisticated tonal/contrapuntal grammar; Aiko's high entropy reflected lack of consistent grammar — closer to controlled randomness

The lesson: Structural complexity lies in the rules (low Kolmogorov complexity), not in the unpredictability of the output (high Shannon entropy). The most creatively sophisticated music often has low conditional entropy because it follows an elegant, constraining system.

Conditional Entropy

Context radically changes information content. The conditional entropy of a note given its previous k notes (bigram, trigram models) is much more musically informative than unigram entropy:

• Tonal music: entropy drops rapidly with context (strong grammar)
• Atonal music: entropy drops slowly with context (weak consistent grammar)
• White noise: entropy does not drop with context (no grammar)

ITPRA Theory (David Huron)

Five stages of musical expectation processing: 1. Imagination — brain generates predictions (computes probability distributions) 2. Tension — arousal as event approaches 3. Prediction — comparison of actual vs. predicted 4. Reaction — automatic response proportional to information content (surprise) 5. Appraisal — conscious evaluation of whether the surprise was good

Neuroscience confirms: the ERAN brain potential scales with harmonic unexpectedness; dopamine peaks at tension-release moments; prediction error drives musical engagement.

Tonality as Information Compression

Tonal harmony is a grammar — a set of rules that reduces the conditional entropy of pitch sequences. Learning to hear tonally is learning a grammar, which: - Reduces cognitive load per note (frees attention for higher-level processing) - Increases sensitivity to meaningful deviations (deceptive cadences, chromatic inflections) - Enables multi-scale musical comprehension (from note to movement)

Genre and Entropy

Spotify-scale data shows that entropy measures are effective genre markers: - High pitch/harmonic entropy: experimental, jazz - Medium entropy: classical (tonal) - Low entropy: pop, gospel - Low rhythmic entropy: EDM, minimalism

Entropy preference correlates with personality (openness to experience), suggesting that music serves different cognitive needs for different listeners.

Lempel-Ziv and Kolmogorov Complexity

LZ complexity: Measures structural novelty (how much new subpattern the sequence introduces). Low LZ complexity = motific, repetitive structure. High LZ complexity = structurally novel.

Kolmogorov complexity: Length of shortest program generating the sequence. Captures the complexity of the rules, not just the output. Musical creativity = low K (elegant rules) generating high H (surprising output).

The Limits of Information Theory

Information theory explains much but not all of music:

Three Themes in This Chapter

1. Reductionism vs. Emergence: Information theory provides a rigorous, mathematical account of musical predictability. But musical meaning — what makes music beautiful, culturally significant, emotionally profound — is not reducible to entropy statistics. Information theory is necessary but not sufficient.

2. Technology as Mediator: Spotify's algorithm is the world's largest information-theoretic music system. It shapes what music billions of people hear, based on information measures that capture some but not all of what matters musically. Understanding algorithmic mediation is a form of musical literacy.

3. Constraint as Creativity: The most structurally sophisticated music (Bach, Schoenberg) often has low conditional entropy — high predictability within a complex system. The constraint of an elegant grammar focuses creativity rather than limiting it.