Chapter 31 — Teaching Notes

One-line purpose. Install the mental model and the arithmetic — the end of the free lunch, Amdahl's and Gustafson's laws, the taxonomy, and a plan-before-you-code discipline — so that Chapters 32–35 land as execution of a plan rather than a grab-bag of directives.

Key ideas to emphasize

  • Moore vs Dennard. This is the most-muddled fact in the whole subject. Drill it: Moore's Law (transistor count) continued; Dennard scaling (power density) ended, which is why clocks stalled and the transistors went into more cores. If students leave with only one crisp fact, make it this one.
  • Amdahl's ceiling is the load-bearing idea. $S_{\max} = 1/(1-p)$. Have them compute it for 90/95/99/99.9% until the "last percent dominates" pattern is visceral. The threshold-concept reframe — "stop asking how many cores, start asking how small the serial fraction" — is the transferable skill.
  • Amdahl and Gustafson are not rivals. They answer different questions (finish-this-faster vs solve-bigger-in-the-same-time). The single most common student error is to think one "refutes" the other. The same 5% serial fraction giving 20× (Amdahl) and 972× (Gustafson at 1024) is the demonstration.
  • The honesty discipline, again. A speedup without its core count is not a claim. Tie parallel-efficiency ($S/N$) reporting to the book's standing rule about benchmark honesty.
  • Plan before you parallelize (§31.5). The five steps, applied to the solver, are the chapter's payoff. Most failed parallel projects skipped Step 4 (estimate with Amdahl) and Step 1 (fast serial first).

Misconceptions to preempt

  • "More cores always means proportional speedup." (Amdahl forbids it for a fixed problem.)
  • "Moore's Law ended in 2005." (No — Dennard scaling did; Moore continued.)
  • "We got 30× — it scales great." (On how many cores? Efficiency? Incomplete claim.)
  • "Let's parallelize the time loop across steps." (Hard sequential dependency: step n+1 needs step n.)
  • "Parallelism fixes slow code." (It multiplies serial performance and is capped by the serial fraction; optimize serially first.)
  • "Coarrays are just Fortran's OpenMP / just Fortran's MPI." (They are a native PGAS model spanning both.)

A live demonstration (5–8 minutes)

Put code/example-01-amdahl.f90 and example-02-gustafson.f90 on the projector. Change p from 0.95 to 0.90 to 0.99 and (hand-)read how the ceiling moves 10 → 20 → 100. Then show Gustafson at the same fraction climbing linearly. The wordless punchline: two functions, a dozen lines, and the entire economics of supercomputing is on the screen. (Do not run students' machines through a real scaling study yet — that is Chapters 33–34; here the "experiment" is arithmetic, which is the point.)

Class-time budget (~50 min)

  • 8 min: the end of the free lunch — Moore vs Dennard, clocks stalled, cores not GHz (§31.1).
  • 15 min: Amdahl's Law, the ceiling, the "last percent" pattern; the threshold reframe (§31.2, first half).
  • 8 min: Gustafson's Law and the two-questions framing (§31.2, second half).
  • 8 min: the taxonomy — shared/distributed/GPU, and why coarrays span two rows (§31.3).
  • 6 min: data vs task, strong vs weak, efficiency (§31.4).
  • 5 min: the five-step plan and the solver's Amdahl estimate (§31.5, Project Checkpoint).

Prerequisites to review

Part VII in spirit: profiling (Ch. 28) produced the serial fraction we use; column-major loop order (Ch. 27) and optimization (Ch. 29) are why "make serial fast first" is Step 1. The heat solver's two-array finite-difference update (Ch. 24) is the structure that makes the interior update data-parallel — worth a one-slide recall before §31.5.

Connections

Forward: every chapter of Part VIII executes this chapter's plan — coarrays (32), OpenMP (33), MPI (34), GPU (35) — and the capstone (38) assembles the parallel solver. The Karp–Flatt diagnostic in Case Study 1 returns implicitly whenever students later measure real scaling and find it below the Amdahl ideal.