Chapter 29 — Instructor Notes

Teaching Notes

Why this chapter is easy to teach badly. Optimization tempts everyone — students most of all — toward the shiny end: SIMD intrinsics, cache tricks, assembly. The chapter deliberately inverts that: the biggest wins are unglamorous (loop order, fusion), and half the wisdom is knowing when to stop and when to call a library. Teach it as discipline, not as a bag of tricks. The narrative arc is "help the compiler, don't outsmart it — except in Knuth's critical 3%." Open with the epigraph and keep returning to it.

The four ideas to emphasize, in order: 1. Loop order is the biggest reliable win, and it costs nothing. Inner loop over the first index (column-major). This is the one rule with no exceptions. Everything else is secondary. If students leave with only this, the chapter succeeded. 2. Most numerical kernels are memory-bound — the roofline threshold concept. You cannot out-compute the memory ceiling, so optimization is mostly moving less data (loop order, fusion), not doing less arithmetic. This reframes the whole subject and is the single most important conceptual takeaway. It is also what tells you cache blocking helps matmul but not a stencil sweep. 3. do concurrent is a promise, not a parallel directive. It asserts independence; the compiler trusts, does not check. Break the promise (in-place update) → undefined result. This is both a performance tool and a trap, and it is the bridge to Part VIII. 4. A tuned BLAS beats you — recognize the shape and call the library. The §29.5 / CS-02 climax. Students should leave able to recognize a library-shaped problem (dense matmul, solve, FFT) and reach for Ch. 21's BLAS instead of a loop.

Misconceptions to preempt: - "Optimizing means doing less arithmetic." Mostly false for numerical code — it means moving less data. The roofline makes this precise; hammer it. - "Cache blocking always helps." No — it helps high-reuse, compute-bound kernels (matmul) and does almost nothing for a memory-bound single stencil sweep. Students who block everything waste days. Use the stencil-vs-matmul contrast (CS-01 vs CS-02) to make it stick. - "do concurrent runs in parallel automatically." No — on most compilers today it compiles serial-but-more- vectorizable; you need flags or a specific compiler to parallelize. It permits, does not guarantee. - "A faster answer is a correct answer." No — verify. A dependency-free reorder is bit-identical (== 0); a reduction reassociates and needs a tolerance. This is the chapter's iron rule; make them state the assertion. - "I can beat the library if I try hard enough." Almost never. CS-02 is designed to humble this gently: build the best loop you can, then measure the 10× gap to dgemm. - Wrong loop order is invisible on a small test (fits in cache). Always benchmark at the real size.

Live-coding demo (25–30 min). Do NOT run timing live and trust it as gospel (variance) — instead make the correctness-preservation visceral, then reason about speed. 1. Start from example-01-loop-order.f90. Run both loop orders; show max |fast - slow| = 0.0. "Reordering never changes the answer." Then, on the board, estimate why the j-inner order strides a column each step. 2. Open example-02-fusion-doconcurrent.f90. Show the two-pass vs fused vs do concurrent all print the identical field. "Fusion and do concurrent change when, never which." 3. Compile project-checkpoint.f90 with -O3 -march=native -fopt-info-vec and read the report live: did the stencil vectorize? Then swap the buffers to plain pointer (drop contiguous) and watch the report flip to "possible aliasing." Restore contiguous; vectorization returns. This is the §29.4 lesson in 3 min. 4. If time: link OpenBLAS and time your blocked matmul vs dgemm (CS-02). The 10× gap is the "aha." NOTE for the instructor: the book never runs code, but in a live class you obviously do. Timing numbers vary run to run — teach students to repeat and take a median (Ch. 28), and treat any single number with suspicion.

Time budget (≈ 6 h, matching the estimate): - §29.1 loop order + fusion/fission + example-01/02 — 75 min (the highest-value hour; do not rush) - §29.2 cache blocking + roofline threshold concept + example-03 — 75 min - §29.3 SIMD + auto-vectorization + do concurrent — 60 min - §29.4 contiguous, aliasing, when hand-tuning wins — 45 min - §29.5 when to stop + BLAS + Project Checkpoint — 60 min - Exercises / case studies — homework

Prerequisites to review before teaching: Ch. 5 (column-major, array sections — the whole foundation), Ch. 27 (no-aliasing advantage, vectorization intro), Ch. 28 (profile first, memory- vs compute-bound — the prerequisite classification), Ch. 24 (the stencil being optimized), Ch. 11 (contiguous, pointer vs allocatable). A 10-minute warm-up recalling the Ch. 5 column-major example is the ideal on-ramp.

Assessment tip. The best single exam item is Case Study 1's scenario: "This smoothing loop is slow on large images; here is the code; diagnose and fix it, and justify each change." It tests loop order, fusion, memory-bound reasoning, correctness verification, and the discipline to stop — the whole chapter in one realistic problem. A close second: "Given this profile (96% in one loop, high cache-miss rate), what do you do first and why not touch the I/O?" (tests profile-first + Amdahl).