Chapter 20 — Teaching Notes

One-line purpose. Replace the student's belief that computer arithmetic behaves like real-number arithmetic with an accurate mental model (the finite, unevenly spaced grid), and give them the tools — epsilon/ULP reasoning, cancellation-avoidance, NaN/Inf detection, conditioning vs stability — that every remaining chapter of Part V assumes.

Key ideas to emphasize

  • The grid, not the line. The 🚪 threshold concept ("a real(dp) is a grid point, not a real number") is the load-bearing idea. Everything else — why == fails, why summation order matters, why cancellation is catastrophic — is a corollary. Keep returning to it.
  • Machine epsilon vs unit roundoff. Students conflate them. Fortran's epsilon = $2^{-52}$ (the gap above 1.0); the unit roundoff $u = \varepsilon/2 = 2^{-53}$ (the max relative error of one op). State the distinction once, cleanly, and use $u$ (not $\varepsilon$) in every error-budget calculation.
  • ULP grows with magnitude. The single most counterintuitive fact. spacing(1.0e6_dp) is a million times spacing(1.0_dp). Drive it home with the absorption demo: near $10^{17}$ the gap exceeds 1, so adding 1 does nothing.
  • Cancellation is self-inflicted and curable. Distinguish it from ordinary rounding. The cure is algebra (conjugate trick, quadratic-root swap, two-pass variance), never "round more carefully."
  • Conditioning ≠ stability. The §20.5 threshold concept ("conditioning is the problem's fault; stability is yours"). This is the intellectual payoff and the through-line for Chapters 21–24.

Misconceptions to preempt

  • "Double precision makes my data more accurate." (No — precision limits arithmetic error, not input error.)
  • "0.1 + 0.2 ≠ 0.3 is a Fortran bug / a Python bug." (No — it is IEEE 754, identical across languages; Python float is binary64.)
  • "Use a tiny tolerance like 1e-15 for float comparison." (Often too tight — scale the tolerance to the magnitudes and to how many operations produced the value; a few ULPs is the principled unit.)
  • "More precision fixes instability." (No — it hides it. The quad oracle in CS-02 gives the right answer, but the one-pass formula is still unstable; grow the mean and quad fails too.)
  • "NaN == NaN should be true." (No — it is the defining self-inequality; it is how you detect a NaN.)

A live demonstration (5–8 minutes)

Open a Python REPL beside a terminal. Type 0.1 + 0.2 in Python → 0.30000000000000004. Then compile and run code/example-01-real-representation.f90 → the same digits. Same for code/example-03-cancellation.f90: (big + 1) - big gives 0.0, big - big + 1 gives 1.0. The wordless punchline — the language did not matter; the hardware did — lands harder than any slide. If time permits, open an IEEE-754 bit visualizer and toggle the fraction bits of 0.1 to show the value it actually stores.

Class-time budget (~50 min)

  • 8 min: the grid model + 0.1 + 0.2 live (§20.1, threshold concept).
  • 10 min: epsilon, ULP, spacing grows with magnitude (§20.2); the error-budget rule ($u$, $Nu$).
  • 12 min: catastrophic cancellation — the conjugate trick and the quadratic-root swap, worked (§20.3); the 🐛 Find-the-Bug on stage.
  • 8 min: NaN/Inf, ieee_is_nan, NaN /= NaN, and -ffpe-trap as the debugging tool (§20.4).
  • 8 min: conditioning vs stability (§20.5) and the precision decision (§20.6) + Project Checkpoint budget.
  • 4 min: the Python bridge and what's next (Ch.21 conditioning → LAPACK).

Prerequisites to review

Chapter 3 (kinds, dp = selected_real_kind(15, 307), the _dp literal suffix) is essential — this chapter is the "why" behind that "what." A one-minute recap of Chapter 5 (arrays, column-major) supports the spaced-review and the summation-order discussion. Chapter 13's -ffpe-trap reappears in §20.4.

Connections

Back: Ch.3 (this chapter is the deep dive Ch.3 §3.2's 🔗 promised), Ch.5 (arrays), Ch.13 (-ffpe-trap). Forward: Ch.21 (conditioning → why call LAPACK), Ch.22 (step-size vs round-off), Ch.23/24 (time-stepping stability, CFL), Ch.27 (single-vs-double bandwidth), Ch.33 (reproducible parallel reductions). Naming the payoffs increases buy-in for what looks, at first, like a detour from "real" numerics.