Chapter 33 — Teaching Notes

One-line purpose. Teach shared-memory parallelism with OpenMP directives, with data scoping as the one non-negotiable skill — and parallelize the heat solver's stencil sweep so the result still matches Chapter 24 exactly.

Key ideas to emphasize

  • Fork–join is the whole model. Serial master → fork a team at !$omp parallel → team runs the region → join back to serial. Draw it. Everything else is detail hung on this picture.
  • Data scoping is where correctness lives. Every variable in a region is shared or private. Read-only → shared; scratch and loop indices → private; a per-thread result to combine → reduction. Drill default(none) as a permanent habit — it converts the most common parallel bug into a compile error.
  • The shared-accumulator race is THE lesson. s = s + x(i) with s shared is wrong and run-to-run varying. Walk the read–add–write interleaving table (in CS-01) live; students remember the mechanism, not the slogan. Then show reduction(+:s) fixing it deterministically.
  • The determinism split. The RESULT must be deterministic (the solver's answer equals Chapter 24 on any thread count); the SCHEDULE is not (which thread does which column, in what order). "If your parallel answer differs from the serial one, that's a bug, not a speedup."
  • Overhead and false sharing are the two performance traps. Fork cost (make regions big / hoist) and false sharing (use reduction, don't hand-roll per-thread arrays). Both are silent — correct answers, bad speed.

Misconceptions to preempt

  • "!$omp do runs the loop once per thread." (No — it partitions the iterations; each runs once.)
  • "The inner loop index is private automatically." (No — only the !$omp do loop's own index is auto-private; inner indices and every scratch temporary are the programmer's job. This is the #1 nested-loop race.)
  • "default(none) prevents all races." (No — it forces you to choose an attribute; you can still choose shared for something you write. It stops accidental races, not deliberate mis-scoping. See CS-01.)
  • "A parallel loop that gives the right answer is fine." (Not if it's slower than serial — false sharing or tiny-region overhead. Correct-but-slow is still a bug.)
  • "Parallel means nondeterministic results." (No — the reduction/solver results are deterministic; only the schedule is nondeterministic. Floating-point reductions differ only in the last bit, a rounding effect, not a race — Chapter 20.)
  • "atomic and critical are interchangeable with reduction." (Reduction is faster and race-free by design; atomic/critical serialize. Rank: reduction > atomic > critical.)

A live demonstration (10 minutes)

Compile example-03-reduction.f90 two ways in front of the class. (1) As written (with reduction(+:s)): run it several times at OMP_NUM_THREADS=8 — always 5050.00. (2) Edit out the reduction, declare s shared, and run it several times — watch the number change run to run and never (reliably) hit 5050. Then restore the reduction. Nothing teaches the race like watching a sum refuse to sit still. (You are demonstrating the bug's existence and nondeterminism, not any specific wrong value.) Follow with godbolt.org (gfortran, -fopenmp -O3) to show !$omp parallel do really emits threaded code.

Class-time budget (~50 min)

  • 8 min: fork–join, the parallel region, thread num/count, -fopenmp (§33.1).
  • 8 min: work sharing — !$omp do, the static iteration→thread map (§33.2).
  • 15 min: data scoping + the reduction race — the heart; the live demo (§33.3).
  • 8 min: synchronization ranking + scheduling static/dynamic (§33.4).
  • 6 min: false sharing (why correct-but-slow) + SIMD/tasks named (§33.5).
  • 5 min: parallelize the solver step; the determinism split (Project Checkpoint).

Prerequisites to review

Ch. 5 (column-major loop order, array sections — the sweep's shape and why i is the inner/private index), Ch. 24 (the step/stencil being parallelized; two-buffer FTCS), Ch. 31 (Amdahl, shared vs distributed, data vs task parallelism — the ceiling this chapter measures against). Confirm students can compile with -fopenmp and set OMP_NUM_THREADS before the lab.

Connections

Back: Ch. 5, 6, 9 (arrays, step, field_t), Ch. 24 (the solver core), Ch. 27–29 (why-fast, profiling, SIMD/do concurrent), Ch. 31 (Amdahl). Forward: Ch. 32 (coarrays — the sibling shared-memory model), Ch. 34 (MPI — distributed, the halo exchange), Ch. 35 (GPU), Ch. 38 (capstone assembles them). Emphasize the same step interface now has serial, OpenMP, coarray, and MPI bodies.