Chapter 35 — Teaching Notes

One-line purpose. Teach the two roads onto the GPU (OpenACC directives, CUDA Fortran kernels) but, above all, install the one discipline that decides success — minimize host–device data movement — so students never build the "accelerated" code that runs slower than the CPU. Closes Part VIII.

Key ideas to emphasize

  • Separate memory is the whole story. The host and device do not share memory; every GPU program is a copy-in / compute / copy-out dance across a slow bus. If students leave with one fact, make it "move the data across the bridge as rarely as possible — once in, once out, compute many times." Everything else is detail.
  • The data region beats the clever kernel. The §35.4 pitfall (per-step copy) is THE common failure. Show Case Study 1's numbers: a genuinely 10×-faster kernel becomes a 10% slowdown purely from where the data clause sits. The fix is structural (!$acc data around the loop), not a better kernel. This is the chapter's payoff.
  • OpenACC first, CUDA Fortran to demystify. Most scientists should start and finish with OpenACC. Teach CUDA Fortran so the machine is not magic (what a kernel/thread/block IS), and so they can read NVIDIA-tuned legacy code — not because they should hand-write kernels by default.
  • The one-based thread index. i = (blockIdx%x - 1)*blockDim%x + threadIdx%x. The single most common porting bug (CUDA C is zero-based). Drill it; it is off-by-one at every block boundary if wrong.
  • Arithmetic intensity as the predictor. flops/byte tells you before you code whether an offload is transfer-bound. A stencil is low-intensity → only wins via on-device reuse (many resident sweeps). Connect to Amdahl (Ch. 31): the transfer is a NEW serial term the GPU adds.
  • Honesty: none of this was compiled (no GPU/nvfortran), and all speedups are illustrative. Model that discipline explicitly — the syntax is written carefully but flagged as unverified; students should confirm on real hardware.

Misconceptions to preempt

  • "A GPU is just a faster CPU; anything is faster on it." (No — it's a differently-shaped throughput machine; wrong-shaped work is slower.)
  • "My kernel is 20× faster, so my program is." (Transfers + serial cap it — Amdahl; often a net slowdown.)
  • "Put a copy on each kernel." (Per-step transfer — the classic killer; use a data region.)
  • "Copy the CUDA C index formula." (Zero-based → off-by-one in CUDA Fortran; subtract 1 from blockIdx%x.)
  • "A reduction is just a parallel loop." (Data race across thousands of threads; use reduction(+:...).)
  • "CPU and GPU sums must match bit-for-bit." (Parallel float sums reorder → not associative; compare by tolerance — Ch. 20/37.)
  • "CUDA Fortran is portable / gfortran can compile it." (NVIDIA-only, nvfortran only; OpenACC is the portable one.)

A live demonstration (5–8 minutes)

Put code/example-01-acc-saxpy.f90 beside example-02-cuda-saxpy.f90 on the projector — same result, one directive vs a whole kernel. Then show example-03-acc-data-region.f90 and ask "how many times does a cross the bus?" (Answer: 2, not 6.) Finally, walk Case Study 1's transfer_audit output: broken 22 s vs CPU 20 s vs fixed 2 s. The wordless punchline: the same kernel is a slowdown or a 10× speedup depending entirely on the data region. Do NOT attempt to run real GPU code — there is none here, and the point is the reasoning.

Class-time budget (~50 min)

  • 8 min: the GPU and the host/device model — separate memory, the three-step dance (§35.1).
  • 10 min: OpenACC — !$acc parallel loop, data clauses, the data region (§35.2).
  • 8 min: CUDA Fortran — kernel, launch config, the one-based index (§35.3).
  • 12 min: host–device transfer as the bottleneck; the per-step-transfer pitfall; Case Study 1 numbers (§35.4).
  • 7 min: when GPUs help vs hurt; arithmetic intensity; Amdahl with transfer (§35.5).
  • 5 min: the Project Checkpoint offload (optional/advanced) and Part VIII wrap-up.

Prerequisites to review

Ch. 31 (the GPU row of the taxonomy; Amdahl; data parallelism) and Ch. 33 (OpenMP directives — OpenACC is the same idea for a different target; reduction; the data race) are the direct scaffolding. Ch. 24's two-array FTCS stencil is the kernel being offloaded; Ch. 5's column-major order returns as GPU memory coalescing. A one-slide recall of "the stencil update is data-parallel because it reads OLD neighbours into a separate array" sets up why it offloads cleanly.

Connections

Back: the parallel-solver arc (Ch. 31 plan → 32 coarrays → 33 OpenMP → 34 MPI → 35 GPU) completes here. Forward: Part IX turns from fast to real — Ch. 36 (anatomy of a scientific code), Ch. 37 (testing/ reproducibility — where CS-02's "compare by tolerance not bits" lands), and the Ch. 38 capstone, whose most ambitious configuration is multi-GPU (one MPI rank per GPU, Ch. 34 halos between devices).