Chapter 25 — Teaching Notes

One-line purpose. Get students to stop dumping science into anonymous text/binary and start writing self-describing, portable, labelled data — so they can write and read a NetCDF and an HDF5 file, and say which to use and why.

Key ideas to emphasize

  • "Self-describing" is the whole chapter. Say it as a slogan: the file carries its own metadata, so it means the same thing to everyone and outlives the code. Everything — NetCDF, HDF5, CF — is machinery in service of that one property. Return to it after every worked example.
  • The four failures of text, and the one raw binary does NOT fix. Bulky/slow/lossy are fixed by unformatted I/O (Ch. 7); mute is not, and raw binary makes portability worse (endianness). This is the motivation for the entire chapter — spend real time on it, ideally with the storage_estimate numbers on the board (3x, exact).
  • Check every library call. NetCDF returns a status; HDF5 hands back hdferr. The check wrapper is non-negotiable; an unchecked failure corrupts everything after it silently. Tie to the iostat habit from Ch. 7 — same idea, applied to a library.
  • Define mode vs data mode (NetCDF). Declare all structure, nf90_enddef, then write. The most common first bug is put_var before enddef (the §25.4 Find-the-Bug). Make them recite the order.
  • HDF5 handles are not Fortran objects. You h5open_f/h5close_f the library and close every dataspace/dataset/property list by hand — the compiler will not, because these are opaque library handles, not allocatables. The loop-that-leaks-dataspaces bug (Ex 25.18) makes it visceral.
  • Chunking is the key HDF5 idea; compression rides on it. Chunk = tile = the atomic unit of I/O and compression. Chunk shape should match the typical read (Case Study 2's whole point). Compression requires chunking because the filter runs per chunk.
  • CF conventions = reproducibility. Units, coordinate variables, and provenance (history, source) are three lines that save afternoons and make a file scientifically usable. standard_name is a controlled vocabulary — never invent one.
  • NetCDF-4 IS HDF5. Students think they must choose; often they don't — NetCDF-4 gives HDF5's speed with NetCDF's conventions.

Misconceptions to preempt

  • "Full precision text is fine for an archive." (It fixes lossiness, not muteness — Case Study 1.)
  • "ncdump shows var(y,x) so my data is transposed." (No — display convention; the library bridges row-major and column-major. Reversing the dimension list would cause a transpose.)
  • "Unformatted binary is portable." (Endianness + record markers; it is a same-machine scratch format.)
  • "HDF5 cleans up when the variable goes out of scope, like allocatable." (No — opaque handles; close them.)
  • "I can make up a standard_name." (Controlled vocabulary; use long_name.)
  • "nf90_open(..., NF90_NOWRITE) then write." (Read-only handle; open NF90_WRITE.)
  • "Compression is a switch I flip on any dataset." (Requires chunking; a contiguous dataset can't deflate.)
  • "Bigger deflate level is always better." (Level 9 buys little extra ratio on smooth fields for lots of CPU; 4–6 is the sweet spot. And it depends on your bottleneck.)
  • "The byte total is just an integer." (8e9 overflows 32-bit integer; use int64 — Ex 25.29.)

A live demonstration (5–10 minutes)

Type example-02-netcdf-write.f90 live and compile it without the library first, so the class sees the "cannot find module netcdf" build error — then add `nf-config --fflags --flibs` and watch it resolve. This teaches the "it's a build error, not a code bug" distinction (mirrors the Ch. 21 -llapack demo). If you have NetCDF installed, run it and immediately do ncdump -h heat.nc so they see the file describe itself — the single most convincing moment in the chapter. Then deliberately move nf90_enddef below nf90_put_var and show the check wrapper catching the define-mode error. (If no library is installed, walk the predicted ncdump -h output on the board instead — the whole chapter is written to support that.)

Class-time budget (~75 min for this intermediate chapter)

  • 12 min: why text/binary don't scale — the four failures, the storage_estimate numbers, self-describing as the fix (§25.1). This is the motivation; do not rush it.
  • 18 min: NetCDF — the data model, define/data mode, the check idiom, the worked writer, ncdump, the row-major display pitfall (§25.2). The core for most students.
  • 15 min: HDF5 — groups/datasets, the h5*_f style and library init, chunking + compression via a property list, close-every-handle (§25.3).
  • 10 min: reading both back; the side-by-side API table; which-to-use (§25.4). The round-trip sum (324) as a determinacy check.
  • 10 min: CF conventions — units, coordinate variables, provenance, reproducibility, the standard_name trap (§25.5).
  • 10 min: the Project Checkpoint (write_field_netcdf on field_t) and the What's Next bridge to Ch. 26.

Prerequisites to review

Chapter 7 (unformatted/stream I/O, iostat/iomsg, write_field — the spaced-review target and the thing this chapter scales up), Chapter 9 (field_t, allocatable components — the other spaced-review target and the project's data structure), and Chapter 16 (NetCDF/HDF5 named in the ecosystem — this chapter cashes that mention). Confirm students remember why unformatted round-trips exactly and what field_t bundles.

Connections

Back: Ch. 7 (I/O foundations — direct predecessor), Ch. 9 (field_t), Ch. 16 (libraries named), Ch. 5 (column-major, the display pitfall), Ch. 15 (Python hand-off). Forward: Ch. 26 (visualization — ParaView reads CF NetCDF directly; VTK next), Ch. 28 (the profiler finds time in write), Ch. 34 (parallel/MPI-IO), Ch. 37 (reproducibility as SE), Ch. 38 (the capstone writes this output). This chapter is where the solver's output becomes a scientific artifact rather than a scratch file.