Ch37 Discussion

Discussion Guide

Prompts. 1. "A result you can't reproduce isn't a result." Push on it: is a hard-to-reproduce experimental result (a rare astronomical event, a one-off) unscientific? How is computational reproducibility different from experimental replication — and why is the computational bar arguably higher (we control every input)? 2. When is bit-for-bit reproducibility worth engineering for (pinned containers, fixed core counts, deterministic reductions), and when is chasing it a waste that a tolerance would settle? Have students name a real scenario for each. 3. The oracle problem: for a solver whose answer you genuinely cannot compute any other way, list every oracle you could still test against. Push until they produce all four sources plus "a trusted reference implementation" and "manufactured solutions." 4. Whose job is testing and reproducibility — the domain scientist's or a "research software engineer's"? What does it cost a lab to skip it, and who pays (the grad student who inherits the code; the reviewer; the field, on retraction)?

Mini group activity (≈30 min). In pairs: each pair takes the working code/project-checkpoint.f90, introduces one silent bug into heat_solver (a sign flip in the y-Laplacian, a dropped /dx**2, an off-by-one loop bound, or a dt that violates CFL), and swaps with another pair. The receiving pair runs the suite and must (a) identify which test(s) caught it, (b) explain the oracle that made the catch, and (c) name a bug that would slip through the current suite and design the test that would catch it. Debrief on which oracles are strongest and where the suite has gaps (e.g. no test currently forces the transient convergence rate — that's the capstone's job). The point students should leave with: a test suite is only as good as the bugs it can catch, and you only know that by breaking things on purpose.