Chapter 33 — Key Takeaways (OpenMP)

A one-page reference for shared-memory parallelism with OpenMP directives. Scan it; the data-scoping table is the one to internalize.

The mental model: fork–join

master --serial--> !$omp parallel [FORK team] --all threads run--> !$omp end parallel [JOIN] --serial-->

Add parallelism one region at a time. Directives are comments (!$omp …): no -fopenmp ⇒ correct serial program; -fopenmp ⇒ parallel.

Directives introduced

Directive What it does
!$omp parallel … !$omp end parallel Fork a team; every thread runs the block.
!$omp do … !$omp end do Work-share a loop's iterations across the team (inside a parallel region).
!$omp parallel do … !$omp end parallel do Fork + work-share a loop in one directive (the common case).
!$omp sections / !$omp section Give different code blocks to different threads (task parallelism).
!$omp workshare Parallelize whole-array Fortran statements (unevenly optimized; prefer do).
!$omp barrier All threads wait until every thread arrives.
!$omp critical … !$omp end critical One thread at a time (general mutual exclusion; relatively slow).
!$omp atomic Make the single next x = x op expr update indivisible (cheap).
!$omp simd` / `!$omp do simd Vectorize a loop within a thread / thread + vectorize together.
!$omp task … !$omp end task, !$omp taskwait Queue irregular/recursive units of work for the team.
!$omp single Exactly one thread runs the block (others skip to the implicit barrier).

Runtime library (use omp_lib)

Call Returns
omp_get_thread_num() This thread's id, 0 … nthreads-1 (master = 0).
omp_get_num_threads() Team size in the current region.
omp_get_max_threads() Threads a parallel region would use.
omp_set_num_threads(n) Set the default team size.
omp_get_wtime() Wall-clock seconds (for timing).

Data scoping — the heart of the chapter

Attribute Meaning Use for
shared(x) One instance, all threads see it Read-only data; distinct-cell output arrays
private(x) One uninitialized copy per thread Loop indices, scratch temporaries
firstprivate(x) Private, initialized to the pre-region value Scratch that must start from a set value
lastprivate(x) Private; last iteration's value copied out When the final scratch value must survive
reduction(op:x) Private per thread, combined by op at the end Running sum/product/max/min/count
default(none) Force explicit scoping of every variable Always. Turns a silent race into a compile error.

Rules of thumb: read-only → shared; scratch & loop indices → private; per-thread result to combine → reduction. The outer (!$omp do) index is auto-private; inner indices and temporaries are your job.

The archetypal bug, and its cure

! WRONG: data race -- s read-add-written by all threads, wrong & run-to-run-varying
!$omp parallel do shared(s, x) private(i)
do i = 1, n;  s = s + x(i);  end do

! RIGHT: private partial sums, combined once at the end -- correct on any thread count
!$omp parallel do default(none) shared(x) private(i) reduction(+:s)
do i = 1, n;  s = s + x(i);  end do

Reduction identities (each thread's private copy starts here)

op + * max min .and. .or.
identity 0 1 −∞ (-huge) +∞ (huge) .true. .false.

Combine per-thread results: prefer, in order

reduction (no contention) → atomic (one cheap indivisible update) → critical (general but serializing). A critical in a hot loop reintroduces Amdahl's serial fraction.

Scheduling — schedule(kind[,chunk])

Kind Assignment Best for
static Equal contiguous chunks, decided up front; reproducible Uniform work (the stencil)
dynamic Grab-a-chunk at run time; self-balancing Uneven / lumpy work
guided Like dynamic, shrinking chunks Uneven work, less hand-out overhead

Pitfalls (all silent — no crash, no warning)

  • The shared-accumulator race — write to one shared scalar in a parallel loop ⇒ wrong, nondeterministic. Use reduction.
  • Unscoped inner index / temporary — the inner j and any rowsum-style scratch must be private. default(none) catches it at compile time.
  • Work-sharing with no team — !$omp do outside a parallel region runs serially; no parallelism.
  • False sharing — threads writing adjacent slots of a per-thread array share a cache line ⇒ correct but slow. Use reduction, or pad to separate cache lines.
  • Tiny regions — forking costs ~microseconds; a region doing little work runs slower parallel. Make regions big; hoist them out of the hot loop.

Compile flag introduced

$ gfortran -std=f2018 -fopenmp -Wall prog.f90 -o prog
$ OMP_NUM_THREADS=4 ./prog

-fopenmp both activates !$omp directives and links the runtime (omp_lib). Set the team size at run time with OMP_NUM_THREADS — same binary, 1 thread on a laptop or 64 on a node.

Numbers/rules worth memorizing

  • Master thread id = 0; ids run 0 … nthreads-1.
  • default(none) on every region. The compiler's default guess (shared) is the bug.
  • Determinism split: the result is deterministic (must match the serial answer); the schedule is not.
  • static is reproducible; dynamic/guided are not.

Project piece added this chapter

step(field, alpha, dt) — same frozen interface, now with !$omp parallel do over the interior sweep: fields shared, loop indices private, static schedule. The two-buffer FTCS structure (old field read, new buffer written) makes the sweep embarrassingly parallel, so the OpenMP result equals the Chapter 24 result on any thread count. Coarray and MPI variants of the same step follow in Chapters 32 and 34. Full directive/clause reference: Appendix G.