Exercises: Coarrays

These exercises make you think in images: who owns which data, when it must be synchronized, and how a coindexed access turns a local array into a distributed one. Several ask you to predict the output of a coarray program on a given number of images — the single most useful skill, because a coarray bug is usually a wrong mental model of which copy you are touching. You can develop and test every program on one core with -fcoarray=single; the multi-image behaviour is reasoned about (and, where you have OpenCoarrays, run with cafrun -n N).

Difficulty: ⭐ warm-up · ⭐⭐ standard · ⭐⭐⭐ deeper. Solutions: worked solutions to the daggered (†) and odd-numbered problems are in appendices/answers-to-selected.md; the computational ones are also worked as a runnable coarray program in code/exercise-solutions.f90. Try every problem before you look. Compile single-image with gfortran -fcoarray=single -std=f2018 -Wall, or parallel with caf + cafrun -n N.

Part A — Images and the SPMD Model ⭐

32.1 † In one sentence each, define image, this_image(), and num_images(). Then explain what "SPMD" means and how an image uses those two intrinsics to find its share of the work.

32.2 Predict the output of this program on (a) 1 image and (b) 3 images. For (b), can you predict the order of the lines?

program p
  implicit none
  print '(a,i0,a,i0)', 'hello from ', this_image(), ' of ', num_images()
end program p

32.3 † Why is the order in which different images print to the screen undefined, and what is the standard idiom for producing deterministic, single-source output from a coarray program?

32.4 Declare, with correct syntax: (a) a scalar real coarray temp; (b) an integer array coarray hist of 10 elements per image; (c) an allocatable real coarray field of rank 2 (to be sized later). Which part of each declaration is the local shape and which is the coshape?

Part B — Codimensions and Remote Access ⭐⭐

32.5 † Given real :: a(50)[*], what does each of these refer to: a, a(10), a[3], a(10)[3]? Which of them may cross a network on a cluster, and which is always local?

32.6 (Type, compile, and run — predict first.) Each image sets token = 10*this_image(); image 1 then reads every image's token and prints it. Write the program (a scalar integer coarray, a sync all, a gather loop on image 1). Predict its output on 4 images, then confirm against code/exercise-solutions.f90.

32.7 † A coarray is declared real :: grid(nx,ny)[np,*] (corank 2). Explain what this_image(grid) returns and how it differs from this_image() with no argument. What does image_index(grid, [2,3]) give you, and why is that the inverse operation?

32.8 Write a single assignment that copies the last column of image q's copy of real :: u(n,m)[*] into the first column of the local copy. Why is a whole-column section far better here than an element-by-element loop?

Part C — Synchronization ⭐⭐

32.9 † Explain, in terms of segments, why the gather in 32.6 needs a sync all between the writes and the reads. What specifically is undefined if you omit it?

32.10 Contrast sync all and sync images. Give one situation where sync images is the better choice and say what its risk is compared to the global barrier.

32.11 † Two images each want to add their local count into a single running total held on image 1. Show how to do it correctly with a critical block, and explain why doing it without mutual exclusion can lose an update even with a sync all in place.

32.12 When would you reach for lock/unlock (with a lock_type coarray) instead of a critical construct? Name one concrete case a single critical cannot express.

Part D — Collectives and Teams ⭐⭐

32.13 † For co_sum(x), where does the result end up, and what changes if you call co_sum(x, result_image=1) instead? Which MPI routines are these two the analogues of?

32.14 (Predict, then confirm.) Every image executes v = this_image(); call co_sum(v) and w = this_image(); call co_max(w). What are v and w afterward on 5 images? On 1 image? Give the general formula for v. (Confirm with code/exercise-solutions.f90.)

32.15 † Describe precisely what call co_broadcast(x, source_image=3) does to x on every image. Why must every image call it, not just image 3?

32.16 What do teams let you do that a single flat set of images cannot, and what is the honest state of gfortran's support for form team/change team today? What would you do if you needed teams now?

Part E — Find the Bug ⭐⭐

Each snippet is wrong. Diagnose it and give the correct version.

32.17 † This "broadcast" of image 1's configuration is meant to leave cfg equal to 7 on every image, but sometimes leaves stale values elsewhere.

integer :: cfg[*]
if (this_image() == 1) cfg = 7
if (this_image() /= 1) cfg = cfg[1]     ! read image 1's value

32.18 A developer writes total[1] = total[1] + this_image() on every image (no critical, no sync), expecting total[1] to end up as $1+2+\dots+N$. Explain the two things wrong with this and give the one-line correct replacement.

32.19 † In a strip decomposition with owned columns 2:nloc+1 and halo columns 1 and nloc+2, a reader writes the right-halo exchange as u(:, nloc+2) = u(:, nloc+1)[me+1]. The stencil then gives wrong values at the strip's right edge. What is the off-by-one, and what is the correct right-hand side?

32.20 A program calls call co_sum(partial) inside if (this_image() <= num_images()/2) then ... end if — i.e. only the lower-half images call it. What goes wrong, and what is the rule about who must call a collective?

Part F — Design It and Back of the Envelope ⭐⭐⭐

32.21 † (Design it — solver.) The Project Checkpoint assumes the number of interior columns divides evenly across the images. Describe, in words and with the key index expressions, how you would extend the decomposition to handle a remainder (e.g. 5 interior columns across 2 images). What must stay true about the coarray's local shape across images, and how do you reconcile that with unequal strips?

32.22 (Back of the envelope — communication cost.) A plate is $N \times N$ and is cut into vertical strips across $P$ images. (a) How many interior cells does each image compute per step (order of magnitude)? (b) How many cells does each image communicate per step (its halo)? (c) Form the ratio (communication / computation) and explain why it shrinks as $N$ grows for fixed $P$ — the reason halo methods weak-scale well (Chapter 31).

32.23 † (Design it.) Sketch how you would decompose the plate as a 2D grid of tiles instead of 1D strips, using a corank-2 coarray u(:,:)[np,*]. What are the four halos each interior tile now exchanges, and which intrinsic maps a tile's grid position to an image number?

32.24 (Judgement.) For each scenario, choose coarrays, OpenMP, or MPI and justify in one sentence: (a) the plate fits in one node's RAM and you want a quick 8× on its cores; (b) you are contributing to an existing 200,000-line MPI weather code; (c) you want native Fortran parallelism, no external dependency in the source, that runs on both your laptop and a small cluster.

Part G — Interleaved (Chapters 5, 8, 24, 31) ⭐⭐

32.25 † (Ch. 31.) From Chapter 31, the solver was estimated at $p \approx 0.98$ parallel, ceiling $50\times$. Explain how the coarray Project Checkpoint realizes that plan: what is the data-parallel part that the images share, and what is the sequential structure the two sync alls enforce?

32.26 (Ch. 8.) When the coarray solver grows up, its field_t and procedures go in a module. Why does a coarray dummy argument specifically require the explicit interface a module provides? (Recall the same argument for class and allocatable dummies in Chapter 9.)

32.27 † (Ch. 5 + 24.) The checkpoint cuts the plate along columns so each halo is a whole column u(:,j). Using column-major order from Chapter 5, explain why that makes each halo contiguous in memory (and thus one efficient message), and what would be worse about cutting along rows.

32.28 † (Synthesis.) Write a coarray program that computes the sum of squares $\sum_{i=1}^{12} i^2$ by giving each image a contiguous chunk of the integers $1..12$, summing $i^2$ over its chunk locally, and combining the partials with co_sum. Confirm the result is 650 regardless of the image count. Why is image-count independence the signature of a correct reduction? (Worked in code/exercise-solutions.f90.)

Solutions to the daggered and odd-numbered problems are in appendices/answers-to-selected.md; the computational ones are worked as a runnable, hand-checked coarray program in code/exercise-solutions.f90. The design problems (32.21, 32.23, 32.24) have model answers plus room for your own reasoning — if your design keeps the coarray's local shape identical across images, communicates only strip edges, and synchronizes every step, you are on the right track.