Chapter 32 — Key Takeaways (Coarrays)

A one-page reference to Fortran's native parallel model: images, coindexing, synchronization, and the 2018 collectives. Keep it beside you through the OpenMP and MPI chapters — the ideas map straight across.

The vocabulary

Term Meaning
image one of $N$ concurrent instances of the program; each has its own private copy of every variable
SPMD Single Program, Multiple Data: same code on every image, each on its own data share
coarray a variable declared with a codimension [*] — one collectively-addressable copy per image
codimension the [ ] bound; indexes images as ( ) indexes elements. corank = number of codimensions
coindexed access a = local copy; a[q] = image $q$'s copy (may be a network message)
segment a stretch of an image's execution between image-control statements; races live in unordered segments
collective subroutine co_sum/co_max/co_broadcast/… — combine or copy across all images; every image must call
team a named subset of images acting as a self-contained image set inside change team (weak gfortran support)

Core syntax (memorize the shapes)

real    :: a[*]                  ! scalar coarray: one copy per image
real    :: v(100)[*]             ! array coarray: 100-element array on every image
real, allocatable :: u(:,:)[:]   ! allocatable coarray; allocate(u(nx,ny)[*])
real    :: g(nx,ny)[np,*]        ! corank-2: images laid out np x (N/np)

me = this_image()                ! this image's number, 1 .. num_images()
n  = num_images()                ! total number of images
x  = a[q]                        ! GET: read image q's copy      (coindexed)
a[q] = x                         ! PUT: write image q's copy      (coindexed)
b = this_image(g)                ! cosubscripts of the caller for coarray g

Synchronization — the correctness rule

A remote read/write must be ORDERED against remote writes, or the program is UNDEFINED (a race).
The pattern:   write coarray  ->  sync all  ->  read it on another image.
Construct What it does Use for
sync all barrier: no image passes until ALL arrive the default; separate a write phase from a read phase
sync images([p]) pairwise sync with listed images only point-to-point handshakes; cheaper than a global barrier
critical … end critical one image at a time in the block shared counters/accumulators/output
lock(l) / unlock(l) (lock_type) explicit mutual-exclusion lock (a coarray) multiple locks, or a lock spanning more than one block
atomic_add/atomic_ref/… hardware-atomic single-variable op the narrow case of one scalar update, cheapest

Collectives (Fortran 2018) — the safe reductions

Call Result (on every image, unless result_image=)
call co_sum(x) sum of all images' x
call co_max(x) / call co_min(x) max / min of all images' x
call co_broadcast(x, source_image=k) image k's x, copied to all
call co_reduce(x, op) reduction by a user operation op

Every image must call a collective. co_sum(x) = MPI's Allreduce; co_sum(x, result_image=1) = MPI's Reduce. Prefer a collective over any hand-rolled coindexed reduction — it cannot race.

Building and running

Goal Command
Test logic on one core (num_images() == 1) gfortran -fcoarray=single -std=f2018 -Wall p.f90 -o p && ./p
Run on N images (needs OpenCoarrays) caf -std=f2018 -Wall p.f90 -o p then cafrun -n N ./p
Intel compiler ifx -coarray p.f90 -o p; set FOR_COARRAY_NUM_IMAGES=N

-fcoarray=single validates logic, never synchronization — one image cannot race. A green single-image test is necessary, not sufficient.

gfortran support, honestly

Feature Status
coarray syntax, this_image/num_images, coindexing, sync, critical, lock supported (multi-image via OpenCoarrays)
collectives co_sum/co_max/co_broadcast/co_reduce supported with OpenCoarrays
teams (form team/change team) incomplete/absent — prefer Intel ifx, or verify your version

When to choose coarrays

You want… Reach for
native parallelism in the language, structured-grid halo exchange, one notation for laptop and cluster coarrays (this chapter)
least-ceremony use of one node's cores, grid fits in RAM OpenMP (Ch. 33)
the most mature/universal option, an existing MPI code, finest tuning MPI (Ch. 34)

Pitfalls

  • Assuming images share memory. They do not — each has a private copy; a (no brackets) is always the local one. Coarrays are distributed memory with convenient syntax.
  • A remote access without a synchronization. Reading a[q] in a segment unordered with the write is a race — undefined, and it changes answer between runs.
  • Two images writing the same location unguarded. Lost updates. Give each image its own slot, use critical/atomic, or call a collective.
  • Relying on print order across images. Undefined; let one image do the printing after a sync.
  • Depending on teams with gfortran today. Standard but poorly supported — recognize the model, verify before building on it.

Numbers and rules worth carrying

  • A coarray is an ordinary variable + [*]; a[q] reaches image $q$'s copy.
  • Correctness in one line: write → sync all → read.
  • Halo cost is a surface, compute is a volume: the ratio $\sim 2P/N$ shrinks as the grid grows → good weak scaling.
  • Coarrays are the only standardized, in-language parallel model among mainstream languages (Fortran 2008; collectives and teams in 2018).

Project piece added this chapter

The heat solver's plate is decomposed across images into vertical strips (cut along columns, so each halo is a contiguous column), with halo exchange by coindexed reads bracketed by two sync all per step. The Chapter 24 stencil is unchanged; only which image computes which columns changes.

   sync all
   if (me > 1)  u(:,1)      = u(:,nloc+1)[me-1]   ! left  halo <- left neighbour's last column
   if (me < ni) u(:,nloc+2) = u(:,2)[me+1]        ! right halo <- right neighbour's first column
   sync all
   u_new(2:nx-1, 2:nloc+1) = u(2:nx-1,2:nloc+1) + r*( ...same five-point stencil as Ch.24... )
   u(2:nx-1, 2:nloc+1) = u_new(2:nx-1, 2:nloc+1)  ! commit; Dirichlet edges untouched

Hand-computed (2 images, r = 0.2, after 2 steps) — the same field for 1, 2, or 4 images:

  100.0  100.0  100.0  100.0  100.0  100.0
    0.0   28.0   32.0   32.0   28.0    0.0
    0.0    4.0    4.0    4.0    4.0    0.0
    0.0    0.0    0.0    0.0    0.0    0.0
    0.0    0.0    0.0    0.0    0.0    0.0

The 32 in row 2 comes across a strip boundary through the halo — proof the exchange works. Same solver goes to OpenMP threads in Chapter 33, to MPI messages in Chapter 34, and to the Chapter 38 capstone.