Chapter 28 Quiz — Profiling and Benchmarking

Twenty questions to check that timing, profiling, the memory/compute distinction, cache analysis, and benchmarking discipline are solid before you start optimizing in Chapter 29. Aim for 16/20. Below that, revisit the section named in the "Topics to review" map at the end. Answers and one-line rationales are in the key.

Multiple choice

Q1. A profiler answers a question a plain timer cannot. Which? - (a) How long did the whole program take? - (b) Where in the program does the time go? - (c) Is the answer numerically correct? - (d) How much memory was allocated?

Q2. system_clock measures: - (a) CPU time consumed by the process. - (b) wall-clock (elapsed real) time. - (c) the number of floating-point operations. - (d) cache misses.

Q3. Given two counts and a count_rate, elapsed seconds is: - (a) count_end - count_start. - (b) (count_end - count_start) * count_rate. - (c) (count_end - count_start) / count_rate. - (d) count_rate / (count_end - count_start).

Q4. The main reason to use integer(int64) counters with system_clock is: - (a) it is required by the standard. - (b) a higher-resolution clock that also wraps far less often. - (c) it makes the program faster. - (d) cpu_time needs it.

Q5. Your program reports wall time 20 s and CPU time 3 s. The most likely explanation: - (a) it is compute-bound and well optimized. - (b) it is waiting — I/O-bound or contended — not computing. - (c) it is using 7 cores. - (d) the timer is broken.

Q6. To profile with gprof, the -pg flag must appear: - (a) only on the compile step. - (b) only on the link step. - (c) on both the compile and the link step. - (d) only when running the program.

Q7. In a gprof flat profile, the column that usually tells you what to optimize is: - (a) calls. - (b) % time (self time). - (c) total ms/call. - (d) the name column.

Q8. A memory-bound loop is limited primarily by: - (a) how fast the CPU can do arithmetic. - (b) how fast data can be moved from memory to the CPU. - (c) the number of function calls. - (d) the compiler version.

Q9. Arithmetic intensity is: - (a) flops per second. - (b) flops per byte of data moved. - (c) bytes per second. - (d) cache misses per instruction.

Q10. valgrind --tool=cachegrind: - (a) reads the CPU's real hardware counters. - (b) simulates a cache model and counts references and misses, deterministically. - (c) makes your program faster. - (d) only detects memory leaks.

True/False (justify in one line)

Q11. True or false: You should profile the -O0 debug build so that every function is visible.

Q12. True or false: The five-point stencil is a compute-bound kernel.

Q13. True or false: For a multithreaded program on several cores, cpu_time can report more seconds than system_clock.

Q14. True or false: A single run time, reported once, is a sound benchmark.

Q15. True or false: A routine that is 20% of the runtime can, if removed entirely, make the whole program at most $1.25\times$ faster.

Short answer

Q16. Give two distinct reasons a benchmark must warm up before it starts timing.

Q17. For "how fast can this kernel possibly go on this CPU?", which statistic of your repeated timings do you report, and why that one?

Q18. State the 80/20 rule in one sentence, and its single practical consequence for how you spend optimization effort.

What does this code print?

Q19. count_rate = 1000000, count_start = 5000000, count_end = 5250000. What does this print?

print '(f6.3)', real(count_end - count_start, dp) / real(count_rate, dp)

Q20. Given samples = [0.20_dp, 0.05_dp, 0.20_dp], what are the printed minimum and median?

print '(2f6.2)', minval(samples), median(samples)   ! median = middle of the sorted three

Answer Key

Q Answer Rationale
1 b A profiler localises time to functions/loops; a timer only totals a region.
2 b system_clock is the wall-clock (elapsed real time) intrinsic; cpu_time is CPU time.
3 c Elapsed seconds = tick difference divided by ticks-per-second.
4 b int64 selects gfortran's high-resolution monotonic clock and a huge count_max (rare wrap).
5 b Wall $\gg$ CPU means the CPU sat idle — waiting on I/O or descheduled; not a compute problem.
6 c -pg instruments both compilation and linking; omit it on either and profiling breaks.
7 b % time (self) ranks where time is actually spent — the flat profile's headline.
8 b Memory-bound = starved for data; the arithmetic units idle waiting on memory.
9 b Arithmetic intensity $I = \text{flops}/\text{byte}$; low $I$ ⇒ memory-bound.
10 b Cachegrind simulates a cache — deterministic counts, slow; perf reads real counters.
11 False Profile the build you ship (-O2); -O0 leaves in calls -O2 inlines away and misleads.
12 False It is memory-bound: ~6 flops per ~16 bytes moved, $I \approx 0.4$ — starved for data.
13 True On most implementations cpu_time sums across cores, so it can exceed wall time.
14 False One run is an anecdote; you must warm up, repeat, and report a robust statistic + spread.
15 True New time $= 0.8$ of original ⇒ speedup $1/0.8 = 1.25\times$ — the Amdahl ceiling for a 20% part.
16 — Any two of: pull data into cache; fault in memory pages; let the CPU ramp to full frequency.
17 — The minimum: every disturbance only slows a run, so the fastest is the least-contaminated, truest cost.
18 — Most runtime lives in a small fraction of code; therefore profile to find that fraction and optimize only it.
19 0.250 $(5250000-5000000)/1000000 = 0.25$.
20 0.05 0.20 min $=0.05$; sorted $[0.05,0.20,0.20]$ ⇒ median (middle) $=0.20$.

Topics to review by question

Questions Section to review
1 §28.2 (what profiling is)
2, 3, 4, 5, 19 §28.1 (cpu_time, system_clock, wall vs CPU)
6, 7 §28.2 (gprof workflow, flat profile)
8, 9, 12 §28.3 (memory- vs compute-bound, arithmetic intensity)
10 §28.4 (cachegrind)
11 §28.2 (never profile -O0)
13 §28.1 (CPU time under multithreading)
14, 16, 17, 20 §28.5 (methodology: warm-up, repetition, statistics)
15, 18 §28.5 (the 80/20 rule and the speedup ceiling)