Further Reading: Benchmarking Quantum Hardware

Tagged Tier 1 (confident it exists and recommended) and Tier 2 (real and worth seeking, but verify the current version or URL).

Randomized benchmarking

• Magesan, Gambetta, and Emerson, "Scalable and robust randomized benchmarking of quantum processes" (2011), Physical Review Letters 106, 180504, and their longer 2012 follow-up in Physical Review A. The standard RB analysis, including the twirl argument and the fit model §30.2 implements. Read the twirl derivation specifically — it is where "robust" and "blind" turn out to be the same property. Tier 1.
• Magesan et al. on interleaved RB (2012). How to isolate a specific gate's error rather than an average over Cliffords. Exercise 30.17's subject, and the version you actually want when comparing two implementations of the same gate. Tier 1.
• Literature on the limits of RB — gate-dependent noise, non-Markovian effects, and cases where the single-exponential model fails. This is the honest counterweight to §30.2, and it is a real and active literature rather than a footnote. Tier 2.
• Simultaneous RB and crosstalk characterization. Standard RB benchmarks qubits in isolation; these measure what happens when neighbours are driven at the same time. Directly relevant to why a benchmark number does not predict a full-width circuit. Tier 2.
• Cycle benchmarking and Pauli-channel estimation. A different family that recovers structure rather than a single average — the answer to §30.5's complaint. Tier 2 — active.

Quantum Volume and holistic metrics

• Cross, Bishop, Sheldon, Nation, and Gambetta, "Validating quantum computers using randomized model circuits" (2019), Physical Review A 100, 032328. The Quantum Volume definition, the heavy-output criterion, and the statistical machinery. Read it to see how carefully the authors bound the claim — the paper is far more measured than the marketing that followed. Tier 1.
• Wack et al. on CLOPS (Circuit Layer Operations Per Second) and the quality/speed/scale framing. Exercise 30.29's subject: QV says nothing about how fast the device runs a variational loop, which after Chapter 24's four-billion-shot budget is not a side issue. Tier 1.
• Current application-benchmark suites — QED-C benchmarks, Metriq, and vendor application-level reporting. The direction the field is moving, and closer to §30.8's advice than QV is. Tier 2 — check what is current.

Cross-entropy benchmarking and the supremacy claims

• Arute et al., "Quantum supremacy using a programmable superconducting processor" (2019), Nature 574, 505. The original claim, and the XEB methodology. Tier 1.
• The classical-simulation rebuttals — tensor-network approaches from the IBM, Alibaba, and Chinese Academy of Sciences groups, among others. Read at least one alongside the original; Exercise 30.30 asks exactly this, and the exercise of tracking what was claimed against what was later simulated is the most useful thing in this section. Tier 1.
• Analyses of XEB's fidelity estimator and its assumptions. Under what conditions the cross-entropy score actually estimates circuit fidelity, and when it does not. Tier 2.

On reading calibration data

• The Target and BackendV2 API documentation. target["ecr"], .error, .duration, and the per-qubit properties. Everything in §30.1 and §30.3 comes from here, and it is public. Tier 1.
• IBM Quantum's calibration pages for a live device. Look at the distribution, then look again a week later. §30.9's last rule — "calibration is a timestamp, not a property" — is best learned by watching one move. Tier 1.
• Chapter 12's material on layout scoring, and VF2Layout / VF2PostLayout. §30.4's explanation of why the median predicts depends entirely on what these do. Tier 1.

The statistics

Mostly not quantum, and mostly more important than the quantum parts.

• Anything on summary statistics and skewed distributions. Median versus mean, and why the choice matters most exactly when the distribution has a long tail — which every error channel here does. Tier 1.
• Literature on benchmark gaming and Goodhart's law. "When a measure becomes a target it ceases to be a good measure" is the entire story of QV's rise and de-emphasis, and it is well studied outside this field. Tier 1.
• Chapter 27's further reading on statistical power. §30.8's "report $1-\text{TVD}$ with a standard error" is that chapter's discipline pointed at hardware instead of code. Tier 1.

Backward references

• Chapter 12 — where this book first read calibration data, and the 288× spread.
• Chapter 28 — the measured 0.1290 that §30.4 predicts to within 12%.
• Chapter 29 — the pinned layout that hit two dead edges, and why the median stopped predicting.

Forward references

• Chapter 31 — the level below the gate, where the error rates in this chapter are ultimately set, and where calibration actually happens.

Where to go next. If one thing: Magesan, Gambetta, and Emerson on RB, and specifically the twirl argument. Once you see that the twirl is what makes the exponential fit valid, "RB averages away the structure" stops being a criticism and becomes a description of the method — which is the correct way to hold it.

If two: add the Quantum Volume paper, read against the marketing you have seen. The paper bounds its claim carefully; the number in a press release does not, and the gap between them is this chapter's subject in miniature.

Then Chapter 31, which goes below the gate to where these numbers are made — and closes Part V.