Chapter 30: Benchmarking Quantum Hardware

Every chapter since Part II has quoted device error rates. Chapter 12's median two-qubit error of 0.0078 and its 288× spread. Chapter 25's threshold comparison. Chapters 28 and 29's fidelity predictions.

This chapter asks where those numbers come from, and how much of your decision they can carry.

The answer is more interesting than "read the calibration data." The same chip, on the same day, supports a quoted two-qubit error of 0.00750 or 0.07205 — a factor of 9.6 — depending on choices that are rarely published. And the standard benchmark for gate quality, randomized benchmarking, is insensitive to readout error by construction, on a device where 12 of 127 qubits have readout error above 10% and one sits at exactly 0.5000.

There is also good news, and it is worth stating first because this chapter is otherwise a list of caveats: the median two-qubit error predicted Chapter 28's measured circuit fidelity to within 12%. The numbers work. They work for a specific reason, and knowing the reason tells you when they will stop.

30.1 What is actually recorded

A backend's Target holds a per-instruction, per-qubit error and duration. Not a number — a distribution:

   two-qubit (ecr):     144 entries, 9 DEAD (error = 1.0)
     min    0.00347       median 0.00750       p95  0.01999
     p25    0.00567       mean   0.01018       max  0.11736   (34x the min)

   single-qubit (sx):   127 entries, 0 dead
     min    0.00011       median 0.00024       p95  0.00112
                          mean   0.00056       max  0.01333   (124x the min)

   readout (measure):   127 entries, 0 dead
     min    0.00293       median 0.01978       p95  0.15859
                          mean   0.04148       max  0.50000   (171x the min)

Three things to notice before any benchmark theory.

Nine of 144 two-qubit edges are dead — error exactly 1.0, uncalibrated or failed. That is 6% of the chip's connectivity, and Chapter 29 §29.4 found what happens when your layout crosses two of them.

The mean exceeds the median everywhere, because the distributions are right-skewed: a few very bad elements pull the average up. For readout the mean is 2.1× the median.

And the spreads are enormous. 34× on two-qubit gates, 124× on single-qubit, 171× on readout. Chapter 12 measured 288× on a different snapshot. A single number summarizing this is summarizing a lot.

Why all three distributions lean the same way

The mean-above-median pattern is not an accident of this snapshot. It is what an error rate is.

An error rate has a floor and no ceiling. Nothing does better than zero, and the physics sets a practical floor some distance above it — a two-qubit gate cannot be cleaner than the coherence time, the control electronics and the calibration procedure jointly allow. Nothing bounds the other end. A qubit with a two-level-system defect sitting near its transition frequency, a coupler that drifted since the last calibration, a readout resonator whose state discrimination collapsed: each produces an element ten or a hundred times worse than its neighbours, and none of them has a counterpart on the good side. There is no mechanism that makes a qubit ten times better than the median.

A quantity with a floor, no ceiling, and multiplicative variation is right-skewed by construction, and the mean of a right-skewed distribution sits above its median. That is the whole explanation for the pattern, and it is why choosing "mean" over "median" is never a neutral choice on any error channel, on any device, in any generation of hardware.

📐 Math Aside: what a mean/median ratio tells you about the tail, and where the model breaks.

For a log-normal distribution with shape parameter $\sigma$, the mean and median are related by

$$\frac{\text{mean}}{\text{median}} = e^{\sigma^2/2} \qquad\Longrightarrow\qquad \sigma = > \sqrt{2\ln\!\left(\frac{\text{mean}}{\text{median}}\right)}$$

Every quantity on the right is already printed above, so the tail parameter comes for free. And the model makes a falsifiable prediction: a log-normal's 95th percentile is $e^{1.645\sigma}$ times its median, and the table above prints the real p95 for all three channels.

text channel median mean sigma model p95 actual p95 model/actual two-qubit ecr 0.00750 0.01018 0.782 0.02713 0.01999 1.36 single-qubit sx 0.00024 0.00056 1.302 0.00204 0.00112 1.82 readout 0.01978 0.04148 1.217 0.14644 0.15859 0.92

The model is good for readout and poor for the gates. It lands within 8% on the readout channel and over-predicts the gate tails by 36% and 82%.

That failure is informative rather than embarrassing. A log-normal is a smooth, single-mode description, and the gate channels are not smooth — they are a tight core of well-calibrated elements plus a handful of pathological ones, which is a mixture of two populations rather than one stretched population. The mean/median ratio cannot tell those apart, which is the point: a two-number summary cannot distinguish "everything is somewhat variable" from "almost everything is fine and a few things are broken", and those two chips call for completely different programs.

A dead element is not an error rate

The nine entries at exactly 1.0 deserve their own sentence, because they are the single largest source of confusion in this chapter.

An error of 1.0 is not a measurement of a very bad gate. It is a placeholder meaning no usable calibration exists for this element — the pair failed its calibration routine, or was never calibrated in this cycle, or has been administratively disabled. A gate with a true error rate of 1.0 would be a gate that reliably produces the wrong answer, which is a perfectly usable gate if you invert it. That is not what these are.

They are missing data wearing a number's clothes, and every summary statistic you compute treats them as data. §30.3 is what that costs.

30.2 Randomized benchmarking

The standard method for gate quality, and it is genuinely clever.

Apply $m$ random Clifford gates, then the single Clifford that inverts their product. A perfect device returns $|0\dots0\rangle$ every time. A noisy one decays, and the decay rate gives the average error per Clifford.

def rb_circuit(n, m, rng):
    qc = QuantumCircuit(n, n)
    total = Clifford(QuantumCircuit(n))
    for _ in range(m):
        c = random_clifford(n, seed=int(rng.integers(1 << 31)))
        qc.compose(c.to_circuit(), inplace=True)
        total = total.compose(c)
    qc.compose(total.adjoint().to_circuit(), inplace=True)
    qc.measure(range(n), range(n))
    return qc

Run it against a known uniform depolarizing error of 0.2% to check it works:

      m      P(0)      std
      1    0.9955   0.0016
      2    0.9928   0.0019
      4    0.9883   0.0036
      8    0.9803   0.0028
     16    0.9605   0.0060
     32    0.9303   0.0060

   fitted decay p = 0.99544   error per Clifford = 0.00228

It recovers the injected error. RB works, and the reason it is the standard is that it is self-calibrating — the fit's decay rate is independent of state-preparation and measurement error, because those affect the amplitude $A$ and offset $B$ rather than the exponent.

⚛️ The Physics Underneath: why RB is robust, and what that robustness costs.

Twirling over the Clifford group converts an arbitrary noise channel into a depolarizing channel with the same average fidelity. That is what makes a single exponential fit valid, and it is why RB gives the same answer regardless of what the noise actually is.

The same twirl is what throws away the structure. Coherent errors, correlated errors, and gate-dependent errors all get averaged into one number, and the averaging is not a limitation of the analysis — it is the mechanism.

Why the decay is a single exponential

The exponential fit is not an empirical convenience that happens to work. It is forced, and seeing why tells you exactly when it stops being forced.

Write the noisy implementation of a Clifford $C$ as the ideal unitary followed by a fixed error channel $\mathcal{E}$. Because the sequence is uniformly random, what the experiment actually applies between one ideal Clifford and the next is not $\mathcal{E}$ but its twirl over the Clifford group:

$$\mathcal{E}_{\text{twirl}}(\rho) = \frac{1}{|\mathcal{C}|}\sum_{C \in \mathcal{C}} C^\dagger\,\mathcal{E}\!\left(C \rho C^\dagger\right) C$$

The Clifford group is a unitary 2-design, which means this average reproduces what you would get by averaging over the full unitary group. And averaging a channel over all unitaries leaves only the part that commutes with every unitary — a space with exactly two basis elements, the identity superoperator and the completely-depolarizing one. So the twirled channel has one free parameter, whatever $\mathcal{E}$ was:

$$\mathcal{E}_{\text{twirl}}(\rho) = p\,\rho + (1-p)\frac{I}{d}$$

Depolarizing channels compose trivially — applying $\mathcal{E}_{\text{twirl}}$ $m$ times gives a depolarizing channel with parameter $p^m$ — so the survival probability after $m$ Cliffords is $A\,p^m + B$ with $B = 1/d$ and $A$ absorbing everything at the ends. One free parameter in the channel, one exponential in the data.

This also names the assumption. The derivation used one fixed $\mathcal{E}$ for every Clifford. If the error depends on which gate you apply, or on what you applied last, the twirl no longer collapses to a single parameter, and a single exponential is no longer the right model. A visibly non-exponential RB curve is not noise in the fit — it is the gate-independence assumption failing, and that is a finding rather than a nuisance.

📐 Math Aside: the $(d-1)/d$ factor, and why the same decay means more error on more qubits.

The average gate fidelity of the depolarizing channel $\mathcal{D}(\rho) = p\rho + (1-p)I/d$ over Haar-random input states is

$$F = \int d\psi\;\langle\psi|\mathcal{D}(|\psi\rangle\langle\psi|)|\psi\rangle > = p + (1-p)\frac{1}{d}$$

because a maximally mixed output still overlaps the target state with probability $1/d$. The infidelity is therefore

$$r = 1 - F = (1-p)\left(1 - \frac{1}{d}\right) = (1-p)\frac{d-1}{d}$$

which is the formula error_per_clifford implements. Check it against §30.2's measurement: $d = 2$, $p = 0.99544$, so $r = 0.00456 \times 0.5 = \mathbf{0.00228}$ — the printed value exactly.

Now hold $p$ fixed and change $d$. On two qubits $d = 4$ and the same decay gives $r = 0.00456 \times 0.75 = 0.00342$, a factor of 1.5 larger. On $n$ qubits the factor tends to 1.

This is not a correction factor bolted on. It says that a wider register has more ways to be wrong, so the same measured decay corresponds to more error. Reporting a two-qubit RB number without saying it is a two-qubit number understates it by 50% — the first of this chapter's several "which statistic?" traps, and the least well known.

Error per Clifford is not error per gate

§30.2's validation looks like it has a discrepancy. A depolarizing error of 0.002 was injected and RB recovered 0.00228 — 14% high. Both numbers are right, and the gap between them is a unit conversion that almost every RB report skips.

Two conversions stand between them.

First, Qiskit's depolarizing_error(0.002, 1) is not an infidelity. Its argument is the channel's mixing parameter $\lambda$; the average gate infidelity is $\lambda(d-1)/d = 0.002 \times 0.5 = \mathbf{0.001}$. So the injected error per gate, in the same units RB reports, is 0.001, not 0.002.

Second, a Clifford is not a gate. RB reports error per Clifford, and each random Clifford decomposes into some number of physical gates in whatever basis the device offers. If that number is $g$, then $\text{EPC} \approx g \times \text{error per gate}$, so §30.2's numbers imply $g = 0.00228 / 0.001 = 2.28$.

That is a prediction, and it is checkable. Sampling 2,000 random single-qubit Cliffords and transpiling each into the same basis §30.2 used:

   physical gates per random 1-qubit Clifford, basis {h,s,sdg,x,y,z,sx}
     mean 2.2815 +/- 0.0233   (sd 1.0425, min 0, max 4)

     0 gates    81   0.0405
     1 gate    384   0.1920
     2 gates   683   0.3415
     3 gates   595   0.2975
     4 gates   257   0.1285

★ Measured 2.2815 against RB's implied 2.28. The 14% "discrepancy" is not a discrepancy at all — it is the compilation cost of a Clifford, and RB measured it correctly.

Keep the consequence. An error per Clifford is not comparable across devices with different native gate sets, because $g$ differs. A device whose Cliffords cost 1.4 gates and a device whose Cliffords cost 2.3 gates can have identical per-gate quality and report RB numbers 64% apart. This is the same disease as §30.3's, one layer down: the units of the quoted number depend on a choice that is not in the quoted number.

Where SPAM actually goes

The claim that RB is self-calibrating is easy to state and easy to accept without checking. It is checkable on the data already printed above.

Refitting §30.2's own decay table — fitting $\ln(P(0) - 1/2)$ against $m$ and keeping the intercept this time — gives both parameters:

   slope     -0.004572   ->  decay p   = 0.99544
   intercept -0.698443   ->  amplitude A = 0.49736      (perfect SPAM would give 0.50000)

Now the test. Scale every point's deviation from the asymptote by 0.80 — exactly what much worse state preparation and readout would do to this curve — and refit:

                          decay p     amplitude A   error per Clifford
   original data          0.99544         0.49736              0.00228
   amplitude x 0.80       0.99544         0.39789              0.00228

The amplitude moved by 20%. The decay and the error per Clifford did not move at all. Not "barely" — the fit is a straight line in $\log(P - B)$, and scaling every $y$ value by a constant shifts the intercept and leaves the slope untouched. The self-calibration is an algebraic identity, not a happy accident.

Which is exactly why §30.5 is a problem. A property that makes readout error invisible to the fit does not make readout error invisible to your circuit.

30.3 The same chip, nine different fidelities

Here is the measurement that should change how you read a spec sheet. All of these are honest summaries of the same calibration record:

   statistic                              value   implied 100-gate survival
   median, dead edges EXCLUDED          0.00750                      0.4710
   mean,   dead edges excluded          0.01018                      0.3593
   median, dead edges INCLUDED          0.00779                      0.4576
   mean,   dead edges INCLUDED          0.07205                      0.0006
   p95,    dead excluded                0.01999                      0.1328
   worst live edge                      0.11736                      0.0000

"Two-qubit error 0.00750" and "two-qubit error 0.07205" describe the same chip on the same day. Including the nine dead edges multiplies the mean by 7.1×, and the implied survival of a 100-gate circuit falls from 47% to 0.06%.

Nobody is lying. Excluding uncalibrated edges is defensible — you would not route through them anyway. Reporting a median rather than a mean is defensible — it resists outliers. Every choice here is defensible and the combination spans a factor of 9.6.

⚠️ Common Pitfall: a quoted device fidelity is a statistic, and the choice of statistic is not usually published.

Before comparing two devices, ask: median or mean? dead elements included? which gate? averaged over all qubits or the best ones? Two vendors making different defensible choices produce numbers that cannot be compared at all.

The fix is to compute the statistic you need from the full record, which is public and one API call away.

Where the 0.07205 actually comes from

The worst number in that table can be taken apart exactly, and it is worth doing, because what falls out is not an error rate.

There are 144 entries, 135 live with mean 0.01018, and 9 dead at exactly 1.0. So:

   live mean 0.01018 x 135 live edges   =  1.37430
   + 9 dead edges at error 1.0          =  9.00000
                                           -------
   sum                                  = 10.37430
   / 144 edges                          =  0.07204     (published: 0.07205)

Now look at the two terms. The nine dead edges contribute $9/144 = 0.06250$ all by themselves, and $0.06250 / 0.07205 = 0.8675$.

★ 86.75% of the "two-qubit error rate 0.07205" is not error. It is the fraction of the chip that is switched off. Every live gate on the device could be made perfect — error identically zero — and this statistic would still read 0.0625, only 13% below its current value.

That is the sharpest form of the section's point. The number is arithmetically correct, defensible in its construction, published without qualification, and it is measuring coverage, not quality. It would move if the vendor recalibrated nine couplers and would barely move if they doubled the fidelity of the other 135.

📊 What the Numbers Say: the 833× is not a range of device quality.

Carry the six statistics through to implied 100-gate survival and the spread is worse than the 9.6× in the error rates:

text median, dead EXCLUDED 0.00750 -> 0.471033 mean, dead excluded 0.01018 -> 0.359437 median, dead INCLUDED 0.00779 -> 0.457467 mean, dead INCLUDED 0.07205 -> 0.000566 833x below the top row p95, dead excluded 0.01999 -> 0.132755 worst live edge 0.11736 -> 0.000004

Exponentiation is why. A factor of 9.6 in a per-gate error becomes a factor of 833 in a 100-gate circuit, because the error enters an exponent and the gate count multiplies it. Any ambiguity in a per-gate number is amplified by your circuit's depth, so the deeper your program the less you can afford not to know which statistic you were handed.

And read the last row correctly: 0.000004 is not a prediction that anything runs at four parts per million. It is the survival of a circuit that routes 100 gates through the single worst live edge on the chip — a circuit nobody would write, described by a statistic somebody might publish.

What each statistic is a claim about

The six rows are not six estimates of one quantity. They are answers to six different questions, and naming the question is what makes each defensible:

   statistic                 the question it actually answers
   median, dead excluded     "what is a typical working edge like?"
   mean,   dead excluded     "what is the average working edge like?" -- outlier-sensitive
   median, dead included     "what is a typical entry in the table?"  -- meaningless: the
                             dead entries are not gates
   mean,   dead included     "what fraction of the chip is unusable?" -- 87% of it, per above
   p95,    dead excluded     "how bad is a bad edge I might get routed through?"
   worst live edge           "what is the floor if everything goes wrong?"

Only two of these are questions anyone actually has, and which two depends entirely on whether the transpiler is choosing your qubits (§30.4) or you are (§30.4's counterweight).

Nothing here is unique to quantum hardware. It is the ordinary problem of summarizing a skewed distribution, which the field inherited without noticing that the tails are longer here than almost anywhere. Chapter 12 measured a 288× spread on a different snapshot of a different chip. Any discipline whose dispersion runs to two orders of magnitude has to name its statistic, and this one mostly does not.

30.4 The median is predictive — and why

Having spent a section on why summary statistics mislead, the honest counterweight.

Chapter 28's Grover-like circuit used 257 two-qubit gates on this backend and measured 0.1290 of its noiseless signal. The naive prediction from the median:

$$(1 - 0.00750)^{257} = 0.1445$$

Predicted 0.1445, measured 0.1290 — a ratio of 1.12. For a one-line estimate ignoring readout, decoherence, crosstalk and single-qubit errors, that is remarkably good.

Why does the median work when the distribution is so wide? Because the transpiler picks good qubits. Chapter 29 §29.4 measured exactly this: Qiskit's VF2Layout scores candidate embeddings against the full error record, so a transpiled circuit lands preferentially on the better part of the distribution. The median describes the qubits you get, not the qubits that exist.

🔬 Honest Assessment: the median predicts your circuit because something is working to make it true.

It stops predicting the moment you pin a layout yourself. Chapter 29's hand-chosen chain sampled two dead edges and returned 0.6790 against an expected 0.9116 — the distribution's tail, not its middle.

A summary statistic is a description of a sampling process. Change the sampling and the statistic stops describing anything.

So: use the median for planning, and never for a circuit whose qubits you chose.

Why a product is the right first model

$(1-e)^{n}$ looks like a guess. It is not, and knowing where it comes from tells you the two ways it fails.

Each two-qubit gate is modelled as an independent depolarizing channel — which §30.2 just showed is what the Clifford twirl makes true for a benchmarked gate. Independent depolarizing channels compose by multiplying their parameters, so $n$ gates each surviving with probability $1-e$ give $(1-e)^n$. Take logs and the model is even simpler:

$$(1-e)^n = e^{\,n\ln(1-e)} \approx e^{-ne} \quad\text{for small } e$$

At the median, $257 \times 0.00750 = 1.9275$ and $e^{-1.9275} = 0.1455$ against the exact 0.1445 — 0.7% apart, so the exponential form is a perfectly good pocket version.

It also gives the useful constant. Circuit fidelity halves every $\ln 2 / e$ gates, which at $e = 0.00750$ is

$$\frac{\ln 2}{-\ln(1 - 0.00750)} = \frac{0.6931}{0.0075282} = 92 \text{ two-qubit gates}$$

★ On this device, at the median, your signal halves every 92 two-qubit gates. That single number does more planning work than any benchmark in this chapter: Chapter 29's linear ansatz at 15 gates spends a sixth of one halving; its full-entanglement sibling at 118 gates spends 1.3; Chapter 28's Grover circuit at 257 spends 2.8 of them.

The two failure modes are visible in the derivation. The model assumes the errors are independent — crosstalk and correlated control errors are not — and it assumes every gate draws $e$ from the same place, which §30.3 just spent a section showing is a strong claim about a 34×-wide distribution.

What the 12% is made of — and which 12%

The "within 12%" above is a real agreement and it deserves auditing, because it is the one piece of good news in the chapter and good news is where you should look hardest.

Start with a detail in the source. Chapter 28 §28.3 reports two different numbers for the same circuit at optimization level 2. Its comparison table gives an 8-seed mean of $P(\text{top}) = 0.0945$; its "what is significant" block gives $P(11111) = 0.0778$. Against the noiseless 0.6027 those are 0.1568 and 0.1291 of the signal retained — the second of which §28.3 states as "12.9%" and this chapter quotes as 0.1290. §30.4 above tests the prediction against that one.

Re-running Chapter 28's own circuit — same transpiler settings, 8 seeds, 20,000 shots on the FakeSherbrooke noise model:

   configuration              ecr mean   signal retained        sd
   FULL noise model              254.0   0.1568 +/- 0.0088    0.0248
   readout error REMOVED         254.0   0.1706 +/- 0.0110    0.0310

The first row reproduces §28.3's table exactly ($0.0945 / 0.6027 = 0.1568$). And it reframes the headline:

   prediction (1 - 0.00750)^257            = 0.1445
     vs Chapter 28's single figure 0.1290  -> ratio 1.120   (12.0% HIGH)
     vs the 8-seed mean          0.1568    -> ratio 0.921   ( 7.9% LOW)

★★ The sign of the error flips depending on which of Chapter 28's two numbers you test against. 0.1290 is not an outlier — it sits 1.12 standard deviations below the seed mean, an entirely ordinary draw — but it is a single draw, and the seed-to-seed standard deviation of this measurement is 0.0248, which is 16% of the mean. A 12% agreement is inside the noise of the thing it agrees with.

The conclusion of §30.4 survives intact: the median predicts this circuit to roughly ten percent, and that is genuinely useful. What does not survive is the precision implied by "12%". The honest claim is "the median predicts this circuit's fidelity to within about 10%, and the measurement it is being compared against has a 16% seed-to-seed spread."

This is the book's most-repeated finding arriving in a chapter about measurement discipline, which is either fitting or embarrassing depending on your mood: a result from one sample is a draw from a distribution. Chapter 27 §27.5 found a 1.0% false-failure rate from 2 failures in 200 runs that was really 0.150% at 2,000 runs. Chapter 28 §28.4 concluded levels 2 and 3 were identical from two circuits. This is the same shape, and it is in this chapter's own headline.

The readout ablation. Removing readout error from the same noise model raises retention from 0.1568 to 0.1706, so readout costs a measured factor of 0.9191. The arithmetic prediction from the median readout error over five measured qubits is $(1 - 0.01978)^5 = \mathbf{0.9049}$ — agreement to 1.6%.

But be careful with that agreement too. The difference between the two configurations is $+0.0138 \pm 0.0141$, which is 0.98 standard errors — not significant at 2σ. Eight seeds is enough to reproduce §28.3's mean and not enough to resolve an 8% effect sitting on a 16% spread. The correct statement is that the ablation is consistent with the readout arithmetic and does not establish it.

📉 Noise Report: what is actually in the residual, and why it has no single sign.

The gate-only model omits four things, and they do not all push the same way.

text readout error on 5 measured qubits (1-0.01978)^5 = 0.9049 pushes DOWN single-qubit gates, 1,749 - 257 = 1,492 median 0.00024 pushes DOWN T1/T2 over a depth-1,080 circuit Ch.39: T1 15.2-483 us pushes DOWN the metric's own uniform floor 1/32 of P_ideal pushes UP

That last term is the one nobody accounts for. "Fraction of the noiseless top-outcome probability retained" is not a fidelity. A fully depolarized 5-qubit circuit returns every one of its 32 outcomes with probability $1/32$, so it still scores $\;(1/32)/0.6027 = 0.0519$ — a circuit with zero fidelity retains 5.19% of its signal by this measure.

Subtract the floor and the picture inverts again:

text retained 0.1568 -> implied fidelity (0.1568 - 0.0519)/(1 - 0.0519) = 0.1107 gate-only prediction 0.1445 ratio 1.305 (+31%) two-term prediction 0.1307 ratio 1.181 (+18%)

Against a floor-corrected fidelity the prediction is 31% high, and the readout term now improves it rather than spoiling it. Same circuit, same noise model, same device statistic — and the prediction is 12% high, 8% low, or 31% high depending on which definition of "measured fidelity" you compare it to.

This chapter's thesis was that you cannot quote a device fidelity without naming its statistic. The same is true of your circuit's fidelity, and almost nobody names that one either.

30.5 What randomized benchmarking cannot see

RB's robustness to state-preparation-and-measurement error is advertised as a feature. It is, and it means RB is blind to something your circuit is not.

   readout error: median 0.0198, mean 0.0415, max 0.5000
   12 of 127 qubits have readout error above 10%
   1 qubit at 0.5000 -- a coin flip

A qubit whose measurement is a coin flip contributes nothing to an RB curve's decay rate — it changes $A$ and $B$, which the fit discards. It destroys any circuit that measures it.

RB also averages away, by construction:

• Coherent errors, which accumulate as $m^2$ rather than $m$ in the worst case and which the Clifford twirl converts into an equivalent depolarizing rate.
• Crosstalk, since standard RB benchmarks qubits in isolation. Simultaneous RB exists and is a different experiment.
• Gate-dependent errors, since the error per Clifford mixes however many physical gates each Clifford decomposes into.
• Drift, since the number is from whenever the calibration ran.

A benchmark that is robust to a class of errors is, from your circuit's perspective, blind to them. Robustness and blindness are the same property described from two directions.

The coin-flip qubit, in bits

"Contributes nothing" is worth making quantitative, because the scale is not linear and people routinely misjudge it.

A measurement with error rate $e$ is a binary symmetric channel, and its capacity is $1 - H(e)$ bits, where $H$ is the binary entropy. Every input below is already in this chapter:

   qubit               readout error   capacity 1 - H(e)
   best on the chip          0.00293        0.9711 bits
   median                    0.01978        0.8598 bits
   mean                      0.04148        0.7510 bits
   p95                       0.15859        0.3691 bits
   worst                     0.50000        0.0000 bits

Two things fall out. The median readout qubit already loses 14% of a bit, which is a larger tax than most people assume from a number that reads as "2% error". And the p95 qubit carries 0.37 bits — it has not degraded gracefully, it has lost nearly two-thirds of its capacity, and there are enough of those that a random 8-qubit layout has a fair chance of touching one.

The 0.5000 qubit is the limiting case and it is genuinely absolute: zero bits. Not "a poor qubit" — its output is statistically independent of its input, so no amount of shots, error mitigation or post-processing recovers anything. Chapter 13's mitigation techniques all assume the measurement carries some signal to correct.

Readout and gates in the same budget

Put readout next to the channel everyone does quote, on a real circuit.

Chapter 29's hardware-aware ansatz is 6 qubits and 15 two-qubit gates. At the medians already established:

   15 two-qubit gates    (1 - 0.00750)^15 = 0.8932    costs 10.68%
    6 measurements       (1 - 0.01978)^6  = 0.8870    costs 11.30%

★ Measuring the qubits costs slightly more than every two-qubit gate in the circuit combined — and randomized benchmarking, the source of the number in the first row, is designed not to see the second.

Do not over-read the arithmetic: these are two terms in an error budget, not a prediction of Chapter 29's measured 0.9116, which was a $1-\text{TVD}$ against a noiseless reference and does not decompose as a product. §29.4 makes that point directly, having watched a survival-product heuristic pick a chain that then lost by $-0.0157$.

And the chip-wide median flatters the readout term here. Chapter 29 §29.4 pulled the actual readout errors for the transpiler's six-qubit layout and got $P(\text{all 6 correct}) = \mathbf{0.8580}$, against the 0.8870 the median predicts — so on the layout that circuit really ran, readout cost 14.2% against the gates' 10.7%.

The comparison of magnitudes is the finding. For any shallow circuit — which is every variational circuit in Part VI, every ansatz in Chapter 36, and most of what actually runs today — readout is the dominant channel, and the standard gate benchmark reports zero information about it.

🐛 Debug This: your weekly regression collapsed and every device benchmark says nothing changed.

The symptom: output fidelity drops from ~0.85 to ~0.31 and stays there. RB is unchanged to three significant figures, Quantum Volume is identical, your equivalence checks all pass, and Chapter 26's bisection finds no divergence.

The cause is almost always that the transpiler's layout moved onto a bad qubit after a recalibration. Your regression pins no layout, so it follows wherever VF2Layout sends it — which is normally the feature that makes §30.4's median predictive, and is exactly what makes the failure untraceable when a readout error moves and the gate score does not.

The check is four lines, and it belongs in the regression itself:

python t = transpile(qc, backend, optimization_level=2, seed_transpiler=seed) used = t.layout.final_index_layout() # the physical qubits you measure ro = [backend.target["measure"][(q,)].error for q in used] print(sorted(ro, reverse=True)[:3], "worst readout on this layout")

Track that alongside your fidelity. On this device the median is 0.01978 and the worst is 0.50000, so a layout shift onto one bad qubit is a factor-of-25 change in one term of your error budget and a zero change in every published device metric.

Two more things worth printing in the same block: t.count_ops()["ecr"], because Chapter 39 measured a 2.03× swing in error across 24 transpiler seeds on a 14-qubit layout, and the number of dead elements, because Chapter 29 §29.4 routed through two of them and lost 0.23 of its fidelity.

30.6 Quantum Volume

QV is the best-known single-number device metric: the largest $2^n$ for which random square circuits — $n$ qubits, $n$ layers of random SU(4) on random pairs — beat a heavy-output threshold of $2/3$.

Its virtues are real. It is holistic, exercising gates, connectivity, routing, compilation and measurement together; it is hard to game by improving one component; and it produces a single ordered number.

Its limits follow directly from "square":

     n   SU(4) blocks   CNOTs   survival @ median   survival @ median      QV
          n*floor(n/2)    (3x)      (per block)          (per CNOT)
     2               2       6            0.9851              0.9558       4
     4               8      24            0.9416              0.8347      16
     6              18      54            0.8733              0.6660      64
     8              32      96            0.7859              0.4854     256

⚠️ Common Pitfall — an SU(4) block is not a two-qubit gate. It is three.

An earlier version of this table had one survival column, computed per block. That is the optimistic reading, and the mistake is easy to make because "$n\lfloor n/2\rfloor$ two-qubit operations" is how QV is usually described. But a general two-qubit unitary needs three CNOTs to synthesize — Chapter 4 §4.6's result, arriving here as a benchmark's hidden multiplier — and transpiling QuantumVolume(n, n) to a CNOT basis confirms the factor is exactly 3.00× at $n = 2, 4, 6, 8$, with no rounding.

At $n = 8$ that is the difference between an expected 79% survival and an expected 49%. The optimistic column is not a small correction to the pessimistic one; it is a different conclusion about whether the circuit runs at all.

QV measures width and depth together, in a fixed ratio. Most real algorithms are not square: Chapter 28's Grover circuit was 5 qubits and 257 two-qubit gates — deep and narrow. Chapter 29's hardware-aware ansatz was 6 qubits and 15 — shallow and narrow. A device tuned for square circuits is not necessarily tuned for either.

And QV uses random SU(4) on random pairs, so it samples the whole connectivity graph including the bad parts. A device with a few excellent qubits and many poor ones scores badly on QV and may run your circuit very well — which, after Chapter 29 §29.2, is exactly the situation a hardware-aware programmer engineers for.

🗝️ Version Note. QV numbers have been reported up to $2^{20}$ or higher on some devices, and the figure is a moving target. It is also increasingly de-emphasized by vendors in favour of application benchmarks and CLOPS-style speed metrics — partly because QV saturates, and partly because square circuits stopped resembling anyone's workload. Check current practice before quoting a number.

Where the 2/3 threshold comes from

The heavy-output criterion looks arbitrary. It is not, and the derivation gives you the one number QV never publishes: the circuit fidelity a device needs to pass.

A heavy output is a bitstring whose ideal probability exceeds the median ideal probability. For a Haar-random circuit on $n$ qubits with $N = 2^n$ outcomes, the ideal probabilities follow the Porter–Thomas distribution — $p$ is exponential with mean $1/N$ — so the median probability is $\ln 2 / N$. Substituting $u = Np$, which is exponential with mean 1:

$$\text{HOP}_{\text{ideal}} = \sum_{x \text{ heavy}} p(x) = \mathbb{E}\!\left[u \cdot \mathbf{1}\{u > \ln 2\}\right] = \int_{\ln 2}^{\infty}\! u\,e^{-u}\,du = \frac{1 + \ln 2}{2} = 0.8466$$

A fully depolarized device outputs the uniform distribution, and the fraction of strings that are heavy is $P(u > \ln 2) = e^{-\ln 2} = 1/2$. So the two ends are pinned:

   perfect device       HOP = 0.8466
   dead device          HOP = 0.5000
   QV threshold         HOP = 0.6667

Under a global depolarizing model with circuit fidelity $F$, the observed HOP interpolates linearly between them, $\text{HOP} = 0.5 + 0.3466\,F$, so passing requires

$$F > \frac{2/3 - 1/2}{0.8466 - 1/2} = \mathbf{0.4809}$$

★ A device passes QV at width $n$ when its $n$-qubit square circuits retain about 48% fidelity. That is the number to carry, because it converts a QV score into something you can compare against your own circuit — and it is a low bar. A circuit at 48% fidelity is not a circuit you would trust an answer from.

What a QV circuit actually costs on hardware

§30.6's table counts $n\lfloor n/2 \rfloor$, which is the number of SU(4) blocks, not the number of two-qubit gates. A general SU(4) needs three two-qubit gates to synthesize, and a heavy-hex lattice cannot host a random pairing, so routing inserts SWAPs on top. Transpiling real QV circuits at optimization level 3 onto FakeSherbrooke, 5 seeds each:

     n     QV   SU(4) blocks   3x blocks   ecr (median of 5)   routing x   survival @ median
     2      4              2           6                   3        0.50              0.9777
     4     16              8          24                  27        1.12              0.8161
     6     64             18          54                  69        1.28              0.5948
     7    128             21          63                  84        1.33              0.5313
     8    256             32          96                 135        1.41              0.3619
    10  1,024             50         150                 264        1.76              0.1370
    12  4,096             72         216                 378        1.75              0.0581

★★ A width-8 QV circuit costs 135 ecr gates on this device, not 32 — a factor of 4.2 over the block count, of which 3× is synthesis and 1.41× is routing. The routing multiplier climbs to 1.75 by $n = 12$ and is still climbing, because a random permutation on a degree-3 lattice gets steadily harder to realise as the width grows.

Now cross the two derivations. The threshold needs survival above 0.4809, and the measured gate counts say that happens at $n = 7$ (0.5313) and fails at $n = 8$ (0.3619):

   error-limited QV ceiling on this device's median two-qubit error:  2^7 = 128
   the same estimate using the BLOCK count instead of real gates:     2^13 = 8,192

The naive version is 64× too optimistic. The difference between them is entirely synthesis and routing — nothing to do with qubit quality — which is why QV is described as holistic and why it is a poor proxy for anything you would compile yourself.

⚙️ Under the Transpiler: QV is largely a routing benchmark, and it is a very noisy one.

The ecr counts above are medians over 5 transpiler seeds. The spread at $n = 8$ was 108 to 150 — a 1.39× swing in gate count from nothing but the seed, which at the median error is a swing from 0.4407 to 0.3242 in survival, i.e. straddling the 0.4809 pass threshold from below.

This is Chapter 28 §28.4's seed variance and Chapter 39's layout roulette (24 seeds, 2.03× in error) arriving in a metric that is reported as a single integer. A published QV is the best of many compilations, and vendors are explicit that heavy compilation effort is permitted — which is defensible, since the compiler is part of the machine, and which also means the number blends device quality with compiler investment in a ratio nobody discloses.

If you want the device half, transpile your circuit and count gates. That number you can attribute.

Why QV saturates, and why it is quantized

Set the survival equal to the threshold and solve for the error rate the device needs at each width:

     n      QV      ecr    error rate needed to pass
     6      64       69                     0.010554
     7     128       84                     0.008678
     8     256      135                     0.005408
    10   1,024      264                     0.002769
    12   4,096      378                     0.001935

Because the gate count grows quadratically — the measured counts run at roughly $2.1n^2$ to $2.6n^2$ — the error rate required scales as $1/n^2$ while the score scales as $2^n$. Going from QV 128 to QV 256 requires gates 1.60× better; from 128 to 1,024 requires 3.13× better.

That is the saturation, and it cuts both ways. Improving the score is hard in the region where devices currently sit, so a stalled QV number can hide real progress. And once a device does get past the hump, the exponential in $2^n$ takes over and the number starts moving in leaps that overstate the underlying improvement.

And QV is quantized. It moves only in powers of two, so a real degradation that halves your circuit's fidelity may not be enough to drop $2^7$ to $2^6$ — which is precisely why a device benchmark makes a poor regression alarm, and why Case Study 2's team watched an unchanged QV while their results collapsed.

CLOPS, and the one metric that is about the clock

Every benchmark so far measures quality. CLOPS — Circuit Layer Operations Per Second — measures speed, and it exists because Chapter 24's variational loops turned out to be throughput-bound rather than fidelity-bound. A VQE run is thousands of small circuits, and if each one costs a round trip, the device's gate quality is not what you are waiting for.

CLOPS measures the rate at which a device executes layers of parameterized circuits, including parameter updates and the classical turnaround inside the runtime. What it excludes is the queue, and Chapter 39 measured what that omission is worth:

   Ch.39   a 4,096-shot Bell job occupies the device for  6.93 ms
   Ch.39   utilization at a 5-minute queue                2.31e-05   (43,340x wall clock)
   Ch.39   120 VQE iterations as 120 jobs: 10 hours waiting, 30 seconds computing

At 2.31 × 10⁻⁵ utilization, 99.99769% of your wall clock is queue. Work the consequence:

   a   2x CLOPS improvement cuts total wall clock by 0.001155%   (1 part in  86,580)
   a  10x CLOPS improvement cuts total wall clock by 0.002079%   (1 part in  48,100)
   a 100x CLOPS improvement cuts total wall clock by 0.002287%   (1 part in  43,727)

★ A hundredfold CLOPS improvement changes your wall clock by two thousandths of one percent, and the returns are visibly saturating: 10× and 100× differ by almost nothing, because once the device time is negligible, making it more negligible achieves nothing.

That is not an argument against CLOPS — it measures a real quantity, and inside a session, where the queue is paid once, throughput genuinely dominates. It is an argument that CLOPS answers a question about the device and you have a question about your Tuesday. Chapter 39's actual lever was structural: the same 120-iteration run in a session instead of 120 jobs, and batching 100 circuits into one submission for roughly 99× the effective throughput.

💰 Cost and Queue: what this chapter's own benchmarks would cost on hardware.

§30.2's RB run is 6 sequence lengths × 12 random sequences × 2,000 shots = 144,000 shots, and that is a one-qubit experiment on a 127-qubit chip. A per-qubit RB campaign across the device is 127 times that; a per-edge two-qubit campaign across 144 edges is more.

This is why the calibration record exists and why you should read it rather than reproduce it. The vendor already ran the experiment, at a cadence you could not afford, and published the result as a Target.

The cost asymmetry is stark against Chapter 39's rate cards for one VQE run — \$50 per-minute, \$7,432 per-shot, \$185,542 on a trapped-ion machine. A benchmark you pay for in shots is a benchmark whose price scales with the thing that is already your bottleneck.

Budget the measurement that answers your question instead. §30.8's protocol — noiseless reference, noisy run, $1 - \text{TVD}$ over several transpiler seeds — costs a handful of jobs against a noise model and runs on a laptop, which is where Chapters 27, 28 and 29 ran every number in them.

30.7 Cross-entropy benchmarking, and the supremacy claims

XEB samples a random circuit's output and scores it by cross-entropy against the simulated ideal distribution. It was the basis of the 2019 quantum supremacy experiment and its successors.

Two properties are worth understanding.

It requires simulating the ideal distribution, which is exactly the thing being claimed intractable. The resolution is that verification happens at sizes where simulation is still possible, and the claim is extrapolated upward — which is why every supremacy claim has been followed by classical simulation work attempting to close the gap, and several have been substantially narrowed.

And it benchmarks a random circuit, which is the most favourable possible case for a quantum device and the least favourable for a classical simulator. Chapter 21 §21.7 and Chapter 24 §24.5 made the same point about Grover and QAOA respectively: a comparison against a task chosen to suit you is not a comparison.

None of that makes XEB dishonest — it measures what it measures well, and the experiments were genuine engineering achievements. It makes XEB a poor guide to whether a device will run your program, because your program is not a random circuit.

The estimator, derived

XEB's headline quantity is a fidelity read straight off sampled bitstrings, which sounds impossible until you see that Porter–Thomas does all the work.

Take the linear estimator. Sample bitstrings $x_1 \dots x_k$ from the device, look up each one's ideal probability $p(x_i)$ by simulation, and compute

$$F_{\text{XEB}} = N\,\langle p(x)\rangle - 1$$

Under Porter–Thomas the ideal probabilities are exponential with mean $1/N$, so:

• Sampling uniformly (a dead device) picks each $x$ with probability $1/N$, giving $\langle p \rangle = \sum_x (1/N)p(x) = 1/N$, so $F_{\text{XEB}} = 0$.
• Sampling from the ideal distribution picks $x$ with probability $p(x)$, giving $\langle p \rangle = \sum_x p(x)^2 = N\,\mathbb{E}[p^2] = N \cdot 2/N^2 = 2/N$, so $F_{\text{XEB}} = 1$.

The factor of 2 is the whole trick: the second moment of an exponential is twice the square of its mean, so a device that concentrates on high-probability strings scores exactly 1 and a device that does not scores exactly 0. Drawing Porter–Thomas samples numerically confirms it converges as $N$ grows:

   n =  8   N =    256    N*sum(p^2) - 1 = +1.2204
   n = 12   N =  4,096    N*sum(p^2) - 1 = +1.0245
   n = 16   N = 65,536    N*sum(p^2) - 1 = +0.9963

At the widths a supremacy experiment uses, the estimator is essentially unbiased. At small $n$ it is not, which is one reason XEB is not a good small-circuit diagnostic.

The assumption XEB shares with RB — and does not earn

Notice what had to be true for $F_{\text{XEB}}$ to be a fidelity. The interpolation between 0 and 1 assumes that whatever the noise does, it moves the output distribution toward uniform in proportion to the fidelity — the global depolarizing, or "white noise", model.

$$P_{\text{noisy}}(x) = F\,p(x) + (1 - F)\frac{1}{N}$$

That is exactly the same assumption §30.4's uniform floor used, and exactly the same assumption RB's exponential rests on. The difference is that RB manufactures it and XEB assumes it.

The Clifford twirl in §30.2 is an active randomization that forces an arbitrary channel into depolarizing form; that is why the exponential is valid regardless of what the noise is. XEB performs no twirl. It relies on random circuits being deep and scrambling enough that the noise behaves like white noise — a plausible claim, well supported empirically at supremacy depths, and an assumption rather than a theorem.

★ Both benchmarks give you one number by making the noise look depolarizing. Only one of them does it on purpose. And on a device where the readout distribution spans 171×, a model that says "the noise pushes everything uniformly toward uniform" is doing a great deal of averaging.

🔀 In Another Framework: Cirq ships this chapter as an API.

Qiskit gives you the Target and expects you to build the experiments. Cirq 1.7.0 ships them:

text cirq.experiments.single_qubit_randomized_benchmarking YES cirq.experiments.two_qubit_randomized_benchmarking YES cirq.experiments.random_rotations_between_grid_interaction_layers... YES cirq.experiments.linear_xeb_fidelity YES cirq.experiments.log_xeb_fidelity YES cirq.experiments.xeb_fitting YES cirq.experiments.estimate_single_qubit_readout_errors YES cirq.experiments.estimate_parallel_single_qubit_readout_errors YES

That inventory is not an accident of taste — it is the supremacy experiments' own toolchain, which is why the XEB machinery is first-class and why the random-circuit generator has "grid interaction layers" in its name.

Note the last two entries especially. Cirq provides readout-error estimation as a named experiment, including a parallel version that measures all qubits simultaneously and therefore catches the crosstalk that §30.5 lists as invisible to standard RB. Chapter 14's framework gives you the measurement this chapter says Qiskit's benchmark cannot make.

🗝️ And check the name before you import it: there is no cirq.experiments.randomized_benchmarking or cirq.experiments.cross_entropy_benchmarking in 1.7.0. The RB entry points are qubit-count specific and the XEB path went through the xeb_fitting module. Both are exactly the kind of rename that turns a five-year-old tutorial into an AttributeError.

30.8 The benchmark that answers your question

Every benchmark in this chapter answers a question about the device. You have a question about your circuit.

Run your circuit. Not a proxy for it — it, or a scaled-down version you can also simulate:

   1. Simulate it noiselessly. That is your reference distribution.
   2. Run it on the device (or a noise model built from the device).
   3. Report 1 - TVD, with a standard error over transpiler seeds.
   4. Repeat at several sizes to see how it degrades.

That is what Chapters 27, 28 and 29 all did, and it is the only measurement that answers "will this work?"

Use the device benchmarks for what they are good at:

   RB / calibration data  -> planning, and choosing between devices
   Quantum Volume         -> a coarse ordering of devices, holistically
   XEB                    -> research claims about random-circuit sampling
   YOUR CIRCUIT'S 1-TVD   -> whether your program will work

And when you cannot simulate the reference — the case that matters most — fall back to what Chapter 26 §26.8 established: verify classically where you can. Chapter 23's Shor checks its factors; Chapter 24's QAOA checks its cut. A result you can verify is a result you can benchmark, at any size.

Building a device metric that predicts your circuit

Between "read the vendor's median" and "run the whole thing" there is a cheap middle option, and it is the one this chapter has been building toward. It costs one transpile and no shots.

   1. Transpile the circuit EXACTLY as you will run it -- same optimization
      level, same seed, same backend.
   2. Read the physical qubits from t.layout.final_index_layout().
   3. For every two-qubit gate INSTANCE in the transpiled circuit, look up
      that edge's error and multiply (1 - e). Repeated edges count each time.
   4. For every qubit you MEASURE, multiply (1 - readout error).
   5. Optionally multiply the single-qubit terms. They are usually small --
      median 0.00024 here -- but Ch.28's circuit has 1,492 of them.

★ This metric never takes a statistic. There is no median in it, no mean, no decision about dead elements — a dead edge, if the layout crosses one, contributes a factor of zero and announces itself immediately, which is exactly the failure Chapter 29 §29.4 spent a section diagnosing. The fix for "which statistic?" is to not take one.

It is also the honest reading of why §30.4 worked. The chip-wide median predicted Chapter 28's circuit because VF2Layout had already selected a good sub-graph; this metric skips the coincidence and asks the qubits you were actually given.

One caveat, and it is measured. Use this to estimate, not to choose. Chapter 29 §29.4 built essentially this heuristic over two-qubit terms only, used it to search all connected 6-chains, found one scoring survival 0.9764 against the transpiler's 0.9587 — and then measured 0.8959 against 0.9116, losing by about four standard errors. Adding step 4 above would have caught it: the winning chain contained qubit 72, whose readout is 9.72% wrong, and the extended product predicts the transpiler's layout ahead by $+0.0172$ against a measured $+0.0157$.

That is one correct prediction out of the three §29.4 tried it on, which is a hypothesis rather than a validated model. A partial error budget is good enough to predict an outcome within tens of percent and not good enough to rank two candidates that differ by 2%.

Predict with it; let VF2Layout choose. That is the same division of labour §29.4 arrived at from the other direction.

🧪 Run It: find out which statistic your own device report is quoting.

Pick any backend you have access to, real or fake, and answer four questions with code rather than from its spec page.

1. Pull the full two-qubit error record and print min, p25, median, mean, p95, max, and the count of entries equal to 1.0. Compare your median against whatever the device page advertises. If they differ, you have just found which statistic the page uses.
2. Compute the mean twice, with and without the dead elements, and take the ratio. On FakeSherbrooke it is 7.1×. If yours is near 1.0, the device has no dead edges today — re-run it next week.
3. Transpile something you actually care about and apply the five-step recipe above. Print your predicted survival next to the chip-wide median prediction. The gap between them is the value the layout pass is adding for you.
4. Print the readout errors of the qubits your circuit measures, sorted worst-first. This is the number no gate benchmark reports and, for any circuit shallower than about 90 two-qubit gates, the one most likely to dominate.

Then re-run all four a week later against the same backend and diff them. §30.9's last rule — calibration is a timestamp, not a property — is a rule you will believe once you have watched one move and not before.

30.9 A protocol

   1. PULL THE FULL DISTRIBUTION, not the summary. min / p25 / median /
      mean / p95 / max, and COUNT THE DEAD ELEMENTS.

   2. COMPUTE THE STATISTIC YOU NEED. If your circuit uses 15 edges, the
      median over 144 is not your answer.

   3. USE THE MEDIAN FOR PLANNING -- it predicted Chapter 28's circuit to
      within 12%. Stop using it the moment you pin a layout.

   4. CHECK READOUT SEPARATELY. RB cannot see it, and 12 of 127 qubits
      here are above 10%.

   5. TREAT QV AS A COARSE ORDERING. It is a square-circuit benchmark and
      your circuit is not square.

   6. BENCHMARK YOUR CIRCUIT. 1 - TVD against a noiseless reference, with
      an error bar, at several sizes.

   7. RE-PULL THE CALIBRATION. It is a timestamp, not a property.

"Which statistic?" is not a hardware question

The discipline this chapter is really teaching has nothing to do with qubits, and the book has been applying it in every part.

   chapter  the same quantity, quoted two ways
   Ch.27    false-failure rate 1.0% (2 of 200 runs) -> 0.150% at 2,000 runs
   Ch.27    shot noise on a TVD at 1,000 shots: mean 0.01313, MAX over 40 runs 0.03700
   Ch.33    quantum beats LogReg by +0.0202 -- with a standard error of 0.0170
   Ch.36    LiH error 0.0201 Ha = 12.62 kcal/mol = 12.6x chemical accuracy
   Ch.39    the SAME VQE run: $50, $7,432, or $185,542 depending on the rate card
   Ch.30    two-qubit error 0.00750 or 0.07205 depending on median-vs-mean and dead edges

Every row is one measurement and two or three honest numbers. Chapter 27's TVD row is the sharpest of them, because the two statistics have different jobs: the mean, 0.01313, tells you how far a correct circuit typically lands from its exact distribution, and the max over 40 runs, 0.03700, tells you where to set a tolerance if you do not want your CI to go red on correct code. They differ by 2.8×, and picking the wrong one does not give you a slightly wrong answer — it gives you a flaky test suite or a blind one.

That is this chapter's §30.3 in a different discipline. A tolerance is a quoted statistic too, and the easy one to compute is the mean.

★ The general rule the book keeps arriving at from different directions: the easy number is almost always the flattering one, because being easy is what stopped the search. Nobody computes six statistics and picks the worst. The median gets quoted because it is the default in numpy, dead edges get dropped because dropping them requires no argument, and the seed that ran first becomes "the measurement".

The defence is procedural rather than moral. Decide which statistic answers your question before you compute any of them, and write the decision down next to the number. That is the entire content of quoted_fidelity requiring both arguments with no defaults.

Where this chapter's conclusions would flip

Each of the three headline findings has a condition that would end it, and two of them are plausible within a hardware generation.

If the nine dead edges were recalibrated, §30.3's 9.6× would mostly evaporate. The median-versus-mean and dead-in-versus-out choices would collapse to a single ratio of $0.01018 / 0.00750 = \mathbf{1.36}$. The order-of-magnitude ambiguity is not a deep property of error distributions — it is one specific failure mode, uncalibrated elements being scored as though they were terrible gates. The residual ambiguity would still be 2.67× between median and p95, which is enough to matter and not enough to make a headline.

If readout error improved tenfold, §30.5's complaint would lose most of its force. At a median readout error of 0.00198, Chapter 29's 6-qubit ansatz would pay 1.18% for its measurements against 10.68% for its gates — a factor of 9 the other way — and a gate benchmark blind to readout would be blind to something that no longer matters. Readout is the weakest of the three channels here precisely because it has had the least attention, and that is a fixable state of affairs rather than a law.

If the transpiler stopped scoring layouts, §30.4's good news would disappear entirely — and this one is under your control, which is why it is the dangerous one. The median predicts because VF2Layout samples the good tail on your behalf. Pin a layout, run on a device whose stack does not score embeddings, or route around a busy region by hand, and the chip-wide median becomes a description of a sampling process you are no longer performing.

What would not flip is the shape of the argument. A benchmark robust to a class of errors is blind to them; that is an identity, not an empirical claim, and no improvement in hardware touches it.

🗝️ Version Note: the Target is now the entire public record.

Everything in §30.1 and §30.3 comes from backend.target[instruction][qubits].error and .duration on a BackendV2. The older path — backend.properties(), .gate_error(...), .readout_error(...) on BackendV1 — is what most tutorials still show and is not the current API.

qiskit.pulse was removed in Qiskit 2.0, taking add_calibration, .calibrations, backend.defaults(), instruction_schedule_map and drive_channel with it. For this chapter that is a narrowing worth naming: the layer where these error rates are physically produced is no longer inspectable from Qiskit, so the Target's scalar error and duration per instruction are now the whole story the vendor tells you. Chapter 31 is about what lives underneath and what its removal cost.

Measured on Qiskit 2.5.1, qiskit-aer 0.17.2, qiskit-ibm-runtime 0.48.0. The layout accessor used throughout §30.5 and §30.8 is t.layout.final_index_layout(); the pre-2.0 spellings of that have changed more than once, and it is the single most common source of a benchmark script that runs without error against the wrong qubits.

What we measured

• The Target holds distributions, not numbers: two-qubit error min 0.00347, median 0.00750, mean 0.01018, max 0.11736 (34×); readout median 0.01978, mean 0.04148, max 0.50000 (171×). 9 of 144 two-qubit edges are dead.
• RB implemented from scratch recovers a known error: injected 0.002 depolarizing, fitted error per Clifford 0.00228.
• ★★ The same chip supports quoted two-qubit errors from 0.00750 to 0.07205 — a factor of 9.6 — across defensible choices of statistic. Implied 100-gate survival: 0.4710 to 0.0006.
• ★★ The median predicted Chapter 28's measured circuit fidelity to within 12% ($(1-0.00750)^{257} = 0.1445$ against a measured 0.1290) — because the transpiler picks good qubits. It fails the moment you pin a layout yourself.
• RB is insensitive to readout error by design, on a device where 12 of 127 qubits exceed 10% readout error and one is at 0.5000 — a coin flip.
• Two devices with identical mean error differ by a factor of 7.8 in INFIDELITY once a circuit can route around one bad edge (survival 0.9910 versus 0.9303).
• QV is a square-circuit benchmark using random SU(4) on random pairs — it samples the whole connectivity graph, including the parts a hardware-aware programmer deliberately avoids.
• ★ 86.75% of the "two-qubit error 0.07205" is not error at all — it is $9/144 = 0.0625$, the fraction of the chip that is switched off. Every live gate could be perfect and the statistic would fall only 13%.
• ★ RB's "error per Clifford" is a per-Clifford number: 2,000 random single-qubit Cliffords cost 2.2815 ± 0.0233 physical gates each in §30.2's basis, against the 2.28 implied by $0.00228 / 0.001$. The apparent 14% overshoot of the injected error is a compilation cost, correctly measured.
• ★ Scaling the RB curve's amplitude by 0.80 leaves the fitted decay and the error per Clifford bit-identical at 0.00228 ($A$ moves 0.49736 → 0.39789). Self-calibration is an algebraic identity, not a tendency.
• ★★ Re-running Chapter 28's circuit over 8 seeds gives 0.1568 ± 0.0088 (sd 0.0248), reproducing §28.3's table. Against that mean the median's prediction is 7.9% LOW, not 12% high — the sign flips depending on which of Chapter 28's two published numbers you test against, and 0.1290 is a single draw sitting 1.12 sd below the mean.
• ★ Removing readout error from the same noise model raises retention to 0.1706 ± 0.0110 — a measured readout factor of 0.9191 against the median's arithmetic 0.9049, agreeing to 1.6% but only 0.98 standard errors, not significant at 2σ.
• The retained-signal metric has a 5.19% floor ($\tfrac{1}{32}$ of the noiseless 0.6027): a zero-fidelity circuit still scores 5.19%. Floor-corrected, the prediction is 31% high and the readout term improves it.
• ★★ A width-8 QV circuit costs 135 ecr gates on this device, not the 32 blocks — 3× synthesis, 1.41× routing — so the error-limited QV ceiling at the median is $2^7 = 128$, where the block count would say $2^{13}$. 64× too optimistic.
• Passing QV needs only 48% circuit fidelity ($F > (2/3 - 1/2)/(0.8466 - 1/2)$, from Porter–Thomas), and the required error rate scales as $1/n^2$: QV 128 → 256 needs gates 1.60× better; 128 → 1,024 needs 3.13×.
• ★ A 100× CLOPS improvement changes wall clock by 0.0023% at Chapter 39's five-minute-queue utilization of $2.31\times10^{-5}$ — one part in 43,727.
• At the median, signal halves every 92 two-qubit gates; readout capacity is 0.8598 bits at the median qubit and exactly zero at the 0.5000 one.

The theme: a benchmark that is robust to a class of errors is blind to them — and a summary statistic describes a sampling process, so it stops describing anything the moment you change how the sampling happens. Including when the sampling process is your own choice of seed.