Case Study 2: Auditing Your First Hardware Result

"An unexplained discrepancy is a bug. An explained discrepancy is a measurement."

Executive Summary

You ran a Bell state and got a few percent of your shots in outcomes that should be impossible. In §2.6 the book told you this was noise. Here you check that claim yourself.

The method: read the device's calibration data, build an error budget from it, predict the error fraction before looking at the result, then compare. If the prediction lands within a factor of two, you have explained your result and can move on. If it does not, you have found something.

This converts "the hardware is noisy" from a shrug into an audit, and it is the same procedure you will use in Chapter 12 to answer the question that governs quantum debugging: is this noise, or is this a bug?

Along the way the audit contradicts something the chapter told you about which direction readout errors go, and predicts something surprising that the naive model misses entirely. Both are worth more than the arithmetic.

Skills applied: the hardware execution path (§2.6); the noise mechanisms (§2.6 Noise Report); reading two histograms (§2.7); little-endian conventions (§2.7).

Reproducibility. Every number below was produced with FakeSherbrooke — a noise model built from a snapshot of a real 127-qubit IBM processor — at 4,096 shots with seed_transpiler=42 and seed_simulator=1234. You can reproduce all of it locally in about two seconds, bit for bit, with no account, no queue, and no quota. Real hardware on a given day will differ; the method is what transfers, and Exercise 2.19 runs it against a live device.

The Result Under Audit

from qiskit import QuantumCircuit
from qiskit.transpiler.preset_passmanagers import generate_preset_pass_manager
from qiskit_ibm_runtime.fake_provider import FakeSherbrooke
from qiskit_ibm_runtime import SamplerV2 as Sampler

backend = FakeSherbrooke()
pm = generate_preset_pass_manager(optimization_level=1, backend=backend, seed_transpiler=42)

def run(circuit, shots=4096):
    sampler = Sampler(mode=backend)
    sampler.options.simulator.seed_simulator = 1234
    return sampler.run([pm.run(circuit)], shots=shots).result()[0].data.c.get_counts()

qc = QuantumCircuit(2, 2)
qc.h(0)
qc.cx(0, 1)
qc.measure([0, 1], [0, 1])

isa = pm.run(qc)
print(f"depth {qc.depth()} -> {isa.depth()},  ops {dict(isa.count_ops())}")
print(f"physical qubits: {isa.layout.final_index_layout()}")
print(dict(sorted(run(qc).items())))

depth 3 -> 8,  ops {'rz': 7, 'sx': 4, 'ecr': 1, 'measure': 2}
physical qubits: [0, 1]
{'00': 1927, '01': 91, '10': 89, '11': 1989}

Error fraction: 180 / 4096 = 4.39%.

The question: is 4.39% the right amount of wrong?

That sounds like a strange question and it is exactly the right one. "The hardware is noisy" predicts nothing and therefore explains nothing. A number you can predict and check is a measurement.

Note also the near-perfect symmetry between 01 (91) and 10 (89). Hold onto that; it turns out to be the most interesting number on the page.

Step 1: Read the Device

Every backend publishes its calibration. This is measured data, updated regularly on real devices, and it is the input to every serious decision you will make about a circuit.

target = backend.target
for q in (0, 1):
    p = target.qubit_properties[q]
    print(f"q{q}: T1={p.t1*1e6:7.1f} us  T2={p.t2*1e6:7.1f} us  "
          f"readout_err={target['measure'][(q,)].error:.4f}  "
          f"readout_dur={target['measure'][(q,)].duration*1e9:.0f} ns")

print(f"ecr(1,0): err={target['ecr'][(1,0)].error:.5f}  dur={target['ecr'][(1,0)].duration*1e9:.0f} ns")
print(f"sx(0):    err={target['sx'][(0,)].error:.6f}  dur={target['sx'][(0,)].duration*1e9:.1f} ns")

q0: T1=  381.6 us  T2=  131.7 us  readout_err=0.0112  readout_dur=1216 ns
q1: T1=  233.8 us  T2=  251.0 us  readout_err=0.0244  readout_dur=1216 ns
ecr(1,0): err=0.00749  dur=533 ns
sx(0):    err=0.000288  dur=56.9 ns

Five things matter here.

$T_1$ (energy relaxation, 234–382 μs). How long an excited qubit stays excited before decaying toward $|0\rangle$.

$T_2$ (dephasing, 132–251 μs). How long a superposition keeps its phase relationship. Bounded above by $2T_1$ and usually well below. Note that q0 has the better $T_1$ and the worse $T_2$ — these are independent quantities, and assuming one tracks the other is a common and costly error.

Readout error (1.12% and 2.44%). The probability that measuring a qubit reports the wrong value. Larger than every other error in this circuit by a factor of three or more, and about to determine the whole answer.

Two-qubit gate error (0.749%) on this specific pair. Other pairs on the same chip are several times worse; choosing among them is Chapter 29's subject.

Durations. ECR 533 ns, each sx 57 ns, and readout 1,216 ns — readout is by far the slowest operation, which surprises most people and dominates the circuit's total duration.

Note what is missing from the error data: rz. On superconducting hardware a $Z$-rotation is implemented virtually — as a bookkeeping change to the phase of subsequent pulses, taking zero time and introducing no error. Seven of the transpiled circuit's twelve operations are therefore free. This is an important practical fact and Chapter 28 exploits it hard.

Step 2: Build the Error Budget

Predict now, before looking back at the result. Take §2.6's four mechanisms in order.

Readout error — expected to dominate

An error fraction counts shots landing in 01 or 10. Starting from a correct 00 or 11, that requires exactly one of the two qubits to be misread:

$$P(\text{exactly one flip}) = p_0(1-p_1) + p_1(1-p_0)$$

$$= 0.0112 \times 0.9756 + 0.0244 \times 0.9888 = 0.01093 + 0.02413 = \mathbf{0.0351}$$

A double misread turns 00 into 11, still an allowed outcome and therefore invisible here. That term is $0.0112 \times 0.0244 = 0.00027$, or 0.03% — negligible. We will check it against data in Step 4 rather than take it on faith.

Readout contribution: 3.51%.

Two-qubit gate error

One ECR at 0.749%. Not every gate error produces a visible 01 or 10: some produce phase errors this measurement cannot see, some produce correlated errors that keep you inside the allowed set. Assume roughly half are visible.

Gate contribution: ~0.37%.

Single-qubit gate error

Four sx at 0.0288% each; the seven rz gates are virtual and free. $4 \times 0.000288 = 0.115\%$, perhaps half visible.

Single-qubit contribution: ~0.06%.

Decoherence

Circuit duration: four sx (228 ns) plus one ECR (533 ns) plus readout (1,216 ns) ≈ 2.0 μs, with readout dominating. Against the worse $T_2$ of 131.7 μs that is $t/T_2 \approx 1.5\%$ of a coherence time. Dephasing does not itself flip a bit in the measurement basis — it destroys the correlation, showing up partly as impossible outcomes and partly as reduced peaks. Assume about a third is visible.

Decoherence contribution: ~0.5%.

Crosstalk

Two active qubits on a 127-qubit chip with everything else idle. Negligible here, and not separately modeled by this backend.

The prediction

Measured: 4.39%.

Step 3: Reconcile

The prediction lands on the measurement, far inside the factor-of-two tolerance set in advance.

Resist being impressed. Three of the five terms involved a guess ("assume roughly half are visible"), and those guesses were made by someone who could see the answer. The honest reading is weaker and more useful: the budget correctly identifies readout error as 80% of the problem, and correctly predicts a few percent rather than a fraction of a percent or tens of percent. That is what an order-of-magnitude model is entitled to claim, and it is enough to separate "expected noise" from "something is wrong."

The signatures that would indicate a real problem:

That last row is where it gets interesting, because our result shows 01 and 10 almost exactly equal (91 and 89) even though the two qubits have readout errors differing by more than a factor of two. The averaged model has no opinion about that. Step 4 explains it, and the explanation is the best thing in this case study.

Step 4: Test the Dominant Term

The audit says readout error dominates. That is testable, and the test is a circuit with no gates at all.

def calibration(prep_ones):
    qc = QuantumCircuit(2, 2)
    for q in prep_ones:
        qc.x(q)
    qc.measure([0, 1], [0, 1])
    return dict(sorted(run(qc).items()))

print("|00> :", calibration([]))
print("|11> :", calibration([0, 1]))

|00> : {'00': 3900, '01': 65, '10': 130, '11': 1}
|11> : {'00': 1, '01': 89, '10': 25, '11': 3981}

Read these carefully. There is a great deal in two lines.

Preparing $|00\rangle$ and measuring it immediately is wrong 4.79% of the time (196 of 4,096). No gates, no entanglement, no meaningful decoherence. That is pure readout error, measured directly.

Preparing $|11\rangle$ is wrong 2.81% of the time (115 of 4,096).

The double-flip terms are visible and tiny: 11 appeared once when preparing $|00\rangle$, 00 once when preparing $|11\rangle$. Predicted rate 0.027%, or about 1.1 counts in 4,096. Observed: 1 and 1. The second-order term was correctly dismissed — and now we know rather than assume.

The four directional error rates

Because the bitstring is q1 q0, each wrong outcome names exactly which qubit flipped and in which direction:

Two things fall out immediately.

The error field in the calibration data is the average of the two directions. For q0: $(1.59 + 0.61)/2 = 1.10\%$, against the published 1.12%. For q1: $(3.17 + 2.17)/2 = 2.67\%$, against the published 2.44%. Both close. The single published number throws away the asymmetry, and the asymmetry is where the interesting behavior lives.

Both qubits are biased toward reading 1. $0 \to 1$ errors outnumber $1 \to 0$ errors by 2.6× on q0 and 1.5× on q1.

The finding that contradicts the chapter

The standard story — the one §2.6's Noise Report warns you about — says the opposite should hold: an excited qubit can decay to the ground state during the 1,216 ns measurement window, so $1 \to 0$ errors should dominate. That mechanism is real. On this device snapshot, on these two qubits, it is not the dominant one, and the bias runs the other way on both.

The likely reason is that readout is a signal-discrimination problem: the two states produce overlapping noisy signals and the threshold is calibrated per qubit. A threshold placed to minimize total error can easily leave a bias in either direction, and it drifts between calibrations.

The lesson is not about transmon physics. It is about method:

The direction of readout asymmetry is a property of your device, your qubits, and your day. It takes two circuits and two seconds to measure. Do not inherit it from a textbook — including this one.

The prediction the averaged model could not make

Now use the directional rates to predict the Bell result properly. A Bell state is an equal mixture of $|00\rangle$ and $|11\rangle$, so each starting state contributes half:

$$P(\texttt{01}) = \tfrac12 P(q_0\!: 0\!\to\!1) + \tfrac12 P(q_1\!: 1\!\to\!0) = \tfrac12(1.59\%) + \tfrac12(2.17\%) = 1.88\%$$

$$P(\texttt{10}) = \tfrac12 P(q_1\!: 0\!\to\!1) + \tfrac12 P(q_0\!: 1\!\to\!0) = \tfrac12(3.17\%) + \tfrac12(0.61\%) = 1.89\%$$

The model predicts that 01 and 10 should be almost exactly equal — and they are.

That is not obvious and the averaged model could not produce it. Each of the two qubits is strongly asymmetric, and the two qubits differ from each other by more than a factor of two. But the Bell state pairs q0's $0\!\to\!1$ rate with q1's $1\!\to\!0$ rate in one bucket, and q1's $0\!\to\!1$ with q0's $1\!\to\!0$ in the other — and on this device those two sums happen to land within 0.01 percentage points of each other. Four strongly unequal numbers, combined by the structure of the state, producing a symmetric result.

You now have a model that explains not just the magnitude of the error but its structure. That is the difference between "the hardware is noisy" and an understanding you can act on.

And you have, incidentally, done the first half of a readout error mitigation experiment. Those four directional rates are the assignment matrix; Chapter 13 inverts it and corrects your counts with it. Two circuits, no new concepts, a measurable improvement.

Analysis

Five things make this an audit rather than a shrug.

A prediction was made before the comparison. Predicting after seeing the answer is not prediction, and the entire value depends on the order.

A tolerance was stated in advance. "Within a factor of two counts as explained" is a criterion that can fail. Without one, every result confirms every model.

The dominant term was tested independently. Two circuits with no gates measure readout error directly, with no reasoning to be wrong about. When you are unsure of a term, isolate it experimentally rather than arguing about it.

A contradiction was followed rather than smoothed over. The data disagreed with the standard explanation of readout asymmetry, and the response was to report it and narrow the claim. In a field where every result is a distribution needing interpretation, letting the measurement win is the whole discipline.

The refined model made a new prediction, and it held. An error budget that only reproduces the number you already had is barely a model. One that predicts a structural feature you had not looked at — the equality of 01 and 10 — has earned some trust.

The transferable move is the third: almost every quantum debugging problem yields to what is the smallest circuit that isolates the thing I am unsure about? — and the answer is nearly always a circuit with fewer gates, not more.

Lessons

1. "The hardware is noisy" is not an explanation. A predicted error fraction is.
2. Readout error dominates shallow circuits — 80% of the budget here, three times any other term. It is also the cheapest error to mitigate.
3. The published per-qubit readout error is an average of two directions, and the asymmetry it hides is often larger than the number itself.
4. Do not assume the direction of readout asymmetry. Measure it. The textbook mechanism predicted the wrong direction on both qubits here.
5. rz is free on superconducting hardware — virtual, zero duration, zero error. Seven of these twelve operations cost nothing.
6. Readout is the slowest operation: 1,216 ns against 533 ns for the entangling gate.
7. $T_1$ and $T_2$ are independent. Here the qubit with the better $T_1$ has the worse $T_2$.
8. Two gate-free circuits give you the full assignment matrix — and that is half of readout mitigation.
9. Know your alarm thresholds. 40% is a bug; 0.1% means you ran an ideal simulator. Print backend.name in every recorded result.

Questions

1. Reproduce every number here with FakeSherbrooke, seed_transpiler=42, and seed_simulator=1234. Then change seed_transpiler to 7 and rerun. Did the transpiler choose different physical qubits? Did the error fraction change? What does that say about quoting a hardware result without recording the layout?

2. Run the two calibration circuits on a live backend. Are your device's readout asymmetries in the same direction as this snapshot's? Report all four directional rates.

3. Redo the directional prediction for a GHZ state on three qubits ($H$ then two CNOTs). Which outcomes are allowed? Write the analogue of the two $P(\texttt{01})$/$P(\texttt{10})$ formulas, then measure and compare.

4. The budget assumed "roughly half of gate errors produce a visible bit flip." Design an experiment that measures this fraction. What circuit isolates gate error from readout error? (Hint: what happens if you apply the same entangling gate many times?)

5. Preparing $|00\rangle$ gave exactly one 11 result in 4,096; the model predicted about 1.1. Compute the Poisson probability of observing 0, 1, or 2 when the mean is 1.1. Is a single count meaningful evidence for anything? How many shots would you need for it to be?

6. The refined model predicted 4.70% total against a measured 4.39% — it over-predicts, while the averaged model predicted 4.4% and landed. Explain why the more accurate readout model produced a worse total, and say which of the two models you would rather have. (This is a real methodological question, not a trick.)

7. Hardest. The budget adds four independent contributions. Identify a regime where independence fails badly. Consider coherent errors — small systematic over-rotations that accumulate in amplitude rather than in probability, so that $n$ of them produce an error growing like $n^2$ rather than $n$. At what circuit depth would that change your estimate by an order of magnitude, and what experiment would reveal it?