Case Study 1: The Energy That Was Confidently Wrong

"The dangerous bug is not the one that crashes. It is the one that returns a number with an error bar attached."

Executive Summary

A variational calculation converges cleanly. The optimizer descends, the energy stabilizes, the error bars are small, and successive runs agree to four decimal places. Everything a working result looks like.

The number is wrong by a factor of two, and the cause is one missing line: the observable was never mapped onto the physical qubits the circuit actually ran on. The Estimator dutifully measured two idle qubits at the other end of the chip, and reported the result with a confident uncertainty.

This case study reproduces the failure, examines why every symptom pointed away from the cause, and builds the two defenses that make it impossible to repeat. It is the practical heart of Chapter 7.

Skills applied: the Estimator primitive and layouts (§7.5, §7.6); transpilation output (Ch. 6 §6.5); noise-aware layout selection (Ch. 4 Case Study 1); reference values (§7.7).

Reproducibility. FakeSherbrooke, seed_transpiler=42, seed_simulator=1234, entirely local.

The Setup

The calculation is the simplest possible stand-in for a VQE energy: the expectation of $H = ZZ + XX$ on a Bell state. The exact value is 2.0 — Chapter 5 §5.5 established both terms are $+1$.

from qiskit import QuantumCircuit
from qiskit.quantum_info import SparsePauliOp
from qiskit.transpiler.preset_passmanagers import generate_preset_pass_manager
from qiskit_ibm_runtime import EstimatorV2
from qiskit_ibm_runtime.fake_provider import FakeSherbrooke

backend = FakeSherbrooke()
pm = generate_preset_pass_manager(optimization_level=3, backend=backend,
                                  seed_transpiler=42)

qc = QuantumCircuit(2)
qc.h(0)
qc.cx(0, 1)

H = SparsePauliOp.from_list([("ZZ", 1.0), ("XX", 1.0)])
isa = pm.run(qc)

The circuit transpiles onto physical qubits [60, 61] — optimization level 3 chose them by error rate, which is exactly the good practice Chapter 4's Case Study 1 argued for.

Now the bug, which is a line that is absent:

observable = H.apply_layout([0, 1], num_qubits=backend.num_qubits)   # <-- wrong qubits
# should be:  observable = H.apply_layout(isa.layout)

estimator = EstimatorV2(mode=backend)
estimator.options.simulator.seed_simulator = 1234
result = estimator.run([(isa, observable)], precision=0.01).result()[0]
print(f"E = {float(result.data.evs):+.4f} +/- {float(result.data.stds):.4f}")

E = +0.8838 +/- 0.0100

A number, with an error bar, to four decimal places. The true answer is 2.0.

Note how the mistake happens in practice. Nobody writes apply_layout([0, 1], ...) on purpose. What they write is code that constructs the padded observable once, early, before the layout is known — because the observable is a property of the problem and the layout is a property of the run, and it feels natural to build the problem first.

Why Every Symptom Pointed Away

This is the instructive part. Six things the team checked, and why each one was reassuring.

The optimizer converged. It descended smoothly and settled. But a variational optimizer minimizes whatever function you give it, and $\langle ZZ + XX\rangle$ on two idle qubits is a perfectly well-behaved function of the ansatz parameters — it just does not depend on them very much, which looks like fast convergence.

The error bars were small and shrank with precision. They are statistical error bars on a quantity that was measured correctly. The Estimator was not wrong about its own uncertainty; it was answering a different question accurately.

Repeated runs agreed. Determinism is what a correct result looks like. It is also what a consistently wrong result looks like, and the two are indistinguishable from repetition alone.

No exception, no warning. The observable had the right width — 127 qubits, matching the device — so nothing type-checked as wrong. Had it been left at 2 qubits, the mismatch would have raised immediately, and the bug would have been found in seconds. The padding that made the code "work" is what made the bug silent.

The simulator agreed with the hardware. Because the same wrong observable was used in both, of course. Comparing your hardware result to your simulator result is a good habit that is blind to any error present in both.

The answer was plausible. 0.88 is not zero, not negative, not infinite. Had the circuit run on qubits that happened to be in $|00\rangle$ with low noise, the result would have been near 1.0 for $\langle ZZ\rangle$ and near 0 for $\langle XX\rangle$ — landing close to 1, which is even more plausible than 0.88.

🐛 Debug This — What would have caught it, in order of cost.

Check	Cost	Would it catch this?
Repeat the run	free	No — the wrong answer is stable
Compare simulator to hardware	cheap	No — same wrong observable in both
Increase shots / tighten precision	expensive	No — measures the wrong thing precisely
Check the error bar	free	No — the statistical error is correct
Compare to an exact reference on the untranspiled circuit	free	YES
Print the layout and the observable's support	free	YES

The two checks that work are the two that step outside the pipeline that contains the bug. Everything else re-runs the same mistake more carefully.

That generalizes well beyond this chapter: a check that uses the same machinery as the thing it is checking cannot find an error in that machinery. Chapter 26 §26.3 makes this the basis of a debugging strategy.

The Reference Check

The cheapest diagnostic in the project, and it is three lines:

from qiskit.primitives import StatevectorEstimator

reference = float(StatevectorEstimator().run([(qc, H)]).result()[0].data.evs)
print(f"reference (exact, untranspiled): {reference:+.4f}")

reference (exact, untranspiled): +2.0000

Against a measured 0.8838, that is a ratio of 0.44 — far outside anything device noise can explain on a two-qubit circuit. Chapter 2's Case Study 2 established that a shallow two-qubit circuit loses a few percent to noise; losing 56% means something structural is wrong.

Correcting the layout:

  reference (exact)                     +2.0000
  measured, observable laid out         +1.8135      ratio 0.907  -- consistent with noise
  measured, observable on qubits 0,1    +0.8838      ratio 0.442  -- structurally wrong

A ratio near 1 means noise. A ratio near 0.5, or near 0, means look at the wiring.

Note what makes this check powerful: it is computed on the untranspiled circuit with a different simulator, so it shares no code path with the failure. It is genuinely independent evidence.

Its limitation is equally important: it requires exact simulation, so it stops working past about thirty qubits — exactly when you most need it. The mitigation is Chapter 26's: build the habit while the circuit is small, and carry the verified pipeline forward rather than the verified result.

The Two Defenses

1. Make the mistake unrepresentable

Do not document "remember to call apply_layout." Arrange for the circuit and observable to be inseparable:

@dataclass(frozen=True)
class Prepared:
    circuit: QuantumCircuit
    observable: SparsePauliOp
    layout: tuple

def prepare(circuit, observable, backend, optimization_level=3):
    pm = generate_preset_pass_manager(optimization_level=optimization_level,
                                      backend=backend, seed_transpiler=42)
    isa = pm.run(circuit)
    return Prepared(isa, observable.apply_layout(isa.layout),
                    tuple(isa.layout.final_index_layout()))

Now there is no code path that produces a transpiled circuit without a matching observable. The layout is applied in exactly one place, by code that has the layout in hand.

This is a general principle worth naming: when a rule can be violated silently, encode it in a type rather than in a comment. The comment is advice; the type is enforcement.

2. Check every result against a reference

def check_against_reference(measured, reference, tolerance=0.25):
    if abs(measured) < 0.1 * abs(reference):
        return False, ("measured value is far smaller than the reference -- "
                       "did you forget apply_layout()? You may be measuring idle qubits.")
    ratio = measured / reference
    if not (1 - tolerance) <= ratio <= (1 + tolerance):
        return False, f"ratio {ratio:.3f} -- larger than noise should explain"
    return True, f"ratio {ratio:.3f} -- consistent with noise"

The specific error message matters. "Value out of range" sends someone to look at their physics. "Did you forget apply_layout()?" sends them to the line that is actually wrong.

🔬 Honest Assessment — How often does this happen in practice?

Often enough that it is worth designing against.

The failure has three properties that make it disproportionately likely to reach a published result. It produces a number rather than a crash. That number is stable, so it survives repetition. And the natural code structure — build the problem, then set up the run — makes the observable get constructed before the layout exists.

The honest broader point is that quantum results are unusually hard to sanity-check because there is no independent source of truth for anything a quantum computer is good at. Here we had a two-qubit circuit and could compute the exact answer, so the reference check works. For a circuit large enough to be interesting, that check does not exist — which is precisely why the discipline has to be built at small scale and carried forward.

This is a real limitation of the field, not a shortcoming of one team's process. Chapter 27's testing strategy and Chapter 30's benchmarks are both, in part, attempts to build sanity checks that survive past the point where exact simulation stops.

Lessons

1. A wrong answer with an error bar is more dangerous than a crash. Design for it.
2. apply_layout every time, taking the layout from the transpiled circuit rather than writing one by hand.
3. The padding that makes the code run is what makes the bug silent. A width mismatch would have raised immediately.
4. A check that shares a code path with the bug cannot find it. Simulator-versus-hardware comparison is blind to any error present in both.
5. Compare every result to an exact reference on the untranspiled circuit. Free, independent, and decisive.
6. Ratio near 1 = noise. Ratio near 0.5 or near 0 = structural error. Learn the shapes.
7. Encode rules in types, not comments. Prepared makes the mistake unrepresentable.
8. Write error messages that name the likely cause, not just the symptom.
9. Build verification habits while circuits are small, because the exact reference disappears around thirty qubits and the habit has to already be there.

Questions

1. Reproduce the failure and the fix. Report all three numbers: reference, correct, and misplaced.

2. Change optimization_level to 1 and rerun. Does the bug still manifest? Explain why the optimization level determines whether this particular mistake is visible, and what that implies about testing on one configuration.

3. Construct a case where the misplaced observable gives a result closer to the truth than the correct one. What does that do to the reference check, and how would you defend against it?

4. The Prepared dataclass prevents the mistake. Find a legitimate reason a caller might want the circuit and observable separately, and design an API that supports it without reopening the trap.

5. check_against_reference uses a fixed 25% tolerance. Replace it with a statistically motivated test using the measured stds and an estimate of systematic error. Where would you get the systematic estimate? (Chapter 12 §12.7.)

6. The reference check stops working past ~30 qubits. Propose two checks that survive at 50 qubits. (Hint: consider circuits whose answers you know for structural reasons rather than by simulation, and consider checking invariants rather than values.)

7. Hardest. This bug was caught by comparing to an exact reference. Design a self-consistency check that needs no reference at all — one that would detect a misplaced observable using only measurements on the device. (Hint: what happens to the measured value if you deliberately change the ansatz parameters? What should happen?)