Case Study 1: The Port That Passed Every Test

The situation

A team has a working Qiskit implementation of a small algorithm — an oracle-based routine of the kind Chapter 19 will build properly. It works. It is tested. They need it in Cirq, because a collaborator's toolchain is Cirq and rewriting the collaborator's toolchain is not on the table.

The port is mechanical. Gates map one-to-one, the circuit is fifteen operations, and it takes an afternoon.

Then they verify it, carefully, the way anyone would.

The verification that passed

Test 1: a Bell state. The canonical smoke test.

  Qiskit:  {'00': 2009, '11': 2087}
  Cirq:    {0: 2043, 3: 2053}

Cirq's 0 is 00 and 3 is 11. Match.

Test 2: a three-qubit GHZ state. Same reasoning, one qubit wider.

  Qiskit:  {'000': ..., '111': ...}
  Cirq:    {0: ..., 7: ...}

Match.

Test 3: a uniform superposition. H on every qubit, all $2^n$ outcomes roughly equal.

Match.

Test 4: the unitary. They compare cirq.unitary(circuit) against Qiskit's Operator(circuit) for the gate-level building blocks and confirm they agree.

Match.

Four independent checks. Two frameworks. Everything agrees. The port ships.

The failure

The full algorithm returns the wrong answer. Not noise-wrong — specifically wrong, and reproducibly so. The marked state comes back as a different state, every time, deterministically.

By Chapter 12 §12.7's procedure this is easy to classify: it fails in noiseless simulation, so it is a bug, not noise. That much they get right immediately.

What follows is four hours in the oracle, because the oracle is where the "which state is marked" logic lives and the symptom is that the wrong state is marked. They check the phase kickback. They check the control conditions. They rebuild the diffuser. They compare the oracle's unitary against the Qiskit version — and it matches, which deepens the confusion rather than resolving it.

The actual bug

Cirq is big-endian. Qiskit is little-endian.

The oracle marks the state |101⟩. In Qiskit, |101⟩ means qubit 0 is 1, qubit 1 is 0, qubit 2 is 1. In Cirq, the same bitstring means qubit 0 is 1, qubit 1 is 0, qubit 2 is 1 — read from the other end. The port marked |101⟩ reversed, which is a different state whenever the target is not a palindrome.

$$\texttt{101 reversed} = \texttt{101}\quad\text{(palindrome — would have worked)}$$ $$\texttt{110 reversed} = \texttt{011}\quad\text{(not — this one broke)}$$

Their target was not a palindrome. It broke.

Why four tests missed it

This is the part worth internalizing, because the tests were not lazy. They were the right tests, chosen for the right reasons, and every one of them is blind to bit reversal.

The first three are symmetric under bit reversal in the most literal sense: applying the reversal to the histogram gives back the identical histogram. Demonstrated directly:

  correct convention: {0: 2043, 3: 2053}
  WRONG convention:   {0: 2043, 3: 2053}
  identical? True

Getting the convention exactly backwards produces the same output. The test passes either way, so passing it carries no information.

The fourth test is subtler and more interesting. cirq.unitary() builds the matrix using Cirq's qubit ordering; Operator() builds it using Qiskit's. Comparing them compares each framework's own self-consistent description, and both are internally correct. The disagreement is not inside either framework — it is at the boundary, and a test conducted entirely within one side of a boundary cannot see it.

The test that takes ten seconds

# In BOTH frameworks: excite qubit 0 only. Where does the amplitude land?
cirq:    cirq.Circuit([cirq.X(q[0]), cirq.I(q[1])])   ->  index 2  (binary 10)
qiskit:  qc = QuantumCircuit(2); qc.x(0)              ->  index 1  (binary 01)

One gate. And unlike the four tests above, it can fail:

  correct convention: {2: 100}
  WRONG convention:   {1: 100}
  identical? False

2 is 10; reversed it is 01 = 1. The wrong convention reports the wrong qubit as excited, and the test catches it.

The rule generalizes past endianness: a test whose expected output is invariant under the bug you are worried about is not a test for that bug.

The fix

One conversion function, at one boundary:

def reverse_bits(value: int, n_qubits: int) -> int:
    return int(format(value, f"0{n_qubits}b")[::-1], 2)

And nothing else in the codebase permitted to reverse bit order.

That constraint is not fussiness. Bit reversal is its own inverse, so two reversals cancel. A codebase with reversals scattered across four modules has a bug whose presence depends on how many of those modules a given code path happens to traverse — which means it appears in some circuits and not others, survives some refactors and not others, and resists every attempt to characterize it.

vqelab/translate.py (§14.5's checkpoint) enforces exactly this, and one of its tests asserts the weakness of the Bell-state check so that nobody later removes the asymmetric tests as redundant.

The lessons

A test that cannot fail is not a test. Four verifications passed, and the port was broken the entire time. Before trusting a test, ask what result would indicate the bug — and if the answer is "the same result," the test is a ritual.

Symmetric test cases hide asymmetric bugs. Bell states, GHZ states, and uniform superpositions are the standard smoke tests precisely because they are simple and symmetric — and their symmetry is exactly what makes them useless here. Reach for an asymmetric case deliberately.

Conventions are invisible until you have seen two. Nobody on that team was careless. Little-endian ordering had been a background fact for as long as they had used Qiskit, and background facts are not things you think to check. This is §14.1's argument for learning a second framework, arriving as a bug report.

Test at the boundary, not inside it. The unitary comparison failed to help because each framework was internally consistent. Cross-framework bugs live at the interface, and detecting them requires a test that spans it — which is what assert_same_state does.

And Chapter 12 §12.7's step 2 did its job. "Does it fail in noiseless simulation?" identified this as a bug within seconds and correctly ruled out every hardware explanation. The procedure worked; the four hours went into the wrong part of the code afterward, because the shape of the symptom pointed at the oracle. Knowing it is a bug does not tell you where the bug is — but it does stop you investigating physics, which is most of the value.

Reproduce it: code/example-03-endianness.py runs every test in this case study, including the demonstration that reversing a Bell histogram changes nothing.