Chapter 5: Measurement, Shots, and Statistics

"The output of a quantum program is a histogram. Everything you will ever conclude, you will conclude from a histogram, and the discipline of doing that honestly is most of the job."

Overview

Every chapter so far has ended in a measure call whose behavior you accepted on faith. This chapter pays that debt.

Measurement is not a passive readout. It is an operation — the only irreversible one in the whole model — and it has a parameter you have not yet been told about: the basis. Change the basis and the same state gives a completely different distribution. Chapter 4 promised that a second basis would separate genuine entanglement from classical correlation, and §5.6 is where that promise is made mechanical.

The other half of the chapter is statistics, and it is the part that separates people who can report a quantum result from people who cannot. A quantum program returns samples. Samples have error. That error shrinks like $1/\sqrt{N}$ and no faster, which means ten times the precision costs a hundred times the runs — the single fact that governs the economics of near-term quantum computing. By the end of §5.9 you will be able to answer "how many shots do I need?" with a number and a justification instead of a shrug.

And §5.8 gives you the tool for the question that recurs constantly and that beginners answer by eyeballing: is this distribution the one I expected?

In this chapter, you will learn to:

• Say what measurement does, and why the basis is part of it.
• Read bitstrings correctly, including with named and multiple classical registers.
• Predict sampling error from shot count, and pick a shot count for a target precision.
• Convert between counts, probabilities, and expectation values, and say why the Estimator primitive exists.
• Measure in the X and Y bases, and compute any Pauli expectation value from counts.
• Compute marginals, and explain what partial measurement does to the rest of the register.
• Apply a chi-squared test and state honestly what it does and does not establish.

Learning Paths

How to read this chapter by track. - 🔰 Beginner — §5.3 (bitstrings), §5.4 (sampling error), and §5.6 (bases) are essential. §5.8 can wait until you have a result you need to defend. - 🔬 Researcher — the whole chapter, and §5.8 twice. Reporting a quantum result without an error bar is the most common defect in this literature. - 🤖 Quantum ML — §5.5 is your chapter. Every loss function you will ever optimize is an expectation value, and §5.9's shot-budget arithmetic determines whether your training loop finishes this week. - 🏗️ Quantum Engineer — §5.4 and §5.9. The shot budget is a cost centre and you will be asked to justify it. - 🔐 Security — §5.6 and §5.8. Basis choice is the heart of BB84 (Chapter 38), and eavesdropper detection is a statistical test on measured error rates.

5.1 What Measurement Does

Every operation you have met so far is unitary: reversible, deterministic, information preserving. Measurement is none of those things.

Measuring a qubit in state $\alpha|0\rangle + \beta|1\rangle$ does two things at once:

1. It returns a classical bit — 0 with probability $|\alpha|^2$, 1 with probability $|\beta|^2$.
2. It replaces the qubit's state with $|0\rangle$ or $|1\rangle$, matching the outcome.

The second part is the one people forget. Measurement does not read the state; it changes it, and the change is irreversible. The information in the amplitudes — everything except which outcome occurred — is gone.

from qiskit import QuantumCircuit
from qiskit.quantum_info import Statevector

qc = QuantumCircuit(1)
qc.h(0)
print("before:", Statevector(qc).data.round(4))
# after measuring, the state is |0> or |1> -- never the superposition again

This is why print(qubit) is impossible (Chapter 1 §1.4), why there is no checkpointing (no-cloning), and why quantum debugging needs the indirect techniques of Chapter 26.

The parameter you have not been told about

Here is the part that reframes everything: measurement always happens in a basis, and Qiskit's measure uses the computational basis — $\{|0\rangle, |1\rangle\}$ — by default.

That default is a choice, not a law. You could measure in the $\{|+\rangle, |-\rangle\}$ basis instead, and you would get different answers from the same state. A state that is a 50/50 coin flip in one basis can be perfectly deterministic in another.

$|+\rangle$ measured in the computational basis: 50/50 random. $|+\rangle$ measured in the $X$ basis: 0 with certainty.

Same state. Different question. Different answer.

⚛️ The Physics Underneath — Measurement as a question, not a reading.

A measurement is specified by a set of orthogonal states — a basis — and it answers: which of these is the system in? The answer is random with probabilities given by the squared overlaps, and afterwards the system genuinely is in the one that came back.

"What is this qubit's state?" is not a question quantum mechanics answers. "Is this qubit $|0\rangle$ or $|1\rangle$?" is. So is "is it $|+\rangle$ or $|-\rangle$?" They are different questions with different answers, and you must choose one — you cannot ask both of the same qubit, because the first destroys the information the second needs.

That is the operational content of complementarity, and §5.6 turns it into two lines of code.

From the Born rule to a histogram

The rule that fixes those probabilities has a name — the Born rule — and it is worth writing in the form that matters for programming rather than the form that matters for physics:

$$P(x) \;=\; \big|\langle x|\psi\rangle\big|^2$$

Three consequences follow directly, and together they are why the second half of this chapter is statistics rather than physics.

One: the theory gives you probabilities, never outcomes. There is no formula anywhere in quantum mechanics that returns the bit you are about to get. A single shot is a draw from $P$, and $|\psi\rangle$ is not recoverable from it. This is not an engineering limitation waiting for better hardware — it is the content of the theory, and it is the reason a quantum program's output type is a histogram.

Two: $N$ shots is one draw from a multinomial distribution, not $N$ facts. The counts dictionary you get back is a single random object. Everything you will ever report from it — a fidelity, an energy, a success rate, a p-value — is a statistic computed from that one draw, and therefore has a distribution of its own. Run the same circuit again with a different seed and every number moves. §5.4 is about how far.

Three: the square is why global phase is unobservable. Multiply the whole state by $e^{i\phi}$ and every amplitude picks up that factor, but

$$\big|e^{i\phi}\langle x|\psi\rangle\big|^2 = \big|e^{i\phi}\big|^2\,\big|\langle x|\psi\rangle\big|^2 = \big|\langle x|\psi\rangle\big|^2$$

since $|e^{i\phi}| = 1$. The probabilities are identical in every basis, because the argument used nothing about $|x\rangle$. Chapter 3 §3.7 asserted that global phase is undetectable; that is the proof, and it is two lines. Relative phase survives, because rotating one amplitude and not another changes the overlaps — which is exactly what the basis rotations of §5.6 are built to expose.

A fourth consequence is quieter and shows up in §5.5's error bars. Amplitudes are what interfere; probabilities are what you can count; and the map between them is a square. Perturb an amplitude to $\alpha(1+\varepsilon)$ and the probability moves to $p(1+\varepsilon)^2 \approx p(1 + 2\varepsilon)$. A 1% error in an amplitude is a 2% error in a probability — so the shot budget you need is set by twice the amplitude-level defect you are trying to see, not by the defect itself.

📊 What the Numbers Say — A histogram is a sample, not the state.

When Qiskit hands you {'00': 2074, '11': 2022}, the temptation is to read it as a description of the state: "the state is 50.6% 00." It is not. It is one draw from a distribution whose true parameters you do not know, and the honest reading is:

text observed p(00) = 0.5063 standard error sqrt(0.5063 x 0.4937 / 4096) = 0.0078 95% interval 0.4910 to 0.5217

The interval contains 0.5. There is no evidence here of any departure from a fair split, and a write-up that says "we measured 50.6%" without the interval has reported a fact about one pseudorandom seed rather than a fact about the circuit.

Every device in this chapter's toolkit exists to turn the first reading into the second.

5.2 Classical Registers and measure

The mechanics, quickly, because there are three ways to get this wrong.

qc = QuantumCircuit(2, 2)
qc.h(0)
qc.cx(0, 1)
qc.measure([0, 1], [0, 1])       # qubit 0 -> clbit 0, qubit 1 -> clbit 1

measure(qubits, clbits) takes two parallel lists. measure_all() is the shortcut that adds a barrier, creates a register named meas, and measures everything:

qc.measure_all()                 # register is named 'meas', not 'c'

That register name matters, because it is how you address results from the primitives:

result[0].data.c.get_counts()       # QuantumCircuit(2, 2)
result[0].data.meas.get_counts()    # after measure_all()
result[0].data.keys()               # when you are not sure

⚠️ Common Pitfall — Three ways measure goes wrong.

Forgetting it entirely. QiskitError: 'No counts for experiment'. The circuit ran and threw away its state. Chapter 2 §2.8.

Measuring mid-circuit by accident. Any measure collapses the state at that point, and everything after it operates on the collapsed state. If you insert a measurement for debugging and leave it in, your algorithm is now a different algorithm. Deliberate mid-circuit measurement is genuinely useful and gets Chapter 9; accidental mid-circuit measurement is a silent, serious bug.

Mismatched lists. measure([0, 1], [1, 0]) is legal and swaps the bits in your output. Everything downstream is then bit-reversed, which — because it is legal — produces no error at all, just wrong answers.

5.3 Reading a Bitstring

Stated once more, because it is the book's running trap:

Qiskit is little-endian. The rightmost character of a bitstring is qubit 0.

   '1 0 1 1'
    │ │ │ └── qubit 0
    │ │ └──── qubit 1
    │ └────── qubit 2
    └──────── qubit 3

qc = QuantumCircuit(3, 3)
qc.x(0)                          # set ONLY qubit 0
qc.measure([0, 1, 2], [0, 1, 2])
# -> {'001': 1024}   the 1 is on the RIGHT

With multiple registers, it gets worse

Qiskit separates registers with a space, and the registers are also ordered right to left:

from qiskit import QuantumRegister, ClassicalRegister

q = QuantumRegister(3, "q")
a = ClassicalRegister(1, "a")
b = ClassicalRegister(2, "b")
qc = QuantumCircuit(q, a, b)
qc.x(0)
qc.measure(q[0], a[0])
qc.measure([q[1], q[2]], [b[0], b[1]])
# counts look like {'00 1': 1024}
#                   └┬┘ │
#                    b  a

The last-declared register appears leftmost. This is consistent with the qubit rule — later things go to the left — and it surprises everyone the first time.

🐛 Debug This — The bit-reversal that produces no error.

python qc = QuantumCircuit(3, 3) qc.h(0) qc.cx(0, 1) qc.cx(1, 2) qc.measure([0, 1, 2], [2, 1, 0]) # <-- reversed clbit list

Symptom: for a GHZ state, nothing. '000' and '111' are palindromes, so the reversal is invisible and every test passes.

Then you change the circuit to something asymmetric — say you add qc.x(2) — and the answers are subtly, consistently wrong, in a way that looks like a physics problem rather than a typo.

The fix: always measure(qubits, clbits) with matching order, or just use measure_all().

The lesson, which generalizes: test with asymmetric states. A symmetric test case cannot detect an ordering bug, and ordering bugs are the most common kind in this field. If your test suite only contains Bell and GHZ states, it cannot catch what Chapter 26 §26.5 is about.

5.4 Shots and Sampling Error

Now the statistics.

You run a circuit $N$ times and count outcomes. The fraction you observe is an estimate of the true probability, and estimates have error.

For a probability $p$ estimated from $N$ shots, the standard error is

$$\sigma = \sqrt{\frac{p(1-p)}{N}}$$

which is at most $\dfrac{1}{2\sqrt{N}}$, attained at $p = 0.5$.

Every additional digit of precision costs a hundred times more shots. That is the fact.

Measured, over 40 independent repetitions at each shot count:

   shots |  mean |p-0.5| |   std of p |  1/(2 sqrt N)
      10 |       0.13750 |    0.17732 |       0.15811
     100 |       0.04875 |    0.05877 |       0.05000
    1000 |       0.01285 |    0.01677 |       0.01581
   10000 |       0.00371 |    0.00477 |       0.00500
  100000 |       0.00111 |    0.00138 |       0.00158

The measured standard deviation tracks $1/(2\sqrt N)$ closely at every scale. A hundredfold increase in shots buys a tenfold reduction in error, and nothing you can do about the circuit changes that.

The rule of four

The hundred-for-ten version of the law is the one to quote in a paper. The one to keep in your head while you are working is smaller and more useful:

$$\sigma \propto \frac{1}{\sqrt N} \quad\Longrightarrow\quad \frac{\sigma(kN)}{\sigma(N)} = \frac{1}{\sqrt k}$$

so $k = 4$ gives exactly $\tfrac12$. Four times the shots halves the error. Always, exactly, at every scale, for any $p$. The $\sqrt{p(1-p)}$ factor cancels out of the ratio, so this holds whether you are estimating a fidelity near 0.99 or a coin flip.

   shots x        error x
        1          1.000
        2          0.707      a doubling buys 29%, not 50%
        4          0.500      <- the rule
       16          0.250
      100          0.100
   10,000          0.010

Read the second row, because it is the one that catches people out. Doubling your shots does not halve your error bar; it shrinks it by 29%. If a reviewer asks you to "run it twice as long," the honest answer is that they have asked for a 1.41× improvement, and if they wanted 2× they should have asked for 4× the shots.

The rule also runs backwards, which is where the money is. You have 40,000 shots and an error bar of ±0.005. Someone asks whether ±0.004 is achievable. That is a 1.25× improvement, costing $1.25^2 = 1.5625$, so 62,500 shots — a 56% increase in cost for a 20% tighter bar. Whether that is worth buying is a judgement, but it is a judgement you can now make in ten seconds and in advance.

📐 Math Aside — Where $1/\sqrt{N}$ comes from, and why it is a hard floor.

Each shot is a Bernoulli trial. Summing $N$ of them gives a binomial with variance $Np(1-p)$; the fraction therefore has variance $p(1-p)/N$ and standard deviation $\sqrt{p(1-p)/N}$.

Inverting for a target precision $\epsilon$ at 95% confidence ($z = 1.96$):

$$N \;=\; \frac{z^2\, p(1-p)}{\epsilon^2} \;\le\; \frac{1.96^2 \times 0.25}{\epsilon^2} > \;\approx\; \frac{0.96}{\epsilon^2}$$

text eps=0.1 -> 97 shots eps=0.05 -> 385 shots eps=0.01 -> 9,604 shots eps=0.005 -> 38,416 shots eps=0.001 -> 960,400 shots

This is not a limitation of current hardware. It is the statistics of sampling, and a perfect fault-tolerant quantum computer would face exactly the same scaling. Techniques exist that beat it in special cases — amplitude estimation achieves $1/N$ rather than $1/\sqrt N$, a quadratic improvement, at the cost of deep coherent circuits — but for anything you can run today, $1/\sqrt{N}$ is the law.

It is also why chemistry is hard: the millihartree precision Chapter 36 needs, on an energy of order 1 Hartree, is a relative precision of $10^{-3}$, which is around $10^6$ shots per expectation value, and VQE needs many of them per optimizer iteration.

📐 Math Aside — The counts are a multinomial, and the correlations are in your favour.

§5.4's formula treats one outcome at a time. The full object is a multinomial: $N$ shots distributed over the outcomes with probabilities $p_i$ summing to 1. That gives the same per-outcome variance, plus a piece the single-outcome formula hides:

$$\operatorname{Var}(\hat p_i) = \frac{p_i(1-p_i)}{N}, > \qquad \operatorname{Cov}(\hat p_i, \hat p_j) = -\frac{p_i p_j}{N} \quad (i \neq j)$$

The covariance is negative, and it has to be: the fractions sum to exactly 1 in every run, so if one goes up another must come down. That constraint is not noise, it is arithmetic, and ignoring it makes you over-estimate the error on anything built from more than one outcome.

The case that matters is a fidelity — the fraction of shots landing on an accepted set of outcomes, like 000 or 111 for GHZ(3). Writing $F = \hat p_{000} + \hat p_{111}$:

$$\operatorname{Var}(F) = \frac{p_{000}(1-p_{000})}{N} + \frac{p_{111}(1-p_{111})}{N} > - \frac{2\,p_{000}\,p_{111}}{N}$$

Work it for a noiseless GHZ(3) at 4,096 shots, $p_{000} = p_{111} = 0.5$:

text naive (add the two variances) sd = 0.011049 with the covariance term sd = 0.000000 <- exactly zero

Which is obviously right, and obviously right only once you look: a noiseless GHZ circuit produces 000 or 111 on every single shot, so $F = 1$ in every run with no error at all. The naive calculation would have put an error bar of ±0.011 on a quantity that cannot vary. Repeat it for a realistic 94% fidelity, $p_{000} = p_{111} = 0.47$:

text naive sd = 0.011029 with the covariance term sd = 0.003711 <- 3.0x smaller direct: sqrt(F(1-F)/N), F=0.94 sd = 0.003711 <- identical

★ The shortcut, and the thing to actually remember: pool first, then apply the formula. A sum of outcomes is a single Bernoulli event — "did this shot land in the accepted set?" — so compute $F$ first and use $\sqrt{F(1-F)/N}$ on the pooled value. The two routes agree to the last digit, because they are the same algebra. Adding variances term by term is the route that is wrong, and it is wrong in the direction that makes your experiment look worse than it is.

Case Study 2's fidelity error bars, and Chapter 30's benchmarking, all use the pooled form.

💰 Cost and Queue — What the shot budget actually costs.

At roughly 100 μs of QPU time per shot:

Precision	Shots	QPU time
±0.05	385	0.04 s
±0.01	9,604	1 s
±0.005	38,416	4 s
±0.001	960,400	96 s

One second of QPU time for two decimal places. A minute and a half for three. And a VQE optimization needs hundreds of such evaluations.

Choose your precision deliberately. The most common beginner mistake is running 100 shots and quoting three decimal places; the second most common is running a million shots when 4,096 would have answered the question.

Confidence intervals, and where the normal one breaks

$\hat p \pm 1.96\,\sigma$ is the interval everyone writes, and it has a name — the Wald interval — and a failure mode that is not a rounding issue. It is built on two approximations: that the binomial is close enough to a normal, and that $\hat p$ is close enough to $p$ to use inside $\sqrt{p(1-p)/N}$. Both hold beautifully at $p = 0.5$ and both collapse when $p$ is small or $N$ is small.

Here is the collapse in its purest form. You run 100 shots of a circuit that should never produce 11 and you observe zero of them:

$$\hat p = 0, \qquad \sigma = \sqrt{\tfrac{0 \times 1}{100}} = 0, \qquad \text{interval} = [0,\, 0]$$

The interval has zero width. The formula has just told you that you know the error rate exactly, to infinite precision, from 100 shots. That is not conservative or approximate — it is a claim no experiment could ever license, produced by a formula that had no way to say "I do not know."

Two standard repairs. The Wilson score interval inverts the test rather than substituting $\hat p$ into the variance:

$$\frac{\hat p + \frac{z^2}{2N} \;\pm\; z\sqrt{\dfrac{\hat p(1-\hat p)}{N} + \dfrac{z^2}{4N^2}}} {1 + \frac{z^2}{N}}$$

The $z^2/2N$ in the numerator pulls the centre away from the boundary, and the $z^2/4N^2$ under the root keeps the width positive even at $\hat p = 0$. The Clopper–Pearson interval goes further and inverts the exact binomial rather than a normal approximation, giving guaranteed-conservative coverage at the cost of being wider than it needs to be.

Four cases, all at 95%:

   observed     Wald                 Wilson               Clopper-Pearson
     0/100      0.00000 - 0.00000    0.00000 - 0.03699    0.00000 - 0.03622
     1/100     -0.00950 - 0.02950    0.00177 - 0.05449    0.00025 - 0.05446
     3/2000    -0.00020 - 0.00320    0.00051 - 0.00440    0.00031 - 0.00438
    50/100      0.40200 - 0.59800    0.40383 - 0.59617    0.39832 - 0.60168

Read the columns downwards. At $50/100$ all three agree to two decimal places — at $p$ near 0.5 the Wald interval is fine and the extra machinery buys nothing. At $1/100$ and $3/2000$ the Wald interval extends below zero, which is a probability it is asserting could be negative. At $0/100$ it has vanished.

Note the third row: $3/2000$ is Chapter 27 §27.5's measured false-failure rate for a distribution test. The honest report of it is 0.150%, 95% interval 0.051% to 0.440% — an interval spanning almost an order of magnitude, which is precisely why that section's earlier $2/200$ estimate of 1.0% needed correcting rather than quoting.

Does the repair actually matter, or is it pedantry? Compute the true coverage — enumerate every possible outcome $k = 0 \ldots N$, weight by its binomial probability, and add up the weight of the outcomes whose interval contains the true $p$. This is exact arithmetic, not a simulation:

   true p     N     Wald coverage   Wilson coverage    (nominal 0.95)
     0.50   100            0.9431            0.9431
     0.05   100            0.8775            0.9659
     0.02   100            0.8664            0.9492
     0.01   100            0.6334            0.9206
     0.05    20            0.6389            0.9245
     0.50    20            0.9586            0.9586

★ A "95% confidence interval" on a true probability of 0.01 from 100 shots contains the truth 63% of the time. It is not 95% confident and it is not 90% confident; it is wrong about one time in three, while being labelled 95%. Wilson holds 0.92 in the same place. And at $p = 0.5$ the two are indistinguishable, which is why this defect survives so long unnoticed — everyone's first experiment is a fair coin.

The practical rule: if $N\hat p < 10$ or $N(1-\hat p) < 10$, do not use $\hat p \pm 1.96\sigma$. This is not a corner case in quantum computing — it is every error rate you will ever quote. A two-qubit gate error of $1.79\times10^{-3}$ (Chapter 39's best measured link) needs about 5,600 shots before the normal approximation is even admissible.

⚠️ Common Pitfall — Zero events does not mean zero rate.

The single most common version of this error: "we saw no failures in 100 runs, so the failure rate is 0%."

The rule of three gives the answer in one step: observing 0 events in $N$ trials puts a 95% upper bound of roughly $3/N$ on the rate. For $N = 100$ that is 3%, and the exact Clopper–Pearson bound is 3.62% — so $3/N$ is a good, slightly optimistic approximation.

text 0 in 20 runs -> 3/N = 15.00% exact upper bound 16.84% 0 in 100 runs -> 3/N = 3.00% exact upper bound 3.62% 0 in 2,000 runs -> 3/N = 0.15% exact upper bound 0.18%

Chapter 27 §27.5 uses exactly this to report its 0-failure rows, and the reason it matters there is the same reason it matters everywhere: a zero count is the weakest possible evidence with the most confident-sounding summary. "We observed no errors" is a description of an experiment. "The error rate is zero" is a claim about a device, and 100 shots cannot support it.

The same arithmetic run forwards tells you how big an experiment you need: to bound a rate below 0.1% you need roughly $3/0.001 = 3{,}000$ clean runs, and no smaller experiment will do it no matter how clean it comes out.

5.5 Counts, Probabilities, and Expectation Values

Three ways to summarize the same data, and choosing the right one is worth real money.

Counts → probabilities

counts = {"00": 2074, "11": 2022}
total = sum(counts.values())
probs = {k: v / total for k, v in counts.items()}

Probabilities → expectation values

An expectation value is a weighted average of an observable over the distribution. For a single-qubit $Z$ measurement, assign $+1$ to outcome 0 and $-1$ to outcome 1:

$$\langle Z\rangle = P(0) - P(1)$$

def expval_z(counts, qubit=0):
    """<Z> on one qubit, from counts. Remember: bitstring[-1-qubit] is qubit `qubit`."""
    total = sum(counts.values())
    return sum(v * (1 if k[-1 - qubit] == "0" else -1) for k, v in counts.items()) / total

For a multi-qubit Pauli string like $Z \otimes Z$, the sign is $+1$ when the relevant bits are equal and $-1$ when they differ:

def expval_zz(counts):
    total = sum(counts.values())
    return sum(v * (1 if k[0] == k[1] else -1) for k, v in counts.items()) / total

Why this matters: one number instead of $2^n$

For $n$ qubits, the full distribution has $2^n$ entries, and estimating all of them to precision $\epsilon$ requires shots growing with $2^n$. An expectation value is a single real number, and estimating it to precision $\epsilon$ takes $O(1/\epsilon^2)$ shots regardless of $n$.

That is the entire reason the Estimator primitive exists alongside Sampler. Asking the right question is worth an exponential factor.

from qiskit.quantum_info import SparsePauliOp
from qiskit.primitives import StatevectorEstimator

qc = QuantumCircuit(2)
qc.h(0)
qc.cx(0, 1)                                     # Bell state

est = StatevectorEstimator()
for obs in ("ZZ", "XX", "YY", "ZI", "IZ", "XI"):
    v = est.run([(qc, SparsePauliOp(obs))]).result()[0].data.evs
    print(f"  <{obs}> = {float(v):+.4f}")

  <ZZ> = +1.0000
  <XX> = +1.0000
  <YY> = -1.0000
  <ZI> = +0.0000
  <IZ> = +0.0000
  <XI> = +0.0000

Read that output carefully — it is Chapter 4 §4.4 in numbers.

The three single-qubit observables ($ZI$, $IZ$, $XI$) are all exactly zero: each qubit on its own carries no information whatsoever. But the two-qubit correlators $\langle ZZ\rangle$ and $\langle XX\rangle$ are $\pm 1$: the pair is perfectly determined.

All the information is in the correlations and none is in the parts, and now you have measured it rather than been told it.

The error bar on an expectation value

An expectation value estimated from shots needs an uncertainty exactly as much as a probability does, and the formula is different enough to be worth deriving rather than looking up.

📐 Math Aside — Why $\sigma = \sqrt{(1 - \langle P\rangle^2)/N}$, and why it is not $1/(2\sqrt N)$.

A Pauli string's measurement outcome per shot is a single random variable $s$ taking values $+1$ and $-1$ — never anything else, because the parity of a bitstring is $\pm1$ by construction. So $s^2 = 1$ identically, and therefore $\mathbb{E}[s^2] = 1$ regardless of the state. The variance of a single shot is then

$$\operatorname{Var}(s) = \mathbb{E}[s^2] - \mathbb{E}[s]^2 = 1 - \langle P\rangle^2$$

and the mean of $N$ independent shots divides that by $N$:

$$\sigma_{\langle P\rangle} = \sqrt{\frac{1 - \langle P\rangle^2}{N}}$$

That is the expval_stderr in this chapter's project checkpoint, and now it is derived rather than asserted. The max(0.0, ...) in that function guards the case where sampling noise pushes an estimate slightly past $\pm1$ — which happens, and would otherwise take the square root of a negative number.

Two sanity checks. At $\langle P\rangle = 0$ the shot is a fair $\pm1$ coin and $\sigma = 1/\sqrt N$ — twice the $1/(2\sqrt N)$ of a probability, because a $\pm1$ variable has twice the range of a $0/1$ one. At $\langle P\rangle = \pm1$ the variance is exactly zero: every shot returns the same value, so there is nothing to average.

The consequence is the one that decides how to spend a VQE shot budget. At 4,096 shots:

   <P>      sigma = sqrt((1 - <P>^2)/N)
   0.00                       0.015625
   0.20                       0.015309
   0.50                       0.013532
   0.80                       0.009375
   0.95                       0.004879
   1.00                       0.000000

★ An observable near $\pm1$ has a small error bar for free; an observable near zero has the largest one there is. The ratio between the extremes at fixed $N$ is $1/\sqrt{1 - 0.95^2} = 3.20$, and in shots — which is what you pay in — the gap is a factor of $1 - \langle P\rangle^2$:

   shots to reach sigma = 0.01 at <P> = 0.00     10,000
   shots to reach sigma = 0.01 at <P> = 0.95        976     <- 10.2x cheaper

So a Hamiltonian's terms are not equally expensive to measure to a given precision, and a uniform shot allocation across them is leaving a factor of ten on the table for the terms near $\pm1$. Chapter 36 §36.7 develops the grown-up version of this, where terms are also weighted by their coefficients $c_k$ and the errors combine in quadrature:

$$\sigma_E = \sqrt{\sum_k c_k^2\,\sigma_k^2}$$

Case Study 1 works that arithmetic end to end for H₂'s 15 terms and lands on the allocation rule $N_k \propto |c_k|$, which is the right answer when every term has about the same variance. The full rule drops out of minimizing $\sigma_E$ under a fixed total $\sum_k N_k = N$ by Lagrange multipliers:

$$N_k \;\propto\; |c_k|\,\sqrt{1 - \langle P_k\rangle^2}$$

— the coefficient times the term's own noisiness. It reduces to Case Study 1's rule exactly when the square roots are equal, and it differs most for the terms sitting near $\pm1$, which the case study's version over-funds. The number that is easy to get is "shots divided by terms." The number that answers the question is not that.

⚠️ Common Pitfall — $\langle YY\rangle = -1$, and why the sign is not a typo.

Everyone expects all three correlators to be $+1$ for $|\Phi^+\rangle$. Two are; $\langle YY\rangle$ is $-1$.

The reason is the factor of $i$ in the $Y$ operator. Applying $Y \otimes Y$ to $|00\rangle + |11\rangle$ gives $-(|00\rangle + |11\rangle)$ — the two factors of $i$ multiply to $-1$. So $\langle YY\rangle = -1$ exactly.

This bites in practice: the stabilizers of $|\Phi^+\rangle$ are $XX$ and $ZZ$ (both $+1$), and $YY = -XX \cdot ZZ$ follows. If you build an entanglement witness assuming all three are $+1$, it will be wrong by a sign and your "witness" will fail on a perfect Bell state. Chapter 4's Case Study 2 used $\langle ZZ\rangle + \langle XX\rangle$ for exactly this reason.

5.6 Measuring in Another Basis

Hardware measures in the computational basis. Full stop — there is no measure_in_x_basis instruction on any device.

So you rotate the state instead. To measure in some basis, apply the gate that maps that basis onto the computational one, then measure normally.

def measure_in(qc, basis, qubit=0):
    if basis == "X":
        qc.h(qubit)
    elif basis == "Y":
        qc.sdg(qubit)
        qc.h(qubit)
    elif basis != "Z":
        raise ValueError(f"unknown basis {basis!r}")
    return qc

📐 Math Aside — Why those two gates, and how to derive the rotation for any observable.

The requirement is a unitary $U$ with $U P U^\dagger = Z$. Then measuring $Z$ after applying $U$ gives the same statistics as measuring $P$ would have. Check the table's entries by matrix multiplication.

For $X$, take $U = H$. Since $H = H^\dagger$ and $H^2 = I$:

$$H X H = \frac{1}{2}\begin{pmatrix}1&1\\1&-1\end{pmatrix} > \begin{pmatrix}0&1\\1&0\end{pmatrix} > \begin{pmatrix}1&1\\1&-1\end{pmatrix} > = \begin{pmatrix}1&0\\0&-1\end{pmatrix} = Z$$

For $Y$, take $U = H S^\dagger$ — note the order: $S^\dagger$ is applied to the state first, so it stands to the right in the operator product. Since $S^\dagger Y S = X$ (a $-90°$ rotation about $z$ carries $y$ onto $x$), applying $H$ afterwards finishes the job:

$$(H S^\dagger)\, Y\, (H S^\dagger)^\dagger = H (S^\dagger Y S) H = H X H = Z$$

The general recipe: any single-qubit observable with eigenvalues $\pm1$ is $\hat n \cdot \vec\sigma$ for a unit vector $\hat n$, and the rotation taking $\hat n$ to $\hat z$ on the Bloch sphere is the $U$ you want. Chapter 3 §3.4's Bloch picture is not decoration — it is the algorithm for constructing basis rotations, and it generalizes to observables that are not Paulis at all, such as the $\cos\theta\,Z + \sin\theta\,X$ that shows up in variational ansätze.

One consequence worth keeping: because $U$ is unitary and appended before the measurement, the basis rotation costs no shots and no accuracy in simulation. On hardware it costs one or two single-qubit gates, which Chapter 31 measured as the cheapest thing on the device — rz at 0.0 ns because it is a virtual frame change, sx at 56.9 ns.

Measured, at 4,096 shots:

  |0>    <Z>=+1.0000  <X>=-0.0015  <Y>=-0.0015
  |+>    <Z>=-0.0015  <X>=+1.0000  <Y>=-0.0015
  |+i>   <Z>=-0.0015  <X>=-0.0015  <Y>=+1.0000

Each state is certain in exactly one basis and maximally random in the other two. That is complementarity, measured. $|0\rangle$ is a definite state — and asking it an $X$ question gives a fair coin.

Note also the $-0.0015$ entries. Those should be exactly zero and they are not, because 4,096 shots gives a standard error of about $1/(2\sqrt{4096}) = 0.008$. A value of $-0.0015$ is entirely consistent with zero — and knowing that, rather than hunting for a bug that explains it, is what §5.4 buys you.

🧪 Run It — Finish the Chapter 4 experiment.

Chapter 4 §4.4 claimed a second basis separates entanglement from classical correlation. Now you know the mechanism. Build the Bell state and the classical impostor, and run both in $Z$ and $X$:

python for basis in ("Z", "X"): for name, prep in (("bell", bell), ("impostor", impostor)): qc = prep() if basis == "X": qc.h(0); qc.h(1) qc.measure([0, 1], [0, 1]) print(basis, name, run(qc))

Two extra gates. That is the entire cost of turning an unsupported claim into a demonstrated one.

Any Pauli expectation value, from counts

Generalizing: to measure $\langle P_{n-1} \otimes \cdots \otimes P_0 \rangle$ for Paulis $P_i \in \{I, X, Y, Z\}$, rotate each qubit according to its Pauli, measure all of them, and take the parity — $+1$ for an even number of 1s among the non-identity positions, $-1$ for odd.

def pauli_expval(counts, pauli):
    """<pauli> from counts. `pauli` is a string like 'ZZ' or 'XIY', qubit 0 RIGHTMOST."""
    total, acc = sum(counts.values()), 0
    for bits, n in counts.items():
        parity = sum(int(b) for b, p in zip(bits, pauli) if p != "I")
        acc += n * (1 if parity % 2 == 0 else -1)
    return acc / total

This is the engine of VQE. A molecular Hamiltonian is a weighted sum of Pauli strings (Chapter 36); its energy is the weighted sum of their expectation values; and each one is measured by rotating, sampling, and taking a parity. Chapter 24 assembles exactly this.

It is also where VQE's cost comes from: each distinct measurement basis needs its own set of shots. A Hamiltonian with 100 Pauli terms may need dozens of separate circuit executions per energy evaluation — which is why commuting-term grouping is a real research area and why Chapter 36 §36.2 cares about it.

Why a Pauli string cannot always share a circuit

The reason is not a software limitation, and it is worth being precise about because the fix — grouping — depends entirely on where the boundary actually falls.

You want $\langle X_0 \rangle$ and $\langle Z_0 \rangle$ from the same run. To get the first you insert h(0); to get the second you insert nothing. There is no single circuit that does both, because the measurement is one irreversible event and §5.1 already told you what it does: after it, the amplitude information the other question needed is gone. $X$ and $Z$ on the same qubit are complementary, and no amount of post-processing recovers one from the other's data. This is the ⚛️ callout in §5.1, arriving as a bill.

Now the part that saves you. Two Pauli strings can share a circuit exactly when they are qubit-wise commuting — that is, when at every qubit position their letters are either identical or one of them is $I$:

   ZZI  and  ZIZ    ->  share.   Positions agree or are I. One circuit, no rotations.
   ZZI  and  IZZ    ->  share.
   XXI  and  XIX    ->  share.   One circuit, h on the non-I positions.
   XZI  and  ZXI    ->  DO NOT.  Qubit 1 wants X in one and Z in the other.
   ZZ   and  XX     ->  DO NOT.  Both positions conflict.

Notice the last two rows. $XZ$ and $ZX$ do commute as operators — $[XZ, ZX] = 0$, since the two anticommutations cancel — and $ZZ$ and $XX$ commute too, which is why they are both stabilizers of $|\Phi^+\rangle$. Commuting is not sufficient; qubit-wise commuting is what buys you a shared circuit. Terms that commute globally but not qubit-wise can still be measured together, but only with an entangling basis change appended to the circuit — extra two-qubit gates, which on hardware is exactly the resource you were trying to save.

So the count that governs VQE's bill is neither the qubit count nor the term count. It is the number of measurement groups, and Chapter 36's measured term counts show how fast that can escape you:

   molecule   orbitals   qubits   Pauli terms
   H2                2        4            15
   LiH               6       12           631
   BeH2              7       14           666
   H2O               7       14         1,086

Read the last two rows together: 14 qubits in both cases, and 1.63× the terms. A device sized in qubits is not a device sized for a molecule. And Chapter 36 §36.2 found that bending BeH₂ by one degree takes it from 666 terms to 1,086 with its electron count untouched — so the number that sets your shot bill is not predictable from the molecular formula either.

There is one free lunch already baked into those figures. LiH's fermionic Hamiltonian holds 1,861 terms, which the Jordan–Wigner mapping collapses onto 631 distinct Pauli strings — a 2.95× reduction obtained for nothing, because distinct fermionic terms that map to the same Pauli operator simply add their coefficients. Chapter 36 §36.3 also measured that Bravyi–Kitaev gives the same 631 terms while cutting the mean Pauli weight from 6.16 to 5.62 and the maximum from 12 to 10 — and weight is exactly what decides qubit-wise commutation, so a lighter mapping groups better.

⚙️ Under the Transpiler — Grouping is a job-count optimization before it is a shot-count one.

The basis change itself is nearly free: a weight-$w$ term costs at most $2w$ single-qubit gates appended to the ansatz, and §5.6's Math Aside priced those at 0.0 ns and 56.9 ns.

The expensive part is that each measurement group is a separate circuit, and therefore potentially a separate job submission. Chapter 39 measured a 4,096-shot Bell job occupying the device for 6.93 ms against a five-minute queue — a utilization of $2.31\times10^{-5}$, or 43,340× more wall clock than device time. At that ratio, cutting 631 circuits to 200 groups saves you almost nothing in device time and saves you 431 queue waits, which is the entire difference between a run that finishes and one that does not. Chapter 39 §39.3 measured the same lever from the other end: batching 100 circuits into one job is worth about 99×.

A second consequence is easy to miss and hard to debug. Appending different basis-change layers to the same ansatz produces circuits the transpiler may lay out differently — and Chapter 39 measured a 14-qubit circuit's fidelity moving from 0.5755 to 0.7911 on transpiler seed alone, a 2.03× swing in error. Two Hamiltonian terms measured through two different layouts are not two measurements of the same state. Pin the layout across a term set, or your energy carries a systematic you will spend a week attributing to chemistry.

5.7 Marginals and Partial Measurement

Two related operations that are easy to confuse.

Marginals: ignoring bits you already measured

If you measured everything and want the distribution of a subset, sum over the rest:

from qiskit.result import marginal_counts

counts = {"100": 2072, "111": 2024}          # 3 qubits: q2=1, and q0/q1 correlated
print(marginal_counts(counts, indices=[0]))        # {'0': 2072, '1': 2024}
print(marginal_counts(counts, indices=[0, 1]))     # {'00': 2072, '11': 2024}

This is pure classical post-processing — no quantum operation, no extra shots, and you can compute as many marginals as you like from one dataset.

Formally, marginalizing sums the joint distribution over the bits you are discarding:

$$P_A(a) \;=\; \sum_b P(a, b)$$

and that sum is the same pooling operation §5.4's multinomial aside was about. So a marginal probability carries a smaller error bar than the outcomes it was built from, and the correct formula is the pooled one:

$$\sigma_{P_A(a)} = \sqrt{\frac{P_A(a)\big(1 - P_A(a)\big)}{N}}$$

Take the counts above, 4,096 shots total, marginalized onto qubit 0:

   full joint      {'100': 2072, '111': 2024}
   marginal q0     {'0': 2072, '1': 2024}
   estimate        P(q0 = 0) = 0.50586
   standard error  sqrt(0.50586 x 0.49414 / 4096) = 0.00781
   95% interval    0.4905 to 0.5212

Adding the two component standard errors in quadrature instead would have given a larger number, and it would have been wrong for the reason §5.4 derived: the counts are negatively correlated because they share a fixed total.

What a marginal cannot see

The saving comes with a loss, and the loss is the interesting part.

Marginals discard correlation, which is the only place quantum information ever lives. Compare a Bell pair against two independent fair coins:

                        joint                              q0 marginal   q1 marginal
   Bell pair     {00: 0.5,          11: 0.5}                 0.5 / 0.5     0.5 / 0.5
   two coins     {00: 0.25, 01: 0.25, 10: 0.25, 11: 0.25}    0.5 / 0.5     0.5 / 0.5

Identical marginals; completely different states. Any test built on single-qubit marginals is blind to the difference — a measurement that cannot detect the thing being asked about, which is this book's most-repeated failure mode and gets six documented instances across Part V.

And the deeper version, which is Chapter 4's whole argument: a GHZ state and a classical 50/50 mixture of 000 and 111 have identical marginals and identical full joint distributions in the $Z$ basis. No amount of marginalizing, and no number of shots, separates them. Only a second basis does — which is why §5.6 exists and why the 🧪 Run It box above is the experiment that closes Chapter 4's open question.

📊 What the Numbers Say — Free marginals are free questions, not free answers.

One 4,096-shot run of a 3-qubit circuit yields marginals over $\{0\}$, $\{1\}$, $\{2\}$, $\{0,1\}$, $\{0,2\}$, $\{1,2\}$ and the full joint: seven distributions for the price of one.

They are not seven independent results. Every one of them is computed from the same 4,096 draws, so their errors are correlated, and finding a small p-value in one of seven is much less surprising than finding one in a single pre-registered test — §5.8's multiple-comparisons arithmetic applies in full.

Marginals cost no shots and no QPU time. They do cost significance.

Partial measurement: measuring only some qubits

Measuring a subset is a quantum operation and it affects the unmeasured qubits:

qc = QuantumCircuit(2, 1)
qc.h(0)
qc.cx(0, 1)
qc.measure(0, 0)              # measure ONLY qubit 0 of a Bell state
# -> {'0': 2045, '1': 2051}

A fair coin, as expected — one qubit of a Bell state is maximally mixed (Chapter 4 §4.4).

But qubit 1 is no longer entangled with anything. It is now in a definite state, $|0\rangle$ or $|1\rangle$, matching whatever qubit 0 returned. Measuring one qubit of an entangled pair collapses both.

That is the difference. A marginal forgets information you already have. A partial measurement destroys information that was there.

⚠️ Common Pitfall — A marginal is not a measurement you did not do.

Marginalizing over qubit 1 gives you the distribution qubit 0 would have shown in that same experiment. It does not tell you what would have happened if you had measured qubit 1 in a different basis — that is a different experiment and the data cannot answer it.

This is a live issue in error mitigation and tomography, where the temptation to squeeze extra conclusions out of one dataset is strong. One dataset answers one set of questions, namely the ones about the basis you actually measured in.

5.8 Is This the Distribution I Expected?

The question you will ask constantly. Eyeballing a histogram is not an answer.

The standard tool is the chi-squared goodness-of-fit test:

$$\chi^2 = \sum_i \frac{(O_i - E_i)^2}{E_i}$$

where $O_i$ are observed counts and $E_i$ expected counts. The test returns a p-value: the probability of seeing a deviation at least this large if the expected distribution were correct. A small p-value means the data are hard to reconcile with your expectation.

from scipy.stats import chisquare

counts = run(bell_circuit, shots=1000)
observed = [counts.get("00", 0), counts.get("11", 0)]
expected = [500, 500]
chi2, p = chisquare(observed, expected)

An ideal Bell state, tested against a fair 50/50 expectation:

  Bell N=   100: obs=[55, 45]     chi2=1.000  p=0.3173
  Bell N=  1000: obs=[515, 485]   chi2=0.900  p=0.3428
  Bell N= 10000: obs=[5002, 4998] chi2=0.002  p=0.9681

All large p-values: the data are entirely consistent with 50/50, as they should be.

Degrees of freedom, and the bins that are not there

Two details decide whether the p-value you just computed means anything.

Degrees of freedom. The statistic follows a $\chi^2$ distribution with $k - 1 - m$ degrees of freedom, where $k$ is the number of bins and $m$ the number of parameters you estimated from the same data. In quantum testing $m$ is almost always 0, because your expected distribution comes from Statevector or from theory rather than from a fit — so dof $= k - 1$, and scipy.stats.chisquare assumes exactly that. If you ever do fit a parameter first (a readout-error rate, say), you must pass ddof or your p-values will be optimistic.

   bins   dof    5% critical value    mean of the distribution
      2     1               3.8415                           1
      4     3               7.8147                           3
      8     7              14.0671                           7
     16    15              24.9958                          15

The last column is the eyeball test worth internalizing: $\chi^2$ should land near its degrees of freedom. The chapter's own Bell table gives $\chi^2 = 0.900$ on 1 dof at 1,000 shots, which is ordinary. It also gives $\chi^2 = 0.002$ at 10,000 shots, $p = 0.9681$ — a suspiciously good fit. One such row is nothing; a whole column of them means your data are too well-behaved, usually because something is deterministic that should not be. A p-value near 1 is as much of a signal as a p-value near 0, and almost nobody looks at that tail.

The expected-count rule. The $\chi^2$ approximation needs every expected count to be at least about 5. With $2^n$ bins and a spread-out distribution that is a hard floor on shots before the test is even admissible:

    qubits   bins    minimum shots (5 per bin)
         3      8                           40
         8    256                        1,280
        20  1,048,576                 5,242,880

Which is Chapter 27 §27.6.2's $2^n$ shot scaling arriving by a completely different route. A distribution test over the full output of a 20-qubit circuit is not a thing you can afford, and no amount of care about the statistic changes that.

🐛 Debug This — The goodness-of-fit helper that works in simulation and crashes on hardware.

This chapter's chi2_against in code/example-03-statistical-tests.py drops bins whose expected count is zero:

python keep = [i for i, e in enumerate(expected) if e > 0] chi2, p = chisquare([observed[i] for i in keep], [expected[i] for i in keep])

Symptom on a noiseless simulator: nothing. A perfect GHZ(3) produces only 000 and 111, the six zero-expected bins are empty, the kept sums agree, and the test returns cleanly.

Symptom on hardware: the first shot that leaks into 001 raises

text ValueError: ... the sum of the observed frequencies must agree with the sum of the expected frequencies to a relative tolerance of 1.49e-08

because dropping the bin dropped its counts too, and 4,095 no longer equals 4,096.

And keeping the bins is worse, not better. Leave the zero-expected bins in and the statistic contains $(O_i - 0)^2/0$; on a realistic 4,096-shot device run with ~6% of shots off the ideal support, chisquare returns $\chi^2 = \infty$ and $p = 0.0$. The test rejects on a single leaked shot, which is true and useless — it is testing "is this device noiseless?", a question you already know the answer to.

The function now refuses instead of crashing, and makes you name the question you are asking. off_support="raise" is the default and reports how many shots leaked where. The two working modes are the ones worth understanding, because they disagree:

```text 4,096 shots, ~6% off the ideal support: {'00':1927, '11':1929, '01':121, '10':119}

off_support="pool" chi2 = 69,866.24 p = 0.0000 REJECT off_support="restrict" chi2 = 0.0010 p = 0.9743 looks perfect ```

pool floors every zero-expectation bin at a small $\epsilon$ and renormalizes, so the test asks "is this the ideal distribution, allowing for a little leakage?" — and 6% is not a little. restrict renormalizes the expectation over the ideal support and tests only those bins, so it asks "given that a shot landed somewhere legal, was it in the right place?" On this data the answer is a resounding yes, because the leakage is symmetric and the surviving 00/11 split is almost exactly even.

Both numbers are correct. Only one of them is an answer to "did my circuit work." restrict is structurally incapable of seeing leakage — it deletes the evidence before computing — which makes it the same failure this book keeps finding: a measurement that cannot detect the thing being asked about. Chapter 26 §26.4's blind bisection, Chapter 27 §27.4's blind test inputs, and Chapter 17's sorted population check are the same shape. Reach for restrict only when leakage is separately accounted for, and say so when you report the p-value.

A third option avoids the choice: Chapter 27 §27.5 uses total variation distance, which has no zero-bin pathology at all — see below.

The generalizable lesson: a test whose validity depends on outcomes having exactly zero probability is a test that only runs in simulation. Every distribution on real hardware has full support.

Statistical power: what the test can and cannot see

Now a circuit that is genuinely biased — $P(1) = 0.55$ rather than 0.5 — tested against the same fair-coin expectation:

  p=0.55 biased N=   100: obs=[47, 53]         p-value=0.54851 -> cannot reject
  p=0.55 biased N=  1000: obs=[467, 533]       p-value=0.03688 -> REJECT fair
  p=0.55 biased N= 10000: obs=[4442, 5558]     p-value=0.00000 -> REJECT fair
  p=0.55 biased N=100000: obs=[44795, 55205]   p-value=0.00000 -> REJECT fair

At 100 shots the test cannot see a 5-percentage-point bias. Not "the bias is small" — the test has no power to detect it, and reports a perfectly comfortable p-value of 0.55 for data that are definitely not fair.

That is the single most important thing to understand about this test:

A large p-value does not mean your circuit is correct. It means your experiment was not sensitive enough to prove otherwise.

"I ran 100 shots and the chi-squared test passed" is not evidence of correctness. It is evidence of an underpowered experiment. Before running, ask: what size of deviation do I need to be able to detect, and how many shots does that require?

Roughly, to detect a bias of size $\delta$ you need on the order of $1/\delta^2$ shots. For $\delta = 0.05$ that is around 400 — consistent with the table, where 1,000 shots detected it reliably and 100 did not.

Sharpening the constant: how many shots is "enough power"?

$1/\delta^2$ is the right shape. The constant in front is where the honesty lives, and it depends on something the rule of thumb never mentions: how often you are willing to miss.

A two-sided test at significance $\alpha$ with power $1 - \beta$ against a bias $\delta$ needs

$$N \;=\; \frac{\big(z_{\alpha/2} + z_\beta\big)^2\, p(1-p)}{\delta^2}$$

Two $z$-values, not one. $z_{\alpha/2} = 1.96$ controls false alarms; $z_\beta$ controls misses, and setting it to zero — which is what happens when you use the precision formula from §5.4 as a power formula — buys you power of exactly 50%.

   (z_a + z_b)^2 :   3.8415 at 50% power
                     7.8489 at 80% power
                    10.5074 at 90% power

     delta |  50% power |  80% power |  90% power
     0.200 |         25 |         50 |         66
     0.100 |         97 |        197 |        263
     0.050 |        385 |        785 |      1,051
     0.020 |      2,401 |      4,906 |      6,568
     0.010 |      9,604 |     19,623 |     26,269
     0.001 |    960,365 |  1,962,220 |  2,626,856

★ Look at the 50%-power column and compare it with §5.4's precision table. They are the same numbers. 385 at $\delta = 0.05$, 9,604 at 0.01, 960,400 at 0.001 — because $1.96^2 \times 0.25 = 0.9604$ appears in both. That coincidence is a trap: the shot count that gives you a ±0.05 error bar is the shot count at which a genuine 5-point bias is detected half the time. "Precise enough" and "powerful enough" are different requirements, and the standard conflation costs a factor of 2.04 in shots.

Run the power curve against the chapter's own measured table, at $\delta = 0.05$:

   shots      power     what the table above showed
     100     0.1688     p = 0.549, cannot reject
     385     0.5009
     785     0.8012
   1,000     0.8865     p = 0.037, REJECT
  10,000     1.0000     p < 0.001, REJECT

Everything lines up. At 100 shots the test rejects a genuinely biased circuit 17% of the time — so the "cannot reject" row is the expected outcome, not bad luck. At 1,000 shots it rejects 89% of the time, and the observed $p = 0.037$ is a marginal rejection, which is exactly what 89% power looks like from the inside.

⚠️ Common Pitfall — A p-value is itself a random variable, and this chapter's table has one draw of each.

Turn the book's own recurring lesson on the table two screens up. Each row is one run at that shot count. At 1,000 shots and a true 5-point bias, the power is 0.8865 — so roughly one run in nine fails to reject, and had the seed landed there, the row would read "cannot reject" and the chapter would appear to say something different.

The conclusion is unaffected, because it does not rest on any single row: the power arithmetic is derived, the trend across four shot counts is monotone, and the boundary lands where $1/\delta^2$ predicts. But the correct reading of a single p-value is "one draw from a distribution that depends on the truth, the shot count, and the seed."

Practical consequence: if a test is near your decision threshold — anything between $p = 0.01$ and $p = 0.10$ — the answer is not "significant" or "not significant." The answer is "run more shots." Chapter 27 §27.5 documents this book's own version of that mistake, where a false-failure rate of 1.0% from 2/200 runs turned out to be 0.150% at 2,000.

The other way to be fooled: asking too many questions

Power protects you from missing a real effect. Nothing so far protects you from finding a fake one, and the exposure grows with every comparison you make:

   independent tests at alpha = 0.05    P(at least one "significant")
                                  1                          0.0500
                                  2                          0.0975
                                  5                          0.2262
                                  8                          0.3366
                                 20                          0.6415

Eight comparisons at the 5% level give you a one-in-three chance of a spurious hit even when every single one of your circuits is perfect. Case Study 2 is built around a report that did exactly this, and §5.7's free marginals are a quiet on-ramp to it: seven marginals from one dataset is seven questions, and the $1 - 0.95^7 = 0.30$ applies.

The defences are all cheap and all social rather than statistical: decide which comparison matters before you look, report how many you ran, and if you must run many, tighten the threshold (Bonferroni divides $\alpha$ by the number of tests). What you must not do is run eight and report the best one, which is the default behaviour of a curious person with a fast simulator.

🔬 Honest Assessment — What a passing test actually licenses you to say.

Three claims, in increasing strength, and only the first two are usually available:

"The data are consistent with my expectation." Fine, if you also state the shot count and the effect size you could have detected. Without those, the claim is empty.

"The data are inconsistent with hypothesis H, at p < 0.01." A strong, honest claim. Rejecting is much easier than confirming, which is why well-designed experiments are usually framed as attempts to reject something.

"My circuit is correct." A statistical test cannot give you this, ever. It can fail to reject a correct circuit and it can fail to reject a subtly broken one. Correctness comes from verification against known cases, statevector comparison in simulation, and property-based tests — which is what Chapter 27 is for.

Always report the shot count with any statistical claim. A p-value without an $N$ is uninterpretable, and it is omitted constantly.

Chi-squared's cousin: total variation distance

$\chi^2$ is not the only way to compare two distributions, and it is not the one this book's testing chapter settles on. The alternative is total variation distance:

$$\mathrm{TVD}(\hat p, q) \;=\; \tfrac12 \sum_i \big|\hat p_i - q_i\big|$$

— half the summed absolute disagreement, which equals the largest probability by which the two distributions can disagree about any event. It lives in $[0, 1]$, it has no zero-bin pathology, and it is a distance rather than a p-value, so you set a tolerance on it and assert.

The two statistics weight errors differently, and the difference decides which one you want:

   chi-squared    divides by E_i -> sensitive to RELATIVE error
                  a bin expected at 0.001 that comes back at 0.002 contributes as much
                  as one expected at 0.5 coming back at 0.6.  Undefined at E_i = 0.

   TVD            absolute error, unweighted
                  a 0.001 -> 0.002 bin contributes 0.0005 and is invisible.
                  Fine at q_i = 0.

If a forbidden outcome appearing at all is the bug, use $\chi^2$ (with the pooling fix above). If the bulk shape is what you care about, use TVD. Most quantum circuit tests want the second, which is why Chapter 27 uses it.

Now the part that makes TVD usable, and the reason it is worth meeting here rather than there: a perfectly correct circuit has a nonzero TVD, and you cannot set a tolerance until you know how big. Chapter 27 §27.5 measured it on a noiseless GHZ(3), 40 independent runs at each shot count:

       shots    mean TVD       std   max over 40 runs
         100     0.04375   0.03199            0.12000
       1,000     0.01313   0.01003            0.03700
      10,000     0.00423   0.00283            0.01160
     100,000     0.00121   0.00101            0.00401

Every number in that table is the error of a circuit with nothing wrong with it. A tolerance tighter than the last column fails correct code some fraction of the time, forever.

The shape is derivable with the tools of §5.4, and the derivation is the useful part because it tells you what the table cannot — how the floor changes for a distribution that is not GHZ(3)'s. Each $\hat p_i$ is a binomial fraction with $\sigma_i = \sqrt{p_i(1-p_i)/N}$; for a roughly normal deviate $\mathbb{E}|\hat p_i - p_i| = \sigma_i \sqrt{2/\pi}$; so

$$\mathbb{E}[\mathrm{TVD}] \;=\; \tfrac12\sqrt{\tfrac{2}{\pi}}\sum_i \sqrt{\frac{p_i(1-p_i)}{N}} \;=\; \frac{1}{\sqrt{2\pi N}}\sum_i \sqrt{p_i(1-p_i)} \;=\; \frac{0.39894\,S}{\sqrt N}$$

where $0.39894 = 1/\sqrt{2\pi}$ and $S = \sum_i \sqrt{p_i(1-p_i)}$ is a shape factor belonging to the distribution, not to the shot count.

Check it. A noiseless GHZ(3) puts $1/2$ on two outcomes and zero on six, so $S = 2\sqrt{0.25} = 1$ exactly and $\mathbb{E}[\mathrm{TVD}] = 0.39894/\sqrt{N}$. At 1,000 shots that predicts 0.01262 against a measured 0.01313 — agreement to 4%, on a mean of 40 runs.

★ The $1/\sqrt N$ is universal. The constant in front is not. For a distribution spread uniformly over $M$ outcomes, $S = \sqrt{M - 1}$, so a full 3-qubit uniform output has $S = 2.6458$ and a floor 2.65× higher than GHZ(3)'s at the same shot count — and a 10-qubit uniform output, $S = 31.98$, has a floor 32× higher. Holding a TVD tolerance fixed while adding qubits therefore costs shots like $2^n$, which is §27.6.2's result. It does not mean testing stops working past ten qubits — it means TVD over the full output distribution does, and that past that width you must test something narrower: a marginal, an expectation value, or a specific property.

For the tolerance itself, Chapter 27 found the max over 40 runs sat at roughly three times the mean at every shot count measured (2.74, 2.82, 2.74, 3.31). Three times the derived mean is $1.197\,S/\sqrt{N}$, which for GHZ(3) at 1,000 shots gives 0.03785 against the measured maximum of 0.03700.

⚠️ Common Pitfall — The shot-noise floor is not a number, and writing it as one is how it goes wrong.

The temptation is to memorize a single tolerance and reuse it. Chapter 27 §27.5.1 documents its own project code doing exactly that — encoding the floor as a safety factor divided by $\sqrt{\text{shots}}$, a factor of three applied to a mean the function never computes. The safety factor is sound; the missing $S$ is not, and the omission is invisible on GHZ-like circuits where $S = 1$ and silently wrong by 32× on a wide output distribution.

Carry the formula, not the number: $\mathbb{E}[\mathrm{TVD}] = 0.39894\,S/\sqrt N$, tolerance about three times that, and compute $S$ from the distribution you are actually testing against — you have it, because you needed the exact distribution to compute TVD in the first place.

§27.5.1 also warns about a second confusion worth importing here: this material contains two different threes. The safety factor above multiplies a mean TVD and scales as $N^{-1/2}$ in shots; the rule of three from §5.4's zero-events callout bounds a rate and scales as $N^{-1}$ in runs of the whole test. They share a digit and nothing else, and mixing them produces a number that is wrong by orders of magnitude and looks entirely plausible.

5.9 Choosing a Shot Count

Putting it together into a decision.

Step 1: What are you estimating? A probability, an expectation value, or a whole distribution? For a distribution over $2^n$ outcomes, costs grow exponentially — usually a sign you should be asking for an expectation value instead.

Step 2: What precision do you need, and why? This should come from the problem, not from habit. Chapter 36 needs 1.6 mHa on an energy near $-1.137$ Ha, so about $10^{-3}$ relative — that number comes from chemistry, not from a default.

Step 3: Compute the shots. $N \approx 0.96/\epsilon^2$ for a probability at 95% confidence.

Step 4: Check it against your budget. Multiply by 100 μs, then by the number of circuits, then by the number of optimizer iterations. This is where variational algorithms become expensive and where the estimate usually forces you to relax step 2.

Step 5: Sanity-check against noise. If device noise shifts your answer by 5%, there is no point estimating to 0.1%. Match your statistical precision to your systematic error — Chapter 2's distinction, now with a decision attached.

That last step is the one people skip. Measured on a device-derived model in Chapter 2, the Bell state's error fraction was around 4%. Estimating it to ±0.01% would be a precise measurement of a biased quantity — a waste of ten thousand times the shots for no gain in what you actually learn.

Practical defaults, to be overridden with reasons:

When more shots are the wrong answer

Everything above prices statistical error. Shots reduce statistical error and nothing else, so before buying more, ask what fraction of your uncertainty is statistical. Three cases where the answer is "almost none," each measured elsewhere in this book:

The variation is between runs, not within them. Chapter 39 transpiled one 14-qubit circuit under 24 seeds and measured fidelities from 0.5755 to 0.7911 — a 2.03× spread in error, with two-qubit gate counts running from 49 to 112. No shot count touches that, because it is not sampling noise: it is 24 different circuits. The fix is repetitions across seeds and a reported distribution, not a bigger $N$. And the trap is in the control: the same test on 4 qubits produced exactly zero variation. A small circuit will tell you the problem does not exist.

The device moved. Chapter 30 measured a single chip's quoted two-qubit error ranging from 0.00750 to 0.07205 across its links — a factor of 9.6 on one device on one day. "The error rate" is not a number, and a million shots gives you a very precise measurement of this morning's calibration.

The quantity is biased. §5.9 step 5, and the reason it is step 5 rather than a footnote.

The positive case is worth naming too, because the discipline is not "never buy shots." Chapter 31's dynamical-decoupling result — XX at $-0.0053 \pm 0.0012$, a 4.4-standard-error effect in the wrong direction — is a conclusion you can only reach with an honest uncertainty. Without the error bar it is a shrug; with it, it is a finding that a widely recommended technique made things significantly worse on that device.

🔬 Honest Assessment — About that 100 μs per shot.

§5.4's cost table converts shots to QPU seconds at 100 μs per shot. That figure is a budgeting convention used throughout Part I, not a measurement, and it is worth saying so before you quote "96 seconds for ±0.001" to anyone.

Chapter 39 measured what a job actually occupies the device for, by scheduling the circuit with alap and reading off the duration:

text circuit duration at 4,096 shots implied per shot Bell 1.69 us 6.93 ms 1.69 us QFT-8 10.55 us 43.20 ms 10.55 us

The planning constant is 59× the measured circuit duration for a Bell state. At 960,400 shots the §5.4 table says 96 s; a Bell circuit's scheduled time is about 1.6 s. The two numbers count different things — Ch.39's is scheduled circuit duration times shots, including the 1,560 ns measurement, while the 100 μs figure is a conservative whole-cycle allowance — but the gap is large enough that the row should be read as an upper bound.

The right conclusion is not "shots are cheap." Chapter 39's actual finding is that at a five-minute queue the utilization of a 4,096-shot Bell job is $2.31\times10^{-5}$ — 43,340× more wall clock than device time. Device seconds are not what decides when your result arrives, so §5.4's table is best read as statistics with a price tag for scale, not as a runtime prediction.

Where the per-shot figure does become the number that matters is billing. Chapter 39 priced one VQE run three ways and got \$50 under per-minute pricing and \$7,432 under per-shot pricing — a 149× spread on identical work. Under per-shot pricing you pay for every shot regardless of how briefly it occupied anything, and §5.4's arithmetic is then exactly the arithmetic of your invoice.

🧱 Project Checkpoint — measure.py v0.

The project gets its measurement layer: counts in, expectation values with error bars out.

```python

vqelab/measure.py -- v0

import math

def probabilities(counts: dict[str, int]) -> dict[str, float]: total = sum(counts.values()) return {k: v / total for k, v in counts.items()}

def pauli_expval(counts: dict[str, int], pauli: str) -> float: """ from counts. Qubit 0 is the RIGHTMOST character of both.""" total, acc = sum(counts.values()), 0 for bits, n in counts.items(): parity = sum(int(b) for b, p in zip(bits, pauli) if p != "I") acc += n * (1 if parity % 2 == 0 else -1) return acc / total

def expval_stderr(counts: dict[str, int], pauli: str) -> float: """Standard error of the above. An expectation value without one is not a result.""" n = sum(counts.values()) ev = pauli_expval(counts, pauli) return math.sqrt(max(0.0, 1 - ev**2) / n) # variance of a +/-1 variable ```

The error bar is not optional. Every energy this project ever reports will carry one, because an expectation value without an uncertainty cannot be compared to a target — and comparing to $-1.137 \pm 0.0016$ Ha is the entire point of the project.

The checkpoint file also adds shots_for_precision(epsilon) and basis_rotation(pauli), which Chapter 24's optimizer loop will call on every iteration.

5.10 Summary

Measurement is an operation, not a reading. It returns a classical bit and replaces the state with the corresponding basis state, irreversibly. Everything the amplitudes encoded, apart from the outcome, is gone.

Measurement happens in a basis, and the computational basis is a default rather than a law. A state that is a fair coin in one basis can be deterministic in another. Hardware only measures in the computational basis, so you rotate the state instead: h for $X$, sdg then h for $Y$.

Qiskit is little-endian — rightmost character is qubit 0 — and with multiple registers the last-declared appears leftmost. Mismatched measure lists are legal and silently bit-reverse your results. Test with asymmetric states; symmetric ones cannot detect ordering bugs.

Sampling error is $\sqrt{p(1-p)/N} \le 1/(2\sqrt N)$. Ten times the precision costs a hundred times the shots, always, on any hardware, forever. $N \approx 0.96/\epsilon^2$ at 95% confidence: ±0.01 needs about 9,600 shots and ±0.001 needs about 960,000.

Counts, probabilities, expectation values are three summaries of one dataset. An expectation value is one number and costs $O(1/\epsilon^2)$ shots regardless of qubit count, while a full distribution over $2^n$ outcomes costs exponentially more. That is why the Estimator primitive exists, and asking the right question is worth an exponential factor.

For a Bell state: $\langle ZZ\rangle = \langle XX\rangle = +1$, $\langle YY\rangle = -1$ (the sign is real — two factors of $i$), and every single-qubit expectation is exactly zero. All the information is in the correlations.

Any Pauli expectation comes from rotating, sampling, and taking a parity. This is the engine of VQE, and the need for a separate basis per Pauli term is where VQE's cost lives.

Marginals are free classical post-processing over data you already have. Partial measurement is a quantum operation that collapses entangled partners too. A marginal forgets; a partial measurement destroys.

The chi-squared test tells you whether data are consistent with an expectation — and a large p-value means your experiment lacked the power to detect a deviation, not that your circuit is correct. At 100 shots the test could not see a 5-percentage-point bias. Always report $N$ with a p-value.

Choose shots deliberately: what are you estimating, what precision does the problem require, what does that cost, and — the step everyone skips — is that precision smaller than your systematic error? Estimating a biased quantity to four decimal places is a waste of a thousandfold in shots.

Next: Chapter 6 — the assembly language underneath every framework. Reading it, writing it, using it to move circuits between tools, and using it to see exactly what the transpiler did to your work.