Case Study 2: The Ansatz That Was Not the Problem

DataField.Dev

Case Study 2: The Ansatz That Was Not the Problem

The symptom

A four-parameter variational ansatz on a two-qubit Hamiltonian:

$$H = Z_0 Z_1 + 0.5\,X_0 + 0.5\,X_1, \qquad E_0 = -\sqrt2 = -1.414214$$

The exact ground state energy comes from diagonalizing $H$ — the reference value Chapter 7 §7.7 insisted every result be measured against.

Run gradient descent for 40 steps:

   step       energy
      0     1.238195
      8    -0.991359
     16    -1.244466
     24    -1.246329
     32    -1.247585
     40    -1.248423

   final -1.248423     error 1.66e-01

Stuck 0.166 above the answer. The trace has clearly flattened — the last eight steps moved it by 0.0008 — and it is not going to close a gap of 0.166 at that rate.

The diagnosis everybody reaches for

The ansatz is not expressive enough.

It is a reasonable inference, and it has a supporting story. Four parameters, two RY layers and one CNOT — that is a very small corner of the two-qubit state space. $H$ contains both $ZZ$ and $X$ terms, so the ground state is genuinely entangled and not obviously reachable. The optimizer descended smoothly, converged to something, and stopped. Textbook underfitting.

The fix follows directly: more parameters.

Testing the diagnosis

   ansatz                                 energy        error
   shallow RY (4 params)                -1.414214     6.66e-16
   RY+RZ (8 params)                     -1.414212     1.33e-06
   StronglyEntanglingLayers x3 (18)     -1.414213     6.32e-08

The four-parameter ansatz reaches the exact answer. To $6.66\times10^{-16}$ — machine precision.

It is also the most accurate of the three. The 8- and 18-parameter versions land slightly further off, because more parameters means more directions to drift in and a flatter approach to the minimum.

The only change between the failing run and this one: AdamOptimizer(0.1) for 400 steps instead of GradientDescentOptimizer(0.25) for 40.

The ansatz was never the problem.

What was the problem

Not the ansatz, and — this is the part that surprised me — not the step size either.

The instinct on a stalled optimizer is that it is overshooting, and the fix is a smaller step. Testing that directly, same ansatz, 200 steps each:

   optimizer                       energy        error
   GradientDescent(0.25)        -1.414214     2.28e-08
   GradientDescent(0.05)        -1.259566     1.55e-01
   Adam(0.1)                    -1.414214     8.76e-09
   Momentum(0.1)                -1.414214     7.80e-10

Plain gradient descent at 0.25 converges fine — given 200 steps instead of 40.

And at 0.05 it does not converge at all, sitting 0.155 away after five times the original step budget. The instinctive fix makes it strictly worse.

So the original failure was the simplest thing it could have been: the run was stopped in the middle of its descent. Not overshooting, not underfitting, not a bad landscape. It had not finished.

Why 40 steps looked like convergence

Because the trace flattens long before it arrives. Between steps 32 and 40 the energy moved by 0.0008 — which reads as "converged to four decimal places" if you are looking at the number rather than at its trend.

Gradient descent on this landscape has a long shallow approach. The gradient is genuinely small near the path, so progress per step is small, and the shape of the curve at step 40 looks exactly like the shape at step 400 — except one of them is at −1.248 and the other at −1.414.

A flattening trace is not a converged trace. The distinguishing question is not "has it stopped moving?" but "has it stopped moving because it arrived?" — and the only way to answer that is to check whether the last step still changed anything above your tolerance.

The two diagnostics

Both are cheap, and either would have prevented the misdiagnosis.

1. Is the trace still descending at the final step?

still_descending = abs(trace[-1] - trace[-2]) > tolerance

If yes, you ran out of steps. This is a stopping-criterion bug, and no amount of extra parameters fixes it. vqelab.variational.optimize returns this as a converged flag precisely because the flag is the entire lesson of this case study — a result object that reports a number without reporting whether the number is finished is inviting exactly this error.

2. Change the optimizer before changing the ansatz.

If a different optimizer closes the gap, the ansatz was never the constraint. This takes one line and one minute, and it distinguishes an optimization failure from a capacity failure definitively.

Only after both come back clean is "the ansatz is too small" a supported conclusion.

Why the wrong fix is expensive

Adding parameters to fix an optimizer problem is not neutral — it actively makes things worse, in three ways this chapter measured.

The gradient bill grows. §16.4: a gradient costs exactly $2n+1$ circuit executions. Going from 4 to 18 parameters takes it from 9 to 37 executions per iteration — four times the shot cost, on every iteration, forever.

The landscape gets flatter. §16.6: gradient variance decays with circuit width, and larger ansätze are wider and deeper. You are moving toward the barren plateau in order to fix a problem that was not about the landscape.

And the answer gets slightly worse, as the table above shows: 6.66e-16 for four parameters against 6.32e-08 for eighteen.

Every axis moves the wrong way, in exchange for fixing nothing.

The lessons

"Not expressive enough" is the most over-diagnosed failure in variational quantum computing. It is plausible, it fits the symptom, and it suggests an action — which is exactly the profile of a diagnosis that gets accepted without testing.

Check convergence before capacity. A stalled optimizer and an inadequate ansatz produce the same plot. One is fixed with more iterations; the other needs a different model. They are trivially distinguishable by whether the trace is still moving.

A flattening trace is not a converged trace. Look at the trend, not the number of stable decimal places.

The instinctive fix can be the wrong direction. Reducing the step size — the standard response to a stall — made this strictly worse. The failure was insufficient distance travelled, and a smaller step travels less.

Always have a reference value. All of this is only diagnosable because $-\sqrt2$ was known independently, by diagonalizing $H$. Without it, −1.248 is just a number the optimizer returned, and there is nothing to be suspicious of. This is Chapter 7 §7.7's point, and it has now paid for itself in Chapters 7, 10, 12, 13, and here.

And, connecting to Case Study 1: both failures in this chapter were misdiagnoses supported by convincing evidence. One produced a clean exponential fit to an artifact; the other produced a smooth convergence curve to the wrong value. In both cases the data looked good and the conclusion was wrong — which is the argument for building the diagnostic into the tooling rather than relying on noticing.

Reproduce it: code/example-04-optimization.py runs the stall, the ansatz comparison, and the optimizer sweep; optimize() in code/project-checkpoint.py returns the converged flag.