What a PASS means¶

A PASS means the checks found nothing wrong. It does not prove the simulation correct. Each check family rests on assumptions; when one is false, a check can pass on wrong code.

Family	Assumes	Independent of the code?
Manufactured solutions	The reference equations are right; the solution is smooth; the steps are in the asymptotic range; the adapter adds the source where the solver evaluates the right-hand side.	Only if the reference equations were written from the physics, not copied from the code.
Reference solutions	The same, plus a trustworthy reference integrator (RK4 at fine steps).	Same as above.
Conserved quantities	The quantity really is conserved (or its budget is known) for this problem.	Yes, if it comes from the physics. A good second check.
Known limits	The limit is exact and the code can reach it.	Yes.
Patch tests	The scheme represents the chosen solution exactly (see patch test theory).	Yes, but only for the parts of the scheme the patch touches.
Runge-Kutta tables	The table you pass is the one the code uses.	Yes: pure algebra on the coefficients.

A check that shares the code's mistake¶

The solver below should solve a spring, x'' = -omega² x, but has the sign of the force wrong. The reference right-hand side was written by looking at the solver, so it has the same wrong sign:

"""A reference that shares the code's mistake passes; an independent check catches it.

The solver below should solve x'' = -omega^2 x (a spring), but has the sign of the
force wrong. A reference right-hand side copied from the solver has the same wrong
sign, so the order checks pass. Energy conservation is derived from the physics,
not from the code, and fails.
"""

import numpy as np

from physverify import ManufacturedODE, invariant_study, mms_order_study, reference_study

OMEGA = 2.0


def rhs(t, y):
    return np.array([y[1], +(OMEGA**2) * y[0]])  # bug: the force should be -omega^2 x


def integrate(source, y0, t_end, n):  # midpoint method with a source-term hook
    h, y = t_end / n, np.array(y0, dtype=float)
    for i in range(n):
        t = i * h
        mid = y + 0.5 * h * (rhs(t, y) + source(t))
        y = y + h * (rhs(t + 0.5 * h, mid) + source(t + 0.5 * h))
    return y


def copied_rhs(t, y):  # written by looking at the solver: the same sign error
    return np.array([y[1], +(OMEGA**2) * y[0]])


def energy(y):  # from the physics: kinetic plus spring energy is constant
    return 0.5 * y[1] ** 2 + 0.5 * OMEGA**2 * y[0] ** 2


ns = (20, 40, 80, 160)
y0 = np.array([1.0, 0.0])
mms = ManufacturedODE(lambda t: np.array([np.sin(t), np.cos(t)]), copied_rhs)
print("mms, copied reference:      ", mms_order_study(integrate, mms, 1.0, ns, 2.0).status)
print("reference, copied reference:", reference_study(integrate, copied_rhs, y0, 1.0, ns, 2.0).status)
print("energy conservation:        ", invariant_study(integrate, energy, y0, 1.0, ns, 2.0).status)

mms, copied reference:       PASS
reference, copied reference: PASS
energy conservation:         FAIL

Both order checks pass, because the code converges at second order, to the wrong equations. The energy check fails, because energy conservation comes from the physics and not from the code. Always include at least one check that is independent of how the equations were typed in: a conserved quantity, a known limit or an exact solution.

Checklist before trusting a PASS¶

Are the reference equations written from the physics (paper, textbook, notes), not copied from the code?
Are units and signs stated, and do they match the code's conventions?
Are initial and boundary conditions checked separately (they are easy to get wrong and a convergence study may not reach them)?
Does the test reach the part of the state space where bugs hide (large amplitudes, boundaries, odd grid sizes)?
Is there a second, independent check, such as a conserved quantity or a limiting case?