The PyPSA ladder¶
mathspec states PyPSA in one file, grown a rung at a time. Each rung here is that file projected onto what its network builds, shown as specsolve builds it beside the PyPSA code that builds the same network, and compared with PyPSA four ways. The PyPSA parity workflow regenerates every page's sources from the pinned mathspec on each run and fails on a diff.
objective — one number, both solves · structure — the same constraint and variable names, one block each · size — the same solver rows, columns and nonzeros · duals — every constraint's dual, per row · linopy lane — the two linopy models, label for label. ✔ identical · ≠ differs, with the recorded reason · ◌ not comparable yet.
| rung | objective | structure | size | duals | linopy lane |
|---|---|---|---|---|---|
| Rung 1 — transport | ✔ 7182.222222222221 |
✔ 7 constraints · 2 variables, name for name | ✔ 45 rows · ✔ 16 columns · ✔ 57 nonzeros | ✔ 45 rows | ✔ 10 equal |
| Rung 2 — storage | ✔ 4456.659315422355 |
✔ 18 constraints · 8 variables, name for name | ✔ 103 rows · ✔ 48 columns · ✔ 166 nonzeros | ✔ 103 rows, 2 negated | ✔ 27 equal |
| Rung 3 — expansion | ✔ 7633.908502024291 |
≠ tech_capacity_expansion_limit 5 vs 1+2+2 — one block per sense — ==, <=, >= — where PyPSA writes one row per labelled constraint whatever its sense; transmission_expansion_cost_limit 2 vs 1+1 — one block per sense — ==, <=, >= — where PyPSA writes one row per labelled constraint whatever its sense |
✔ 184 rows · ✔ 73 columns · ✔ 328 nonzeros | ✔ 184 rows, 4 negated | ✔ 57 equal, 3 split |
| Rung 4 — ramps | ✔ 8785.0 |
✔ 9 constraints · 2 variables, name for name | ✔ 64 rows · ✔ 20 columns · ✔ 92 nonzeros | ✔ 64 rows, 2 negated | ✔ 12 equal |
| Rung 5 — global constraints | ✔ 10282.833333333332 |
≠ operational_limit 3 vs 1+1+1 — one block per sense — ==, <=, >= — where PyPSA writes one row per labelled constraint whatever its sense; primary_energy 3 vs 1+1+1 — one block per sense — ==, <=, >= — where PyPSA writes one row per labelled constraint whatever its sense |
✔ 102 rows · ✔ 44 columns · ✔ 190 nonzeros | ✔ 102 rows, 2 negated | ✔ 23 equal, 2 split |
| Rung 6 — KVL | ✔ 23962.0 |
≠ transmission_expansion_cost_limit 2 vs 1+1 — one block per sense — ==, <=, >= — where PyPSA writes one row per labelled constraint whatever its sense; transmission_volume_expansion_limit 2 vs 1+1 — one block per sense — ==, <=, >= — where PyPSA writes one row per labelled constraint whatever its sense |
✔ 123 rows · ✔ 42 columns · ✔ 204 nonzeros | ✔ 123 rows | ✔ 19 equal, 3 split |
| Rung 7 — commitment | ✔ 7775.0 |
✔ 17 constraints · 5 variables, name for name | ✔ 116 rows · ✔ 44 columns · ✔ 237 nonzeros | — integer model, no duals | ✔ 20 equal · ≠ 3 recorded |
| Rung 8 — modular and big-M | ✔ 15915.0 |
✔ 28 constraints · 7 variables, name for name | ✔ 191 rows · ✔ 80 columns · ✔ 379 nonzeros | — integer model, no duals | ✔ 36 equal |
| Rung 9 — a multi-link with four output ports | ✔ 11714.4 |
✔ 5 constraints · 2 variables, name for name | ✔ 92 rows · ✔ 32 columns · ✔ 116 nonzeros | ✔ 92 rows | ✔ 8 equal |
| Rung 10 — quadratic costs | ✔ 12587.437500000098 |
✔ 5 constraints · 2 variables, name for name | ✔ 60 rows · ✔ 24 columns · ✔ 80 nonzeros | ✔ 60 rows | ✔ 8 equal |
Rung 11 — PyPSA's own ac_dc_meshed example, whole |
✔ 18441021.477729 |
≠ objective_constant 1 vs 0 — PyPSA carries a nonzero objective constant as a fixed variable of that name; the file states no constant, and the objective is compared net of it |
✔ 468 rows · ≠ 188 vs 187 columns · ✔ 1007 nonzeros | ✔ 468 rows | ✔ 17 equal, 1 split · ≠ 1 recorded |
| Rung 12 — linearized unit commitment | ✔ 7775.0 |
✔ 21 constraints · 5 variables, name for name | ✔ 128 rows · ✔ 44 columns · ✔ 288 nonzeros | ≠ 128 rows, 1 negated, Generator-com-status-min_up_time_must_stay_up off by 490.0 — degenerate with PyPSA's duplicated cap — a unit held on has status pinned at 1 by this row from below and by the variable bound and the cap row from above, and HiGHS may put the dual on any of them; Generator-status-p-fixed-upper off by 1447.5 — PyPSA bounds the status at 1 on the variable and writes the cap row as well, so a binding cap's dual may sit on the bound and leave the row at zero; the file states the row alone (rung 12, the linearized relaxation) |
✔ 24 equal · ≠ 3 recorded |
| Rung 13 — transmission losses in tangent form | ✔ 10805.29588 |
≠ objective_constant 1 vs 0 — PyPSA carries a nonzero objective constant as a fixed variable of that name; the file states no constant, and the objective is compared net of it |
✔ 150 rows · ≠ 46 vs 45 columns · ✔ 290 nonzeros | ✔ 150 rows | ✔ 22 equal · ≠ 1 recorded |
| Rung 14 — two futures and a risk preference | ✔ 9267.386667 |
≠ Generator-ext-p_nom-lower 2 vs 1 — PyPSA writes the build's floor once per scenario, each row over the one capacity variable; the file states it once — capacity is chosen before the future is known, and a copy per future is the same row again.; Generator-ext-p_nom-upper 2 vs 1 — PyPSA writes the build's cap once per scenario, each row over the one capacity variable; the file states it once — capacity is chosen before the future is known, and a copy per future is the same row again. |
≠ 87 vs 85 rows · ✔ 37 columns · ≠ 148 vs 146 nonzeros | ≠ 87 rows, Generator-ext-p_nom-upper off by 36.714666667 — two copies of one binding row are degenerate — the solver may put the cap's whole price on either copy, so PyPSA's per-scenario duals are shares of the file's one. |
✔ 17 equal · ≠ 2 recorded |
| Rung 15 — two investment periods | ✔ 12747.191096 |
✔ 10 constraints · 3 variables, name for name | ✔ 80 rows · ✔ 31 columns · ✔ 117 nonzeros | ✔ 80 rows | ✔ 14 equal |
| Rung 16 — link delay | ✔ 5262.5 |
✔ 5 constraints · 2 variables, name for name | ✔ 52 rows · ✔ 20 columns · ✔ 67 nonzeros | ✔ 52 rows | ✔ 8 equal |
The four comparisons¶
Both sides start from one object, the network the rung's script builds. PyPSA solves it directly; specsolve solves the file attached to the tables prep.py makes of it.
| column | specsolve | PyPSA | identical means |
|---|---|---|---|
| objective | result.objective |
n.objective + n.objective_constant |
equal, relative 1e-9 |
| structure | len(result.activity(block)), len(result.primal(variable)) |
rows and columns of n.model per name, masked labels excluded |
one block per PyPSA name, equal count — a split counts as a difference |
| size | diagnostics() rows, columns, nonzeros |
n.model.solver_model rows, columns, nonzeros |
the model handed to HiGHS is the same size on both sides |
| duals | result.dual(block) |
n.model.constraints[name].dual |
every row's dual equal, absolute 1e-6 — against the negative where the file writes the row negated; an integer model has none |
| linopy lane | the test oracle, tests/linopy_lane |
n.optimize.create_model() |
label for label: coefficients, sense, right-hand side, bounds, integrality, objective terms |
Both sides solve one object, the network the rung's script builds — PyPSA directly, specsolve through the file attached to the tables prep.py makes of it. A difference in structure, duals or the linopy lane is allowed only with a reason in differential/pypsa/deviations.yaml; the runner fails on one recorded nowhere and on a reason no rung needs. A rung the linopy lane cannot build yet names the blocker instead. Not compared: primals (an optimum need not be unique).
One kind of reason is checked rather than excused. Where the file states a row as PyPSA writes it negated — a storage balance with the charge on the left, a ramp written the other way about — the dual is the negative of PyPSA's, exactly, so the runner compares it against the negative at the same tolerance and the claim is under test. Those names are marked negated below; a difference surviving the negation is red.
Recorded deviations¶
| PyPSA name (comparison) | why specsolve differs | on rungs |
|---|---|---|
Generator-com-status-min_up_time_must_stay_up (duals) |
degenerate with PyPSA's duplicated cap — a unit held on has status pinned at 1 by this row from below and by the variable bound and the cap row from above, and HiGHS may put the dual on any of them | Rung 12 — linearized unit commitment |
Generator-ext-p_nom-lower (linopy lane) |
PyPSA keys the build's floor by scenario as well as by generator, one copy of the row per future; the file's one row has no scenario to key by | Rung 14 — two futures and a risk preference |
Generator-ext-p_nom-lower (structure) |
PyPSA writes the build's floor once per scenario, each row over the one capacity variable; the file states it once — capacity is chosen before the future is known, and a copy per future is the same row again. | Rung 14 — two futures and a risk preference |
Generator-ext-p_nom-upper (duals) |
two copies of one binding row are degenerate — the solver may put the cap's whole price on either copy, so PyPSA's per-scenario duals are shares of the file's one. | Rung 14 — two futures and a risk preference |
Generator-ext-p_nom-upper (linopy lane) |
PyPSA keys the build's cap by scenario as well as by generator, one copy of the row per future; the file's one row has no scenario to key by | Rung 14 — two futures and a risk preference |
Generator-ext-p_nom-upper (structure) |
PyPSA writes the build's cap once per scenario, each row over the one capacity variable; the file states it once — capacity is chosen before the future is known, and a copy per future is the same row again. | Rung 14 — two futures and a risk preference |
Generator-p-ramp_limit_down (duals, negated) |
the initial-snapshot block states −p on the left where PyPSA writes p[t] − p[t−1] ≥ −rd·p_nom — the same row negated, so its dual is the negative | Rung 3 — expansion, Rung 4 — ramps, Rung 12 — linearized unit commitment |
Generator-shut_down (linopy lane) |
the cap is a row here, where PyPSA bounds the variable at 1 as well — a 0-1 binary, or a share in [0, 1] under the linearized relaxation | Rung 7 — commitment, Rung 12 — linearized unit commitment |
Generator-start_up (linopy lane) |
the cap is a row here, where PyPSA bounds the variable at 1 as well — a 0-1 binary, or a share in [0, 1] under the linearized relaxation | Rung 7 — commitment, Rung 12 — linearized unit commitment |
Generator-status (linopy lane) |
the cap is a row here, where PyPSA bounds the variable at 1 as well — a 0-1 binary, or a share in [0, 1] under the linearized relaxation | Rung 7 — commitment, Rung 12 — linearized unit commitment |
Generator-status-p-fixed-upper (duals) |
PyPSA bounds the status at 1 on the variable and writes the cap row as well, so a binding cap's dual may sit on the bound and leave the row at zero; the file states the row alone (rung 12, the linearized relaxation) | Rung 12 — linearized unit commitment |
Link-p-ramp_limit_down (duals, negated) |
the file states the ramp as p[t-1] - p[t] <= rd * p_nom where PyPSA writes p[t] - p[t-1] >= -rd * p_nom — the same row negated, so its dual is the negative | Rung 3 — expansion, Rung 4 — ramps |
StorageUnit-energy_balance (duals, negated) |
the file states the balance with the charge on the left; PyPSA writes −soc there, the same row negated, so its dual is the negative | Rung 2 — storage, Rung 3 — expansion, Rung 5 — global constraints |
Store-energy_balance (duals, negated) |
the file states the balance with the level on the left; PyPSA writes −e there, the same row negated, so its dual is the negative | Rung 2 — storage, Rung 3 — expansion, Rung 5 — global constraints |
objective (linopy lane) |
PyPSA's objective carries its fixed objective_constant variable as a term; the file states no constant, and the objective value is compared net of it | Rung 11 — PyPSA's own ac_dc_meshed example, whole, Rung 13 — transmission losses in tangent form |
objective_constant (structure) |
PyPSA carries a nonzero objective constant as a fixed variable of that name; the file states no constant, and the objective is compared net of it | Rung 11 — PyPSA's own ac_dc_meshed example, whole, Rung 13 — transmission losses in tangent form |
operational_limit (structure) |
one block per sense — ==, <=, >= — where PyPSA writes one row per labelled constraint whatever its sense | Rung 5 — global constraints |
primary_energy (structure) |
one block per sense — ==, <=, >= — where PyPSA writes one row per labelled constraint whatever its sense | Rung 5 — global constraints |
solver model (structure) |
what the file states otherwise than PyPSA writes it — the objective_constant variable's column and row (rungs 11, 13), the build's floor and cap once rather than once per scenario (rung 14) | Rung 11 — PyPSA's own ac_dc_meshed example, whole, Rung 13 — transmission losses in tangent form, Rung 14 — two futures and a risk preference |
tech_capacity_expansion_limit (structure) |
one block per sense — ==, <=, >= — where PyPSA writes one row per labelled constraint whatever its sense | Rung 3 — expansion |
transmission_expansion_cost_limit (structure) |
one block per sense — ==, <=, >= — where PyPSA writes one row per labelled constraint whatever its sense | Rung 3 — expansion, Rung 6 — KVL |
transmission_volume_expansion_limit (structure) |
one block per sense — ==, <=, >= — where PyPSA writes one row per labelled constraint whatever its sense | Rung 6 — KVL |
Not compared, deliberately: primals — an optimum need not be unique. Counted rather than compared: the rows built per block, on each rung's page, and over the whole ladder that every block is built by some rung, every mask is partially true somewhere and every parameter is fed somewhere — the runner fails on a gap.
Each rung's own model is the file projected onto what the rung builds; the runner solves the projection too and holds it to the full file's objective (relative 1e-9), so a cut that lost a term is a red run rather than a shorter page.