Translator timeslice implementation notes (decisions from staging PR #121)

[nick-gorman]

Context

PR #117 (new-format-translator) is the omnibus/baseline implementation of the new-format pipeline, including the timeslice work. It is being brought onto main as a series of small, separately-reviewable PRs rather than merged wholesale.

PR #121 stages the templater half (decoding AEMO's PLEXOS calendar into one window pattern per reference year). While compiling it, we made/validated several decisions that belong to the translator half and so land in later staging PRs. This issue records them so they aren't lost between PRs.

Established along the way: within each region and reference year the shipped v7.5 windows form an exact partition of the year (every day covered by exactly one timeslice — verified across all 75 region×reference-year groups). winter_reference is already the contiguous complement of peak+summer (a fixed 04-01 → 11-01 block for every reference year), with the weather variation carried by the peak/summer windows. The cycle boundary is 1 November; consecutive reference years abut there. The downstream "gap fallback" is therefore defensive code the real data never exercises. The items below lean on that structure holding.

Enforcement, as of #121

• Tiling is enforced inline: _template_timeslices calls _raise_unless_windows_tile_the_year before returning, so a future calendar that fails to tile fails at template time. The check is boundary-based (sorted by start, each window must end where the next begins, the last wrapping to the first) — no day counting, leap-safe.
• Two structural assumptions underpin the reference-year attribution, and are checked by standalone guards, not inline: _raise_unless_only_winter_crosses_financial_year (only winter may span 1 July, so weather-varying timeslices stay within one reference year) and _raise_unless_winter_is_constant_per_region (the one boundary-crossing timeslice is identical across reference years, so its attribution can't blur). These run on the real calendar inside test_shipped_calendar_decodes, so they're enforced per workbook version rather than on every decode — deliberately, to keep the focused decoder tests on minimal data.
• The timeslices schema custom_validation rules (no-overlap, full-coverage) remain inert until the schema-validation layer is built; the guards above are the active enforcement until then.

To action in later 117 staging PRs

1. Re-check coverage/non-overlap after pattern expansion. The templater's check is leap-safe (it verifies the windows' month-day boundaries chain, without counting days), and the templater deliberately preserves 02-29 (clamping is the translator's job — covered by test_convert_windows_to_month_days_preserves_leap_day). The translator re-expands patterns into concrete model years where a 02-29 boundary must be clamped to 02-28 in non-leap years. This is concrete, not hypothetical: reference year 2024 has a 29 February hot day (NSW and VIC) in the shipped calendar, so the clamp fires whenever the configured cycle places 2024 in a non-leap model year. Re-assert the partition on the expanded snapshot mapping, where that clamp could open a one-day seam.

2. Add a round-trip test: decode → re-expand → original calendar. Decoding timeslice_RefYear5000.csv to per-reference-year patterns and re-expanding with the original reference_year_sequence reproduces the kept windows exactly (verified on v7.5: 1460/1460 windows match, start and end). This is the strongest correctness check on the decode — it catches mis-paired windows, reference-year mis-attribution, and any non-invertible decode, none of which the consistency guard catches (AEMO's cycle order is fixed, so a same-reference-year contamination would be identical across occurrences and pass consistency silently). It belongs in the translator PR because it needs the translator's _place_pattern_in_financial_year — run it against the real re-expansion, not a test-local copy.

3. Use max per-direction capacity as link p_nom, not winter_reference. Removes the str.endswith(\"_winter_reference\") special-case in _extract_static_limits and keeps p_max_pu <= 1. Verify winter is the max for every path: if so this is a no-op rename; if not, the current code already emits p_max_pu > 1 for that path (a latent over-crediting bug this fixes).

4. Remove the now-dead gap fallback in _build_link_pu_overrides. Once coverage is enforced and guaranteed, every snapshot is set by exactly one timeslice — drop the static-default seed (the 1.0 / static p_min_pu init) and instead assert the series is fully covered. Constraints need no change: the partition makes the additive "one instance per timeslice, scoped to its snapshots" model exactly-once-per-snapshot already.

5. (Optional) Forbid mixing static (NaN) and timeslice-tagged rows for one entity in the network_transmission_path_limits and custom_constraints_rhs schemas. Each entity is either fully static or fully timeslice-resolved — never a partial mix.

[nick-gorman]

PR #126 (Translate the new-format network tables to buses and links) implements item 3.

_extract_static_limits — which selected the static p_nom by matching str.endswith("_winter_reference") — is replaced by _extract_max_capacities, which takes the maximum capacity per (path_id, direction) across timeslices. The forward maximum becomes p_nom, so every forward per-timeslice limit lands at or below 1 (p_max_pu <= 1) by construction, and the winter_reference special-case is gone.

On the shipped v7.5 data this is the no-op rename the issue anticipated: winter_reference is the max forward limit on every path, so p_nom is unchanged. The new test_translate_network_to_links_uses_max_capacity_as_p_nom covers the divergent case — a path whose peak forward limit exceeds winter — and pins p_max_pu = 1.0 on the peak condition, where the old code would have emitted 1500/1400 = 1.07 and over-credited the link.

Items 1, 2, 4 and 5 remain for their respective staging PRs (timeslice decode/expand, pypsa_build's _build_link_pu_overrides gap-fallback removal, and the schema custom_validation rules). Item 4 in particular is the natural follow-on once pypsa_build consumes the link_timeslice_limits table this PR produces.

[nick-gorman]

Decided — snapshot-coverage invariant for network_transmission_path_limits (sharpens items 3, 4 and optional 5).

For each (path/rez, direction) the per-snapshot limit must be completely defined across the model year. Completeness is satisfied either way:

• By tiling — a set of named timeslices whose windows partition the year (e.g. peak + summer + winter_reference, winter being the contiguous complement). No fallback row is needed; the link's static attribute is never consulted.
• By a timeslice = NaN fallback row — its capacity applies to every snapshot no named timeslice covers. This is also how a fully-static path (no named timeslices) is expressed.

Precedence per snapshot: named timeslice → timeslice = NaN fallback → system default_limit (the last applies when even the NaN row's capacity is itself NaN, i.e. the path has no data at all — today's collapsed all-NaN row). Holds independently for forward and reverse.

This refines optional item 5: a path may mix named-timeslice rows with a NaN fallback row — that's the point — what's forbidden is leaving a snapshot uncovered in either direction. It mirrors the NaN-timeslice-as-fallback convention already introduced for custom_constraints_rhs (commit 5294cc8), so the two tables read the same way.

Consequence for the links table (item 3): because coverage is now guaranteed, p_nom = max(forward, reverse) is safe — the link's static p_max_pu is never reached on an uncovered snapshot — so every per-snapshot limit can be expressed per unit of p_nom in [-1, 1]. p_nom is taken as the max over all rows including the timeslice = NaN fallback, since at translate time we don't yet know which snapshots the fallback will cover. This is the form PR #126 implements; the matching _build_link_pu_overrides simplification (item 4) then becomes "never build the gap fallback" rather than "remove it later", since pypsa_build is still unwritten.