fix: use (path, segment_code) instead of id_path as diff JOIN key

[hf-mrdachner]

Summary

• Replace id_path with (path, segment_code) as the JOIN key in v_ahb_diff
• id_path contains positional indices that shift on insertions, producing false add/delete pairs
• (path, segment_code) is unique per pruefidentifikator+version and stable across versions

Impact

Example: UTILMD/44001 FV2510→FV2604

	Before (id_path)	After (path, segment_code)
added	69	5
deleted	64	0
modified	27	1
unchanged	122	212

Test plan

• Run existing test_ahb_diff_view.py tests — 7/7 passed
• Verify UTILMD/44001 FV2510→FV2604 produces 5 added, 0 deleted, 1 modified
• Verify INVOIC/31002 FV2510→FV2604 still produces 1 modified
• Check (path, segment_code) uniqueness — unique for 97% of (version, pruefi) combinations

Known limitations

• IFTSTA (21045): path is not unique enough due to flat segment group naming — tracked in IFTSTA: path column not unique enough for stable diff JOIN key #258
• PARTIN/MSCONS/PRICAT: A few benign cnt=2 duplicates from repeated identical elements (e.g. two "Name des Kontoinhabers" fields) — diff result still correct because content is identical

Fixes #256

🤖 Generated with Claude Code

[hf-kklein]

I understand the problem and commented on #256 in a longer comment.

Specifically for this PR: I'd not merge it like that because we loose the only purpose that id-path has: to be unique. Instead of a single column that is unique we introduce a mechanism that requires the users of the DB to think and interpret themselves. The intention of the pre-computed ID path was to obey the "don't make me think" principle. The users of the DB shouldn't solve the uniqueness problem on their own but build upon guarantees that we as creators of the DB give and enforce.

Can we keep the id-path as a SINGLE column that is unique per ahb-line (together with formatversion and prüfi)? I like that it's a SINGLE column, not a tuple of columns. But this also means you have to dig deeper into the code and parts of the code I'm not proud of ;)

[hf-kklein]

this is why I wouldn't merge the PR as of now: if the id is not unique anymore we can drop it right away. that's the only (and a very important) purpose of the id_path column.

the id path MUST stay unique

[hf-kklein]

removing the guarantees that id-path gives right now would also mean to rewrite parts of the ahb-tabellen codebase. ahb-tabellen is agnostic on how idpath looks like but it needs the guarantee that it's unique (and that sort_path has the correct order).

[hf-kklein]

ok, noch ein letzter kommentar: ich glaube es ist besser das "id-path ist falsch konstruiert und nicht für cross-formatversions-vergleiche zu gebrauchen" zu lösen als den vergleich symptombekämpfend anzupassen. verstehst du, was ich meine? auch wenn das schwieriger ist, wäre es glaube ich der ursächliche fix. das problem versteh ich und das ist vollkommen richtig in #256 zusammengefasst. aber die lösung sollte imho auch direkt bei der konstruktion vom id-path ansetzen

[hf-kklein]

Hi @hf-mrdachner! An alternative fix is now available in #260 that addresses the root cause — fixing id_path construction itself in materialize_ahb_view.sql — rather than changing the diff view JOIN key.

The approach in #260 injects semantic qualifiers (like DTM+92) directly into the id_path during the recursive CTE, keeping it as a single unique column. This preserves all the downstream guarantees that @hf-kklein mentioned in his review.

Your bug report in #256 and the edge case analysis in #258 and #259 were essential foundation work for this fix — the IFTSTA, PARTIN, MSCONS, and PRICAT cases you identified are now all tested explicitly. Please keep filing issues like these; they make the codebase better! 🙏

Would you be open to reviewing #260? Your deep understanding of the data would be very valuable for the review.

[hf-mrdachner]

Closing in favor of #260 which fixes the root cause (id_path construction) instead of working around it in the view JOIN. The remaining MSCONS cross-version instability is a data quality issue tracked in xml-migs-and-ahbs#80.