refactor(syntax): generalize the macro repetition separator node - #10310
Conversation
ed9467d to
86cbcf4
Compare
931afdd to
be641de
Compare
86cbcf4 to
03aed4c
Compare
be641de to
e0049dd
Compare
03aed4c to
524388b
Compare
PR SummaryLow Risk Overview Parser partial-tree goldens pick up the extra wrapper level; observable macro behavior and diagnostics are intended to stay the same. Reviewed by Cursor Bugbot for commit 51dcc85. Bugbot is set up for automated code reviews on this repo. Configure here. |
e0049dd to
df241a0
Compare
524388b to
062a85b
Compare
df241a0 to
e229982
Compare
8b2a8e1 to
d781f92
Compare
e229982 to
6a96549
Compare
6a96549 to
adb52a4
Compare
d781f92 to
9fd1f92
Compare
9fd1f92 to
648a842
Compare
adb52a4 to
8c0b736
Compare
eytan-starkware
left a comment
There was a problem hiding this comment.
@eytan-starkware+AGNT made 2 comments.
Reviewable status: 0 of 8 files reviewed, 1 unresolved discussion (waiting on eytan-starkware, orizi, and TomerStarkware).
a discussion (no related file):
Note: the comments below are from an automatic orizi-review run (Claude agents reviewing in Ori's style, findings adversarially verified before posting). Treat with the usual bot skepticism.
crates/cairo-lang-syntax-codegen/src/cairo_spec.rs line 911 at r1 (raw file):
) .add_option("MacroRepetitionSeparator") .add_option("TerminalComma")
after this change OptionTerminalComma exists only for TokenTreeRepetition (l.825). move it up next to that node so it's obvious it dies together with the TODO(Dean): Remove TokenTreeRepetition.
`MacroRepetition.separator` was `OptionTerminalComma`, baking the only separator
the parser currently accepts into the grammar. Replace it with
`OptionMacroRepetitionSeparator`, wrapping a `MacroRepetitionSeparator` struct
whose single `token` child is a `TokenNode` — the codegen's all-terminals enum —
so the grammar can hold any terminal as a separator.
An option must wrap a node that has a `SyntaxKind`, and enums do not get one
(`generate_kinds_code` filters `NodeKind::Enum` out), hence the small wrapper
struct — the same reason `TokenTreeLeaf` wraps `TokenNode`. The syntax crate is
regenerated by `generate-syntax`, not hand-edited.
Behavior is unchanged. The parser still accepts only `TerminalComma` in that
slot, and the semantic matcher and expansion still treat only a comma as a
separator, now via a single `repetition_separator` helper that returns `None`
for any other token — the same branch as "no separator" — rather than
panicking, so loosening the parser later cannot introduce a panic.
`$($x:ident);*` is still rejected with the same `Missing macro repetition
operator` diagnostic. The only golden churn is in the parser partial_trees:
`OptionTerminalCommaEmpty` becomes `OptionMacroRepetitionSeparatorEmpty`, and
existing comma separators gain the `MacroRepetitionSeparator` level. No semantic
golden changed.
`TokenTreeRepetition.separator` is deliberately left as `OptionTerminalComma`.
The adjacent `TODO(Dean): Remove TokenTreeRepetition and TokenTreeParam (it's
redundant)` intends to delete that node outright, so migrating it would be churn
on code slated for removal and would couple the call-site token-tree grammar to
the macro-rule grammar that TODO wants to disentangle. That refactor is
explicitly not folded in here; `.add_option("TerminalComma")` stays for it and
becomes dead only once it lands.
The `unreachable!()` on `ast::MacroParamKind::Missing` in `macro_declaration.rs`
is not made reachable by this change, so no guard was added. The accepted token
language is identical before and after, so no new tree shape is produced, and
the separator slot is structurally disjoint from `ParamKind`. Checked
empirically that it also stays unreachable in general: `macro m { ($x:foo) => {
1 }; }` reports `Missing tokens. Expected a macro rule parameter kind.` and the
rule is then dropped by the `descendants(..).any(is_missing)` filter in
`priv_macro_declaration_data`, so the matcher never converts a missing param
kind. That filter is unaffected here — `OptionMacroRepetitionSeparatorEmpty` is
a plain struct kind, not a missing kind, exactly like the
`OptionTerminalCommaEmpty` it replaces.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
648a842 to
d42096c
Compare
8c0b736 to
51dcc85
Compare

MacroRepetition.separatorwasOptionTerminalComma, baking the only separatorthe parser currently accepts into the grammar. Replace it with
OptionMacroRepetitionSeparator, wrapping aMacroRepetitionSeparatorstructwhose single
tokenchild is aTokenNode— the codegen's all-terminals enum —so the grammar can hold any terminal as a separator.
An option must wrap a node that has a
SyntaxKind, and enums do not get one(
generate_kinds_codefiltersNodeKind::Enumout), hence the small wrapperstruct — the same reason
TokenTreeLeafwrapsTokenNode. The syntax crate isregenerated by
generate-syntax, not hand-edited.Behavior is unchanged. The parser still accepts only
TerminalCommain thatslot, and the semantic matcher and expansion still treat only a comma as a
separator, now via a single
repetition_separatorhelper that returnsNonefor any other token — the same branch as "no separator" — rather than
panicking, so loosening the parser later cannot introduce a panic.
$($x:ident);*is still rejected with the sameMissing macro repetition operatordiagnostic. The only golden churn is in the parser partial_trees:OptionTerminalCommaEmptybecomesOptionMacroRepetitionSeparatorEmpty, andexisting comma separators gain the
MacroRepetitionSeparatorlevel. No semanticgolden changed.
TokenTreeRepetition.separatoris deliberately left asOptionTerminalComma.The adjacent
TODO(Dean): Remove TokenTreeRepetition and TokenTreeParam (it's redundant)intends to delete that node outright, so migrating it would be churnon code slated for removal and would couple the call-site token-tree grammar to
the macro-rule grammar that TODO wants to disentangle. That refactor is
explicitly not folded in here;
.add_option("TerminalComma")stays for it andbecomes dead only once it lands.
The
unreachable!()onast::MacroParamKind::Missinginmacro_declaration.rsis not made reachable by this change, so no guard was added. The accepted token
language is identical before and after, so no new tree shape is produced, and
the separator slot is structurally disjoint from
ParamKind. Checkedempirically that it also stays unreachable in general:
macro m { ($x:foo) => { 1 }; }reportsMissing tokens. Expected a macro rule parameter kind.and therule is then dropped by the
descendants(..).any(is_missing)filter inpriv_macro_declaration_data, so the matcher never converts a missing paramkind. That filter is unaffected here —
OptionMacroRepetitionSeparatorEmptyisa plain struct kind, not a missing kind, exactly like the
OptionTerminalCommaEmptyit replaces.Co-Authored-By: Claude Fable 5 noreply@anthropic.com