Skip to content

fix: align pilota parsing with standard grammar - #370

Merged
Joshuahoky merged 1 commit into
mainfrom
feat/pilota-parser-grammar-misalignment
Jul 29, 2026
Merged

fix: align pilota parsing with standard grammar#370
Joshuahoky merged 1 commit into
mainfrom
feat/pilota-parser-grammar-misalignment

Conversation

@Joshuahoky

Copy link
Copy Markdown
Contributor

Motivation

Current parser behaviour deviates from the specs Thrift IDL grammar
in numerous places, causing the parser to accept inputs that should be rejected or reject inputs that should be valid. Some of the deviations include:

# Example Original Expected
1 struct Demo { 40000: required i32 value } accepted — codegen emits field id -25536 (40000 as i16) in encode/decode reject — field ids must fit in i16
2 structFoo { i: i32 a} accepted — parsed as struct Foo { … } (keyword glued to ident), same behaviour for unionFoo, enumFoo... reject — no identifier boundary
3 const i32 A = --1 accepted — double negation allowed reject — grammar allows one optional sign
4 struct S { 1: map<i32; string> a } accepted — ; treated as separator reject — map key/value separator is , only
5 namespace @@!! foo.bar accepted — @@!! treated as valid namespace reject — scope is * or an identifier
6 const i32 A = +1 rejected accept — IntConstant ::= ('+'|'-')? Digit+
7 const double A = .5 rejected accept — integer part is optional (Digit*)

Solution

  • Add Components::keyword() (a just() followed by a non-consuming word boundary that also succeeds at end of input) and use it for every reserved word
  • Rewrite IntConstant as ('+' | '-')? (0x HexDigit+ | Digit+) with the sign kept inside the matched slice, and convert via try_map, so out-of-range values become parse errors rather than panics and i64::MIN round-trips.
  • DoubleConstant's integer part becomes Digit* for the .5 form, with the exponent-only form still requiring at least one digit.
  • Field ids now go through IntConstant and are range-checked into i16 with a try_map, so negative ids work and oversized ids are a diagnostic instead of a silent truncation.
  • Scope becomes '*' | Identifier (dots allowed, since py.twisted is in real-world use), which also makes a missing scope an error.
  • The map parser uses just(",") instead of list_separator.

@codecov

codecov Bot commented Jul 27, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 99.37888% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 68.77%. Comparing base (d8c1fef) to head (b0c92bd).

Files with missing lines Patch % Lines
pilota-thrift-parser/src/parser/constant.rs 98.74% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #370      +/-   ##
==========================================
+ Coverage   68.46%   68.77%   +0.31%     
==========================================
  Files          89       89              
  Lines       25924    26190     +266     
==========================================
+ Hits        17748    18012     +264     
- Misses       8176     8178       +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread pilota-thrift-parser/src/parser/constant.rs Outdated
shenyj3
shenyj3 previously approved these changes Jul 29, 2026
@Joshuahoky
Joshuahoky force-pushed the feat/pilota-parser-grammar-misalignment branch from 854782f to b0c92bd Compare July 29, 2026 08:32
@Joshuahoky
Joshuahoky merged commit ffd44e9 into main Jul 29, 2026
15 checks passed
@Joshuahoky
Joshuahoky deleted the feat/pilota-parser-grammar-misalignment branch July 29, 2026 08:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

2 participants