Implement #254: preserve TOML table syntax on round-trip#670
Implement #254: preserve TOML table syntax on round-trip#670rajulbhatnagar wants to merge 3 commits into
Conversation
|
Very intrusive changes for what is frankly a minor issue |
|
Definitely work needs to be split into smaller prs, 900+ file pr is a no-go. |
|
This topic has my interest. As stated in the description commit 1 is the compatibility test, which should be removed from this PR. (it makes more sense when there's a separate project that runs this testset again all toml parser, so it can make comparison statistics) |
|
@rfscholte There's now #678 which I think adds compatibility testing suite (once fixes are in). Or if not, separate PR for testing part would make sense. |
|
Compliancy test added for CI; compliancy deviations fixed. This PR needs merging from |
|
Ya makes sense for the compatibility suite to run as part of CI. Will remove that commit and rebase. Also happy to break this into more than 1 PR if you folks prefer |
Parsing a TOML document with [section] headers or [[array of tables]] and writing the resulting tree back now preserves those forms instead of collapsing to dotted keys. - Origin-flag node subclasses (TomlObjectNode, TomlArrayNode) carry fromTableHeader / fromArrayOfTables flags set by the parser. TomlNodeFactory produces them so createObjectNode and deepCopy round-trip the metadata. TomlMapper._readTreeAndClose short-circuits the deserializer reconstruction when the parser is one we created (a TomlTreeTraversingParser marker), preserving flags that the standard JsonNodeDeserializer path would have dropped. The short-circuit is disabled when the user has overridden the node factory, so a user-configured factory is honored. - Strict-ordering enforcement: TOML rejects scalars after a sub-table at the same level. TomlObjectNode.serialize reorders children up front, deferring any branch that contains a table-flagged descendant. TomlGenerator also enforces the rule on the explicit-API path: scalars after a writeStartTable() at any ancestor scope (not just the immediate context) throw TomlStreamWriteException. - Explicit generator API: writeStartTable() and writeStartArrayOfTables() emit [path] / [[path]] directly without requiring a parsed origin tree. - BigDecimal trailing-zero handling honors JsonNodeFeature.STRIP_TRAILING_BIGDECIMAL_ZEROES on the short-circuit path so the override matches what JsonNodeDeserializer would have applied. Backward compatibility: trees built via createObjectNode + scalar children only (no parsed origin) still serialize as dotted keys, so existing users on upgrade paths see no behavior change.
a6cfbb8 to
a8593bb
Compare
Summary
Resolves #254. Parsing a TOML document with
[section]headers or[[array of tables]]and writing the resulting tree back now preservesthose forms instead of collapsing to dotted keys.
How it works
Origin-flag node subclasses + factory. Package-private
TomlObjectNode/TomlArrayNodecarryfromTableHeader/fromArrayOfTablesflags set by the parser.TomlNodeFactoryproduces them so
mapper.createObjectNode()anddeepCopy()round-trip the metadata.
TomlMapper._readTreeAndCloseshort-circuits the deserializer reconstruction when the parser is
one we created (a
TomlTreeTraversingParsermarker), preservingflags that the standard
JsonNodeDeserializerpath would havedropped. The short-circuit is disabled when the user has overridden
the node factory, so a user-configured factory is honored (at the
cost of flag preservation, the explicit opt-out trade-off).
Strict-ordering enforcement. TOML rejects scalars after a
sub-table at the same level.
TomlObjectNode.serializereorderschildren up front, deferring any branch that contains a table-
flagged descendant.
TomlGeneratoralso enforces the rule on theexplicit-API path: scalars after
writeStartTable()at anyancestor scope (not just the immediate context) throw
TomlStreamWriteException.Explicit generator API. New public methods on
TomlGenerator:writeStartTable(),writeStartArrayOfTables(). Callers buildingTOML from scratch can emit
[path]and[[path]]directly withoutparsing first.
BigDecimaltrailing-zero handling. HonorsJsonNodeFeature.STRIP_TRAILING_BIGDECIMAL_ZEROESon theshort-circuit path so the override matches what
JsonNodeDeserializerwould have applied.Backward compatibility
Trees built via
mapper.createObjectNode()with only scalar children(no parsed
[section]origin) still serialize as dotted keys. Verifiedby
unflaggedNestedObjectStillUsesDottedKeys. Existing users onupgrade paths see no behavior change.
This branch is rebased on the current
3.xHEAD, so it composes withthe recently-merged #669 fix (dotted key cannot extend an explicitly
defined table). The
explicitlyDefinedflag now lives on the promotedTomlObjectNode; two tests(
dottedKeyExtendingExplicitTableStillRejected,explicitTableThenExplicitSubtableRoundTrips) confirm the two featurescoexist.
Test plan
./mvnw -B -ntp -pl toml test— all green (TomlRoundTripTest: 33cases; full module: 195 run, 2 skipped — the pre-existing
@Disabledcompliance stubs, untouched by this PR)
over
[table]/[[aot]], explicit-API emission, strict-orderingparameterized over 5 illegal-write cases, ancestor-scope scalar
lockout, inline-scope fallthrough, custom-factory honor, BigDecimal
feature on/off,
TreeTraversingParsersource-exposure contract,end-to-end fixture, TOML parser: invalid dotted-key extension of explicit tables / AOT not rejected #669 interaction
Notes for reviewers
TomlTreeTraversingParseris a no-op marker subclass used onlyto discriminate parsers we created from caller-supplied
TreeTraversingParserinstances in_readTreeAndClose. Caller-supplied parsers go through the standard deserializer path so their
source tree is not mutated (verified by
externalTreeTraversingParserDoesNotMutateSource)._scalarsClosedpropagation walks ancestors up to the nearestenclosing
TABLE/ARRAY_OF_TABLES, fixing a subtle issue wherea
writeStartTable()invoked from inside aDOTTED_OBJECTwouldnot lock the root scope, allowing later root-level scalars to
serialize inside the just-emitted section. Regression test:
scalarAtAncestorAfterNestedTableThrows.several of the tracked parser bugs is intentionally not in this
PR; it will be submitted separately.