Skip to content

Fix obscure symbol reassociation bug - #457

Merged
cirras merged 2 commits into
masterfrom
token-indices-are-not-unique-identifiers-and-my-life-is-a-lie
Sep 2, 2026
Merged

Fix obscure symbol reassociation bug#457
cirras merged 2 commits into
masterfrom
token-indices-are-not-unique-identifiers-and-my-life-is-a-lie

Conversation

@cirras

@cirras cirras commented Aug 31, 2026

Copy link
Copy Markdown
Collaborator

This PR fixes an obscure bug related to the 2-pass semantic analysis we're doing where we build the symbol table in one pass, throw away ASTs as we go, and re-parse source files (reassociating symbol information) in a second pass.

For symbol reassociation purposes, we were registering declarations, occurences, and scopes against token indices as if they were unique at the node level. They're not.

This led to some tricky situations where scopes in particular would be incorrectly attributed to the wrong nodes. This bug was masked very well because most name resolution occurs during symbol table construction.

The problem came about after symbol reassociation. In the analysis phase, on-demand name resolution occurs during expression type resolution. Any analysis rule calling ExpressionNode::getType was potentially impacted.

@cirras
cirras requested a review from fourls August 31, 2026 03:36
@cirras cirras changed the title Token indices are not unique identifiers and my life is a lie Fix obscure symbol reassociation bug Aug 31, 2026
@cirras
cirras force-pushed the token-indices-are-not-unique-identifiers-and-my-life-is-a-lie branch from d0b5db0 to 5252de0 Compare August 31, 2026 06:39
@cirras

cirras commented Aug 31, 2026

Copy link
Copy Markdown
Collaborator Author

Fixed the failing tests and added a new pile of unit tests covering the SymbolAssociationVisitor.

Tests were failing because we were actually relying on this awful "multiple nodes return the same token index behavior" in several ways. Most notably, the registered location for every name occurrence is an IdentifierNode, but we were reattaching via the containing NameReferenceNode which had the same token index.

fourls
fourls previously approved these changes Sep 1, 2026

@fourls fourls left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this looks good!
As far as I can see we don't rely on this shared-index behaviour anywhere else.
The new SymbolAssociationVisitor tests are really good to have.

We were registering declarations, occurences, and scopes against token
indices as if they were unique at the node level. They're not.

This led to some tricky situations where scopes in particular would be
incorrectly attributed to the wrong nodes. This bug was masked very well
because most name resolution occurs during symbol table construction.

The problem came about after symbol reassociation. In the analysis
phase, on-demand name resolution occurs during expression type
resolution. Any analysis rule calling `ExpressionNode::getType` was
potentially impacted.
@cirras
cirras force-pushed the token-indices-are-not-unique-identifiers-and-my-life-is-a-lie branch from 5252de0 to 7bd6758 Compare September 2, 2026 00:50
@cirras

cirras commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator Author

Realized I did the classic "wrong year in license header" mistake with the new file.
@fourls, can you reapprove?

@cirras
cirras requested a review from fourls September 2, 2026 00:51

@fourls fourls left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't notice, good catch 😆

@cirras
cirras merged commit 08adf3d into master Sep 2, 2026
4 checks passed
@cirras
cirras deleted the token-indices-are-not-unique-identifiers-and-my-life-is-a-lie branch September 2, 2026 01:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants