Materialize singleton classes for namespace declarations

[soutaro]

This PR materializes all singleton class declarations for class and module namespaces.

This is required for descendants enumeration and for find_member_in_ancestors correctness. Without these declarations, a subclass singleton class such as Child::<Child> may not exist when Child is only defined as class Child < Parent, so inherited singleton ancestors and members cannot be found.

Impacted Ruby methods

• Namespace#descendants now includes materialized singleton class declarations.
• Namespace#find_member now returns members inherited through singleton class ancestors.

Other user-facing changes

• Declaration search and completion results exclude materialized singleton class declarations, so newly materialized declarations do not appear as extra workspace symbols.

Performance impact

Because this materializes singleton classes for every namespace declaration, it can add memory use and runtime cost. In practice, the measured impact was small.

• RSS: no measurable increase in synthetic corpus benchmarks or a large internal Ruby codebase
• Runtime: about 1.8% slower on a large internal Ruby codebase; not measurable in synthetic corpus benchmarks

Validation

• Confirmed the superclass-extend descendants regression fails on origin/main: Extension.descendants lacks Child::<Child>
• Confirmed the singleton find_member regression fails on origin/main: Child::<Child> returns DeclarationNotFound
• shadowenv exec -- cargo test -p rubydex find_member_in_ancestors_returns_inherited_singleton_member
• shadowenv exec -- bundle exec ruby -Itest test/declaration_test.rb -n /test_find_member_returns_inherited_members_on_singleton_class/
• shadowenv exec -- cargo test -p rubydex exact_match_empty_query_returns_all
• shadowenv exec -- cargo test -p rubydex
• shadowenv exec -- bundle exec rake ruby_test
• git diff --check HEAD~2..HEAD

[vinistock]

If we make the move to always create singleton classes eagerly, then I think the right layer for the fix would to create a singleton class whenever the class gets created. It may require studying the impact for lazily creating singleton class of singleton classes (e.g.: Foo::<Foo>::<<Foo>>).

Also, I wonder if it would still be worth trying to minimize the amount of work. So far, if no operation in the code reaches into the singleton class, we've been considering that it doesn't exist in our modelling (lazy). Maybe we should only create singleton classes for the descendants of existing singleton classes (instead of all of them).

[soutaro]

@vinistock

think the right layer for the fix would to create a singleton class whenever the class gets created.

It makes sense. Will fix the PR to do so.

And we still need to keep the lazy path for meta-singleton classes anyway. Ancestor/descendant enumeration will remain broken for meta-singleton classes, but that seems acceptable.

if no operation in the code reaches into the singleton class, we've been considering that it doesn't exist in our modelling

It would be possible, but it would require a significant re-architecture of the ancestor/descendant handling. I think we should keep the eager materialization strategy for this issue for now.

[vinistock]

When you say meta singleton classes, do you mean multiple levels of singleton classes (e.g.: Foo::<Foo>::<<Foo>>)?

If this eager creation of singleton classes doesn't handle that, then it's not quite fitting the algorithm at the right stop and we may need to invest a bit more time thinking about how to address the problem.

Maybe when a singleton class is created, we need to trigger automatic creation of singleton classes for all of the descendants of the attached object, keeping the mechanism lazy, but ensuring that it works for any depth of singleton.

[soutaro]

Oops, I mixed up the similar concepts in Smalltalk (metaclasses) and Ruby (singleton classes)...

I understand this fix is incomplete, but I think it’s still worth merging. This patch doesn’t make the singleton-class-of-a-singleton-class case any worse, and it fixes the ancestor/descendant relationships for singleton classes.

We can probably improve the implementation further in the future.

[vinistock]

We should be able to fix the problem completely. In singleton_parent_id, we already descend on the chain to find the ultimate attached object.

Once we reach the attached object, could we check which descendants of the attached object are missing singleton classes and then enqueue their creation? That would likely both continue to have lazy creation and fix the issue no matter how deep the singleton class level.