[conformance] Model Annotated with a distinct Type case

[fangyi-zhou]

Summary

Before this change, we are treating an annotated type Annotated[T, ...] as the base type T. This is fine in the majority of the places, but does not capture some subtleties according to the spec.

We have failing conformance test cases due to an Annotated type being accepted as a type or TypeAlias, as well as being callable, whereas the spec doesn't allow so.

https://typing.python.org/en/latest/spec/qualifiers.html#annotated

As with most special forms, Annotated is not assignable to type or type[T]

An attempt to call Annotated (whether parameterized or not) should be treated as a type error by type checkers

This diff adds a new Type case for Annotated, which contains the base type. The type behaves like the base type, except it cannot be passed to type or called.

Test Plan

Ran conformance testsuite, and observed that qualifiers_annotated.py is passing.

[github-actions]

Diff from mypy_primer, showing the effect of this PR on open source code:

Tanjun (https://github.com/FasterSpeeding/Tanjun)
+ ERROR tanjun/annotations.py:1601:12-43: Returned type `Annotated[_T]` is not assignable to declared return type `type[_T]` [bad-return]
+ ERROR tanjun/annotations.py:1855:16-42: Returned type `Annotated[str]` is not assignable to declared return type `type[str]` [bad-return]
+ ERROR tanjun/annotations.py:2268:16-64: Returned type `Annotated[PartialChannel]` is not assignable to declared return type `type[PartialChannel]` [bad-return]

pydantic (https://github.com/pydantic/pydantic)
+ ERROR pydantic/types.py:230:12-235:6: Returned type `Annotated[int]` is not assignable to declared return type `type[int]` [bad-return]
+ ERROR pydantic/types.py:491:12-497:6: Returned type `Annotated[float]` is not assignable to declared return type `type[float]` [bad-return]
+ ERROR pydantic/types.py:679:12-683:6: Returned type `Annotated[bytes]` is not assignable to declared return type `type[bytes]` [bad-return]
+ ERROR pydantic/types.py:817:12-828:6: Returned type `Annotated[str]` is not assignable to declared return type `type[str]` [bad-return]
+ ERROR pydantic/types.py:852:12-87: Returned type `Annotated[set[HashableItemType]]` is not assignable to declared return type `type[set[HashableItemType]]` [bad-return]
+ ERROR pydantic/types.py:868:12-93: Returned type `Annotated[frozenset[HashableItemType]]` is not assignable to declared return type `type[frozenset[HashableItemType]]` [bad-return]
+ ERROR pydantic/types.py:903:12-88: Returned type `Annotated[list[AnyItemType]]` is not assignable to declared return type `type[list[AnyItemType]]` [bad-return]
+ ERROR pydantic/types.py:1124:12-1131:6: Returned type `Annotated[Decimal]` is not assignable to declared return type `type[Decimal]` [bad-return]
+ ERROR pydantic/types.py:2257:12-2261:6: Returned type `Annotated[date]` is not assignable to declared return type `type[date]` [bad-return]
+ ERROR pydantic/v1/fields.py:586:45-49: Argument `Annotated[T]` is not assignable to parameter `class_or_tuple` with type `tuple[type[Any], ...] | type[Any] | None` in function `pydantic.v1.utils.lenient_issubclass` [bad-argument-type]
+ ERROR pydantic/v1/types.py:832:20-24: Returned type `Annotated[T]` is not assignable to declared return type `type[JsonWrapper]` [bad-return]

prefect (https://github.com/PrefectHQ/prefect)
- ERROR src/prefect/blocks/core.py:1046:16-51: Returned type `tuple[BlockDocument, str | None]` is not assignable to declared return type `tuple[BlockDocument, str]` [bad-return]
+ ERROR src/prefect/blocks/core.py:1046:16-51: Returned type `tuple[BlockDocument, Name | None]` is not assignable to declared return type `tuple[BlockDocument, str]` [bad-return]

[github-actions]

Primer Diff Classification

❌ 2 regression(s) | ➖ 1 neutral | 3 project(s) total

2 regression(s) across Tanjun, pydantic. error kinds: bad-return, bad-argument-type. caused by mk_type_form().

Project	Verdict	Changes	Error Kinds	Root Cause
Tanjun	❌ Regression	+3	bad-return	mk_type_form()
pydantic	❌ Regression	+11	bad-argument-type, bad-return	pyrefly/lib/alt/specials.rs
prefect	➖ Neutral	+1, -1	bad-return

Detailed analysis

❌ Regression (2)

Tanjun (+3)

These are new errors where pyrefly now treats Annotated[T, ...] as incompatible with type[T] in value contexts. While this may be technically more accurate to runtime behavior (since Annotated[int, ...] creates an _AnnotatedAlias object, not int itself), this appears to be stricter than the established ecosystem standard. The typing spec treats Annotated[T, ...] as equivalent to T for type checking, and most code expects to use them interchangeably. The functions like _annotated() are clearly designed to return type aliases that should work as type objects. This represents pyrefly being more restrictive than the practical typing ecosystem expects.
Attribution: The change in pyrefly/lib/alt/specials.rs modified how Annotated is handled - instead of unwrapping to the inner type with mk_type_form(), it now creates a distinct Type::Annotated variant. The subset checking logic in pyrefly/lib/solver/subset.rs was updated to explicitly reject assignments from Type::Annotated(_) to Type::Type(_), causing these new errors.

pydantic (+11)

These are regressions. While the code has a theoretical type-level inconsistency (returning Annotated[T, ...] instead of type[T]), pyrefly is being stricter than the established ecosystem standard. Mypy and pyright don't flag these patterns because they work correctly in practice - the functions return type aliases that behave appropriately when used in annotations. The errors create noise without practical value since the code functions as intended.
Attribution: The change to introduce Type::Annotated in pyrefly/lib/alt/specials.rs and the subset checking logic in pyrefly/lib/solver/subset.rs that prevents Annotated[T, ...] from being assigned to type[T] caused these errors.

➖ Neutral (1)

prefect (+1, -1)

Same errors at same locations with same error kinds — message wording changed, no behavioral impact.

Suggested Fix

Summary: The PR introduced a new Type::Annotated variant that prevents Annotated[T, ...] from being assignable to type[T], causing regressions in Tanjun and pydantic where this pattern is commonly used.

1. In is_subtype_of() in solver/subset.rs, add a special case for Type::Annotated: when checking if Annotated[T, ...] is assignable to type[U], unwrap the Annotated and check if T is assignable to type[U] instead. This restores the previous behavior where Annotated types were treated as equivalent to their inner type for assignment purposes.

Files: pyrefly/lib/solver/subset.rs
Confidence: high
Affected projects: Tanjun, pydantic
Fixes: bad-assignment, bad-argument-type
The regression analysis clearly shows that the subset checking logic now explicitly rejects assignments from Type::Annotated() to Type::Type(). Adding a special case to unwrap Annotated types when checking assignability to type[T] would restore the ecosystem-standard behavior. Expected outcome: eliminates 14 bad-assignment and bad-argument-type errors across 2 projects

2. In check_callable() in solver/subset.rs or the relevant callable checking function, add a guard: when checking if Type::Annotated(inner) is callable, delegate to checking if the inner type is callable. This prevents not-callable errors when Annotated types wrap callable types.

Files: pyrefly/lib/solver/subset.rs
Confidence: medium
Affected projects: Tanjun, pydantic
Fixes: not-callable
The conformance test shows new not-callable errors for Annotated types. Since Annotated[T, ...] should behave like T for most purposes, callable checking should unwrap the annotation. Expected outcome: eliminates 2 not-callable errors in conformance tests

---

_{Classification by primer-classifier (1 heuristic, 2 LLM) · Was this helpful? React with 👍 or 👎}

[migeed-z]

I'm working on a tool to classify those errors easily. Those look like expected errors to me. I think the LLM classified them as regressions, because it doesn't have the PR title and explanation. It thinks the regressions were not intentional and its feedback is essentially undoing the PR. I will fix that.

[meta-codesync]

@migeed-z has imported this pull request. If you are a Meta employee, you can view this in D94607593.

[migeed-z]

Hmm.. so the diff ended up introducing 100+ errors on IG. cc @rchen152 for opinions on this.

[rchen152]

Hmm.. so the diff ended up introducing 100+ errors on IG. cc @rchen152 for opinions on this.

I only looked at a few errors, but the ones I dug into were caused by pyrefly no longer treating things like Annotated[int, "metadata"] as callable. As per https://discuss.python.org/t/is-annotated-compatible-with-type-t/43898, this (pyrefly's new behavior as of this PR) is correct and spec-compliant, and IG was essentially relying on a bug in pyrefly.

[fangyi-zhou]

The errors are true positives given a strict reading of the spec, in practice people have been using that in the wild quite often.

See discussion here

https://discuss.python.org/t/is-annotated-compatible-with-type-t/43898

[grievejia]

Review automatically exported from Phabricator review in Meta.

[meta-codesync]

@migeed-z merged this pull request in 14be156.