Commit 1b7e717
authored
_(This PR replaces an earlier draft of the same feature:
#18690 )_
Feedback from @JukkaL integrated since the last PR, by commit title:
* Apply feedback: Change MAYBE_UNRECOGNIZED_STR_TYPEFORM from
unaccompanied note to standalone error
* Apply feedback: Refactor extract save/restore of SemanticAnalyzer
state to a new context manager
* Apply feedback: Suppress SyntaxWarnings when parsing strings as types
at the _most-targeted_ location
* Apply feedback: Add TypeForm profiling counters to SemanticAnalyzer
and the --dump-build-stats option
* Increase efficiency of quick rejection heuristic from 85.8% -> 99.6%
in SemanticAnalyzer.try_parse_as_type_expression()
* Apply feedback: Recognize assignment to union of TypeForm with
non-TypeForm
* Apply feedback: Alter primitives.pyi fixture rather than tuple.pyi and
dict.pyi
Feedback NOT integrated, with rationale:
* ✖️ Add tests related to recursive types
* Recursive cases are already well-covered by tests related to TypeType
(is_type_form=False).
* I _did_ find an infinite recursion bug affecting garden-variety
`Type[...]`, which I can fix in a separate PR.
* ✖️ Define `TypeForm(...)` in value contexts as a regular function like
`Callable[[TypeForm[T]], TypeForm[T]]` rather than as a special
expression node (TypeFormExpr).
* The special expression node allows mypy to print out _better error
messages_ when a user puts an invalid type expression inside
`TypeForm(...)`. See case 4 of testTypeFormExpression in
check-typeform.test
There is one commit unrelated to the core function of this PR that could
be split to a separate PR:
* Allow TypeAlias and PlaceholderNode to be stringified/printed
Closes #9773
---
_(Most of the following description is copied from the original PR,
**except for the text in bold**)_
Implements the [TypeForm PEP 747](https://peps.python.org/pep-0747/), as
an opt-in feature enabled by the CLI flag
`--enable-incomplete-feature=TypeForm`.
Implementation approach:
* The `TypeForm[T]` is represented as a type using the existing
`TypeType` class, with an `is_type_form=True` constructor parameter.
`Type[C]` continues to be represented using `TypeType`, but with
`is_type_form=False` (the default).
* Recognizing a type expression literal such as `int | str` requires
parsing an `Expression` as a type expression. Only the SemanticAnalyzer
pass has the ability to parse **arbitrary** type expressions
**(including stringified annotations)**, using
`SemanticAnalyzer.expr_to_analyzed_type()`. **(I've extended the
`TypeChecker` pass to parse all kinds of type expressions except
stringified annotations, using the new `TypeCheckerAsSemanticAnalyzer`
adapter.)**
* Therefore during the SemanticAnalyzer pass, at certain syntactic
locations (i.e. assignment r-values, callable arguments, returned
expressions), the analyzer tries to parse the `Expression` it is looking
at using `try_parse_as_type_expression()` - a new function - and stores
the result (a `Type`) in `{IndexExpr, OpExpr, StrExpr}.as_type` - a new
attribute.
* During the later TypeChecker pass, when looking at an `Expression` to
determine its type, if the expression is in a type context that expects
some kind of `TypeForm[...]` and the expression was successfully parsed
as a type expression by the earlier SemanticAnalyzer pass **(or can be
parsed as a type expression immediately during the type checker pass)**,
the expression will be given the type `TypeForm[expr.as_type]` rather
than using the regular type inference rules for a value expression.
* Key relationships between `TypeForm[T]`, `Type[C]`, and `object` types
are defined in the visitors powering `is_subtype`, `join_types`, and
`meet_types`.
* The `TypeForm(T)` expression is recognized as a `TypeFormExpr` and has
the return type `TypeForm[T]`.
* The new test suite in `check-typeform.test` is a good reference to the
expected behaviors for operations that interact with `TypeForm` in some
way.
Controversial parts of this PR, in @davidfstr 's opinion:
* Type form literals **containing stringified annotations** are only
recognized in certain syntactic locations (and not ALL possible
locations). Namely they are recognized as (1) assignment r-values, (2)
callable expression arguments, and (3) as returned expressions, but
nowhere else. For example they aren't recognized in expressions like
`dict_with_typx_keys[int | str]`. **Attempting to use stringified
annotations in other locations will emit a
MAYBE_UNRECOGNIZED_STR_TYPEFORM error.**
* The existing `TypeType` class is now used to represent BOTH the
`Type[T]` and `TypeForm[T]` types, rather than introducing a distinct
subclass of `Type` to represent the `TypeForm[T]` type. This was done to
simplify logic that manipulates both `Type[T]` and `TypeForm[T]` values,
since they are both manipulated in very similar ways.
* The "normalized" form of `TypeForm[X | Y]` - as returned by
`TypeType.make_normalized()` - is just `TypeForm[X | Y]` rather than
`TypeForm[X] | TypeForm[Y]`, differing from the normalization behavior
of `Type[X | Y]`.
1 parent 2809328 commit 1b7e717
File tree
38 files changed
+1814
-73
lines changed- docs/source
- misc
- mypyc/irbuild
- mypy
- server
- test-data/unit
- fixtures
- lib-stub
38 files changed
+1814
-73
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1286 | 1286 | | |
1287 | 1287 | | |
1288 | 1288 | | |
| 1289 | + | |
| 1290 | + | |
| 1291 | + | |
| 1292 | + | |
| 1293 | + | |
| 1294 | + | |
| 1295 | + | |
| 1296 | + | |
| 1297 | + | |
| 1298 | + | |
| 1299 | + | |
| 1300 | + | |
| 1301 | + | |
| 1302 | + | |
| 1303 | + | |
| 1304 | + | |
| 1305 | + | |
| 1306 | + | |
| 1307 | + | |
| 1308 | + | |
| 1309 | + | |
| 1310 | + | |
| 1311 | + | |
| 1312 | + | |
| 1313 | + | |
| 1314 | + | |
| 1315 | + | |
| 1316 | + | |
| 1317 | + | |
| 1318 | + | |
| 1319 | + | |
| 1320 | + | |
| 1321 | + | |
| 1322 | + | |
| 1323 | + | |
| 1324 | + | |
| 1325 | + | |
| 1326 | + | |
| 1327 | + | |
| 1328 | + | |
| 1329 | + | |
| 1330 | + | |
| 1331 | + | |
| 1332 | + | |
| 1333 | + | |
| 1334 | + | |
| 1335 | + | |
| 1336 | + | |
| 1337 | + | |
| 1338 | + | |
| 1339 | + | |
| 1340 | + | |
| 1341 | + | |
| 1342 | + | |
| 1343 | + | |
| 1344 | + | |
| 1345 | + | |
| 1346 | + | |
| 1347 | + | |
1289 | 1348 | | |
1290 | 1349 | | |
1291 | 1350 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
155 | 155 | | |
156 | 156 | | |
157 | 157 | | |
| 158 | + | |
158 | 159 | | |
159 | 160 | | |
160 | 161 | | |
| |||
346 | 347 | | |
347 | 348 | | |
348 | 349 | | |
| 350 | + | |
| 351 | + | |
349 | 352 | | |
350 | 353 | | |
351 | 354 | | |
| |||
423 | 426 | | |
424 | 427 | | |
425 | 428 | | |
| 429 | + | |
426 | 430 | | |
427 | 431 | | |
428 | 432 | | |
| |||
2661 | 2665 | | |
2662 | 2666 | | |
2663 | 2667 | | |
2664 | | - | |
| 2668 | + | |
| 2669 | + | |
| 2670 | + | |
| 2671 | + | |
| 2672 | + | |
2665 | 2673 | | |
2666 | 2674 | | |
2667 | 2675 | | |
| |||
2729 | 2737 | | |
2730 | 2738 | | |
2731 | 2739 | | |
| 2740 | + | |
| 2741 | + | |
| 2742 | + | |
| 2743 | + | |
| 2744 | + | |
| 2745 | + | |
| 2746 | + | |
| 2747 | + | |
| 2748 | + | |
2732 | 2749 | | |
2733 | 2750 | | |
2734 | 2751 | | |
| |||
8023 | 8040 | | |
8024 | 8041 | | |
8025 | 8042 | | |
8026 | | - | |
| 8043 | + | |
| 8044 | + | |
| 8045 | + | |
8027 | 8046 | | |
8028 | 8047 | | |
8029 | 8048 | | |
| |||
8151 | 8170 | | |
8152 | 8171 | | |
8153 | 8172 | | |
| 8173 | + | |
| 8174 | + | |
| 8175 | + | |
| 8176 | + | |
| 8177 | + | |
| 8178 | + | |
| 8179 | + | |
| 8180 | + | |
| 8181 | + | |
| 8182 | + | |
| 8183 | + | |
| 8184 | + | |
| 8185 | + | |
| 8186 | + | |
| 8187 | + | |
| 8188 | + | |
| 8189 | + | |
| 8190 | + | |
| 8191 | + | |
| 8192 | + | |
| 8193 | + | |
| 8194 | + | |
| 8195 | + | |
| 8196 | + | |
| 8197 | + | |
| 8198 | + | |
| 8199 | + | |
| 8200 | + | |
| 8201 | + | |
| 8202 | + | |
| 8203 | + | |
| 8204 | + | |
| 8205 | + | |
| 8206 | + | |
| 8207 | + | |
| 8208 | + | |
| 8209 | + | |
| 8210 | + | |
| 8211 | + | |
| 8212 | + | |
| 8213 | + | |
| 8214 | + | |
| 8215 | + | |
| 8216 | + | |
| 8217 | + | |
| 8218 | + | |
| 8219 | + | |
| 8220 | + | |
| 8221 | + | |
| 8222 | + | |
| 8223 | + | |
| 8224 | + | |
| 8225 | + | |
| 8226 | + | |
| 8227 | + | |
| 8228 | + | |
| 8229 | + | |
| 8230 | + | |
| 8231 | + | |
| 8232 | + | |
| 8233 | + | |
| 8234 | + | |
| 8235 | + | |
| 8236 | + | |
| 8237 | + | |
| 8238 | + | |
| 8239 | + | |
| 8240 | + | |
| 8241 | + | |
| 8242 | + | |
| 8243 | + | |
| 8244 | + | |
| 8245 | + | |
| 8246 | + | |
| 8247 | + | |
| 8248 | + | |
| 8249 | + | |
| 8250 | + | |
| 8251 | + | |
| 8252 | + | |
| 8253 | + | |
| 8254 | + | |
| 8255 | + | |
| 8256 | + | |
| 8257 | + | |
| 8258 | + | |
| 8259 | + | |
| 8260 | + | |
| 8261 | + | |
| 8262 | + | |
| 8263 | + | |
8154 | 8264 | | |
8155 | 8265 | | |
8156 | 8266 | | |
| |||
0 commit comments