-
-
Notifications
You must be signed in to change notification settings - Fork 3.1k
[wip] Improve reachability in narrowing logic, avoid losing narrowing constraints #20660
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
| def __ge__(self, x: i64) -> bool: ... | ||
| def __gt__(self, x: i64) -> bool: ... | ||
| def __index__(self) -> int: ... | ||
| def __eq__(self, x: object) -> bool: ... |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Without (correctly) annotating the custom equality here, we would consider otherwise reachable code to be unreachable. Specifically the right side of this or clause:
Line 1627 in d475659
| if meta[0] != cache_version() or meta[1] != CACHE_VERSION: |
|
@p-sawicki I'm getting https://github.com/python/mypy/actions/runs/21421020108/job/61680277800?pr=20660 |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
Primer needs some work, specifically I need to handle promotions better |
This comment has been minimized.
This comment has been minimized.
4849aaf to
35a3b2d
Compare
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
the assertions are meant to catch mistakes when modifying mypyc so it's unfortunate that changes in mypy triggered them. are tests still failing because of them? if they block unrelated PRs then i think we would have to disable them. |
09a0db0 to
e9c1274
Compare
#20660 is proving a little tricky, so I'm going to try and land some of the diff independently. I think of this as mostly a cleanup PR, but it does change a few semantics. In particular, the places where we previously added a `{}` to the list of type maps used to clobber some narrowing
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
|
@p-sawicki looks like later versions of this PR no longer trigger it, for whatever reason, so I'm not blocked! But if you |
|
Diff from mypy_primer, showing the effect of this PR on open source code: beartype (https://github.com/beartype/beartype)
+ beartype/claw/_ast/_kind/clawastimport.py:1354: error: Statement is unreachable [unreachable]
optuna (https://github.com/optuna/optuna)
+ tests/study_tests/test_study.py:1428: error: Statement is unreachable [unreachable]
psycopg (https://github.com/psycopg/psycopg)
+ tests/test_cursor_server_async.py:416: error: Statement is unreachable [unreachable]
+ tests/test_cursor_server.py:410: error: Statement is unreachable [unreachable]
- tests/test_cursor_common_async.py:543: error: Statement is unreachable [unreachable]
+ tests/test_cursor_common_async.py:151: error: Statement is unreachable [unreachable]
- tests/test_cursor_common.py:538: error: Statement is unreachable [unreachable]
+ tests/test_cursor_common_async.py:534: error: Statement is unreachable [unreachable]
+ tests/test_cursor_common_async.py:574: error: Statement is unreachable [unreachable]
+ tests/test_cursor_common.py:153: error: Statement is unreachable [unreachable]
+ tests/test_cursor_common.py:529: error: Statement is unreachable [unreachable]
+ tests/test_cursor_common.py:569: error: Statement is unreachable [unreachable]
prefect (https://github.com/PrefectHQ/prefect)
+ src/prefect/filesystems.py:252: error: Statement is unreachable [unreachable]
+ src/prefect/filesystems.py:296: error: Statement is unreachable [unreachable]
- src/prefect/server/api/server.py:664: error: Invalid index type "tuple[str, bool, bool]" for "dict[tuple[Settings, bool], Any]"; expected type "tuple[Settings, bool]" [index]
+ src/prefect/server/api/server.py:663: error: Right operand of "and" is never evaluated [unreachable]
+ src/prefect/server/api/server.py:664: error: Statement is unreachable [unreachable]
pydantic (https://github.com/pydantic/pydantic)
+ pydantic/v1/typing.py:623: error: Statement is unreachable [unreachable]
+ pydantic/v1/utils.py:768: error: Statement is unreachable [unreachable]
+ pydantic/v1/fields.py:459: error: Statement is unreachable [unreachable]
+ pydantic/v1/fields.py:641: error: Statement is unreachable [unreachable]
+ pydantic/v1/schema.py:1055: error: Statement is unreachable [unreachable]
+ pydantic/v1/generics.py:270: error: Statement is unreachable [unreachable]
+ pydantic/v1/generics.py:274: error: Statement is unreachable [unreachable]
+ pydantic/main.py:1363: error: Statement is unreachable [unreachable]
colour (https://github.com/colour-science/colour)
+ colour/utilities/tests/test_structures.py:71: error: Statement is unreachable [unreachable]
- colour/utilities/tests/test_network.py:337: error: Statement is unreachable [unreachable]
+ colour/utilities/tests/test_network.py:323: error: Statement is unreachable [unreachable]
mypy (https://github.com/python/mypy)
+ mypy/server/aststrip.py:140: error: If condition in comprehension is always true [redundant-expr]
+ mypy/server/aststrip.py:140: note: Error code "redundant-expr" not covered by "type: ignore" comment
+ mypy/server/aststrip.py:140: note: See https://mypy.rtfd.io/en/stable/_refs.html#code-redundant-expr for more info
+ mypy/stubutil.py:76: error: Statement is unreachable [unreachable]
+ mypy/stubutil.py:76: note: See https://mypy.rtfd.io/en/stable/_refs.html#code-unreachable for more info
altair (https://github.com/vega/altair)
+ altair/utils/schemapi.py:413: error: Right operand of "and" is never evaluated [unreachable]
+ altair/utils/schemapi.py:479: error: Right operand of "and" is never evaluated [unreachable]
+ altair/utils/schemapi.py:481: error: Statement is unreachable [unreachable]
+ altair/utils/schemapi.py:717: error: Statement is unreachable [unreachable]
+ altair/vegalite/v6/api.py:1536: error: Statement is unreachable [unreachable]
+ tools/schemapi/schemapi.py:411: error: Right operand of "and" is never evaluated [unreachable]
+ tools/schemapi/schemapi.py:477: error: Right operand of "and" is never evaluated [unreachable]
+ tools/schemapi/schemapi.py:479: error: Statement is unreachable [unreachable]
+ tools/schemapi/schemapi.py:715: error: Statement is unreachable [unreachable]
graphql-core (https://github.com/graphql-python/graphql-core)
+ tests/pyutils/test_identity_func.py:7: error: Statement is unreachable [unreachable]
+ tests/pyutils/test_boxed_awaitabe_or_value.py:44: error: Statement is unreachable [unreachable]
+ tests/pyutils/test_description.py:92: error: Statement is unreachable [unreachable]
schemathesis (https://github.com/schemathesis/schemathesis)
+ src/schemathesis/specs/openapi/extra_data_source.py: note: In member "weighted_select" of class "VariantUsageTracker":
+ src/schemathesis/specs/openapi/extra_data_source.py:85: error: Statement is unreachable [unreachable]
strawberry (https://github.com/strawberry-graphql/strawberry)
+ strawberry/types/fields/resolver.py:179: error: Statement is unreachable [unreachable]
operator (https://github.com/canonical/operator)
+ ops/framework.py:678: error: Statement is unreachable [unreachable]
- ops/_private/harness.py:2563: error: Invalid index type "str" for "dict[int, dict[str, Any]]"; expected type "int" [index]
+ ops/_private/harness.py:2563: error: Statement is unreachable [unreachable]
core (https://github.com/home-assistant/core)
+ homeassistant/components/homekit/__init__.py:982: error: Right operand of "or" is never evaluated [unreachable]
+ homeassistant/components/firefly_iii/sensor.py:190: error: Statement is unreachable [unreachable]
sphinx (https://github.com/sphinx-doc/sphinx)
+ sphinx/util/nodes.py:268:13: error: Statement is unreachable [unreachable]
+ sphinx/config.py: note: In member "convert_overrides" of class "Config":
+ sphinx/config.py:359:13: error: Statement is unreachable [unreachable]
+ sphinx/config.py: note: In function "_validate_valid_types":
+ sphinx/config.py:626:9: error: Statement is unreachable [unreachable]
+ sphinx/domains/python/_annotations.py: note: In function "_parse_annotation":
+ sphinx/domains/python/_annotations.py:166:17: error: Statement is unreachable [unreachable]
+ sphinx/builders/linkcheck.py: note: In function "_allowed_redirect":
+ sphinx/builders/linkcheck.py:757:9: error: Statement is unreachable [unreachable]
pytest (https://github.com/pytest-dev/pytest)
+ testing/test_capture.py:923: error: Statement is unreachable [unreachable]
mitmproxy (https://github.com/mitmproxy/mitmproxy)
+ mitmproxy/utils/typecheck.py:25: error: Statement is unreachable [unreachable]
+ mitmproxy/utils/typecheck.py:34: error: Statement is unreachable [unreachable]
+ mitmproxy/utils/typecheck.py:43: error: Statement is unreachable [unreachable]
sympy (https://github.com/sympy/sympy)
+ sympy/polys/puiseux.py:563: error: Statement is unreachable [unreachable]
+ sympy/polys/puiseux.py:565: error: Statement is unreachable [unreachable]
+ sympy/matrices/matrixbase.py:5258: error: Statement is unreachable [unreachable]
+ sympy/matrices/matrixbase.py:5261: error: Statement is unreachable [unreachable]
+ sympy/matrices/matrixbase.py:5285: error: Statement is unreachable [unreachable]
+ sympy/integrals/manualintegrate.py:1581: error: Statement is unreachable [unreachable]
+ sympy/integrals/manualintegrate.py:1592: error: Statement is unreachable [unreachable]
+ sympy/integrals/manualintegrate.py:2381: error: Statement is unreachable [unreachable]
+ sympy/polys/series/ringpython.py:410: error: Statement is unreachable [unreachable]
+ sympy/polys/series/ringpython.py:412: error: Statement is unreachable [unreachable]
pandas (https://github.com/pandas-dev/pandas)
+ pandas/core/arrays/datetimes.py:2976: error: Statement is unreachable [unreachable]
+ pandas/core/arrays/datetimes.py:2984: error: Statement is unreachable [unreachable]
+ pandas/core/arrays/datetimes.py:2997: error: Unused "type: ignore" comment [unused-ignore]
+ pandas/core/arrays/datetimes.py:3009: error: Unused "type: ignore" comment [unused-ignore]
+ pandas/core/arrays/datetimes.py:3009: error: Statement is unreachable [unreachable]
+ pandas/core/arrays/datetimes.py:3009: note: Error code "unreachable" not covered by "type: ignore" comment
+ pandas/core/arrays/datetimes.py:3011: error: Redundant cast to "Timestamp" [redundant-cast]
+ pandas/core/arrays/datetimes.py:3012: error: Redundant cast to "Timestamp" [redundant-cast]
+ pandas/core/arrays/numpy_.py:258: error: Statement is unreachable [unreachable]
discord.py (https://github.com/Rapptz/discord.py)
+ discord/player.py:167: error: Statement is unreachable [unreachable]
+ discord/http.py:170: error: Statement is unreachable [unreachable]
+ discord/http.py:274: error: Unused "type: ignore" comment [unused-ignore]
- discord/http.py:190: error: Incompatible types in assignment (expression has type "str", target has type "list[Embed]") [assignment]
- discord/http.py:192: error: Incompatible types in assignment (expression has type "None", target has type "list[Embed]") [assignment]
- discord/http.py:196: error: Incompatible types in assignment (expression has type "list[dict[str, Any]]", target has type "list[Embed]") [assignment]
- discord/http.py:207: error: Incompatible types in assignment (expression has type "str", target has type "list[Embed]") [assignment]
- discord/http.py:208: error: Incompatible types in assignment (expression has type "bool", target has type "list[Embed]") [assignment]
- discord/http.py:211: error: Incompatible types in assignment (expression has type "MessageReference | None", target has type "list[Embed]") [assignment]
- discord/http.py:215: error: Incompatible types in assignment (expression has type "list[str | int]", target has type "list[Embed]") [assignment]
- discord/http.py:219: error: Incompatible types in assignment (expression has type "bool", target has type "list[Embed]") [assignment]
- discord/http.py:221: error: Incompatible types in assignment (expression has type "str", target has type "list[Embed]") [assignment]
- discord/http.py:223: error: Incompatible types in assignment (expression has type "str", target has type "list[Embed]") [assignment]
- discord/http.py:226: error: Incompatible types in assignment (expression has type "int", target has type "list[Embed]") [assignment]
- discord/http.py:229: error: Incompatible types in assignment (expression has type "str", target has type "list[Embed]") [assignment]
- discord/http.py:233: error: Incompatible types in assignment (expression has type "AllowedMentions", target has type "list[Embed]") [assignment]
- discord/http.py:235: error: Incompatible types in assignment (expression has type "AllowedMentions", target has type "list[Embed]") [assignment]
- discord/http.py:237: error: Incompatible types in assignment (expression has type "AllowedMentions", target has type "list[Embed]") [assignment]
- discord/http.py:241: error: Incompatible types in assignment (expression has type "AllowedMentions", target has type "list[Embed]") [assignment]
- discord/http.py:242: error: No overload variant of "__setitem__" of "list" matches argument types "str", "bool" [call-overload]
- discord/http.py:242: note: Possible overload variants:
- discord/http.py:242: note: def __setitem__(self, SupportsIndex, Embed, /) -> None
- discord/http.py:242: note: def __setitem__(self, slice[Any, Any, Any], Iterable[Embed], /) -> None
- discord/http.py:257: error: Argument 1 to "append" of "list" has incompatible type "Attachment"; expected "dict[str, Any]" [arg-type]
- discord/http.py:259: error: Incompatible types in assignment (expression has type "list[dict[str, Any]]", target has type "list[Embed]") [assignment]
- discord/http.py:263: error: Incompatible types in assignment (expression has type "list[str | int]", target has type "list[Embed]") [assignment]
- discord/http.py:269: error: Dict entry 0 has incompatible type "str": "dict[str, list[Embed]]"; expected "str": "list[Embed]" [dict-item]
- discord/http.py:279: error: Incompatible types in assignment (expression has type "None", variable has type "dict[str, list[Embed]]") [assignment]
- discord/http.py:284: error: Dict entry 1 has incompatible type "str": "BufferedIOBase"; expected "str": "str" [dict-item]
+ discord/webhook/sync.py:1088: error: Statement is unreachable [unreachable]
- discord/webhook/sync.py:1105: error: Argument "applied_tags" to "handle_message_parameters" has incompatible type "str | Snowflake"; expected "list[str | int] | None" [arg-type]
+ discord/webhook/sync.py:1105: error: Cannot determine type of "applied_tag_ids" [has-type]
+ discord/webhook/async_.py:565: error: Statement is unreachable [unreachable]
- discord/webhook/async_.py:578: error: Unsupported target for indexed assignment ("dict[str, Any] | None") [index]
- discord/webhook/async_.py:582: error: Unsupported target for indexed assignment ("dict[str, Any] | None") [index]
- discord/webhook/async_.py:584: error: Unsupported target for indexed assignment ("dict[str, Any] | None") [index]
- discord/webhook/async_.py:588: error: Unsupported target for indexed assignment ("dict[str, Any] | None") [index]
- discord/webhook/async_.py:590: error: Unsupported target for indexed assignment ("dict[str, Any] | None") [index]
- discord/webhook/async_.py:594: error: Unsupported target for indexed assignment ("dict[str, Any] | None") [index]
- discord/webhook/async_.py:603: error: Unsupported target for indexed assignment ("dict[str, Any] | None") [index]
- discord/webhook/async_.py:606: error: Unsupported target for indexed assignment ("dict[str, Any] | None") [index]
- discord/webhook/async_.py:610: error: Unsupported target for indexed assignment ("dict[str, Any] | None") [index]
- discord/webhook/async_.py:612: error: Unsupported target for indexed assignment ("dict[str, Any] | None") [index]
- discord/webhook/async_.py:614: error: Unsupported target for indexed assignment ("dict[str, Any] | None") [index]
- discord/webhook/async_.py:629: error: Argument 1 to "append" of "list" has incompatible type "Attachment"; expected "dict[str, Any]" [arg-type]
- discord/webhook/async_.py:631: error: Unsupported target for indexed assignment ("dict[str, Any] | None") [index]
- discord/webhook/async_.py:634: error: Unsupported target for indexed assignment ("dict[str, Any] | None") [index]
- discord/webhook/async_.py:645: error: Dict entry 1 has incompatible type "str": "BufferedIOBase"; expected "str": "str" [dict-item]
+ discord/webhook/async_.py:1874: error: Statement is unreachable [unreachable]
- discord/webhook/async_.py:1892: error: Argument "applied_tags" to "handle_message_parameters" has incompatible type "str | Snowflake"; expected "list[str | int] | None" [arg-type]
+ discord/webhook/async_.py:1892: error: Cannot determine type of "applied_tag_ids" [has-type]
+ discord/app_commands/tree.py:96: error: Statement is unreachable [unreachable]
+ discord/ext/tasks/__init__.py:645: error: Statement is unreachable [unreachable]
dd-trace-py (https://github.com/DataDog/dd-trace-py)
+ ddtrace/debugging/_import.py:23: error: Statement is unreachable [unreachable]
trio (https://github.com/python-trio/trio)
+ src/trio/_tests/test_path.py:163: error: Statement is unreachable [unreachable]
+ src/trio/_tests/test_socket.py:1127: error: Statement is unreachable [unreachable]
+ src/trio/_tests/test_exports.py:574: error: Statement is unreachable [unreachable]
- src/trio/_core/_tests/test_run.py:2467: error: Statement is unreachable [unreachable]
+ src/trio/_core/_tests/test_run.py:2456: error: Statement is unreachable [unreachable]
+ src/trio/_core/_tests/test_run.py:2457: error: Unused "type: ignore" comment [unused-ignore]
+ src/trio/_core/_tests/test_guest_mode.py:161: error: Statement is unreachable [unreachable]
ibis (https://github.com/ibis-project/ibis)
+ ibis/backends/polars/__init__.py:452: error: Statement is unreachable [unreachable]
jax (https://github.com/google/jax)
+ jax/_src/internal_test_util/test_harnesses.py:2608: error: Statement is unreachable [unreachable]
+ jax/_src/pallas/core.py:1063: error: Statement is unreachable [unreachable]
Expression (https://github.com/cognitedata/Expression)
+ README.py:394: error: Statement is unreachable [unreachable]
- README.py:460: error: Incompatible types in assignment (expression has type "Result[int, Exception]", variable has type "Seq[int]") [assignment]
- README.py:461: error: Subclass of "Seq[int]" and "Result[Any, Any]" cannot exist: would have incompatible method signatures [unreachable]
- README.py:464: error: Statement is unreachable [unreachable]
pyodide (https://github.com/pyodide/pyodide)
+ pyodide-build/pyodide_build/tests/test_xbuildenv.py:88: error: Statement is unreachable [unreachable]
pandera (https://github.com/pandera-dev/pandera)
+ tests/pandas/test_schemas.py:1179: error: Statement is unreachable [unreachable]
+ tests/pandas/test_model.py:43: error: Statement is unreachable [unreachable]
+ tests/pandas/test_model.py:385: error: Statement is unreachable [unreachable]
+ tests/pandas/test_model.py:401: error: Statement is unreachable [unreachable]
+ tests/pandas/test_model.py:406: error: Statement is unreachable [unreachable]
+ tests/pandas/test_model.py:421: error: Statement is unreachable [unreachable]
+ tests/pandas/test_model.py:437: error: Statement is unreachable [unreachable]
+ tests/pandas/test_model.py:455: error: Statement is unreachable [unreachable]
poetry (https://github.com/python-poetry/poetry)
+ tests/installation/test_chooser.py:344: error: Statement is unreachable [unreachable]
rotki (https://github.com/rotki/rotki)
+ rotkehlchen/exchanges/binance.py:391: error: Statement is unreachable [unreachable]
+ rotkehlchen/tests/api/test_evmlike.py:212: error: Statement is unreachable [unreachable]
+ rotkehlchen/tests/api/test_history_base_entry.py:1263: error: Need type annotation for "ids_per_subtype" [var-annotated]
static-frame (https://github.com/static-frame/static-frame)
+ static_frame/core/archive_npy.py:232: error: Statement is unreachable [unreachable]
+ static_frame/core/interface.py:1041: error: Statement is unreachable [unreachable]
+ static_frame/core/interface.py:1043: error: Statement is unreachable [unreachable]
+ static_frame/core/interface.py:1066: error: Item "type" of "type[InterfaceValues[TVContainer_co@InterfaceValues]] | type[InterfaceString[TVContainer_co@InterfaceString]] | type[InterfaceDatetime[TVContainer_co@InterfaceDatetime]] | type[InterfaceTranspose[TVContainer_co@InterfaceTranspose]] | type[InterfaceFillValue[TVContainer_co@InterfaceFillValue]] | type[InterfaceRe[TVContainer_co@InterfaceRe]] | type[ReduceDispatch] | type[<subclass of "static_frame.core.node_selector.Interface" and "typing.Mapping[Any, Any]">]" has no attribute "CLS_DELEGATE" [union-attr]
cloud-init (https://github.com/canonical/cloud-init)
+ tests/unittests/sources/test_maas.py:269: error: Statement is unreachable [unreachable]
+ tests/unittests/cmd/test_status.py:260: error: Statement is unreachable [unreachable]
xarray (https://github.com/pydata/xarray)
+ xarray/core/extension_array.py: note: In function "union_unordered_categorical_and_scalar":
+ xarray/core/extension_array.py:163: error: If condition in comprehension is always true [redundant-expr]
+ xarray/coding/cftime_offsets.py: note: In member "_next_higher_resolution" of class "Tick":
+ xarray/coding/cftime_offsets.py:206: error: Statement is unreachable [unreachable]
freqtrade (https://github.com/freqtrade/freqtrade)
+ freqtrade/wallets.py:380: error: Statement is unreachable [unreachable]
|
By decoupling if_map's and else_map's we don't have to make up empty dictionaries to make them zip well, which enables us to preserve the None's that indicate a lack of reachability better.
This also prevents us from losing narrowing constraints in some cases, see e.g. the new
testNarrowingConstrainedTypeVarTypetest which we previously errored onImproves #17372
Fixes #17045
Improves #14965
Fixes this comment #9003 (comment)