Incorrect type annotations and mypy ≥0.960 failure in macro_collector.py #50

New issue

Open

opened 2024-09-23 12:05:35 -05:00 by gilles-peskine-arm · 0 comments

gilles-peskine-arm commented

2024-09-23 12:05:35 -05:00

(Migrated from github.com)

scripts/mbedtls_framework/macro_collector.py fails to typecheck with mypy ≥0.960.

framework/scripts/mbedtls_framework/macro_collector.py:488: error: No overload variant of "sub" matches argument types "Pattern[Any]", "bytes", "str"
framework/scripts/mbedtls_framework/macro_collector.py:488: note: Possible overload variants:
framework/scripts/mbedtls_framework/macro_collector.py:488: note:     def sub(pattern: Union[str, Pattern[str]], repl: Union[str, Callable[[Match[str]], str]], string: str, count: int = ..., flags: Union[int, RegexFlag] = ...) -> str
framework/scripts/mbedtls_framework/macro_collector.py:488: note:     def sub(pattern: Union[bytes, Pattern[bytes]], repl: Union[Union[bytes, Union[bytearray, memoryview, array[Any], mmap, _CData]], Callable[[Match[bytes]], Union[bytes, Union[bytearray, memoryview, array[Any], mmap, _CData]]]], string: Union[bytes, Union[bytearray, memoryview, array[Any], mmap, _CData]], count: int = ..., flags: Union[int, RegexFlag] = ...) -> bytes

The problem is that mypy's knowledge of read_file_lines is not precise enough, or perhaps outright wrong. The type annotation on read_file_lines.__iter__ says that it's an Iterator[str], but it's actually an iterator over bytes when the context manager was initialized with binary=True. Earlier versions of mypy were fine with that, but since mypy 0.960, they're complaining, I think rightfully, I think thanks to improvements in how overloads for open are handled.

The code isn't wrong, but it has a dependent type, which is too hard for mypy.

The goal of this issue is to fix macro_collector.py so that our code typechecks under modern mypy.

`scripts/mbedtls_framework/macro_collector.py` fails to typecheck with mypy ≥0.960. ``` framework/scripts/mbedtls_framework/macro_collector.py:488: error: No overload variant of "sub" matches argument types "Pattern[Any]", "bytes", "str" framework/scripts/mbedtls_framework/macro_collector.py:488: note: Possible overload variants: framework/scripts/mbedtls_framework/macro_collector.py:488: note: def sub(pattern: Union[str, Pattern[str]], repl: Union[str, Callable[[Match[str]], str]], string: str, count: int = ..., flags: Union[int, RegexFlag] = ...) -> str framework/scripts/mbedtls_framework/macro_collector.py:488: note: def sub(pattern: Union[bytes, Pattern[bytes]], repl: Union[Union[bytes, Union[bytearray, memoryview, array[Any], mmap, _CData]], Callable[[Match[bytes]], Union[bytes, Union[bytearray, memoryview, array[Any], mmap, _CData]]]], string: Union[bytes, Union[bytearray, memoryview, array[Any], mmap, _CData]], count: int = ..., flags: Union[int, RegexFlag] = ...) -> bytes ``` The problem is that mypy's knowledge of `read_file_lines` is not precise enough, or perhaps outright wrong. The type annotation on `read_file_lines.__iter__` says that it's an `Iterator[str]`, but it's actually an iterator over `bytes` when the context manager was initialized with `binary=True`. Earlier versions of mypy were fine with that, but since [mypy 0.960](https://mypy-lang.blogspot.com/2022/05/mypy-0960-released.html), they're complaining, I think rightfully, I think thanks to improvements in how overloads for `open` are handled. The code isn't wrong, but it has a dependent type, which is too hard for mypy. The goal of this issue is to fix `macro_collector.py` so that our code typechecks under modern mypy.

No milestone

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference

gh-Mbed-TLS/mbedtls-framework#50

No description provided.

Rows
Columns