Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed up pytype using --precompiled-builtins #1501

Open
rchen152 opened this issue Sep 15, 2023 · 0 comments
Open

Speed up pytype using --precompiled-builtins #1501

rchen152 opened this issue Sep 15, 2023 · 0 comments
Labels
cat: stubs and 3p type stubs and third-party types enhancement

Comments

@rchen152
Copy link
Contributor

We observed that pytype is surprisingly slow when analyzing https://github.com/google/pycnite, a small project with only 9 .py files. I profiled pytype on a few of the files and discovered that it is spending an unreasonable amount of time (~2s per file) parsing the builtins and typing stubs. We already have a way to parse and pickle these as a preprocessing step (--generate-builtins) and to load the pickle rather than re-parsing every time (--precompiled-builtins).

Perhaps we could generate the builtins in out/bin/pytype/ to speed up the CI workflow and also check them in every time a commit is made to main. I briefly tried to do this and ran into a few issues:

  • --generate-builtins assumes that typeshed contains only .pyi files, stdlib/VERSIONS, tests/pytype_exclude_list.txt, and the METADATA.toml files in each stubs subdirectory. In particular, it crashes when it encounters stdlib/_typeshed/README.md and the @tests dirctories in stubs/. This wasn't an issue previously because --generate-builtins was being run with a copy of typeshed into which we'd copied only the files we wanted.
  • typeshed/stubs/ causes dependency resolution issues. The stubs have dependencies on external packages (typeshed's pytype_test has some special logic for this: https://github.com/python/typeshed/blob/0ea043253e70d0304478a6d0b58bcda4cc583d08/tests/pytype_test.py#L163). Plus, for whatever reason, trying to pickle them leads to random issues with resolving class attributes. This wasn't an issue previously because we were only pickling the stdlib and the six stubs.
  • Because of the above problem, I tried deleting everything under stubs/ and pickling just the stdlib. This seems to work as long as I set options.typeshed = True so stubs/ files missing from the pickle can still be loaded: http://google3/third_party/py/pytype/config.py;l=663;rcl=564564177.
  • out/bin/pytype and out/bin/pytype-single set TYPESHED_HOME to the typeshed submodule. I had to manually set/unset TYPESHED_HOME depending on whether I wanted it point to a modified typeshed for pickling or the full typeshed so that stubs left out of the pickle could still be found.
@rchen152 rchen152 added enhancement cat: stubs and 3p type stubs and third-party types labels Sep 15, 2023
rchen152 added a commit to rchen152/typing that referenced this issue Jan 2, 2024
This change gets the pytype test down to ~30s. It's still considerably
slower than the others, but any further improvements will likely require
untangling the sad state of affairs that is
google/pytype#1501, which will take some time.

* Runs pytype's 'check' option through pytype.io.check_py to skip the
  costly type inference that it does by default.
* Adds a progress bar so that it doesn't look like pytype is hanging.

This generates a few changes to results/pytype/ due to error messages
being formatted and parsed in a different way.
JelleZijlstra pushed a commit to python/typing that referenced this issue Jan 3, 2024
This change gets the pytype test down to ~30s. It's still considerably
slower than the others, but any further improvements will likely require
untangling the sad state of affairs that is
google/pytype#1501, which will take some time.

* Runs pytype's 'check' option through pytype.io.check_py to skip the
  costly type inference that it does by default.
* Adds a progress bar so that it doesn't look like pytype is hanging.

This generates a few changes to results/pytype/ due to error messages
being formatted and parsed in a different way.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cat: stubs and 3p type stubs and third-party types enhancement
Projects
None yet
Development

No branches or pull requests

1 participant