-
-
Notifications
You must be signed in to change notification settings - Fork 5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ENH/BLD: Add install tags for tests
#20712
Conversation
This PR doesn't have a description, should I assume what you're trying to do is build on Ralf's work in NumPy to install the tests as a separate "package" for the purposes of PyPI releases? Or something else? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks @czgdp1807, this looks very clean now and all seems to work as advertised.
$ python -m build -wnx -Cinstall-args=--tags=runtime,python-runtime,devel
$ mv scipy-1.14.0.dev0-cp312-cp312-linux_x86_64.whl scipy-1.14.0.dev0-cp312-cp312-linux_x86_64-no-tests.whl
$ python -m build -wnx
$ ls -l dist/*.whl
... 22140538 ... dist/scipy-1.14.0.dev0-cp312-cp312-linux_x86_64-no-tests.whl
... 25101662 ... dist/scipy-1.14.0.dev0-cp312-cp312-linux_x86_64.whl
So the produced wheels are ~12% smaller. In addition, this will help people who want to strip out the tests, for example:
- Conda-forge: fragile hacks in build scripts to remove tests: https://github.com/conda-forge/scipy-feedstock/blob/39c367144d7519ad131c7fdb073bf0d2476a374c/recipe/build-output.bat#L25-L46
- Pyodide: has generic post-processing to strip tests, but that is less reliable since it's filename-based, requiring things like: https://github.com/pyodide/pyodide/blob/0c2dea7882e1cda56da7ee31c73e286461451cd5/packages/scipy/meta.yaml#L91-L93
- AWS Lamba layer for SciPy: Scipy lambda layer for 3.9 and 3.10 keithrozario/Klayers#360
A wheel for separately installable tests can be produced with a one-line diff:
diff --git a/pyproject.toml b/pyproject.toml
index ab241ac23..3fc1eecfd 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -32,7 +32,7 @@ requires = [
]
[project]
-name = "scipy"
+name = "scipy-tests"
version = "1.14.0.dev0"
# TODO: add `license-files` once PEP 639 is accepted (see meson-python#88)
# at that point, no longer include them in `py3.install_sources()`
$ python -m build -wnx -Cinstall-args=--tags=tests
$ ls -l dist/*.whl
... 2966947 ... dist/scipy_tests-1.14.0.dev0-cp312-cp312-linux_x86_64.whl
There is another set of test files to strip, which is test extension modules:
$ ls -l build/scipy/*/*.so | rg test
-rwxr-xr-x 1 rgommers rgommers 28664 17 mei 18:04 build/scipy/integrate/_test_multivariate.cpython-312-x86_64-linux-gnu.so
-rwxr-xr-x 1 rgommers rgommers 270968 17 mei 18:04 build/scipy/integrate/_test_odeint_banded.cpython-312-x86_64-linux-gnu.so
-rwxr-xr-x 1 rgommers rgommers 151456 17 mei 18:04 build/scipy/io/_test_fortran.cpython-312-x86_64-linux-gnu.so
-rwxr-xr-x 1 rgommers rgommers 52912 17 mei 18:04 build/scipy/_lib/_test_ccallback.cpython-312-x86_64-linux-gnu.so
-rwxr-xr-x 1 rgommers rgommers 158752 21 mei 13:45 build/scipy/_lib/_test_deprecation_call.cpython-312-x86_64-linux-gnu.so
-rwxr-xr-x 1 rgommers rgommers 92272 21 mei 13:45 build/scipy/_lib/_test_deprecation_def.cpython-312-x86_64-linux-gnu.so
-rwxr-xr-x 1 rgommers rgommers 31336 17 mei 18:04 build/scipy/ndimage/_ctest.cpython-312-x86_64-linux-gnu.so
-rwxr-xr-x 1 rgommers rgommers 386480 21 mei 13:45 build/scipy/ndimage/_cytest.cpython-312-x86_64-linux-gnu.so
-rwxr-xr-x 1 rgommers rgommers 1095216 21 mei 13:45 build/scipy/special/_test_internal.cpython-312-x86_64-linux-gnu.so
Could you change the install tags on those targets to tests
as well?
I think for |
Yeah don't worry about that. It's not something we have to do anything about before starting to actually produce wheels without tests included. And two |
All ready on this PR. |
Found one more sizable set of files to remove, pushed a fix. With that, we're at:
Once we remove the already-deprecated |
Okay, let's declare victory here - in it went! Nice work @czgdp1807. Cc @h-vetinari for visibility: for the next release it may be nice to start using this in conda-forge to produce |
That's great news; this took a lot of manual processing on the conda-forge that will be great to get rid of. Thanks for the heads-up and the work here! |
gh-20812 includes docs for this new feature |
Same as numpy/numpy#26274
Reference issue
What does this implement/fix?
Additional information