Support package release dependencies in multi-Python packages #372

davidbrochart · 2022-09-05T16:16:57Z

Closes #371.

This fails with check-python, extract-release... because the dependent packages have not been released on PyPI yet.
Maybe there is a way to add the just-created (local) releases in the package providers?

blink1073 · 2022-09-06T02:23:03Z

Yeah, I finally figured out how to test local changes in #363. Perhaps we should wait for that before doing this?

davidbrochart · 2022-09-06T06:08:23Z

Perhaps we should wait for that before doing this?

Sounds good. In the meantime, I can always achieve what this PR does in two steps:

make a first release with version x.y.z.
pin sub-packages with ==x.y.z, and release again.

blink1073 · 2022-09-12T01:28:36Z

Okay, you can rebase now that #363 is merged.

codecov-commenter · 2022-09-12T07:37:37Z

Codecov Report

Merging #372 (2651a46) into main (2cd3e74) will decrease coverage by 0.32%.
The diff coverage is 58.33%.

@@            Coverage Diff             @@
##             main     #372      +/-   ##
==========================================
- Coverage   82.93%   82.60%   -0.33%     
==========================================
  Files          21       21              
  Lines        2866     2898      +32     
  Branches      374      382       +8     
==========================================
+ Hits         2377     2394      +17     
- Misses        356      368      +12     
- Partials      133      136       +3

Impacted Files	Coverage Δ
jupyter_releaser/lib.py	`78.68% <51.61%> (-2.15%)`	⬇️
jupyter_releaser/cli.py	`91.00% <100.00%> (+0.09%)`	⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

davidbrochart · 2022-09-12T07:58:33Z

I rebased an tried to release jupyverse with this branch, but it failed in check-changelog with:

fatal: malformed object name refs/heads/release_deps

Any idea?

blink1073 · 2022-09-12T13:30:27Z

Ah, I see that we need to store the RH_REF in the metadata file and use that.

blink1073 · 2022-09-12T13:31:59Z

#375

davidbrochart · 2022-09-12T14:49:55Z

I rebased on #375, but it still fails.

davidbrochart · 2022-09-12T15:36:02Z

With the latest changes in #375, it fails at "Publish Release" in "Prepare Environment":

ValueError: No release found for url https://github.com/jupyter-server/jupyverse/releases/tag/untagged-3ab212a9b6e7bceff24f

But it could be because of my changes.

blink1073 · 2022-09-12T16:00:10Z

I think it is because the original draft release was over one day old and got pruned. You'll have to start over.

blink1073 · 2022-09-12T16:00:50Z

Oh, I see it is still there. Maybe try with the full url: https://github.com/jupyter-server/jupyverse/releases/tag/untagged-43adf70df0043151e18b

davidbrochart · 2022-09-12T16:28:48Z

Still fails:

--------------------------------------------------
bump-version
--------------------------------------------------


/home/runner/work/jupyter_releaser/jupyter_releaser/jupyter_releaser/util.py:174: UserWarning: Using deprecated setup.py invocation
  warnings.warn("Using deprecated setup.py invocation")
0.0.39
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.10.6/x64/bin/tbump", line 8, in <module>
    sys.exit(main())
  File "/opt/hostedtoolcache/Python/3.10.6/x64/lib/python3.10/site-packages/tbump/cli.py", line 245, in main
    run(args)
  File "/opt/hostedtoolcache/Python/3.10.6/x64/lib/python3.10/site-packages/tbump/cli.py", line 141, in run
    run_bump(arguments)
  File "/opt/hostedtoolcache/Python/3.10.6/x64/lib/python3.10/site-packages/tbump/cli.py", line 162, in run_bump
    bump(bump_options, arguments.operations)
  File "/opt/hostedtoolcache/Python/3.10.6/x64/lib/python3.10/site-packages/tbump/cli.py", line 172, in bump
    config_file = get_config_file(
  File "/opt/hostedtoolcache/Python/3.10.6/x64/lib/python3.10/site-packages/tbump/config.py", line 222, in get_config_file
    res.get_config()
  File "/opt/hostedtoolcache/Python/3.10.6/x64/lib/python3.10/site-packages/tbump/config.py", line 68, in get_config
    parsed = self.get_parsed()
  File "/opt/hostedtoolcache/Python/3.10.6/x64/lib/python3.10/site-packages/tbump/config.py", line [103](https://github.com/davidbrochart/jupyter_releaser/runs/8310556740?check_suite_focus=true#step:4:106), in get_parsed
    return tool_section.value  # type: ignore
AttributeError: 'dict' object has no attribute 'value'. Did you mean: 'values'?

fcollonval · 2022-09-12T16:35:13Z

I don't think this is related to the error from David above.

When trying to release JupyterLab, the draft PR got converted and populated with all to-be-published assets see draft release. But the publication failed because the target GitHub repository was not pass to the sub-action Publish of the Full release workflow: https://github.com/fcollonval/jupyter_releaser/runs/8308943651?check_suite_focus=true

blink1073 · 2022-09-12T18:09:43Z

For now you can create a tag for 519b8dd on your fork, and select that tag when running your workflows, you can get unblocked.

blink1073 · 2022-09-12T20:58:02Z

@davidbrochart I think the error you are seeing is from the multi-dependency handling.

davidbrochart · 2022-09-13T06:48:26Z

I'm not sure, because there is the same issue in the "Check Release" action of this PR.

blink1073 · 2022-09-13T19:49:51Z

@davidbrochart I'd recommend switching away from tbump in general. See #374 for an example of how to switch to hatchling version instead.

davidbrochart · 2022-09-16T08:45:29Z

This PR now fails at check-python:

Checking dist/fps_login-0.0.41-py3-none-any.whl: PASSED
Checking dist/fps_kernels-0.0.41.tar.gz: PASSED
Checking dist/jupyverse-0.0.41.tar.gz: PASSED
ERROR: Could not find a version that satisfies the requirement fps-kernels==0.0.41 (from jupyverse) (from versions: 0.0.0, 0.0.1, 0.0.2, 0.0.3, 0.0.4, 0.0.11, 0.0.12, 0.0.13, 0.0.15, 0.0.16, 0.0.17, 0.0.18, 0.0.19, 0.0.21, 0.0.22, 0.0.24, 0.0.25, 0.0.26, 0.0.27, 0.0.28, 0.0.29, 0.0.30, 0.0.31, 0.0.32, 0.0.33, 0.0.34, 0.0.35, 0.0.36, 0.0.37, 0.0.38, 0.0.40)

Here jupyverse depends on the just-released fps-kernels==0.0.41, but it's not taken into account.
Should I just skip check-python?

davidbrochart · 2022-09-16T08:51:47Z

Should I just skip check-python?

If I do so it fails at extract-release:

41898282+github-actions[bot]@users.noreply.github.com
Git checkout already exists
Already on 'main'
Your branch is up to date with 'origin/main'.
refs/heads/main
WARNING: jupyverse 0.0.41 does not provide the extra 'dev'
ERROR: Could not find a version that satisfies the requirement fps-contents==0.0.41 (from jupyverse[dev,test]) (from versions: 0.0.0, 0.0.1, 0.0.2, 0.0.3, 0.0.4, 0.0.5, 0.0.11, 0.0.12, 0.0.13, 0.0.15, 0.0.16, 0.0.17, 0.0.18, 0.0.19, 0.0.21, 0.0.22, 0.0.24, 0.0.25, 0.0.26, 0.0.27, 0.0.28, 0.0.29, 0.0.30, 0.0.31, 0.0.32, 0.0.33, 0.0.34, 0.0.35, 0.0.36, 0.0.37, 0.0.38, 0.0.40)
ERROR: No matching distribution found for fps-contents==0.0.41

davidbrochart · 2022-09-22T08:03:01Z

@blink1073 this still fails. Any idea what can be done?

blink1073 · 2022-09-22T15:46:08Z

You'd have to publish the packages to the local pypi server before calling check-python, and then make sure you are running the local pypi server during check-python, or skip check-python.

jtpio · 2023-09-20T06:45:09Z

You'd have to publish the packages to the local pypi server before calling check-python, and then make sure you are running the local pypi server during check-python, or skip check-python.

The releaser has a utility function to start the local PyPI server:

jupyter_releaser/jupyter_releaser/python.py

Lines 183 to 196 in a4de95e

    
           def start_local_pypi(): 
        
               """Start a local PyPI server""" 
        
               temp_dir = TemporaryDirectory() 
        
               cmd = f"pypi-server run -p 8081  -P . -a . -o  -v {temp_dir.name}" 
        
               proc = Popen(shlex.split(cmd), stdout=PIPE)  # noqa 
        
               # Wait for the server to start 
        
               while True: 
        
                   assert proc.stdout is not None 
        
                   line = proc.stdout.readline().decode("utf-8").strip() 
        
                   util.log(line) 
        
                   if "Listening on" in line: 
        
                       break 
        
               atexit.register(proc.kill) 
        
               atexit.register(temp_dir.cleanup)

There doesn't seem to be a way to stop it though. So not sure if starting it before check-python would then cause issues when the publish-assets step tries to start it again.

davidbrochart marked this pull request as draft September 5, 2022 16:17

davidbrochart marked this pull request as ready for review September 6, 2022 03:58

davidbrochart force-pushed the release_deps branch 2 times, most recently from 509a7d2 to 2651a46 Compare September 12, 2022 07:25

davidbrochart force-pushed the release_deps branch from 2651a46 to 15dc994 Compare September 12, 2022 14:44

davidbrochart force-pushed the release_deps branch from 15dc994 to 89499ad Compare September 12, 2022 15:24

blink1073 added the enhancement New feature or request label Sep 12, 2022

davidbrochart force-pushed the release_deps branch from 89499ad to 2141efd Compare September 12, 2022 16:13

blink1073 mentioned this pull request Sep 12, 2022

Get repository from release url if given #377

Merged

davidbrochart force-pushed the release_deps branch 3 times, most recently from 1b6ef17 to 58ae52a Compare September 16, 2022 08:18

Support package release dependencies in multi-Python packages

39c3044

davidbrochart force-pushed the release_deps branch from 58ae52a to 39c3044 Compare September 22, 2022 07:23

jtpio mentioned this pull request Mar 16, 2023

Allow skipping pip install step in check-python #499

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support package release dependencies in multi-Python packages #372

Support package release dependencies in multi-Python packages #372

davidbrochart commented Sep 5, 2022

blink1073 commented Sep 6, 2022 •

edited

Loading

davidbrochart commented Sep 6, 2022

blink1073 commented Sep 12, 2022

codecov-commenter commented Sep 12, 2022

davidbrochart commented Sep 12, 2022

blink1073 commented Sep 12, 2022

blink1073 commented Sep 12, 2022

davidbrochart commented Sep 12, 2022

davidbrochart commented Sep 12, 2022 •

edited

Loading

blink1073 commented Sep 12, 2022

blink1073 commented Sep 12, 2022

davidbrochart commented Sep 12, 2022

fcollonval commented Sep 12, 2022

blink1073 commented Sep 12, 2022 •

edited

Loading

blink1073 commented Sep 12, 2022

davidbrochart commented Sep 13, 2022

blink1073 commented Sep 13, 2022

davidbrochart commented Sep 16, 2022

davidbrochart commented Sep 16, 2022

davidbrochart commented Sep 22, 2022

blink1073 commented Sep 22, 2022

jtpio commented Sep 20, 2023

Support package release dependencies in multi-Python packages #372

Are you sure you want to change the base?

Support package release dependencies in multi-Python packages #372

Conversation

davidbrochart commented Sep 5, 2022

blink1073 commented Sep 6, 2022 • edited Loading

davidbrochart commented Sep 6, 2022

blink1073 commented Sep 12, 2022

codecov-commenter commented Sep 12, 2022

Codecov Report

davidbrochart commented Sep 12, 2022

blink1073 commented Sep 12, 2022

blink1073 commented Sep 12, 2022

davidbrochart commented Sep 12, 2022

davidbrochart commented Sep 12, 2022 • edited Loading

blink1073 commented Sep 12, 2022

blink1073 commented Sep 12, 2022

davidbrochart commented Sep 12, 2022

fcollonval commented Sep 12, 2022

blink1073 commented Sep 12, 2022 • edited Loading

blink1073 commented Sep 12, 2022

davidbrochart commented Sep 13, 2022

blink1073 commented Sep 13, 2022

davidbrochart commented Sep 16, 2022

davidbrochart commented Sep 16, 2022

davidbrochart commented Sep 22, 2022

blink1073 commented Sep 22, 2022

jtpio commented Sep 20, 2023

blink1073 commented Sep 6, 2022 •

edited

Loading

davidbrochart commented Sep 12, 2022 •

edited

Loading

blink1073 commented Sep 12, 2022 •

edited

Loading