Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to scheme version 4.0.0 #157

Open
wants to merge 39 commits into
base: master
Choose a base branch
from

Conversation

deathaxe
Copy link

@deathaxe deathaxe commented Aug 19, 2022

This PR introduces required changes to support scheme 4.0.0 for Package Control 4.0

Overview

  1. Updates database scheme to
    • replace dependency by library
    • add python_versions key
    • drop load_order
  2. Provides a setup/sql/migrations/library.sql script to update production database.
    Tested by creating a DB with 3.0.0 scheme, fetching some libs/packages and finally running library.sql
  3. Updates related controllers/models/views/libs/tasks to make use of new database scheme.
  4. Pull in package_control from https://github.com/wbond/package_control/tree/four-point-oh
  5. Add a generate_channel_v4_json task and update generate_channel_v3_json to only provide PC.

Why channel_v4.json

Package Control 4.0 handles both 3.0.0 and 4.0.0 schemes well. Just requires 4.0.0 to pull dependencies/libraries for python 3.8. So its safe (and required) to publish it before this PR is published to packagecontrol.io

If we would keep going with channel_v3.json and just update its scheme version no Package Control 3.x client was able to fetch packages from official sources due to mismatching scheme version. We would end up in users needing manually update Package Control, which may cause some complains.

By providing a dummy channel_v3.json with only Package Control in it, all clients have a chance to update to 4.0 even after packagecontrol.io is updated.

In the meanwhile Package Control 4.0 can be preconfigured to pull in channel_v3.json and channel_v4.json, so it keeps going with v3 until packagecontrol.io is updated.

This should ensure a smooth transition from 3.x to 4.0.

Required changes to channel can be found at wbond/package_control_channel#8713

- rename dependency to library
- drop load_order from libraries table
- add python_versions to library_releases table

- add migration scheme file
commit aede06d414bb9a3a6f710690f2f3f698c7303beb
Scheme 4.0.0 is incompatible with 3.0.0 and will cause current PC 3.4.x
releases to completely fail. They wouldn't even be able to fetch data
to upgrade itself.

Hence a dummy channel_v3.json needs to be served for all clients not
updated before new packagecontrol.io release goes online.

PC 4.0 can ship with both channel_v3 and _v4 pre-configured to ensure a
smooth transition to new scheme version.

The only thing changing for end users is availability of libraries for
python 3.8 then.
This change caused explicit data from JSON to be overwritten with
fallback defaults.

see: wbond/package_control@a8262a9
__all__ is a list of names (strings) of exported modules.
@deathaxe
Copy link
Author

For this PR to work #155 and #156 should be merged before.

This commit enables packagecontrol.io to optionally store packages'
python_versions, which are declared within a repository.json

This way old packages, which are proofed compatible with python 3.8, can
be pushed to new plugin_host without touching their github repo.

A local .python-version file always overrides upstream settings.
Provide fallbacks if release keys are not found. Primarily possible only for
`python_versions` as it is optional for packages.
@deathaxe
Copy link
Author

Got it up and running on Debian 12 with python 3.11 and PostgreSQL 15 with a little patch to gears library.

This commit

1. moves asset storage functionality to a dedicated module as it might also be
   used to provide compressed stylesheets or javascript assets in future.

2. fixes an issue which caused generated channel.json files to look different
   after each poll. This causes the file to change each 10mins even if content
   hasn't changed. This degrades chance to return 304 so clients could re-use
   local cache.

   To achieve it, json objects content is sorted by key and a hash file is used
   to check, whether content has changed since last run.
@adamlaska
Copy link

!

@jwortmann jwortmann mentioned this pull request Jan 27, 2024
15 tasks
@toddwildey
Copy link

@wbond Can we merge this change into Package Control? Do we require an ownership transfer first?

I would be willing to host/contribute maintenance to packagecontrol.io to get this merged.

@deathaxe
Copy link
Author

deathaxe commented Jun 9, 2024

Package Control is critical for Sublime Text package ecosystem's security.

It should therefore be hosted and administrated by trusted and well-known actors, only, especially after recently disclosed XS supply chain attack, which targeted compromizing SSH connections of widely used linux distributions.

@toddwildey
Copy link

toddwildey commented Jun 10, 2024

  1. I've been using Sublime Text for almost ~15 years.
  2. I currently operate as a Senior Software Development Engineer at Amazon professionally, where I've worked for full time for nearly a decade.
    1. Compromising security for Package Control or Sublime Text is absolutely not in my best interest professionally.
    2. More so, I can help establish zero-trust mechanisms that require trusted maintainers to curate the channel.
  3. I am happy for someone else to maintain the packagecontrol.io repository, but progress on the repository appears to have stalled out for the last 3 years due to @wbond's effective departure. It is critical for the success of Sublime Text and all the developers who use it for Package Control to not be blocked by one person.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants