Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor image download in rhizome side & add optional image version. #1524

Merged
merged 1 commit into from May 7, 2024

Conversation

pykello
Copy link
Contributor

@pykello pykello commented Apr 30, 2024

This is in preparation to make boot images versioned. Some of the reasons to do this includes:

  • Allow installing a newer version of an image without breaking existing VMs which have references to the older versions.
  • Make storage snapshots easier: we know exactly which base image is being used, and we can just snapshot the diff.
  • Making correlation of failures to bad OS images easier.

Current patch:

  • Moves almost all of the image download logic to the standalone class BootImage.
  • Adds optional version and sha256sum parameters to images. If these are null or empty, the behaviour is exactly same as before.
  • Uses BootImage in VmSetup and download-boot-image.

If an image is versioned, it'll be stored as /var/storage/images/$name-$version.raw. Legacy unversioned images are still supported and are stored at /var/storage/images/$name.raw.

@enescakir
Copy link
Member

I was planning to implement fixed versioning for the runner image. This PR will be very helpful. Thank you.

@pykello pykello force-pushed the pykello/image_versioning_rhizome branch from 07df48c to 8d80e38 Compare May 1, 2024 07:21
@pykello pykello force-pushed the pykello/image_versioning_rhizome branch 3 times, most recently from 962df73 to 306554f Compare May 1, 2024 07:46
@pykello pykello requested a review from enescakir May 1, 2024 07:50
@pykello pykello force-pushed the pykello/image_versioning_rhizome branch from 306554f to 313676e Compare May 1, 2024 17:16
pykello added a commit that referenced this pull request May 2, 2024
In #1524 we changed rhizome to
be able to download versioned boot images. In this change, we actually
use that infrastructure.

To download a versioned boot image, one can do:

```
> vmh.download_boot_image("ubuntu-jammy", version: "20240319")
```

After this has been done, all VMs on that host which want to use the
`ubuntu-jammy` image will use the latest boot image.

This allows us to be able to download new image versions without
impacting previous VMs at all. Previous VMs will continue to use the
older images.

As follow up items:
* VmHost setup will include downloading some default images (e.g.
 `ubuntu-jammy`) with explicit. vm_setup won't download the images
  automatically.
* We will enforce to use versioned images always
* Add a program to remove unused images
Copy link
Member

@enescakir enescakir left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@byucesoy byucesoy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good. My only concern is the difficulty of updating hashes. I think it would be good to have a simpler way to store the list of hashes; such as passing them during first download.

rhizome/host/lib/boot_image.rb Outdated Show resolved Hide resolved
This is in preparation to make boot images versioned. Some of the
reasons to do this includes:

* Allow installing a newer version of an image without breaking existing
  VMs which have references to the older versions.
* Make storage snapshots easier: we know exactly which base image is
  being used, and we can just snapshot the diff.
* Making correlation of failures to bad OS images easier.

Current patch:
* Moves almost all of the image download logic to the standalone class
  BootImage.
* Adds optional version and sha256sum parameters to images. If these are
  null or empty, the behaviour is exactly same as before.
* Uses BootImage in VmSetup and `download-boot-image`.

If an image is versioned, it'll be stored as
`/var/storage/images/$name-$version.raw`. Legacy unversioned images are
still supported and are stored at `/var/storage/$name.raw`.
@pykello pykello force-pushed the pykello/image_versioning_rhizome branch from d54aa7a to cd545b0 Compare May 7, 2024 17:28
@pykello pykello merged commit 33f87d0 into main May 7, 2024
7 checks passed
@pykello pykello deleted the pykello/image_versioning_rhizome branch May 7, 2024 19:34
@github-actions github-actions bot locked and limited conversation to collaborators May 7, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants