Prebuilt Ruby ADR #49
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -3,43 +3,78 @@ | |
| ***Status***: proposed | ||
|
|
||
| ## Context | ||
| Currently, only a few versions of Ruby are available at runtime prebuilt by the [virtual-environment images](https://github.com/actions/virtual-environments) as part of the hosted build VM tools cache. The setup-ruby action only supports using what's in the cache and does not support acquiring other versions at runtime. | ||
|
|
||
| This introduces two problems: | ||
|
|
||
| 1. New version latency: When new versions are released, they are not available for use until a new image is released. | ||
| 2. Versions can be removed. The current image policy is latest of each minor version. Even if that is increased to n - 1, [builds that were working can break](https://github.com/actions/virtual-environments/issues/281). And since images roll through scale units with the readme only being updated once it's everywhere, it's hard to plan for the change and react. | ||
|
|
||
| This is in contrast to other setup actions like setup-node where the cache is an optimization but supports pulling any version by a specified semver at runtime. | ||
|
|
||
| Ruby officially only supports source code distribution. There are no "official" pre built distributions since there are system and environmental dependencies. | ||
|
|
||
| ## Goals | ||
|
|
||
| - Offer as many versions as possible by platforms | ||
| - Offer popular Ruby variants like JRuby | ||
| - Consistent with other setup-xxx actions patterns and Actions workflows. | ||
| - Ideally works across all supported scenarios including self-hosted runners. | ||
| - Ideally works on all the supported runner platforms (windows, mac-os, all linux variants) | ||
| - Ideally works with actions container scenarios | ||
|
|
||
| ## Options | ||
|
|
||
| ### Download Prebuilt Rubies on Misses | ||
|
|
||
| Other tools like offer a [distribution](https://nodejs.org/dist/) which offers a [queryable endpoint](https://nodejs.org/dist/index.json) which allows the desired version [by a version spec which is semver](https://github.com/actions/setup-node#setup-node). | ||
|
|
||
| Other CI services like Travis support pulling a wide variety of [ruby versions offered here](http://rubies.travis-ci.org/). | ||
|
|
||
| NOTE: The prebuilt rubies offered will only be supported for use by GitHub Actions. | ||
|
|
||
| The scripts and tooling to build the Rubies will be at `actions/build-ruby` and it will be consumed by the existing action `actions/setup-ruby`. Since the capabilities are additive, there's no need to create a `v2`. There is no compat break. | ||
|
|
||
| A specific version of a Ruby will be offered as an individual build-ruby repo release similar to how the [actions runner exposes versions](https://github.com/actions/runner/releases/tag/v2.164.0) where each version offers n platforms. This is also similar to [Travis individual versions](http://rubies.travis-ci.org/ubuntu/18.04/s390x/ruby-2.6.5). | ||
|
|
||
| **Pros** | ||
|
|
||
| On a toolkit cache miss, the first build is slower but not that slow since it's pre-built. | ||
|
There was a problem hiding this comment. It downloads in less than 5 seconds, I don't think any strategy would be faster (as long as there are not all versions in the base image). There was a problem hiding this comment. It’s only the first build that’s slower. And it’s ultimately not about pure speed. It’s about the huge breadth of platforms, self hosted and GHES scenarios this affords. In addition alignment with the only officially supported dist model. This is early and we need numbers and analysis. To early to make a call |
||
|
|
||
| **Cons** | ||
|
|
||
| Will not work with self-hosted scenarios | ||
| Will not work across all platforms the runner supports | ||
|
There was a problem hiding this comment. Could you explain why it would not work on all platforms? There was a problem hiding this comment. It effectively won't work against all linux variants (rh, ubuntu, arch, arm etc.) times all versions (18.04, 18.10 etc) which our runner currently supports. From discussions, this is why Ruby only officially supports building from source - the environmental and dependency differences. That is unless you want to build an impossible matrix of platform, all versions, arch etc. prebuilt. There was a problem hiding this comment. Ah right, I understood that as "all the platforms GitHub-hosted runners support". |
||
| Will not work across various container scenarios | ||
| Only supported on hosted VMs | ||
|
|
||
| ### Build and Cache Misses at Runtime | ||
|
|
||
| [rbenv/ruby-build](https://github.com/rbenv/ruby-build) offers a convenient way to build at runtime. | ||
|
|
||
| Build on cache miss of an exact version. Offer an option to cache using the upcoming [toolkit cache lib](https://github.com/actions/toolkit). There are plans to factor out the caching logic in the [actions/cache](https://github.com/actions/cache) so other actions can take advantage of caching services. | ||
|
|
||
| **Pros** | ||
|
|
||
| Aligns well with the only supported Ruby distribution ... source code. | ||
| Works on self-hosted, | ||
|
|
||
| **Cons** | ||
|
|
||
| The first build on a miss will be slower but will not fail. | ||
|
There was a problem hiding this comment. That strategy would be similar to https://github.com/clupprich/ruby-build-action. There was a problem hiding this comment. It would also do so many duplicate builds of the same version in many repositories, i.e., I think it would waste resources significantly. There was a problem hiding this comment. See comment below. We need to look at first build per org and only on cache miss ( folks that bound to specific version ). The mainline minor binding bypasses all of this with vm cache of latest versions There was a problem hiding this comment. After the miss on hosted (single use), it would be cached in the same data center for future builds and very fast downloads. On self-hosted, it would live past that job and be in the machines cache and instantly use it. There was a problem hiding this comment. remember that the hosted VMs are caching the latest (and possibly n -1 ) of every minor version so this is only first build for folks that pin back to an old specific version. |
||
| Ingress / Egress costs for storage | ||
|
|
||
| ## Prebuilt Implementation Details | ||
|
|
||
| Option 1: Download Virtual Environment Prebuilt Rubies | ||
|
|
||
| The virtual-environments team is working on opening the tool cache prebuilt versions for download. This is the best download option since it would be the exact same binaries on a cache miss after a version is removed from the images. | ||
|
|
||
| Option 2: GitHub GPR Universal Package | ||
|
|
||
| The `actions/build-ruby` repo offers packages for [example](https://github.com/actions/setup-ruby/packages). | ||
|
There was a problem hiding this comment. What's the difference between packages and releases? From https://github.com/features/packages#pricing I can see 500MB (total) could quickly become too tight for hosting all Ruby versions. Releases don't have such restrictions AFAIK (just no single file >2GB). There was a problem hiding this comment. I'm discussing with packaging folks. But essentially, packaging has full semver support without putting that burden on the action or some other index json or api. I think if we went the packaging route, then each distribution, per version, per arch would be an independent package (how big is a discrete ruby? 10 - 20 MB?) There was a problem hiding this comment. See https://github.com/eregon/ruby-install-builder/releases/tag/builds-bundler1 I think Rubyists are BTW more used to |
||
|
|
||
| Option 3: GitHub Release | ||
|
|
||
| The `actions/build-ruby` repo offers releases for [example](https://github.com/actions/runner/releases/tag/v2.164.0) | ||
|
|
||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you add TruffleRuby in the
Goalsabove since it's removed here?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm focusing this change request on (1) getting versions of Ruby at runtime and (2) self-hosted scenarios for on-prem and GHES scenarios (hi pri right now). We can write another change request on adding variations. We still haven't done a good job with Ruby ...