From a5a471aed0b0821892603fc81f98fdea47c95751 Mon Sep 17 00:00:00 2001 From: Josh Dolitsky Date: Fri, 7 Jul 2023 14:56:56 -0500 Subject: [PATCH 1/4] Add blog post to summarize upcoming spec changes Signed-off-by: Josh Dolitsky --- ...-oci-image-and-distribution-specs-v-1-1.md | 147 ++++++++++++++++++ 1 file changed, 147 insertions(+) create mode 100644 content/posts/blog/2023-07-07-summary-of-upcoming-changes-in-oci-image-and-distribution-specs-v-1-1.md diff --git a/content/posts/blog/2023-07-07-summary-of-upcoming-changes-in-oci-image-and-distribution-specs-v-1-1.md b/content/posts/blog/2023-07-07-summary-of-upcoming-changes-in-oci-image-and-distribution-specs-v-1-1.md new file mode 100644 index 0000000..f00368e --- /dev/null +++ b/content/posts/blog/2023-07-07-summary-of-upcoming-changes-in-oci-image-and-distribution-specs-v-1-1.md @@ -0,0 +1,147 @@ +--- +title: Summary of Upcoming Changes in OCI Image and Distribution Specs v1.1 +author: + name: Open Container Initiative + tag: oci +tags: blog +date: 2023-07-07 +--- + +As mentioned in an [earlier post](./2023-01-26-oci-in-2023.md), the +OCI Reference Types Working Group delivered a suggested set changes +in OCI Image and Distribution specs to enable the creation and +discovery of relationships between objects stored in an OCI registry. + +These changes (with some subsequent modifications) will be included in +upcoming minor releases of both OCI Image and Distribution spec. + +# Notes on Versioning and the Term "OCI 1.1" + +The version of the upcoming minor releases of both OCI Image and +Distribution spec happens to be the same: `v1.1.0`. + +This has undoubtedly led to the term "OCI 1.1" which is a bit vague and +incorrect since these changes do not affect the Runtime spec. These versions +of the individual specifications are all technically independent of each other. + +However, when dealing with clients responsible with pushing/pulling from +registries, or registries themselves, hearing claims of "support for OCI 1.1" +can be assumed to be relating to reference types and the changes +outlined in this post. + +# List of Major Changes (3) + +## 1. New Registry API Endpoint for Querying Relationships + +The following HTTP API endpoint has been added to enable +querying for relationships between objects in the registry: +`GET /v2//referrers/` + +The contents of a valid response to this request is a valid OCI +image index containing a list of descriptors which point to the +provided digest: + +```json +{ + "schemaVersion": 2, + "mediaType": "application/vnd.oci.image.index.v1+json", + "manifests": [ + { + "mediaType": "application/vnd.oci.image.manifest.v1+json", + "size": 1234, + "digest": "sha256:a1a1a1...", + "artifactType": "application/vnd.example.sbom.v1", + "annotations": { + "org.opencontainers.artifact.created": "2022-01-01T14:42:55Z", + "org.example.sbom.format": "json" + } + }, + { + "mediaType": "application/vnd.oci.image.manifest.v1+json", + "size": 1234, + "digest": "sha256:a2a2a2...", + "artifactType": "application/vnd.example.signature.v1", + "annotations": { + "org.opencontainers.artifact.created": "2022-01-01T07:21:33Z", + "org.example.signature.fingerprint": "abcd" + } + } + ] +} +``` + +If the response contains a `Link` header +(see [RFC5988](https://www.rfc-editor.org/rfc/rfc5988.html)), +this indicates that there is another page of results which the client +should fetch. + +In addition, clients may provide a filter in querystring in the form +`?artifactType=`. Registries may or may not support this +server-side filtering mechanism. If supported, however, the registry is +required to respond with a header indicating the filter(s) applied: +`OCI-Filters-Applied: artifactType`. Note: the only filter defined at +this time is `artifactType`. + +## 2. New Manifest Field for Establishing Relationships + +A new field called `subject` can now be included on manifests +(including on an index) which points to another object in the registry: + +```json +{ +... + "subject": { + "mediaType": "application/vnd.oci.image.manifest.v1+json", + "digest": "sha256:5b0bca...", + "size": 7682 + }, +... +} +``` + +This is the primary mechanism for linking objects to one another. This +is what is used to determine the proper list returned by the API endpoint +described in #1 above. + +If the registry supports the processing of the `subject` field, it is required +to respond with a header in the following form: `OCI-Subject: sha256:5b0bca...` +( the value being the `digest` field in the `subject` JSON object). + +## 3. Official Guidance on How to Specify Alternatative Artifacts + +For several years, people have been coming up with different ways to +specify alternative (non-image) artifacts in the registry. + +One such method relied on using a custom value for the `config.mediaType` +field on the image manifest. This is the same method used by tools such +as Helm, OPA, WASM etc. + +In the new OCI releases, this behavior has been finally codified as +a valid way of specifying this information. + +If you look again at #1 above, in the sample JSON response you will notice the +new field `artifactType` on descriptors. The value of this field is either a.) the +`artifactType` provided on the manifest when it is pushed or b.) the value of `config.mediaType` if the `artifactType` field is missing. + +TLDR; for new clients pushing artifacts, there is a new top-level field called +`artifactType` which can be used to denote a custom, non-image artifact. If +an object is instead pushed with a custom artifact type in the `config.mediaType` +field (the old way), this value will be surfaced in the `artifactType` field +in the API response. + +# Closing Thoughts + +This post attemts to cover all of the major changes involved in adopting +OCI Image and Distribution specs v1.1, however there are several other +fine-grained details which can only be understood by carefully reading +through the specifications. + +We encourage you to take a look at both of the latest specs to see for yourself: + +- [Image spec](https://github.com/opencontainers/image-spec/blob/main/spec.md) +- [Distribution spec](https://github.com/opencontainers/distribution-spec/blob/main/spec.md) + +As always, if you need assistance, please reach out to the OCI +community via GitHub, Slack, or the mailing list. You can find more +info on how to get in touch with the OCI community +[here](../../community/overview/). From 200c4369c55e4b3f0d2ecddab1ec866d15c0903f Mon Sep 17 00:00:00 2001 From: Josh Dolitsky <393494+jdolitsky@users.noreply.github.com> Date: Fri, 7 Jul 2023 16:12:37 -0500 Subject: [PATCH 2/4] Apply suggestions from code review fix typos found by tianon in review Co-authored-by: Tianon Gravi Signed-off-by: Josh Dolitsky <393494+jdolitsky@users.noreply.github.com> --- ...ing-changes-in-oci-image-and-distribution-specs-v-1-1.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/content/posts/blog/2023-07-07-summary-of-upcoming-changes-in-oci-image-and-distribution-specs-v-1-1.md b/content/posts/blog/2023-07-07-summary-of-upcoming-changes-in-oci-image-and-distribution-specs-v-1-1.md index f00368e..004fa8a 100644 --- a/content/posts/blog/2023-07-07-summary-of-upcoming-changes-in-oci-image-and-distribution-specs-v-1-1.md +++ b/content/posts/blog/2023-07-07-summary-of-upcoming-changes-in-oci-image-and-distribution-specs-v-1-1.md @@ -8,8 +8,8 @@ date: 2023-07-07 --- As mentioned in an [earlier post](./2023-01-26-oci-in-2023.md), the -OCI Reference Types Working Group delivered a suggested set changes -in OCI Image and Distribution specs to enable the creation and +OCI Reference Types Working Group delivered a suggested set of changes +in the OCI Image and Distribution specs to enable the creation and discovery of relationships between objects stored in an OCI registry. These changes (with some subsequent modifications) will be included in @@ -105,7 +105,7 @@ described in #1 above. If the registry supports the processing of the `subject` field, it is required to respond with a header in the following form: `OCI-Subject: sha256:5b0bca...` -( the value being the `digest` field in the `subject` JSON object). +(the value being the `digest` field in the `subject` JSON object). ## 3. Official Guidance on How to Specify Alternatative Artifacts From 8e29c303d9370aadfa1eae1cf39f52d66510f62c Mon Sep 17 00:00:00 2001 From: Josh Dolitsky Date: Fri, 7 Jul 2023 16:32:50 -0500 Subject: [PATCH 3/4] address various review feedback Signed-off-by: Josh Dolitsky --- ...-oci-image-and-distribution-specs-v-1-1.md | 109 ++++++++++-------- 1 file changed, 61 insertions(+), 48 deletions(-) diff --git a/content/posts/blog/2023-07-07-summary-of-upcoming-changes-in-oci-image-and-distribution-specs-v-1-1.md b/content/posts/blog/2023-07-07-summary-of-upcoming-changes-in-oci-image-and-distribution-specs-v-1-1.md index f00368e..5a0db88 100644 --- a/content/posts/blog/2023-07-07-summary-of-upcoming-changes-in-oci-image-and-distribution-specs-v-1-1.md +++ b/content/posts/blog/2023-07-07-summary-of-upcoming-changes-in-oci-image-and-distribution-specs-v-1-1.md @@ -31,7 +31,58 @@ outlined in this post. # List of Major Changes (3) -## 1. New Registry API Endpoint for Querying Relationships +## 1. Official Guidance on How to Create and Store Alternative (Even Non-Container) Artifacts + +For several years, people have been coming up with different ways to +specify alternative (non-image) artifacts in the registry. + +One such method relied on using a custom value for the `config.mediaType` +field on the image manifest. This is the same method used by tools such +as Helm, OPA, WASM etc. + +In the new OCI releases, this behavior has been finally codified as +a valid way of specifying this information. + +If you look at change #3 below, in the sample JSON response from the registry API, +you will notice the new field `artifactType` on descriptors. The value of this field +is either a.) the `artifactType` provided on the manifest when it is pushed or +b.) the value of `config.mediaType` if the `artifactType` field is missing. + +TLDR; for new clients pushing artifacts, there is a new top-level field called +`artifactType` which can be used to denote a custom, non-image artifact. If +an object is instead pushed with a custom artifact type in the `config.mediaType` +field (the old way), this value will be surfaced in the `artifactType` field +in the API response. + +For more information, please see our +[Guidelines for Artifact Usage](https://github.com/opencontainers/image-spec/blob/main/manifest.md#guidelines-for-artifact-usage). + +## 2. New Manifest Field for Establishing Relationships + +A new field called `subject` can now be included on manifests +(including on an index) which points to another object in the registry: + +```json +{ +... + "subject": { + "mediaType": "application/vnd.oci.image.manifest.v1+json", + "digest": "sha256:5b0bca...", + "size": 7682 + }, +... +} +``` + +This is the primary mechanism for linking objects to one another. This +is what is used to determine the proper list returned by the registry +API endpoint described in #3 below. + +If the registry supports the processing of the `subject` field, it is required +to respond with a header in the following form: `OCI-Subject: sha256:5b0bca...` +( the value being the `digest` field in the `subject` JSON object). + +## 3. New Registry API Endpoint for Querying Relationships The following HTTP API endpoint has been added to enable querying for relationships between objects in the registry: @@ -82,53 +133,6 @@ required to respond with a header indicating the filter(s) applied: `OCI-Filters-Applied: artifactType`. Note: the only filter defined at this time is `artifactType`. -## 2. New Manifest Field for Establishing Relationships - -A new field called `subject` can now be included on manifests -(including on an index) which points to another object in the registry: - -```json -{ -... - "subject": { - "mediaType": "application/vnd.oci.image.manifest.v1+json", - "digest": "sha256:5b0bca...", - "size": 7682 - }, -... -} -``` - -This is the primary mechanism for linking objects to one another. This -is what is used to determine the proper list returned by the API endpoint -described in #1 above. - -If the registry supports the processing of the `subject` field, it is required -to respond with a header in the following form: `OCI-Subject: sha256:5b0bca...` -( the value being the `digest` field in the `subject` JSON object). - -## 3. Official Guidance on How to Specify Alternatative Artifacts - -For several years, people have been coming up with different ways to -specify alternative (non-image) artifacts in the registry. - -One such method relied on using a custom value for the `config.mediaType` -field on the image manifest. This is the same method used by tools such -as Helm, OPA, WASM etc. - -In the new OCI releases, this behavior has been finally codified as -a valid way of specifying this information. - -If you look again at #1 above, in the sample JSON response you will notice the -new field `artifactType` on descriptors. The value of this field is either a.) the -`artifactType` provided on the manifest when it is pushed or b.) the value of `config.mediaType` if the `artifactType` field is missing. - -TLDR; for new clients pushing artifacts, there is a new top-level field called -`artifactType` which can be used to denote a custom, non-image artifact. If -an object is instead pushed with a custom artifact type in the `config.mediaType` -field (the old way), this value will be surfaced in the `artifactType` field -in the API response. - # Closing Thoughts This post attemts to cover all of the major changes involved in adopting @@ -136,6 +140,15 @@ OCI Image and Distribution specs v1.1, however there are several other fine-grained details which can only be understood by carefully reading through the specifications. +Some other changes include: + +- `data`` field in the descriptor +- non-distributable layers are deprecated +- zstd compression support +- extension support in distribution-spec +- support for resuming chunked blob push +- anonymous blob mount support in distribution-spec + We encourage you to take a look at both of the latest specs to see for yourself: - [Image spec](https://github.com/opencontainers/image-spec/blob/main/spec.md) From c06515e63862d2114f0644d2582bde487843c269 Mon Sep 17 00:00:00 2001 From: Josh Dolitsky Date: Fri, 7 Jul 2023 16:50:16 -0500 Subject: [PATCH 4/4] Add warning header to bullet list Signed-off-by: Josh Dolitsky --- ...upcoming-changes-in-oci-image-and-distribution-specs-v-1-1.md | 1 + 1 file changed, 1 insertion(+) diff --git a/content/posts/blog/2023-07-07-summary-of-upcoming-changes-in-oci-image-and-distribution-specs-v-1-1.md b/content/posts/blog/2023-07-07-summary-of-upcoming-changes-in-oci-image-and-distribution-specs-v-1-1.md index 210cc9c..ab7c5b4 100644 --- a/content/posts/blog/2023-07-07-summary-of-upcoming-changes-in-oci-image-and-distribution-specs-v-1-1.md +++ b/content/posts/blog/2023-07-07-summary-of-upcoming-changes-in-oci-image-and-distribution-specs-v-1-1.md @@ -149,6 +149,7 @@ Some other notable changes include the following: - extension support in distribution-spec - support for resuming chunked blob push - anonymous blob mount support in distribution-spec +- a `Warning` header for surfacing user-facing client warnings We encourage you to take a look at both of the latest specs to see for yourself: