Only the legacy docker format tar is supported

[vaskozl]

Inspect fails on many valid container tarballs I would expect it to work on.

So far, I've tried inspect and failed to get inspect with skopeo tarballs and those generated with buildx. E.g.

skopeo copy docker://alpine:latest docker-archive://image.tar the image inspector fails with an exception on this line:
https://github.com/blackducksoftware/hub-imageinspector-lib/blob/master/src/main/java/com/synopsys/integration/blackduck/imageinspector/image/docker/manifest/DockerManifest.java#L85

This is because a slash in the manifest layers is expected - something that is only present in the legacy spec, see this skopeo issue explaining it.

I also get a null pointer exception when the tar is exported with buildkit via docker buildx's tar outpout, which is official way to get a tarball when using buildkit/buildx.

The reason this is problematic is that docker save requires the docker daemon to be running. Running the docker daemon is not possible in environments where only normal unprivileged containers are allowed. We can't allow privileged containers and currently blackduck only seems to support legacy docker save tarballs which we can't easily use it in Gitlab CI.

The different manifests.json for comparison:

manifest.json with docker save, which the inspector succeeds on:

[
  {
    "Config": "9c9c8fde770e6dba7db819d333ec653501bb4f7da7de873255574c64e50d8fcc.json",
    "RepoTags": null,
    "Layers": [
      "5be2576eb46b8d1aadb38945eb5fa1b4d1641e7074293c83ee9c45bee7eb71d1/layer.tar",
      "73abe3705c93199fcaf80f557a7648cc3c4faca2c3ef8a6b521d13ee87f4ab16/layer.tar",
      "17113dbc5a00432ff552acda6b9fa6382b40539ded5c1a5312b1f8159fc3e39c/layer.tar",
      "d8ab616399c1e0cf2a2b4ccc3e4ed884ef45e15df8a243541f709f143a3da7cb/layer.tar",
...

skopeo with the docker-archive, setting which fails with "out of bounds" due to the lib looking for slash:

[
  {
    "Config": "9c9c8fde770e6dba7db819d333ec653501bb4f7da7de873255574c64e50d8fcc.json",
    "RepoTags": [],
    "Layers": [
      "0ad3ddf4a4ce0fc9d65021897541c99e5497a11ced2419016147267df61732d3.tar",
      "2f7b7ce167b3050f45c6d6da4d212766dc75fcabe73a1fccf3f1aa48022498b5.tar",
      "7677e51e347aed6a8ee916111e2a2cd7bc07d2a9d9082bb2ed46c63192eca25d.tar",
      "ccaa8e196143f23c40caac3b279ef719cc8c09b531109fc3dfa44c30521fd558.tar",
...
    ]
  }
]

docker buildx based produced tar (fails with NullPointerException):

[
  {
    "Config": "blobs/sha256/9c9c8fde770e6dba7db819d333ec653501bb4f7da7de873255574c64e50d8fcc",
    "RepoTags": null,
    "Layers": [
      "blobs/sha256/4be315f6562fccf08fd6c749557e31e45ab6d987370e20e2c4933ddb04ddd5ff",
      "blobs/sha256/96866e173b1b6b5665940125744372af9c6c534ffbe034a9187670b5e04e3950",
      "blobs/sha256/314d33570bef2c2d4fd28455d65c9646eece372a093ee675b1370fc58fa9982e",
...
    ]
  }
]

[gopannabd]

@vaskozl Thank you for submitting this issue. We will respond after an internal review.

[vaskozl]

I note that the lack of schema2-like config+blob formats prevents us from updating our build infrastructure and may lead to us not renewing our blackduck subscription.

[vaskozl]

After discovering that "mediaType":"application/vnd.oci.image.manifest.v1+json" archives produced by podman save work and digging into the reason why, I can now provide some more insight into the exact issue.

New versions of docker builder docker buildx build -o type=docker,dest=- . > myimage.tar produce:

* blobs/
* index.json
* manifest.json
* oci-layout

The matcher which checks whether the image is the old repositories style docker format runs first, and check for a manifest.json and incorrectly assumes that the image is in the legacy format, despite it having an oci-layout file:

hub-imageinspector-lib/src/main/java/com/synopsys/integration/blackduck/imageinspector/image/docker/DockerImageFormatMatchesChecker.java

Line 24 in 3c48b8a

logger.info("Docker image manifest.json file found");

(Note also the wrong logger.debug line that was copy pasted from the OCI matcher)

Also note that application/vnd.docker.distribution.manifest.v2+json (type=docker) tarballs also have an oci-layout and a manifest.json file and are incompatible with the legacy repositories style format. They should use the new OCI layer detection logic.

The fix should be rather easy and can be done multiple ways:

• Run the OCI matcher first checking for oci-layout before assuming manifest.json -> legacy docker repo style
• Make the legacy docker mather return false given the presence of oci-layout
• use index.json to correctly identify the mediatype rather than just looking for oci-layout/manifest.json

If you'd like to reproduce the tar's I'm talking about you can simply do:

Dockerfile:

FROM alpine

docker buildx build -o type=oci,dest=- . > oci.tar
docker buildx build -o type=docker,dest=- . > docker-v2.tar

[gopannabd]

@vaskozl Thank you for providing the detailed information. May I know the customer organization that is affected? I can pass on the information to the corresponding Customer Relationship manager. Alternatively, please file a Salesforce ticket and/or contact your customer relationship manager for further assistance.