Zip file option for tile delivery.

[artunit]

There may be value in using the zip file format in combination with byte ranges for efficient delivery of tiles. This makes a large set of precut tiles more manageable while leveraging long-standing web plumbing. OurDigitalWorld is testing this approach out with a middleware implmentation but has also done a simple proof-of-concept with javascript in the Universal Viewer. Perhaps this could be a IIIF extension? One possible syntax might be something like:

{
   "ImageZip":{
      "id": "/np/essex/1971-08-04/1971-08-04-0001/tiles.zip",
      "directory": "dir.bin"
      }
}

Here the id would form the access point to the zip archive, and the "dir.bin" file would be in the same location but just consist of the zip directory itself (to allow quick look-ups in the zip archive if javascript can't efficiently extract it, see this issue). This idea builds on a discussion in the IIIF slack channel. Some links from there include the wacx and cdxj specs.

[stephenwf]

Also interested. It could be possible on both the server and in browser potentially (using something like zip.js)

Another proof-of-concept: https://github.com/digirati-co-uk/storage-api/blob/main/src/routes/iiif/level-0.ts

Although serving up the whole zip to the client defeats the purpose of tiling - and we could get better compression with a full image sent instead. Would be useful as a server-side thing through a proxy or middleware.

[artunit]

Javascript can do byte range requests on the zip archive, so the tile handling can theoretically be quite efficient, but the web server has to support the range request It seems to be widely available on the web side, I think Apache has had byte range support since the late 1990s. I am not enough of javascript guru to know what the trade-offs are for tasking the viewer with this kind of thing.

[atomotic]

Also every S3 cloud storage (aws, digitalocean, cloudflare, etc) supports byte range.
The benefit of a feature like this is that static tiles could be easily hosted as ZIP on those storages (very cheap)

[atomotic]

Even if not directly related to the subject of this discussion (serving static tiles as ZIP files) I report here something discovered in the slack discussion:
there is this interesting Openseadragon plugin https://github.com/pearcetm/GeoTIFFTileSource that allows to load a remote pyramidal TIFF files using the byte range approach.

This is a test I've made https://pub-0f1c9e6ddb92456a85802303778fa724.r2.dev/demo/index.html
the viewer here is loading a 600M TIFF file https://pub-0f1c9e6ddb92456a85802303778fa724.r2.dev/map.tif created using vips

$ vips im_vips2tiff original.jpg map.tif:deflate,tile:256x256,pyramid

and the ongoing discussion on openseadragon github:
openseadragon/openseadragon#944

[artunit]

This is very relevant, I could see an organization loving the idea of a pyramidal TIFF for this purpose. Maybe the discussion should be more about using a Static file option for tile delivery. I could rework my UV example to be a similar plugin. You wouldn't need a dir option for the TIFF, so something like: { "StaticFile":{ "id": "/np/essex/1971-08-04/1971-08-04-0001/image.tiff" } }

[atomotic]

TIL PMTiles

PMTiles is a single-file archive format for tiled data. It enables low-cost, zero-maintenance map applications for "serverless" environments without having to rely on a custom tile backend or a third-party provider. This is achieved by packing all tiles of a tileset into an archive so that all tiles can be accessed easily and without much overhead via HTTP range requests. By combining all the tiles into one archive, hosting costs are kept low, as it is usually a lot cheaper to update one large file than to update thousands or even millions of small files.

https://protomaps.com/docs/pmtiles
https://github.com/protomaps/PMTiles/blob/main/spec/v3/spec.md

[artunit]

Great point about updates in this project, thousands of tiny files can be pain to manage and move around.

[atomotic]

an update on PMTiles:
https://twitter.com/protomaps/status/1714279337258778911
https://docs.protomaps.com/deploy/server