sumdb responses caching

[pboguslawski]

Is your feature request related to a problem? Please describe.

According to #1506 (comment) Athens does not cache checksums just packages.

The idea behind having a checksum server is that there's a third party that ensures the proxy is behaving as expected. Having Athens serve its own checksums defeats that purpose.

If Athens is compromised, proxied sumdb responses may be compromised as well probably (not sure if checksums are signed)?

According to https://goproxy.io/docs/GOSUMDB-env.html there is public key used with GOSUMDB so compromising athens problaby won't allow bad guys to change cached checksums without being detected.

Now every sumdb query goes to public net which also may be privacy problem; consider adding option to allow caching checksum responses from upstream sumdb also if proxy owner wants it. If one wants to verify checksum with third party - should not use proxy but use this third party's service directly probably.

Describe the solution you'd like

(1) Add parameter (disabled by default for backward compatibility) to enable caching module checksums in a same way as modules are cached (same storage, similar download modes).

(2) To be considered if not possible already: add another athens parameter to force module checksum verification before storing it in athens cache (athens should calculate checksum and compare it with upstream checsum and abort caching module if checksums differ - similar as go does).

[matt0x6F]

Just want to be clear for anyone stumbling on this issue that GoNoSumPatterns prevents sending private module checksums on to a public sum database. Go (as a client) supports this via GO_NOSUMDB. What that leaves open is what you mentioned - Athens can verify this package, but it cannot validate the progeny of the release. The real power of the Go public Sum database is that it quickly picks up on new packages and package versions on major VCS providers. At worst, it picks up those changes on the first request to the public proxy for a particular package version.

It would be a lot of work to implement an append only, merkle tree based DB in the way that Go has it that facilitates validation of a particular package versions progeny. I also don't think the usecase is the same when using Athens, so this is probably a no go.

We probably could run our own database that stores the hash of a version upon initial download and hashes the hashes of the artifacts similar to the way Go Sum DB does with package dependencies. We could at least then prove that the artifact that exists on storage matches what we originally downloaded. @pboguslawski thoughts on that?