Refactor logic for fetching files and listing directories

[flakey5]

Background

As of right now the code behind fetching files, heading files, and listing directories is functional and working fine. However, it isn't the cleanest.

Current Request Flow

Notable Points

• GET & HEAD requests are squished into the same functions. This can get messy, especially when one of them requires certain behaviors that the other one doesn't
• Many functions are long and hard to follow in some cases
• R2 and the origin fallback behaviors are baked in

Proposal

I have a few ideas that can make the code nicer to read and add some additional functionality that is/could be useful. I've implemented most of these ideas in my personal fork as POCs, https://github.com/flakey5/release-cloudflare-worker/tree/flakey5/20240330/contributors (term POC being used liberally in some cases)

A Provider

A Provider essentially acts like an API client. It abstracts the calls to a data source (e.g. r2, origin.nodejs.org) and returns the data in a shared type (e.g. GetFileResult for getting a file). It also should handle retry logic if necessary.

export interface Provider {
  head(path: string): Promise<HeadResult>;
  getFile(path: string, options?: GetFileOptions): Promise<GetFileResult>;
  readDirectory(path: string, options?: ReadDirectoryOptions): Promise<ReadDirectoryResult>;
}

So, an example of using a Provider to get a file from R2 would look something like:

const provider = new R2Provider({ ctx });
const result = await provider.getFile('some-file', { /* additional options, e.g. conditional headers */ });
if (!result.exists) {
  // file doesn't exist
} else {
  // do something with result.body
}

Providers will be created for R2 (R2Provider), R2's S3 API (S3Provider), and the origin server (OriginProvider). POC located here.

This API is a bit more complicated than what is currently implemented. However, it gives us a few benefits:

• We can introduce and swap out Providers easily.
• We can change where the worker is providing assets from without modifying code
  • E.g. if R2 were to go down for a significant amount of time, we can have something in place that allows us to just swap out the R2Provider for the OriginProvider without needing to change any code and without needing to modify the Cloudflare routing config (see https://github.com/flakey5/release-cloudflare-worker/blob/flakey5/20240330/contributors/src/providers/providerFactory.ts#L6-L33)
• Provides a single API for interacting with the few data sources we have
• We can chain multiple Providers
  • E.g. creating fallbacks if one exhausts all of their retries (see https://github.com/flakey5/release-cloudflare-worker/blob/efb7ad9e5cf06a254a2f0912586b1276cfc7739c/src/providers/r2Provider.ts#L99)

Our current implementation does not give us these benefits. It is hard coded to go to R2 and then the origin server if all retries to R2 get exhausted.

Additional changes

• Handle GET and HEAD requests separately.
• Refactor files in src/handlers/strategies to files in src/actions
  • POC code here
• Move code that renders the directory listing HTML into a response like we do with errors (e.g. src/responses/directoryNotFound.ts)
  • POC code here

Proposed Request Flow

GET

HEAD

Wdyt cc @nodejs/web-infra ?