Workbox Support for Remix-Pwa

[dev-afzalansari]

Workbox Support for Remix-Pwa

Even though without workbox modules remix-pwa is doing great but we still need to add support for workbox for users who prefer workbox to build PWA over implementing their own logic in interpreting and caching the requests. Many other popular PWA libraries have support for the workbox.

It will be easy for users who are familiar with the workbox to extend the service worker.

How to Add this Feature

By simply extending the options in prompted question and adding expected cli flags remix-pwa can generate a service worker written with workbox modules and add dependencies for this those modules in package.json.

[ShafSpecs]

This is a splendid idea! I am giving it a read-up.

from what I see so far, it might be more efficient to build this as a separate package (@remix-pwa/sw for example)

[ShafSpecs]

It could still work in remix-pwa tho, we just add more dependencies if workbox is chosen. And create another service worker template file that's more barebones

[ShafSpecs]

Still figuring out how this would work flawlessly with Remix. Seems like workbox has additional scripts and build steps? I am currently playing around with some of their apis and want to replicate the functionalities and create a separate package that could be used like workbox to create your own SW

[ShafSpecs]

I am open to ideas about how this would integrate with esbuild and remix. Workbox has some too many packages, and SWs in remix for now have to be bundled (.worker files).

We would need to still ask devs what workbox features they want to work with, and install the appropriate packages. That would increase the steps to quickly turn a remix app to PWA. I would draw some sketches up to see how this could fit in well and simple when I get back to it tmrw

[dev-afzalansari]

I am open to ideas about how this would integrate with esbuild and remix. Workbox has some too many packages, and SWs in remix for now have to be bundled (.worker files).

A service worker with workbox modules can be easily bundled using esbuild plus we will give users options only for pre-caching and runtime/just-in-time caching and dependencies responding to their choice will be added. Later users can extend SW by adding more modules/plugins or changing strategies.

Still figuring out how this would work flawlessly with Remix.

I have worked with workbox but not integrated with the remix but I will try to provide an example repo when I get things working.

[ShafSpecs]

A service worker with workbox modules can be easily bundled using esbuild

Oh cool! It can very much work then

I have worked with workbox but not integrated with the remix but I will try to provide an example repo when I get things working.

👍👍

Limiting the strategies to just preaching and runtime Caching with the option to change strategies later on with Workbox works very well. I'm currently experimenting with extending the strategies available

[ShafSpecs]

Still not figured this out yet. Got ideas from Workbox I want to implement soon though: stale while revalidate caching, etc.

Got an example with remix @dev-afzalansari? I tried looking at the CRA examples for a simple integration but nothing suits remix afaik

[dev-afzalansari]

@ShafSpecs

I tried looking at the CRA examples for a simple integration but nothing suits remix afaik

Yes, I think we should take more time to dig into this issue as I also cannot get remix's CatchBoundary to render when there is not any cached response for a particular request.

[ShafSpecs]

Do you have something up already? I could play around and maybe see how it fits in

[dev-afzalansari]

I was trying to implement runtime caching with very basic code. I used registerRoute to match requests and created custom strategies that mimic NetworkFirst strategy but set the header X-Remix-Catch and X-Remix-Worker to Yes if there is neither network nor the cached response available in order to render the CatchBoundary just like the vanilla service worker from remix-pwa.

Unfortunately, It's not working and I am trying to figure it out.

I could play around and maybe see how it fits in

Yes, I will update you on that tomorrow maybe.

[dev-afzalansari]

@ShafSpecs

Here you go remix-workbox-example, It's very basic but Good Luck.

[ShafSpecs]

@dev-afzalansari Been working with the repo you sent me, I did some adjustments to separate the caches and avoid them being stored in one bucket, I also did a few fixes:

• Loaders weren't cached before. It was just the document, the reason why it doesn't seem obvious at first is because the loader data is being rendered and not being used under the hood.
• The client was never hydrated. That was due to deleting the contents of entry.client

I keep running into the issue with the catch boundary though. The reason being that the route is never cached, so it doesn't work offfline. I've tried a few fixes but they aren't workbox. The only solution I've been able to come up with was by writing my own custom code which removes the need for Workbox entirely.

What I'm thinking of doing now is trying to get a small postMessage handler into Workbox, I'm not sure if they have (still searching docs), and then try to handle caching there also for both document and loader requests. registerRoute triggers on a fetch event but not on a postMessage event and that means detecting navigation to ErrorBoundaries do not occur by default

[dev-afzalansari]

@ShafSpecs

You should open PR on the example repo with your work or when you have got things working properly so I can keep up with that.

[ShafSpecs]

dev-afzalansari/remix-workbox-example#1 opened a PR

[dev-afzalansari]

@ShafSpecs

I think I have got things working in the example repo plus I have integrated a workbox-background-sync plugin also to demonstrate that now another workbox modules can be consumed easily.

[ShafSpecs]

Awesome! I love it. I should add the workbox-background-sync feature into remix-pwa.

One thing I noticed though, not sure if it is on purpose but if I go offline and attempt to visit a page that hasn't been cached, I get redirected back

[ShafSpecs]

We can start polishing the Workbox integration for a new update. I'm currently working on more integrations too besides Workbox like OneSignal

[dev-afzalansari]

One thing I noticed though, not sure if it is on purpose but if I go offline and attempt to visit a page that hasn't been cached, I get redirected back

I experienced this issue initially while building the integrating the workbox but I cannot reproduce it further, everything is working fine currently on my side. But for the record, this issue may arise when javascript bundles are not cached so when you run into this issue do check the asset cache in the dev tools.

[ShafSpecs]

Okay. I'm currently thinking of how we integrate it. I want to avoid making the remix-pwa template too large, and Workbox is the first of many plugins I hope to integrate. I'm thinking of smthg like @remix-pwa/plugin. It could still work in template though, we just create two new worker template files and install the dependencies on install.

[ShafSpecs]

Or we bundle everything into a sw package, including plugins and integrations and primary functions (postMessage handlers, Caching, etc.). Install the package if the user wants to set up their worker file themselves or we use the worker files in the template folder if they would like a working service worker to start of with.

[dev-afzalansari]

Or we bundle everything into a sw package, including plugins and integrations and primary functions (postMessage handlers, Caching, etc.)

Sounds good. It will make templates look lighter and clear, and I would also recommend since we are going to make a @remix-pwa/sw we should also make a @remix-pwa/client-utils package it will be more convenient to abstract these client APIs as well and then we will only need to add a dependency of it if the user chooses to use client-utils.

[ShafSpecs]

we should also make a @remix-pwa/client-utils package it will be more convenient to abstract these client APIs as well and then we will only need to add a dependency of it if the user chooses to use client-utils.

Agreed. I did something like this with Web Assembly (@remix-pwa/client), yet to finish the docs yet.

I would start working on both of them today 👍

[dev-afzalansari]

@ShafSpecs

I would start working on both of them today

Great. But it looks like a lot of work. You should create initially configured repos for them so I can start working on them too.

[ShafSpecs]

@dev-afzalansari Do you mean example repos? Like the remix-workbox-example you used to demo Workbox?

[dev-afzalansari]

Do you mean example repos?

No, I meant the actual repos for the discussed packages or the monorepo if that is what you are planning for.

[ShafSpecs]

Do you mean example repos?

No, I meant the actual repos for the discussed packages or the monorepo if that is what you are planning for.

Created a simple Notion template containing the packages and their basic features:
https://dot-albatross-15c.notion.site/Remix-PWA-56f8f85adb9d48bdb5958384ffba9d62

Monorepo for packages:
https://github.com/remix-pwa/modules (modules sound like a terrible repo name, suggestions are open 😅)

[ShafSpecs]

@dev-afzalansari I created a basic sw package with basic functionalities. I currently haven't added the modules to the monorepo yet. I am still testing and working on it, I would create a demo repo later this weekend for testing and improving the modules.

[ShafSpecs]

It was heavily inspired by Workbox too.

[ShafSpecs]

https://github.com/ShafSpecs/sw-example
The sandbox I'm using to put together the APIs. It's currently unreadable because of I crammed a lot of things into few files. Currently arranging the directories

[ShafSpecs]

Given the sw-example/worker module a README of sorts. You can test it out and play with the features 👍. Currently, it resembles Workbox a lot (thx to copilot) but this weekend, I would re-write the necessary parts for Remix. I also want it to resemble Workbox a bit, so if you've used Workbox before, it's no problem setting up your own in remix-pwa with additional extensibility and vice-versa

[dev-afzalansari]

I'll take a look at it tomorrow and will let you know my points about it.

[dev-afzalansari]

@ShafSpecs

Looking at the sw-example repo feels like we are rewriting the workbox which doesn't make sense because there is no need to write a package that already exists. Although it's good to get inspired by their logger and stuff but not the whole implementation.

My point here is to provide the developer's options for a vanilla-sw or sw-with-workbox. Our vanilla-sw here should look lighter and easy to understand and most importantly it should be extensible just like a worker should be. In the case of the sw-example repo, I have seen it is registering worker events such as fetch and activate internally which will make our vanilla-sw limited since we can only register one event at a time in a worker's lifespan, the most latest event will override the other registered events.

It means if we register events internally we have to write our own version of the workbox which we don't want here.

API Design

According to my point, @remix-pwa/sw imports and fetch handler should look like this:

vanilla-sw

import { 
  matchRequest,
  handleMessage,
  handleInstall,
  handleActivate,
  handleFetch,
  handlePush
 } from '@remix-pwa/sw'

type MatchRequest = (request) => 'asset' | 'loader' | 'document' | null

self.addEventListener('fetch', (event) => {
  event.respondWith((async () => {
      const type = matchRequest(event.request)
      if(type) return handleFetch(event.request)
      
     return fetch(event.request)
  })())
})

// other handlers registration.....

sw-with-workbox

import { 
  matchAssetRequest,
  matchLoaderRequest,
  matchDocumentRequest,
  handleMessage,
  handleInstall,
  handleActivate,
  remixLoaderPlugin,
  handlePush
 } from '@remix-pwa/sw/workbox'

registerRoute(matchAssetRequest, new CacheFirst())
registerRoute(matchDocumentRequest, new NetworkFirst())
registerRoute(matchLoaderRequest, new NetworkFirst({
  plugins: [remixLoaderPlugin]
}))

// other handler registrations.....

It's a minimal example and the APIs may need to be more advanced since @remix-pwa/sw needs to be aware of cache names and static assets paths.

Let me know if I am missing any points here.

[ShafSpecs]

Completely agree 👍. The issue you raised about events being registered internally is something I was concerned about too, but your proposal for handleFetch, etc. would work perfectly. We could make them return a promise so users can extend their capabilities

const fetchHandler = async (event: FetchEvent) => {
      const type = matchRequest(event.request)
      if(type) return handleFetch(event.request)

      return fetch(event.request)
}
self.addEventListener('fetch', (event) => {
  event.respondWith(() => fetchHandler(event).then(res => /* Your extended event handler here */))
})

About caching, I would prefer devs have more control over how they cache (currently, it's network first and preaching hard coded).

[dev-afzalansari]

About caching, I would prefer devs have more control over how they cache (currently, it's network first and preaching hard coded).

👍👍, You should give me a branch in the sw-example repo something named like afzals-patch so I can push my work as well and you can keep up with that.

[ShafSpecs]

moving this reply to a separate comment

[ShafSpecs]

About caching, I would prefer devs have more control over how they cache (currently, it's network first and preaching hard coded).

👍👍, You should give me a branch in the sw-example repo something named like afzals-patch so I can push my work as well and you can keep up with that.

Sent an invite 👍

[ShafSpecs]

Moved this comment to a new thread due to length 👍

@dev-afzalansari Would do some editing this morning and update the repo. I'm still not sure how to would handle caching strategies yet. I'm thinking something like:

import { handleFetch, handleAssetFetch, handleLoaderFetch } from '@remix-pwa/sw'

type CacheStrategy = "NetworkFirst" | "CacheFirst" | "CacheOnly" | undefined

const fetchHandler = async (event: FetchEvent): Promise<Response> => {
  const type: MatchRequest = matchRequest(event.request)
  
  switch (type) {
    case 'asset':
      return handleAssetFetch(
        request: Request, 
        assetUrls: string[], 
        {
           cacheStrategy?: CacheStrategy, 
           cacheName?: string, 
           plugins?: Plugin[]
        }
      ) 
    case 'loader':
      return handleLoaderFetch(request: Request, { cacheStrategy?: CacheStrategy, cacheName?: string, plugins?: Plugin[] })
    case null:
      return fetch(event.request)
    // ...other cases
    case default:
      // In the scenario it didn't match any of the above cases, 
      // handle the fetch with default settings (behaves like defaultHandler() in `workbox`)
      return handleFetch(event.request)
  }    
}

self.addEventListener('fetch', (event) => {
  event.respondWith(() => fetchHandler(event))
})

I kinda like this setup, what do you think? If the dev wants to handle only loader-related request (GET request) with the service worker, they simple match type 'loader' in the switch statement and use the default to handle the rest. And if for example, they want to extend the handlers, e.g handleImageAssetFetch, they simply write a function that returns a Promise<Response> and they can use whatever caching strategy, plugin or write their own custom ones. For instance, to match just images and loader requests:

// `handleCustomFetch` is a function for allowing you to write your own custom fetch handler
import { handleFetch, handleCustomFetch, handleLoaderFetch } from '@remix-pwa/sw'

const customMatchRequest = (request: Request) => /* my custom matcher */;

const fetchHandler = async (event: FetchEvent): Promise<Response> => {
  const type: 'loader' | 'image' = customMatchRequest(event.request)
  
  switch (type) {
    case 'loader':
      return handleLoaderFetch(request: Request, { cacheStrategy?: CacheStrategy, cacheName?: string, plugins?: Plugin[] }) 
    case 'image':
      // the `fetchCallback` param is the function that would be called to handle the match
      return handleCustomFetch(fetchCallback: (request: Request) => Promise<Response>)   
    case default:
      return handleFetch(event.request)
  }    
}

I'm still thinking of how handleMessage would function with minimal code.

[ShafSpecs]

@dev-afzalansari
Currently implementing a simpler, minimal design with a bit of work done on the caching part. I showcased as much as I could in the worker. I am open to suggestions and criticism, even if it includes changing implementations (done that a few times already, one more time should be ok)

https://github.com/ShafSpecs/sw-example/blob/shaf-patch/app/entry.worker.ts

[dev-afzalansari]

@ShafSpecs

I see you have got the point in your patch and I will love to improve that.

[ShafSpecs]

@dev-afzalansari Started mapping out a design for how plugins would work and function with strategies. I think I got a real simplified process for creating and using plugins plus ideas for plugins. I srsly can't wait to see what people would build with the template

A pseudocode example of how it would work: https://github.com/ShafSpecs/sw-example/blob/shaf-patch/app/worker/background-sync.ts
A whole lot of ideas for plugins: https://github.com/ShafSpecs/sw-example/blob/shaf-patch/app/worker/README.md

[dev-afzalansari]

Took a look at it and I think those plugins are amazing would love to help you make a space for them in the new sw-package once I get my patch working with all four types of service workers.

[ShafSpecs]

Having ideas for using a similar approach for handling message event. Using a base MessageHandler and extending that:

// if you send `ping` from client, you get a `pong` response from the service worker 😁 
class PingHandler extends MessageHandler {
  async handleMessage(event: ExtendableMessageEvent, state: Record<string, any> = {}) {
    await super.handleMessage(event, state);

    event.source.postMessage('pong');
  }
}

const pingHandler = new PingHandler([new LoggingPlugin(), new AuthPlugin()]);

self.addEventListener('message', (event: ExtendableMessageEvent) => {
  if (event.data.type === 'ping') {
    pingHandler.handleMessage(event);
  }
});

Not sure yet, still considering the possibilities. This imo opens up further opportunities and makes your SW a progressive powerhouse

[ShafSpecs]

So we can create a default RemixNavigationHandler that contains the current implementation for handling message events in Remix and leave extensibility for adding other types of implementations

[ShafSpecs]

And now, I'm thinking of how to integrate this approach for the major event handlers 😆 (push, sync, etc.)

[ShafSpecs]

@dev-afzalansari Worked on the message handlers and did some testing, saw you went via the function route for message handlers, I used abstract classes and inheritance instead to allow for extensibility and to change core implementations later without having to go and change every message handler

https://github.com/ShafSpecs/sw-example/blob/shaf-patch/app/remix-pwa-sw/handler/message.ts

Also reworked fetch module a bit, still in progress: https://github.com/ShafSpecs/sw-example/blob/shaf-patch/app/remix-pwa-sw/handler/fetch.ts

[ShafSpecs]

Also, I saw you did all four workers. Superb! I would pull them and run testing and try to iron it out, if it needs. Then start building the package

[ShafSpecs]

@dev-afzalansari Went on a little (work) break and did some work in the meantime: https://github.com/remix-pwa/sw

Currently working on the docs and prepping remix-pwa cli for its biggest update yet

[typytypytypy]

Really excited about these updates, especially the ability to do background syncing! I know you've said the update is coming "soon", do you have a more concrete timeframe? I tried to view the docs to get a head start, but had trouble accessing the site.

[ShafSpecs]

Really excited about these updates, especially the ability to do background syncing! I know you've said the update is coming "soon", do you have a more concrete timeframe?

I would say this month, not sure when though as I build in my spare time. I am done with the package though, just to get the docs to a readable degree then I publish a pre-release you can use to get a hang of it. Then publish an official v2 release at the end of the month

I tried to view the docs to get a head start, but had trouble accessing the site.

Weird. I haven't bought a domain name yet but https://remix-pwa-docs.vercel.app should work just fine.

[dev-afzalansari]

@ShafSpecs

I tested with vanilla service workers and it's working fine, but workbox exports are missing notify me when you publish the next alpha version so I can test with workbox as well.

[ShafSpecs]

I tested with vanilla service workers and it's working fine

😊

but workbox exports are missing notify me when you publish the next alpha version so I can test with workbox as well.

That should be later today (a few hours time)

[ShafSpecs]

@dev-afzalansari New version published 🚀

[ShafSpecs]

@dev-afzalansari @typytypytypy New pre-release is out! I would do a pre-release in github releases this weekend. Feeback would definitely be appreciated

#85

[ShafSpecs]

Btw, already building out APIs for @remix-pwa/push. Not sure of a release timeline yet

[ShafSpecs]

https://remix-pwa-docs.vercel.app/push

Push is already out! @dev-afzalansari
Btw, are you on Discord?

[dev-afzalansari]

Yes, I am on discord but not active that much. My username Afzal Ansari#5076

[ShafSpecs]

Yes, I am on discord but not active that much. My username Afzal Ansari#5076

Sent an invite (FR)