Support rebuild event from dev server

[antgel]

This feature adds the ability to define a rebuildUrl in the config. When eleventy-dev-server is running and a POST request is made to that URL, 11ty rebuilds the site.

Usage:

  eleventyConfig.setServerOptions({
    rebuildUrl: "/rebuild/",
    rebuildUrlToken: "foobar",
  });

...

$ curl -vsSo/dev/null -X POST http://localhost:8080/rebuild/
*   Trying 127.0.0.1:8080...
* Connected to localhost (127.0.0.1) port 8080 (#0)
> POST /rebuild/ HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.81.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 403 Forbidden
< Content-Type: text/plain
< Date: Mon, 30 Oct 2023 11:43:04 GMT
< Connection: keep-alive
< Keep-Alive: timeout=5
< Transfer-Encoding: chunked
< 
{ [19 bytes data]
* Connection #0 to host localhost left intact

$ curl -vsSo/dev/null -X POST -H 'x-11ty-rebuild-token: foobar' http://localhost:8080/rebuild/
*   Trying 127.0.0.1:8080...
* Connected to localhost (127.0.0.1) port 8080 (#0)
> POST /rebuild/ HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.81.0
> Accept: */*
> x-11ty-rebuild-token: foobar
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Date: Mon, 30 Oct 2023 11:43:20 GMT
< Connection: keep-alive
< Keep-Alive: timeout=5
< Transfer-Encoding: chunked
< 
{ [5 bytes data]
* Connection #0 to host localhost left intact

Rationale: I'm working on a cool 11ty / Strapi project. In the cloud, when content is updated on Strapi, Strapi can send a webhook to whichever service is hosting 11ty, to rebuild the 11ty site. However this doesn't work for local development, as discussed at #604, #691, #1198, #1358, and elsewhere.

There have been suggested workarounds that generally involve the 11ty user writing an(other) HTTP server to accept the request and dumping a file somewhere to trigger a rebuild. But it is clunky and we already have an HTTP server. I reused existing EventBus functionality; really most of my time was spent reading and understanding the code.

I hope this is useful; it's my first 11ty contribution so feel free to let me know how it may be improved. :)

Corresponding dev server PR: 11ty/eleventy-dev-server#70

[Aankhen]

There have been suggested workarounds that generally involve the 11ty user writing an(other) HTTP server to accept the request and dumping a file somewhere to trigger a rebuild. But it is clunky and we already have an HTTP server.

That’s right, I suggested a bespoke HTTP server that uses Eleventy’s programmatic interface. My main concern is that implementing this means encouraging what is only meant to be a dev server to accept arbitrary Internet requests, increasing its responsibilities and attack surface. Given that the requirements you’ve outlined are reasonable but also atypical, it seems fair to me to place some of the burden in this scenario on the user: I would personally prefer an out-of-tree plugin, a wrapper script, or even a code snippet in the documentation to integrating it into Eleventy proper.

[antgel]

Thanks for the comment, yes indeed I forgot to mention that suggestion in the OP.

My main concern is that implementing this means encouraging what is only meant to be a dev server to accept arbitrary Internet requests, increasing its responsibilities and attack surface.

Nobody is "encouraging" the server to do anything except accept requests, which is the point of a server. The requests are not "arbitrary", on the contrary have a very specific purpose, nor are they "Internet" unless one runs the dev server on a publicly-accessible IP address, which is orthogonal from this PR.

In fact this PR (locally) replicates the functionality that one gets on the public Internet when hosting 11ty on a service such as Netlify or Vercel which accept webhooks and rebuild the site. So "increasing the server's responsibilities" in such a small way doesn't seem like a Bad Thing to me.

I have added 11ty/eleventy-dev-server@1c242aa which should go even further to assuage your concerns about the "attack surface".

Given that the requirements you’ve outlined are reasonable but also atypical, it seems fair to me to place some of the burden in this scenario on the user

I don't think that these requirements are "atypical", as seen by the Issues linked in the OP and other various Internet discussions with hacky workarounds. Use of headless CMSs is on the rise .

Also, "some of the burden" on the user? Currently all of the burden is on the user to be able to develop Headless CMS / Jamstack with the same low-friction approach, and I haven't seen any other slick implementation out there. In any event, why not put the user first? We're talking about a handful of lines of code.

[Aankhen]

Nobody is "encouraging" the server to do anything except accept requests, which is the point of a server. The requests are not "arbitrary", on the contrary have a very specific purpose, nor are they "Internet" unless one runs the dev server on a publicly-accessible IP address, which is orthogonal from this PR.

Maybe I’ve misunderstood something. How does the server accept requests from Strapi if it isn’t publicly accessible? (And to be clear, I didn’t mean ‘arbitrary’ to refer to these well-specified webhook requests, but rather the ceaseless flood of requests that any publicly accessible server receives, deliberately or not.)

In fact this PR (locally) replicates the functionality that one gets on the public Internet when hosting 11ty on a service such as Netlify or Vercel which accept webhooks and rebuild the site. So "increasing the server's responsibilities" in such a small way doesn't seem like a Bad Thing to me.

I understand what you’re saying and I think it’s a sensible and useful feature in the abstract, but it’s (naturally) extra code to design, maintain, and reason about the effects of, so it’s worth exploring the justifications and ramifications.

I have added 11ty/eleventy-dev-server@1c242aa which should go even further to assuage your concerns about the "attack surface".

I’m definitely happy to see that. Thank you.

I don't think that these requirements are "atypical", as seen by the Issues linked in the OP and other various Internet discussions with hacky workarounds. Use of headless CMSs is on the rise .

This is hard to quantify. I know headless CMSs have become quite popular; on the other hand, most Eleventy users I interact with don’t use headless CMSs… which could easily be caused in part by the absence of features like this. Either way, I do think better integration with headless CMSs is a good idea. I’m only examining the design of this particular feature.

Also, "some of the burden" on the user? Currently all of the burden is on the user to be able to develop Headless CMS / Jamstack with the same low-friction approach, and I haven't seen any other slick implementation out there.

I meant that Eleventy’s programmatic API gives you the low-level ability to trigger the rebuild. ‘Some of the burden’ is connecting that to whichever mechanism you want to use to trigger it. (I agree that it’s not a trivial task.)

In any event, why not put the user first? We're talking about a handful of lines of code.

The user absolutely needs to come first, and a crucial aspect of that is not inadvertently creating vulnerabilities that don’t exist without Eleventy, which is why a feature that expects a dev server to be exposed to the Internet gives me pause. As I said above, if I’ve misunderstood how it works, then my concerns are irrelevant and this is unequivocally a great feature that I’m sure will be deeply appreciated.

[antgel]

Nobody is "encouraging" the server to do anything except accept requests, which is the point of a server. The requests are not "arbitrary", on the contrary have a very specific purpose, nor are they "Internet" unless one runs the dev server on a publicly-accessible IP address, which is orthogonal from this PR.

Maybe I’ve misunderstood something. How does the server accept requests from Strapi if it isn’t publicly accessible? (And to be clear, I didn’t mean ‘arbitrary’ to refer to these well-specified webhook requests, but rather the ceaseless flood of requests that any publicly accessible server receives, deliberately or not.)

Thanks for clarifying. I think you have misunderstood the level of exposure required / expected to use this feature.

The intention is that the 11ty dev server accepts rebuild HTTP requests from a Strapi instance (or indeed any other service that wishes to signal to 11ty via HTTP to rebuild) that may be running:

1. On localhost e.g. 11ty on http://localhost:8080. More common i.e. Strapi and 11ty both running local servers on a development machine.
2. In the cloud tunnelling to the development machine via e.g. ngrok, serveo i.e. 11ty on https://some-random-url.ngrok.dev. This is less common but would be used if the development Strapi instance was not on localhost.

In either instance, there is no intention nor need to run the 11ty dev server on a publicly IP. (2) simulates this for certain situations, but either way, whether or not the Strapi dev server is accessible to the public Internet is irrelevant to the feature. Developers may already expose their dev server if they feel they wish or need to.

In fact this PR (locally) replicates the functionality that one gets on the public Internet when hosting 11ty on a service such as Netlify or Vercel which accept webhooks and rebuild the site. So "increasing the server's responsibilities" in such a small way doesn't seem like a Bad Thing to me.

I understand what you’re saying and I think it’s a sensible and useful feature in the abstract, but it’s (naturally) extra code to design, maintain, and reason about the effects of, so it’s worth exploring the justifications and ramifications.

Fair enough.

I have added 11ty/eleventy-dev-server@1c242aa which should go even further to assuage your concerns about the "attack surface".

I’m definitely happy to see that. Thank you.

No problem and thank you for the feedback, it's an easy and useful addition.

In any event, why not put the user first? We're talking about a handful of lines of code.

The user absolutely needs to come first, and a crucial aspect of that is not inadvertently creating vulnerabilities that don’t exist without Eleventy, which is why a feature that expects a dev server to be exposed to the Internet gives me pause. As I said above, if I’ve misunderstood how it works, then my concerns are irrelevant and this is unequivocally a great feature that I’m sure will be deeply appreciated.

Hopefully the previous part has explained this. :)