On-demand Incremental Static Regeneration (next.js 12.1)

[priard]

Hi, next.js 12.1 introduces On-demand Incremental Static Regeneration.

"Next.js now exposes a function unstable_revalidate () allowing you to revalidate individual pages that use getStaticProps. (...) You can now manually purge the Next.js cache for a specific page on-demand."

This gives the possibility to trigger the revalidate cache of selected paths, e.g. during publishing, saving and updating (WordPress backend) without the need to set revalidation for a specific time. With larger websites this will save a lot of server resources while still maintaining excellent performance.

I wonder if you are considering introducing this possibility in future versions of Faust?

More info on next.js On-demend ISR: https://nextjs.org/blog/next-12-1

[theodesp]

Hello @priard. Thank you for the heads up. We will investigate this feature as part of the larger effort to revalidate content on demand.

Looking at this workflow though, I would be very cautious on having the main WordPress backend trying to trigger anything to the Front-end site URL. With regular WP this is not an issue. But in the Headless worlds there should be only a one way communication (from the Front-end site URL to WP backend and not vice versa) to reduce any coupling scenarios.

However ( I haven't verified though ) this could be done with any plugin/hook that triggers webhooks on post update/create like this one. Here are some rough steps:

1. You would have to configure the plugin to send a webhook request to the Front-end site URL.
2. You would have to create a special api handler as shown in the example to listen to those requests coming from the plugin.
3. Whenever the post is updated it will post to the api handler and revalidate the page.

I hope this helps.

[alexjpanagis]

Hi Theofanis, in that case – what is the recommended method to handle updating content when changes are made without redeploying the entire site?

[theodesp]

Using the default settings in Faust you get 900 seconds. If you made changes during that time, visitors would only see them after the 900 seconds have passed. This is no different from Next.js as Faust.js just sets the revalidate property for you by default. You don't have to redeploy the site if you only made content changes.

[nazmateo]

hello @theodesp thank you for your information on this. So basically I used atlas blueprint to create an app which pulls data from one of wpengine environment, so problem is when page content is updated, frontend site (deployed via wpengine, hosted in github using atlas blueprint) is not updated even after 900 seconds. page.js is basically a has same content from atlas blueprint portfolio.

I checked the docs here and here and here but I can't be sure If i need to setup up something for revalidation or page contents to update.
Hope you can help us with this
Here is our page.js inside wp-templates folder

`import * as MENUS from 'constants/menus';

import { gql } from '@apollo/client';
import { BlogInfoFragment } from 'fragments/GeneralSettings';
import { pageTitle } from 'utilities';
import 'slick-carousel/slick/slick.css';
import 'slick-carousel/slick/slick-theme.css';

import {
Header,
Footer,
Main,
ContentWrapper,
NavigationMenu,
FeaturedImage,
SEO,
} from '../components';

export default function Component(props) {

// Loading state for previews
if (props.loading) {
return <>Loading...</>;
}

const { title: siteTitle, description: siteDescription } =
props?.data?.generalSettings;
const primaryMenu = props?.data?.headerMenuItems?.nodes ?? [];
const footerMenu = props?.data?.footerMenuItems?.nodes ?? [];
const { title, content, featuredImage } = props?.data?.page ?? { title: '' };

return (
<>
<SEO
title={pageTitle(
props?.data?.generalSettings,
title,
props?.data?.generalSettings?.title
)}
description={siteDescription}
imageUrl={featuredImage?.node?.sourceUrl}
/>

<>

</>

</>
);
}

Component.variables = ({ databaseId }, ctx) => {
return {
databaseId,
headerLocation: MENUS.PRIMARY_LOCATION,
footerLocation: MENUS.FOOTER_LOCATION,
asPreview: ctx?.asPreview,
};
};

Component.query = gql${BlogInfoFragment} ${NavigationMenu.fragments.entry} ${FeaturedImage.fragments.entry} query GetPageData( $databaseId: ID! $headerLocation: MenuLocationEnum $footerLocation: MenuLocationEnum $asPreview: Boolean = false ) { page(id: $databaseId, idType: DATABASE_ID, asPreview: $asPreview) { title content ...FeaturedImageFragment } generalSettings { ...BlogInfoFragment } footerMenuItems: menuItems(where: { location: $footerLocation }) { nodes { ...NavigationMenuItemFragment } } headerMenuItems: menuItems(where: { location: $headerLocation }) { nodes { ...NavigationMenuItemFragment } } };
`

[bebjakub]

Hi @theodesp I'm trying to implement on demand ISR revalidation. On the homepage, I'm using default usePosts hook.
The homepage's posts list reveals post items duplicates, after multiple times switching the post status between publish and draft and homepage revalidating. Is it because of GraphQL client cache? After full nextjs rebuild duplicates disappear.
Thank you!

[bebjakub]

I started with the official example https://github.com/wpengine/faustjs/blob/canary/examples/next/getting-started/src/pages/posts/index.tsx#L18
Currently, I'm getting list items with fetch in getStaticProps.

my /api/revalidate.ts:

import { NextApiRequest } from "next"

export default async function handler(req: NextApiRequest, res) {
    const {
        body: { paths },
        method,
    } = req

    if (req.headers.authorization !== process.env.REVALIDATE_SECRET_KEY) {
        return res.status(401).json({ message: 'Invalid token' })
    }

    if (method !== 'PUT') {
        return res.status(405).json({ message: `Method ${method} Not Allowed` })
    }

    try {
        const revalidatePaths = paths
            .filter((path: string) => path.startsWith('/'))
            .map((path: string) => res.unstable_revalidate(path, { unstable_onlyGenerated: false }));

        await Promise.all(revalidatePaths);

        return res.json({ revalidated: true, message: 'Paths revalidated' })
    } catch (err) {
        return res.status(500).json({ message: err.message })
    }
}

The paths I'm sending from a custom WordPress plugin on post updates.

[priard]

Finally, I had some time to play with revalidation with my own implementation. Works decently well in general, but only if the post is refreshed, for example (page reload). I do have a lot of different query caching problems. Various strange things are happening here, from overwriting with the wrong cache queries when switching pages to a few others.

Supporting prepass and other helpers did not solve anything. The ISR set in Faust also doesn't revalidate the query cache correctly. I mean, of course, if we do page reload, yes, revalidation works, but when we switch between pages, the queries are retrieved from the cache (if they existed before) and revalidation does not refresh them.

@bebjakub, any luck? Have you made any progress on the topic? ;)

[bebjakub]

@Priad for post/page querying I'm not using gqty, I'm querying data via fetch :D It's annoying but better as install another gql client package when I need to make a non-cached query :)
btw. I published a wp plugin for easier On-Demand Revalidation https://github.com/gdidentity/on-demand-revalidation fell free to create a PR if you miss something... :)

[priard]

Hey, @bebjakub, I saw your "On-Demand Revalidation", in my free time I'll probably create PR. I added an additional function to revalidate the main page and recent posts (x count) and call it e.g. when saving ACF Option Page settings.

As for the problems with mixing posts (due to query cache), GQty solved it in the latest version 2.3.0 using "Change variable aliasing to be deterministic with cached and sliced sha1 hashing". I've already tested it, it works fine. Hopefully the Faust team will quickly update GQty on their side. CC: @theodesp

@bebjakub, I saw that you also created a Cloudflare cache cleaner plugin for your needs. Can you tell us what your Cloudflare configuration looks like? Of course, if you used some Page Rules or, for example, changed the Cache Control origin headers. I'm curious.

[bebjakub]

@priard Yes, create PR, and add also options fields for revalidate recent posts at least, pls. To revalidate the main page there is a checkbox "Revalidate Homepage on post update" already. If you want to revalidate on the saving ACF options page, maybe it'll be the best option to copy existing options in the "On Post update" section to the new "On ACF Options update" with a checkbox to enable the whole section to allow users to use just what they need.

My Cloudflare configuration:

1. Caching > Configuration Browser Cache TTL: Respect Existing Headers
2. Page rules:

• https://mywebsite.com/* Cache Level: Cache Everything, Origin Cache Control: On
• https://mywebsite.com/api/faust/auth/* Cache Level: Bypass

[priard]

So the revalidation basically works, not without problems (problems with the query cache mixing), but there is also a solution for that (the latest version of GQty 2.3.0).

@bebjakub showed what a more advanced implementation might look like. I think it is worth extending FaustWP with these functions and adding revalidation in the example implementation + expand the documentation.

[1] On-Demand Revalidation
[2] GQty 2.3.0

[priard]

I have a moment, so little update on on-demend revalidation.

I basically rewritten my application to switch from GQty to Apollo Client and the revalidation problems are completely gone. Additionally, this way I managed to reduce "First Load JS" by 30-40 kB per route. I also rewritten WordPress Post / Page preview to work with Apollo, but that's where the problems started.

It turned out that Apollo has a few long-unresolved bugs that make working with useQuery and useLazyQuery difficult (infinite loops, non-firing queries, and other bugs). So I decided to rewrite the application again and replace Apollo with urql library. From that moment on, the problems ended. Everything works correctly and quickly, and interestingly again - thanks to this change - I was able to reduce the size of the application by another ~ 30 kB.

I rewrote the preview, rewritten handleSitemapRequests (middleware does rewrite to api / faust / xml route I added to handle sitemap / rss / robots requests) to work with next.js 12.2.x, added sitemap-like rss support and it works together very well.

I wrote all of this to let you know that it's worth switching to urql right away and not bothering with either GQty or Apollo Client. Especially if you plan to create a fairly extensive website with many functions that are based on graphql api.

As for On-demand Incremental Static Regeneration, both Apollo and urql handle that flawlessly. However, if I had to choose, it would definitely be urql. Works great, easy to implement, easy to migrate from Apollo, fast and small, but most importantly, well documented and heavily developed.

Good luck.

cc: @bebjakub

[bebjakub]

@priard thanks a lot for the info!