-
Notifications
You must be signed in to change notification settings - Fork 32
IDEA: Support cached mode for plugin so metalsmith builds can be faster when we know the content hasn't changed #49
Comments
hi @jideshv, Best, |
hey @jideshv, I'm up totally for that. 👍 It would save requests and speed up development flow. That's a great idea! I don't think it would be that difficult. I'm just thinking out loud here.
And then we could wrap
Totally! 👍 |
@stefanjudis I like the approach but the question now is when do we invalidate the cache ? |
@Khaledgarbaya Hmm good question...
Let me just think out loud here. Production buildNothing's gonna change here.
Development flowThis mode would fill up the cache but also serve cached results if available.
Development flow invalidationThis mode drops everything initially and then write all data to cache.
Does this make sense? |
I'm guessing everyone's moved on to other projects perhaps, and feeling less of a burn when rebuilding. Has anyone made any progress on this? |
Hey @larrybotha, unfortunately, I've to say that we won't be able to tackle this in the near future. :( But I could support you in case you want to tackle it. :) |
@stefanjudis thanks! I do need to level up my node skills, so this would be a good way to both get some practise there, and learn more about the Contentful API. Got a bunch of things on my learning road map, and the next couple weeks are nuts, but if I find time between other goals I may just give it a bash. Don't hold thumbs, future people who come this way! I am known to do what I say, but also to not do what I say, too. |
This is my first look at contentful so I might be way off base.. but I have a few thoughts. @stefanjudis re: invalidating cache, it would be frustrating if the only way to invalidate the cache was with a specific call like This kind of approach means you still have a single api request per build, but for most use cases that's a reasonable trade off. @Khaledgarbaya re: sync api, looking at the docs about the sync api, it seems like it would be quite difficult to implement in this plugin. That said, the subsequent calls to the sync api get "delta updates", and while I understand the concept, not knowing what those deltas might look like means IDK what applying them would mean on a practical level. metalsmith-cache (self plug sorry) might be a good fit for actually storing and retrieving cached files, pretty much just a wrapper around lokijs to make it play well with metalsmith. Storing and retrieving files is straight forward (per readme). Merging from cache to your metalsmithfiles structure is as easy as |
When I get your approach correctly, this means depending on how many resources you include you'd have to make one API call per resource to get the And usually (at least for my case) this is exactly what I need. I want to develop a site and I know that the data didn't change (or I don't care) – I only care about fast build times and independence of the API in development. I'd totally be open for metalsmith-cache if it does the job. ;) |
I take your point. Thinking through this a little more it's starting to get complicated. I guess there's two types of changes contentful-metalsmith makes. Firstly the files which are created by this plugin with This first case for created files is easy.. you write every file created to a cache, and just retrieve all the cached files every build. It's this type of query which filtering by The second case for effected files is much more difficult. You can't just cache the state of a file after this plugin has effected it, because when you return it later on it will be uneffected by any other plugins earlier in the build process which have changed that file in some way. So for this case you need to cache queries rather than files, which is more complex. |
Hi Team, first of all amazing plugin. As we get more and more dependent on this plugin one area that has been a bit frustrating is the time it takes to pull data from contentful (we are pulling 400+ objects) even if the data hasn't changed. Was wondering how difficult it would be to allow a cached mode of the plugin. Thoughts on this? I was considering forking and adding this feature. Would there be interest from others?
The text was updated successfully, but these errors were encountered: