Storage plugin redesign

[Carniatto]

Context

After a recent feature request (#1760) to implement an async version of the storage plugin I had a look at what plugins have been developed by the community. It seems that there are quite a few plugins that have been developed that solve various different use cases, but for me, this does indicate that the core plugin is lacking in some areas.
While we generally prefer to maintain backward compatibility as much as possible, we are working on NGXS v4 and this offers the opportunity for a bit of a rethink around this plugin and to release a reworked version of this plugin.
We are looking for some members of the community that would like to be involved with this plugin and to head up the efforts to make a really awesome new version that handles a broader set of use cases.
If you would like to be involved, please join the #plugins_storage-plugin channel and indicate your intention with a ✋ emoji response to this message on the channel in Slack.

Initial Requirements

• Connect to multiple storage engines
• Connect to Async/Observable storage engines
• Should work with Ionic? Capacitor?
• Should work with Electron?
• Feature Modules (lazy loading)
• Add extra default engines:
  • SessionStorage
  • IndexedDb (async)
  • WebSQL (async)
  • SQLite (async)
  • File
  • URL query params
  • LocalForage
  • HTTP/Rest API
• Debounce time option (similar to Electron Plugin)
• Provide Fallback storage options (similar to Ionic Plugin)
• Migration between versions of an App
• Interaction with default state

The purpose of this discussion is to iterate on the ideas. The final design will be documented on #1771

[Carniatto]

Meeting notes from the kickstart session (15/07/2021):

Use cases

• Ionic (w/ NGXS) [Nick]
  • Feature Module (race condition)
    • hydrate on state init
    • changes lost on state update
  • using example from async-storage-plugin for Ionic
  • state token was not correct
  • persisting only part of the data
  • transforming data
    • Serialiser/Deserialiser could not transform the whole state
    • synchronising parts of the state
  • Ionic recommended engine: SQLite plugin to Cordova [build to native], other options�:
    • localStorage —> LocalForage
    • indexedDb
• [Phillip]
  • Noob to the storage plugin
  • uses NGXS in multiple projects
  • wants to get involved
• PWA (no localStorage) [Ally]
  • use local-storage-plugin
  • async-storage-plugin looked stale
  • IndexedDb, WebSQL, SQLite etc �(PWA / Web workers / Service worker)
  • support PWA with offline storage
  • persisting two slices each in different engines
  • phonebook app
    • records go to indexedDb
    • user session data stored in SessionStorage

LocalForage

• abstraction for different kinds of storage
• tries to best choose which storage engine to use

Challenges:

• How does it impact the initial state
• Interaction with default state
  • using the default from State definition in the engine
  • when to update the state
• Migration between versions of an App
• Ionic: foreground vs background
• Define an extensible API using Engines, Migration, and Converters

Opportunities

• Bringing part Firestore Plugin as an engine

recording

[ishanuda]

Hi,

NW.js is also a cross-platform desktop application development ecosystem using web technologies similar to the Electron.
https://nwjs.io/

Under the initial requirement, I see Should work with Electron?

I think the NW.js is also working as same as Electron still there are differences like features, the way we build the app etc.

Hence, could you please add the Should work with NW.js to the list?

Thank you very much

[profanis]

In the application I currently work on I had to use LocalForage to accomplish an auto-save functionality. The idea is that a user can play around with the application, create entities, link them together, and if he is happy with what has been created, can click the save button to persist everything in the server using a REST API.
I use the ngxsOnInit hook to register an observer whose responsibility is to listen to specific actions.

Based on this experience, I would like to share my thoughts on the LocalForage/IndexDB field:

• Will you update the IndexDB on every dispatched action? An idea would be to have a setting to allow all, or to specify which actions are eligible to do so (via a decorator)
• Will the persistence be in a key-value pair?
• Can I persist more than one key in IndexDB? If so, what will be the key identifier?
• When will I flush the NGXS state from the IndexDB?
• I should have a way to destroy the created DB
• Since the persisted data will be there until we delete them, how about securing the persisted data?

[markwhitfeld]

@Carniatto & others. What is the plan to move forward with this?

[DwieDima]

Is there any update on whether NGXS will ever support an async storage engine? It's truly unfortunate that for applications relying on async libraries like SQLite, it's currently impossible to integrate them effectively with NGXS. This limitation feels like a significant hurdle, and it's disappointing that such a powerful state management library cannot support these use cases. I genuinely hope the team might reconsider addressing this, as it would greatly enhance NGXS's flexibility and usability for modern applications!