RFC: keep form values and it's defaults in sync with remote data

[Moshyfawn]

Intro

This is a big topic about managing a close-to-real-time (in some cases, actually real-time) form data. I think it's important to introduce a solid way to sync the remote data with the currently active (editable) form.

The current state of things

At this time, React Hook Form is focused on "traditional" more Redux-like type of data flow (a very stretch comparison, IK. Gotta anchor to smth lol): you fetch some async default values and reset the form when you get the data, not thinking about the remote data anymore; you've set the client-side form values and operate the client state, assuming the remote data never changes without the current form submission (mutating remote data on form submission)

const { reset } = useForm()
useEffect(() => {
  if (remoteData) {
    reset(remoteData)
  }
}, [reset, remoteData])

The problem

The issue arises when the data becomes more "unstable": introduction of background re-fetches, stale data, subscriptions, etc.

The JS eco-system is greatly advancing in the field of cache management, back-end - front-end data syncing. Just look at packages like SWR and Apollo. Even "the one store - 99 problems" Redux is moving towards dividing the data cache and the client store: RTK Query.

I'm going to use the React Query cache management (and a ton more) package as a base to showcase the ever-changing data use-case for simplicity-sake.

Consider some back-end data entry you can update (mutate) via your form on the client (your basic CRUD, if you will):

{
  "name": "John Doe"
}

The potential form setup to update the entry with React Hook Form can look like this:

const { register } = useForm()

return (
  <form>
    <input {...register("name")} defaultValue="" />
  </form>
)

Now, our data is stored and the back-end and we need to retrieve it to initialize our form with the actual default values with out entry data. We don't have the needed data to be set as form's defaultValues initially. As I've mentioned previously, the current React Hook Form recommended pattern is to reset the form's values as soon as our async data comes in. Let's try applying it to our form (Keep in mind that we're not looking at such concepts as suspense as it's a whole another topic)

// INFO: data can be `{ name: string }` or `undefined`
+ const { data: user } = useQuery(["users", { userId }], getUser)
  const {
    register,
+   reset
  } = useForm()

+ useEffect(() => {
+   if (user) {
+     reset(user)
+   }
+ }, [user, reset])

  return (
    <form>
      <input
        {...register("name")}
-       defaultValue=""
      />
    </form>
  )

Provided our back-end user data changes only when we submit the form successfully, problem solved.

Introducing background re-fetch. I won't go into great detail about how it works or what it means for our form. For that you can read up the @TkDodo's blog-post React Query and Forms. For our form, background refetch means that our client state form data will be eventually synced with the remote data during the life-time of our form, meaning, with our current setup the user form input will be reset with the back-end data, which can potentially differ; not a greate UX at all.

You can check out this codesanbox which demonstrates our current issue:

1. Open the user form
2. Enter some value different from defaults
3. Cause background refetch
   • Un-focus the current browser window (or switch to a different tab)
   • Click Re-fetch User button (not so much of a background re-fetch, but a re-fetch nonetheless)
4. See the form input's data has changed

The current solution

There's a lot going on and there're multiple moving pieces to this problem. There's no one way to tackle this topic.

It's tough to solve an issue, the solution to which depends on the business requirements, app architecture, performance, etc. etc. Simply put: it's case-by-case study. To list a few:

• Do you want to override the user's form input with the new remote data?
• Do you want to show a notification about the new remote data?
• Give the user control over syncing client and server state
• Ignore the background refetch all together?

That said, there're a couple of things you can decide on the form's side.

The solution that comes to mind is to check the dirtyFields values and override only the yet to be modified fields, leaving the dirty fields with the current user-entered values. This would look something like

const { data: user } = useQuery(["users", { userId }], getUser)

const { formState, getValues, reset } = useForm()
const { dirtyFields } = formState

useEffect(() => {
  if (user) {
    const newDefaultValues = Object.fromEntries(
      Object.entries(user).filter(([userKey]) => !dirtyFields[userKey])
    )
    const currentValues = getValues()

    reset(
      {
        ...currentValues,
        ...newDefaultValues
      },
      {
        keepDirty: true
      }
    )
  }
}, [user, dirtyFields, getValues, reset])

Here's a codesanbox to play around with it live

This approach while being a bit convoluted, allows to us to override only the currently unmodified form fields, leaving the user's progress alone. Is it the solution? No, it leaves a lot to be desired in terms of UX: updating form values, even the user hasn't yet touched them can easily get confusing.

@TkDodo provides a slightly different approach to managing the form values updates using the controlled inputs. You can see his example here.

The proper solution would combine something from the method mentioned above with potentially a better React Hook Form API to update the intended fields + the user notification system + potentially a completely different approach + ...

You can notice how untapped this question is. Hence we need your help :P

Outro

I'm very curious to hear your opinion on the matter. Please, share you experience and potential solutions you might have came up with, so we can all discuss it and improve React Hook Form and sequentially the developer's quality of life (your life, lol)

P.S. For those who are not familiar with the cache management concept described above, please, take a look at the greatest information source there's about React Query and back-end data synchronization in general: @TkDodo's blog. Or feel free to ask me questions on the topic here.

P.P.S. Here's a list of discussion resources:

• @TkDodo's RQ-forms blog comments
• @ciaoimxiao's "keepDirtyOnReinitialize flag" Twitter thread
• @TkDodo's RQ-forms Twitter thread

P.P.P.S. Sry for a long and crumpled description. it's such a big topic; my little brain doesn't know how to bite into it at times xD

[bluebill1049]

relevant: #2335

[Moshyfawn]

Yep!

Tho, the initial default async values (the topic raised in #2335) has it's own context to discuss. There's a suspense solution also.

The OG post is more about the whole life-cycle of the form

[TkDodo]

I have to admit I haven't fully understood the reset api 😅 , but generally, I don't think it's a bad idea to have this done with a useEffect. Users already have great control over what they want to do:

• overwrite the user input with every background data change? Just call reset like described:

useEffect(() => {
  if (user) {
    reset(user)
  }
}, [user, reset])

• show a toast with action when new data comes in to let the user choose what to do? Easy:

useEffect(() => {
  if (user) {
    // some async dialog that returns a Promise
    dialog.confirm().then(() => reset(user))
  }
}, [user, reset])

However, what is missing is that you can keep your own input on big forms, but get untouched fields updated by background updates. What I would like to see is a way to sync only the defaultValues:

const {
   register,
   reset,
   defaultValue: { name: "" }
} = useForm()

useEffect(() => {
  if (user) {
    // hypothetical api that only updates the defaultValues
    reset({ defaultValues: user })
  }
}, [user, reset])

now if we have changed the name already, this wouldn't do anything. Maybe, if we have changed the name to Alice, and the background update comes in that also sets the name to Alice, our field would be reset to be in a non-dirty state, because it is now the same as the default value. Maybe the value itself could even be reset to undefined. That would be sweet.

If we have set the name to Bob in the meantime, and the background update yields Alice, the defaultValue would be Alice, but the value would still be Bob, meaning our field is still dirty,. But if we now reset, it would go back to Alice instead of empty string.

I hope I'm making somewhat sense 😅

[bluebill1049]

Thanks, @TkDodo for the awesome feedback and input!

I quite like this idea (API design): reset({ defaultValues: user }) which translates to the current API is something like

reset(undefined, { defaultValues: users }) as the first argument would update both both values and defautlValues (to keep form data sync)

[bluebill1049]

reset(data, {
  keepDirtyValues: true
})

I think this API is probably closest to the existing pattern. options to have a new keepDirtyValues , so defaultValues will be updated, and values will be updated except for those dirtied fields.

[TkDodo]

your example shows the issue very well: I refetch the user, and the default values update (internally), but they are not visible on the screen, even though no field is dirty. I would expect non-dirty fields to update visually on the screen as well - or is that a weird ask 😅 ?

also, side node: there is no good way to implement usePrevious without technically violating the rules of react. The obvious version reads from a ref during render, which is "not allowed" and can get you in troubles with concurrent features. see: https://twitter.com/brian_d_vaughn/status/1317536300845666304

[Moshyfawn]

I would expect non-dirty fields to update visually on the screen as well - or is that a weird ask 😅 ?

The point of the example below is more to demonstrate the behavior of the keepValues: true reset option set to this point:

My take is that if a background refetch comes in, it would just update those default values

I believe, I see where the disconnect between what you expect to happen to defaultValues and what RHF is doing is (even tho, I can't quite put it into words yet lol).

Take a look at #8237. It's a reset API proposal to handle resetting the specific field values; not resetting the dirty fields.

there is no good way to implement usePrevious without technically violating the rules of react.

Oh, shoot! That I completely skipped.. Hm, it's supper weird to me that the use of functional components has such a drawback! I guess I can use something like immer to check the previous value, but that's.. that's interesting! 🤔

[Moshyfawn]

also, side node: there is no good way to implement usePrevious without technically violating the rules of react. The obvious version reads from a ref during render, which is "not allowed" and can get you in troubles with concurrent features. see: https://twitter.com/brian_d_vaughn/status/1317536300845666304

I see the most basic usePrevious hook implementation on the official React docs page. It uses refs to read the value during render. Isn't it weird?

[TkDodo]

Isn't it weird?

very! rules of react are "weird" 😅

[TkDodo]

https://twitter.com/TkDodo/status/1518301353231138816

[Moshyfawn]

I guess the solution with the current API is this weird setup (reusing the logic from the initial "solution" example from the OG post)

useEffect(() => {
  if (user) {
    const newDefaultValues = Object.fromEntries(
      Object.entries(user).filter(([userKey]) => !dirtyFields[userKey])
    )
    const currentValues = getValues()

    reset(user, {
      keepValues: true
    })

    reset(
      {
        ...currentValues,
        ...newDefaultValues
      },
      {
        keepDefaultValues: true
      }
    )
  }
}, [user, dirtyFields, getValues, reset])

It seems to work as expected. I know it's not pretty, but that's what I can come up with at this time.

Here's a codesanbox

reset-screencast.mp4

[bluebill1049]

Thanks very much for everyone's input. I think we are heading to a great result here. I have thought about this issue for a while in bed last night.

There are always going to be two-piece of form values state to be managed. Let's call it

• defaultValues
• currentFormValues

1. keepValues: true

Update defaultValues and skip update all currentFormValues, which mark dirtyFields and isDirty

2. keepDefaultValues: true

Update currentFormValues and skip update all defaultValues, which mark dirtyFields and isDirty

So above is basically defaultValues vs currentFormValues

---

3. keepDirtyValues: true

Update defaultValues and skip all modified fields currentFormValues, which mark dirtyFields and isDirty

I did aim to retain a copy of the user's modified data, but I think that's probably not a valid case for the above which @TkDodo pointed out, we should strictly follow what's involved with defaultValues vs currentFormValues update by the reset API instead user's action.

https://codesandbox.io/s/tender-panini-o8to91?file=/src/App.js

Screen.Recording.2022-04-24.at.8.58.25.am.mov

---

In terms of the API design, I think in the next major version we can improve upon it (breaking change)

reset(data); // covers most of user cases
reset(data, { keep: ['values' ] });
reset(data, { keep: ['defaultValues' ] });
reset(data, { keep: ['dirtyValues', 'isDirty', 'errors' ] }); // when user want to keep certain states.

while most users' reset actions wouldn't be involved with defaultValues, it comes in handy when you want to sync with external remote data.

[mikestopcontinues]

@bluebill1049 I would avoid the keep array syntax. It's non-standard, so it's less intuitive to newbs and non-TS users. Also, it makes conditionals much uglier:

const opts = {
  keep: [
    isSomeState ? 'dirtyValues' : undefined, // will the type even support undef/null?
  ],
};

// VS...

const opts = {
  keepDefaultValues: isSomeState,
};

[bluebill1049]

thanks for the feedback! Make sense 👍

[Moshyfawn]

@bluebill1049 I would avoid the keep array syntax. It's non-standard, so it's less intuitive to newbs and non-TS users. Also, it makes conditionals much uglier:

const opts = {
  keep: [
    isSomeState ? 'dirtyValues' : undefined, // will the type even support undef/null?
  ],
};

// VS...

const opts = {
  keepDefaultValues: isSomeState,
};

I see your point. It definitely makes sense, especially the conditional array part.

What I was trying to diviate from is an obnoxious amount of boolean options which, say, nooby wouldn't know how they're dependant on each other.

I agree that the array syntax doesn't suffices here, but I'd love to explore possible solutions to help with the amount of options we have

[mikestopcontinues]

There are quite a lot of options. I wonder if it's possible to reduce them down. E.G.

• keepDirty = keepDirty + keepDirtyValues
• keepSubmit = keepSubmitCount + keepIsSubmitted
• KeepErrors = keepErrors + keepIsValid

[uncvrd]

Hi @Moshyfawn ! Out of curiosity, have you discovered a useful way of handling server state with keepDirtyValues when using useFieldArray? I've been following the react-query forms post from TkDoDo that you have commented on and have had some issues figuring out the best way to implement.

For example, say that your server stores a collection of 2 items to display within your form (to do this we'll utilize useFieldArray, while a user edits a form, they decide to remove one of the items within the field array, then react-query refetches data from the server. How would RHF know to not merge in the removed (on the client side) field value?

Here's a codesandbox which shows this issue: https://codesandbox.io/s/react-keepdirtyvalues-forked-jyezhw?file=/src/App.tsx

Ideally my example would load in data and reset within a useEffect but since I'm loading in fake data, we're triggering it by a button click, but to recreate:

1. Click Start Fetch to "load" in data from the database
2. Change the input value in the first input
3. Remove the second item by clicking Remove on the right side
4. Click Start Fetch to simulate react-query fetching data again
5. Observe that the first row stays the same with the new modified value (yay! curtesy of the new keepDirtyValues) but the previously removed user value (the second row), is obviously added back again.

Any thoughts on the best approach for handling this scenario?

[dbousamra]

Hi all. Reading this thread reminds me a lot about our autosave implementation. We allow editing of forms, where the server can occasionally compute new values based on what the user submitted, and we want to update the form with those new values, but not erase any unsynced changes.

Here's our implementation. I know for sure it's not very good, and has a small bug in it (have yet to solve it), but it works for the most part. https://gist.github.com/dbousamra/37d67e2f7b43261d424e7b0d75032d5b

There's some extra stuff in here we do around optimistic locking (taking ownership), that can be ignored.

[tjhrulz]

Man, I'm used to random usernames with two names followed by four numbers trying to add me and spam me on every service that isn't GitHub. Now I have to deal with them on GitHub?

For anyone else that shows up there isn't aware the previous comment is spam probably trying to get you to go to some shady website to call you out of money that you'll give them probably via gift cards. Don't click on the links in the comment mentioning a bunch of usernames above and I already reported it so if you would report it to that may speed things up to get it removed.

Edit: well that was quick. Comment is already deleted. Everything on this thread is probably safe now.

[Moshyfawn]

I'm trying to keep up with this shit popping up everywhere in real-time. It's being going for almost an hour now: every 5 mins: 2 - 3 spam comments like you just saw. Freaking hell

[bluebill1049]

I am getting those as well... annoying

[Moshyfawn]

But thx for looking after the community and educating ppl about spam! Appreciate it!

[Moshyfawn]

It's also happening in a bunch of other repos of other orgs :(