Immer perf appears to have gotten worse over time, based on updated benchmarks #1152
Labels
Comments
|
Hey @markerikson, Thanks! These findings are very interesting. I'll try to dig deeper here and see how the regressions were introduced, and thanks for setting up that repo! Two high level thoughts jump to mind as to causes:
I hope to explore both, but apologies upfront that it might take a while as we'll have a move coming up :) Edit: related thought, I'm wondering if Redux would overall would be faster if you'd deep-freeze the event object before sending it into the immer reducer. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Summary
I've put together some updated benchmarks for various versions of Immer and other immutable update libs vs a hand-written reducer, especially since the current docs at https://immerjs.github.io/immer/performance show benchmarks that were last run against Node 10 and much older versions of Immer.
Overall, it does appear that Immer is significantly slower than both hand-written reducers and
mutative. It looks like the majority of that time is due to freezing, but it also appears that Immer's perf has gotten worse over time.I know that Immer has a lot of logic for correctness. I'm not sure how much performance optimization can be wrung out of the current approach, but the results here do seem concerning, and I figured this was worth sharing for discussion.
Overview
It looks like Immer's perf got significantly worse starting with Immer 8. This is especially true with freezing turned on, but also even with freezing turned off. (This is admittedly a bit surprising given that Immer 8's changes were basically just turning on freezing by default, no other major logic changes.)
As an example, note this set of results for one benchmark case. The vanilla reducer is 11 microseconds. Immer is in the milliseconds range, and for both freezing and non-freezing Immer gets worse over time:
Background
There was some extensive discussion of Immer perf and freezing behavior over in the Redux Toolkit repo:
In that issue, @gentlee set up both a vanilla hand-written reducer, and an RTK
createSlicereducer with Immer, and compared them in four scenarios dealing with a nested large array (add, update, remove, concat + truncate).The issue discussion noted that Immer appears to be significantly slower than hand-written - not just 2-3x, but 100x or more.
I've taken the sample scenarios from that repo, and created another new benchmark repo at https://github.com/markerikson/immer-perf-tests that improves on the benchmarking process in a few ways:
faster-iteration-experimentbranch from Faster iteration experiment #1120 )I did run this on a few different Node versions and got roughly similar results each time. I also tried to run it in a browser, but ran into issues with Vite failing to load a nested dependency for the
mitatabenchmarking lib, so didn't get that working atm.Detailed Output
Here's the output of a benchmark run:
The text was updated successfully, but these errors were encountered: