Measured performance decrease between major releases of Marked, time for a performance pass? #2355
Comments
|
It would be great to get something in the pipeline to measure performance. We tried that at one point but it wasn't reliable when run in GitHub actions. PRs are appreciated 😁👍 |
|
Well, it's not something that has to be done often so this could be a manual operation by generating flamegraphs and attempting to optimize the functions that 'waste' the most time - just looking at the output above the lexer is a good place to start. |
|
What are you using to produce those flame graphs? |
|
Chrome DevTools via "Performance" tab recordings |
|
Is there perhaps a good standard test Markdown document that would give Marked.js a thorough performance stress test? I know we have our bench script, but that uses only our spec tests from Commonmark if I recall, which is heavily weighted toward specs with lots of odd edge cases (i.e., I think almost a fourth of our tests are for Things like this tend to have large numbers of code blocks, etc., which is a similar problem, but at least it's a normal document that can be pasted into Chrome if we use their performance tab. |
|
Safari also has some interesting tooling but DevTools should be sufficient. You can roll your own perf using the Performance API however it of course may add a few precious ns/ms to runtime ;) |
|
@UziTech Since we run each spec 1000 times in our bench test, would it be possible to modify our bench tests to track the average time taken for each spec and output which cases are slowing us down the most? |
|
Ya we could bench each test individually to see which ones are slowest relative to other packages. |
|
@UziTech I put together a rough version of this, but realized analyzing becomes very confusing if we are comparing 6 versions/options of Marked against Commonmark and Markdown-It. Given that, does it make sense to compare just the CJS (no GFM) version of Marked with Commonmark? If so I can make a PR. Essentially for each spec, I found the ratio of execution times The ranking is as follows (top 100 worst specs): KeySpec # - The commonmark spec example number. Note that example 523 is consistently the worst performing (first place almost every time), however in general
|
|
Brilliant research - do the majority of these use regex? My vague hypothesis is that some regexes could be optimized to reduce the number of steps needed for matching. Using a 3rd party tool such as https://regex101.com/ (although it only works for Python/PHP) you can tell the number of steps it had to take - there's probably an equiv for JS somewhere? |
|
Regex101 works fine. Even though the debugger is "only for PHP" they are nearly identical so it works well enough for tracking things down. Regex is always a possible source of improvement, and you are right, pretty much every token uses Regex at it's core. I can already see a possible improvement in the Feel free to poke around if you notice any potential improvements. |
|
Wish I had the time! My personal priority is taking Marked (and other renderers such as highlight.js) out of the client critical path in my upcoming library by caching results server side where possible. |
What pain point are you perceiving?
Since MarkedJS's inception most major versions have increased time needed to complete parsing. The numbers shown below were captured on second page load /w 6x CPU slowdown - only Marked version being revised, nothing else. Marked processing time has increased more than 2x since v0.8.2
Version 0.8.2
Version 2.0.3
Version 4.0.9
Describe the solution you'd like
Perhaps it's time to look at an optimization pass and potential short circuits that could be taken to restore some previously lost performance? I feel performance should remain somewhat consistent...
The text was updated successfully, but these errors were encountered: