Slow rendering with OffthreadVideo

[coleatkinson1]

First of all I just want to say that Remotion is amazing! I'm having a lot of fun with it :)

I have a composition that contains two side-by-side OffthreadVideo components. Each of the videos are ~40 minutes long, and are static files in my public dir. The composition also contains a few other basic HTML elements. The video also contains intro and outro compositions, and this is all housed in Sequence components.

Rendering starts off fine, but after about 20%-30% of the rendering is complete (by frame count) it grinds to a crawl, and renders at about 5fps and continues to slow.

This is with default render settings, and I have tried a few of the options, without any luck. The same thing occurs with faster render presets, and other encoders. Does anybody have an idea as to how I can attempt to solve this? Or is there an inherent issue with off-thread videos? I previously rendered this with shorter videos (~5 mins) and didn't have any issue.

Thanks

[JonnyBurger]

Hey @coleatkinson1!

A few things you can try:

• Increase the Offthread video cache size (https://www.remotion.dev/docs/renderer/render-media#offthreadvideocachesizeinbytes, also available anywhere else). Maybe the video gets busted from the cache and needs to be re-opened again
• Render the video with --log=verbose, you will get very exact timings about which frames are extracted, the seeking positions in the video and how long it took to extract frames.

Do you see a lot of "cache pruned" messages in the logs? Then you could also fix it by increasing the cache.
If you see something strange that is not explainable and you think Remotion should do better, you may also send me the video for investigation and I'll see if I can improve it. [email protected]

[coleatkinson1]

Hi @JonnyBurger , thanks for the fast reply :)

Rendering with verbose logging, I am see a lot of cache pruned messages. I increased the cache to ~8gb, which is the maximum free memory that I have available on my system, and I'm still seeing a lot of these messages:
Got packet dts = Some(1072128) pts = Some(1072128) key = false compositor received frame 1072128 compositor Pruned 3 to save memory, keeping 3109. Total cache size: 8189MB compositor Pruned 1 to save memory, keeping 3108. Total cache size: 8189MB compositor Pruned 1 to save memory, keeping 3107. Total cache size: 8189MB compositor Pruned 1 to save memory, keeping 3106. Total cache size: 8189MB compositor Got packet dts = Some(1249792) pts = Some(1250816) key = false compositor received frame 1249792 compositor Got packet dts = Some(1250304) pts = Some(1250304) key = false compositor received frame 1250304 compositor Got packet dts = Some(1250816) pts = Some(1251840) key = false compositor received frame 1250816 compositor Got packet dts = Some(1251328) pts = Some(1251328) key = false compositor received frame 1251328 compositor Pruned 2 to save memory, keeping 3108. Total cache size: 8191MB compositor Got packet dts = Some(1072640) pts = Some(1073664) key = false compositor received frame 1072640 compositor Got packet dts = Some(1073152) pts = Some(1073152) key = false compositor received frame 1073152

FYI I'm running this on ubuntu 22.04. I'm going to try rendering on Windows 10 to see if I get similar results. I also ran it on a VPS (running ubuntu), and it timed out with this error:
Error: Timeout exceeded rendering the component. at /root/Remotion/srcomparison/node_modules/@remotion/renderer/dist/seek-to-frame.js:44:28 at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

[JonnyBurger]

@coleatkinson1 This makes me think that you are including a video with very few keyframes.

FFmpeg has to decode a big portion of the video before it receives the frame that we want.

If you can, re-encode the video with FFmpeg and it should speed up the render afterwards

My offer still stands to send the video for introspection

[coleatkinson1]

@JonnyBurger
I re-encoded the video with FFmpeg, which didn't have any impact.

To test whether it was something that I was doing wrong in that project, I started a new remotion project with a blank canvas, which only has 1 composition with 1 offthreadvideo (also tested with the normal Video component). I tested with multiple different videos of length 30mins - 45mins and they all had the same issue. I also tried rendering on different machines, with the same result. They all encountered lots of cache pruning messages and significant reduction in render/encoding speed at ~20% and would continue to slow.

This leads me to believe that there is some fundamental issue with the video component(s) when using videos >30mins in length.

[JonnyBurger]

I can only examine the problem if I have the video.

[coleatkinson1]

Sorry, do you want me to send one of the videos that I used offthreadvideo component?

[JonnyBurger]

Yes please! For example over Discord or [email protected]. I'll do a render myself.

[JonnyBurger]

Thanks for sending the video! I can see some slowdown as well once the cache pruning hits.
I'll look into if we can optimize this.

[tzvc]

Running into the same issue! Rendering a 1m video containing an offthread video with a 1h source takes ages.