Leaking temp files in PHP-FPM when max_execution_time interrupts the script

[zerodeux]

This is a follow-up of ImageMagick/ImageMagick#395 (comment)

I encountered a case where a PHP app ran imagick-related jobs that were too long wrt. to the max_execution_time limit. As a consequence, temporary image files woud not being cleaned up and would pile up in /tmp.

At this moment in time, I'm not sure if the PHP app's framework is responsible for explicitly creating the temp files or if they are automatically created by imagick (lack of RAM, swapping to disk).

I am wondering if the imagick extension would take care of releasing the Imagick objects and implicitly remove temp files in the case of a request terminated by a max_execution_time limit ?

Because if that's not the case, I would recommend to the PHP developpers to make sure to always define a cleanup function with register_shutdown_function() and push the reponsability to them.

[exetico]

It's not lack of ram; as I've seen this going on, with plenty of free ram. Not even close to the limit.
One of the system's there I'm seeing it in WordPress does have 16GB of ram, and never jumps above 8GB. There's even a swap file of 2GB, but it's never touched, cause the ram does handle every possible demand. The resources are monitored, so I can guarantee that it's not even close to the 16GB.

The bug shows periodically. The most problematisk thing is that a uploaded file of 400KB, turns into a 300-450MB file in /tmp, not ever to be cleaned up.

I've been waiting with a clean-up solution, but today I ended up adding a job to the crontab, and extended our /tmp space to 5GB. It's a real pain, and I'm very surprised that it's still a thing. Googling the specific issue shows plenty of cases with people blaming WordPress, hosting solutions, LiteSpeed and so on 😆 ... But in the end Imagick are the one, eating up the /tmp folder.

[zerodeux]

It's not lack of ram

Indeed it's not, Imagemagick switches from RAM to swapfile using a configured limit, see the <policy domain="resource" ...> tags in your /etc/ImageMagick*/policy.xml.

This is documented and controlled behaviour.

I have not tested it, but I guess setting <policy domain="resource" name="disk" value="0"/> would prevent ImageMagick to create swapfiles, but in exchange you would get 1/ well... no leftover swapfiles and 2/ aborted image operations when they don't feet the RAM budget (as configured by policy.xml and as limited by the effectively available RAM). That's a tradeoff some people would be willing to select on production servers, because using a swapfile is much more expensive than plain RAM, and to scale you might need to guarantee that image operations dont generate unbounded IOs.

[linuxmalaysia]

Our PHP application using tcpdf thats will call imagick to produce a PDF output. This will create large imagick temporay files in /tmp . We have users thats will generate their report in pdf. This create many imagick temp files that nearly 400MB each that eatup free space. The report not larger then 1MB.

During peak hours, the execution time increase and users may aborted the PDF report generation, due to its take longer time to produce. This will leave the imagick temp files in /tmp without been deleted.

We have create a cron to monitor this behavior and remove them if needed.

We increase the max_execution_time to monitor this will help during peak hours.