How lazy-decoding mechanism works

[hp1909]

Hello, I'm investigating my application bug related to displaying animated images (APNG). Therefore, I'm reading the SDWebImage source code and don't quite understand exactly the lazy-decoding mechanism. I'd be very appreciate if someone could explain it. Thank you in advance!

[dreampiggy]

Apple's ImageIO and UIKit use lazy decoding.

CGImage is lazy, not a data which hold memory buffer. It query the pixel buffer using callback (not like Android Bitmap or other things you familiar)

A CGImage can be created with CGDataProvider, which can provide the callback to access underlying pixel buffer (a 2-dimension array). So this not means the data must be always in memory (surprise ?)

See: https://developer.apple.com/documentation/coregraphics/cgdataprovidergetbytescallback?language=objc

So, ask yourself:

If CGImage is lazy, when UI render on screen (this happended on main queue), will this effect frame rate ?

For JPEG and PNG, the lazy cause the decoding happened on main queue, but iPhone or iPad's hardware-accelerate codec is fast enough, so it's OK

But for WebP/HEIC, this is not quite enough, especially for low model like iPhone 11 before. It use software codec.

So, this is why we introduce the force decode, force the lazy to become non-lazy in the background queue, then render on screen does not trigger decoding at all.

But force decode will also allocate pixel buffer in the background queue before rendering, so this will pre-allocate memory, and may contains more memory footprint.

In previous 10 years (iOS 4-iOS 14), UIKit team hide this internal detail, but this indeed effect performance. We provide the choice to you.

In iOS 15, UIKit team finally provide the public API to do force decode as well, see: https://developer.apple.com/documentation/uikit/uiimage/3750834-preparingfordisplay

[hp1909]

Thank you so much for the detailed explanation!

[dreampiggy]

https://github.com/path/FastImageCache

Expain the lazy decode steps in

The compressed image data is decoded into its uncompressed bitmap form, which is typically a very CPU-intensive operation.3

[dreampiggy]

If you want, just test by yourself, create a UIImage using UIImage imageWithData:, then render it into UIImageView on screen.

Set a breakpoint at CGDataProviderRetainBytePtr and see when it get called, and step to see the actual decoding happended (you will see like AppleJPEGPlugin or PNGPlugin, GIFPlugin actual codec doing decoding.