heif-convert not showing image

[rickysarraf]

I think I may have found a bug in the heif-convert tool used in the photoprism builds.

Consider the below message from the current preview branch build:

photoprism_1  | 2024-02-22T09:30:01.019534915Z time="2024-02-22T09:30:01Z" level=info msg="index: ignored \".Moved\""                                                                             
photoprism_1  | 2024-02-22T09:30:01.019859438Z time="2024-02-22T09:30:01Z" level=info msg="index: ignored \".dtrash\""                                                                            
photoprism_1  | 2024-02-22T09:30:03.743863261Z time="2024-02-22T09:30:03Z" level=info msg="index: ignored \"Encoded\""                                                                            
photoprism_1  | 2024-02-22T09:30:03.745474947Z time="2024-02-22T09:30:03Z" level=info msg="index: ignored \"Imgs\""
photoprism_1  | 2024-02-22T09:30:03.746046641Z time="2024-02-22T09:30:03Z" level=info msg="index: ignored \"configs\""
photoprism_1  | 2024-02-22T09:30:03.826699014Z time="2024-02-22T09:30:03Z" level=info msg="convert: converting IMG20240221203917.heic to IMG20240221203917.heic.jpg (heif-convert)"
photoprism_1  | 2024-02-22T09:30:04.361180446Z time="2024-02-22T09:30:04Z" level=debug msg="faces: found no orphan markers [153.326979ms]"
photoprism_1  | 2024-02-22T09:30:04.788562586Z time="2024-02-22T09:30:04Z" level=debug msg="markers: found no invalid references [427.354251ms]"
photoprism_1  | 2024-02-22T09:30:04.789331262Z time="2024-02-22T09:30:04Z" level=debug msg="markers: found no missing subjects [716.737µs]"
photoprism_1  | 2024-02-22T09:30:10.476847770Z time="2024-02-22T09:30:10Z" level=debug msg="faces: found no ambiguous subjects [5.687433513s]"
photoprism_1  | 2024-02-22T09:30:10.477988625Z time="2024-02-22T09:30:10Z" level=debug msg="faces: found no clusters to be merged [1.105375ms]"
photoprism_1  | 2024-02-22T09:30:10.594137235Z time="2024-02-22T09:30:10Z" level=debug msg="faces: skipped clustering"
photoprism_1  | 2024-02-22T09:30:10.594174546Z time="2024-02-22T09:30:10Z" level=debug msg="faces: found no new faces [116.138593ms]"
photoprism_1  | 2024-02-22T09:30:10.594885664Z time="2024-02-22T09:30:10Z" level=debug msg="faces: found no unmatched markers"
photoprism_1  | 2024-02-22T09:30:10.676002324Z time="2024-02-22T09:30:10Z" level=debug msg="faces: updated 0 markers, recognized 0 faces, 0 unknown [81.793469ms]"
photoprism_1  | 2024-02-22T09:30:10.679655895Z time="2024-02-22T09:30:10Z" level=debug msg="index: updating counts"
photoprism_1  | 2024-02-22T09:30:10.734102136Z time="2024-02-22T09:30:10Z" level=debug msg="counts: updated 0 places [54.431675ms]"
photoprism_1  | 2024-02-22T09:30:10.913915392Z time="2024-02-22T09:30:10Z" level=debug msg="counts: updated 0 subjects [179.758334ms]"
photoprism_1  | 2024-02-22T09:30:11.176690329Z time="2024-02-22T09:30:11Z" level=debug msg="counts: updated 0 labels [262.730781ms]"
photoprism_1  | 2024-02-22T09:30:11.192065839Z time="2024-02-22T09:30:11Z" level=info msg="index: updated 1 file [10.479064594s]"
photoprism_1  | 2024-02-22T09:30:11.262213113Z time="2024-02-22T09:30:11Z" level=debug msg="covers: updated 0 folders [70.182466ms]"
photoprism_1  | 2024-02-22T09:30:11.262248749Z time="2024-02-22T09:30:11Z" level=debug msg="removed folder:/:true:false from cache"
photoprism_1  | 2024-02-22T09:30:14.521209946Z time="2024-02-22T09:30:14Z" level=debug msg="purge: updated 0 files [3.258826524s]"
photoprism_1  | 2024-02-22T09:30:14.522454643Z time="2024-02-22T09:30:14Z" level=debug msg="purge: updated 0 duplicates [1.099402ms]"
photoprism_1  | 2024-02-22T09:30:14.671715311Z time="2024-02-22T09:30:14Z" level=debug msg="purge: updated 0 photos [149.322932ms]"
photoprism_1  | 2024-02-22T09:30:14.808087186Z time="2024-02-22T09:30:14Z" level=debug msg="index: found no photos without primary file [136.250022ms]"
photoprism_1  | 2024-02-22T09:30:14.969885201Z time="2024-02-22T09:30:14Z" level=debug msg="index: found no orphan files [113.564258ms]"
photoprism_1  | 2024-02-22T09:30:15.571670767Z time="2024-02-22T09:30:15Z" level=debug msg="search: updated index [308.319668ms]"
photoprism_1  | 2024-02-22T09:30:15.840049428Z time="2024-02-22T09:30:15Z" level=debug msg="index: updating counts"

It gives a message that the file was converted (correct word should be generated) but nothing is the resultant output. And thus, that image is never displayed in the photo view, once the indexing is complete.

Looking further into the photoprism docker instance, I found:

photoprism@:/photoprism/originals$ heif-convert ./2024/February/IMG20240221203917.heic
Unknown file type in 
Warning: Invalid EXIF text encoding for UserComment - ./2024/February/IMG20240221203917.heic
Error: Zero-length file name - ""
photoprism@:/photoprism/originals$ 

photoprism@:/photoprism/originals$ heif-convert -h
 heif-convert  libheif version: 1.15.1
-------------------------------------------
Usage: heif-convert [options]  <input-image> <output-image>

The program determines the output file format from the output filename suffix.
These suffices are recognized: jpg, jpeg, png, y4m.
Options:
  -h, --help              show help
  -q, --quality           quality (for JPEG output)
  -d, --decoder ID        use a specific decoder (see --list-decoders)
      --with-aux          also write auxiliary images (e.g. depth images)
      --with-xmp          write XMP metadata to file (output filename with .xmp suffix)
      --with-exif         write EXIF metadata to file (output filename with .exif suffix)
      --skip-exif-offset  skip EXIF metadata offset bytes
      --no-colons         replace ':' characters in auxiliary image filenames with '_'
      --list-decoders     list all available decoders (built-in and plugins)
      --quiet             do not output status messages to console
No file specified
Error: Zero-length file name - ""

photoprism@:/photoprism/originals$ ls -lh ./2024/February/IMG20240221203917.heic
-rwxrw-rw- 1 photoprism photoprism 1.8M Feb 21 15:09 ./2024/February/IMG20240221203917.heic


photoprism@:/photoprism/originals$ cd
photoprism@:~$ 
exit
15:42 ♒♒♒   ☺    

The above is very weird. Made me feel my image is corrupt.

But on further check on the host machine (Debian Testing), I get:

$ heif-convert ./2024/February/IMG20240221203917.heic
File contains 1 image
Written to ./2024/February/IMG20240221203917.jpg


rrs@chutzpah: $ file ./2024/February/IMG20240221203917.jpg
./2024/February/IMG20240221203917.jpg: JPEG image data, JFIF standard 1.01, aspect ratio, density 1x1, segment length 16, Exif Standard: [TIFF image data, little-endian, direntries=12, height=4000, manufacturer=OnePlus, model=OnePlus Open, orientation=upper-left, xresolution=158, yresolution=166, resolutionunit=2, datetime=2024:02:21 20:39:17, GPS-Data, width=3008], baseline, precision 8, 3008x4000, components 3
17:04 ♒♒♒   ☺    
rrs@chutzpah: $ file ./2024/February/IMG20240221203917.heic
./2024/February/IMG20240221203917.heic: ISO Media, HEIF Image HEVC Main or Main Still Picture Profile
17:04 ♒♒♒   ☺    
rrs@chutzpah: $ 


rrs@chutzpah: $ heif-convert -h
 heif-convert  libheif version: 1.17.6
-------------------------------------------
Usage: heif-convert [options]  <input-image> [output-image]

The program determines the output file format from the output filename suffix.
These suffixes are recognized: jpg, jpeg, png, y4m. If no output filename is specified, 'jpg' is used.

Options:
  -h, --help                     show help
  -v, --version                  show version
  -q, --quality                  quality (for JPEG output)
  -o, --output FILENAME          write output to FILENAME (optional)
  -d, --decoder ID               use a specific decoder (see --list-decoders)
      --with-aux                 also write auxiliary images (e.g. depth images)
      --with-xmp                 write XMP metadata to file (output filename with .xmp suffix)
      --with-exif                write EXIF metadata to file (output filename with .exif suffix)
      --skip-exif-offset         skip EXIF metadata offset bytes
      --no-colons                replace ':' characters in auxiliary image filenames with '_'
      --list-decoders            list all available decoders (built-in and plugins)
      --quiet                    do not output status messages to console
  -C, --chroma-upsampling ALGO   Force chroma upsampling algorithm (nn = nearest-neighbor / bilinear)
      --png-compression-level #  Set to integer between 0 (fastest) and 9 (best). Use -1 for default.
17:05 ♒♒♒   ☺    

@lastzero Do you think this is a bug ? Maybe just an outdated version of libheif1 that needs to be updated ?

[rickysarraf]

After the manually generated jpeg, the index was good enough to stack it along with the heic image. So, I think the libheif1 version shipped in Debian Bookworm, of which the photoprism build is that I'm using, is the cause of the problem.

photoprism_1  | 2024-02-22T11:51:34.595547429Z time="2024-02-22T11:51:34Z" level=info msg="index: ignored \".Moved\""                                                                             
photoprism_1  | 2024-02-22T11:51:34.595595389Z time="2024-02-22T11:51:34Z" level=info msg="index: ignored \".dtrash\""
photoprism_1  | 2024-02-22T11:51:36.375060929Z time="2024-02-22T11:51:36Z" level=info msg="index: ignored \"Encoded\""
photoprism_1  | 2024-02-22T11:51:36.375481368Z time="2024-02-22T11:51:36Z" level=info msg="index: ignored \"Imgs\""
photoprism_1  | 2024-02-22T11:51:36.375496311Z time="2024-02-22T11:51:36Z" level=info msg="index: ignored \"configs\""
photoprism_1  | 2024-02-22T11:51:36.446190722Z time="2024-02-22T11:51:36Z" level=info msg="index: added main heic file 2024/February/IMG20240221203917.heic"                                      
photoprism_1  | 2024-02-22T11:51:36.449187507Z time="2024-02-22T11:51:36Z" level=debug msg="exiftool: extracting metadata from 2024/February/IMG20240221203917.jpg"                               
photoprism_1  | 2024-02-22T11:51:36.640859643Z time="2024-02-22T11:51:36Z" level=debug msg="cache: created b8accea139e35e64ccaa7002ebbcbdcd0d8bfeca_exiftool.json"                                
photoprism_1  | 2024-02-22T11:51:36.675854303Z time="2024-02-22T11:51:36Z" level=debug msg="metadata: no XMP data in IMG20240221203917.jpg (read xmp)"                                            
photoprism_1  | 2024-02-22T11:51:37.502526441Z time="2024-02-22T11:51:37Z" level=debug msg="media: opened 2024/February/IMG20240221203917.jpg [48 MB]"
photoprism_1  | 2024-02-22T11:51:38.125579203Z time="2024-02-22T11:51:38Z" level=info msg="media: created 8 thumbnails for 2024/February/IMG20240221203917.jpg [1.466607405s]"                    
photoprism_1  | 2024-02-22T11:51:39.062445894Z time="2024-02-22T11:51:39Z" level=debug msg="photo: changing location of 2024/February/IMG20240221203917 from zz to s2:XXXXXXXXXXXXX"
photoprism_1  | 2024-02-22T11:51:39.080141069Z time="2024-02-22T11:51:39Z" level=debug msg="photo: 2024/February/IMG20240221203917 title based on label 'XXXXXXX"                           
photoprism_1  | 2024-02-22T11:51:39.080182569Z time="2024-02-22T11:51:39Z" level=debug msg="photo: 2024/February/IMG20240221203917 has new title 'XXXXXXXX' [872.282µs]"                                                                                                                                                                                                 
photoprism_1  | 2024-02-22T11:51:39.178096638Z time="2024-02-22T11:51:39Z" level=debug msg="search: updated index by photo id [14.172782ms]"                                                      

[rickysarraf]

Thank you. That explains why I was exceptionally seeing the issue.

[lastzero]

I've removed this from our Debian Dockerfiles to avoid any further confusion: 11d27da

[lastzero]

@rickysarraf I also created additional LibHEIF binaries yesterday, including versions for ARMv7 and Debian Bookworm:

• https://dl.photoprism.app/dist/libheif/

Unfortunately, v1.17.6 seems to introduce regressions with respect to the image orientation, so I had to revert back to v1.17.1.

Any help comparing the versions and testing with HEIC images that have a non-default orientation (e.g. portrait) would be much appreciated!

[rickysarraf]

Dear @lastzero

Would you be able to confirm that there's no /usr/local/bin/heif-convert in your photoprism/photoprism:bookworm images ?
I build the PhotoPrism docker images locally, using your bookworm image. I think there's a leftover script in /usr/local/bin/

[lastzero]

I've removed it last week, but only from the develop image and not from the one for production.