Post Thu Mar 22, 2018 1:52 pm

Workaround/Bug: photo thumbnail creation during indexing

Hi,

I have described some odd behaviour of Serviio when indexing large photo libraries and/or large photos (both have issues). The bug report can be found here:
-> https://bitbucket.org/xnejp03/serviio/issues/1057/buggy-photo-thumbnail-creation-during

This is my problem:

I have a directory of 250000 of my photos I'd like to include in Serviio's Images service, only useful if it provides thumbnails too. There are four associated problems here:

  1. The server process simply crashes for the larger of my photos, with OutOfMemory exceptions. The larger photos are about 80 MPixels JPGs (medium format camera).
  2. The server process simply crashes if the database grows larger. My Serviio database grows to about 64 GB on disk.
  3. I can resolve both issues above by providing 4GB memory to Java in the bin/serviio.sh script (MacOS). I.e., I modify file /Library/Application Support/bin/serviio.sh. However, with each update, this file gets overidden (although changes to log4j are honoured). Maybe, serviio.sh could include a javaconfig.sh file after defaults are set?
  4. All of the above can be worked around, except for this: the indexing of my library takes more than a week to finish!
I understand why the library with many thumbnails grows large. However, I propose to use a similiar approach to ffmpeg (like ImageMagick) to create image/photo thumbnails. Java has better libraries than BufferedImage but may be too cumbersome to integrate. It would be easier to fork thumbnail creation off for both image and video, rather than just video. This additionally frees memory to remain available to use an in-memory database with many thumbnails. Alternatively, Serviio could store thumbnail files outside the database.

In any case, 512MB heap space is NOT enough for real world usage of Serviio with a large photo library, i.e., when the feature to index images is to be used seriously.