FAQ  •  Register  •  Login

Interrupted library scan - upper limit of 11886 pictures?

<<

Rolf_is_watching

Serviio newbie

Posts: 4

Joined: Fri Sep 14, 2018 4:10 pm

Post Fri Sep 14, 2018 4:43 pm

Interrupted library scan - upper limit of 11886 pictures?

Dear owner, dear experts,

for many years I was happy user of Serviio (1.9.2 and earlier) on Windows 10 and earlier. But since several months, Serviio 1.9.2 doesn't serve my content completely any more. It started with Serviio having problems to find and serve added files correctly. So I switched to "polling" and using "Force refresh", as described in other posts. This worked for a while. But in the mean time, after more images were added to my hard disk, even this doesn't help any more to serve all files. So I did a clean-up and tried from scratch. This is the steps I took and the resulting behavior:

- Uninstall Serviio 1.9.2
- Delete library and old logs
- Reinstall Serviio 1.9.2
- Turn on DEBUG in file log4j.xml
- Stop server
- Add single folder D:\Bilder\Fotos in polling mode to library. It contains 38526 image files in three layers of subfolders
- Wait until scanning ended, result:

Serviio stops scanning after 11886 files.
This is reported by the Serviio GUI and confirmed by the Serviio log.
But actually there are 38526 pictures.
-> Library is incomplete


- Click "Force refresh"

Numbers and served content doesn't change.
Only the log file contains two more lines:
2018-09-14 18:04:47,442 INFO [LocalLibraryManager] Starting manual library scan
2018-09-14 18:04:47,443 INFO [LibraryOneTimeScanner] Interrupting library scan


To me it looks like some unintended interrupt of the library scan.
I checked the folder where scanning stopped. There is no specifically suspicious file there:
No corrupt file, all with read permission, no unusual file or image size, no unusual file format.
I tried this also with Serviio 1.9.1 - same problem.

I attach my log files.
I also tried to attach my database, but it's too big, even when zipped (~4MB).
So I tried to create an archive split into files smaller than 500kB. Unfortunately that results in more than the allowed 3 attachments.
I can provide the library by email to anybody who is willing to help.

I would really like to stay with Serviio (instead of movint to Plex, for example), but I need help to solve the reported issue.

Thanks,
Rolf
Attachments
log.zip
(155.44 KiB) Downloaded 185 times
<<

Rolf_is_watching

Serviio newbie

Posts: 4

Joined: Fri Sep 14, 2018 4:10 pm

Post Fri Sep 14, 2018 4:48 pm

Re: Interrupted library scan - upper limit of 11886 pictures

In case, this helps to analyze:
The library database itself under C:\Program Files\Serviio\library is roughly 350MB in size.
Most of it is the following single file:
c590.dat with 328MB.
<<

Rolf_is_watching

Serviio newbie

Posts: 4

Joined: Fri Sep 14, 2018 4:10 pm

Post Fri Nov 23, 2018 10:46 pm

Re: Interrupted library scan - upper limit of 11886 pictures

Problem still exists in Serviio PRO v 1.10 :(
<<

atc98092

User avatar

DLNA master

Posts: 5205

Joined: Fri Aug 17, 2012 10:22 pm

Location: Washington (the state)

Post Sat Nov 24, 2018 6:38 pm

Re: Interrupted library scan - upper limit of 11886 pictures

I just looked through the logs you posted, and only found an error in one of them. The error was this:
  Code:
2018-09-14 17:38:31,895 WARN  [MediaFileIndexer] Cannot add file D:\Bilder\Fotos\2013\2013_05_31 Kroatien Crikvenica Dramalj Gleitschirmurlaub\20130531-170100.jpg because of an unexpected error. Message: Cannot read Folder with name = 2013

That error appears to have stopped the library scan from continuing. Since it appears that your files have names in a non-English language, I might guess that you might have a character in a filename that is tripping up the scanner. Zip will need to weigh in on what that might be. But you might look over your folder structure, especially the D:\Bilder\Fotos\2013 folder, since that's the one that tripped it up. Maybe create a new folder with a slightly different name, move your photos into the new folder, and see if it will now scan. Perhaps there's something corrupted in that file within Windows.

I don't think there's a limit to the number of media files that can be scanned, as I've heard of other users with more than that. But I don't have anywhere near that number so can't speak from experience. Since I don't speak or read anything but English, I doubt I can be of much more assistance.
Dan

LG NANO85 4K TV, Samsung JU7100 4K TV, Sony BDP-S3500, Sharp 4K Roku TV, Insignia Roku TV, Roku Ultra, Premiere and Stick, Nvidia Shield, Yamaha RX-V583 AVR.
Primary server: Intel i5-6400, 16 gig ram, Windows 10 Pro, 22 TB hard drive space | Test server Windows 10 Pro, AMD Phenom II X4 965, 8 gig ram

HOWTO: Enable debug logging HOWTO: Identify media file contents
<<

Rolf_is_watching

Serviio newbie

Posts: 4

Joined: Fri Sep 14, 2018 4:10 pm

Post Sat Nov 24, 2018 9:20 pm

Re: Interrupted library scan - upper limit of 11886 pictures

Dear atc98092,
thank you very much for looking into the problem and log file.

Yes, I also considered earlier that special characters (ÄÖÜäöüß) could be the root cause. But I found out, that there are actually many other directories with those special characters successfully scanned.
I also found, that the scan does not stop always at the same file and also not after the same number of files. So it seemingly is not a particular file or particular folder which breaks it.

Another suspicion I had, was that JPGs with broken meta data could be the root cause. But I could not confirm this. I will try to attach the file you identified in the log. It is OK and meta data can also be parsed by all other programs I have in use.
EDIT: OK, file is too big to attach, sorry.

I have no further ideas. I switched over to Plex again for my pictures (movies I still serve via Serviio), because 1.10 also didn't solve the problem. I really would like to serve movies and pictures again with Serviio.

Rolf


atc98092 wrote:I just looked through the logs you posted, and only found an error in one of them. The error was this:
  Code:
2018-09-14 17:38:31,895 WARN  [MediaFileIndexer] Cannot add file D:\Bilder\Fotos\2013\2013_05_31 Kroatien Crikvenica Dramalj Gleitschirmurlaub\20130531-170100.jpg because of an unexpected error. Message: Cannot read Folder with name = 2013

That error appears to have stopped the library scan from continuing. Since it appears that your files have names in a non-English language, I might guess that you might have a character in a filename that is tripping up the scanner. Zip will need to weigh in on what that might be. But you might look over your folder structure, especially the D:\Bilder\Fotos\2013 folder, since that's the one that tripped it up. Maybe create a new folder with a slightly different name, move your photos into the new folder, and see if it will now scan. Perhaps there's something corrupted in that file within Windows.

I don't think there's a limit to the number of media files that can be scanned, as I've heard of other users with more than that. But I don't have anywhere near that number so can't speak from experience. Since I don't speak or read anything but English, I doubt I can be of much more assistance.
<<

LaKn57

Serviio newbie

Posts: 3

Joined: Mon Feb 11, 2019 11:47 pm

Post Sat Feb 16, 2019 8:27 pm

Re: Interrupted library scan - upper limit of 11886 pictures

I have been trying to get Serviio Pro 1.10.1 working on a large number of images, but have given up for the time being.
Before I increased the heapspace of the derby server, i also reached a limit of just less than 120,000 images.
Serviio server would stop in matter of minutes.
After increasing the heapspace alotted to derby I increased that number upto 400,000 + images, but Serviio becomes unstable.
Unable to load image, Timeout failures.
What I have seen in the derby log :
Can't get itemscount for mediatype errors and out of memory errors.
There is a ticket for memoryfootprint of the derby server and I think this could be related to the limited amount of images
you can handle.

LaKn57

Return to Serviio Support & Help

Who is online

Users browsing this forum: Google [Bot] and 29 guests

Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Designed by ST Software for PTF.