Library refresh on 1.5

No its not updating...
Also on windows it does not work

[zip]

That's not very helpful without a detailed log. Could you provide some pls?

[zip]

Actually, I think I can reproduce it on a Mac, will try to fix and release a new version if required.

I think the problem has been identified, but I would like to add my experience nonetheless.

I'm running a clean install of Serviio 1.5.1 on ubuntu server 14.04 running java 8 and I can confirm that library refresh doesn't work.
I even enabled debug mode, but when I add a file nothing happens after 30 minutes or more.
If I login into the web console "Keep library automatically updated" is flagged and when I click Save it updates the library once. Then again no more changes detected until I click Save again.
Both with or without "Search for updates of currently shared files" enabled.

my java is:
openjdk version "1.8.0_01-internal"
OpenJDK Runtime Environment (build 1.8.0_01-internal-b04)
OpenJDK Server VM (build 25.40-b08, mixed mode)

logs after adding a file:
2015-03-05 13:38:44,679 DEBUG [FeedUpdaterWorker] Checking for new and expired online resources
2015-03-05 13:38:44,679 DEBUG [OnlineRepositoryDAOImpl] Reading all OnlineRepositories
2015-03-05 13:38:44,681 DEBUG [OnlineRepositoryDAOImpl] Reading an OnlineRepository (id = 2)
2015-03-05 13:38:45,917 DEBUG [SearchManager] Committing search index
2015-03-05 13:39:15,917 DEBUG [SearchManager] Committing search index
2015-03-05 13:39:44,683 DEBUG [FeedUpdaterWorker] Checking for new and expired online resources
2015-03-05 13:39:44,683 DEBUG [OnlineRepositoryDAOImpl] Reading all OnlineRepositories
2015-03-05 13:39:44,684 DEBUG [OnlineRepositoryDAOImpl] Reading an OnlineRepository (id = 2)
2015-03-05 13:39:45,918 DEBUG [SearchManager] Committing search index
[CUT]
and so on and on

[zip]

I think the problem has been identified, but I would like to add my experience nonetheless.

I'm running a clean install of Serviio 1.5.1 on ubuntu server 14.04 running java 8 and I can confirm that library refresh doesn't work.
I even enabled debug mode, but when I add a file nothing happens after 30 minutes or more.
If I login into the web console "Keep library automatically updated" is flagged and when I click Save it updates the library once. Then again no more changes detected until I click Save again.
Both with or without "Search for updates of currently shared files" enabled.

my java is:
openjdk version "1.8.0_01-internal"
OpenJDK Runtime Environment (build 1.8.0_01-internal-b04)
OpenJDK Server VM (build 25.40-b08, mixed mode)

logs after adding a file:
2015-03-05 13:38:44,679 DEBUG [FeedUpdaterWorker] Checking for new and expired online resources
2015-03-05 13:38:44,679 DEBUG [OnlineRepositoryDAOImpl] Reading all OnlineRepositories
2015-03-05 13:38:44,681 DEBUG [OnlineRepositoryDAOImpl] Reading an OnlineRepository (id = 2)
2015-03-05 13:38:45,917 DEBUG [SearchManager] Committing search index
2015-03-05 13:39:15,917 DEBUG [SearchManager] Committing search index
2015-03-05 13:39:44,683 DEBUG [FeedUpdaterWorker] Checking for new and expired online resources
2015-03-05 13:39:44,683 DEBUG [OnlineRepositoryDAOImpl] Reading all OnlineRepositories
2015-03-05 13:39:44,684 DEBUG [OnlineRepositoryDAOImpl] Reading an OnlineRepository (id = 2)
2015-03-05 13:39:45,918 DEBUG [SearchManager] Committing search index
[CUT]
and so on and on

Can you post the detailed log of the server starting and the library engine kicking in (about 20 secs after the server starts)? You should see something like here: http://wiki.serviio.org/doku.php?id=lib ... delta_scan

I have a different problem with 1.5.1:

Everything is working fine, i mean: when I disconnect an external drive, the library doesn't clean up and this is very desirable. But the problem occurs when I copy some files inside a folder that serviio is watching. For example after i copy a movie, sometimes happens that when I am watching it, it stops at a certain time, let's say 32:14. When i try watch the movie again, there is no way to go further than 32:14. I think this is related to serviio not updating correctly the file. I have "search for updates of the currently shared file" turned on, by the way.
That's why I sticked to 1.4.1. It sucks when you have to rescan all the library, but when it scans it does it in the right way.

Thanks for developing serviio

[zip]

I have a different problem with 1.5.1:

Everything is working fine, i mean: when I disconnect an external drive, the library doesn't clean up and this is very desirable. But the problem occurs when I copy some files inside a folder that serviio is watching. For example after i copy a movie, sometimes happens that when I am watching it, it stops at a certain time, let's say 32:14. When i try watch the movie again, there is no way to go further than 32:14. I think this is related to serviio not updating correctly the file. I have "search for updates of the currently shared file" turned on, by the way.
That's why I sticked to 1.4.1. It sucks when you have to rescan all the library, but when it scans it does it in the right way.

Thanks for developing serviio

What is your OS and are the files on a local drive? When you copy a file, can you see in a detailed log if it starts adding the file (running ffmpeg -i) before it finished copying it? If yes, it might detect the file with a shorter duration than what it really is.

Yes, i will provide you with the detailed log, give me few days to reinstall 1.5.1 and repeat the test (quite busy). I strongly agree with you about ffmpeg, because the file was good opened with samba. By the way i have a raspberry pi 2 with raspbian. I compiled ffmpeg as in the guide for raspberry (it took hours). If it is that ffmpeg is analyzing the file too quickly, how can avoid it? Why with force refresh it doesn't work? Does serviio store the hash of the scanned files?

[zip]

Yes, i will provide you with the detailed log, give me few days to reinstall 1.5.1 and repeat the test (quite busy). I strongly agree with you about ffmpeg, because the file was good opened with samba. By the way i have a raspberry pi 2 with raspbian. I compiled ffmpeg as in the guide for raspberry (it took hours). If it is that ffmpeg is analyzing the file too quickly, how can avoid it? Why with force refresh it doesn't work? Does serviio store the hash of the scanned files?

I'm not sure if this is the case, should see from the log.

I had the same problem with v1.5.1 on CentOS 7.0: When removing or adding files, nothing was updated. Due to several other issues I went back to v1.4.1.2.

Dude, i tried to reproduce the same but i cannot for the moment. If you think about it it's logical: i don't know how the scanning is implemented, but if ffmpeg starts too soon, it's obvious that that files are somehow corrupted. Hashing files it's easy, costless and i think it can completely avoid to scan the file again for no reason or resolve add/update issues. The idea of the removable drives was great, but far from perfect as 1.4.1. When i will reproduce the error i will send the whole log to you. Today i only used 1.5.1 and the issue was that the downloaded file was not seen after some time (like 30 min or 1h) after finished download. I had to force refresh. With 1.4.1 everything was added or removed to library after 5 minutes from the time of complete download

A new video don't show up in Serviio 1.5.1 when browsing for it. You find attrached the serviio.log, ffmpeg -i dump and system information. I have processed the file with HandBrake, but it does not show up in the file list on the TV. The original unprocessed file does. I have been processing files with the same setup for months. The only changes are the Serviio update to 1.5.1 and java runtime update to 1.8_31 64 bit version.
I can add that other files ripped or processed with MakeMKV and handBrake don't show up either. All files processed after installing Servioo 1.5.1.

Serviio.log:
2015-03-08 08:32:57,023 ERROR [ProcessExecutor] Fatal error in process starting: Cannot run program "cmd": CreateProcess failed, error=216
java.io.IOException: Cannot run program "cmd": CreateProcess failed, error=216
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
at org.serviio.external.ProcessExecutor.run(ProcessExecutor.java:135)
Caused by: java.io.IOException: CreateProcess failed, error=216
at java.lang.ProcessImpl.create(Native Method)
at java.lang.ProcessImpl.<init>(ProcessImpl.java:386)
at java.lang.ProcessImpl.start(ProcessImpl.java:137)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
... 1 more
2015-03-08 08:32:57,023 WARN [ProcessExecutor] Cannot retrieve results, output reader is null
2015-03-08 08:32:57,023 WARN [MetadataExtractionCoordinator] Skipping processing metadata for an unsupported file (C:\mkv\Video.mkv). Message: Unknown video file type.

C:\mkv>"C:\Program Files (x86)\Serviio\lib\"ffmpeg -i Video.mkv
ffmpeg version 2.4.git Copyright (c) 2000-2014 the FFmpeg developers
built on Nov 19 2014 22:12:09 with gcc 4.7.2 (GCC)
configuration: --enable-static --disable-shared --disable-ffplay --disable-ffserver --enable-memalign-hack --enable-libmp3lame --enable-libass --enable-librtm
p --enable-fontconfig --enable-libfreetype --enable-zlib --enable-libx264 --enable-libspeex --extra-libs='-lrtmp -lpolarssl -lws2_32 -lwinmm -lexpat -lfreetype
-lfribidi -lz' --arch=x86 --enable-runtime-cpudetect --enable-pthreads --target-os=mingw32 --cross-prefix=i686-w64-mingw32- --enable-gpl --pkg-config=pkg-config

libavutil 54. 14.100 / 54. 14.100
libavcodec 56. 12.101 / 56. 12.101
libavformat 56. 14.100 / 56. 14.100
libavdevice 56. 3.100 / 56. 3.100
libavfilter 5. 2.103 / 5. 2.103
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 1.100 / 1. 1.100
libpostproc 53. 3.100 / 53. 3.100
[matroska,webm @ 03C762E0] Could not find codec parameters for stream 2 (Subtitle: hdmv_pgs_subtitle): unspecified size
Consider increasing the value for the 'analyzeduration' and 'probesize' options
[matroska,webm @ 03C762E0] Could not find codec parameters for stream 3 (Subtitle: hdmv_pgs_subtitle): unspecified size
Consider increasing the value for the 'analyzeduration' and 'probesize' options
Input #0, matroska,webm, from 'Video.mkv':
Metadata:
encoder : libmkv 0.6.5
Duration: 01:40:13.29, start: 0.000000, bitrate: 9665 kb/s
Chapter #0:0: start 0.000000, end 671.833000
Metadata:
title : Chapter 1
Chapter #0:1: start 671.833000, end 1175.417000
Metadata:
title : Chapter 2
Chapter #0:2: start 1175.417000, end 1813.542000
Metadata:
title : Chapter 3
Chapter #0:3: start 1813.542000, end 2474.417000
Metadata:
title : Chapter 4
Chapter #0:4: start 2474.417000, end 2914.458000
Metadata:
title : Chapter 5
Chapter #0:5: start 2914.458000, end 3501.417000
Metadata:
title : Chapter 6
Chapter #0:6: start 3501.417000, end 4200.250000
Metadata:
title : Chapter 7
Chapter #0:7: start 4200.250000, end 4753.125000
Metadata:
title : Chapter 8
Chapter #0:8: start 4753.125000, end 5240.500000
Metadata:
title : Chapter 9
Chapter #0:9: start 5240.500000, end 5934.708000
Metadata:
title : Chapter 10
Chapter #0:10: start 5934.708000, end 6013.291500
Metadata:
title : Chapter 11
Stream #0:0(eng): Video: h264 (High), yuv420p(tv, bt709), 1920x1088, SAR 1:1 DAR 30:17, 24 fps, 24 tbr, 1k tbn, 180k tbc (default)
Stream #0:1(non): Audio: dts (DTS-HD MA), 48000 Hz, 5.1(side), fltp, 1536 kb/s (default)
Stream #0:2(non): Subtitle: hdmv_pgs_subtitle
Stream #0:3(non): Subtitle: hdmv_pgs_subtitle
[NULL @ 0471ADA0] Unable to find a suitable output format for 'ffmpeg'
ffmpeg: Invalid argument

System information
OS updated: Windows Home Premium
Java version: 1.8_031 64 bit version
Serviio version: 1.5.1 pro (3 days left)
HandBrake version: 0.9.9.5530

[zip]

@janrou - what happens if you try to execute 'cmd' on your PC? i mean, open command prompt and type

  Code:

cmd


same problem here on a Mac Mini with Yosemite.
With 1.5 version the library update was instant ...spectacular. Upgrading to 1.5.1 the auto refresh wont work and I have to update manually.

I would to upgrade my serviio.sh file to reduce the time and force the upgrade. I found this string:
... -Ddcraw.location="$SERVIIO_HOME/lib/dcraw -Dserviio.LibraryPollerFrequency=5" ....
where i can add serviio.forceLibraryPoller=true? And, why -Dserviio??

An updated version of serviio will very appreciated

[zip]

working on it. On Mac the update will be every 5 minutes now, as it was otherwise using too much CPU for large libraries.

ok, perfect. Thank you

I wanna add an info: seems that the 5 minutes upgrade works with audio and images but not with video. very strange.
I have 3 folders on the same SSD

i have the same issue, it takes a while to update the library, it waits like 5-10 min before its updated, its really annoying
the period of *instant* needs to be abit more *instant*

working on it. On Mac the update will be every 5 minutes now, as it was otherwise using too much CPU for large libraries.

please atleast give us a option to change it. i dont mind using a little more cpu for instant file add to libary

My library will not refresh at all. I can't watch any any videos that weren't in my library before I updated to 1.5.1.
I even tried going back to the previous version. Didn't work. I re-installed 1.5.1 and still can't see new files. Pretty much makes Serviio useless.
Force rescan does nothing. I even tried to add new folders. They don't show up.
I am using Windows version.

serviio - Copy (2).zip

My log file

(270.57 KiB) Downloaded 286 times

What worked for me:
1- Removed all shared folders from the library.
2- Uninstalled Serviio.
3- Deleted installation folder (Windows OS. Don't know if possible in Mac).
4- Reinstalled Serviio.
5- Readded shared folders to library.
Note: It will take a lot of time (hours in my case) to rebuild the library. My strategy was to add first what I wanted to watch 'urgently' (TV Shows) and later I added the movies, videos, pictures and music.

All that shouldn't be necessary. I already tried reinstalling the previous version. That didn't work. Plus I lost everything that I had set up with online sources. Where is the file that contains all of that? Can I save that and add it back after the re-install?

Thanks.