FAQ  •  Register  •  Login

new files not refreshed - due to DVR lock?

<<

Brett

Serviio newbie

Posts: 12

Joined: Thu Mar 12, 2015 4:39 am

Post Thu Mar 12, 2015 5:03 am

new files not refreshed - due to DVR lock?

Hi,

I just updated/upgraded (Pro) and my "auto" refresh no longer works.

The logs indicate the delta scanner is detecting the new file.
However, the file never gets added to the library.

My DVR (BeyondTV) records directly into the directory watched by Serviio.
So, I'm not sure if this problem is due to the file being locked, unreadable or incomplete.

In the attached log, search for "The Late Late Show".

You can see Serviio is doing something with the new file. The following log is repeated until the recording stops at 11:52

2015-03-11 23:51:43,827 DEBUG [MediaItemDAOImpl] Looking up a media item for file path: F:\dvr\The Late Late Show-2015-03-11-0.tp, ignore case: true

The logging for this file stops and the file is never added to the library.

Note that a manual "Force Refresh" picks up the new files.
If I rename the file, it is detected by the delta scanner and added to the library.
If I copy the file, it is detected by the delta scanner and added to the library.

With this version of Serviio I never see new DVR'ed files w/o a manual "Force Refresh". :^(


System info:
Windows 7
F: is an external drive
Serviio PRO v 1.5.1

Thanks for any assistance.
Brett
Attachments
serviio.log.refresh.problem.gz
(10.16 KiB) Downloaded 419 times
<<

zip

User avatar

Serviio developer / Site Admin

Posts: 17212

Joined: Sat Oct 24, 2009 12:24 pm

Location: London, UK

Post Thu Mar 12, 2015 9:53 pm

Re: new files not refreshed - due to DVR lock?

yep, that is not working anymore in 1.5. I assume when the DVR starts storing the file, Serviio detects it and at that point it's not complete. It will then try to add the file, if the filesize keeps increasing.

What is your detailed log settings? It seems to me there are some entries missing.

Alternatively you could try to use POLLER for your shared folders. http://wiki.serviio.org/doku.php?id=lib ... er_scanner
<<

Brett

Serviio newbie

Posts: 12

Joined: Thu Mar 12, 2015 4:39 am

Post Fri Mar 13, 2015 4:32 am

Re: new files not refreshed - due to DVR lock?

zip wrote:yep, that is not working anymore in 1.5. I assume when the DVR starts storing the file, Serviio detects it and at that point it's not complete. It will then try to add the file, if the filesize keeps increasing.

What is your detailed log settings? It seems to me there are some entries missing.

Alternatively you could try to use POLLER for your shared folders. http://wiki.serviio.org/doku.php?id=lib ... er_scanner


Logs are set to debug. This is the only thing I changed in log4j.xml

<category name="org.serviio">
<priority value="DEBUG"/>
</category>

The logs do indicate that it's watching the file. And, perhaps, after the file stops increasing it attempts something. But, there are no ERROR logs and no indication of successfully adding the new file to the library.

I will set the poller property to workaround this issue. But, having the delta-scanner detect in real-time is pretty awesome.

Any chance you could add more debug logging around the logic in question? The next release I can retry, collect logs and report back.

Thanks!
Brett
<<

BadBunny

User avatar

Streaming enthusiast

Posts: 26

Joined: Sat Mar 14, 2015 9:56 pm

Post Sun Mar 15, 2015 12:32 am

Re: new files not refreshed - due to DVR lock?

Hello,

I am also having the problem of the library not refreshing automatically with v1.5.1 on Linux Mint 17.1 KDE 64bit.
Is this the correct way to try forcing the POLLER mechanism in the serviio.sh file?

# Setup Serviio specific properties
JAVA_OPTS="-serviio.forceLibraryPoller=true -serviio.libraryPollerFrequency=30 -Djava.net.preferIPv4Stack=true -Djava.awt.headless=true -Dorg.restlet.engine.loggerFacadeClass=org.restlet.ext.slf4j.Slf4jLoggerFacade -Dderby.system.home=$SERVIIO_HOME/library -Dserviio.home=$SERVIIO_HOME -Dffmpeg.location=ffmpeg -Ddcraw.location=dcraw"


I have tried it a couple of times now and it seems to work ok until i restart my PC, then Serviio crashes on startup with the following error:

lee@BadBunnix:~/Programs/serviio-1.5.1/bin > ./serviio.sh
Unrecognized option: -serviio.forceLibraryPoller=true
Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.


(It is set to auto-start but after it has failed to start this is the error I get when attempting to manually start it)

Thank you.
Windows 7 64bit / Linux Mint 17.1 KDE 64bit (Dual boot)
Serviio v1.5.2
Enjoys a pint of fine wine and as many french fries as I can stuff in my face.
<<

zip

User avatar

Serviio developer / Site Admin

Posts: 17212

Joined: Sat Oct 24, 2009 12:24 pm

Location: London, UK

Post Sun Mar 15, 2015 11:49 am

Re: new files not refreshed - due to DVR lock?

You have to prefix the parameters with '-D', so

  Code:
-Dserviio.forceLibraryPoller=true
<<

BadBunny

User avatar

Streaming enthusiast

Posts: 26

Joined: Sat Mar 14, 2015 9:56 pm

Post Sun Mar 15, 2015 1:18 pm

Re: new files not refreshed - due to DVR lock?

Thank you Zip, that worked to start Serviio without the error.

The only problem now is that it still does not auto-scan to reflect changes to my scanned folders, I tested both local linux folders and auto-mounted SMB shares on the same computer (It dual-boots with Windows 7). If i start the Serviio console and press the refresh button then then it scans and the changes are made.
I set a 5 minute scan interval to test with.
Windows 7 64bit / Linux Mint 17.1 KDE 64bit (Dual boot)
Serviio v1.5.2
Enjoys a pint of fine wine and as many french fries as I can stuff in my face.
<<

BadBunny

User avatar

Streaming enthusiast

Posts: 26

Joined: Sat Mar 14, 2015 9:56 pm

Post Sun Mar 15, 2015 1:51 pm

Re: new files not refreshed - due to DVR lock?

I tell a lie, I waited longer and Serviio is recognizing changes made to the local Linux folder but not the local SMB shares.
Windows 7 64bit / Linux Mint 17.1 KDE 64bit (Dual boot)
Serviio v1.5.2
Enjoys a pint of fine wine and as many french fries as I can stuff in my face.
<<

zip

User avatar

Serviio developer / Site Admin

Posts: 17212

Joined: Sat Oct 24, 2009 12:24 pm

Location: London, UK

Post Sun Mar 15, 2015 8:41 pm

Re: new files not refreshed - due to DVR lock?

Post detailed log of startup and then adding a file to the samba share, please.

Also tell me what is the samba share path and what is the added file path.
<<

BadBunny

User avatar

Streaming enthusiast

Posts: 26

Joined: Sat Mar 14, 2015 9:56 pm

Post Mon Mar 16, 2015 6:41 pm

Re: new files not refreshed - due to DVR lock?

Sorry, I am only using Linux a couple of weeks now and mixed up the terminology a bit.
I am not using SMB shares, I have a few folders from local Windows drives that are auto-mounted on startup.
(It's a dual-boot Windows 7 / Linux Mint 17.1 KDE system)

Logs are attached. Thanks.

Badbunny.log.tar.gz
(16.78 KiB) Downloaded 431 times
Windows 7 64bit / Linux Mint 17.1 KDE 64bit (Dual boot)
Serviio v1.5.2
Enjoys a pint of fine wine and as many french fries as I can stuff in my face.
<<

zip

User avatar

Serviio developer / Site Admin

Posts: 17212

Joined: Sat Oct 24, 2009 12:24 pm

Location: London, UK

Post Mon Mar 16, 2015 10:02 pm

Re: new files not refreshed - due to DVR lock?

Thanks, so I see this in your log:

  Code:
2015-03-16 19:27:33,474 INFO  [RepositoriesStatus] Repository '/home/lee/Video 1' is not currently available. Deferring all operations for later.
2015-03-16 19:27:33,499 INFO  [RepositoriesStatus] Repository '/home/lee/Video 2' is not currently available. Deferring all operations for later.
2015-03-16 19:27:33,522 INFO  [RepositoriesStatus] Repository '/home/lee/Video 3' is not currently available. Deferring all operations for later.
2015-03-16 19:27:33,539 INFO  [RepositoriesStatus] Repository '/media/lee/2Tb Storage 1/Downloaded Movies' is not currently available. Deferring all operations for later.
2015-03-16 19:27:33,576 INFO  [LibraryDeltaScanner] Added repository '/home/lee/Videos' to delta scanner (POLLER)


It looks like only /home/lee/Videos is available, the other 3 are either disconnected drives or they don't exist anymore or the access rights are not correct.

Can you confirm which folder you added a file to?

POLLER is used, btw - from out beta testing this week, it looks like inotify (which the NATIVE scanner on Linux is based on) doesn't work well with the Samba shares, so you'll need to use POLLER for these folders.
<<

BadBunny

User avatar

Streaming enthusiast

Posts: 26

Joined: Sat Mar 14, 2015 9:56 pm

Post Mon Mar 16, 2015 10:12 pm

Re: new files not refreshed - due to DVR lock?

Hi Petr, thanks for the reply.

/home/lee/Videos is a folder in my Linux Home directory.
The other 3 are Windows folders that I mounted in Linux, and for this test I added video files to /home/lee/Video 2.

I had already set POLLER as follows:

# Setup Serviio specific properties
JAVA_OPTS="-Dserviio.forceLibraryPoller=true -Dserviio.libraryPollerFrequency=5 -Djava.net.preferIPv4Stack=true -Djava.awt.headless=true -Dorg.restlet.engine.loggerFacadeClass=org.restlet.ext.slf4j.Slf4jLoggerFacade -Dderby.system.home=$SERVIIO_HOME/library -Dserviio.home=$SERVIIO_HOME -Dffmpeg.location=ffmpeg -Ddcraw.location=dcraw"


If i manually press the Refresh button in the Serviio console my media are added OK to the 3 Windows folders.
Windows 7 64bit / Linux Mint 17.1 KDE 64bit (Dual boot)
Serviio v1.5.2
Enjoys a pint of fine wine and as many french fries as I can stuff in my face.
<<

zip

User avatar

Serviio developer / Site Admin

Posts: 17212

Joined: Sat Oct 24, 2009 12:24 pm

Location: London, UK

Post Mon Mar 16, 2015 10:30 pm

Re: new files not refreshed - due to DVR lock?

Ok. later in the log you can see

  Code:
2015-03-16 19:27:53,590 DEBUG [RepositoriesStatus] Repository /home/lee/Video 1 has become available
2015-03-16 19:27:53,590 DEBUG [RepositoriesStatus] Repository /home/lee/Video 2 has become available
2015-03-16 19:27:53,591 DEBUG [RepositoriesStatus] Repository /home/lee/Video 3 has become available
2015-03-16 19:27:53,591 DEBUG [RepositoriesStatus] Repository /media/lee/2Tb Storage 1/Downloaded Movies has become available


So combined with what i posted above, it looks like these 4 drives are not available on the server startup - for one of the listed reasons and then become available roughly 20 seconds later. At this point it should start detecting the files.

Any idea when you added the file - ie, was it while thew server was starting or within 1 minutes of it starting? or later?
<<

BadBunny

User avatar

Streaming enthusiast

Posts: 26

Joined: Sat Mar 14, 2015 9:56 pm

Post Mon Mar 16, 2015 10:48 pm

Re: new files not refreshed - due to DVR lock?

I had added a file before I restarted the computer, and later even added a few more, though none showed up.

I just made a test. I created a new folder on the same drive that has my Video 1 folder.
I added that folder in the Serviio console using the 'Add Path' option instead of using the 'Add Local' option.
I put a test video file in that folder and waited.
A few minutes later I saw it had been detected by Serviio automatically.

I will now re-add my real folders in the same way and make further tests, but I think this is the solution.
I will update here and let you know how it goes.
Many thanks for all your help!
Windows 7 64bit / Linux Mint 17.1 KDE 64bit (Dual boot)
Serviio v1.5.2
Enjoys a pint of fine wine and as many french fries as I can stuff in my face.
<<

zip

User avatar

Serviio developer / Site Admin

Posts: 17212

Joined: Sat Oct 24, 2009 12:24 pm

Location: London, UK

Post Mon Mar 16, 2015 10:54 pm

Re: new files not refreshed - due to DVR lock?

Oh - is there a difference in the log on how it detects the drive? Something else instead of '/home/lee/Video 1' ?

It should not really make a difference.
<<

BadBunny

User avatar

Streaming enthusiast

Posts: 26

Joined: Sat Mar 14, 2015 9:56 pm

Post Tue Mar 17, 2015 11:39 am

Re: new files not refreshed - due to DVR lock?

This seems to have been a false alarm, i am testing it some more and found that it only adds media to the library when i press the Force Refresh button in the console or the moment after I add a new folder to scan.

And using the Add Path option added my folder in the same way as Add Local, according to the logs, so as you say, it does not make any difference.

And I found the difference between this:

  Code:
2015-03-16 19:27:33,474 INFO  [RepositoriesStatus] Repository '/home/lee/Video 1' is not currently available. Deferring all operations for later.
2015-03-16 19:27:33,499 INFO  [RepositoriesStatus] Repository '/home/lee/Video 2' is not currently available. Deferring all operations for later.
2015-03-16 19:27:33,522 INFO  [RepositoriesStatus] Repository '/home/lee/Video 3' is not currently available. Deferring all operations for later.
2015-03-16 19:27:33,539 INFO  [RepositoriesStatus] Repository '/media/lee/2Tb Storage 1/Downloaded Movies' is not currently available. Deferring all operations for later.
2015-03-16 19:27:33,576 INFO  [LibraryDeltaScanner] Added repository '/home/lee/Videos' to delta scanner (POLLER)


and this:

  Code:
2015-03-16 19:27:53,590 DEBUG [RepositoriesStatus] Repository /home/lee/Video 1 has become available
2015-03-16 19:27:53,590 DEBUG [RepositoriesStatus] Repository /home/lee/Video 2 has become available
2015-03-16 19:27:53,591 DEBUG [RepositoriesStatus] Repository /home/lee/Video 3 has become available
2015-03-16 19:27:53,591 DEBUG [RepositoriesStatus] Repository /media/lee/2Tb Storage 1/Downloaded Movies has become available


At the first point the computer has just started up but has not yet been logged in, so it seems the drives are only becoming available (and still not being automatically scanned) once I log in.
Windows 7 64bit / Linux Mint 17.1 KDE 64bit (Dual boot)
Serviio v1.5.2
Enjoys a pint of fine wine and as many french fries as I can stuff in my face.
<<

zip

User avatar

Serviio developer / Site Admin

Posts: 17212

Joined: Sat Oct 24, 2009 12:24 pm

Location: London, UK

Post Tue Mar 17, 2015 1:33 pm

Re: new files not refreshed - due to DVR lock?

yes, that's what I posted earlier.

I tried that on my machine with a USB drive (detached, started Serviio, added a file) and it picked up the file.

Can you try to stop the Serviio server (not the PC), start it again (at this point the drives should be already available) and se if it picks up changes then? If not, please post another log.
<<

BadBunny

User avatar

Streaming enthusiast

Posts: 26

Joined: Sat Mar 14, 2015 9:56 pm

Post Tue Mar 17, 2015 9:49 pm

Re: new files not refreshed - due to DVR lock?

I have been playing about with this during the day, looking at folder permissions of current and new folders to see if there was a visible difference, nothing was different that i saw but i played about ticking various options for the folders to see if that helped but nothing did.

I had the idea that somehow the word 'video' in the folder name might be a problem as my previous test folders on the same hard drive with other names had worked for a few files, so I set up a few new folders and put some video files into them and this does randomly at first seem to work, but stops after one or two files.

I had left a video file in one of my original video folders during all this and the only time it was recognized was when i had added the new folder path to the Serviio console and pressed save, after which i saw it made a new scan.

From what I have found so far this is only working reliably when the video files are located in my 'Videos' drive that is a part of my Linux Home folder.

Logs are attached.

serviio.log.tar.gz
(28.92 KiB) Downloaded 405 times
Windows 7 64bit / Linux Mint 17.1 KDE 64bit (Dual boot)
Serviio v1.5.2
Enjoys a pint of fine wine and as many french fries as I can stuff in my face.
<<

zip

User avatar

Serviio developer / Site Admin

Posts: 17212

Joined: Sat Oct 24, 2009 12:24 pm

Location: London, UK

Post Tue Mar 17, 2015 9:58 pm

Re: new files not refreshed - due to DVR lock?

Could you try what I asked above pls?
<<

BadBunny

User avatar

Streaming enthusiast

Posts: 26

Joined: Sat Mar 14, 2015 9:56 pm

Post Tue Mar 17, 2015 10:26 pm

Re: new files not refreshed - due to DVR lock?

I had done that as you asked during my various tests, but just did it once more to make it clearer in the logs attached.

Thanks.

serviio.log.tar.gz
(45 KiB) Downloaded 420 times
Windows 7 64bit / Linux Mint 17.1 KDE 64bit (Dual boot)
Serviio v1.5.2
Enjoys a pint of fine wine and as many french fries as I can stuff in my face.
<<

zip

User avatar

Serviio developer / Site Admin

Posts: 17212

Joined: Sat Oct 24, 2009 12:24 pm

Location: London, UK

Post Tue Mar 17, 2015 11:49 pm

Re: new files not refreshed - due to DVR lock?

Could you stop the server, remove all the logs, then start, wait 1 minute, add a file, wait 2 more minutes and post the log? This last log doesn't seem to include the server starting up.
Next

Return to User experiences

Who is online

Users browsing this forum: No registered users and 18 guests

Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Designed by ST Software for PTF.