Page 1 of 1

Customizable per-folder media rescan period

PostPosted: Tue Feb 14, 2012 8:24 pm
by lampsound
In my local home network i have a NAS server and Popcorn C200 mediaplayer.
Currently Serviio handles both of them and flawlessly translates their content to Sony Bravia televisions in kitchen and bedroom (so big thanks to you, zip :) ).
Popcorn has two internal hard drives and placed in bedroom. Small spin-down time set to both of drives (via smartctl) because WD 20EARS is noisy sometimes and this noise usually can significantly disturb the sleep.

My feature request is to have customizable per-folder media rescan period.
For example - 15 minutes rescan period for NAS and 04:00AM-06:00AM each day for Popcorn.
I have some Java experience and will be glad to help you implement this feature (with Quartz Scheduler lib, for example), GUI console changes will be very small.

Re: Customizable per-folder media rescan period

PostPosted: Tue Feb 14, 2012 8:42 pm
by zip
I don't think I'll be putting any effort in this as with Java 7 it should be notification based. Once java 7 is widely available I'll implement that.

Re: Customizable per-folder media rescan period

PostPosted: Tue Feb 14, 2012 8:48 pm
by lampsound
zip wrote:I don't think I'll be putting any effort in this as with Java 7 it should be notification based. Once java 7 is widely available I'll implement that.

Java 7 will provide a WatchService:
https://blogs.oracle.com/thejavatutoria ... or_changes

  Code:
WatchService watcher = FileSystems.getDefault().newWatchService();
// add a user-defied file-change listener


But this don't change the general behavior - hard drive should spin-up to provide file-system changes to WatchService and this spin-up cause noise. So i think this spin-up's can be made controllable and customizable.

Re: Customizable per-folder media rescan period

PostPosted: Tue Feb 14, 2012 8:59 pm
by zip
I assume they will only spin up when you copy a new file there, at which point Serviio will get the notification.

Re: Customizable per-folder media rescan period

PostPosted: Tue Feb 14, 2012 9:31 pm
by lampsound
zip wrote:I assume they will only spin up when you copy a new file there, at which point Serviio will get the notification.

After little googling i found some technical details about WatchService in JDK 7 - it works 100% good only with local filesystem.

People reports this:
http://stackoverflow.com/questions/8476419/java-watchservice-not-generating-events-while-watching-mapped-drives
http://www.java.net/node/701540

Here it's javadoc:
"If a watched file is not located on a local storage device then it is implementation specific if changes to the file can be detected. In particular, it is not required that changes to files carried out on remote systems be detected."

My opinion is that hand-made file changes polling will be potentially more deterministic, controllable and stable.
Also here is some alternatives to hand-made changes poller:
http://commons.apache.org/io/api-2.0/org/apache/commons/io/monitor/FileAlterationObserver.html

Later i can made a little test with JDK7 on Windows and file changes on Popcorn (it's linux based device with Samba).

Re: Customizable per-folder media rescan period

PostPosted: Tue Feb 14, 2012 9:59 pm
by lampsound
A little test:
  Code:
import java.nio.file.*;
import java.util.List;

public class WatcherTest {
    private static final String PATH = "\\\\POPCORN\\ShareCenter\\Video";
//    private static final String PATH = "\\\\STORAGE\\ShareCenter\\Public";

    public static void main(String[] args) throws Exception {
        FileSystem fs = FileSystems.getDefault();
        WatchService ws = fs.newWatchService();

        System.out.println(String.format("Start polling for [%s]", PATH));
        Path path = fs.getPath(PATH);
        path.register(ws, StandardWatchEventKinds.ENTRY_CREATE, StandardWatchEventKinds.ENTRY_MODIFY, StandardWatchEventKinds.OVERFLOW, StandardWatchEventKinds.ENTRY_DELETE);
        WatchKey watchKey;
        do {
            watchKey = ws.take();
            List<WatchEvent<?>> events = watchKey.pollEvents();
            for (WatchEvent object : events) {
                if (object.kind().equals(StandardWatchEventKinds.ENTRY_MODIFY)) {
                    System.out.println("Modify: " + object.context().toString());
                }
                if (object.kind().equals(StandardWatchEventKinds.ENTRY_DELETE)) {
                    System.out.println("Delete: " + object.context().toString());
                }
                if (object.kind().equals(StandardWatchEventKinds.ENTRY_CREATE)) {
                    System.out.println("Created: " + object.context().toString());
                }
            }
        } while (watchKey.reset());
    }
}


The results for application running with JDK7-final on Windows 7 x64 are:
1. WatchService correctly handles changes on remote drive hosted at Windows-based PC (\\STORAGE in example)
2. WatchService exits immediately (ws.take() returns empty list and watchKey.reset() returns 'false') on remote drive hosted at Linux-based device with Samba (\\POPCORN in example)

Re: Customizable per-folder media rescan period

PostPosted: Tue Feb 14, 2012 11:03 pm
by zip
thanks, I had no idea. Looks like the Apache class might be a way to go. It's I assume polling for new files, updates and deletions, so there would still be some delay and CPU usage.

Re: Customizable per-folder media rescan period

PostPosted: Tue Apr 17, 2012 1:29 am
by Anaerin
From the looks of it, it appears you can "fall-through", so if using the Java nio classes fails to start the watch (returns immediately), you can begin polling (as serviio does at the moment) instead. This will give maximum flexibility with minimum breakage and minimum CPU usage when available. The only thing I'd really suggest testing is if there is any situation where the watcher starts but doesn't actually return.

Re: Customizable per-folder media rescan period

PostPosted: Tue Apr 17, 2012 1:59 am
by jhb50
Using curl and an ffmpeg intercept, I've implemented a refresh on demand capability for any folder. I use this for rapidly changing sports and news online feeds to get the latest changes on demand and let the folder refresh on serviios console cycle. Best of both worlds and less cpu load. See the wiki for my current implementation which I plan to improve with a better interface to the library id.