Page 1 of 1

Slowing down ffmpeg scraping

PostPosted: Mon Feb 04, 2013 10:17 pm
by mqojdn
In what order Serviio works?
- Runs extractUrl for all items in the feed and then starts with ffmpeg probing.
or
- Runs extractUrl and ffmpeg for each item and then proceeds to the next.

I'm asking because some sites detect the scraping and begin timing out on ffmpeg requests. And if Serviio goes item by item then I can slow it down by not returning from the plugin until the 29th second.
If not, then I propose returning some variable from extractUrl, holding the "wait before next" value.

Re: Slowing down ffmpeg scraping

PostPosted: Mon Feb 04, 2013 10:52 pm
by zip
The former

Re: Slowing down ffmpeg scraping

PostPosted: Tue Feb 05, 2013 9:59 am
by mqojdn
Here is a sample http://pastie.org/pastes/6051906/download?key=xen0ogeinqtnt0wuw0bynw (the links are temporary, not sure for how long), just add that as an RSS feed. Some of the items wouldn't load because they allow only 2 concurrent connections per IP.
And for example putlocker.com would start timing out and forwarding to a captcha page, when they detect excessive scraping.