Problem with dailymotion.groovy

Ok Petr...

For me, some streams work... My own personal one does not

http://dailymotion.com/rss/user/tstrike34/1

*scratches head*

I'm confused : when I search on DM with your user id (tstrike34)
I get
Résultats de recherche pour tstrike34
"Woops! Pas de vidéos disponibles pour cette recherche. "
(Search results for tstrike34
No videos available for this search)

It should include videos I have saved in my favorites list.

[zip]

This is the rss feed of the non working video : http://www.dailymotion.com/rss/video/xyrx4v
Of course you won't see it inside Serviio.
And a working one : http://www.dailymotion.com/rss/video/xyrzqs/
(I've not found how to create a single request for 2 different video_id)

Ok, so I tried the one that doesn't work and it seems it takes too long to get the results of FFmpeg -i for the stream. There is a limit of 1 minute to get the results and this failed on my PC. Therefore Serviio assumes the link is broken.

The problem (in my view Petr) is that Serviio has a very tough time dealing with old (or poorly done) TV/VCR rips that are uploaded to Dailymotion. I have other Dailymotion feeds that Serviio (I really mean ffmpeg) pefectly reads the metadata and feeds ffmpeg the parameters needed to do its thing.

Here is a perfect example of what I am talking about based on my serviio.log I watched this morning.

Here is the output I am getting from Serviio running on debug:

TV/VCR oriented feeds/uploads, I get this:

  Code:

2013-05-17 03:23:52,664 DEBUG [MediaItemDAOImpl] Reading dirty MediaItems for Repository (id = 1)
2013-05-17 03:23:52,696 DEBUG [RepositoryDAOImpl] Marking Repository 1 as scanned with current timestamp
2013-05-17 03:23:53,364 DEBUG [FeedParser] Found matching url extractor (DailyMotion) for resource http://www.dailymotion.com/rss/playlist/x2my
xv_tstrike34_mystuff
2013-05-17 03:23:53,366 DEBUG [FeedItemUrlExtractor] DailyMotion: Starting extraction of url for item: ECW Hardcore TV 7/3/1997
2013-05-17 03:23:54,156 DEBUG [FeedParser] Unexpected error during url extractor plugin invocation (DailyMotion) for item ECW Hardcore TV 7/3/1
997: index is out of range 0..-1 (index = 0)


Here is what is interesting... For the videos that appears to be DVD rips, Serviio can read them just fine... See here:

  Code:

2013-05-17 03:24:03,456 DEBUG [FeedItemUrlExtractor] DailyMotion: Starting extraction of url for item: ECW - Guilty As Charged 1999 Part 2/3
2013-05-17 03:24:04,246 DEBUG [FeedItemUrlExtractor] DailyMotion: Finished extraction of url: ContentURLContainer [fileType=VIDEO, contentUrl=h
ttp://www.dailymotion.com/cdn/H264-512x384/video/xzgngo.mp4?auth=1368951843-2a0906061cfa7e251fa6171fa37a149c, expiresImmediately=false, live=fa
lse, ]
2013-05-17 03:24:04,246 DEBUG [FeedParser] Added feed item 1: 'ECW - Guilty As Charged 1999 Part 2/3' (http://www.dailymotion.com/cdn/H264-512x
384/video/xzgngo.mp4?auth=1368951843-2a0906061cfa7e251fa6171fa37a149c)


And then once Serviio determines its metadata, it really starts processing (per the DailyMotion groovy plugin):

  Code:

2013-05-17 03:24:07,539 DEBUG [OnlineContentCacheDecorator] Stored online content 'http://www.dailymotion.com/rss/playlist/x2myxv_tstrike34_mys
tuff' in the cache (online_feeds), returning it
2013-05-17 03:24:07,539 DEBUG [OnlineLibraryManager] Resource http://www.dailymotion.com/rss/playlist/x2myxv_tstrike34_mystuff will expire in t
he cache on Mon May 20 16:24:07 CDT 2013
2013-05-17 03:24:07,540 DEBUG [FeedUpdaterThread] Retrieving file size from the URL connection
2013-05-17 03:24:08,092 DEBUG [FeedUpdaterThread] Retrieving information about the video stream 'ECW - Guilty As Charged 1999 Part 2/3'
2013-05-17 03:24:08,092 DEBUG [FFMPEGWrapper] Invoking FFMPEG to retrieve media information for file: http://www.dailymotion.com/cdn/H264-512x3
84/video/xzgngo.mp4?auth=1368951843-2a0906061cfa7e251fa6171fa37a149c
2013-05-17 03:24:08,092 DEBUG [ProcessExecutor] Starting ffmpeg -analyzeduration 10000000 -i http://www.dailymotion.com/cdn/H264-512x384/video/
xzgngo.mp4?auth=1368951843-2a0906061cfa7e251fa6171fa37a149c
2013-05-17 03:24:18,725 DEBUG [DiscoverySSDPMessageListener] Received a valid NOTIFY (ssdp:alive) message from Renderer 00000000-0000-1010-8000
-30f9ed75b44c from address /192.168.1.82:52323
2013-05-17 03:24:19,002 DEBUG [DiscoverySSDPMessageListener] Received a valid NOTIFY (ssdp:alive) message from Renderer 00000000-0000-1010-8000
-30f9ed75b44c from address /192.168.1.82:52323
2013-05-17 03:24:19,096 DEBUG [DiscoverySSDPMessageListener] Received a valid M-SEARCH message for search target urn:schemas-upnp-org:device:In
ternetGatewayDevice:1 from address /192.168.1.70:3840s
2013-05-17 03:24:29,476 DEBUG [FFmpegMetadataRetriever] Retrieving H264 profile/level for file 'http://www.dailymotion.com/cdn/H264-512x384/vid
eo/xzgngo.mp4?auth=1368951843-2a0906061cfa7e251fa6171fa37a149c'
2013-05-17 03:24:29,476 DEBUG [FFMPEGWrapper] Invoking FFMPEG to retrieve H264 header for file: http://www.dailymotion.com/cdn/H264-512x384/vid
eo/xzgngo.mp4?auth=1368951843-2a0906061cfa7e251fa6171fa37a149c
2013-05-17 03:24:29,477 DEBUG [ProcessExecutor] Starting ffmpeg -analyzeduration 10000000 -i http://www.dailymotion.com/cdn/H264-512x384/video/
xzgngo.mp4?auth=1368951843-2a0906061cfa7e251fa6171fa37a149c -frames:v 1 -c:v copy -f h264 -bsf:v h264_mp4toannexb -an pipe:
2013-05-17 03:24:50,889 DEBUG [FFmpegMetadataRetriever] File 'http://www.dailymotion.com/cdn/H264-512x384/video/xzgngo.mp4?auth=1368951843-2a09
06061cfa7e251fa6171fa37a149c' has H264 profile C_BASELINE, levels [{RF=1.3, H=2.1}] and 3 ref frames
2013-05-17 03:24:50,891 DEBUG [TechnicalMetadataCacheDecorator] Stored technical metadata for online item 'http://www.dailymotion.com/cdn/H264-
512x384/video/xzgngo.mp4?auth=1368951843-2a0906061cfa7e251fa6171fa37a149c' in the cache (online_technical_metadata), returning it
2013-05-17 03:24:50,897 DEBUG [FeedUpdaterThread] Retrieving file size from the URL connection
2013-05-17 03:24:51,393 DEBUG [FeedUpdaterThread] Retrieving information about the video stream 'ECW WrestlePalooza 1995 - UNRELEASED'
2013-05-17 03:24:51,393 DEBUG [FFMPEGWrapper] Invoking FFMPEG to retrieve media information for file: http://www.dailymotion.com/cdn/H264-512x3
84/video/xzhiy0.mp4?auth=1368951846-4c4d0052cd4a21f2dd0bc37fc1630e91
2013-05-17 03:24:51,393 DEBUG [ProcessExecutor] Starting ffmpeg -analyzeduration 10000000 -i http://www.dailymotion.com/cdn/H264-512x384/video/
xzhiy0.mp4?auth=1368951846-4c4d0052cd4a21f2dd0bc37fc1630e91


Ok, so I tried the one that doesn't work and it seems it takes too long to get the results of FFmpeg -i for the stream. There is a limit of 1 minute to get the results and this failed on my PC. Therefore Serviio assumes the link is broken.

I see. This could be caused by the size of this stream (>2Gb, 2:11:41).
Who's timing out ? FFmpeg waiting for the stream or Serviio waiting for FFmpeg ?
Could it be posible to make this time out a user defined setting in the future ?

Anyway, it's rare that we find stream of this size online so this is not a very big issue.
I've tried to find another big stream to test but didn't find one.

  Code:

2013-05-17 03:23:54,156 DEBUG [FeedParser] Unexpected error during url extractor plugin invocation (DailyMotion) for item ECW Hardcore TV 7/3/1
997: index is out of range 0..-1 (index = 0)


Petr,

There appears to be a problem with a get or set method that is causing the Column out of Index error.

Is this something we users (who have some coding experience) can fix ourselves?

It should include videos I have saved in my favorites list.

Well, it seems you're not known by DM as tstrike34 !
Except I'm wrong a search on this user name retrieves nothing...

This is getting slightly off topic

The problem (in my view Petr) is that Serviio has a very tough time dealing with old (or poorly done) TV/VCR rips that are uploaded to Dailymotion. I have other Dailymotion feeds that Serviio (I really mean ffmpeg) pefectly reads the metadata and feeds ffmpeg the parameters needed to do its thing.

The stream I have problem with is not an old TV rip.
It's an archive of a live stream streamed by DailyMotion a few weeks ago.

I have the feeling the problems you are reporting are nothing to do with mine.
Mine is about one particular very big stream that don't play when others from the same day (but smaller) of archived live streaming play perfectly.

May I suggest you open a different thread concerning your problem ?

It should include videos I have saved in my favorites list.

Well, it seems you're not known by DM as tstrike34 !
Except I'm wrong a search on this user name retrieves nothing...

This is getting slightly off topic

Its perfectly on topic. Its about the dailymotion.groovy plugin. As for the user... let me resolve that...

Its a username alias... you are probably on to something.

Its perfectly on topic. Its about the dailymotion.groovy plugin. As for the user... let me resolve that...

Its a username alias... you are probably on to something.

OK, I incorrectly named this thread. It was not intended for all dailymotion.groovy problems but for a particular one I have. Mea culpa, mea maxima culpa

BTW I'm on to nothing. When not logged to DM when I search with jdegreef (my user name in DM) I find me, when I search for tstrike34 I find nothing ! What do you mean alias ? In a previous post you used tstrike34 as your DM user name !
What's the url you use in Serviio to get your problematic streams ?

Its perfectly on topic. Its about the dailymotion.groovy plugin. As for the user... let me resolve that...

Its a username alias... you are probably on to something.

OK, I incorrectly named this thread. It was not intended for all dailymotion.groovy problems but for a particular one I have. Mea culpa, mea maxima culpa

BTW I'm on to nothing. When not logged to DM when I search with jdegreef (my user name in DM) I find me, when I search for tstrike34 I find nothing ! What do you mean alias ? In a previous post you used tstrike34 as your DM user name !

Yes I did. DM asked me if I wanted to change my username from their standard user naming convention (it was dm_50b021929c6a8)...

I usually loathe to create a single thread for a related subject matter, in fact I found my way here to this thread from a thread I started (someone put that link to this thread, so I merely followed).

May I suggest you open a different thread concerning your problem ?

Suggestion noted, but unfounded.

I found the root of my problem and landed in your problem roughly 15 minutes ago.

And now I agree... Seems like trying to stream or play big files with DailyMotion.groovy times out a bit too quickly. I was watching the real time (tail -f) writing to serviio.log and performed the same tests Petr had suggested

  Code:

ffmpeg -i " http://www.dailymotion.com/cdn/H264-512x384/video/x3ler0.mp4?auth=1369008349-1b86249a4a47baee3f7c9b297013a0ca"


Here was the output:

  Code:

ffmpeg version N-52953-g50e6672 Copyright (c) 2000-2013 the FFmpeg developers
  built on May 10 2013 14:16:15 with gcc 4.1.2 (GCC) 20080704 (Red Hat 4.1.2-54)
  configuration: --enable-version3 --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libvpx --enable-libfaac --enable-libmp3lame --enable-libtheora --enable-libx264 --enable-libxvid --enable-gpl --enable-postproc --enable-nonfree
  libavutil      52. 30.100 / 52. 30.100
  libavcodec     55.  7.100 / 55.  7.100
  libavformat    55.  4.101 / 55.  4.101
  libavdevice    55.  0.100 / 55.  0.100
  libavfilter     3. 63.101 /  3. 63.101
  libswscale      2.  3.100 /  2.  3.100
  libswresample   0. 17.102 /  0. 17.102
  libpostproc    52.  3.100 / 52.  3.100
 http://www.dailymotion.com/cdn/H264-512x384/video/x3ler0.mp4?auth=1369008349-1b86249a4a47baee3f7c9b297013a0ca: No such file or directory


Later on the links would expire as seen here:

  Code:

ffmpeg -i "http://www.dailymotion.com/cdn/H264-512x384/video/x33v1r.mp4?auth=1369008360-4c99673215247f84896c38c5d7c0fe90"
ffmpeg version N-52953-g50e6672 Copyright (c) 2000-2013 the FFmpeg developers
  built on May 10 2013 14:16:15 with gcc 4.1.2 (GCC) 20080704 (Red Hat 4.1.2-54)
  configuration: --enable-version3 --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libvpx --enable-libfaac --enable-libmp3lame --enable-libtheora --enable-libx264 --enable-libxvid --enable-gpl --enable-postproc --enable-nonfree
  libavutil      52. 30.100 / 52. 30.100
  libavcodec     55.  7.100 / 55.  7.100
  libavformat    55.  4.101 / 55.  4.101
  libavdevice    55.  0.100 / 55.  0.100
  libavfilter     3. 63.101 /  3. 63.101
  libswscale      2.  3.100 /  2.  3.100
  libswresample   0. 17.102 /  0. 17.102
  libpostproc    52.  3.100 / 52.  3.100
[http @ 0x189fe660] Invalid cookie found, no value, path or domain specified
[http @ 0x189fe5a0] HTTP error 404 Not Found
http://www.dailymotion.com/cdn/H264-512x384/video/x33v1r.mp4?auth=1369008360-4c99673215247f84896c38c5d7c0fe90: Input/output error


Weird.

Sidebar : I learned that a variety of special characters in the video's title name actually borks the groovy scripts' ability to extract the real URL of the video in DM. It warrants some examination (that is what led to my earlier inquiries).

*Edit*
I write code (mostly CORBA) so I was fascinated by the inner workings of Serviio. Appears more to me like the root cause centers around the activity of FFMPEG. Take note that I am using the latest and stable version of FFMPEG versus the compiled version that ships with Serviio 1.2.1... I would guess that would be ok?

So ... any merit to my observations?

So ... any merit to my observations?

Well, when I type http://www.dailymotion.com/rss/user/tstrike34/1 in my browser I get this (see screen capture).
What could I say ?

So ... any merit to my observations?

Well, when I type http://www.dailymotion.com/rss/user/tstrike34/1 in my browser I get this (see screen capture).
What could I say ?

I have no videos there...

I have built playlists from other videos.

With that said, please see my comments related to your earlier query. I believe both you and Petr have valid points regarding the delay in reading the metadata from the Dailymotion videos.

Tell me exactly how to reproduce your problem. What url do you put in Serviio ?
I still have the feeling that you problem is different from mine.
For me FFmpeg -i doesn't time out there's nothing in the log. It's only taking so long that Serviio times out waiting de response from FFmpeg.

Tell me exactly how to reproduce your problem. What url do you put in Serviio ?
I still have the feeling that you problem is different from mine.
For me FFmpeg -i doesn't time out there's nothing in the log. It's only taking so long that Serviio times out waiting de response from FFmpeg.

Please go back and look at the past output I posted.

In fact, you were the one that posted on my thread suggesting our issues are related. After a correction or two in configuration, I see the same time of issues.

If you closely read the DEBUG output from the serviio.log, and actively watches what happens during the reading of the metadata from videos from Dailymotion, you will note that if the metadata is not provided within 30 seconds, Serviio times out.

[zip]

What is the problem still? I already said that Seviio has a 30 seconds limit to get the metadata and if it takes longer it skips the file.

What is the problem still? I already said that Seviio has a 30 seconds limit to get the metadata and if it takes longer it skips the file.

Yep, let's stop arguing.
For me your answer do it
Problem closed for me.

What is the problem still? I already said that Seviio has a 30 seconds limit to get the metadata and if it takes longer it skips the file.

Hey Petr, is there a way (as far as a configuration file or setting) that we could get Serviio to wait longer than 30 seconds?

By the way (seeing that you are from London), are you a Chelsea fan? If so, congrat on the Europa Cup. Looks like you have a good coach coming in (although Benito did you all some good, presuming you are a Blue fan).

[zip]

No, there is not, but I'll make it configurable via system property for 1.3.

Yes, I'm a Blue, hoping Jose's coming home