FAQ  •  Register  •  Login

[Patch] HD stream selection in HTTP Live Streaming (HLS)

<<

stefana888

User avatar

DLNA master

Posts: 134

Joined: Mon Aug 18, 2014 6:24 am

Post Thu May 21, 2015 2:32 am

[Patch] HD stream selection in HTTP Live Streaming (HLS)

I am utilizing onlive live content in HLS where the manifest file contains multiple streams with different bandwiths (programs). (The url to a single stream can not be extracted by hand as far as I know from the current type of manifest, see example.) Serviio then defaults to the first video and audio streams it finds which isn't very flexible. In fact, it will probably be a "worst" approach always returning the streams with the lowest bandwith. A "best" approach would be returning the streams with the hightest assumed bandwith i.e. in HD-format. The following modification manages that;

In org.serviio.library.metadata.FFmpegMetadataRetriever

  Code:
private static Map<String, Object> getParametersMap(List<String> ffmpegMediaDescription)
          throws InvalidMediaFormatException
          {

 . . .

             List<EmbeddedSubtitles> subtitlesList = new ArrayList();

             String[] array = ffmpegMediaDescription.toArray(new String[ffmpegMediaDescription.size()]);
              int i = 0;
              while (i < array.length)
               {
                   String line = array[i].trim();
                   int inputPos = line.indexOf("Input #0");

 . . .

                           parameters.put("bitrate", bitrate);
                           }
                         }
                       }
                     }
                      else if (line.startsWith("Program"))
                      {
                        int j = i + 1;
                        while (j < array.length)
                        {
                           if (array[j].trim().startsWith("Program"))
                           {
                             i = j - 1;
                             break;
                           }
                           j++;
                        }
                      }
                      else if ((line.startsWith("Stream")) && (line.indexOf("Video:") > -1) && (parameters.get("video_codec") == null))
       
 . . .

               subtitlesList.add(new EmbeddedSubtitles((Integer)streamInfo.getValueA(), sCodec, (String)streamInfo.getValueB(), defaultLanguage));
               }
             }
           }
           i++;
         }
         parameters.put("embedded_subtitles", subtitlesList);
         return parameters;

. . .


(Example manifest)

  Code:
#EXTM3U
#EXT-X-VERSION:3
#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="English",FORCED=NO,AUTOSELECT=YES,URI="subtitlelist_leng_b110000.m3u8",LANGUAGE="eng"
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=110000,RESOLUTION=320x180,SUBTITLES="subs"
chunklist_b110000.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=64000,CODECS="mp4a.40.2",SUBTITLES="subs"
chunklist_b64000.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=268000,RESOLUTION=320x180,SUBTITLES="subs"
chunklist_b268000.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=668000,RESOLUTION=640x360,SUBTITLES="subs"
chunklist_b668000.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1000000,RESOLUTION=880x480,SUBTITLES="subs"
chunklist_b1000000.m3u8
#EXT-X-STREAM-INF:PROGRAM-ID=1,BANDWIDTH=1800000,RESOLUTION=1024x576,SUBTITLES="subs"
chunklist_b1800000.m3u8
<<

zip

User avatar

Serviio developer / Site Admin

Posts: 17212

Joined: Sat Oct 24, 2009 12:24 pm

Location: London, UK

Post Mon May 25, 2015 6:24 pm

Re: [Patch] HD stream selection in HTTP Live Streaming (HLS)

I think there is no given order of the items in the m3u file, so picking first / last will not make it better.

In general, Serviio will picj the first video stream and audio stream that FFmpeg suggests. Supporting playback / transcode of any other video stream that the first would need further updates in the app, i think.
<<

stefana888

User avatar

DLNA master

Posts: 134

Joined: Mon Aug 18, 2014 6:24 am

Post Mon May 25, 2015 7:12 pm

Re: [Patch] HD stream selection in HTTP Live Streaming (HLS)

Take a look at this ffmpeg output. As one can see higher bitrates here have higher 'Program #'. The suggested code takes the streams with the highest 'Program #'. The ffmpeg output corresponds to the m3u8 manifest stream order. Actually they seem standardized in order from low to high stream bitrates.

Who can live with Serviio producing a 320 x 180 video format when there is a 1024 x 576 video format available? The idea works for all my media, not only HLS online content.

  Code:
Input #0, hls,applehttp, from 'http://xxxxxxxxx.net/xxxxxxxxxx/c51eb4af0c90422baad58832447f31de/manifest.
m3u8':
  Duration: N/A, start: 13730.119344, bitrate: N/A
  Program 0
    Metadata:
      variant_bitrate : 110000
    Stream #0:0: Data: timed_id3 (ID3  / 0x20334449)
    Metadata:
      variant_bitrate : 110000
    Stream #0:1: Video: h264 (Baseline) ([27][0][0][0] / 0x001B), yuv420p, 320x180 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 9
0k tbn, 50 tbc
    Metadata:
      variant_bitrate : 110000
    Stream #0:2: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 67 kb/s
    Metadata:
      variant_bitrate : 110000
  Program 1
    Metadata:
      variant_bitrate : 64000
    Stream #0:3: Audio: aac (LC), 44100 Hz, stereo, fltp, 65 kb/s
    Metadata:
      variant_bitrate : 64000
  Program 2
    Metadata:
      variant_bitrate : 268000
    Stream #0:4: Data: timed_id3 (ID3  / 0x20334449)
    Metadata:
      variant_bitrate : 268000
    Stream #0:5: Video: h264 (Baseline) ([27][0][0][0] / 0x001B), yuv420p, 320x180 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 9
0k tbn, 50 tbc
    Metadata:
      variant_bitrate : 268000
    Stream #0:6: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 67 kb/s
    Metadata:
      variant_bitrate : 268000
  Program 3
    Metadata:
      variant_bitrate : 668000
    Stream #0:7: Data: timed_id3 (ID3  / 0x20334449)
    Metadata:
      variant_bitrate : 668000
    Stream #0:8: Video: h264 (Baseline) ([27][0][0][0] / 0x001B), yuv420p, 640x360 [SAR 1:1 DAR 16:9], 25 tbr, 90k tbn,
50 tbc
    Metadata:
      variant_bitrate : 668000
    Stream #0:9: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 59 kb/s
    Metadata:
      variant_bitrate : 668000
  Program 4
    Metadata:
      variant_bitrate : 1000000
    Stream #0:10: Data: timed_id3 (ID3  / 0x20334449)
    Metadata:
      variant_bitrate : 1000000
    Stream #0:11: Video: h264 (Baseline) ([27][0][0][0] / 0x001B), yuv420p, 880x480 [SAR 1:1 DAR 11:6], 25 tbr, 90k tbn,
 50 tbc
    Metadata:
      variant_bitrate : 1000000
    Stream #0:12: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 117 kb/s
    Metadata:
      variant_bitrate : 1000000
  Program 5
    Metadata:
      variant_bitrate : 1800000
    Stream #0:13: Data: timed_id3 (ID3  / 0x20334449)
    Metadata:
      variant_bitrate : 1800000
    Stream #0:14: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p, 1024x576 [SAR 1:1 DAR 16:9], 25 tbr, 90k tbn, 50
 tbc
    Metadata:
      variant_bitrate : 1800000
    Stream #0:15: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 117 kb/s
    Metadata:
      variant_bitrate : 1800000
At least one output file must be specified
Last edited by stefana888 on Sun May 31, 2015 7:29 pm, edited 1 time in total.
<<

jhb50

DLNA master

Posts: 2843

Joined: Thu Jun 30, 2011 9:32 pm

Post Tue May 26, 2015 12:17 pm

Re: [Patch] HD stream selection in HTTP Live Streaming (HLS)

I have always found the resolution in increasing order as well.
<<

zip

User avatar

Serviio developer / Site Admin

Posts: 17212

Joined: Sat Oct 24, 2009 12:24 pm

Location: London, UK

Post Thu May 28, 2015 1:10 pm

Re: [Patch] HD stream selection in HTTP Live Streaming (HLS)

<<

stefana888

User avatar

DLNA master

Posts: 134

Joined: Mon Aug 18, 2014 6:24 am

Post Fri Jul 03, 2015 6:33 pm

Re: [Patch] HD stream selection in HTTP Live Streaming (HLS)

@jhb50's bitbucket comment

https://developer.apple.com/library/ios ... MENDATIONS

"First bit rate should be one that most clients can sustain", for testing purposes.
Thus it is not likely that the first stream have the highest bitrate. An exception as in jhb50's bitbucket comment shouldn't decide the rule...

It is no problem for me, I can patch Serviio class files myself. But what about a potential customer with demands in live streaming...
<<

jhb50

DLNA master

Posts: 2843

Joined: Thu Jun 30, 2011 9:32 pm

Post Sat Jul 04, 2015 12:01 am

Re: [Patch] HD stream selection in HTTP Live Streaming (HLS)

Why not alter the patch to select the stream based on the "Preferred Online Quality" set in the console rather than the highest.
Until these options are expanded to separate 1080 and 720, high should select 720 since 1080 overwhelms many systems that are capable of 720.
Perhaps you can post a patch to allow XHigh,High,Medium and Low, or even better add the values, 1080,720,480,360,240 to the existing ones which would provide compatibility.
<<

stefana888

User avatar

DLNA master

Posts: 134

Joined: Mon Aug 18, 2014 6:24 am

Post Sat Jul 04, 2015 4:06 pm

Re: [Patch] HD stream selection in HTTP Live Streaming (HLS)

jhb50 wrote:Why not alter the patch to select the stream based on the "Preferred Online Quality" set in the console rather than the highest.
Until these options are expanded to separate 1080 and 720, high should select 720 since 1080 overwhelms many systems that are capable of 720.
Perhaps you can post a patch to allow XHigh,High,Medium and Low, or even better add the values, 1080,720,480,360,240 to the existing ones which would provide compatibility.
It would no longer be a simple patch to introduce a complete new feature that needs an alternate FFmpegMetadataRetriever. But I agree something is definitely missing here when multiple resolutions are available. Only picking the first stream does not satisfy any reasonable needs in http live streaming.
<<

stefana888

User avatar

DLNA master

Posts: 134

Joined: Mon Aug 18, 2014 6:24 am

Post Sun Jul 05, 2015 4:27 am

Re: [Patch] HD stream selection in HTTP Live Streaming (HLS)

(Changed my last post.)
<<

zip

User avatar

Serviio developer / Site Admin

Posts: 17212

Joined: Sat Oct 24, 2009 12:24 pm

Location: London, UK

Post Sat Jul 11, 2015 10:18 am

Re: [Patch] HD stream selection in HTTP Live Streaming (HLS)

The preferred quality is there for the outgoing quality / bitrate (to enable streaming for users with lower upstream bandwidth).

It might have to be a different config setting altogether, or use a -D system property for now.
<<

jhb50

DLNA master

Posts: 2843

Joined: Thu Jun 30, 2011 9:32 pm

Post Sat Jul 11, 2015 12:55 pm

Re: [Patch] HD stream selection in HTTP Live Streaming (HLS)

Am I missing something? I assumed you wanted to pick the hls stream based on the users preferred outgoing quality... so is not the console setting the value to use just like we pick the youtube stream quality? I don't understand why a new setting would be required or why this would be a complete new feature that needs an alternate FFmpegMetadataRetriever?
<<

stefana888

User avatar

DLNA master

Posts: 134

Joined: Mon Aug 18, 2014 6:24 am

Post Sun Jul 12, 2015 8:26 pm

Re: [Patch] HD stream selection in HTTP Live Streaming (HLS)

A -D system property would be the easiest way, toggling if the patch functionality (test for 'Program') is applied. Else I don't see how completely rewrite FFMpegMetadataRetriever to tie it to the console is avoided.

I will deliver the source code to Zip for consideration.
<<

jhb50

DLNA master

Posts: 2843

Joined: Thu Jun 30, 2011 9:32 pm

Post Sun Jul 12, 2015 11:11 pm

Re: [Patch] HD stream selection in HTTP Live Streaming (HLS)

Here is a stream where all Programs are 640x360 but different bitrates. I don't understand why there are 2 Programs for each bitrate with the second not showing fps and a different audio bit rate. I guess these would need to be handled too but I don't know which Program to suggest be used.
NHK provides high,mid and low streams and uses the bitrates for adaptive streaming.
  Code:
Input #0, hls,applehttp, from 'http://www3.nhk.or.jp/nhkworld/app/tv/stream/global/live_high.m3u8':
  Duration: N/A, start: 78198.375189, bitrate: 423 kb/s
  Program 0
    Metadata:
      variant_bitrate : 1368400
    Stream #0:0: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p, 640x360 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 90k tbn, 60 tbc
    Metadata:
      variant_bitrate : 1368400
    Stream #0:1: Audio: aac ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 74 kb/s
    Metadata:
      variant_bitrate : 1368400
    Stream #0:2: Unknown: none ([5][0][0][0] / 0x0005)
    Metadata:
      variant_bitrate : 1368400
 
 Program 1
    Metadata:
      variant_bitrate : 1368400
    Stream #0:3: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p, 640x360 [SAR 1:1 DAR 16:9], 30 tbr, 90k tbn, 60 tbc
    Metadata:
      variant_bitrate : 1368400
    Stream #0:4: Audio: aac ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 66 kb/s
    Metadata:
      variant_bitrate : 1368400
    Stream #0:5: Unknown: none ([5][0][0][0] / 0x0005)
    Metadata:
      variant_bitrate : 1368400
 
Program 2
    Metadata:
      variant_bitrate : 990000
    Stream #0:6: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p, 640x360 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 90k tbn, 60 tbc
    Metadata:
      variant_bitrate : 990000
    Stream #0:7: Audio: aac ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 74 kb/s
    Metadata:
      variant_bitrate : 990000
    Stream #0:8: Unknown: none ([5][0][0][0] / 0x0005)
    Metadata:
      variant_bitrate : 990000
 
Program 3
    Metadata:
      variant_bitrate : 990000
    Stream #0:9: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p, 640x360 [SAR 1:1 DAR 16:9], 30 tbr, 90k tbn, 60 tbc
    Metadata:
      variant_bitrate : 990000
    Stream #0:10: Audio: aac ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 66 kb/s
    Metadata:
      variant_bitrate : 990000
    Stream #0:11: Unknown: none ([5][0][0][0] / 0x0005)
    Metadata:
      variant_bitrate : 990000
 
 Program 4
    Metadata:
      variant_bitrate : 660000
    Stream #0:12: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p, 640x360 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 90k tbn, 60 tbc
    Metadata:
      variant_bitrate : 660000
    Stream #0:13: Audio: aac ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 74 kb/s
    Metadata:
      variant_bitrate : 660000
    Stream #0:14: Unknown: none ([5][0][0][0] / 0x0005)
    Metadata:
      variant_bitrate : 660000
 
 Program 5
    Metadata:
      variant_bitrate : 660000
    Stream #0:15: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p, 640x360 [SAR 1:1 DAR 16:9], 30 tbr, 90k tbn, 60 tbc
    Metadata:
      variant_bitrate : 660000
    Stream #0:16: Audio: aac ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 66 kb/s
    Metadata:
      variant_bitrate : 660000
    Stream #0:17: Unknown: none ([5][0][0][0] / 0x0005)
    Metadata:
      variant_bitrate : 660000
<<

stefana888

User avatar

DLNA master

Posts: 134

Joined: Mon Aug 18, 2014 6:24 am

Post Mon Jul 13, 2015 5:45 am

Re: [Patch] HD stream selection in HTTP Live Streaming (HLS)

jhb50 wrote:Here is a stream where all Programs are 640x360 but different bitrates. I don't understand why there are 2 Programs for each bitrate with the second not showing fps and a different audio bit rate. I guess these would need to be handled too but I don't know which Program to suggest be used.
NHK provides high,mid and low streams and uses the bitrates for adaptive streaming.
  Code:
Input #0, hls,applehttp, from 'http://www3.nhk.or.jp/nhkworld/app/tv/stream/global/live_high.m3u8':
  Duration: N/A, start: 78198.375189, bitrate: 423 kb/s
  Program 0
    Metadata:
      variant_bitrate : 1368400
    Stream #0:0: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p, 640x360 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 90k tbn, 60 tbc
    Metadata:
      variant_bitrate : 1368400
    Stream #0:1: Audio: aac ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 74 kb/s
    Metadata:
      variant_bitrate : 1368400
    Stream #0:2: Unknown: none ([5][0][0][0] / 0x0005)
    Metadata:
      variant_bitrate : 1368400
 
 Program 1
    Metadata:
      variant_bitrate : 1368400
    Stream #0:3: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p, 640x360 [SAR 1:1 DAR 16:9], 30 tbr, 90k tbn, 60 tbc
    Metadata:
      variant_bitrate : 1368400
    Stream #0:4: Audio: aac ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 66 kb/s
    Metadata:
      variant_bitrate : 1368400
    Stream #0:5: Unknown: none ([5][0][0][0] / 0x0005)
    Metadata:
      variant_bitrate : 1368400
 
Program 2
    Metadata:
      variant_bitrate : 990000
    Stream #0:6: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p, 640x360 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 90k tbn, 60 tbc
    Metadata:
      variant_bitrate : 990000
    Stream #0:7: Audio: aac ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 74 kb/s
    Metadata:
      variant_bitrate : 990000
    Stream #0:8: Unknown: none ([5][0][0][0] / 0x0005)
    Metadata:
      variant_bitrate : 990000
 
Program 3
    Metadata:
      variant_bitrate : 990000
    Stream #0:9: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p, 640x360 [SAR 1:1 DAR 16:9], 30 tbr, 90k tbn, 60 tbc
    Metadata:
      variant_bitrate : 990000
    Stream #0:10: Audio: aac ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 66 kb/s
    Metadata:
      variant_bitrate : 990000
    Stream #0:11: Unknown: none ([5][0][0][0] / 0x0005)
    Metadata:
      variant_bitrate : 990000
 
 Program 4
    Metadata:
      variant_bitrate : 660000
    Stream #0:12: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p, 640x360 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 90k tbn, 60 tbc
    Metadata:
      variant_bitrate : 660000
    Stream #0:13: Audio: aac ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 74 kb/s
    Metadata:
      variant_bitrate : 660000
    Stream #0:14: Unknown: none ([5][0][0][0] / 0x0005)
    Metadata:
      variant_bitrate : 660000
 
 Program 5
    Metadata:
      variant_bitrate : 660000
    Stream #0:15: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p, 640x360 [SAR 1:1 DAR 16:9], 30 tbr, 90k tbn, 60 tbc
    Metadata:
      variant_bitrate : 660000
    Stream #0:16: Audio: aac ([15][0][0][0] / 0x000F), 44100 Hz, stereo, fltp, 66 kb/s
    Metadata:
      variant_bitrate : 660000
    Stream #0:17: Unknown: none ([5][0][0][0] / 0x0005)
    Metadata:
      variant_bitrate : 660000


Sparate playlists for high, medium, low bitrates seem uncommon. They tend to bunch them all in a single ordered playlist. Often from low to high.

Return to Feature requests

Who is online

Users browsing this forum: No registered users and 29 guests

cron
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group.
Designed by ST Software for PTF.