iPad and AirPlayer app

[Elvis71]

Hi, Im using iPad with the Airplayer and Serviio as the mediaserver. It works well, also with a lot of other (HD) filetypes than MP4. Except srt subtitles, I can't get those to work.

Any experiences/ideas anyone, or tips for another iPad app that may to the trick?

[patters]

AirPlayer cannot use hardware h264 decoding for anything other than mp4 apparently so it can be very taxing on the battery/CPU.

As someone who'll be getting an iPad2 I've just been looking into this, and it seems that by jailbreaking you can use XBMC which has hacky access to this framework.

Looks like even the iPad2's GPU (likely to be a dual core Imagination Technologies SGX543) will still be limited to 720p though which I just can't understand given that my Sony Vaio P with Intel GMA 500 GPU (which is a much older Imagination Technologies SGX535) can decode 1080p in hardware. I wouldn't really want to have to specifically download 720p content.

[patters]

I've got my iPad 2 and it's great. No queuing for 33 hours outside an Apple Store for me. Just a trip to the reseller we use at work

There's a free DLNA client called Media Link Player Lite which plays all the mp4 content in my Serviio library, even the 1080p stuff apparently - which is also High Profile Level 4.1. Purportedly the iPads are only meant to do Main Profile Level 3.1 with no more than 720p. Nothing else plays though, so it's a real shame that there's no way to do live remuxing to mp4.

I did manually convert a 3gp file from a Blackberry phone by copying the video codec (MPEG-4 part 2) into an mp4 container and transcoding the sound to mp3. The video plays but it's silent. So there may be a decent way to inexpensively transcode all Xvid/DivX avi files without touching the video.

I'm determined to improve this one way or another though - this device is tantalisingly close to being the ultimate renderer. Why be concerned about how many gigs of flash memory you have when you can stream from your NAS?

[patters]

Doing some reading it does look like the native QuickTime player in iPhones and iPads can deal with h264 in mpegts but only if it's served up in the Apple HTTP streaming way (cut into small chunks). Hopefully this may be achievable on the fly though it may ultimately entail a complex mod to Serviio.

[patters]

I have been experimenting with Wowza Media Server (a commercial web streaming server) which I use at work to see if I could get it to serve On Demand m2ts streams. Unfortunately, though I could get it to stream mp4 content to the iPad easily, it can only stream live m2ts streams directly from an encoder at this time. On Demand mpegts is not supported.

Apple have an overview of their HTTP Streaming here, and it spells out that iOS clients definitely support h264 with aac in m2ts.

Serviio should be able to use FFmpeg to remux h264 streams in mkv or m2ts, transcoding audio to aac where necessary (we'll need libfaac support in FFmpeg) and then segment and serve the file as requested. Wowza for instance doesn't actually chop the file up - it merely presents a virtual folder structure to the client via its web service which presumably point to byte ranges of the requested media file, so I think it should be possible.

It would then just be a case of making a web app for the iPhone/iPad that would run on the Serviio server showing the DLNA folder structure and thumbnails etc. Not strictly DLNA, but it would be far better than the current alternatives.

More info and an open source HTTP segmenter here:
http://code.google.com/p/httpsegmenter/
http://www.ioncannon.net/programming/45 ... nt-page-3/
http://www.tandasoft.com/2011/02/01/iph ... n-windows/
http://www.espend.de/artikel/iphone-ipa ... ndows.html

[patters]

Great - it works! I installed IIS on my Windows 7 machine and remuxed an h264+aac mp4 sample file into mpegts. I then used the open source segmenter and dumped all the chunks plus the .m3u8 in the web root. And it plays!

So it should be possible to play mkv and m2ts files on iPad/iPhone with simple remuxing and audio-only transcoding where it's not aac. I'll experiment some more with 1080p content, and with mp3 audio (which I think I've read is only supported with audio-only streams), and to see if it supports MPEG-4 ASP. If that works then even DivX/Xvid won't need transcoding.

[zip]

Good job patters. So the IPad app would get the m3u file as another DLNA resource? Or is the Http streaming completely different to DLNA?

In the Apple doc it says:

(if the file is already encapsulated in an MPEG-2 transport stream, the file segmenter omits this step)

which sounds to me the file doesn't have to be segmented if it already is M2TS.. ?

[patters]

The way I envisage it, the iPad would browse Serviio's DLNA server as normal. Serviio would detect the iPad renderer, realising thanks to the profiles.xml that it only supports h264 with aac, and to remux everything to m2ts. It would advertise the resource absolutely as per the DLNA spec, but the actual hyperlink to the media file would need to be an m3u8 file which, assuming that's allowed, ought to cause the iOS device to spawn the native QuickTime player. I guess that breaks the cleanliness of Serviio, because it has to fork its behaviour depending on the renderer. Still, it's probably a price worth playing.

The only issue really is this segmenting problem. I can only assume that Wowza Media Server is able to create a virtual m3u8 file on demand, and has a way of pointing it at byte ranges of the media file. This would be the tough part, though that open source segmenter should give you the info you'd need to implement it.

I hadn't noticed that bit you quoted in that Apple doc. I'll give it a try now without segmenting (just putting one single entry in the m3u8 file)...

...and it plays, but it seems you can't seek. So I guess Serviio would need its own Apple HTTP streamer service.

[zip]

can you try splitting a m2ts file into 'segments' with something like this: http://www.techiecorner.com/107/how-to- ... les-linux/

If that works your byte-range theory would work, if not, each segment would probably need a m2ts header and that would be harder to implement.

[patters]

It is more complex than that. It splits on key frames only, and this means that if the media file's keyframe interval isn't aligned with that 10 second target chunk size you end up (as I did for my test) with chunks of differing sizes:

So I think you'd need to sort of abstract all that within the streaming server. Take a look at Wowza if you have time. The eval license is free. Content that's in an on demand folder isn't chunked, but the server obviously parses the media file at the time it's requested and figures it all out. Clearly a significant endeavour, but to my knowledge no other server is able to stream remuxed h264 to iOS devices (and there are lots of them).

  Code:

#EXTM3U
#EXT-X-TARGETDURATION:10
#EXTINF:12,
http://192.168.1.69/moon1-1.ts
#EXTINF:10,
http://192.168.1.69/moon1-2.ts
#EXTINF:11,
http://192.168.1.69/moon1-3.ts
#EXTINF:10,
http://192.168.1.69/moon1-4.ts
#EXTINF:12,
http://192.168.1.69/moon1-5.ts
#EXTINF:11,
http://192.168.1.69/moon1-6.ts
#EXTINF:12,
http://192.168.1.69/moon1-7.ts
#EXTINF:10,
http://192.168.1.69/moon1-8.ts
#EXTINF:10,
http://192.168.1.69/moon1-9.ts
#EXTINF:10,
http://192.168.1.69/moon1-10.ts
#EXTINF:13,
http://192.168.1.69/moon1-11.ts
#EXTINF:7,
http://192.168.1.69/moon1-12.ts


In other testing, it seems that in Apple's eyes at least, MPEG-4 ASP is not a legal video codec for the mpegts container. It should be - the resulting file does play in VLC.

It is possible to convert DivX/Xvid AVI files to mp4 pretty quickly without touching the video, though not on the fly because you can't live stream mp4. I had hoped the iPad would play mp3 audio in mp4 video files, meaning a simple remux would be all that was needed - but sadly it really does need to be aac audio

Once they're remuxed to mp4 and audio transcoded with the following command line, they're playable from within the Media Link Player Lite DLNA client:

C:\Temp>ffmpeg -i "g:\video\Trailers and samples\testclip.avi" -vcodec copy -acodec libfaac -aq 85 -ac 2 "g:\video\Trailers and samples\testclip.mp4"

You can get the Win32 binary of the latest git version of FFmpeg compiled with libfaac support, and libfaac.dll from:
http://oss.netfarm.it/mplayer-win32.php

[zip]

The chunks are with the same duration, how can that be split exactly on keyframes (they happen randomly)?

Anyway, it looks like a lot of work, maybe something for the future Bug keep digging.

[patters]

I think it means that you might specify a desired chunk size of 10 seconds (as Apple recommends) but your keyframe interval might be say 3 seconds. So the first chunk would be at 9 seconds, the second at 21 seconds (a 12 second chunk) and so on.

[patters]

More testing - the iPad2 can definitely play h264 with High Profile Level 4.1 at 1920x1080 but only at lower bitrates - say around 10Mbps. I have a high bitrate copy of Big Buck Bunny from the x264 encoder showcase AVCHD DVD (which reaches around 30Mbps) and that judders a lot, losing A/V sync. I guess this could be caused by wifi though. I don't have an 802.11n router to try it on - only 802.11g (the iPad supports n).

I also compiled libfaac and recompiled FFmpeg with support for it on my Synology NAS. Unfortunately the performance on ARM is too slow for realtime audio transcoding to aac. It can only manage 13fps. Perhaps there will be an ARM-optimized aac encoder some day.

[patters]

The chunks are with the same duration, how can that be split exactly on keyframes (they happen randomly)?

Anyway, it looks like a lot of work, maybe something for the future Bug keep digging.

Microsoft have implemented the exact functionality I had in mind, with IIS Media Services:
http://learn.iis.net/page.aspx/854/appl ... -services/

Particularly:

The MPEG-2 TS files on IIS are not stored in multiple segments on disk like other solutions for Apple HTTP Live Streaming. IIS Media Services dynamically serves the segments on request and leaves the full MPEG-2 TS files available to make asset management and post-production easier.

[patters]

Been doing more research and it seems that even the most popular media server for iPad - Air Video - is brute force transcoding everything non-compliant to h264, then presumably streaming using Apple HTTP pseudo streaming. We can certainly do better than this by remuxing streams that are already h264, and audio transcoding.

And now for some good news - it looks like FFmpeg may soon be integrating support for authoring Apple HTTP segmented streams with m3u8 playlists:
http://article.gmane.org/gmane.comp.vid ... vel/128320
http://permalink.gmane.org/gmane.comp.v ... vel/128988
http://permalink.gmane.org/gmane.comp.v ... vel/128998

If you look at the output of ffmpeg -formats on a recent build you'll see applehttp is listed but I think it might only be an input format for the time being. I've tried giving that as an output argument with -f and it didn't like it. This pretty much confirms it:
http://article.gmane.org/gmane.comp.vid ... =applehttp

If they integrate output format support, then I reckon it should be very easy to get Serviio serving the chunks while they're being transcoded from its usual transcoding temp folder - since the whole point of Apple HTTP is that it uses a regular web server, and it's the client that checks the playlist at defined intervals for new chunks.

What does need testing is whether we can feed an iPad an m3u8 playlist file via DLNA - will existing DLNA software know what to do with it if we give it a resource type of something like AVC_TS_MP_HD_AAC_MULT5, or do we need to suggest a mod to the authors of current clients? Or do we need a Serviio app?

I also discovered another AAC encoder - the reference HE-AACv2 encoder from 3GPP which is supposed to be pretty good (unlike most reference encoders). This guy has packaged it up as a library:
http://tipok.org.ua/ru/node/17

FFmpeg can be patched to support it as an external encoder so there's a chance it may be faster than libfaac which everyone always seems keen to rubbish. Though iPads support HE-AACv1 and v2, iPhones (pre-iPhone 4) don't support anything above LC-AAC but I don't think that's too important given you'd have to re-encode any h264 content for their limited resolution support anyway (so you probably wouldn't bother), and it won't mess up audio transcoding too much because we could fall back to mp3 for audio-only files.

[zip]

That sounds interesting. When it becomes possible to implement this (ie ffmpeg support is in place), we can create a ticket with all the info you found and try implementing this.

Hello,

I know it's been a long time - has this become possible to implement yet? I'm just now trying to use my iPod with Serviio.

Thanks.

That sounds interesting. When it becomes possible to implement this (ie ffmpeg support is in place), we can create a ticket with all the info you found and try implementing this.

Is there any progress on this? I just got my iPad and it appears that I need to transcode all my video library as it appears AC3 is not supported. As audio only transcoding is fast enough to happen live, I would highly appreciate a suitable transcoding profile on serviio.

ffmpeg support already segmenting, not sure whether this could be done on the fly.
Exmaple:
ffmpeg -i "Ice Age.mp4" -c copy -map 0 -vbsf h264_mp4toannexb -f segment -segment_time 10 -segment_list "Ice Age.m3u8" -segment_format mpegts "Ice Age%d.ts"

In addition, Air Video Server is hosting their modified version of the ffmpeg, so their solution could be used as well. As I can see they use alternative options, different from official ffmpeg:
Example:
/home/as/ffmpeg/ffmpeg --segment-length 4 --segment-offset 0 --port-number 46631

[zip]

I noticed ffmpeg supports segmenting so it's something I will look at in the future.