Help - why are some MP4's playing, and some others not

Hi,

I've identified that my Humax Fox T2 Freeview can natively play mp4 (AVC-H4.1, AAC) videos from serviio - however they will only do so if I've any kind of editing to them. Otherwise, the error message "Cannot support this file format" error pops up on the TV.

More detail:
I have 2 test Serviio systems - both clean build Serviio 1.4 on a WinXP and Win8.1 system. The results are always consistent across both. All these test files WILL play natively (aka direct stream, without transcoding) to my Samsung C-Series TV.

I have over 2000 home video clips/films etc all encoded over the years using handbrake (via Vidcoder) using the default HighProfile
As they stand these only play on my Humax when Serviio is configured to use a profile such as Sony Bravia 2009 which does on the fly transcoding of MP4. This, however, causes a nearly 1 min delay before the start of the video, and prevents any other streams being delivered to other devices, as my media server is maxed out transcoding this stream.

I use a non-destructive editor (Machete) which can split, crop, and rejoin mp4's without re-encoding
Once I've used Machete, these videos will stream from Serviio to my Humax without any transcoding, using the Generic profile. These start playing immediately, and only use a few % processor overhead on my serviio media server.
The Machete editing required could simply be to de-mux the video stream to a separate file, or to remove a single key frame from the beginning/end of the mp4 - neither of which mean re-encoding the original mp4 file.

Below, I've listed MEDIAINFO details on 2 test video clips - I've tried it only many others, and the result is the same.

Thoughts and guidance are very much appreciated, however, PLEASE don't reply with answers which suggest "re-encode your entire library using another encoder" or "buy a more powerful media server" - this is not identifying and solving the issue. I GUARENTEE others are actually having to use transcode profiles to overcome this issue without realising they could use direct streaming to their device, simply because of some values in the header/config info in the mp4.

Here is a MEDIAINFO report of a test file (an episode of Endevour) which will not play natively.
I also attached the debug log taken during 2 tests of these files

serviio-debug.zip

Debug log of video tests

(25.65 KiB) Downloaded 445 times

General
Complete name : D:\Media\TV\Endeavour\Season 1\Endeavour.S01E03.Rocket - orig.mp4
Format : MPEG-4
Format profile : Base Media / Version 2
Codec ID : mp42
File size : 793 MiB
Duration : 1h 28mn
Overall bit rate mode : Variable
Overall bit rate : 1 250 Kbps
Encoded date : UTC 2013-09-13 00:34:59
Tagged date : UTC 2013-09-13 03:23:04
Writing application : HandBrake 0.9.9 2013051800

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4.1
Format settings, CABAC : Yes
Format settings, ReFrames : 6 frames
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 1h 28mn
Bit rate : 1 086 Kbps
Width : 960 pixels
Height : 718 pixels
Display aspect ratio : 16:9
Frame rate mode : Variable
Frame rate : 25.000 fps
Minimum frame rate : 16.667 fps
Maximum frame rate : 25.000 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.063
Stream size : 689 MiB (87%)
Writing library : x264 core 130 r2273 b3065e6
Encoding settings : cabac=1 / ref=6 / deblock=1:-1:-1 / analyse=0x3:0x133 / me=umh / subme=11 / psy=1 / psy_rd=1.00:0.15 / mixed_ref=1 / me_range=24 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=0 / chroma_qp_offset=-3 / threads=9 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=6 / b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=60 / rc=crf / mbtree=1 / crf=20.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / vbv_maxrate=62500 / vbv_bufsize=78125 / crf_max=0.0 / nal_hrd=none / ip_ratio=1.40 / aq=1:1.00
Encoded date : UTC 2013-09-13 00:34:59
Tagged date : UTC 2013-09-13 03:23:04
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709

Audio
ID : 2
Format : AAC
Format/Info : Advanced Audio Codec
Format profile : LC
Codec ID : 40
Duration : 1h 28mn
Bit rate mode : Variable
Bit rate : 160 Kbps
Maximum bit rate : 276 Kbps
Channel(s) : 2 channels
Channel positions : Front: L R
Sampling rate : 48.0 KHz
Compression mode : Lossy
Stream size : 101 MiB (13%)
Language : English
Encoded date : UTC 2013-09-13 00:34:59
Tagged date : UTC 2013-09-13 03:23:03


==================
Here's the same file stripped of the audio (if I re-muxed the audio back in it still works, so the audio is not relevant). This file will stream natively - naturally you get an audio error (!).
==================


General
Complete name : D:\Media\TV\Endeavour\Season 1\Demux Endeavour.S01E03.Rocket - origv noa.mp4
Format : MPEG-4
Format profile : Base Media / Version 2
Codec ID : mp42
File size : 690 MiB
Duration : 1h 28mn
Overall bit rate : 1 088 Kbps
Writing application : Machete 4.1

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4.1
Format settings, CABAC : Yes
Format settings, ReFrames : 6 frames
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 1h 28mn
Source duration : 1h 28mn
Bit rate : 1 086 Kbps
Width : 960 pixels
Height : 718 pixels
Display aspect ratio : 4:3
Original display aspect ratio : 16:9
Frame rate mode : Variable
Frame rate : 25.000 fps
Minimum frame rate : 16.667 fps
Maximum frame rate : 25.000 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.063
Stream size : 689 MiB (100%)
Source stream size : 689 MiB (100%)
Writing library : x264 core 130 r2273 b3065e6
Encoding settings : cabac=1 / ref=6 / deblock=1:-1:-1 / analyse=0x3:0x133 / me=umh / subme=11 / psy=1 / psy_rd=1.00:0.15 / mixed_ref=1 / me_range=24 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=0 / chroma_qp_offset=-3 / threads=9 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=6 / b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=60 / rc=crf / mbtree=1 / crf=20.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / vbv_maxrate=62500 / vbv_bufsize=78125 / crf_max=0.0 / nal_hrd=none / ip_ratio=1.40 / aq=1:1.00
Language : English
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709
mdhd_Duration : 5322920


Many thanks for help in resolving this, enabling me to play my files natively.
Cheers
Ti

[DenyAll]

If you stream via another Media Server (eg. enable streaming from Windows Media Player) does it exhibit the same issue - ie. the original file does not play but the remuxed file does? This would isolate the issue to Serviio.

Does the file play on your PC without remuxing?

What happens if you remux the file using the ffmpeg command:

"C:\Program Files\Serviio\lib\ffmpeg" -i "D:\Media\TV\Endeavour\Season 1\Endeavour.S01E03.Rocket - orig.mp4" -map 0 -c copy "D:\Media\TV\Endeavour\Season 1\Endeavour.S01E03.Rocket - remux.mp4"

Please note any warnings and errors during the remux process.

My suspicion is that there is something in the MP4 header that is not liked by your T2 (maybe aspect ratio, which changes...). While this "something" is likely ignored by software players and other renderer devices such as your Samsung, your Humax requires it (this is reasonably common: some encoding software, for whatever reason, does not write a particular field to the container header. Some players on seeing the missing or blank field, derive it from the audio/video stream and continue playing... other players spit the dummy and refuse to play the file. I had the same issue with some old AVI's that had the non-zero-bit audio issue - played fine on the PS3 and on software players, but my Panasonic rejected it).

If it is as I suspect, remuxing the file, recreating the container while keeping original audio and video, will fix the issue. As to what is actually corrected by the remux process - if the ffmpeg command doesn't tell you, then you will need something that looks deeply into the container headers of both files and then do a field by field comparison... if you are really keen

Gaaaargh!

DenyAll, your 'DLNA master' status is justified!

Yep, exactly as you predicted - using your ffmpeg command, the remux played perfectly. There were no errors in the remux (see below, along with mediainfo)
Also playing either version DIRECTLY via USB stick in the Humax worked perfectly

The Remux 'YES', - original 'NO' - network playback was consistent across Serviio, Plex, and my ReadyNAS DLNA streaming to the Humax

Both versions played perfectly via any delivery method on my Samsung TV, PC, phone, and anything else I could get my hands on.

So, the $60,000 question is can we intercept/alter/change any of these fields during playback, as I really don't want to remux 2,317 files, mainly because of the time, space, and availability (some I have archived off onto Bluray discs), also because the original timestamps are destroyed.

The other reason is of course to be able to identify the problem, and maybe alter/reconfig something in Handbrake/Vidcoder to avoid this problem in the future.

Any other thoughts or help appreciated.

I'm stuck and don't know how to proceed

Thanks
Ti

====
FFMPEG log
====

"C:\Program Files\Serviio\lib\ffmpeg" -i "D:\Media\TV\Endeavour\Season 1\Endeavour.S01E03.Rocket - orig.mp4" -map 0 -c copy "D:\Media\TV\Ende
avour\Season 1\!Endeavour.S01E03.Rocket - remux.mp4"
ffmpeg version N-54096-ge41bf19 Copyright (c) 2000-2013 the FFmpeg developers
built on Jun 18 2013 10:49:28 with gcc 4.7.2 (GCC)
configuration: --enable-static --disable-shared --disable-ffplay --disable-ffserver --enable-memalign-hack --enable-libmp3lame --enable-libass --enable-librtm
p --enable-fontconfig --enable-libfreetype --enable-zlib --enable-libx264 --extra-libs='-lrtmp -lpolarssl -lws2_32 -lwinmm -lexpat -lfreetype -lfribidi -lz' --a
rch=x86 --enable-runtime-cpudetect --enable-pthreads --target-os=mingw32 --cross-prefix=i686-w64-mingw32- --enable-gpl --pkg-config=pkg-config
libavutil 52. 37.101 / 52. 37.101
libavcodec 55. 16.100 / 55. 16.100
libavformat 55. 9.100 / 55. 9.100
libavdevice 55. 2.100 / 55. 2.100
libavfilter 3. 77.101 / 3. 77.101
libswscale 2. 3.100 / 2. 3.100
libswresample 0. 17.102 / 0. 17.102
libpostproc 52. 3.100 / 52. 3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'D:\Media\TV\Endeavour\Season 1\Endeavour.S01E03.Rocket - orig.mp4':
Metadata:
major_brand : mp42
minor_version : 0
compatible_brands: mp42isomavc1
creation_time : 2013-09-13 00:34:59
encoder : HandBrake 0.9.9 2013051800
Duration: 01:28:42.90, start: 0.000000, bitrate: 1250 kb/s
Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 960x718 [SAR 4:3 DAR 640:359], 1085 kb/s, 25 fps, 25 tbr, 90k tbn, 180k tbc
Metadata:
creation_time : 2013-09-13 00:34:59
Stream #0:1(eng): Audio: aac (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 159 kb/s
Metadata:
creation_time : 2013-09-13 00:34:59
Output #0, mp4, to 'D:\Media\TV\Endeavour\Season 1\!Endeavour.S01E03.Rocket - remux.mp4':
Metadata:
major_brand : mp42
minor_version : 0
compatible_brands: mp42isomavc1
encoder : Lavf55.9.100
Stream #0:0(und): Video: h264 ([33][0][0][0] / 0x0021), yuv420p, 960x718 [SAR 4:3 DAR 640:359], q=2-31, 1085 kb/s, 25 fps, 90k tbn, 90k tbc
Metadata:
creation_time : 2013-09-13 00:34:59
Stream #0:1(eng): Audio: aac ([64][0][0][0] / 0x0040), 48000 Hz, stereo, 159 kb/s
Metadata:
creation_time : 2013-09-13 00:34:59
Stream mapping:
Stream #0:0 -> #0:0 (copy)
Stream #0:1 -> #0:1 (copy)
Press [q] to stop, [?] for help
frame=133055 fps=26504 q=-1.0 Lsize= 813304kB time=01:28:42.90 bitrate=1251.7kbits/s
video:705413kB audio:103932kB subtitle:0 global headers:0kB muxing overhead 0.489042%

====
MediaInfo
====

General
Complete name : D:\Media\TV\Endeavour\Season 1\!Endeavour.S01E03.Rocket - remux.mp4
Format : MPEG-4
Format profile : Base Media
Codec ID : isom
File size : 794 MiB
Duration : 1h 28mn
Overall bit rate : 1 252 Kbps
Writing application : Lavf55.9.100

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4.1
Format settings, CABAC : Yes
Format settings, ReFrames : 6 frames
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 1h 28mn
Bit rate : 1 086 Kbps
Width : 960 pixels
Height : 718 pixels
Display aspect ratio : 16:9
Frame rate mode : Variable
Frame rate : 25.000 fps
Minimum frame rate : 16.667 fps
Maximum frame rate : 25.000 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.063
Stream size : 689 MiB (87%)
Writing library : x264 core 130 r2273 b3065e6
Encoding settings : cabac=1 / ref=6 / deblock=1:-1:-1 / analyse=0x3:0x133 / me=umh / subme=11 / psy=1 / psy_rd=1.00:0.15 / mixed_ref=1 / me_range=24 / chroma_me=1 / trellis=2 / 8x8dct=1 / cqm=0 / deadzone=21,11 / fast_pskip=0 / chroma_qp_offset=-3 / threads=9 / lookahead_threads=1 / sliced_threads=0 / nr=0 / decimate=1 / interlaced=0 / bluray_compat=0 / constrained_intra=0 / bframes=6 / b_pyramid=2 / b_adapt=2 / b_bias=0 / direct=3 / weightb=1 / open_gop=0 / weightp=2 / keyint=250 / keyint_min=25 / scenecut=40 / intra_refresh=0 / rc_lookahead=60 / rc=crf / mbtree=1 / crf=20.0 / qcomp=0.60 / qpmin=0 / qpmax=69 / qpstep=4 / vbv_maxrate=62500 / vbv_bufsize=78125 / crf_max=0.0 / nal_hrd=none / ip_ratio=1.40 / aq=1:1.00
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709

Audio
ID : 2
Format : AAC
Format/Info : Advanced Audio Codec
Format profile : LC
Codec ID : 40
Duration : 1h 28mn
Bit rate mode : Constant
Bit rate : 160 Kbps
Channel(s) : 2 channels
Channel positions : Front: L R
Sampling rate : 48.0 KHz
Compression mode : Lossy
Stream size : 101 MiB (13%)
Language : English

[DenyAll]

As I see it you only have two options to deal with the existing files - either remux the file permanently or remux it on the fly:

Permanantly: I know, not preferred, but if you do decide to go down this path then look at my tool here: viewtopic.php?f=17&t=12136. This will allow you to do bulk file remuxes using ffmpeg (although never tested with that many - you may want to do in batches).

On the fly: Create a small profile in Serviio to remux your MP4 files on the fly eg.

  Code:

<Profile id="humax" name="Humax Fox T2 Freeview" extendsProfileId="1">
   <Transcoding>
      <Video targetContainer="mpegts">
         <Matches container="mp4" />
      </Video>
   </Transcoding>
</Profile>

This profile will remux the existing video and audio streams for all MP4 files into an MPEG-TS container (I assume the Humax supports this, if not try changing "mpegts" to "mpeg"). Copy and paste this profile into your Serviio\Config\Profiles.xml file (use any text editor, and paste it above the last line ie. between the existing </Profile> and </Profiles> statements.) You need to restart your PC after editing this file. Then assign this profile to your Humax Ip address in Serviio Console.

The problem with remuxing like this on the fly is that it may have unwanted side effects (depends on the Humax) - test your pause, fast forward, rewind functions. Ensure aspect ratio is correct.

...to be able to identify the problem, and maybe alter/reconfig something in Handbrake/Vidcoder to avoid this problem in the future.

Identifying what the field is will be extremely difficult. The only thing I can suggest is to use a hex editor, or find specific software to do a deep look into mp4 headers (I couldn't find any in my short search). Make sure you have the latest versions of Handbrake/Vidcoder as they may have corrected the issue.

You could also look at the Handbrake forums and see what settings others use. I'd also download some of their test files and see if they have the same issue.

....also because the original timestamps are destroyed.

Unfortunately any permanent fix will do that....

Hi DenyAll,

Thanks for your time helping me with this, it's soo great to have someone who not only understands the problem exactly, as you do, but also gives super clear help and guidance.

I've tried your profile, it works perfectly - far quicker to start, and less CPU overhead than any of the other profiles I've tried - I assume the remixing into ts streams takes less overhead.

So that's my solution!

I did try the latest builds of handbrake and Vidcoder, but they still produce incompatible mp4's - so annoying.

Anyway many thanks again
Ti