Unable to play TVheadend recordings (mkv container)

Hi,
if i recorded with TVH and the buildin profile settings "matroska" TV shows.
I spend some time to figure out why they wont play, neither on my Samsung nor on my Philips TV.
Finally it seems to be a problem with Serviio, as all works well with minidlna.
The effects is that it starts playing for less than a second and then stops.
I use the latest Serviio on Ubnutu 16.04.

Can you please check why it is not working and maybe fix it?

Uploaded a demo file here:
http://www.file-upload.net/download-118 ... 5.mkv.html


Here the file info:

  Code:

ffmpeg version 2.8.6-1ubuntu2 Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 5.3.1 (Ubuntu 5.3.1-11ubuntu1) 20160311
  configuration: --prefix=/usr --extra-version=1ubuntu2 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-libx264 --enable-libopencv
  libavutil      54. 31.100 / 54. 31.100
  libavcodec     56. 60.100 / 56. 60.100
  libavformat    56. 40.101 / 56. 40.101
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 40.101 /  5. 40.101
  libavresample   2.  1.  0 /  2.  1.  0
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  2.101 /  1.  2.101
  libpostproc    53.  3.100 / 53.  3.100
Input #0, matroska,webm, from 'Hart aber fair 2016-08-30.10-15.mkv':
  Metadata:
    title           : Hart aber fair
    encoder         : Tvheadend Matroska muxer
    DATE_BROADCASTED: 2016-08-30 10:15:00
    ORIGINAL_MEDIA_TYPE: TV
    CONTENT_TYPE    : News / Current affairs
    TVCHANNEL       : 3sat HD
    SUMMARY         : Der Feind in meinem Essen -
    SUMMARY-ger     : Der Feind in meinem Essen -
    DESCRIPTION     : Deutschland 2016Tim Mälzer - Koch und RestaurantinhaberChristian Schmidt, CSU - Bundesminister für Ernährung und LandwirtschaftGünter Tissen - Hauptgeschäftsführer der Wirtschaftlichen Vereinigung ZuckerSilke Schwartau - Ernährungsexpertin, Verbrau
    DESCRIPTION-ger : Deutschland 2016Tim Mälzer - Koch und RestaurantinhaberChristian Schmidt, CSU - Bundesminister für Ernährung und LandwirtschaftGünter Tissen - Hauptgeschäftsführer der Wirtschaftlichen Vereinigung ZuckerSilke Schwartau - Ernährungsexpertin, Verbrau
  Duration: 00:01:01.72, start: 0.005000, bitrate: 13013 kb/s
    Stream #0:0(eng): Video: h264 (High), yuv420p(tv, bt709), 1280x720 [SAR 1:1 DAR 16:9], 50 fps, 50 tbr, 1k tbn, 100 tbc (default)
    Stream #0:1(ger): Audio: mp3, 48000 Hz, stereo, s16p, 256 kb/s (default)
    Stream #0:2(mis): Audio: mp3, 48000 Hz, stereo, s16p, 192 kb/s (default)
    Stream #0:3(mul): Audio: mp2, 48000 Hz, stereo, s16p, 192 kb/s (default)
    Stream #0:4(ger): Audio: ac3, 48000 Hz, stereo, fltp, 448 kb/s (default)
    Stream #0:5(ger): Subtitle: dvb_subtitle (default)

It seems i am getting closer to the issue.
The problem is not the file or file format itself. The problem seems to be way how the file is added to the folder which is scanned by serviio.
As tvheadend is "recording" the file directly into the folder which serviio is watching for changes i found the entry of the recorded file appear 3 times in the log:

  Code:

serviio.log:2016-09-02 23:27:01,648 INFO  [MediaFileIndexer] Added file 'Sherlock - Sein letzter Schwur 2016-09-02.23-30.mkv' (title: Sherlock - Sein letzter Schwur 2016-09-02.23-30) to Library
serviio.log:2016-09-02 23:27:02,313 INFO  [MediaFileIndexer] Updated 'Sherlock - Sein letzter Schwur 2016-09-02.23-30' (/mpool/tv/Sherlock - Sein letzter Schwur/Sherlock - Sein letzter Schwur 2016-09-02.23-30.mkv) in Library
serviio.log:2016-09-05 21:11:50,245 INFO  [MediaFileIndexer] Updated 'Sherlock - Sein letzter Schwur 2016-09-02.23-30' (/mpool/tv/Sherlock - Sein letzter Schwur/Sherlock - Sein letzter Schwur 2016-09-02.23-30.mkv) in Library


The first time when it is created and the other two times when serviio realizes that it is updated due to the fact that the file size is of course growing because tvheadend is recording it.
It seems that serviio is storing the wrong information of the length of this file. I assume the movie length is only scanned the first time it is create in the serviio library, when the recording was just started and not updated when the recording finally finished. The effect is what i described already: if i try to play it serviio only shows the frist 1-2 seconds and then it stops.

When i rename this file after the recording was finished so that serviio removes the file entry from its library and scans the file again with the new name everything works fine!!!:

  Code:

serviio.log:2016-09-05 21:21:34,884 INFO  [MediaFileIndexer] Removing 'Sherlock - Sein letzter Schwur 2016-09-02.23-30' (/mpool/tv/Sherlock - Sein letzter Schwur/Sherlock - Sein letzter Schwur 2016-09-02.23-30.mkv) from Library
serviio.log:2016-09-05 21:21:36,633 INFO  [MediaFileIndexer] Added file 'Sherlock - Sein letzter Schwur 2016-09-02.23-30 1.mkv' (title: Sherlock - Sein letzter Schwur 2016-09-02.23-30 1) to Library


@zip: I think this behaviour of serviio can be easiliy corrected, or?

@zip

Any short feedback ?

@zip: please just some remark is you can image it touch this point?

[zip]

Normally duration would be updated on file update as well. But it depends what FFMpeg outputs.

You could try to run ffmpeg -i for the file while it's being written to see if the file duration changes correctly, on this line

  Code:

Duration: 00:01:01.72, start: 0.005000, bitrate: 13013 kb/s

Also your FFmpeg is kind of outdated, you could try to upgrade (Serviio 1.7 now uses 3.0.2)

Normally duration would be updated on file update as well. But it depends what FFMpeg outputs.

You could try to run ffmpeg -i for the file while it's being written to see if the file duration changes correctly, on this line

  Code:

Duration: 00:01:01.72, start: 0.005000, bitrate: 13013 kb/s

Also your FFmpeg is kind of outdated, you could try to upgrade (Serviio 1.7 now uses 3.0.2)

THX for the feedback.

What ffmpeg produces DURING the recording is:

  Code:

Duration: N/A, start: 0.000000, bitrate: N/A

after the recoring (exmaple):

  Code:

Duration: 00:00:42.85, start: 0.000000, bitrate: 11480 kb/s

If i use the "old" ffmpeg or a newer release like 3.1.3 makes no difference.

Also the serviio log never reports an update activity from the MediaFileIndexer AFTER the recoring is finished.

Can this be fixed?

[zip]

yeah, so normally MKV is not streamable format, ie it needs to be fully written in order to be playable (thus FFMpeg being not able to detect duration etc).

not sure this is easily fixable.

yeah, so normally MKV is not streamable format, ie it needs to be fully written in order to be playable (thus FFMpeg being not able to detect duration etc).

not sure this is easily fixable.

ok, but i can say that for example Minidlna is able to detect the length correctly after it was recorded.

BTW, i guess it is not a streaming issue. Maybe because of the fact that at the beginning of the recording the final length is not absolutely sure as you can anytime stop it for exmaple.

[zip]

You could use the poller library mechanism for your library instead. That might help with detecting the final state after its been finished. Check wiki for details, under Library scanning mechanism

You could use the poller library mechanism for your library instead. That might help with detecting the final state after its been finished. Check wiki for details, under Library scanning mechanism

Isn't this not just a workaround for the problem?

[zip]

I'm guessing that Java/ OS cannot detect change in the file as it's being written. So workaround is to manually check for filesize at an interval.

I'm guessing that Java/ OS cannot detect change in the file as it's being written. So workaround is to manually check for filesize at an interval.

Polling workaround seems to work, but not sure if this is the best solution...

[zip]

Best solution would be to download to a different directory than what you share.

Best solution would be to download to a different directory than what you share.

what are the disadvantages of this poll method besides what is written in your faq?

[zip]

It's more CPU / IO intensive if you have a lot of files / directories that you share.

There is a ticket to be able to poll for selected folders rather than everything so keep an eye on it.

https://bitbucket.org/xnejp03/serviio/i ... red-folder

even polling methode seem to have an issue now.
same effect again.
what else can we do?

[zip]

post the detailed log of the file first being added and played.

post the detailed log of the file first being added and played.

Check the log for the TV recording called "Mitternachtsspitzen".

The issue is like in the posts above: Serviio added the file to the lib but doesn't update its final lenght or other meta data at the end.

As soon as i rename the finished recoring Serviio see the whole file as new one and adds it completly to the lib with all needed data.

I think the issue appeared again after using serviio 1.8 (maybe since 1.7 too).

What can we do?

[DenyAll]

Are you sure you are using the Polling method? This suggests not:

2016-12-28 00:23:23,241 DEBUG [PipeManager] Adding file '/mpool/tv/Mitternachtsspitzen/Mitternachtsspitzen 2016-12-27.23-40.mkv' to the delta queue (file added)

From version 1.8 the Poller needs to be enabled for each Shared folder (that you require it for) in Serviio Console (edit the Shared folder and you will see it).

The old switch used for version 1.7 and earlier is no longer valid (not sure whether it produces an error if still used, or is just ignored), so upgrading from 1.7 to 1.8 without doing anything else will (I believe) result in you "reverting" to the Delta scanning method. Version 1.8 allows you to set polling per shared folder (rather than all-in as was previously the case).

Are you sure you are using the Polling method? This suggests not:

2016-12-28 00:23:23,241 DEBUG [PipeManager] Adding file '/mpool/tv/Mitternachtsspitzen/Mitternachtsspitzen 2016-12-27.23-40.mkv' to the delta queue (file added)

From version 1.8 the Poller needs to be enabled for each Shared folder (that you require it for) in Serviio Console (edit the Shared folder and you will see it).

The old switch used for version 1.7 and earlier is no longer valid (not sure whether it produces an error if still used, or is just ignored), so upgrading from 1.7 to 1.8 without doing anything else will (I believe) result in you "reverting" to the Delta scanning method. Version 1.8 allows you to set polling per shared folder (rather than all-in as was previously the case).

ok, greate.
Thanks, maybe overseen this change in 1.8.