Serviio

Hello to All,
I have spend quite a some time searching for solutions, but to no avail.
My problem(s):
Serviio v1.9.0 (did not try any earlier version or beta)
Tryingt to stream web stream segmented (HLS), HD, audio aac and video h264 (High) 1920x1080, and stream it to WDTV profile.
WDTV by itself supports ts and aac without any issues (no transcoding needed).
running command ffmpeg -i http://STREAM_TO_WATCH I am getting all this info, also duration of the stream: Duration: 00:MM:SS.00
No Internet problems.
I also tried to web stream locally on my 1Gb LAN with the same issues and results.
However when trying to watch the stream, since for web streams ffmpeg transcoding is build-in into the serviio code (according to ZIP)), my laptop CPU goes to 100% while transcoding, and shortly after the stream begins, it just stops or looses audio or entirely dies because of CPU usage.
There is no problem if I transmux the segmented stream into an entire .ts file (no more transcoding)
Under serviio I have transcoding disabled. I tried to disable it in WDTV profile, but since for web streams it is build into serviio code, this overtakes and forcefully transcodes the stream, forcing CPU into 100% and dying video stream.
I have also tried to run ffmpeg command but I never succeeded to send the stream to WDTV in a serviio style (or in any ways).
I was able to save it (the stream segments) to a file, but this is not what I do want to do.
Maybe anyone could give me a constructive piece of advice how could I stop serviio transcoding entirely, even web streams, or possibly what command to issue to ffmpeg to send this http:// stream to WDTV?
In detailed serviio logs I do see FFMPEG WRAPPER starting the stream, but there is no command part showing redirection to WDTV.
Also, is there any way to pause or froward stream segments since ffmpeg is aware of the stream length (yet serviio displays it incorrectly)?
Now I only see 00:18:18 as the length of any web stream.
Once again, any constructive advice will be greatly appreciated and welcomed.
Thanks.
P-B

Bump
No one? No hints or suggestions?
P-B

If you set debug logging on you will see the ffmpeg command that is being executed. You can then run that from a command line and see what ffmpeg says is happening. May provide some hints.

You can also try playing with the profile:

• Edit config/profiles.xml using your favourite text editor (Notepad++ recommended).
• Find the following within the "WDTV Live" profile:
  

    Code:

  <OnlineTranscoding>
   <Video targetContainer="mpeg" targetVCodec="mpeg2video" targetACodec="ac3" aBitrate="384">
      <Matches container="mp4" />
      <Matches container="asf" /> <!-- for mms:// -->
   </Video>
</OnlineTranscoding>


• Change the first line to:
  <Video targetContainer="mpeg" targetACodec="ac3" aBitrate="384">
• Save the file and restart your PC

This will remux the video rather than transcoding it. No idea if this will work. Try targetContainer="mpegts" as well.

Hello again,
@DenyAll---Thanks for your advice, however this does not solve the problem.
Let me reiterate it again:
1. Serviio works fine in general, however,
2. when it invokes ffmpeg on m3u8 web live streams, ffmpeg during transcoding/remuxing uses 100% CPU of my machine and ultimately videos die (stutter, loose audio, etc, they are just simply not watchable)
3. Changing profile for WDTV (my renderer) as DenyAll suggested, does not make any difference since Serviio overrides profile settings while live web streaming.
4. I tried to stream simple .ts file and there is no transcoding/remuxing, but .m3u8 "playlist" files are always transcoded/remuxed.
5. When I try commandline ffmpeg: ffmpeg -i http://MY_STREAMED_URL.m3u8 -c copy -f h.264 udp://localhost:1234, I can stream it to vlc without any problems
6. when I try 5. to pipe: to any process (| vlc or java) ffmpeg sends it somewhere, but I cannot find it or play it. (I also tried to invoke ffserver, but this one is becoming deprecated, and has issues with mp4 files, among other issues).
7. I wish, I could know how to pipe/redirect ffmpeg stream to serviio's java process to override serviio's forceful transcoding/remuxing of web live m3u8 streams.
8. Or maybe there is a hidden door to completely disable serviio's forceful transcoding/remuxing of web live m3u8 streams?
I tried PM ZIP for any hints, however the board will not let me (will not allow me) do it, since I am too new member.
Any suggestions in this matter?
Sincerely,
P-B

try to add this as another entry to the WDTV profile's Online Transcoding:

Hello Again,
Thank you Zip for responding to my call for help.
Also, again thanks to DenyAll for some hints.
I did some research and attaching zipped serviio logs (in DEBUG mode) and additional info which might be useful for debugging.
But making long story short Zips hint:

or DenyAll hint:

were only partially helpful.
applehttp container seems to be incompatible with WDTV: I could not open any live web streams.
I ended up with mpegts, which is not perfect but seems to cycle CPU more frequently, although still well above 90% usage.
I have tested streams with 1920x1080, 1280x720, and 960x540 video resolutions (should be included in serviio logs).
I also tested local (python http invoking command; I did not feel like involving Apache server into the puzzle), and only plain .ts file seems not to transcode/transmux. If m3u8 playlists are involved, ffmpeg "lays its hands" on them and CPU goes way up.
Well, anyway, please see attached zipped logs and other info, and if you would need any additional info, please, do not hesitate to contact me.
BTW streams urls in logs are geo-localized and will not play in other geo-locations.
Sincerely,
P-B

how does your wdtv profile look like after my suggestion? and did you restart serviio serivce / pc after you made that change

Hello Zip,
I just appended the selection you indicated:

<Video targetContainer="mpegts">
<Matches container="applehttp"/>
</Video>

to original profoles.xml file:
to the front of the WDTV Live section:

Sections BEFORE this part were left UNCHANGED (NOT MODIFIED)
</Transcoding>
<OnlineTranscoding>
<Video targetContainer="mpeg" targetVCodec="mpeg2video" targetACodec="ac3" aBitrate="384">
<Matches container="mp4" />
<Matches container="asf" /> <!-- for mms:// -->
</Video>
</OnlineTranscoding>
<AudioTrackRemux> Following Sections FURTHER Down were left UNCHANGED (NOT MODIFIED)

So ultimately it looked this way:

BEFORE UNCHANGED (NOT MODIFIED)
</Transcoding>
<OnlineTranscoding>
<Video targetContainer="mpegts">
<Matches container="applehttp"/>
</Video>
<Video targetContainer="mpeg" targetVCodec="mpeg2video" targetACodec="ac3" aBitrate="384">
<Matches container="mp4" />
<Matches container="asf" /> <!-- for mms:// -->
</Video>
</OnlineTranscoding>
<AudioTrackRemux> FURTHER UNCHANGED (NOT MODIFIED)

So, the profile looked as the original file with the insert in the WDTV Live OnlineTranscoding section. I did not remove (delete) anything, any other parts from the original profiles.xml file.

Also, before I made any changes to serviio configuration I did STOP the service:
/opt/serviio/bin/serviio.sh -stop (java process disappeared from processes list; serviio was not on media servers list)
and after configuration modifications I did restart serviio:
/opt/serviio/bin/serviio.sh -start (java process re-appeared on processes list; serviio appeared again on media servers list)

On occasion I would reboot the system entirely, but not every time I did made changes to serviio configurations.
FYI: This is Ubuntu 17.10 x64 (not Windows), so I did not think it was necessary to reboot ever time after configuration changes, just stopping and re-starting the service/process was sufficient enough (but maybe this was wrong?).
Sincerely,
P-B

can you send the detailed log again, trying to play that stream with this updated profile?

Hello Zip,
the attempt to use your suggested profile additions:
<Video targetContainer="mpegts">
<Matches container="applehttp"/>
</Video>
were logged and are in the attached files.
Beeing more specific:
Serviio.log.1 (3 hits)
Line 1383: [hls,applehttp @ 0x557294368c20] Opening 'https://sdt-plix35-12.tvp.pl/token/video/vod/35924191/20180225/3240244527/619e41f0-1299-4b5d-87bb-844c4af9fb06/video.ism/video-audio=173000-video=1991000-33.ts' for reading
Line 2715: [hls,applehttp @ 0x558432f6dc20] Opening 'https://sdt-plix35-12.tvp.pl/token/video/vod/35924191/20180225/3240244527/619e41f0-1299-4b5d-87bb-844c4af9fb06/video.ism/video-audio=173000-video=4785000-18.ts' for reading
Line 2879: [hls,applehttp @ 0x5588459fbc20] Opening 'https://sdt-plix35-12.tvp.pl/token/video/vod/35924191/20180225/3240244527/619e41f0-1299-4b5d-87bb-844c4af9fb06/video.ism/video-audio=173000-video=3280000-27.ts' for reading
Serviio.log.2 (2 hits)
Line 1528: [hls,applehttp @ 0x5604ed123c20] Opening 'https://sdt-plix35-12.tvp.pl/token/video/vod/35924191/20180225/3240244527/619e41f0-1299-4b5d-87bb-844c4af9fb06/video.ism/video-audio=173000-video=3280000-27.ts' for reading
Line 1694: [hls,applehttp @ 0x5588dba51c20] Opening 'https://sdt-plix35-12.tvp.pl/token/video/vod/35924191/20180225/3240244527/619e41f0-1299-4b5d-87bb-844c4af9fb06/video.ism/video-audio=173000-video=1991000-35.ts' for reading

But before these line I can see some kind of ffmpeg error:
2018-02-25 16:40:22,415 DEBUG [TimeoutStreamDelegator] Closing piped input stream and closing related feeder process
2018-02-25 16:40:22,415 DEBUG [ProcessExecutor] Stopping external process: Thread[Thread-102,5,main]
2018-02-25 16:40:22,415 DEBUG [ProcessUtils] Sending kill -9 to the Unix process: 23270
2018-02-25 16:40:22,415 DEBUG [ProcessExecutor] Starting kill -9 23270
2018-02-25 16:40:22,426 WARN [ProcessExecutor] Process ffmpeg has a return code of 137! This is a possible error. Detailed output follows.
frame= 3099 fps= 71 q=1.0 size= 57236kB time=00:02:03.98 bitrate=3781.6kbits/s dup=2 drop=0 speed=2.82x
frame= 3143 fps= 71 q=1.0 size= 57780kB time=00:02:05.69 bitrate=3765.7kbits/s dup=2 drop=0 speed=2.83x

In every listed above instance of applehttp container, such ffmpeg 137! error shows up.
Not sure if this is what you are looking for.
Please, advice what others steps should I undertake, to provide better logs and possible resolution to this puzzle.

As a side note, would it be possible to add me to serviio beta-testers group, if I could possibly contribute with something to this project, I will gladly do it.

Thank You. Very Sincerely,
P-B

There is something weird. First, it uses the wrong transcoding profile to what I'd expect, and second the WDTV sends a bye-bye message after a while and that kills the transcoding job (or more likely the job is killed after 20 seconds of inactivity on the device part).

Did you turn off the device during streaming or something?

Hello Zip,
please see attached more serviio logs.
Just to clarify previous logs contained entire profile without any deletions, just suggested additions and modifications.
And after the video started streaming (or not) I would terminate serviio to proceed to another test option, that is why they were terminating after 20 secs.
This time I did as follows:
I edited profiles.xml file in such a way, that I left only default DLNA ID=1 profile and WDTV (since without profile 1 serviio would not start).
OK, so, now 1st log it shows original profile (WDTV only, but no changes to profile), and video stream starts. I ran it about 20-30 secs, and proceeded to the next option (after stopping serviio, emptying log file, making modifications to profiles file, and re-starting serviio).
Profile and log 2 contained only your suggested modifications: applehttp (exactly as indicated in previous posts, replacing original entries of WDTV section). It was not possible to play video with WDTV error that it was unknown (non-compatible) file format.
Profile and log 3 I replaced applehttp with "*" (asterisk). Results as with applehttp-No video play.
And finally I did modified original WDTV profile, it would be:
<OnlineTranscoding>
<Video targetContainer="mpegts">
<Matches container="mp4" />
<Matches container="asf" /> <!-- for mms:// -->
</Video>
</OnlineTranscoding>
<AudioTrackRemux>.......
and video stream played again, of course transcoded.
If You need more logs and info, please do not hesitate to contact me.
Sincerely,
P-B

so it looks like with log2, it's what I was trying to do - transcode HLS to mpegts (byt with the audio and video streams left untouched, thus using very little CPU).
But it looks like the device doesn't like this format.

Serviio doesn't support passing through HLS streams, without touching them somehow.

Hi Zip,
thanks a lot for your assistance regarding this issue.
So, I guess I will have to live with the current situation and just hope for future developments.
(Unfortunately WDTV is out of support; EOL=End_of_Life, so this side of equation will not change). Maybe some other future developments will bring a viable solution to the issue, lets hope.
Or maybe there will be a way to pipe ffmpeg i/o to serviio any other way (eg from cli?) to pass it to WDTV with ffmpeg directives and without serviio's ffmpeg transcoding attempts.
But, once again, truly thank you for your time and assistance.
Very Sincerely,
P-B

Dude it might be worth it to state what your usual bandwidth is since what you're watching is online? That's why it doesn't work for me.

Hello again,
Just to let you know I think I have found a temp solution to my problem.
I have scanned all available serviio profiles for different devices and assigned one-by-one all of them to my WDTV SMP box.
I have found that 3 profiles would play web live streams of my desire with minimal ffmpeg involvement (almost none; 0-1% CPU).
Roku 4K TV, Roku TV (non 4K), and Sony BDP-Sx80/SMP-N200 (2011) would produce similar (not exact: -map 0:X, X would vary) ffmpeg command execution:
ffmpeg -fflags +genpts -threads 0 -analyzeduration 10000000 -i http://MY_WEB_LIVE_STREAM.m3u8 -y -threads 0 -c:v copy -c:a:0 copy -map 0:1 -map 0:0 -sn -f mpegts pipe:
The web streams startup takes a longer while up to 1-2 mins to start playing, and sometimes they abort to play, but at some point they do play.
I have noticed that Roku devices are referencing some other Roku profiles. Similar with Sony.
So, now I have to sift through Roku profiles to figure out what parts are relevant to succeed with such ffmpeg behaviour.
Debug logs reffer to something like this: AVC_TS_MP_HD_AAC_MULT5_ISO as successful format.
This item is included in "Generic DLNA profile", however WDTV and most of other profiles seem to ignore it.
It must be something additional in Roku 4K profile that produces such an output. Even Roku Media Player, which profile is referenced in Roku 4K does not produce proper output.
Since I am not good with codecs and containers, it might take me a while to sift through and sort it out, but as soon as I figure it out (I hope so), I will report it here for others to make use of it whenever needed.
Sincerely,
P-B

The same media profile is used when you added the applehttp transocidng to wdtv profile (log 2 from your last attachment).



Only there is no ffmpeg invoked then, and you stop the server. So maybe it was the case when it sometimes starts, and sometimes it does not.

Hello Zip,
Not sure what to say.
When I used the profile attached to this log 2 file, I got response from my WDTV that it was unsupported file, and it would not want to play it. It gives the only option to dismiss this info screen and try again.
Why nothing showed up in the log file, I do not know. Logging was set up to DEBUG.
After 4-5 attempts to play the file and the same response from WDTV: Unsupported file and no way to get it to play, I just stopped serviio server and moved on to the new configuration with the new empty log file to log other profile debug info.
I have also noticed some other strange behavior while trying to play web live streams playlists and .ts files, which I will try to describe as another topic in this forum, but, this is not only related to WDTV, but also to other renderers, like Android VLC player.
Sincerely,
P-B