Transcoding testing? Supposed to work in latest master?

Rainer Schütz

Hi,

I am pretty excited about TVHeadend, thanks very much for this project and community! I would like to use TV Headend to complement my Plex server. I was hoping I could use it to provide transcoded TV streams to Plex to be able to get them on Chromecast. I that a feasible/possible working mode? What I'd hope is that TVHeadend takes a IPTV-supplied sd-stream from a sat-receiver and transcodes it on the fly to mpeg4(h264/AAC).

I compiled from latest master and it seems - after some trial and error - I got TVHeadend to receive streams from an Eyetv netstream. What I did to verify this was: Configure until I had a Channel in the system that seemed active within TVHeadends, hit the "play" button, paste the resulting m3u file in VLC, and I get a working stream originating from TVHeadend. So far great!

I cannot watch the stream within TVHeadends though. Hitting 'Watch TV' in the Electronic Program Guide produces a Window, but it remains black whichever resolution or program I select (sd-TV). I get the following output in the log-pane:

014-10-12 09:38:23.389 subscription: "HTTP" subscribing on "BBC One London", weight: 100, adapter: "IPTV", network: "EyeTV", mux: "london.bbc1.bbc.co.uk", provider: "BSkyB", service: "BBC One Lon", hostname="10.0.1.100", username="", client="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/38.0.2125.101 Safari/537.36"
2014-10-12 09:38:24.978 transcode: Unable to find libx264 encoder
2014-10-12 09:38:24.978 transcode: 1:MPEG2VIDEO ==> Passthrough

2014-10-12 09:38:24.979 transcode: 2:MPEG2AUDIO ==> AAC

Is that "2014-10-12 09:38:24.978 transcode: Unable to find libx264 encoder" a significant clue, is there anything I can do? My build seemed to run through well (after a few tweaks I found in the forum). This is on Ubuntu 14.04. What also seems a bit puzzling is that the message shows transcoding going on, although I did not request anything. I meant just to use the "pass" profile (set to default with no stream filter enabled/configured).

I saw quite a few commits relating to transcoding, so I thought you might appreciate the report (of course I don't expect everything to work in a bleeding edge version)

Thanks for any hint/help!
.r.

ProfYaffle

Try sudo apt-get install libx264-dev (you can use apt-cache policy libx264* to check what you already have and what's available).

Transcoding is an ongoing work, so there are daily commits to get this working - keep an eye on the source. I know there have been issues with certain types of CODEC for audio or video streams, for example, and even specific versions of ffmpeg/libav. Accept that it's a moving target, and you'll be fine :)

Be careful of the overhead, though... re-encoding MPEG2 to H.264 in realtime does need reasonable grunt, and to do anything with an HD signal (all H.264 anyway in the UK) is seriously heavy lifting. Squashing an MPEG2 stream down to something you can then relay upstream on a low-bandwidth link should be doable if you're not really picky about quality, though, I'd guess.

Also, in terms of playing the live stream, try using the Play link in the programme information instead of the web player. That works relatively reliably for me - depending on the client, at worst I've perhaps had to save the Play link as a file and rename it as .xspf (or .m3u).

Rainer Schütz

Thanks a lot Prof :) I guess I'll just be patient for a while. I verified I have everything libx264-related on my system, I even compiled the library from videolan. But maybe there is something messy in my setup or there is something wrong with tvheadends detection. Let's see, I am not in a particular hurry. This is my output for apt-cache policy libx264*:

libx264-dev:
Installed: 3:0.142.2431+git76440~trusty1
Candidate: 3:0.142.2431+git76440~trusty1
Version table:
* 3:0.142.2431+git76440~trusty1 0
500 http://ppa.launchpad.net/mc3man/trusty-media/ubuntu/ trusty/main amd64 Packages
100 /var/lib/dpkg/status
2:0.142.2389+git956c8d8-2 0
500 http://gb.archive.ubuntu.com/ubuntu/ trusty/universe amd64 Packages
libx264-142:
Installed: 3:0.142.2431+git76440~trusty1
Candidate: 3:0.142.2431+git76440~trusty1
Version table:
* 3:0.142.2431+git76440~trusty1 0
500 http://ppa.launchpad.net/mc3man/trusty-media/ubuntu/ trusty/main amd64 Packages
100 /var/lib/dpkg/status
2:0.142.2389+git956c8d8-2 0

500 http://gb.archive.ubuntu.com/ubuntu/ trusty/universe amd64 Packages

And additionally a lot of stuff relating to libx265, not sure why that shows up...

Thanks again
.r.

Rainer Schütz

This is getting very exciting! If I just do a default build, libx264 is still not found,but if I build with the (new?) build option --enable-liffmpeg_static codecs are pulled in and built together with tvheadend. The result: libx264 is found and I do get various transcoded streams that play beautifully in VLC! For example: a DVBS stream sent from eyetv netstream sat (old nodel, not the one with mp4-hardware-transcoding) is transcoded on the fly to webm-libvorbis in 384 resolution, and it plays nicely and perfectly in sync in VLC. My server (with a i5 m560 processor - so middle powered) runs at around 50% countinuous cpu-load (I think it's only one core) - not too bad, I think. At full resolution (576) it goes up to 100 and above (continues to play cleanly), which makes me think the second core still has some reserve. Very exciting!!!

I don't have the time this morning to test out all codec-combinations and post my findings, but I'll do so later. On a first quick check I found the following:

- WebM&libvorbis seems to work beautifully

- h264 works well too

- there seems to be a problem with AAC sound. So a combination of WebM or h264 with AAC will both stream nicely - but there is just no sound. VLC-Mediainformation reports an AAC/mp4a audio-stream, but I hear nothing. A (possibly) interesting detail: when I switched the soundcodec back to libvorbis, VLC continued to report a AAC-stream but produced sound!

- the tvheadend internal playback still doesn't work (has low priority for me, just reporting in case you would like to know). I get this output in the log:
2014-10-14 10:28:55.404 subscription: "HTTP" subscribing on "BBC One London", weight: 100, adapter: "IPTV", network: "EyeTV", mux: "london.bbc1.bbc.co.uk", provider: "BSkyB", service: "BBC One Lon", hostname="10.0.1.100", username="", client="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/38.0.2125.101 Safari/537.36"
2014-10-14 10:28:57.042 TS: EyeTV/london.bbc1.bbc.co.uk/BBC One Lon: MPEG2VIDEO @ #5000 Continuity counter error (total 1)
2014-10-14 10:28:57.089 transcode: 1:MPEG2VIDEO 720x576 ==> H264 480x384
2014-10-14 10:28:57.089 transcode: 2:MPEG2AUDIO ==> AAC
2014-10-14 10:28:57.690 libav: using SAR=64/45
2014-10-14 10:28:57.694 libav: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2
2014-10-14 10:28:57.696 libav: profile Constrained Baseline, level 2.1
2014-10-14 10:28:57.696 libav: 264 - core 142 - H.264/MPEG-4 AVC codec - Copyleft 2003-2014 - http://www.videolan.org/x264.html - options: cabac=0 ref=3 deblock=1:0:0 analyse=0x1:0x111 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=4 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=25 keyint_min=2 scenecut=40 intra_refresh=0 rc_lookahead=25 rc=abr mbtree=1 bitrate=368 ratetol=1.0 qcomp=0.60 qpmin=10 qpmax=30 qpstep=4 vbv_maxrate=3670 vbv_bufsize=1835 nal_hrd=none filler=0 ip_ratio=1.41 aq=1:1.00

2014-10-14 10:29:15.243 subscription: "HTTP" unsubscribing from "BBC One London", hostname="10.0.1.100", username="", client="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/38.0.2125.101 Safari/537.36"

Will test with more detail later (the log produces a lot of data when I play from VLC, including some error-messages), but I have to run now!

Thanks for this, very exciting!
Best
.r.

Matthias Strack

Sorry capturing your thread, but i have a smiliar issue with transcoding.
It works with THVClient & TVHGuide, but not in VLC.
Whats the correct URL to enable transcoding while vieweing a stream with VLC?
Tried with http://192.168.1.17:9981/play/stream/channel/28ef44674923cbf29fccdb757bf19dff??mux=matroska&acodec=AAC&vcodec=H264&resolution=576 but i still get the 'normal' untranscoded stream.

alf alfonsius

master is working with profiles now, like /stream/channelnumber/1?profile=webtv-h264-aac-mpegts

Rainer Schütz

Thanks for that tip!

I've done my tests by now, the picture is pretty conclusive, the main problems are ac3 and aac:

Note: build used from 15th Oct. Last commit: linuxdvb_adapter: remove.. (2637ee0aad72f1cff14814e3f4782405b1094d64)

## vlc-playback
- aac-transcoded-streams have no sound, a stream shows up, but no sound (actually
that can only be verified for mpga->aac, as ac3->aac doesn't work for another
reason).
- no ac3-decoding, log message: ac3-decoder not found (although this test used a
fresh build from a fresh checkout with --enable-libffmpeg_static). So ac3-sound
works if the client supports it, but it cannot be transcoded.
- hd-transcodings return two audio-streams (as reported by vlc) while the source
has only one. Vlc does not allow to select that second stream, though. And
the dvbs-stream from the original is not returned in the transcode.

- the mpegts-stream seems to be pretty broken

So at the moment HD-streams cannot be transcoded for clients that only support aac

or libvorbis audio.

### Profiles:
- matroska (passthrough): SD & HD sound&video ok
- pass: SD & HD sound&video ok
- transcoding codecs
-- webtv-h264-aac-matroska (384): SD: no-sound, no image, HD: sound-and image,
but: wrong audio-codec (a52), two audio streams, lost undf stream
-- webtv-h264-aac-mpegts (384): HD&SD: stream never get's stable, automatic unsubscribe
-- webtv-h264-vorbis-webm (384): SD&HD: sound & image work, but HD audio wrong
codec (as above)
-- webtv-vp8-aac-webm (384): SD: no sound, video works, HD: video and sound work,
but sound is wrong codec (as above)
-- webtv-vp8-vorbis-webm (384): SD&HD work, but HD using ac3 (as above)
## tvh WebUI-playback:
- SD: no image no sound
- HD: image but no sound (ac3-decoder not found)
- resolution-switch not-functional

- cosmetical WebUI-player often opens with "Unspecified error"

## little/unrelated issues found on the way
- help in stream-configuration missing
- build-problem (?): the libav_static build seems not to be changed once detected,

even if there are build-configuration changes

I made tables where log messages for each profile are copied. If somebody would
like to see them let me know. It's a lot of lines, so I omitted posting it for now...
Best
.r.

Rainer Schütz

One example of the logs where you can see the failing ac3:

- webtv-h264-aac-matroska (384) video & audio, but wrong audio-codec
- str0:avc1
- str1&2:ac3 (!)
- str3:subt
- 5 unnumbered streams with provider info
- cpu-usage: 160%

- remark: something is fishy with audio-streams: vlc reports 2 ac3-streams and no undf stream. Sound works because of ac3 passthrough, but it should be aac. Only one audio-stream is accessible through vlc-menu, source only contains one audio-stream

2014-10-15 12:25:00.499 mpegts: hd.news.bbc.co.uk in EyeTV - tuning on IPTV
2014-10-15 12:25:00.500 subscription: 0029: "HTTP" subscribing on "BBC News HD", weight: 100, adapter: "IPTV", network: "EyeTV", mux: "hd.news.bbc.co.uk", provider: "BSkyB", service: "BBC NEWS HD", hostname="10.0.1.100", username="", client="VLC/2.1.5 LibVLC/2.1.5"
2014-10-15 12:25:02.144 TS: EyeTV/hd.news.bbc.co.uk/BBC NEWS HD: H264 @ #5000 Continuity counter error (total 1)
2014-10-15 12:25:02.168 transcode: 1:H264 1920x1080 ==> H264 682x384
2014-10-15 12:25:02.168 transcode: 2:TELETEXT ==> Filtered
2014-10-15 12:25:02.168 transcode: 3:DVBSUB ==> Filtered
2014-10-15 12:25:02.168 transcode: Unable to find AC3 decoder
2014-10-15 12:25:02.168 transcode: 4:AC3 ==> Passthrough
2014-10-15 12:25:03.193 libav: mmco: unref short failure
2014-10-15 12:25:03.195 libav: Increasing reorder buffer to 1
2014-10-15 12:25:03.218 libav: Increasing reorder buffer to 2
2014-10-15 12:25:03.220 libav: using SAR=1/1
2014-10-15 12:25:03.221 libav: Increasing reorder buffer to 3
2014-10-15 12:25:03.226 libav: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2
2014-10-15 12:25:03.229 libav: profile Constrained Baseline, level 3.0
2014-10-15 12:25:03.229 libav: 264 - core 142 - H.264/MPEG-4 AVC codec - Copyleft 2003-2014 - http://www.videolan.org/x264.html - options: cabac=0 ref=3 deblock=1:0:0 analyse=0x1:0x111 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=4 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=25 keyint_min=2 scenecut=40 intra_refresh=0 rc_lookahead=25 rc=abr mbtree=1 bitrate=523 ratetol=1.0 qcomp=0.60 qpmin=10 qpmax=30 qpstep=4 vbv_maxrate=3670 vbv_bufsize=1835 nal_hrd=none filler=0 ip_ratio=1.41 aq=1:1.00
2014-10-15 12:25:03.435 libav: mmco: unref short failure

Rainer Schütz

Fantastic news! ac3 now found and something significantly changed on aac. Even some profiles work in the Web-viewer now. I will report soon which ones work and which ones not.

The new htsp stream is probably something that should be used with dedicated clients only? I tried it twice with VLC and the tvh-server immediately crashed. It's a mild crash, the system is not affected and the tvh-server can be restarted immediately. In case somebody is interested in the syslog-output:

Oct 16 17:51:10 lg-vaio tvheadend[1118]: AVAHI: Service 'Tvheadend' successfully established.
Oct 16 17:51:32 lg-vaio tvheadend[1118]: CRASH: Signal: 11 in PRG: /usr/local/bin/tvheadend (3.9.1871~g26ccee6) [a26c01bb09dffe6f8c814e6645a5dc1d1cda758d] CWD: /
Oct 16 17:51:32 lg-vaio tvheadend[1118]: CRASH: Fault address (nil) (Address not mapped)
Oct 16 17:51:32 lg-vaio tvheadend[1118]: CRASH: Loaded libraries: /lib/x86_64-linux-gnu/libssl.so.1.0.0 /lib/x86_64-linux-gnu/libcrypto.so.1.0.0 /lib/x86_64-linux-gnu/libz.so.1 /usr/lib/liburiparser.so.1 /usr/lib/x86_64-linux-gnu/libavahi-common.so.3 /usr/lib/x86_64-linux-gnu/libavahi-client.so.3 /lib/x86_64-linux-gnu/libdbus-1.so.3 /lib/x86_64-linux-gnu/libdl.so.2 /lib/x86_64-linux-gnu/libpthread.so.0 /lib/x86_64-linux-gnu/libm.so.6 /usr/local/lib/libiconv.so.2 /lib/x86_64-linux-gnu/librt.so.1 /lib/x86_64-linux-gnu/libc.so.6 /lib64/ld-linux-x86-64.so.2 /lib/x86_64-linux-gnu/libnss_compat.so.2 /lib/x86_64-linux-gnu/libnsl.so.1 /lib/x86_64-linux-gnu/libnss_nis.so.2 /lib/x86_64-linux-gnu/libnss_files.so.2
Oct 16 17:51:32 lg-vaio tvheadend[1118]: CRASH: Register dump [23]: 000000000016e36000007fb1f40008c0000000007fffffff00000000000002830000000001201d70000000000000000000000000011fef0000007fb1fbffe79000000000011fef0000007fb1fbffe3c000007fb1fbffe4d000007fb1fbffe73000000000000000000000000000000000000000000000000000007fb1fbffe3580000000000000000000000000001024600000000000000330000000000000014000000000000000efffffffe7ffbba170000000000000000
Oct 16 17:51:32 lg-vaio tvheadend[1118]: CRASH: STACKTRACE
Oct 16 17:51:32 lg-vaio tvheadend[1118]: CRASH: /home/ri/Projects/tvheadend/src/trap.c:148 0x459989

Oct 16 17:51:32 lg-vaio kernel: [89289.658003] tcp_server_star[1170]: segfault at 0 ip (null) sp 00007fb1fbffe358 error 14 in tvheadend[400000+794000]

I'll be back with more on how things now work in my environment. Things get more and more exciting!
Thanks a lot for all of this, it's much appreciated!
.r.

Rainer Schütz

More good news: AAC audio works with the new libav-matroska container!!! So there seems to be an interdependence between getting AAC-transcoding to work and the container used.

Atm the libav-matroska container displays a wrong image-proportion. While the resolution in my testscreen should be 480x384 (4:3) the image displays in a wide-screen proportion with objects horizontally stretched.

I'll post a more comprehensive test soon. But the general conclusion for now is: AAC does not yet work with all of the old containers, libvorbis does. So since AC3 decoding works now, one can get video&sound from sd and hd DVBS as broadcasted in UK/Freesat. Yet the probably most useful (in terms of general support) combination h264-aac only works with the new libav-matroska container yet.

Apparently additional streams like epg, channel status etc are now consistently filtered out in transcoded streams - I guess that's on purpose and it looks reassuring to me.

In general I am positively surprised that resource consumption for full resolution SD-transcode (720x576) is still doable on a mainstream-consumer-ware server (eats about 200% of 400% available). Vp8 seems a little more economic than h264 and I haven't seen a significant difference in quality yet.

Passing through a HD-video and just converting audio ac3->libvorbis only eats about 10% of system resources - so it might be possible to use that path for HD as the Chromecast should be able to process a HD-h264-stream. Actual transcoding of a full-resolution HD signal, eg to vp8 eats about 380% on average on my system and is unstable. But then I wouldn't see a practical use-scenario for that anyways.

Thanks again for all your efforts!
Best
.r.

Rainer Schütz

To be a bit more precise on how AAC does not work:
Transcoding seems to be going on, I get a stable stream and tvh eats about 10% system-resources if I pass through the video and ask for a ac3->aac or mpga->aac transcode only. VLC Media Information also reports a AAC-audio stream.
But there is no sound (except with the new matroska container)...
Best
.r.

Rainer Schütz

AAC works now with all containers and video-codecs, this is fantastic!!!! Congratulations and big thanks!!!

So from my perspective (and from what I know up to now) I got all I need!

What I've found in passing by though, just for your info:
- trying to open the htsp-profile from VLC crashes tvh instantly (requires specific client?)
- the mpeg-ps profile doesn't build up (not fro streaming?)

- the new ffmpeg-matroska opens with a horizontal stretch

But once again, thanks a bunch for those speedy reactions!

.r.