TVHeadend Raspberry Pi and Transcoding

lord slash · 2014-11-04T22:19:08+00:00

Hi guys, I've setup a TVHeadend back end with my raspberry pi and raspbian, where I stream the italian tv to my home in Germany. The system works perfectly d...

Antonio S

I did it except for:

install MMAL : https://github.com/raspberrypi/userland (to compile)

Shouldn't userland already be part of raspian? ffmpeg didn't complained about missing mmal

Antonio S

The HW seems to be ok. I have built a custom ffmpeg and tried an encoding:


pi@raspberrypi:~ $ ffmpeg -i dolbyaurora.vob -c:a copy -c:v h264_omx -b:v 128k test.mp4
ffmpeg version 3.2.2 Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 4.9.2 (Raspbian 4.9.2-10)
  configuration: --prefix=/usr --enable-mmal --enable-omx --enable-omx-rpi --enable-decoder=h264_mmal --enable-encoder=h264_omx --enable-encoder=h264_omx --enable-libx264 --enable-libx265 --enable-nonfree --enable-gpl
  libavutil      55. 34.100 / 55. 34.100
  libavcodec     57. 64.101 / 57. 64.101
  libavformat    57. 56.100 / 57. 56.100
  libavdevice    57.  1.100 / 57.  1.100
  libavfilter     6. 65.100 /  6. 65.100
  libswscale      4.  2.100 /  4.  2.100
  libswresample   2.  3.100 /  2.  3.100
  libpostproc    54.  1.100 / 54.  1.100
Input #0, mpeg, from 'dolbyaurora.vob':
  Duration: 00:00:34.98, start: 0.049756, bitrate: 5296 kb/s
    Stream #0:0[0x1bf]: Data: dvd_nav_packet
    Stream #0:1[0x1e0]: Video: mpeg2video (Main), yuv420p(tv, top first), 720x576 [SAR 64:45 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:2[0x80]: Audio: ac3, 48000 Hz, 5.1(side), fltp, 384 kb/s
[h264_omx @ 0x2e032d0] Using OMX.broadcom.video_encode
[mp4 @ 0x2e020e0] track 1: codec frame size is not set
Output #0, mp4, to 'test.mp4':
  Metadata:
    encoder         : Lavf57.56.100
    Stream #0:0: Video: h264 (h264_omx) ([33][0][0][0] / 0x0021), yuv420p, 720x576 [SAR 64:45 DAR 16:9], q=2-31, 128 kb/s, 25 fps, 12800 tbn, 25 tbc
    Metadata:
      encoder         : Lavc57.64.101 h264_omx
    Stream #0:1: Audio: ac3 ([165][0][0][0] / 0x00A5), 48000 Hz, 5.1(side), 384 kb/s
Stream mapping:
  Stream #0:1 -> #0:0 (mpeg2video (native) -> h264 (h264_omx))
  Stream #0:2 -> #0:1 (copy)
Press [q] to stop, [?] for help
frame=   56 fps=0.0 q=-0.0 size=     127kB time=00:00:02.20 bitrate= 473.0kbits/
frame=  110 fps=110 q=-0.0 size=     273kB time=00:00:04.38 bitrate= 510.2kbits/
frame=  163 fps=108 q=-0.0 size=     406kB time=00:00:06.49 bitrate= 512.1kbits/
frame=  214 fps=106 q=-0.0 size=     530kB time=00:00:08.54 bitrate= 508.5kbits/
frame=  265 fps=105 q=-0.0 size=     658kB time=00:00:10.56 bitrate= 510.6kbits/
frame=  316 fps=105 q=-0.0 size=     788kB time=00:00:12.60 bitrate= 512.0kbits/
frame=  366 fps=104 q=-0.0 size=     914kB time=00:00:14.62 bitrate= 511.9kbits/
frame=  416 fps=103 q=-0.0 size=    1038kB time=00:00:16.60 bitrate= 512.2kbits/
frame=  467 fps=103 q=-0.0 size=    1165kB time=00:00:18.62 bitrate= 512.3kbits/
frame=  515 fps=102 q=-0.0 size=    1285kB time=00:00:20.54 bitrate= 512.6kbits/
frame=  565 fps=102 q=-0.0 size=    1411kB time=00:00:22.56 bitrate= 512.5kbits/
frame=  615 fps=102 q=-0.0 size=    1534kB time=00:00:24.57 bitrate= 511.5kbits/
frame=  667 fps=102 q=-0.0 size=    1664kB time=00:00:26.62 bitrate= 511.9kbits/
frame=  716 fps=101 q=-0.0 size=    1789kB time=00:00:28.60 bitrate= 512.2kbits/
frame=  767 fps=101 q=-0.0 size=    1916kB time=00:00:30.62 bitrate= 512.5kbits/
frame=  818 fps=101 q=-0.0 size=    2042kB time=00:00:32.70 bitrate= 511.5kbits/
frame=  872 fps=102 q=-0.0 size=    2176kB time=00:00:34.81 bitrate= 511.9kbits/
frame=  875 fps=102 q=-0.0 Lsize=    2203kB time=00:00:34.96 bitrate= 516.2kbits/

s speed=4.06x    
video:547kB audio:1640kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.768599%

HW accelerations is picked up and the speed is very good. the only thing is puzzling me is this:

pi@raspberrypi:~/ffmpeg-3.2.2 $ ffmpeg -hwaccels ffmpeg version 3.2.2 Copyright (c) 2000-2016 the FFmpeg developers built with gcc 4.9.2 (Raspbian 4.9.2-10) configuration: --prefix=/usr --enable-mmal --enable-omx --enable-omx-rpi --enable-decoder=h264_mmal --enable-encoder=h264_omx --enable-encoder=h264_omx --enable-libx264 --enable-libx265 --enable-nonfree --enable-gpl libavutil 55. 34.100 / 55. 34.100 libavcodec 57. 64.101 / 57. 64.101 libavformat 57. 56.100 / 57. 56.100 libavdevice 57. 1.100 / 57. 1.100 libavfilter 6. 65.100 / 6. 65.100 libswscale 4. 2.100 / 4. 2.100 libswresample 2. 3.100 / 2. 3.100 libpostproc 54. 1.100 / 54. 1.100 Hardware acceleration methods:

Apparently hwaccels list is empty. Could be this causing the problem?

Antonio S

from tvheadend log


2017-01-09 19:04:49.490 [   INFO]:subscription: 007A: "192.168.182.243 [ menion
| Kodi Media Center ]" subscribing on channel "Rete4", weight: 100, adapter: "Sk
y IT Digital Key (green led) : DVB-T #0", network: "DVB-T Network", mux: "698MHz
", provider: "Mediaset", service: "Rete4", profile="omx", hostname="192.168.182.
243", username="menion", client="Kodi Media Center"
2017-01-09 19:04:50.367 [  TRACE]:transcode: 0001: transcoder_calc_stream_count=
2 (video=1, audio=1, subtitle=0)
2017-01-09 19:04:50.367 [   INFO]:transcode: 0001: 2:TELETEXT ==> Filtered
2017-01-09 19:04:50.367 [  TRACE]:transcode: 0001: Using decoder mpeg2video
2017-01-09 19:04:50.367 [  ERROR]:transcode: 0001: Unable to find h264_omx encod
er
2017-01-09 19:04:50.367 [   INFO]:transcode: 0001: 3:MPEG2VIDEO ==> Passthrough
2017-01-09 19:04:50.367 [  TRACE]:transcode: 0001: Using decoder mp2

I have forced h264_omx in the stream. The string is passed correclty, but for some reason the libavcodec doesn' support it.
I don't understand

Alexander Meinke

Hello Antonio,

I do the following to get h264_omx in tvheadend's stream profile tab:

1. git clone <tvheadend>

2. apply the following patches:


diff --git a/Makefile.ffmpeg b/Makefile.ffmpeg
index aa36eae..66ad3ba 100644
--- a/Makefile.ffmpeg
+++ b/Makefile.ffmpeg
@@ -581,7 +581,13 @@ $(LIB_ROOT)/$(FFMPEG)/.tvh_build: \
 		$(foreach encoder,$(ENCODERS),--enable-encoder=$(encoder)) \
 		$(foreach muxer,$(MUXERS),--enable-muxer=$(muxer)) \
 		$(foreach bsf,$(BSFS),--enable-bsf=$(bsf)) \
-		$(foreach filter,$(FILTERS),--enable-filter=$(filter))
+		$(foreach filter,$(FILTERS),--enable-filter=$(filter)) \
+		--enable-nonfree \
+		--enable-mmal \
+		--enable-omx \
+		--enable-omx-rpi \
+		--enable-decoder=h264_mmal \
+		--enable-encoder=h264_omx
 	DESTDIR=$(EBUILDIR) \
 		$(MAKE) -C $(LIB_ROOT)/$(FFMPEG) install
 	@touch $@


diff --git a/src/plumbing/transcoding.c b/src/plumbing/transcoding.c
index 1f2427b..4281b3b 100644
--- a/src/plumbing/transcoding.c
+++ b/src/plumbing/transcoding.c
@@ -1348,7 +1348,7 @@ transcoder_stream_video(transcoder_t *t, transcoder_stream_t *ts, th_pkt_t *pkt)
       tvhinfo(LS_TRANSCODE, "%04X: Using preset %s", shortid(t), t->t_props.tp_vcodec_preset);
 
       // All modern devices should support "high" profile
-      av_dict_set(&opts, "profile", "high", 0);
+      //av_dict_set(&opts, "profile", "high", 0);
 
       if (t->t_props.tp_vbitrate < 64) {
         // encode with specified quality and optimize for low latency

3. Add Autobuild/jessie-armhf.sh with the following content:

AUTOBUILD_CONFIGURE_EXTRA="${AUTOBUILD_CONFIGURE_EXTRA:-} --arch=armhf --disable-libx265_static --disable-libx265 --disable-libvpx_static --disable-libvpx --disable-bintray_cache" DEBDIST=jessie source Autobuild/debian.sh

4. Execute ./Autobuild.sh -t jessie-armhf from source repository

5. dpkg -i <tvheadend>.deb (The package will be placed outside the source repository)

Don't forget - as you already mentioned - to do distclean so ffmpeg will be rebuild.

Antonio S

ahhhh ok
I think I have put the ffmpeg switch in the wrong position (global ffmpeg configure options)
I will check this
Bye

Antonio S

Ok
I have compiled tvheadend to use my self-built shared ffmpeg, I thinks it's way better to have full control of the ffmpeg/libavcodec capabilities
Now I get the h264_omx and I can reproduce the Joseph issue, CPU still overkilled

Looking at the logs I see

2017-01-09 20:34:38.899 [ INFO]:subscription: 000B: "192.168.182.243 [ menion | Kodi Media Center ]" subscribing on chann
el "Canale5", weight: 100, adapter: "Sky IT Digital Key (green led) : DVB-T #0", network: "DVB-T Network", mux: "698MHz", p
rovider: "Mediaset", service: "Canale5", profile="omx", hostname="192.168.182.243", username="menion", client="Kodi Media C
enter"
2017-01-09 20:34:39.383 [ TRACE]:transcode: 0001: transcoder_calc_stream_count=2 (video=1, audio=1, subtitle=0)
2017-01-09 20:34:39.383 [ TRACE]:transcode: 0001:Using decoder mpeg2video 2017-01-09 20:34:39.383 [ TRACE]:transcode: 0001: Using encoder h264_omx
2017-01-09 20:34:39.392 [ INFO]:transcode: 0001: 1:MPEG2VIDEO 720x576 ==> H264 720x576 (h264_omx)
2017-01-09 20:34:39.392 [ TRACE]:transcode: 0001: Using decoder mp2
2017-01-09 20:34:39.392 [ TRACE]:transcode: 0001: Using encoder aac
2017-01-09 20:34:39.394 [ INFO]:transcode: 0001: 2:MPEG2AUDIO ==> AAC (aac)
2017-01-09 20:34:39.487 [WARNING]:tsfix: The timediff for TELETEXT is big (3215664514), using current dts
2017-01-09 20:34:39.586 [ INFO]:transcode: 0001: Using preset faster

2017-01-09 20:34:39.608 [ INFO]:libav: Using OMX.broadcom.video_encode

The problem is the bold part, tvheadedn pickup mpeg2video decoder which is (as far as I understand), the software decoder of mpeg2

Antonio S

I have tried what I suggested and what Morten also tried
The path was too quick and dirty, because hardcoding mpeg2_mmal was forcing tvheadend to pickup mpeg2_mmal also for audio

So I made a "less" quick and dirty patch, but still I get the error:

2017-01-09 20:50:07.370 [ INFO]:subscription: 0007: "192.168.182.243 [ menion | Kodi Media Center ]" subscribing on chann
el "Canale5", weight: 100, adapter: "Sky IT Digital Key (green led) : DVB-T #0", network: "DVB-T Network", mux: "698MHz", p
rovider: "Mediaset", service: "Canale5", profile="omx", hostname="192.168.182.243", username="menion", client="Kodi Media C
enter"
2017-01-09 20:50:07.728 [ TRACE]:transcode: 0001: transcoder_calc_stream_count=2 (video=1, audio=1, subtitle=0)
2017-01-09 20:50:07.728 [ TRACE]:transcode: 0001: Using decoder mpeg2_mmal
2017-01-09 20:50:07.728 [ TRACE]:transcode: 0001: Using encoder h264_omx
2017-01-09 20:50:07.729 [ INFO]:transcode: 0001: 1:MPEG2VIDEO 720x576 ==> H264 720x576 (h264_omx)
2017-01-09 20:50:07.729 [ TRACE]:transcode: 0001: Using decoder mp2
2017-01-09 20:50:07.729 [ TRACE]:transcode: 0001: Using encoder aac
2017-01-09 20:50:07.731 [ INFO]:transcode: 0001: 2:MPEG2AUDIO ==> AAC (aac)
2017-01-09 20:50:07.887 [ ERROR]:transcode: 0001: Unable to decode video (0, Success)
2017-01-09 20:50:07.888 [WARNING]:tsfix: The timediff for TELETEXT is big (3215668114), using current dts
2017-01-09 20:50:07.888 [ ERROR]:transcode: 0001: Unable to decode video (0, Success)
2017-01-09 20:50:07.890 [ ERROR]:transcode: 0001: Unable to decode video (0, Success)
2017-01-09 20:50:07.890 [ ERROR]:libav: MMAL error 2 on control port
2017-01-09 20:50:07.890 [ ERROR]:libav: MMAL error 2 on control port
2017-01-09 20:50:07.891 [ ERROR]:libav: MMAL error 2 on control port
2017-01-09 20:50:07.891 [ ERROR]:libav: MMAL error 2 on control port
2017-01-09 20:50:07.891 [ ERROR]:libav: MMAL error 2 on control port
2017-01-09 20:50:07.891 [ ERROR]:libav: MMAL error 2 on control port
2017-01-09 20:50:07.892 [ ERROR]:libav: MMAL error 2 on control port
2017-01-09 20:50:07.892 [ ERROR]:libav: MMAL error 2 on control port

2017-01-09 20:50:07.892 [ ERROR]:libav: MMAL error 2 on control port

I have to find some info about this MMAL error or asking support from the libav-dev mailing list, even if that list is not so "supportive" for the moment.
Let's keep in touch

Antonio S

Hi all
I have made some more performance test.

I'm using a record from DVB-T so we are 100% aligned with the video source we have to deal with, when transcoding with tvheadend and this are the results (all test with mp2 to aac transcoding):

1) native(sw) -> no deinterlace -> h264_omx


pi@raspberrypi:/media/tvrec $ ffmpeg -i Prova.ts -c:a aac -c:v h264_omx -sn -b:v 128k prova_sw_nodeinterlace.mkv
ffmpeg version 3.2.2 Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 4.9.2 (Raspbian 4.9.2-10)
  configuration: --prefix=/usr --enable-mmal --enable-omx --enable-omx-rpi --enable-decoder=h264_mmal --enable-encoder=h264_omx --enable-encoder=h264_omx --enable-libx264 --enable-libx265 --enable-nonfree --enable-gpl --enable-avresample
  libavutil      55. 34.100 / 55. 34.100
  libavcodec     57. 64.101 / 57. 64.101
  libavformat    57. 56.100 / 57. 56.100
  libavdevice    57.  1.100 / 57.  1.100
  libavfilter     6. 65.100 /  6. 65.100
  libavresample   3.  1.  0 /  3.  1.  0
  libswscale      4.  2.100 /  4.  2.100
  libswresample   2.  3.100 /  2.  3.100
  libpostproc    54.  1.100 / 54.  1.100
[mpeg2video @ 0x2def800] Invalid frame dimensions 0x0.
    Last message repeated 14 times
[mpegts @ 0x2deb2b0] PES packet size mismatch
    Last message repeated 2 times
Input #0, mpegts, from 'Prova.ts':
  Duration: 09:56:08.63, start: 117669.938122, bitrate: 6 kb/s
  Program 4005 
    Metadata:
      service_name    : Canale5
      service_provider: Mediaset
    Stream #0:0[0x64a]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv, top first), 720x576 [SAR 64:45 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x64b](ita): Audio: mp2 ([4][0][0][0] / 0x0004), 48000 Hz, stereo, s16p, 192 kb/s
    Stream #0:2[0x64c](eng): Audio: mp2 ([4][0][0][0] / 0x0004), 48000 Hz, stereo, s16p, 128 kb/s
    Stream #0:3[0x653](ita,ita): Subtitle: dvb_teletext ([6][0][0][0] / 0x0006)
[h264_omx @ 0x2e27e20] Using OMX.broadcom.video_encode
Output #0, matroska, to 'prova_sw_nodeinterlace.mkv':
  Metadata:
    encoder         : Lavf57.56.100
    Stream #0:0: Video: h264 (h264_omx) (H264 / 0x34363248), yuv420p, 720x576 [SAR 64:45 DAR 16:9], q=2-31, 128 kb/s, 25 fps, 1k tbn, 25 tbc
    Metadata:
      encoder         : Lavc57.64.101 h264_omx
    Stream #0:1(ita): Audio: aac (LC) ([255][0][0][0] / 0x00FF), 48000 Hz, stereo, fltp, 128 kb/s
    Metadata:
      encoder         : Lavc57.64.101 aac
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg2video (native) -> h264 (h264_omx))
  Stream #0:1 -> #0:1 (mp2 (native) -> aac (native))
Press [q] to stop, [?] for help
[mpegts @ 0x2deb2b0] PES packet size mismatchme=00:00:38.28 bitrate= 258.3kbits/s speed=1.72x    
[mpeg2video @ 0x2e303a0] ac-tex damaged at 35 33
[mpeg2video @ 0x2e303a0] Warning MVs not available
[mpeg2video @ 0x2e303a0] concealing 135 DC, 135 AC, 135 MV errors in P frame
frame=  961 fps= 42 q=-0.0 Lsize=    1242kB time=00:00:39.24 bitrate= 259.2kbits/s speed=1.73x    
video:605kB audio:614kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.860181%
[aac @ 0x2e29450] Qavg: 777.114

2) native(sw) -> deinterlace -> h264_omx


pi@raspberrypi:/media/tvrec $ ffmpeg -i Prova.ts -vf yadif -c:a aac -c:v h264_omx -sn -b:v 128k prova_sw_deinterlace.mkv
ffmpeg version 3.2.2 Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 4.9.2 (Raspbian 4.9.2-10)
  configuration: --prefix=/usr --enable-mmal --enable-omx --enable-omx-rpi --enable-decoder=h264_mmal --enable-encoder=h264_omx --enable-encoder=h264_omx --enable-libx264 --enable-libx265 --enable-nonfree --enable-gpl --enable-avresample
  libavutil      55. 34.100 / 55. 34.100
  libavcodec     57. 64.101 / 57. 64.101
  libavformat    57. 56.100 / 57. 56.100
  libavdevice    57.  1.100 / 57.  1.100
  libavfilter     6. 65.100 /  6. 65.100
  libavresample   3.  1.  0 /  3.  1.  0
  libswscale      4.  2.100 /  4.  2.100
  libswresample   2.  3.100 /  2.  3.100
  libpostproc    54.  1.100 / 54.  1.100
[mpeg2video @ 0x239d780] Invalid frame dimensions 0x0.
    Last message repeated 14 times
[mpegts @ 0x23992d0] PES packet size mismatch
    Last message repeated 2 times
Input #0, mpegts, from 'Prova.ts':
  Duration: 09:56:08.63, start: 117669.938122, bitrate: 6 kb/s
  Program 4005 
    Metadata:
      service_name    : Canale5
      service_provider: Mediaset
    Stream #0:0[0x64a]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv, top first), 720x576 [SAR 64:45 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x64b](ita): Audio: mp2 ([4][0][0][0] / 0x0004), 48000 Hz, stereo, s16p, 192 kb/s
    Stream #0:2[0x64c](eng): Audio: mp2 ([4][0][0][0] / 0x0004), 48000 Hz, stereo, s16p, 128 kb/s
    Stream #0:3[0x653](ita,ita): Subtitle: dvb_teletext ([6][0][0][0] / 0x0006)
[h264_omx @ 0x2423e40] Using OMX.broadcom.video_encode
Output #0, matroska, to 'prova_sw_deinterlace.mkv':
  Metadata:
    encoder         : Lavf57.56.100
    Stream #0:0: Video: h264 (h264_omx) (H264 / 0x34363248), yuv420p, 720x576 [SAR 64:45 DAR 16:9], q=2-31, 128 kb/s, 25 fps, 1k tbn, 25 tbc
    Metadata:
      encoder         : Lavc57.64.101 h264_omx
    Stream #0:1(ita): Audio: aac (LC) ([255][0][0][0] / 0x00FF), 48000 Hz, stereo, fltp, 128 kb/s
    Metadata:
      encoder         : Lavc57.64.101 aac
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg2video (native) -> h264 (h264_omx))
  Stream #0:1 -> #0:1 (mp2 (native) -> aac (native))
Press [q] to stop, [?] for help
[mpegts @ 0x23992d0] PES packet size mismatchme=00:00:38.88 bitrate= 258.0kbits/s speed=1.07x    
[mpeg2video @ 0x23a0c90] ac-tex damaged at 35 33
[mpeg2video @ 0x23a0c90] Warning MVs not available
[mpeg2video @ 0x23a0c90] concealing 135 DC, 135 AC, 135 MV errors in P frame
frame=  961 fps= 26 q=-0.0 Lsize=    1240kB time=00:00:39.24 bitrate= 258.9kbits/s speed=1.08x    
video:603kB audio:614kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.866869%
[aac @ 0x2425500] Qavg: 777.114

3) mpeg2_mmal(hw) -> no deinterlace -> h264_omx


pi@raspberrypi:/media/tvrec $ ffmpeg -vcodec mpeg2_mmal -i Prova.ts -c:a aac -c:v h264_omx -sn -b:v 128k prova2.mkv
ffmpeg version 3.2.2 Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 4.9.2 (Raspbian 4.9.2-10)
  configuration: --prefix=/usr --enable-mmal --enable-omx --enable-omx-rpi --enable-decoder=h264_mmal --enable-encoder=h264_omx --enable-encoder=h264_omx --enable-libx264 --enable-libx265 --enable-nonfree --enable-gpl --enable-avresample
  libavutil      55. 34.100 / 55. 34.100
  libavcodec     57. 64.101 / 57. 64.101
  libavformat    57. 56.100 / 57. 56.100
  libavdevice    57.  1.100 / 57.  1.100
  libavfilter     6. 65.100 /  6. 65.100
  libavresample   3.  1.  0 /  3.  1.  0
  libswscale      4.  2.100 /  4.  2.100
  libswresample   2.  3.100 /  2.  3.100
  libpostproc    54.  1.100 / 54.  1.100
[mpegts @ 0x153c2e0] PES packet size mismatch
    Last message repeated 2 times
Input #0, mpegts, from 'Prova.ts':
  Duration: 09:56:08.63, start: 117669.938122, bitrate: 6 kb/s
  Program 4005 
    Metadata:
      service_name    : Canale5
      service_provider: Mediaset
    Stream #0:0[0x64a]: Video: mpeg2video ([2][0][0][0] / 0x0002), yuv420p(top first), 320x240, 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x64b](ita): Audio: mp2 ([4][0][0][0] / 0x0004), 48000 Hz, stereo, s16p, 192 kb/s
    Stream #0:2[0x64c](eng): Audio: mp2 ([4][0][0][0] / 0x0004), 48000 Hz, stereo, s16p, 128 kb/s
    Stream #0:3[0x653](ita,ita): Subtitle: dvb_teletext ([6][0][0][0] / 0x0006)
File 'prova2.mkv' already exists. Overwrite ? [y/N] y
[h264_omx @ 0x1695980] Using OMX.broadcom.video_encode
Output #0, matroska, to 'prova2.mkv':
  Metadata:
    encoder         : Lavf57.56.100
    Stream #0:0: Video: h264 (h264_omx) (H264 / 0x34363248), yuv420p, 320x240, q=2-31, 128 kb/s, 25 fps, 1k tbn, 25 tbc
    Metadata:
      encoder         : Lavc57.64.101 h264_omx
    Stream #0:1(ita): Audio: aac (LC) ([255][0][0][0] / 0x00FF), 48000 Hz, stereo, fltp, 128 kb/s
    Metadata:
      encoder         : Lavc57.64.101 aac
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg2video (mpeg2_mmal) -> h264 (h264_omx))
  Stream #0:1 -> #0:1 (mp2 (native) -> aac (native))
Press [q] to stop, [?] for help
[mpeg2_mmal @ 0x15445e0] Changing output format.
Input stream #0:0 frame changed from size:320x240 fmt:yuv420p to size:720x576 fmt:yuv420p53x    
[mpegts @ 0x153c2e0] PES packet size mismatchme=00:00:38.84 bitrate= 258.0kbits/s speed=1.44x    
frame=  961 fps= 35 q=-0.0 Lsize=    1240kB time=00:00:39.24 bitrate= 258.9kbits/s speed=1.45x    
video:603kB audio:614kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.906249%
[aac @ 0x16bea00] Qavg: 777.114

4) mpeg2_mmal(hw) -> deinterlace -> h264_omx


pi@raspberrypi:/media/tvrec $ ffmpeg -vcodec mpeg2_mmal -i Prova.ts -vf yadif -c:a aac -c:v h264_omx -sn -b:v 128k prova2.mkv
ffmpeg version 3.2.2 Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 4.9.2 (Raspbian 4.9.2-10)
  configuration: --prefix=/usr --enable-mmal --enable-omx --enable-omx-rpi --enable-decoder=h264_mmal --enable-encoder=h264_omx --enable-encoder=h264_omx --enable-libx264 --enable-libx265 --enable-nonfree --enable-gpl --enable-avresample
  libavutil      55. 34.100 / 55. 34.100
  libavcodec     57. 64.101 / 57. 64.101
  libavformat    57. 56.100 / 57. 56.100
  libavdevice    57.  1.100 / 57.  1.100
  libavfilter     6. 65.100 /  6. 65.100
  libavresample   3.  1.  0 /  3.  1.  0
  libswscale      4.  2.100 /  4.  2.100
  libswresample   2.  3.100 /  2.  3.100
  libpostproc    54.  1.100 / 54.  1.100
[mpegts @ 0x185a300] PES packet size mismatch
    Last message repeated 2 times
Input #0, mpegts, from 'Prova.ts':
  Duration: 09:56:08.63, start: 117669.938122, bitrate: 6 kb/s
  Program 4005 
    Metadata:
      service_name    : Canale5
      service_provider: Mediaset
    Stream #0:0[0x64a]: Video: mpeg2video ([2][0][0][0] / 0x0002), yuv420p(top first), 320x240, 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x64b](ita): Audio: mp2 ([4][0][0][0] / 0x0004), 48000 Hz, stereo, s16p, 192 kb/s
    Stream #0:2[0x64c](eng): Audio: mp2 ([4][0][0][0] / 0x0004), 48000 Hz, stereo, s16p, 128 kb/s
    Stream #0:3[0x653](ita,ita): Subtitle: dvb_teletext ([6][0][0][0] / 0x0006)
File 'prova2.mkv' already exists. Overwrite ? [y/N] y
[h264_omx @ 0x19b4440] Using OMX.broadcom.video_encode
Output #0, matroska, to 'prova2.mkv':
  Metadata:
    encoder         : Lavf57.56.100
    Stream #0:0: Video: h264 (h264_omx) (H264 / 0x34363248), yuv420p, 320x240, q=2-31, 128 kb/s, 25 fps, 1k tbn, 25 tbc
    Metadata:
      encoder         : Lavc57.64.101 h264_omx
    Stream #0:1(ita): Audio: aac (LC) ([255][0][0][0] / 0x00FF), 48000 Hz, stereo, fltp, 128 kb/s
    Metadata:
      encoder         : Lavc57.64.101 aac
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg2video (mpeg2_mmal) -> h264 (h264_omx))
  Stream #0:1 -> #0:1 (mp2 (native) -> aac (native))
Press [q] to stop, [?] for help
[mpeg2_mmal @ 0x19ca8f0] Changing output format.
Input stream #0:0 frame changed from size:320x240 fmt:yuv420p to size:720x576 fmt:yuv420p48x    
[mpegts @ 0x185a300] PES packet size mismatchme=00:00:38.55 bitrate= 256.8kbits/s speed=0.987x    
frame=  961 fps= 24 q=-0.0 Lsize=    1240kB time=00:00:39.24 bitrate= 259.0kbits/s speed=0.989x    
video:603kB audio:614kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.905568%
[aac @ 0x19f05a0] Qavg: 777.114

So in summary

1) sw->hw no deinterlace: speed 1.73x
2) sw->hw deinterlace: speed 1.08x
3) hw->hw no deinterlace: speed 1.45x

4) hw->hw deinterlace: speed 0.97x

So, we can clearly see that the deinterlace put an huge impact on the performance, but also it seems that the hw decoding doesn't help. However looking better at the logs, you can see that the mpeg2_mmal seems to output a fixed 720x576 size despite of the input, that in this case is 320x240, so the encoder has more to do and also the output of the encoder is even scaled back to 320x240

So I have tried to put a scaling before the encoding:

5) mpeg2_mmal(hw) -> deinterlace -> scale-> h264_omx


pi@raspberrypi:/media/tvrec $ ffmpeg -vcodec mpeg2_mmal -i Prova.ts -vf yadif -vf scale=320:240 -c:a aac -c:v h264_omx -sn -b:v 128k prova_hw_deinterlace_scaled.mkv
ffmpeg version 3.2.2 Copyright (c) 2000-2016 the FFmpeg developers
  built with gcc 4.9.2 (Raspbian 4.9.2-10)
  configuration: --prefix=/usr --enable-mmal --enable-omx --enable-omx-rpi --enable-decoder=h264_mmal --enable-encoder=h264_omx --enable-encoder=h264_omx --enable-libx264 --enable-libx265 --enable-nonfree --enable-gpl --enable-avresample
  libavutil      55. 34.100 / 55. 34.100
  libavcodec     57. 64.101 / 57. 64.101
  libavformat    57. 56.100 / 57. 56.100
  libavdevice    57.  1.100 / 57.  1.100
  libavfilter     6. 65.100 /  6. 65.100
  libavresample   3.  1.  0 /  3.  1.  0
  libswscale      4.  2.100 /  4.  2.100
  libswresample   2.  3.100 /  2.  3.100
  libpostproc    54.  1.100 / 54.  1.100
[mpegts @ 0x31de310] PES packet size mismatch
    Last message repeated 2 times
Input #0, mpegts, from 'Prova.ts':
  Duration: 09:56:08.63, start: 117669.938122, bitrate: 6 kb/s
  Program 4005 
    Metadata:
      service_name    : Canale5
      service_provider: Mediaset
    Stream #0:0[0x64a]: Video: mpeg2video ([2][0][0][0] / 0x0002), yuv420p(top first), 320x240, 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x64b](ita): Audio: mp2 ([4][0][0][0] / 0x0004), 48000 Hz, stereo, s16p, 192 kb/s
    Stream #0:2[0x64c](eng): Audio: mp2 ([4][0][0][0] / 0x0004), 48000 Hz, stereo, s16p, 128 kb/s
    Stream #0:3[0x653](ita,ita): Subtitle: dvb_teletext ([6][0][0][0] / 0x0006)
[h264_omx @ 0x3338450] Using OMX.broadcom.video_encode
Output #0, matroska, to 'prova_hw_deinterlace_scaled.mkv':
  Metadata:
    encoder         : Lavf57.56.100
    Stream #0:0: Video: h264 (h264_omx) (H264 / 0x34363248), yuv420p, 320x240, q=2-31, 128 kb/s, 25 fps, 1k tbn, 25 tbc
    Metadata:
      encoder         : Lavc57.64.101 h264_omx
    Stream #0:1(ita): Audio: aac (LC) ([255][0][0][0] / 0x00FF), 48000 Hz, stereo, fltp, 128 kb/s
    Metadata:
      encoder         : Lavc57.64.101 aac
Stream mapping:
  Stream #0:0 -> #0:0 (mpeg2video (mpeg2_mmal) -> h264 (h264_omx))
  Stream #0:1 -> #0:1 (mp2 (native) -> aac (native))
Press [q] to stop, [?] for help
[mpeg2_mmal @ 0x334e900] Changing output format.
Input stream #0:0 frame changed from size:320x240 fmt:yuv420p to size:720x576 fmt:yuv420p
[mpegts @ 0x31de310] PES packet size mismatchme=00:00:38.57 bitrate= 256.6kbits/s speed=1.48x    
frame=  961 fps= 36 q=-0.0 Lsize=    1240kB time=00:00:39.24 bitrate= 258.9kbits/s speed=1.48x    
video:603kB audio:614kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 1.906249%
[aac @ 0x33745a0] Qavg: 777.114

It's quite better now, this speed, 1.48x is possible that makes the math to fill a real time transcoding on Rpi3, even if there is little margin.
Still it needs to be understood why the MMAL error when attempting to use this encoder in tvheadend, I'm building a custom ffmpeg for trying to understand what the error ""2" means.

Joseph ZACHARIE

Hi,

During my test, I've try to use a h264 source (From TVheadend et from a file).

The file was correclty compress with good performance.

From TVheadend they was some performance issue.

The MPG2 compression to h264_omx look like a software compression with your performance figures.

Try to compress direclty a .ts with ffmpeg to omx you will see the specificity of HW compression in the picture result.

Joseph

Antonio S

Hi Joseph
No, I believe that with my testing, transcoding with tvheadend was actually using hardware h264_omx for encodinf and software mpeg2 decoding. In fact if I look at the CPU usage of my ffmpeg test case #2 (sw dec, hw enc plus deinterlace) and case #5 (hw dev, he enc plus deinterlace and resize) I get 156% and 98%. The 156% is the CPU usage we get also with tvheadend
According also to the Kodi recomandation for MPEG2 playback, Rpi3 cannot sustain a stable, realtime, mpeg2 decoding.
So for having a chance of success, we need to use the GPU for the decoding also, and we still hit the MMAL error.

I have just completed my test here and what I get is:

tvheadend logs


2017-01-10 15:17:12.424 [  ERROR]:libav: MMAL error 2(ENOSPC) on control port
2017-01-10 15:17:12.444 [  ERROR]:libav: MMAL error 2(ENOSPC) on control port
2017-01-10 15:17:12.464 [  ERROR]:libav: MMAL error 2(ENOSPC) on control port
2017-01-10 15:17:12.484 [  ERROR]:libav: MMAL error 2(ENOSPC) on control port

sudo vcdbg log msg


338795.311: alloc_compact_internal(space=3291328, in_low_region=0, mode=7) failed
338815.315: alloc_compact_internal(space=3291328, in_low_region=0, mode=7) failed
338835.322: alloc_compact_internal(space=3291328, in_low_region=0, mode=7) failed
338855.326: alloc_compact_internal(space=3291328, in_low_region=0, mode=7) failed
338875.317: alloc_compact_internal(space=3291328, in_low_region=0, mode=7) failed
338895.315: alloc_compact_internal(space=3291328, in_low_region=0, mode=7) failed

I think we may miss some initialization in libav when using mpeg2_mmal

Antonio S

Hi
I have moved a little bit ahead. The MMAL 2 error was caused by the fact that mpeg2_mmal needs a valid width height in AVContex. Doing so, there is no error, the decoder kind of works (even if it return 0 but got_frame is 1). The problem is that the decoded video is weird, in the sense it drives h264_omx crazy (high Quantizer, no valid video out). Looking at the ffmpeg mmal init function, it seems that all the required parameters are initialized now, so I cannot understand what's the problem.
And the problem is that I cannot find anyone to support in this...

Morten Gade Sørensen

Really great that you're doing progress Antonio. Please let me know if there is anything I can do to test...

Have you tried transcoding/transrating when the original/source codec is H.264 already (h264_mmal)? Do you get the same behavior?

Sincerely,
Morten

Antonio S

No I didn't. Actually a guy on the ffmpeg-user mailing list replied. Tvheadend use an old, deprecated api for decode/encode, he recommend to use the new ones, that actually are the ones ffmpeg utils use. The modification should not be difficult, I will look into that

Antonio S

Hi all

I have made some more tests

1) mpeg2 -> decode (hw) -> deinterlace -> yuv (raw) output: 38fps avg / 40fps peak / 184% CPU
2) mpeg2 -> decode (sw> -> deinterlace -> yuv (raw) output: 35fps avg / 45fps peak / 184% CPU

3) yuv -> encode (hw) -> h264 output: : 70fps avg / 122fps peak / 25% CPU

So, this gives me a clearer picture on the process. As you can see the encoding performances are very very good, CPU usage is low and fps very high. The problem is with the decoding and +deinterlacing+. Here the things are quite strange, apparently there is no big difference between hw and sw decoding, but if you look at the differences between the average fps and peak fps I believe the problem here is the SD card write/read performance.
But this is not the point, the point is that the tvheadend transcoding engine is not pipelined. As far as I have seen in the code, it transcode in parallel audio/video/subtitles, BUT, the video transcoding itself is not parallel, it is serial. So, even if we accelerate to the hell, still the encoder waits resize, resize waits deinterlace, deinterlace waits decoder... This will endup no where.

So.... according to the number, it is possible to match realtime encoding on a Rpi3. Because, given 4 cores (to simplify I assume that the cores can run in parallel with no penalties and this is not the real case) with a +new+ transcoding architecture we can run

CPU[0]: Audio encoding
CPU[1]: Video decoding
CPU[2]: Filtering

CPU[3]: TVHeadend core, subtitles, encoding control, etc...

But to do so we have to modify the TV Headend transcoder to run in multithread. In general you cannot just pick a core for your thread, Linux will handle it, but without being multithread, even if Linux will try to optimize the code on the available cores, we will not make it.
I should have the skills to do so, I have actually forked the tvheadend repo, the problem is the time... I have two baby twins and a full time job, I'm really not sure how fast I could get some result, probably after a Rpi10 is out wil 16 cores@4Ghz in a credit card size, able to transcode 10 channels in parallel on one core, smoking a cigarete on the others... :)
If anyone would join the task, let's discuss it

Frank Schimmack

Hi all,
I am not an expert, so I am just thinking like a kind of "end user".

It is possible to watch DVDs (MPEG2) and deinterlaced on a raspi without high CPU load, or not?

So the Job: mpeg2 -> decode (hw) -> deinterlace -> yuv (raw) should be possible to do in my opinion. Perhaps I am missing something....

Antonio S

Your question is out of topic here. Anyhow, TV Headend is not a player, it is a DVB streaming backend. Your question should be placed to the frontend (player) you use. For instance, Kodi on Rpi can perfectly decode mpeg2 streaming, assuming that the mpeg2 decoder license is available

Alexander Meinke

Hi Antonio,

great to see you moving forward and I really appreciate it. I could imagine to spend some spare time working on it.

Well, talking about sw and hw en-/decoding is a bit "non-expert" as everything is running on hw :). The point is - as you already pointed out - the balancing, sequential vs. parallel. The RPi accommodate a NDA SoC with a very powerful GPU. So I believe the GPU has four cores (abbreviated from the name VideoCore IV). If you want less load on your CPU (which is doing more general tasks) you can offload most tasks to the highly parallel GPU. This is achieved byprogramming the CPU to offload tasks to the GPU. That means, there will be load on CPU - more or less for control, devices, dma etc. - and you will need or create something that will offload the tasks for you. For Multimedia there is OpenMaX (OMX) from the Krhonos Group that is also responsible for most Open* (OpenGL, OpenCL, OpenAL, OpenGL ES ....) standards.
To get the turn, there are OMX parts missing for ffmpeg and also the openmaxil library on raspbian.
MMAL is something ARM specific (MultiMediaAbstractionLayer I think) and therefore is CPU and not GPU related. Broadcom produces SoCs with IP from ARM and a custom GPU in the RPi case. There are also Broadcom SoCs for STBs that do "hw" transcoding running MIPS CPUs and custom GPUs.

It's a bit tricky as SoC can mean CPU + GPU but also only CPU and there are even more IP like Modems (3G, 4G ...), GPS and so on. The classic PC can do have all this too but it is commonly a separate device. Even PCs do have some special interfaces like vdpau, vaapi, nvenc.

The answer to Frank would be yes, but only with mpeg license and omxplayer. Sorry, I'am not really sure about omxplayer.

Antonio, what would you do or plan next?

noggin

MMAL can include hardware decoding - it's one of the two hardware acceleration options in Raspberry Pi Kodi. OMXPlayer and MMALPlayer are two different acceleration options. AIUI they access the VPU and GPU hardware via different software routes - but both use hardware acceleration. (De-interlacing was shifted between VPU and GPU I believe - but still doesn't hit the CPU)

Antonio S

Hi Alexander, on Monday I will write some details on the architecture I have in mind.

Antonio S

So, here I am :)

Lukely the current transcoding architecture, should make the modification for parallel transconding almost transparent to the rest of the TVHeadend system:


static void
transcoder_stream_video(transcoder_t *t, transcoder_stream_t *ts, th_pkt_t *pkt)

This function perform a complete transcode over one input video packed. The good thing is that the transcoded packet is sent via an async function called later in this function body:


if (got_output)
    send_video_packet(t, ts, pkt, &packet2, octx);

So, in my mind I have this idea: in the function transcoder_init_video together with the codec pickup we create also the threads we need to run in parallel. The transcoder_stream_video, instead of doing the transcoding, will sync these threads to perform their own transcoding step (or steps) iteration. The last stage (must be the encoder), if a packet is ready, calls send_video_packet. Every AVFrame output from one stage will be an AVFrame input to the next one. But there must be a pipeline of frames, since we run in parallel. So we need a number of AVFrame equal to the number of stages we run in parallel, and the stage process the older frame out from the previous, something like:

                
         | Stage0| Stage1| Stage2|
         | frame | frame | frame |
         | In|Out| In|Out| In|Out|     
In pkt 0 | 0 | 0 | x | x | x | x | None
In pkt 1 | 1 | 1 | 0 | 0 | x | x | None
In pkt 2 | 2 | 2 | 1 | 1 | 0 | 0 | Out pkt 0|
In pkt 3 | 0 | 0 | 2 | 2 | 1 | 1 | Out pkt 1|
In pkt 4 | 1 | 1 | 0 | 0 | 2 | 2 | Out pkt 2|
In pkt 5 | 2 | 2 | 1 | 1 | 0 | 0 | Out pkt 3|
In pkt 6 | 0 | 0 | 2 | 2 | 1 | 1 | Out pkt 4|
In pkt 7 | 1 | 1 | 0 | 0 | 2 | 2 | Out pkt 5|
In pkt 8 | 2 | 2 | 1 | 1 | 0 | 0 | Out pkt 6|
In pkt 9 | 0 | 0 | 2 | 2 | 1 | 1 | Out pkt 7|
In pkt a | 1 | 1 | 0 | 0 | 2 | 2 | Out pkt 8|
In pkt b | 2 | 2 | 1 | 1 | 0 | 0 | Out pkt 9|
In pkt c | 0 | 0 | 2 | 2 | 1 | 1 | Out pkt a|
In pkt d | 1 | 1 | 0 | 0 | 2 | 2 | Out pkt b|

So we have a frame pipeline of N stages and also a packet output latency of N, but it should not be a problem.

What do you think about?

« Previous Page Next Page »