Feature #4572

OMX Support

Added by Mark Clarkstone 3 months ago. Updated about 1 month ago.

Status:AcceptedStart date:2017-09-05
Priority:NormalDue date:
Assignee:-% Done:

100%

Category:Transcoding
Target version:-

Description

I'm trying to build ffmpeg on the Pi using the following options.

sudo AUTOBUILD_CONFIGURE_EXTRA=--enable-omx\ --enable-ffmpeg_static\ --disable-bintray_cache\ --disable-libvpx_static\ --disable-libvpx\ --disable-nvenc ./Autobuild.sh -t raspbianstretch-armhf

This is to test out hardware transcoding on the Pi, unfortunately it fails to build as it always tries to build with nvenc and throws:-

ERROR: nvenc requested, but not all dependencies are satisfied: cuda

Even if you pass ---disable-nvenc.

Associated revisions

Revision e10c3008
Added by Mark Clarkstone 3 months ago

configure: only enable nvenc if asked, not by default, fixes #4572

Revision 3391e1d9
Added by Jaroslav Kysela 2 months ago

trancode: a blind shot to make functional rpi mmal decoders, issue #4572

Revision 3e2ac87e
Added by Jaroslav Kysela 2 months ago

transcode: try to fix the mmal decoder selection, issue #4572

Revision a62a38e9
Added by Mark Clarkstone 2 months ago

configure: only enable nvenc if asked, not by default, fixes #4572

History

#1 Updated by Klara Jansen 3 months ago

Same here...

#2 Updated by Mark Clarkstone 3 months ago

  • % Done changed from 0 to 50

Klara Jansen wrote:

Same here...

I've added a PR to "fix" this.

#3 Updated by Mark Clarkstone 3 months ago

  • Subject changed from GIT: Cannot disable nvenc to OMX Support

I've changed the title as so all omx related info is here.

Well, with that PR it builds and it looks extremely promising, but I get no video output. The good news is, tvh or the Pi doesn't crash.

Here is what I get when I try.

2017-09-05 22:32:49.179 subscription: 000D: "HTTP" subscribing on channel "BBC One E Mid", weight: 100, adapter: "IPTV", network: "tvh1", mux: "playlist - BBC One E Mid", service: "BBC ONE E Mid", profile="test", hostname="192.168.1.103", client="VLC/2.2.2 LibVLC/2.2.2" 
2017-09-05 22:32:50.162 libav: AVCodecContext: Invalid frame dimensions 0x0.
2017-09-05 22:32:50.170 libav: AVCodecContext: Invalid frame dimensions 0x0.
2017-09-05 22:32:50.199 libav: AVCodecContext: Invalid frame dimensions 0x0.
2017-09-05 22:32:50.240 libav: AVCodecContext: Invalid frame dimensions 0x0.
2017-09-05 22:32:50.270 libav: AVCodecContext: Invalid frame dimensions 0x0.
2017-09-05 22:32:50.302 libav: AVCodecContext: Invalid frame dimensions 0x0.
2017-09-05 22:32:50.374 libav: AVCodecContext: Invalid frame dimensions 0x0.
2017-09-05 22:32:50.414 libav: AVCodecContext: Invalid frame dimensions 0x0.
2017-09-05 22:32:50.454 libav: AVCodecContext: Invalid frame dimensions 0x0.
2017-09-05 22:32:50.496 libav: AVCodecContext: Invalid frame dimensions 0x0.
2017-09-05 22:32:50.504 libav: AVCodecContext: Invalid frame dimensions 0x0.
2017-09-05 22:32:50.534 libav: AVCodecContext: Invalid frame dimensions 0x0.
2017-09-05 22:32:50.606 libav: AVCodecContext: Invalid frame dimensions 0x0.
2017-09-05 22:32:50.613 libav: AVCodecContext: Invalid frame dimensions 0x0.
2017-09-05 22:32:50.646 libav: AVCodecContext: Invalid frame dimensions 0x0.
2017-09-05 22:32:57.174 libav: AVFormatContext: Could not find codec parameters for stream 1 (Audio: mp3 ([3][0][0][0] / 0x0003), 0 channels): unspecified frame size
Consider increasing the value for the 'analyzeduration' and 'probesize' options
2017-09-05 22:32:57.183 libav: AVFormatContext: Could not find codec parameters for stream 2 (Audio: mp3 ([3][0][0][0] / 0x0003), 0 channels): unspecified frame size
Consider increasing the value for the 'analyzeduration' and 'probesize' options
2017-09-05 22:32:57.193 libav: AVFormatContext: sample rate not set
2017-09-05 22:32:57.196 iptv: libav: Unable to write header

Seems like some values aren't being set?

Jaroslav, let me know if you want me to try any patches at all, I'm more than happy to try anything.

Here are a few links that may help.

#4 Updated by Mark Clarkstone 3 months ago

  • Tracker changed from Bug to Feature

#5 Updated by Mark Clarkstone 3 months ago

  • Category set to Transcoding

#6 Updated by Jaroslav Kysela 3 months ago

These errors are not from transcoder, but from IPTV libav input. There's trouble with the audio detection. Where you get the input stream? Could you run ffprobe on the input stream?

#7 Updated by Mark Clarkstone 3 months ago

  • Status changed from New to Fixed
  • % Done changed from 50 to 100

#8 Updated by Jaroslav Kysela 3 months ago

  • Status changed from Fixed to Accepted

#9 Updated by Mark Clarkstone 3 months ago

Jaroslav Kysela wrote:

These errors are not from transcoder, but from IPTV libav input. There's trouble with the audio detection. Where you get the input stream? Could you run ffprobe on the input stream?

It's an IPTV auto network using the playlist from my main tvh server.

The channel is bog standard DVB-T mpeg2/mp2 - no funky profiles, just pass.

Here's the probe.

[email protected]:~$ ffprobe http://192.168.1.3:9981/stream/channel/9f8e24ade33ee9c2622823db82c07cc0
ffprobe version 2.8.11-0ubuntu0.16.04.1 Copyright (c) 2007-2017 the FFmpeg developers
  built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.4) 20160609
  configuration: --prefix=/usr --extra-version=0ubuntu0.16.04.1 --build-suffix=-ffmpeg --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --cc=cc --cxx=g++ --enable-gpl --enable-shared --disable-stripping --disable-decoder=libopenjpeg --disable-decoder=libschroedinger --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmodplug --enable-libmp3lame --enable-libopenjpeg --enable-libopus --enable-libpulse --enable-librtmp --enable-libschroedinger --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxvid --enable-libzvbi --enable-openal --enable-opengl --enable-x11grab --enable-libdc1394 --enable-libiec61883 --enable-libzmq --enable-frei0r --enable-libx264 --enable-libopencv
  libavutil      54. 31.100 / 54. 31.100
  libavcodec     56. 60.100 / 56. 60.100
  libavformat    56. 40.101 / 56. 40.101
  libavdevice    56.  4.100 / 56.  4.100
  libavfilter     5. 40.101 /  5. 40.101
  libavresample   2.  1.  0 /  2.  1.  0
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  2.101 /  1.  2.101
  libpostproc    53.  3.100 / 53.  3.100
[mpeg2video @ 0x1df5ba0] Invalid frame dimensions 0x0.
    Last message repeated 7 times
[mpegts @ 0x1de7c40] Could not find codec parameters for stream 2 (Audio: mp3 ([3][0][0][0] / 0x0003), 0 channels, s16p): unspecified frame size
Consider increasing the value for the 'analyzeduration' and 'probesize' options
Input #0, mpegts, from 'http://192.168.1.3:9981/stream/channel/9f8e24ade33ee9c2622823db82c07cc0':
  Duration: N/A, start: 1007.201578, bitrate: N/A
  Program 1 
    Metadata:
      service_name    : BBC ONE E Mid
      service_provider: 
    Stream #0:0[0x65]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv), 704x576 [SAR 16:11 DAR 16:9], max. 15000 kb/s, 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x66](eng): Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, s16p, 256 kb/s
    Stream #0:2[0x6a](eng): Audio: mp3 ([3][0][0][0] / 0x0003), 0 channels, s16p (visual impaired)
    Stream #0:3[0x69](eng): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)

#10 Updated by Jaroslav Kysela 3 months ago

Could you try this? It's not a complete fix, it's just for testing:

diff --git a/src/input/mpegts/iptv/iptv_libav.c b/src/input/mpegts/iptv/iptv_libav.c
index bd14b380e..2389d279c 100644
--- a/src/input/mpegts/iptv/iptv_libav.c
+++ b/src/input/mpegts/iptv/iptv_libav.c
@@ -115,6 +115,8 @@ iptv_libav_thread(void *aux)

   for (i = 0; i < la->ictx->nb_streams; i++) {
     in_stream = la->ictx->streams[i];
+    if (in_stream->codec->frame_size == 0)
+      continue;
     out_stream = avformat_new_stream(la->octx, in_stream->codec->codec);
     if (out_stream == NULL) {
       tvherror(LS_IPTV, "libav: Failed allocating output stream");

#11 Updated by Mark Clarkstone 3 months ago

Jaroslav Kysela wrote:

Could you try this? It's not a complete fix, it's just for testing:

[...]

Sure, no problem, could take a little time to build though, will report results when possible.

#12 Updated by Mark Clarkstone 3 months ago

Update:

I'm now getting audio, but no video, using a profile that tries to use the omx hardware causes the cpu usage to sky rocket.

017-09-06 21:36:23.891 subscription: 0008: "HTTP" subscribing on channel "BBC One E Mid", weight: 100, adapter: "IPTV", network: "iptvbbc", mux: "115 - BBC One E Mid", service: "BBC ONE E Mid", profile="test", hostname="192.168.1.103", client="VLC/2.2.2 LibVLC/2.2.2" 
2017-09-06 21:36:24.880 transcode: 0007: 01:MPEG2VIDEO: ==> Using profile omx1
2017-09-06 21:36:24.880 transcode: 0007: 02:MPEG2AUDIO: ==> Using profile copy
2017-09-06 21:36:24.880 transcode: 0007: 03:MPEG2AUDIO: ==> Using profile copy
2017-09-06 21:36:24.880 transcode: 0007: 04:DVBSUB: ==> Filtered out
2017-09-06 21:36:25.585 libav: AVCodecContext: Invalid frame dimensions 0x0.
2017-09-06 21:36:25.671 libav: AVCodecContext: Invalid frame dimensions 0x0.
2017-09-06 21:36:25.672 libav: AVCodecContext: Invalid frame dimensions 0x0.
2017-09-06 21:36:25.680 libav: AVCodecContext: Invalid frame dimensions 0x0.
2017-09-06 21:36:25.680 libav: AVCodecContext: Invalid frame dimensions 0x0.
2017-09-06 21:36:25.743 libav: AVCodecContext: Invalid frame dimensions 0x0.
2017-09-06 21:36:25.815 libav: AVCodecContext: Invalid frame dimensions 0x0.
2017-09-06 21:36:25.878 libav: AVCodecContext: Invalid frame dimensions 0x0.
2017-09-06 21:36:25.879 libav: AVCodecContext: Invalid frame dimensions 0x0.
2017-09-06 21:36:25.912 libav: AVCodecContext: Invalid frame dimensions 0x0.
2017-09-06 21:36:25.913 libav: AVCodecContext: Invalid frame dimensions 0x0.
2017-09-06 21:36:26.173 libav: AVCodecContext: Using OMX.broadcom.video_encode

The good news is, it's at least trying to transcode.

Setting both profiles to copy & the usage stays normal, so I suspect it's not transcoding using the hardware but trying in software?

#13 Updated by Benjamin Challinor 3 months ago

I've noticed the same issue - transcoding a DVB-T UK stream results in no video, with the audio working fine & the same warnings in the log.

Turning trace on for libav/transcode gives a bit more:

Sep  8 17:10:56 tvheadend[6448]: libav: AVCodecContext: Skipping B slice due to open GOP
Sep  8 17:10:56 tvheadend[6448]: transcode: 0006: 01:H264: [mpeg2video => h264_omx]: opts: flags=+global_header,zerocopy=1,tvh_filter_deint=0,width=0,height=0,pix_fmt=0,bf=3,tvh_require_meta=1
Sep  8 17:10:56 tvheadend[6448]: libav: AVCodecContext: dimensions not set

The 'skipping B slice' warning is repeated a lot of times before presumably giving up without any successful determination of the stream parameters as width/height equate to 0. I'm guessing this is why no video output is observed - it does seem to be attempting to transcode something given CPU usage and a small reservation of GPU memory at the time, although with 0x0 dimensions its no surprise the video is missing (which it is - there is no video stream in the output at all)

#14 Updated by Klara Jansen 3 months ago

Mark Clarkstone wrote:

I've changed the title as so all omx related info is here.

Well, with that PR it builds and it looks extremely promising, but I get no video output. The good news is, tvh or the Pi doesn't crash.

I seem to be missing something as the build on my PI2 still crashes, using exactly the line as described in the first post. It throws the following error:
Markdown: docs/wizard/muxes.md
Markdown: docs/wizard/network.md
Markdown: docs/wizard/status.md
CC src/docs.o
CC build.o
CC timestamp.o
CC tvheadend
/usr/bin/ld: /usr/src/tvheadend/build.linux/ffmpeg/build/ffmpeg/lib/libtheoraenc.a(encode.o): relocation R_ARM_THM_MOVW_ABS_NC against `__stack_chk_guard' can not be used when making a shared object; recompile with -fPIC
/usr/src/tvheadend/build.linux/ffmpeg/build/ffmpeg/lib/libtheoraenc.a: error adding symbols: Bad value
collect2: error: ld returned 1 exit status
distcc[21353] ERROR: compile (null) on localhost failed
Makefile:684: recipe for target '/usr/src/tvheadend/build.linux/tvheadend' failed
make[3]: *** [/usr/src/tvheadend/build.linux/tvheadend] Error 1
make[3]: Leaving directory '/usr/src/tvheadend'
Makefile:99: recipe for target 'ffmpeg_all' failed
make[2]: *** [ffmpeg_all] Error 2
make[2]: Leaving directory '/usr/src/tvheadend'
debian/rules:11: recipe for target 'override_dh_auto_build' failed
make[1]: *** [override_dh_auto_build] Error 2
make[1]: Leaving directory '/usr/src/tvheadend'
debian/rules:5: recipe for target 'build' failed
make: *** [build] Error 2
dpkg-buildpackage: error: debian/rules build gave error exit status 2

I do have libtheora installed:
dpkg -l | grep libtheora
ii libtheora-dev:armhf 1.1.1+dfsg.1-6 armhf Theora Video Compression Codec (development files)
ii libtheora0:armhf 1.1.1+dfsg.1-6 armhf Theora Video Compression Codec

What's going wrong with my build?

#15 Updated by Jaroslav Kysela 3 months ago

@Klara: Try --disable-pie (configure).

#16 Updated by Klara Jansen 3 months ago

Jaroslav Kysela wrote:

@Klara: Try --disable-pie (configure).

Same result...

#17 Updated by Mark Clarkstone 3 months ago

Klara Jansen wrote:

Jaroslav Kysela wrote:

@Klara: Try --disable-pie (configure).

Same result...

This is what I used for configure during testing.

./configure --enable-omx --disable-libvpx --disable-libvpx_static --disable-libx265 --disable-libvpx_static --disable-libtheora --disable-libtheora_static --disable-libvorbis --disable-libvorbis_static --disable-libopus --disable-libopus_static

#18 Updated by Klara Jansen 3 months ago

Mark Clarkstone wrote:

This is what I used for configure during testing.
[...]

That's working! Thanks!

#19 Updated by Peter Leehrmund 2 months ago

Klara Jansen wrote:

Mark Clarkstone wrote:

This is what I used for configure during testing.
[...]

That's working! Thanks!

Sorry for the question, i am „new“.

Im searching for a solution to transcode one HD-Streams with GPU (OMX) on a Raspberry Pi3.
Is it possible with this solution so that i can choose a Stream-Profile (h264 omx)?

#20 Updated by Mark Clarkstone 2 months ago

Peter Leehrmund wrote:

Klara Jansen wrote:

Mark Clarkstone wrote:

This is what I used for configure during testing.
[...]

That's working! Thanks!

Sorry for the question, i am „new“.

Im searching for a solution to transcode one HD-Streams with GPU (OMX) on a Raspberry Pi3.

I'm only testing using a Pi1, unfortunately I don't think it's going to be possible to transcode in hardware on the Pi at the moment - at least not fully anyway.

My Pi isn't fast enough to do software decoding & tvh can't utilise the hardware decoder (yet?). I suspect it's possible using transcode spawn to an external app that can do both decode & encode. I have yet to try it, but I don't see why it won't work.

Is it possible with this solution so that i can choose a Stream-Profile (h264 omx)?

@Jaroslav
I was going to suggest you hide or disable the omx transcoder until it's possible to do both decode & encode in hardware as it's just going to confuse people. Up to you though.

#21 Updated by Jaroslav Kysela 2 months ago

Can someone with rpi show me a command-line using the ffmpeg command which is doing hardware transcoding from .ts to .mkv for example? We need probably mpeg2video and h264 examples.

#22 Updated by Mark Clarkstone 2 months ago

Jaroslav Kysela wrote:

Can someone with rpi show me a command-line using the ffmpeg command which is doing hardware transcoding from .ts to .mkv for example? We need probably mpeg2video and h264 examples.

Is this helpful? :)

#23 Updated by Jaroslav Kysela 2 months ago

Try v4.3-453-g3391e1d99 ...

#24 Updated by Jaroslav Kysela 2 months ago

One more fix in v4.3-460-g3e2ac87ea .

#25 Updated by C vH 2 months ago

@Mark Clarkstone
did it work for you ?
The log tells with >100% cpu usage at top

[WARNING]:transcode: 0006: 01:H264: [h264_mmal => h264_omx]: Invalid pts (2300202) <= last (2320402), dropping frame

As we build it differently it may be a build problem, but the ffmpeg we build works well with omx/mmal, for x86_64 it works well at Tvh with vaapi.

#26 Updated by Jaroslav Kysela 2 months ago

The scaling (height) and deinterlacer should be off (zero) for tests (no software video filters). Anyway, h264_omx encoder does not set pict_type in AVFrame, so tvh probably might not work for this reason.

#27 Updated by Mark Clarkstone 2 months ago

Just had a chance to test again & provide better logs this time!

Logs:
http://termbin.com/rrjx
http://termbin.com/fdbl

I also did the most basic tests I can think of..

Settings: Using the profile with copy & HW accel - nothing else is set.

  Container: Matroska (mkv)/av-lib 
  Result: Hardware not used (expected), partial garbled video, there's sound though 

  Container: Matroska (mkv)/built-in
  Result: Hardware not used (expected), fine, no issues at all

  Container: WEBM/built-in, MPEG-PS
  Result: Hardware not used (expected), inconclusive - doesn't support h264/aac/dvd muxer not found

  Container: MPEG-TS/av-lib
  Result: Hardware not used (expected), OK

  Container: MP4/av-lib
  Result: Error
  Log: 
    2017-09-16 05:46:54.793 libav: AVFormatContext: Application provided invalid, non monotonically increasing dts to muxer in stream 0: 23304 >= 23304
    2017-09-16 05:46:54.813 libav: Failed to write frame

Settings: Using the profile with copy & the bitrate set to 1000 

  Container: Matroska (mkv)/built-in,  Matroska (mkv)/av-lib, MP4/av-lib
  Result: Hardware used, just audio, attemps to transcode video, tvheadend hangs once playback is stopped - must be killed

  Container: MPEG-TS/av-lib
  Result: Hardware used, just audio - attemps to transcode video, _doesn't hang_
  Log for MP4/av-lib:
    2017-09-16 06:04:12.416 libav: AVCodecContext: Changing output format.
    2017-09-16 06:04:12.744 libav: AVCodecContext: Using OMX.broadcom.video_encode
    2017-09-16 06:04:16.227 libav: AVFormatContext: Using AVStream.codec to pass codec parameters to muxers is deprecated, use AVStream.codecpar instead.
    2017-09-16 06:04:16.237 libav: AVFormatContext: track 0: codec frame size is not set

Jaroslav, I hope this information helps!

#28 Updated by Mark Clarkstone 2 months ago

  • Status changed from Accepted to Fixed

#29 Updated by Jaroslav Kysela 2 months ago

  • Status changed from Fixed to Accepted

#30 Updated by Manfred Kreisl about 1 month ago

Mark Clarkstone wrote:

My Pi isn't fast enough to do software decoding & tvh can't utilise the hardware decoder (yet?). I suspect it's possible using transcode spawn to an external app that can do both decode & encode. I have yet to try it, but I don't see why it won't work.

Could anybody explain what --enable-omx does?

I would suppose that this option would use hardware encoding using ffmpeg's builtin support for omx.

But I was never able to get it work. Either there is no video stream in the output mkv (in that case i get

Oct 17 19:50:17 kmxbilr2 dvr:  1  H264                    1280x720    16:9                                 <disabled, no valid input>

in the log [1]) or the video stream is just copied into the mkv

Thanks

[1] http://paste.debian.net/991325/

Also available in: Atom PDF