Hardware transcoding

Added by Josu Lazkano about 1 year ago

Hello,

I have a Tvheadened server that feed 4 Libreelec frontends, it really works great. But I want to compress the video stream to some remote users.

Which is the best hardware to trancode the streams?

My CPU is "Intel(R) Core(TM)2 Duo CPU E8200 @ 2.66GHz", so I can not transcore it.

I will appreciate your help.

Best regards.


Replies (21)

RE: Hardware transcoding - Added by Robert Cameron about 1 year ago

The best would be the h264_vaapi that is now in ffmpeg. However, currently the vaapi hardware encoding support in Tvheadend is not working. Your hardware is sufficient, though. On a less powered Intel system, I am able to get ~5% CPU usage while transcoding 1080P MPEG2 into H264 using ffmpeg's h264_vaapi.

RE: Hardware transcoding - Added by Josu Lazkano about 1 year ago

Thanks Robert,

Please, could you show how to configure it?

I never use transcoding profiles in Tvheadend.

About vaapi. Isn't it related to GPU? My server has not any GPU card. It could be enough adding a specific card? It will be great for me.

Thanks for your help and best regards.

RE: Hardware transcoding - Added by Robert Cameron about 1 year ago

VAAPI is Intel's open/general hardware en-/decoding API that uses its HD Graphics system, which is the integrated GPU in Intel processors.

QSV (Quick Sync Video) is Intel's closed/proprietray hardware en-/decoding API; QSV requires additional kernel modules and specific SDKs from Intel, whereas VAAPI does not.

ffmpeg has supported QSV for quite some time, but getting it configured properly is no easy feat; starting in 3.0, ffmpeg also includes VAAPI support, which is much easier to use.

Unfortunately, the VAAPI encoders as compiled into Tvheadend are broken, so I can't really give any tutorials. You can use an IPTV pipe:// input and use ffmpeg to do your transcoding, but it's not particularly easy to set up, either. (It rarely works for me ... but it's not a huge issue for me because I don't use transcoding.)

RE: Hardware transcoding - Added by Josu Lazkano about 1 year ago

Thanks again Robert,

So I need to use ffmpeg trancoding pipe.

Anyone could help with a example? I use HD content in most of the channels.

Kind regards.

RE: Hardware transcoding - Added by Josu Lazkano about 1 year ago

I try "webtv-h264-aac-mpegts" in "Streaming profiles" and the server goues 100% CPU load.

I will appreciate your help.

Regards.

RE: Hardware transcoding - Added by Robert Cameron about 1 year ago

To use VAAPI with ffmpeg, check if your build of ffmpeg has vaapi support built in with : ffmpeg -codecs | grep vaapi.

Then, when you construct the command line for ffmpeg:

ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/render/D128 -hwaccel_output_format vaapi -i - -vf 'format=nv12|vaapi,hwupload' -c:v h264_vaapi -c:a copy pipe:1

That will take input from the standard input (such as for use with pipe://), use VAAPI to encode the video the H.264, but copy the audio stream as-is without re- or transcoding, and pass it back out to the standard input.

If you also need to resize the video, use the scale_vaapi video filter; just add it as an additional filter to the -vf option. So from the previous example, the video filter becomes: -vf 'format=nv12|vaapi,hwupload,scale_vaapi=w:h' where w and h are the width and height of your new video.

Hope this helps as a starting point.

As a note, you might need to ensure that the user running the command is a member of the video group, or which group has write access to /dev/dri in order to use VAAPI.

RE: Hardware transcoding - Added by Josu Lazkano about 1 year ago

Thanks again Robert,

I have no GPU card in the server, but I have this DRI devices:

# ls -l /dev/dri/
total 0
crw-rw---- 1 root video 226,  0 sep 18 00:11 card0
crw-rw---- 1 root video 226, 64 sep 18 00:11 controlD64
# lspci | grep VGA
00:02.0 VGA compatible controller: Intel Corporation 82G33/G31 Express Integrated Graphics Controller (rev 02)

I try to add a Stream Profile, but which option must I select here?

I really appreciate your help.

Kind regards.

RE: Hardware transcoding - Added by Robert Cameron about 1 year ago

Looks like you may not the necessary drivers, or your hardware may not support VAAPI.

As far as the streaming profile, if you add -f mpegts before the pipe:1 in the ffmpeg command line, it will create a MPEG-TS, and then just use the pass-through profile.

(Of course, this is for ffmpeg being used with the IPTV Network input, and a mux defined as a pipe:// resource.)

To see if your setup has the proper drivers and hardware, check what the output of vainfo is.

RE: Hardware transcoding - Added by Josu Lazkano about 1 year ago

Thanks again.

I will try to add a ATI GPU in ther server and test it.

Which GPU card could be good for that? It just wotk with ATI? Or could I use Nvidia?

Best regards.

RE: Hardware transcoding - Added by Christian Brauwers about 1 year ago

Have an Eye on this Board : https://www.supermicro.com/products/motherboard/Xeon/C236_C232/X11SSV-M4.cfm

The Intel Iris P580 GPU can transcode up to 15 HD Channel at time !

RE: Hardware transcoding - Added by Josu Lazkano about 1 year ago

Thanks Christian,

It cost 1000$. Too much for me.

It looks great, but 15 HD transcoding is too much, I have enough with 2-3 HD transcoding.

Is it possible adding a GPU card to actual server?

Regards.

RE: Hardware transcoding - Added by Robert Cameron about 1 year ago

Of course it is possible to add a GPU to a server and use it for transcoding. It all depends upon the drivers and tools you want to use, and what types of hardware acceleration they support.

RE: Hardware transcoding - Added by Josu Lazkano about 1 year ago

Thanks,

I want to know your experience about GPU cards. I need just trancode 2 HD streams.

Regards.

RE: Hardware transcoding - Added by Christian Brauwers about 1 year ago

couldn't you try Rpi 2/3 for transcoding? with new ffmpeg 3.1 HW encoding is supported ... cost round about 35 eur per Rpi

RE: Hardware transcoding - Added by Jonathan Thomson about 1 year ago

It still needs the TVHeadend developers to build support for it though :(

I think there's a way to use pipe to transcode but it sounds messy to me.

RE: Hardware transcoding - Added by Christian Brauwers about 1 year ago

why? you can build simple php script or something else to map this ...
i will get some new rpi3 these days so i have 2 rpi2 for testing ... will try with ffmpeg 3.1 on rpi2 ...

RE: Hardware transcoding - Added by Peter Tofte 10 months ago

Hi Robert

I have the ffmpeg command working with vaapi.

/usr/bin/ffmpeg -re -y -vaapi_device /dev/dri/card0 -hwaccel vaapi -hwaccel_output_format vaapi -i /home/hts/in.mkv -an -vf 'format=vaapi|nv12,hwupload' -c:v h264_vaapi -qp 20 -f mpegts -tune zerolatency -f mpegts out.ts

When I try to create a pipe in tvheadend (in the url text box) i put

/usr/bin/ffmpeg -re -y -vaapi_device /dev/dri/card0 -hwaccel vaapi -hwaccel_output_format vaapi -i /home/hts/in.mkv -an -vf 'format=vaapi|nv12,hwupload' -c:v h264_vaapi -qp 20 -f mpegts -tune zerolatency -f mpegts pipe:1

I get the following error message:
2017-01-21 09:48:31.621 spawn: Stream #0:0(eng): Video: h264 (High), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 29.97 fps, 29.97 tbr, 1k tbn, 59.94 tbc (default)
2017-01-21 09:48:31.621 spawn: [AVFilterGraph
0x5635f9d5ae00] No such filter: 'format=nv12|vaapi,hwupload'
2017-01-21 09:48:31.621 spawn: Error opening filters!
2017-01-21 09:48:31.623 iptv: stdin pipe unexpectedly closed: No [email protected]

I was able to make a pipe working with avconv and an online rstp stream, but not with vaapi. So pipe is fundamentally working in tvheadend. Question is, why is it not working with my vaapi command.

I also tried to create a #bin/bash script and calling this script from the pipe url in tvheadend, but no luck.

I am opening the iptv mux with vlc.

Any ideas? Did you get it to work?

-Peter

Robert Cameron wrote:

To use VAAPI with ffmpeg, check if your build of ffmpeg has vaapi support built in with : ffmpeg -codecs | grep vaapi.

Then, when you construct the command line for ffmpeg:

ffmpeg -hwaccel vaapi -hwaccel_device /dev/dri/render/D128 -hwaccel_output_format vaapi -i - -vf 'format=nv12|vaapi,hwupload' -c:v h264_vaapi -c:a copy pipe:1

That will take input from the standard input (such as for use with pipe://), use VAAPI to encode the video the H.264, but copy the audio stream as-is without re- or transcoding, and pass it back out to the standard input.

If you also need to resize the video, use the scale_vaapi video filter; just add it as an additional filter to the -vf option. So from the previous example, the video filter becomes: -vf 'format=nv12|vaapi,hwupload,scale_vaapi=w:h' where w and h are the width and height of your new video.

Hope this helps as a starting point.

As a note, you might need to ensure that the user running the command is a member of the video group, or which group has write access to /dev/dri in order to use VAAPI.

RE: Hardware transcoding - Added by Robert Cameron 10 months ago

Peter Tofte wrote:

Hi Robert

I have the ffmpeg command working with vaapi.

/usr/bin/ffmpeg -re -y -vaapi_device /dev/dri/card0 -hwaccel vaapi -hwaccel_output_format vaapi -i /home/hts/in.mkv -an -vf 'format=vaapi|nv12,hwupload' -c:v h264_vaapi -qp 20 -f mpegts -tune zerolatency -f mpegts out.ts

When I try to create a pipe in tvheadend (in the url text box) i put

/usr/bin/ffmpeg -re -y -vaapi_device /dev/dri/card0 -hwaccel vaapi -hwaccel_output_format vaapi -i /home/hts/in.mkv -an -vf 'format=vaapi|nv12,hwupload' -c:v h264_vaapi -qp 20 -f mpegts -tune zerolatency -f mpegts pipe:1

I get the following error message:
2017-01-21 09:48:31.621 spawn: Stream #0:0(eng): Video: h264 (High), yuv420p(tv, bt709, progressive), 1920x1080 [SAR 1:1 DAR 16:9], 29.97 fps, 29.97 tbr, 1k tbn, 59.94 tbc (default)
2017-01-21 09:48:31.621 spawn: [AVFilterGraph
0x5635f9d5ae00] No such filter: 'format=nv12|vaapi,hwupload'
2017-01-21 09:48:31.621 spawn: Error opening filters!
2017-01-21 09:48:31.623 iptv: stdin pipe unexpectedly closed: No [email protected]

I was able to make a pipe working with avconv and an online rstp stream, but not with vaapi. So pipe is fundamentally working in tvheadend. Question is, why is it not working with my vaapi command.

I also tried to create a #bin/bash script and calling this script from the pipe url in tvheadend, but no luck.

I am opening the iptv mux with vlc.

Any ideas? Did you get it to work?

You have a couple of problems with your options for ffmpeg.

  • -re – This option is used to slow the reading of on-disk files to real–time for live streaming output. When used with live stream inputs, it can cause problems when ffmpeg attempts to re-adjust the framerates. Do not use this option live/real–time inputs. (Looking at your example, it is hard to tell if you are using the /home/hts/in.mkv as a test input, or if you are really using a file for your IPTV mux. If you are really using an on-disk file, then keep the -re option.)
  • -vaapi_device /dev/dri/card0 – VA-API hardware encoding does not use the /dev/dri/card* device, but rather /dev/dri/render*. On my systems the device is /dev/dri/renderD128, but it may be different on yours.
  • -tune zerolatency – The -tune options are for using libx264 as your encoder, and have no effect when using h264_vaapi as your encoder.
  • -qp 20 – This is basically unnecessary, as the default -qp setting for h264_vaapi is already 20. The scale is the same as the -qp and -crf options for libx264: 0-51, with lower numbers having higher quality; sane values should remain between 18–28 if you are going to use this setting.

So, you may want to try this as your URL for the IPTV mux:

pipe:///usr/bin/ffmpeg -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -re -i /home/hts/in.mkv -vf format=nv12|vaapi,hwupload -an -c:v h264_vaapi -f mpegts -flags +global_header pipe:1

I have had problems with quotes in pipe:// URLs, so I tend to leave them out as they only seem to make things more difficult, regardless of how I escape them, and regardless of whether they are ' or ". Also, I have had problems with MPEG-TS headers needing to be present for Tvheadend, so I have added the -flags +global_header option to ensure those are generated and added into the MPEG-TS.

RE: Hardware transcoding - Added by Peter Tofte 10 months ago

Hi Robert,

that did the trick. I didn't notice the card0 issue, I know it has to be renderD128. The " ' "s are causing problems..I think its better to place the actual ffmpeg command in a shell script. I will ditch the "-re" option, it doesn't make sense for live tv anyway, as the in-pipe will only provide data real-time anyway - right?

I am now able to transcode live tv (yes the in.mkv was merely a test setup) using vaapi. Only thing missing now is a way to cast to chromecast. Unfortunately chromecast only supports webm (transcoding required, and i dont have hardware supporting VP8 encoding) or MP4 which is no good for live TV due to some MOOV thing in the file header.

-Peter

RE: Hardware transcoding - Added by Robert Cameron 10 months ago

Peter Tofte wrote:

Hi Robert,

that did the trick. I didn't notice the card0 issue, I know it has to be renderD128. The " ' "s are causing problems..I think its better to place the actual ffmpeg command in a shell script. I will ditch the "-re" option, it doesn't make sense for live tv anyway, as the in-pipe will only provide data real-time anyway - right?

I am now able to transcode live tv (yes the in.mkv was merely a test setup) using vaapi. Only thing missing now is a way to cast to chromecast. Unfortunately chromecast only supports webm (transcoding required, and i dont have hardware supporting VP8 encoding) or MP4 which is no good for live TV due to some MOOV thing in the file header.

-Peter

Since the file was merely a placeholder, then yes, ditch the -re option. As far as MP4 support, you can try thisj: you can change -f mpegts to -f mp4 -movflags isml. Generally streaming requires the MOOV to be moved to the beginning, and with ffmpeg this requires a second pass which will not work with live streaming (-movflags faststart), but the isml flag is supposed to be for a streaming site endpoint, so perhaps you can give that a try.

Otherwise, glad you finally got it working.

RE: Hardware transcoding - Added by Lalyu Lalev 6 months ago

Hello guys,
I know that the threat is old now but can you share some results? Also can this approach be used for live tv?

(1-21/21)