TVHeadend Raspberry Pi and Transcoding

lord slash · 2014-11-04T22:19:08+00:00

Hi guys, I've setup a TVHeadend back end with my raspberry pi and raspbian, where I stream the italian tv to my home in Germany. The system works perfectly d...

Morten Gade Sørensen

Sounds like a really good plan! So this will actually be a general improvement that will benefit not only the Raspberry Pi builds?

Unfortunately I don't have the skills to help with the coding - I wish I could. Can you do it?

Thumbs up!

mpmc

Antonio S wrote:
> So, here I am :)
> Lukely the current transcoding architecture, should make the modification for parallel transconding almost transparent to the rest of the TVHeadend system:
>
> [...]
>
> This function perform a complete transcode over one input video packed. The good thing is that the transcoded packet is sent via an async function called later in this function body:
>
> [...]
>
> So, in my mind I have this idea: in the function transcoder_init_video together with the codec pickup we create also the threads we need to run in parallel. The transcoder_stream_video, instead of doing the transcoding, will sync these threads to perform their own transcoding step (or steps) iteration. The last stage (must be the encoder), if a packet is ready, calls send_video_packet. Every AVFrame output from one stage will be an AVFrame input to the next one. But there must be a pipeline of frames, since we run in parallel. So we need a number of AVFrame equal to the number of stages we run in parallel, and the stage process the older frame out from the previous, something like:
>
> [...]
>
> So we have a frame pipeline of N stages and also a packet output latency of N, but it should not be a problem.

> What do you think about?

This would be a welcomed feature for many that use Tvheadend on their Pis. If you're using the master branch please do consider making a PR :)

Alexander Meinke

Hey Antonio,

nice to hear from you. I had to think about it to get your idea and resulting benefits. The conclusion is, that I am not your man for this. Nevertheless, I have some questions.

Is it right that your going to create the threads ininit, sync them viastream and send them viasend after all frames has been transocded?
Do you think it will be possible to split transcoding between the threads with ffmpeg?

Would the architecture make it possible to pass the number of threads to ffmpeg?

So it looks like I can't imagine what is going to be done in parallel. There is the pipeline where every stage does it's own thing and outputs to its successor stage. Last stage will output the the full transcoded frame within a packet.

Sorry for delay,

Alex.

Antonio S

Alexander Meinke wrote:
> Hey Antonio,
>
> nice to hear from you. I had to think about it to get your idea and resulting benefits. The conclusion is, that I am not your man for this. Nevertheless, I have some questions.
>

> Is it right that your going to create the threads ininit, sync them viastream and send them viasend after all frames has been transocded?

Yes, this is the plan, however the _send will be called when the last stage (the encoder) has one packet to send

> Do you think it will be possible to split transcoding between the threads with ffmpeg?

> Would the architecture make it possible to pass the number of threads to ffmpeg?

It's possible and tvheadend apparently ask ffmpeg to run encoding in parallel with a number of threads equal to the CPU core detected. I have not checked if the information passed are consistent, but this is another topic. Parallelizing heavy computation stage over thread is of course good for multi core architecture, but still with the current transcoding code, every stage (decoding, deinterlacing, encoding) waits each other. With this architecture you will never get benefits from GPU accelerated capabilities
>
> So it looks like I can't imagine what is going to be done in parallel. There is the pipeline where every stage does it's own thing and outputs to its successor stage. Last stage will output the the full transcoded frame within a packet.

Let me try to be more simplier here. With the current code, as said every stage waits each other. Suppose we have a GPU that is capable to accelerate each stages, le'ts say that this acceleration won't consume any CPU power. But let's say that every stage gets 1/50 seconds for complete the job. So we would have

decoding (takes 1/50sec, no CPU) -> deinterlacing (takes 1/50sec, no CPU) -> encoding (takes 1/50sec, no CPU)

For completing a single frame trascoding we need 3/50 sec. If the source frame rate is 1/25 (which is 2/50 sec) we don't do transcoding in real time, even if we are using 0% of the CPU!!
Can you see now the difference between CPU usage and real time? ;)

With a pipelined transcoding we will have, after the third frame punt in the pipeline

1) decoding of frame 2 (takes 1/50sec, no CPU)
2) deinterlacing frame 1 (takes 1/50sec, no CPU)

3) encoding of frame 0 (takes 1/50sec, no CPU) ---> output

So we perform the entire trnascoding in 1/50 sec with no CPU usage and we run realtime :)

>
>
> Sorry for delay,
>

> Alex.

No problem. I'm done with the coding of the pipeline, debugging now. Next step is to implement specific handler for the decoding/filtering/enconding stage (trivial) and then put these handlers in task (not complicated). Stay tuned

Antonio S

Hi
pipeline is working now, I'm developing on branch: https://github.com/Menion2k/tvheadend/tree/rpi/encoding Still it run serial, so there is no improvement, but now I can move forward to the last two steps
Bye

Morten Gade Sørensen

Really great job. Antonio!
Can't wait to test it...

Broc Miller

This is fantastic! Will this benefit hardware that has no hardware acceleration?

Antonio S

Hi all
I have completed (and committed in my branch) the implementation of the pipeline
Unfortunately I'm in a dead path now

This are the figures from a Rpi3 with standard clocking

1) Decoding SW stage takes betwen 18ms and 35ms
2) Deinterlacing with no resize takes between 35ms and 50ms

3) Encoding HW takes between 8ms and 15ms

So the problem is with the deinterlacing. It's purely done in SW and Rpi3 is too slow for the task (we must stay 100% of the times above 40ms)
There are two problems here: I cannot make the hw accelerated decoder working, I don't know why, probably there is something needed in the codec open. This would lower a little bit the CPU load and give more power to the deinterlacing, BUT, I'm not sure that ffmpeg supports multhread deinterlacing...
If, and only IF, we can fix the HW mpeg decoder and when ffmpeg would implements some MMAL accelerated deinterlacing filter, we may have some success, but for the moment there is nothing we can do for it. I will write a question to libav mailing list for the mmal deinterlacing, chance to get some answer are really low...

Pablo Alone

Hi Antonio,

I know I speak for many reading this thread when I say that your efforts are very much appreciated.

It would indeed appear to be a dead end. Having said that, thank you for taking the time and effort to investigate this to the degree that you have.

Regards
AAA

Matt T

Agreed. Fantastic work on this, and thanks to everyone involved.

I wonder if all is not lost though.

1) Is it possible to not deinterlace and simply leave it for the player to do it? In Australia we broadcast interlaced h264, so it must be a thing...

2) I personally am interested not to much in a live stream but in the final file format. Perhaps this development could be leveraged to allow TVHeadend to record and compress to h264, if not in realtime? It would be nice to have it all built in without having to fuss about with post processing options etc (currently I use post processing to call a bash script which then calls ffmpeg)

Thanks again
Matt

noggin

Matt T wrote:
> Agreed. Fantastic work on this, and thanks to everyone involved.
>
> I wonder if all is not lost though.
>

> 1) Is it possible to not deinterlace and simply leave it for the player to do it? In Australia we broadcast interlaced h264, so it must be a thing...

It should be possible to do this IF you don't scale vertically - i.e. keep 1080i as 1080i or 576i as 576i and keep an identical field-rate.

It IS possible to scale in the interlaced domain too - by simply scaling fields, not frames, but you have to do it in the field domain, which may be tricky with some codecs. You don't get the same quality - but it was how early 'DVE's (Digital Video Effects devices that squeezed or zoomed and repositioned video digitally) worked.

If you try scaling in the frame domain without deinterlacing, you end up with all sorts of nasty artefacts (commonly seen as line-grouping, banding or 'wobbly edged motion) as field 1 motion leaks into field 2 and vice versa.

Morten Gade Sørensen

You've done an amazing job on improving the code!

I was wondering if de-interlacing is neccesarry when the source codec is H.264? So maybe we can still benefit from your improvements when the goal is to transrate (H.264->H.264 with lower bitrate)?

dorin ilut

A big thanks also from me. I would also like what Morten is stating. For me all HD programs are H.264 and those are the ones I would like to transcode. The SD channels are MPEG-2 and those are at an acceptable bandwidth rate.

Dorin

Antonio S

Matt T wrote:
> Agreed. Fantastic work on this, and thanks to everyone involved.
>
> I wonder if all is not lost though.
>
> 1) Is it possible to not deinterlace and simply leave it for the player to do it? In Australia we broadcast interlaced h264, so it must be a thing...
>
> 2) I personally am interested not to much in a live stream but in the final file format. Perhaps this development could be leveraged to allow TVHeadend to record and compress to h264, if not in realtime? It would be nice to have it all built in without having to fuss about with post processing options etc (currently I use post processing to call a bash script which then calls ffmpeg)
>
> Thanks again

> Matt

No, it's not possible, at least it is not possible with the OMX encoder of Rpi3, that woeks only for progressive inputs

Antonio S

dorin ilut wrote:
> A big thanks also from me. I would also like what Morten is stating. For me all HD programs are H.264 and those are the ones I would like to transcode. The SD channels are MPEG-2 and those are at an acceptable bandwidth rate.
>

> Dorin

Hi
I will give a try with the HD channels

noggin

Antonio S wrote:
> Matt T wrote:
> > Agreed. Fantastic work on this, and thanks to everyone involved.
> >
> > I wonder if all is not lost though.
> >
> > 1) Is it possible to not deinterlace and simply leave it for the player to do it? In Australia we broadcast interlaced h264, so it must be a thing...
> >
> > 2) I personally am interested not to much in a live stream but in the final file format. Perhaps this development could be leveraged to allow TVHeadend to record and compress to h264, if not in realtime? It would be nice to have it all built in without having to fuss about with post processing options etc (currently I use post processing to call a bash script which then calls ffmpeg)
> >
> > Thanks again
> > Matt
>

> No, it's not possible, at least it is not possible with the OMX encoder of Rpi3, that woeks only for progressive inputs

Ah - so that means 576/50i and 1080/50i (i.e. all UK TV) would have to be re-encoded either to 25p or 50p (both requiring a deinterlace)... What a pain. (I guess the encoder was designed primarily to compress camera output in a mobile phone-type scenario - which would be natively progressive)

dorin ilut

Hi Antonio,

Do not know if it is relevant but all my HD channels look like this. Check the attachment.

Thibaut monet

As anyone tried the omxtx transcoding option
https://github.com/dickontoo/omxtx

Following the author it should be able to transcode on the pi

https://www.raspberrypi.org/forums/viewtopic.php?f=70&t=25022

Matt T

What about using the hardware MMAL decoder to deinterlace? It is definitely capable of doing it, because my copy of Rasplex (based on Libreelec) does it. So the MMAL could decode, deinterlace, and then send it to the OMX for encoding.

Matt T

And come to think of it, it can probably hardware scale too. I can play a 576i mpg2 video upscaling to 1080p with CPU at like 3%. So the deinterlacing and upscaling must be happening in hardware.

« Previous Page Next Page »