Project

General

Profile

Changes requires to clarify some of the settings used in Codec Profiles

Added by Alin Gherghescu 26 days ago

The settings Profile Settings are applied to Decode side of the Codec. 
I think it make sense to rename the section from "Profile Settings" to "Profile/Decoder Settings" --> this way when user with check the hardware Acceleration will understand that this applies to Decoder (I know tooltip is providing this info ... but still).
Also all the rest of the sections are Encoder related ... should we add Encoder at the beginning of each line? : "Encoder Codec Settings", "Encoder Advanced Settings" and "Encoder Expert Settings"?

BTW: 
1. Decoder HW + Encoder HW --> works (with VAAPI)
2. Decoder SW + Encoder HW --> works (with VAAPI)
3. Decoder SW + Encoder SW --> works
4. Decoder HW + Encoder SW --> not working (with VAAPI)

Let me know your opinions.

Error:
2023-01-09 12:02:30.266 transcode: 0015: 01:MPEG2VIDEO: > Using profile webtv-h264
2023-01-09 12:02:30.267 transcode: 0015: 02:AC3: > Using profile webtv-aac
2023-01-09 12:02:31.195 libav: AVCodecContext: using SAR=40/33
2023-01-09 12:02:31.196 libav: AVCodecContext: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 AVX512
2023-01-09 12:02:31.197 libav: AVCodecContext: profile High, level 3.0, 4:2:0, 8-bit
2023-01-09 12:02:31.198 libav: TVHGraph: Impossible to convert between the formats supported by the filter 'Parsed_scale_vaapi_1' and the filter 'auto_scaler_0'
2023-01-09 12:02:31.198 transcode: 0015: 01:H264: [mpeg2video => libx264]: filters: failed to config filter graph


Replies (15)

RE: Changes requires to clarify some of the settings used in Codec Profiles - Added by saen acro 26 days ago

Because source is not some Capture card with send RAW data only Encoding is not enough ;)
Transcoding is other thing ;)
source > decoding > processing > encoding > output ~ transcoding

RE: Changes requires to clarify some of the settings used in Codec Profiles - Added by Alin Gherghescu 26 days ago

Alin Gherghescu wrote:

4. Decoder HW + Encoder SW --> not working (with VAAPI)

the fix is:
replace function _video_filters_get_filters() from src/transcoding/transcode/video.c with function below.

static int
_video_filters_get_filters(TVHContext *self, AVDictionary **opts, char **filters)
{
    char download[48];
    char deint[8];
    char hw_deint[64];
    char scale[24];
    char hw_scale[64];
    char upload[48];
    int ihw = _video_filters_hw_pix_fmt(self->iavctx->pix_fmt);
    int ohw = _video_filters_hw_pix_fmt(self->oavctx->pix_fmt);
    int filter_scale = (self->iavctx->height != self->oavctx->height);
    int filter_deint = 0, filter_download = 0, filter_upload = 0;

    if (tvh_context_get_int_opt(opts, "tvh_filter_deint", &filter_deint)) {
        return -1;
    }
    filter_download = (ihw && (!ohw || filter_scale || filter_deint)) ? 1 : 0;
    filter_upload = ((filter_download || !ihw) && ohw) ? 1 : 0;

    memset(deint, 0, sizeof(deint));
    memset(hw_deint, 0, sizeof(hw_deint));
#if ENABLE_HWACCELS
    if (filter_deint) {
        // when hwaccel is enabled we have two optios:
        if (ihw) {
            // hw deint
            hwaccels_get_deint_filter(self->iavctx, hw_deint, sizeof(hw_deint));
        }
        else {
            // sw deint
            if (str_snprintf(deint, sizeof(deint), "yadif")) {
                return -1;
            }
        }
    }
#else
    if (filter_deint) {
        if (str_snprintf(deint, sizeof(deint), "yadif")) {
            return -1;
        }
    }
#endif

    memset(scale, 0, sizeof(scale));
    memset(hw_scale, 0, sizeof(hw_scale));
#if ENABLE_HWACCELS
    if (filter_scale) {
        // when hwaccel is enabled we have two optios:
        if (ihw) {
            // hw scale
            hwaccels_get_scale_filter(self->iavctx, self->oavctx, hw_scale, sizeof(hw_scale));
        }
        else {
            // sw scale
            if (str_snprintf(scale, sizeof(scale), "scale=w=-2:h=%d",
                         self->oavctx->height)) {
                return -1;
            }
        }
    }
#else
    if (filter_scale) {
        if (str_snprintf(scale, sizeof(scale), "scale=w=-2:h=%d",
                         self->oavctx->height)) {
            return -1;
        }
    }
#endif

#if ENABLE_HWACCELS
    if (hw_deint[0] == '\0' && deint[0] == '\0' && hw_scale[0] == '\0' && scale[0] == '\0') {
        filter_download = filter_upload = 0;
    }
#else
    if (deint[0] == '\0' && scale[0] == '\0') {
        filter_download = filter_upload = 0;
    }
#endif

    memset(download, 0, sizeof(download));
    if (filter_download &&
        str_snprintf(download, sizeof(download), "hwdownload,format=pix_fmts=%s",
                     av_get_pix_fmt_name(self->iavctx->sw_pix_fmt))) {
        return -1;
    }

    memset(upload, 0, sizeof(upload));
    if (filter_upload &&
        str_snprintf(upload, sizeof(upload), "format=pix_fmts=%s|%s,hwupload",
                     av_get_pix_fmt_name(self->oavctx->sw_pix_fmt),
                     av_get_pix_fmt_name(self->oavctx->pix_fmt))) {
        return -1;
    }

    if (!(*filters = str_join(",", hw_deint, hw_scale, download, deint, scale, upload, NULL))) {
        return -1;
    }

    return 0;
}

RE: Changes requires to clarify some of the settings used in Codec Profiles - Added by Alin Gherghescu 26 days ago

saen acro wrote:

Because source is not some Capture card with send RAW data only Encoding is not enough ;)
Transcoding is other thing ;)
source > decoding > processing > encoding > output ~ transcoding

I don't understand your comment ... let me add a cartoon (to clarify).

info.png (30.5 KB) info.png

RE: Changes requires to clarify some of the settings used in Codec Profiles - Added by saen acro 26 days ago

Decoder and processing is combined in section
aka only for decoding is check box for HW decoding.
Resize and Deinterlace is processing.

It will be very good if there appear also "denoise" and "sharpen"

https://www.intel.com/content/www/us/en/developer/articles/technical/media-framework-upstream-patches-2022-q3.html

RE: Changes requires to clarify some of the settings used in Codec Profiles - Added by Kiril St. 26 days ago

Alin Gherghescu wrote:

Alin Gherghescu wrote:

4. Decoder HW + Encoder SW --> not working (with VAAPI)

the fix is:
replace function _video_filters_get_filters() from src/transcoding/transcode/video.c with function below.

[...]

Its possible in this code to add hw encoding???
We have some progress! Now i have picture but cpu is verry slow and do not encode properly! With checkbox, on or off I dont have any difeerence in encoding speed!

RE: Changes requires to clarify some of the settings used in Codec Profiles - Added by saen acro 26 days ago

Kiril St. wrote:

Alin Gherghescu wrote:

Alin Gherghescu wrote:

4. Decoder HW + Encoder SW --> not working (with VAAPI)

the fix is:
replace function _video_filters_get_filters() from src/transcoding/transcode/video.c with function below.

[...]

Its possible in this code to add hw encoding???
We have some progress! Now i have picture but cpu is verry slow and do not encode properly! With checkbox, on or off I dont have any difeerence in encoding speed!

Monitor your GPU with this
https://github.com/Syllo/nvtop
For intel GPU need kernel 5.19+ to see all telemetry, but and basic will be enough.

Its good idea for feature to have some GPU capability checker when TVH start
something similar to nv-video-info
with can post in log what can do GPU HW transcoding.

RE: Changes requires to clarify some of the settings used in Codec Profiles - Added by Alin Gherghescu 25 days ago

Kiril St. wrote:

Its possible in this code to add hw encoding???
We have some progress! Now i have picture but cpu is verry slow and do not encode properly! With checkbox, on or off I dont have any difeerence in encoding speed!

Can you provide: CPU (eventually what motherboard or PC you use), OS and what HW encoder you try to use (Nvidia, Intel AMD / nvec or vaapi). Also resolution IN, resolution OUT, channel is encoded by provider in ... (MPEG2, H264 ...) and you are trying to transcode that in .... (H264, VP8 ...). I hope you build TVH from Master (and not use some prebuild binary ... ) - please confirm how you installed tvh.

RE: Changes requires to clarify some of the settings used in Codec Profiles - Added by Alin Gherghescu 23 days ago

saen acro wrote:

It will be very good if there appear also "denoise" and "sharpen"

looking at:
https://ffmpeg.org/doxygen/4.0/index.html

I got to:
https://ffmpeg.org/doxygen/4.0/dir_07c4be17d34b4a7a38dbd1b8cb6b1e6c.html

But If you search for "VAAPI" you will see that "denoise" and "sharpen" are not available as filter for VAAPI (HW filter) ... there are other SW filters ... but I don't think is practical to try to use those in real time.

Even latest ffmpeg git doesn't seem to have:
https://github.com/FFmpeg/FFmpeg/tree/master/libavfilter
denoise_vaapi or sharpen_vaapi available.
Do you think I am missing something?

RE: Changes requires to clarify some of the settings used in Codec Profiles - Added by Alin Gherghescu 23 days ago

I found half of your request (denoise) ... but bad news: is only available in qsv (not in vaapi) :(
https://github.com/FFmpeg/FFmpeg/blob/master/libavfilter/vf_vpp_qsv.c
So ... there is a long path to that.

RE: Changes requires to clarify some of the settings used in Codec Profiles - Added by Alin Gherghescu 23 days ago

saen acro wrote:

You search them on wrong place ;)
https://github.com/intel/cartwheel-ffmpeg/tree/4.4/patches

Correct me if I am wrong: those are not patches applied to ffmpeg 4 that are already in ffmpeg 5?
I checked all patches but could not find anything related to "denoise" and "sharpen".
https://github.com/intel/cartwheel-ffmpeg/blob/4.4/patches/0074-avutils-hwcontext_qsv-set-the-source-device-in-qsv_d.patch
this one has some example filter with "unsharp_opencl" ... but this I think is inverse of "sharpen"

my comment previous post are from ffmpeg master ... long path to intercept :( .

ffmpeg compiled couple of months a go is confirming this:

RE: Changes requires to clarify some of the settings used in Codec Profiles - Added by saen acro 23 days ago

Alin Gherghescu wrote:

saen acro wrote:

You search them on wrong place ;)
https://github.com/intel/cartwheel-ffmpeg/tree/4.4/patches

Correct me if I am wrong: those are not patches applied to ffmpeg 4 that are already in ffmpeg 5?
I checked all patches but could not find anything related to "denoise" and "sharpen".
https://github.com/intel/cartwheel-ffmpeg/blob/4.4/patches/0074-avutils-hwcontext_qsv-set-the-source-device-in-qsv_d.patch
this one has some example filter with "unsharp_opencl" ... but this I think is inverse of "sharpen"

my comment previous post are from ffmpeg master ... long path to intercept :( .

With version of FFMPEG is used in TVH in the moment 4.0 or 5.0?
Change branch on top and will have them for 5.0 or 5.1 or master for current.

RE: Changes requires to clarify some of the settings used in Codec Profiles - Added by Alin Gherghescu 23 days ago

saen acro wrote:

Alin Gherghescu wrote:

saen acro wrote:

With version of FFMPEG is used in TVH in the moment 4.0 or 5.0?

bad news: we are on 4.4.

RE: Changes requires to clarify some of the settings used in Codec Profiles - Added by Alin Gherghescu 23 days ago

or maybe you are referring to sharpness (not 'sharpen').

RE: Changes requires to clarify some of the settings used in Codec Profiles - Added by saen acro 23 days ago

Alin Gherghescu wrote:

or maybe you are referring to sharpness (not 'sharpen').

synonym of the same word

intel page from upper post

Tested Features
Decode: AVC/H264, HEVC/H265 (8/10/12bit), AV1 (8/10bit), VP9 (8, 10, 12bit), VP8, JPEG/MJPEG, MPEG2, VC1
Encode: AV1 (ffmpeg-vaapi, ffmpeg-qsv and gst-msdk supported but not gst-vaapi), AVC/H264, HEVC/H265 (8/10bit), VP9 (8/10bit), VP8, JPEG/MJPEG, MPEG2
VPP : brightness/contrast/saturation/hue, csc, deinterlace, denoise, scale, sharpen, mirroring, rotation, transpose

Sharpening is needed for upscaling

    (1-15/15)