Bug #6228

h264_vaapi parameter 'quality' is not used properly

Added by Alin Gherghescu about 1 year ago. Updated 12 months ago.

Target version:
Start date:
Due date:
% Done:


Estimated time:
Found in version:
Affected Versions:


Reviewing the code related to vaapi I think I found a parameter that is mapped wrong.
The problem is in src/transcoding/codec/codecs/lib/vaapi.c

Based on description from tvh webpage (and form source code) 'quality' is an integer value between 0 and 8 that is used for Quality vs. Speed trade-off. See attachment for details.
If I understand the code, this value is passed to ffmpeg on line: 229 as "quality INT"

AV_DICT_SET_INT(opts, "quality", self->quality, 0);

Looking add ffmpeg documentation I see some conflict information:
9.14.8 Options

-compression_level integer
For lossy, this is a quality/speed tradeoff. Higher values give better quality for a given size at the cost of increased encoding time. For lossless, this is a size/speed tradeoff. Higher values give smaller size at the cost of increased encoding time. More specifically, it controls the number of extra algorithms and compression tools used, and varies the combination of these tools. This maps to the method option in libwebp. The valid range is 0 to 6. Default is 4.

-quality float
For lossy encoding, this controls image quality. For lossless encoding, this controls the effort and time spent in compression. Range is 0 to 100. Default is 75.

another source of info:

There is no complete analogue of the -preset option. The -compression_level option controls the local speed/quality tradeoff in the encoder (that is, the amount of effort expended on trying to get the best results from local choices like motion estimation and mode decision), using a nebulous per-device scale. The argument is a small integer, from 1 up to some limit dependent on the device (not more than 7) - higher values are faster / lower stream quality. Separately, some hardware (Intel gen9) supports a low-power mode with more restricted features. It is accessible via the -low_power option.

1. I think the ffmpeg parameter is compression_level (not quality)
2. Higher value is higher quality or lower quality? is not clear for me.
3. Valid range is 0 to 6 in one document and 1 to 7 in the other ... anyway is different from the source code (0 to 8). What should be the right range?


h254_vaapi_quality.png (151 KB) h254_vaapi_quality.png Alin Gherghescu, 2022-12-06 05:17
h264_quality.png (26.5 KB) h264_quality.png Alin Gherghescu, 2022-12-12 02:26
comp_level.png (28.7 KB) comp_level.png Alin Gherghescu, 2022-12-12 06:00



Updated by Alin Gherghescu about 1 year ago

based on answer from:
and based on info from:


Speed / quality tradeoff: higher values are faster / worse quality. 

I think we have the answer for:
2. Higher value is higher quality or lower quality? is not clear for me.
--> higher values are faster / worse quality.
3. Valid range is 0 to 6 in one document and 1 to 7 in the other ... anyway is different from the source code (0 to 8). What should be the right range?
--> 1 to 7.


Updated by Flole Systems about 1 year ago

As it's written it seems like higher means faster which also means lower quality.


Updated by Alin Gherghescu 12 months ago

After some investigation I can tell you that quality has an impact on the GPU utilization (Video):
GK41 1080p 25 h264 --> h264:
8 --> close 7
7 --> 15 - 18%
4 --> 16 - 19%
1 --> 21 - 24%
0 --> close 1

Probably there is some undocumented argument ... I don't know.
I used the following settings:


Updated by Alin Gherghescu 12 months ago

another test with a pentium 5405u (nuc) shows small difference between same values quality or compression_level. CPU loading being content dependent I selected one station that was relatively steady and resize to large value to get higher value for GPU usage.

mpeg2 480p 30 --> h264 2160p:
8 --> close 7
7 --> 22 - 27%
4 --> 22 - 31%
1 --> 31 - 38%
0 --> close 1

8 --> close 7
7 --> 24 - 28%
4 --> 25 - 31%
1 --> 31 - 37%
0 --> close 1

there seems to be an undecomented alias between quality and compression_level.


Updated by Alin Gherghescu 12 months ago

finally I found where was quality taken from:

alin@alin-cm8pcb4r:~$ ffmpeg -hide_banner -h encoder=h264_vaapi
Encoder h264_vaapi [H.264/AVC (VAAPI)]:
General capabilities: delay hardware
Threading capabilities: none
Supported pixel formats: vaapi_vld
h264_vaapi AVOptions:
-low_power <boolean> E..V..... Use low-power encoding mode (only available on some platforms; may not support all encoding features) (default false)
-idr_interval <int> E..V..... Distance (in I-frames) between IDR frames (from 0 to INT_MAX) (default 0)
-b_depth <int> E..V..... Maximum B-frame reference depth (from 1 to INT_MAX) (default 1)
-rc_mode <int> E..V..... Set rate control mode (from 0 to 6) (default auto)
auto E..V..... Choose mode automatically based on other parameters
CQP E..V..... Constant-quality
CBR E..V..... Constant-bitrate
VBR E..V..... Variable-bitrate
ICQ E..V..... Intelligent constant-quality
QVBR E..V..... Quality-defined variable-bitrate
AVBR E..V..... Average variable-bitrate
-qp <int> E..V..... Constant QP (for P-frames; scaled by qfactor/qoffset for I/B) (from 0 to 52) (default 0)
-quality <int> E..V..... Set encode quality (trades off against speed, higher is faster) (from -1 to INT_MAX) (default -1)
-coder <int> E..V..... Entropy coder type (from 0 to 1) (default cabac)
cavlc E..V.....
cabac E..V.....
vlc E..V.....
ac E..V.....
-aud <boolean> E..V..... Include AUD (default false)
-sei <flags> E..V..... Set SEI to include (default identifier+timing+recovery_point)
identifier E..V..... Include encoder version identifier
timing E..V..... Include timing parameters (buffering_period and pic_timing)
recovery_point E..V..... Include recovery points where appropriate
-profile <int> E..V..... Set profile (profile_idc and constraint_set*_flag) (from -99 to 65535) (default -99)
constrained_baseline E..V.....
main E..V.....
high E..V.....
-level <int> E..V..... Set level (level_idc) (from -99 to 255) (default -99)
1 E..V.....
1.1 E..V.....
1.2 E..V.....
1.3 E..V.....
2 E..V.....
2.1 E..V.....
2.2 E..V.....
3 E..V.....
3.1 E..V.....
3.2 E..V.....
4 E..V.....
4.1 E..V.....
4.2 E..V.....
5 E..V.....
5.1 E..V.....
5.2 E..V.....
6 E..V.....
6.1 E..V.....
6.2 E..V.....


Updated by Alin Gherghescu 12 months ago

Taking into account VAAPI is reporting 'quality' my proposal is to keep the same name for parameter.
But the range of this value seems to be only up to 7


Can we ask the person who implemented 0 --> 8 where did he found number '8'?


Updated by Flole Systems 12 months ago

Good luck with that, it's most likely someone at intel and the only way to get support from them is through their premier support system. To get access to that you need to fullfil certain criteria and so on.

You can try to open a GitHub issue but don't expect an answer there.

Also available in: Atom PDF