Hello,
I've struggling with this for a week and I finally need serious help.

Platform: unRAID, LinuxServer official docker.

Was running i5-9600t/UHD 630 iGPU with no issues for years. Updgraded hardware to 11700t/ UHD 750 and at the same time updated my unRAID software from 6.12 to 7.2.3. I don't think the host os software is the issue though.

vaapi_h264 works; vaapi_h265 pretends to be working (not seeing anything obvious in the normal log), but the video looks very scrambled. After my long running recording profiles failed, I tried adjusting many if not all options of the MKV/Built in.

Intel_GPU_top reports the video render is used, so don't think passing hardware is an issue (+ vaapi_h264 works fine). Libx265 works, but obviously that is not preferred as it uses CPU/soft.

Any suggestions what to look into next? What kind of debug log to pull?

`
Build: 4.3-2600_g5abbcda4d (2026-01-17T15:48:11+0000)
Configure arguments:
--disable-ffmpeg_static --disable-libfdkaac_static --disable-libtheora_static --disable-libopus_static --disable-libvorbis_static --disable-libvpx_static --disable-libx264_static --disable-libx265_static --disable-libfdkaac --enable-libopus --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --disable-avahi --disable-dbus_1 --disable-bintray_cache --disable-execinfo --disable-hdhomerun_static --enable-hdhomerun_client --enable-libav --enable-pngquant --enable-trace --enable-vaapi --infodir=/usr/share/info --localstatedir=/var --mandir=/usr/share/man --prefix=/usr --python=python3 --sysconfdir=/config

Compiler:
Using C compiler: cc
Using LD flags: -lhdhomerun -ldvbcsa
Build for arch: x86_64

Binaries:
Using PYTHON: python3
Using GZIP: gzip
Using BZIP2: bzip2

Options:
pie yes
ccdebug no
cardclient yes
cwc yes
cccam yes
capmt yes
constcw yes
linuxdvb yes
satip_server yes
satip_client yes
hdhomerun_client yes
hdhomerun_server yes
hdhomerun_static no
iptv yes
tsfile yes
dvbscan yes
timeshift yes
trace yes
avahi no
zlib yes
libav yes
ffmpeg_static no
libx264 yes
libx264_static no
libx265 yes
libx265_static no
libvpx yes
libvpx_static no
libtheora yes
libtheora_static no
libvorbis yes
libvorbis_static no
libfdkaac no
libfdkaac_static no
libopus yes
libopus_static no
nvenc no
vaapi yes
mmal no
omx no
inotify yes
epoll yes
pcre no
pcre2 yes
uriparser yes
ccache no
tvhcsa yes
bundle no
pngquant yes
kqueue no
dbus_1 no
android no
gtimer_check no
slow_memoryinfo no
libsystemd_daemon no
pcloud_cache yes
ddci yes
cclang_threadsan no
gperftools no
execinfo no
mmx yes
sse2 yes
W_unused_result yes
f_stack_protector yes
f_stack_protector_strong yes
f_stack_check yes
f_PIE yes
strlcat yes
strlcpy yes
fdatasync yes
getloadavg yes
atomic32 yes
atomic64 yes
atomic_time_t yes
atomic_ptr yes
bitops64 yes
qsort_r yes
time_ld yes
stime yes
gmtoff yes
recvmmsg yes
sendmmsg yes
gnu_libiconv yes
libiconv yes
ifnames yes
py_gzip yes
bin_pkg_config yes
bin_xgettext yes
bin_msgmerge yes
bin_gzip yes
bin_bzip2 yes
ssl yes
linuxdvbapi yes
linuxdvb_ca yes
libhdhomerun yes
upnp yes
hwaccels yes
inotify_h yes
inotify_init1 yes
dvbcsa yes
epoll_create1 yes
mpegts yes
mpegts_dvb yes

Packages:
openssl 3.3.5
zlib 1.3.1
libpcre2-8 10.43
liburiparser 0.9.8
libavfilter 7.110.100
libswresample 3.9.100
libswscale 5.9.100
libavformat 58.76.100
libavcodec 58.134.100
libavutil 56.70.100
libva 1.21.0
libva-drm 1.21.0

Installation paths:
Prefix: /usr
Binaries: ${prefix}/bin
Libraries: ${prefix}/lib
Data files: ${prefix}/share
Man pages: /usr/share/man
`