Project

General

Profile

Bug #1592

MKV generated is still incorrect for hardware decoders (e.g panasonic TV)

Added by Eric Valette almost 7 years ago. Updated over 6 years ago.

Status:
Accepted
Priority:
Normal
Category:
Muxers
Target version:
-
Start date:
2013-02-03
Due date:
% Done:

0%

Estimated time:
Found in version:
3.3.338~gbb51160
Affected Versions:

Description

I have a brand new panasonic TV, with ethernet connexion and DLNA capabilities. I tested several material on the TV and they mostly worked. But none of the tvheadend recording were working although the TV supports mkv, H264, EAC3, ... Saving as TS and using mkvmerge to produce MKV make it work on TV. Same channel, same codec identifications by mediainfo. Firts MKV generated by mkmerge works, second not. The things I see is the

Height : 1 088 pixels
Original height : 1 080 pixels
Frame rate mode : Variable

Whereas working one has only

Height : 1 080 pixels
Frame rate mode : Constant

Can send sample of files.

Here are the complete mediainfo output:

mediainfo TF1-Le-transporteur.2013-02-03.20-50.mkv
General
Unique ID : 20373502504667733180652587223598795499 (0xF53CB3A0DBDB2F8C0BE7DB62D008AEB)
Complete name : TF1-Le-transporteur.2013-02-03.20-50.mkv
Format : Matroska
Format version : Version 4 / Version 2
File size : 1.64 GiB
Duration : 30mn 57s
Overall bit rate : 7 588 Kbps
Encoded date : UTC 2013-02-03 21:57:55
Writing application : mkvmerge v6.0.0 ('Coming Up For Air') built on Jan 21 2013 10:34:30
Writing library : libebml v1.3.0 + libmatroska v1.4.0

Video
ID : 2
Format : AVC
Format/Info : Advanced Video Codec
Format profile :
Format settings, CABAC : Yes
Format settings, ReFrames : 4 frames
Codec ID : V_MPEG4/ISO/AVC
Duration : 30mn 57s
Bit rate : 7 053 Kbps
Width : 1 920 pixels
Height : 1 080 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 25.000 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : MBAFF
Bits/(Pixel*Frame) : 0.136
Stream size : 1.53 GiB (93%)
Default : Yes
Forced : No
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709

Audio #1
ID : 1
Format : E-AC-3
Format/Info : Audio Coding 3
Format settings, Endianness : Big
Codec ID : A_EAC3
Duration : 30mn 57s
Bit rate mode : Constant
Bit rate : 128 Kbps
Channel(s) : 2 channels
Channel positions : Front: L R
Sampling rate : 48.0 KHz
Compression mode : Lossy
Delay relative to video : -119ms
Stream size : 28.3 MiB (2%)
Language : French
Default : Yes
Forced : No

Audio #2
ID : 3
Format : E-AC-3
Format/Info : Audio Coding 3
Format settings, Endianness : Big
Codec ID : A_EAC3
Duration : 30mn 57s
Bit rate mode : Constant
Bit rate : 128 Kbps
Channel(s) : 2 channels
Channel positions : Front: L R
Sampling rate : 48.0 KHz
Compression mode : Lossy
Delay relative to video : -101ms
Stream size : 28.3 MiB (2%)
Default : No
Forced : No

Audio #3
ID : 4
Format : E-AC-3
Format/Info : Audio Coding 3
Format settings, Endianness : Big
Codec ID : A_EAC3
Duration : 30mn 57s
Bit rate mode : Constant
Bit rate : 128 Kbps
Channel(s) : 2 channels
Channel positions : Front: L R
Sampling rate : 48.0 KHz
Compression mode : Lossy
Delay relative to video : -101ms
Stream size : 28.3 MiB (2%)
Default : No
Forced : No

[email protected]:/videoRecords# mediainfo TF1-Doc-Martin.2013-01-28.20-50.mkv
General
Unique ID : 206141404959851820191439781740831413314 (0x9B1563F51F5FF7478327902710119442)
Complete name : TF1-Doc-Martin.2013-01-28.20-50.mkv
Format : Matroska
Format version : Version 2
File size : 3.35 GiB
Duration : 1h 6mn
Overall bit rate : 7 162 Kbps
Movie name : Doc Martin
Writing application : Tvheadend 3.3.338~gbb51160
Writing library : Tvheadend Matroska muxer
Original source form : TV
DATE_BROADCASTED : 2013-01-28 20:50:00
TVCHANNEL : TF1
PART_NUMBER : 3

Video
ID : 2
Format : AVC
Format/Info : Advanced Video Codec
Format profile :
Format settings, CABAC : Yes
Format settings, ReFrames : 4 frames
Format settings, GOP : M=8, N=32
Codec ID : V_MPEG4/ISO/AVC
Bit rate : 6 635 Kbps
Width : 1 920 pixels
Height : 1 088 pixels
Original height : 1 080 pixels
Display aspect ratio : 16:9
Original display aspect ratio : 16:9
Frame rate mode : Variable
Original frame rate : 25.000 fps
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : MBAFF
Language : French
Default : Yes
Forced : No
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709

Audio #1
ID : 1
Format : E-AC-3
Format/Info : Audio Coding 3
Format settings, Endianness : Big
Codec ID : A_EAC3
Duration : 1h 6mn
Bit rate mode : Constant
Bit rate : 128 Kbps
Channel(s) : 2 channels
Channel positions : Front: L R
Sampling rate : 48.0 KHz
Compression mode : Lossy
Delay relative to video : 1mn 5s
Stream size : 61.3 MiB (2%)
Language : French
Default : Yes
Forced : No

Audio #2
ID : 3
Format : E-AC-3
Format/Info : Audio Coding 3
Format settings, Endianness : Big
Codec ID : A_EAC3
Duration : 1h 6mn
Bit rate mode : Constant
Bit rate : 128 Kbps
Channel(s) : 2 channels
Channel positions : Front: L R
Sampling rate : 48.0 KHz
Compression mode : Lossy
Delay relative to video : 1mn 5s
Stream size : 61.3 MiB (2%)
Language : qaa
Default : Yes
Forced : No

Audio #3
ID : 4
Format : E-AC-3
Format/Info : Audio Coding 3
Format settings, Endianness : Big
Codec ID : A_EAC3
Duration : 1h 6mn
Bit rate mode : Constant
Bit rate : 128 Kbps
Channel(s) : 2 channels
Channel positions : Front: L R
Sampling rate : 48.0 KHz
Compression mode : Lossy
Delay relative to video : 1mn 5s
Stream size : 61.3 MiB (2%)
Language : qaa
Default : Yes
Forced : No

Text #1
ID : 5
Format : S_DVBSUB
Codec ID : S_DVBSUB
Language : French
Default : Yes
Forced : No

Text #2
ID : 6
Format : S_DVBSUB
Codec ID : S_DVBSUB
Language : French
Default : Yes
Forced : No


Files

avc.c (6.6 KB) avc.c Modified avc.c file with fixed avc_find_startcode Eric Valette, 2013-02-12 21:07

History

#1

Updated by Eric Valette almost 7 years ago

So I've tried various things that may help others hunting the bug. I first edited video track header to describe the video height and width in the same way mkvmerge does => no change. I then used mkvmerge to extract H264 track as elementary stream from a tvheadend non working mkv and remuxed it using mkvmerge => works (no audio of course but will see that later). Seems to prove that its not the H264 recorded byte stream per se that is problematic but its description or the way the panasonic TV interprets it.

I really would like to get exactly the same ts stream remuxed both by tvheadend and mkvmerge to analyze headers. Looking at ffmpeg code that correspond to what is in the avc.c file, I saw some additional checks have been introduced to the routine that rewrites the sps/pps and that is used in the video header to provide codec private data. Will try to update the code to ffmpeg current and see if it changes anything. BTW now that it link with libav, ffmpeg routines could be reused as is...

#2

Updated by John Törnblom almost 7 years ago

I'm guessing the subtitles might have something to do with it.

#3

Updated by Eric Valette almost 7 years ago

John Törnblom wrote:

I'm guessing the subtitles might have something to do with it.

Can try to remove subtitle from faulty mkv to see if this changes something. Will do and report. Note that the tv works with blueray-rip that have subtitles.

#4

Updated by Eric Valette almost 7 years ago

Eric Valette - wrote:

John Törnblom wrote:

I'm guessing the subtitles might have something to do with it.

Can try to remove subtitle from faulty mkv to see if this changes something. Will do and report. Note that the tv works with blueray-rip that have subtitles.

Also the PS as subtitle too and works...

#5

Updated by Eric Valette almost 7 years ago

Next batch of tests:
1) Removing all video subtitle tracks doesn't change anything,
2) Putting the video track as first track in the file without subtitles idem
3) Extracting the video track as elementary stream, deleting the original video track and re-adding the extracted ones produce a working mkv,

So I really believes this has to do with the video track description. Furthermore I believe this is in the video segment description as remuxing the complete files via mkvmerge changes the clusters layout but not the track description and also produce a non working mkv.

#6

Updated by Eric Valette almost 7 years ago

After analysis by one of our H264 expert it turns out that, as expected, the CodecPrivateData field is indeed incorrect and that it is probably due to the avc_find_startcode routine. The original code in ffmpeg has now evolved. This field is used by hardware to preprogram the decoder before it finds the sps/pps info directly in the stream.

So I'm going to retake the current ffmpeg code (and add the credits in this file).

#7

Updated by Eric Valette almost 7 years ago

Even the improved current ffmpeg avc_find_startcode code is incorrect and lead to generation of wrong CodecPrivateData field (and wrong sps/pps info). We had to dump the original stream and simulate parsing done during mkv remuxing to find what is wrong.

If someone has a 100% proven avc_find_startcode routine in C, I'm all ears. In the meantime I will test a fix to original ffmpeg code (mkvmerge code is correct by highly depends on C++).

#8

Updated by Eric Valette almost 7 years ago

I have a working version tested with two hardware decoders (including the panasonic TV) that also works fine on PC with hardware decoding like vdpau, xvba, software decoding. Anyone having problem with a hardware decoder may test

#9

Updated by Adam Sutton almost 7 years ago

  • Status changed from New to Accepted

Eric,

Can you submit this via github PR and I'll see about getting it merged.

Ta
Adam

#10

Updated by Eric Valette almost 7 years ago

Adam Sutton wrote:

Eric,

Can you submit this via github PR and I'll see about getting it merged.

Yes I was just expecting some wider testing before pushing it. But I can do a PR for this single file for sure (any way 90% of it is ffmpeg code like previous version). I just fixed the location where ffmpeg code does not conform to standard!

#11

Updated by Adam Sutton almost 7 years ago

Sure, but if you think it fixes it and John gives the nod I'll merge it into master (post 3.4 branch) and we can use trial by crowd (it works quite well ;) and you'll at least get feedback if it breaks something else).

Adam

#12

Updated by Eric Valette almost 7 years ago

Adam Sutton wrote:

Sure, but if you think it fixes it and John gives the nod I'll merge it into master (post 3.4 branch) and we can use trial by crowd (it works quite well ;) and you'll at least get feedback if it breaks something else).

Adam

Works for me on all my hardware platform. Run it at home. Have quite a few but...

#13

Updated by Eric Valette almost 7 years ago

Adam Sutton wrote:

Sure, but if you think it fixes it and John gives the nod I'll merge it into master (post 3.4 branch) and we can use trial by crowd (it works quite well ;) and you'll at least get feedback if it breaks something else).

Adam

PR done.

#14

Updated by Adam Sutton over 6 years ago

I can't seem to find the PR? Eric do you still want this included?

Adam

#15

Updated by Eric Valette over 6 years ago

Adam Sutton wrote:

I can't seem to find the PR? Eric do you still want this included?

Well I have it in my tree so personnaly I don't care. The bug is definitively there and you do have the file. I did a PR but now I shall create a dedicated branch to reuse the PR and I do not knpw how to do it... I could return the question : do you want to have correct code?

#16

Updated by Rob vh over 6 years ago

I have seen 15-20% of the MKVs fail on a Samsung TV, so I am interested in inherent improvements of MKV build too.

#17

Updated by xraynorm - over 6 years ago

Just tested TVH recordings (.mkv) on a Panasonic TXL42E5B over DLNA and the all work fine (SD HD DVB-T)

HTS Tvheadend 3.3.491~g6990005~precise

#18

Updated by Eric Valette over 6 years ago

xraynorm - wrote:

Just tested TVH recordings (.mkv) on a Panasonic TXL42E5B over DLNA and the all work fine (SD HD DVB-T)

HTS Tvheadend 3.3.491~g6990005~precise

Do not understand what we shall understand. First Did you test with or without the patch applied? The bug is there but depends on the way the NALU padding is done and thus depend on the DVB source H264 stream itself. When you are in a situation wrere the compuation for avc_find_startcode is incorrect the PrivateCodecDate generation and PPS/SPS generation is bugged. The stream I have with upstream generation ARE bugged. I do not see it on many devices because some chipsets use them for decoding while others do not. The TV is just what enabled to discover the bug.

#19

Updated by xraynorm - over 6 years ago

No patch just standard install of 3.3.491.

UK Freeview.

I was not implying this was not an issue. Just that it works on my TV which is probably very similar to yours.

#20

Updated by Eric Valette over 6 years ago

xraynorm - wrote:

No patch just standard install of 3.3.491.

UK Freeview.

I was not implying this was not an issue. Just that it works on my TV which is probably very similar to yours.

Your model is a LCD model while I have a plasma 3D one and the price range is quite diffrent so I doubt it uses the same hardware components. Can send you a bugged mkv if for testing purpose if you want to discriminate chipsets vs dvb source.

The fact that ffmpeg did change the code for detection compared to the old one still used in tvheadend is just an hint that the bug exists. I just needed to change a for loop, to a while one to skip one more 0 padding compared to current ffmpeg routine.

#21

Updated by Eric Valette over 6 years ago

BTW : someone gave me the way to do a PR from an alaredy forked branch. And its not at all that simple when you main tree contains already many patches. Will try to send a new PR but I'm in the process of swapping my main PC so not before a few days... If someone as simple git commands to create a new branch to and existing tree with the patch already included I'm glad to try it.

Also available in: Atom PDF