Project

General

Profile

Bug #555

mkv produced by tvheadend are not playable by dedicated hardware players that support othrwyse most mkv

Added by Eric Valette over 11 years ago. Updated almost 11 years ago.

Status:
Fixed
Priority:
High
Assignee:
Category:
PVR / DVR
Target version:
Start date:
2011-06-07
Due date:
% Done:

60%

Estimated time:
Found in version:
git 5 June.
Affected Versions:

Description

I have two DLNA 1.5 DMP that refuse to read the tvheadend produced mkv:
1) Orange UHD86 STB <http://www.porciello.com/inventel/uhd86.html&gt;. The firmware I have now does support mkv and I do manage to stream to the box mkv I created form ARTE HD using ffmepg to manually convert mpeg2ts to mkv. The TVheadend generated mkv are unplayable,
2) LG DP1B <http://www.reviewbloger.com/lg-dp1b-media-player-review> that also plays the file I created or downloaded from internet

mkvinfo of a good file :

mkvinfo Tucker.mkv
+ EBML head |+ EBML version: 1 |+ EBML read version: 1 |+ EBML maximum ID length: 4 |+ EBML maximum size length: 8 |+ Doc type: matroska |+ Doc type version: 2 |+ Doc type read version: 2
+ Segment, size 5791460748 |+ Seek head (subentries will be skipped) |+ EbmlVoid (size: 143) |+ Segment information | + Timecode scale: 1000000 | + Title: Tucker | + Muxing application: Lavf52.56.1 | + Writing application: Lavf52.56.1 | + Segment UID: 0x4e 0x92 0x4c 0xea 0x4f 0x56 0xe8 0x5e 0x65 0xa4 0x1d 0x0c 0x55 0x66 0xe4 0x8f | + Duration: 6217.016s (01:43:37.016) |+ Segment tracks | + A track | + Track number: 1 | + Track UID: 1 | + Lacing flag: 0 | + Timecode scale: 1 | + Language: und | + Codec ID: V_MPEG4/ISO/AVC | + Track type: video | + Video track | + Pixel width: 1440 | + Pixel height: 1080 | + Display width: 1920 | + Display height: 1080 | + CodecPrivate, length 43 (h.264 profile: Main @L4.0) | + A track | + Track number: 2 | + Track UID: 2 | + Lacing flag: 0 | + Timecode scale: 1 | + Language: fra | + Codec ID: A_MPEG/L2 | + Track type: audio | + Audio track | + Channels: 2 | + Sampling frequency: 48000 |+ Cluster

mkvinfo of a bad file:

+ EBML head |+ EBML version: 1 |+ EBML read version: 1 |+ EBML maximum ID length: 4 |+ EBML maximum size length: 8 |+ Doc type: matroska |+ Doc type version: 2 |+ Doc type read version: 2
+ Segment, size 5217377982 |+ Seek head (subentries will be skipped) |+ EbmlVoid (size: 4025) |+ Segment information | + Timecode scale: 1000000 | + Muxing application: libebml v1.0.0 + libmatroska v1.0.0 | + Writing application: mkvmerge v4.4.0 ('Die Wiederkehr') built on Oct 31 2010 17:31:36 | + Duration: 5319.783s (01:28:39.783) | + Date: Mon Dec 20 20:18:55 2010 UTC | + Title: Sting Concert | + Segment UID: 0x51 0x39 0x3b 0x76 0x7d 0xd9 0x60 0x28 0x5b 0xb0 0x9f 0x3a 0xa0 0x13 0x0f 0x92 |+ Segment tracks | + A track | + Track number: 1 | + Track UID: 1 | + Track type: video | + Lacing flag: 0 | + MinCache: 1 | + Codec ID: V_MPEG4/ISO/AVC | + CodecPrivate, length 39 (h.264 profile: Unknown @L10.3) | + Default duration: 40.000ms (25.000 fps for a video track) | + Video track | + Pixel width: 1440 | + Pixel height: 1088 | + Display width: 30 | + Display height: 17 | + A track | + Track number: 2 | + Track UID: 2 | + Track type: audio | + Codec ID: A_MPEG/L2 | + Default duration: 24.000ms (41.667 fps for a video track) | + Language: fra | + Audio track | + Sampling frequency: 48000 | + Channels: 2 | + Content encodings | + Content encoding | + Content compression | + Algorithm: 3 (header removal) | + Settings: length 1, data: 0xff | + A track | + Track number: 3 | + Track UID: 3 | + Track type: audio | + Default flag: 0 | + Codec ID: A_MPEG/L2 | + Default duration: 24.000ms (41.667 fps for a video track) | + Language: deu | + Audio track | + Sampling frequency: 48000 | + Channels: 2 | + Content encodings | + Content encoding | + Content compression | + Algorithm: 3 (header removal) | + Settings: length 1, data: 0xff | + A track | + Track number: 4 | + Track UID: 4 | + Track type: subtitles | + Lacing flag: 0 | + Codec ID: S_DVBSUB | + CodecPrivate, length 4 | + Language: fra | + A track | + Track number: 5 | + Track UID: 5 | + Track type: subtitles | + Default flag: 0 | + Lacing flag: 0 | + Codec ID: S_DVBSUB | + CodecPrivate, length 4 | + Language: ger | + A track | + Track number: 6 | + Track UID: 6 | + Track type: subtitles | + Default flag: 0 | + Lacing flag: 0 | + Codec ID: S_DVBSUB | + CodecPrivate, length 4 | + Language: fra |+ EbmlVoid (size: 1226) |+ Cluster

Note that the producer is marked to be mkvmerge instead of tvheadend because I used it to remove start and end but the stream description is otherwyse unmodified. I checked with original tvheadend recordings.

Now the differences:
1) The fact that the tvheadend file has subtitle and additional audio does not matter. I removed them using mmg and it did not chnage anything
2)Video track | + Pixel width: 1440 | + Pixel height: 1088 <=== strange should be 1080 as on ffmpeg produced output
missing: | + Display width: 1920 | + Display height: 1080
Probably bad:
CodecPrivate, length 39 (h.264 profile: Unknown @L10.3). All file I have seen have something like
CodecPrivate, length 43 (h.264 profile: Main @L4.0) (or @L4.1) but not unknown. I can alter other flags using mmg
but not this one. Altering other flags do not fix the problem on at least one hardware decoder.


Files

codecs.pdf (122 KB) codecs.pdf Eric Valette, 2011-06-14 18:28
mkv_h264_fix_v2.patch (9.45 KB) mkv_h264_fix_v2.patch tvheadend H264 fix patrick boissonade, 2011-10-20 14:44
mkvmerge_support_bad_mkvh264.patch (12 KB) mkvmerge_support_bad_mkvh264.patch mkvmerge patch to fix bogus mkv patrick boissonade, 2011-10-20 14:44
tvheadend-mkvinfo.txt (1.96 KB) tvheadend-mkvinfo.txt William Velde, 2011-10-23 21:06
tvheadend-patched-mkvinfo.txt (2 KB) tvheadend-patched-mkvinfo.txt William Velde, 2011-10-23 21:06
test.ts (5.72 MB) test.ts William Velde, 2011-10-23 22:05

History

#1

Updated by Yura Scheglyuk over 11 years ago

There are a lot of bugreports about hardware players. I made workaround - see bug #392

#2

Updated by Eric Valette over 11 years ago

Yura Scheglyuk wrote:

There are a lot of bugreports about hardware players. I made workaround - see bug #392

Using mkmerge over the produced file to remux does not solve the problem for me. However I think it now proved that its a container bug because:
1) The device do play the same video and audio codec using the mepgs2ts stream directly
2) When converting the ts to mkv using ffmepg with copy option for video and audio codec (arte hd for exemple), the resulting file plays correctly,
3) the only things that differ between the tvheadend mkv and the ffmpeg mkv is only the mkv container.
4) there are at least two error in the container (1088 instead of 1080) and the CodecPrivate field

I will ask people if I can patch the two files (mmg can patch the stream description but not the CodecPrivate field).

#3

Updated by Eric Valette over 11 years ago

Attached is the description of the codecprivate field that is mandatory see <https://roundup.libav.org/issue314>

#4

Updated by Eric Valette over 11 years ago

case HB_VCODEC_X264:
77 track->codecID = MK_VCODEC_MP4AVC;
78 /* Taken from x264 muxers.c /
79 avcC_len = 5 + 1 + 2 + job->config.h264.sps_length + 1 + 2 + job->config.h264.pps_length;
80 avcC = malloc(avcC_len);
81 if (avcC == NULL)
82 return 1;
83
84 avcC0 = 1;
85 avcC1 = job
>config.h264.sps1; /
AVCProfileIndication /
86 avcC2 = job->config.h264.sps2; /
profile_compat /
87 avcC3 = job->config.h264.sps3; /
AVCLevelIndication */
88 avcC4 = 0xff; // nalu size length is four bytes
89 avcC5 = 0xe1; // one sps
90
91 avcC6 = job->config.h264.sps_length >> 8;
92 avcC7 = job->config.h264.sps_length;
93
94 memcpy(avcC+8, job->config.h264.sps, job->config.h264.sps_length);
95
96 avcC[8+job->config.h264.sps_length] = 1; // one pps
97 avcC[9+job->config.h264.sps_length] = job->config.h264.pps_length >> 8;
98 avcC[10+job->config.h264.sps_length] = job->config.h264.pps_length;
99
100 memcpy( avcC+11+job->config.h264.sps_length, job->config.h264.pps, job->config.h264.pps_length );
101 track->codecPrivate = avcC;

#5

Updated by Eric Valette over 11 years ago

Eric Valette - wrote:

> case HB_VCODEC_X264:
> 77                track->codecID = MK_VCODEC_MP4AVC;
> 78                /* Taken from x264 muxers.c */
> 79                avcC_len = 5 + 1 + 2 + job->config.h264.sps_length + 1 + 2 + job->config.h264.pps_length;
> 80                avcC = malloc(avcC_len);
> 81                if (avcC == NULL)
> 82                    return -1;
> 83    
> 84                avcC[0] = 1;
> 85                avcC[1] = job->config.h264.sps[1];      /* AVCProfileIndication */
> 86                avcC[2] = job->config.h264.sps[2];      /* profile_compat */
> 87                avcC[3] = job->config.h264.sps[3];      /* AVCLevelIndication */
> 88                avcC[4] = 0xff; // nalu size length is four bytes
> 89                avcC[5] = 0xe1; // one sps
> 90    
> 91                avcC[6] = job->config.h264.sps_length >> 8;
> 92                avcC[7] = job->config.h264.sps_length;
> 93    
> 94                memcpy(avcC+8, job->config.h264.sps, job->config.h264.sps_length);
> 95    
> 96                avcC[8+job->config.h264.sps_length] = 1; // one pps
> 97                avcC[9+job->config.h264.sps_length] = job->config.h264.pps_length >> 8;
> 98                avcC[10+job->config.h264.sps_length] = job->config.h264.pps_length;
> 99    
> 100                memcpy( avcC+11+job->config.h264.sps_length, job->config.h264.pps, job->config.h264.pps_length );
> 101                track->codecPrivate = avcC;

#6

Updated by Eric Valette over 11 years ago

BTW isom_write_avcc in avc.c has already the code in question. Just it not used...

#7

Updated by Andreas Smas over 11 years ago

  • Status changed from New to Accepted

Thanks for the troubleshooting.

There is some code in Tvheadend that should be able to do this conversion (from annex-b to AVC) but i never got it to work correctly.

And it seemed that most software based (ie. libav) players worked fine anyway. But apparently the hardware players have problems.
I'll try to finish the conversion code.

#8

Updated by Eric Valette over 11 years ago

Andreas Öman wrote:

Thanks for the troubleshooting.

There is some code in Tvheadend that should be able to do this conversion (from annex-b to AVC) but i never got it to work correctly.

And it seemed that most software based (ie. libav) players worked fine anyway. But apparently the hardware players have problems.
I'll try to finish the conversion code.

Yes as mentioned above the code isom_write_avcc in tvheadends avc.c does it. Note that the bug is worse because it will be complicated for people to remux HD streams:

if you try mkvexetract to try to remux with ffmpeg you get:

mkvextract tracks Retour-vers-le-futur-2.mkv 1:extracted.h264
Erreur : Track 1: NAL too big (because it look into the CodecPrivate to get the the NAL lenght) (checked with author)

and
ffmpeg barks about non monotonically increasing dts as already recorded in other bug reports.

#9

Updated by Eric Valette over 11 years ago

I gave up trying to rescue my files on Linux and switched to windows. I tested the Haali mkvspliter and indeed it does not recognize the video part of the mkv file (exactly for the same reason as in the ffmpeg bug reported in this thread). I also installed the (once free) cyberlink H264 and VC1 codec in addition to Haali codecs and tried to play various in Windows Media Player (XP). The mkv file that play correctly are the same that play correctly on my hardware player. So maybe this could help you for your tests.

#10

Updated by patrick boissonade about 11 years ago

Find attached two patches. First one is for tvheadend to generate correct mkv file (h264 codec private data and h264 nalu format). Second one is a patch for mkvmerge 5.0.1 enabling to fix bogus mkv generated by tvheadend. The second patch is a kludge that should be used only to fix tvheadend mkv NOT a mkvmerge enhancement!!! Do not submit it upstream.

#11

Updated by Hein Rigolo about 11 years ago

question: is the second patch only needed if you want to fix tvheadend .mkv's that are created before the first patch has been applied to the tvheadend source?

If so, then the second patch is only "temporarly" needed ....

#12

Updated by Eric Valette about 11 years ago

Hein Rigolo wrote:

question: is the second patch only needed if you want to fix tvheadend .mkv's that are created before the first patch has been applied to the tvheadend source?

Yes.

If so, then the second patch is only "temporarly" needed ....

But I think all people wanting to rescue their bogus files will appreciate it like I did.

Thanks again patrick.

#13

Updated by Hein Rigolo about 11 years ago

I am also happy that a source of problems with the mkv has been solved. I was just wondering what the status of the second patch was exactly.

Patrick,
Could you provide this first patch as a pull request on github? Then it can be included in the main brache quicker.

Hein

#14

Updated by Hein Rigolo about 11 years ago

  • Target version set to 2.13
  • % Done changed from 0 to 60
#15

Updated by William Velde about 11 years ago

Hello all,
i thought i had the same issue but not sure what is going on.

When i record hd video which i can watch using xbmc without problems i cannot record the stream and then playback the stream using mplayer or xbmc with audio.
So i patched tvheadend with the above patch to see what happens but tvheadend stil does not record my hd stream with audio.

it does fix the mkvinfo with a known h264 profile

Attached is the output from mkvinfo on the same recording (few minutes later) using tvheadend without the patch and then the output with the patched tvheadend.
Anyone an idea?

With kind regards,

William van de Velde

#16

Updated by William Velde about 11 years ago

William Velde wrote:

Hello all,
i thought i had the same issue but not sure what is going on.

When i record hd video which i can watch using xbmc without problems i cannot record the stream and then playback the stream using mplayer or xbmc with audio.
So i patched tvheadend with the above patch to see what happens but tvheadend stil does not record my hd stream with audio.

it does fix the mkvinfo with a known h264 profile

Attached is the output from mkvinfo on the same recording (few minutes later) using tvheadend without the patch and then the output with the patched tvheadend.
Anyone an idea?

With kind regards,

William van de Velde

Logfile from tvheadend displays:
Oct 23 20:49:55 backend tvheadend16807: dvr: # type lang resolution samplerate channels
Oct 23 20:49:55 backend tvheadend16807: dvr: 1 H264 1920 x 1088
Oct 23 20:49:55 backend tvheadend16807: dvr: 2 AC3 dut 96000 0 <disabled, no valid input>
Oct 23 20:49:55 backend tvheadend16807: dvr: 3 TELETEXT
Oct 23 20:49:55 backend tvheadend16807: dvr: 4 CA

#17

Updated by Eric Valette about 11 years ago

William Velde wrote:

Hello all,
i thought i had the same issue but not sure what is going on.

When i record hd video which i can watch using xbmc without problems i cannot record the stream and then playback the stream using mplayer or xbmc with audio.
So i patched tvheadend with the above patch to see what happens but tvheadend stil does not record my hd stream with audio.

it does fix the mkvinfo with a known h264 profile

Attached is the output from mkvinfo on the same recording (few minutes later) using tvheadend without the patch and then the output with the patched tvheadend.
Anyone an idea?

With kind regards,

William van de Velde

What Is see ffrom the mkvinfo file, is that without beeing patched the H264 field stream is incorrect and correct with patched version.
The patch itself does not change anything for audio stream compared to unpatched version.

Probably the audio stream in the streamed TS is recorded in a way that tvheadend does not recognize. Could you dump the TS stream using the TS patch available as a pull request or simply using tzap and use tsinfo on it.

#18

Updated by William Velde about 11 years ago

Hello Eric,

how can i dump the ts stream using the ts patch?
Could i otherwise use mplayer -dumpstream?

When i directly play the stream with mplayer i see the following info:
VIDEO H264 AUDIO A52 NO SUBS (yet)! PROGRAM N. 1
Stream not seekable!
FPS seems to be: 25.000000 ==========================================================================
Opening video decoder: [ffmpeg] FFmpeg's libavcodec codec family
Selected video codec: [ffh264] vfm: ffmpeg (FFmpeg H.264) ========================================================================== ==========================================================================
Opening audio decoder: [ffmpeg] FFmpeg/libavcodec audio decoders
AUDIO: 48000 Hz, 2 ch, s16le, 384.0 kbit/25.00% (ratio: 48000->192000)
Selected audio codec: [ffac3] afm: ffmpeg (FFmpeg AC-3) ==========================================================================
AO: [pulse] 48000Hz 2ch s16le (2 bytes per sample)
Starting playback...

And i do get the audio.
The text from above disabled, no valid input is in the file: dvr_rec.c but i'm not a programmer

With kind regards,

William van de Velde

#19

Updated by Eric Valette about 11 years ago

William Velde wrote:

Hello Eric,

how can i dump the ts stream using the ts patch?
Could i otherwise use mplayer -dumpstream?

As long as you take the initial stream and not something remuxed by tvheadend its OK.

When i directly play the stream with mplayer i see the following info:
VIDEO H264 AUDIO A52 NO SUBS (yet)! PROGRAM N. 1

What is the URL of the stream you give to mplayer? directly from /dev/dvb/adapter0/....

#20

Updated by William Velde about 11 years ago

No i use the url from tvheadend:
mplayer -dumpstream http://192.168.10.10:9981/playlist/channelid/44 -dumpfile test.ts

i attached the test.ts file

With kind regards,

William van de Velde

#21

Updated by Eric Valette about 11 years ago

William Velde wrote:

No i use the url from tvheadend:
mplayer -dumpstream http://192.168.10.10:9981/playlist/channelid/44 -dumpfile test.ts

i attached the test.ts file

The file is remuxed by tvheadend so that's not good. Plus there is a bug in EAC3 audio support via the web streaming API that is also corrected by the TS patch...

I propose we continue discution outside this bug because it has nothing to do with this bug.

#22

Updated by Eric Valette about 11 years ago

Eric Valette - wrote:

William Velde wrote:

No i use the url from tvheadend:
mplayer -dumpstream http://192.168.10.10:9981/playlist/channelid/44 -dumpfile test.ts

i attached the test.ts file

The file is remuxed by tvheadend so that's not good. Plus there is a bug in EAC3 audio support via the web streaming API that is also corrected by the TS patch...

I propose we continue discution outside this bug because it has nothing to do with this bug.

BTW: I have sound with the ts sample you provided.

#23

Updated by Hein Rigolo about 11 years ago

You can use tvheadends dump mux option to create a unmodified dump of your complete transportstream directly fro tvheadend.

#24

Updated by patrick boissonade about 11 years ago

Hein,

I can't provide the patch with a pull request on githubt now as I won't be available for the next 15 days. If you can integrate the patch in the main branch, I will appreciate; else I will do it when I come back.

#25

Updated by Andreas Smas about 11 years ago

  • Status changed from Accepted to Fixed

Fixed in commit:fb106fa1

I modified the patch a bit so it does not depend on libav

Thanks everyone involved.

#26

Updated by B. J. almost 11 years ago

Bug is back with ~pulse-4 provided by Dushmaniac. Changing output to ts in dvr-settings works very well. But matroska (mkv) never work again (H.264-stream is simply missing) - this has been fixed in previous versions (at least in ~pulse-3).

Hmm

Also available in: Atom PDF