Bug #4987

Unable to map channels for some HDHomerun Prime IPTV services

Added by Ted Romer 5 months ago. Updated 4 months ago.

Status:FixedStart date:2018-03-04
Priority:NormalDue date:
Assignee:-% Done:

100%

Category:IPTV
Target version:-
Found in version:4.2.5 Affected Versions:

Description

For 17 out of 106 channels tvheadend didn't successfully map channels to auto-created muxes. This is in the US, using Comcast, and an HDHomerun Prime CableCard. I am able to stream all the channels through other services, e.g. MythTV.

I've attached mux captures for 3 of the channels.

Happy to re-run with higher levels of debugging or try other diagnostic techniques.

NBCSNHD.ts (14.7 MB) Ted Romer, 2018-03-04 00:49

RTN5HD.ts (14.8 MB) Ted Romer, 2018-03-04 00:50

KCPQDT3.ts (5.75 MB) Ted Romer, 2018-03-04 00:50

ESPNHD.iptv.ts (12.4 MB) Ted Romer, 2018-03-06 06:44

OXYGNHP.iptv.ts (13.8 MB) Ted Romer, 2018-03-06 06:44

TBSHDP.iptv.ts (13.4 MB) Ted Romer, 2018-03-06 06:45

TNTPHD.iptv.ts (13.3 MB) Ted Romer, 2018-03-06 06:45

Associated revisions

Revision d3fc9696
Added by Jaroslav Kysela 4 months ago

dvb psi: improve section parser, fixes #4987

Revision 06613a2f
Added by Jaroslav Kysela 4 months ago

dvb psi: improve section parser, fixes #4987

History

#1 Updated by Jaroslav Kysela 5 months ago

It seems that there's non-standard PMT table with table ID 0xc0. According https://en.wikipedia.org/wiki/Program-specific_information, 0xc0 is used by DigiCipher II/ATSC/SCTE. We need to find the specification which describes this format.

#2 Updated by Ted Romer 5 months ago

There's this OpenCable ETV specification, "Enhanced TV Application Messaging Protocol 1.0" from CableLabs that looks like a match.

https://community.cablelabs.com/wiki/plugins/servlet/cablelabs/alfresco/download?id=bd611368-6f40-4d92-9dad-5d48116a79a4

Found via the ATSC code points registry spreadsheet at https://www.atsc.org/techdoc/code-point-registry/ -- "PMT Stream Type" and "References" tabs.

#3 Updated by Jaroslav Kysela 4 months ago

Unfortunately, I don't think that it's the proper specification. There is mentioned 0xc0 tag (not table id), but it's for the elementary stream.

#4 Updated by Robert Cameron 4 months ago

Jaroslav Kysela wrote:

It seems that there's non-standard PMT table with table ID 0xc0. According https://en.wikipedia.org/wiki/Program-specific_information, 0xc0 is used by DigiCipher II/ATSC/SCTE. We need to find the specification which describes this format.

According to SiliconDust's documentation, when the Prime decodes a stream, it rewrites the PMT. I don't know to what extent the PMT is rewritten, but I can try a capture to see if anyone else can make sense of it if desired.

For reference, the statement about rewriting the PMT is found in the TECH manual. While technically a different device, the firmware is only slightly different from a consumer Prime, and the hardware is identical, IIRC. The document is at http://www.silicondust.com/hdhomerun/hdhomerun_tech.pdf and the notations is at the bottom when referencing tuning by PID or vchannel. Since tuning by vchannel is the only means by which to achieve CableCARD decryption, my reading of this is to infer that the PMT is different than the unfiltered stream. Perhaps I'm wrong, though.

#5 Updated by Robert Cameron 4 months ago

I also found the same reference in their standard programming documentation on page 7 at https://www.silicondust.com/hdhomerun/hdhomerun_development.pdf with the same wording as the TECH documentation.

#6 Updated by Jaroslav Kysela 4 months ago

Unfortunately, the 0xc0 table id is not described in these documents - all ATSC/SCTE documents I found (including from 2017) refer this table as 'other standard'. I tried ffmpeg and it probes the streams manually (tries to decode PIDs using a blind scan). I believe that those are mangled MPEG-TS streams which are out of the specification. I would report the firmware problem.

#7 Updated by Ted Romer 4 months ago

I might be missing something, but all of the channels stream fine when I point VLC at the same M3U that I'm using for the IPTV automatic network, and MythTV + Kodi works as well, which makes me think it's possible to get TVHeadend to work with these streams.

Would it be helpful to provide some captures made directly via the Prime's IPTV URLs?

#8 Updated by Ted Romer 4 months ago

Observations: as with the report in the forums from Reggie Burnett https://tvheadend.org/boards/5/topics/31546?r=31710

(1) the services that fail to map to channels are exactly those that tvheadend marks as unencrypted.
(2) the results aren't consistent: if I delete and recreate a problematic mux+service it will sometimes succeed.

I've attached a few captures directly from the HDHR IPTV URLs for channels that resulted in problematic muxes, though given (2), it's hard to say whether they'll repro the problem.

#9 Updated by Jaroslav Kysela 4 months ago

Ted Romer wrote:

I might be missing something, but all of the channels stream fine when I point VLC at the same M3U that I'm using for the IPTV automatic network, and MythTV + Kodi works as well, which makes me think it's possible to get TVHeadend to work with these streams.

It's possible. As I wrote, appearently, players are doing extra lookup for the separate PIDs in the MPEG-TS stream, so they work without the correct PMT table. Unfortunately, TVH requires the valid PMT table for the automatic import and no-one created a PMT table editor yet. You may modify the PMT

Would it be helpful to provide some captures made directly via the Prime's IPTV URLs?

Just tried OXYGENHP.iptv.ts and there's valid PMT table (table id 0x02):

Sync-Byte 0x47: 71 (0x47)
Transport_error_indicator: 0 (0x00)  [= packet ok]
Payload_unit_start_indicator: 1 (0x01)  [= Packet data starts]
transport_priority: 0 (0x00)
PID: 74 (0x004a)  [= ]
transport_scrambling_control: 0 (0x00)  [= No scrambling of TS packet payload]
adaptation_field_control: 1 (0x01)  [= no adaptation_field, payload only]
continuity_counter: 7 (0x07)  [= (sequence ok)]
    Payload: (len: 184)
        ==> pointer_field: 0 (0x00)
        ==> Section table: 2 (0x02)  [= Program Map Table (PMT)]
    Data-Bytes:
          0000:  00 02 b0 83 00 07 c1 00  00 f5 39 f0 18 09 04 47   ..........9....G
          0010:  49 e1 1a 05 04 41 43 2d  33 05 04 45 41 43 33 05   I....AC-3..EAC3.
          0020:  04 43 55 45 49 1b f5 39  f0 10 2a 02 7e 1f 97 00   .CUEI..9..*.~...
          0030:  e9 08 0c 00 1f 41 85 07  d0 41 81 f5 3a f0 0f 81   .....A...A..:...
          0040:  07 06 38 0f ff 1f 00 3f  0a 04 65 6e 67 00 81 f5   ..8....?..eng...
          0050:  3b f0 0f 81 07 06 28 05  ff 1f 00 3f 0a 04 73 70   ;.....(....?..sp
          0060:  61 00 86 f5 3c f0 0f 8a  01 00 97 00 e9 08 0c 00   a...<...........
          0070:  1f 41 85 07 d0 41 c0 f5  3d f0 08 bf 06 49 6e 76   .A...A..=....Inv
          0080:  69 64 69 26 c3 28 a0 ff  ff ff ff ff ff ff ff ff   idi&.(..........
          0090:  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff   ................
          00a0:  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff   ................
          00b0:  ff ff ff ff ff ff ff ff                            ........

But in the 'non-working' streams, there's only some uknown table with table id 0xc0:

Sync-Byte 0x47: 71 (0x47)
Transport_error_indicator: 0 (0x00)  [= packet ok]
Payload_unit_start_indicator: 1 (0x01)  [= Packet data starts]
transport_priority: 0 (0x00)
PID: 67 (0x0043)  [= ]
transport_scrambling_control: 0 (0x00)  [= No scrambling of TS packet payload]
adaptation_field_control: 1 (0x01)  [= no adaptation_field, payload only]
continuity_counter: 12 (0x0c)  [= (sequence ok)]
    Payload: (len: 184)
        ==> pointer_field: 0 (0x00)
        ==> Section table: 192 (0xc0)  [= ATSC reserved]
    Data-Bytes:
          0000:  00 c0 00 15 00 00 05 00  52 00 00 00 00 00 00 01   ........R.......
          0010:  00 00 00 00 00 ae 2f fb  27 02 b0 9f 00 05 cf 00   ....../.'.......
          0020:  00 f2 a6 f0 18 09 04 47  49 e1 13 05 04 41 43 2d   .......GI....AC-
          0030:  33 05 04 45 41 43 33 05  04 43 55 45 49 1b f2 a6   3..EAC3..CUEI...
          0040:  f0 10 2a 02 7e 1f 97 00  e9 08 0c 00 1f 41 85 07   ..*.~........A..
          0050:  d0 41 81 f2 a7 f0 0f 81  07 06 38 0f ff 1f 00 3f   .A........8....?
          0060:  0a 04 65 6e 67 00 81 f2  a8 f0 10 81 08 06 28 01   ..eng.........(.
          0070:  ff ff 1f 00 3f 0a 04 73  70 61 00 86 f2 a9 f0 0f   ....?..spa......
          0080:  8a 01 00 97 00 e9 08 0c  00 1f 41 85 07 d0 41 c0   ..........A...A.
          0090:  f2 aa f0 08 05 04 45 54  56 31 a1 00 c0 f2 ab f0   ......ETV1......
          00a0:  09 05 04 45 54 56 31 a2  01 00 c0 f2 ac f0 08 bf   ...ETV1.........
          00b0:  06 49 6e 76 69 64 69 bf                            .Invidi.
Sync-Byte 0x47: 71 (0x47)
Transport_error_indicator: 0 (0x00)  [= packet ok]
Payload_unit_start_indicator: 0 (0x00)  [= Packet data continues]
transport_priority: 0 (0x00)
PID: 67 (0x0043)  [= ]
transport_scrambling_control: 0 (0x00)  [= No scrambling of TS packet payload]
adaptation_field_control: 1 (0x01)  [= no adaptation_field, payload only]
continuity_counter: 13 (0x0d)  [= (sequence ok)]
    Payload: (len: 184)
    Data-Bytes:
          0000:  37 54 67 ff ff ff ff ff  ff ff ff ff ff ff ff ff   7Tg.............
          0010:  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff   ................
          0020:  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff   ................
          0030:  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff   ................
          0040:  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff   ................
          0050:  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff   ................
          0060:  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff   ................
          0070:  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff   ................
          0080:  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff   ................
          0090:  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff   ................
          00a0:  ff ff ff ff ff ff ff ff  ff ff ff ff ff ff ff ff   ................
          00b0:  ff ff ff ff ff ff ff ff                            ........

But wait.. If I look further inside this, it appears that there are TWO tables inside one MPEG-TS packet 0xC0 and 0x02:

          0000:  00 c0 00 15 00 00 05 00  52 00 00 00 00 00 00 01   ........R.......
          0010:  00 00 00 00 00 ae 2f fb  27
          0010:                              02 b0 9f 00 05 cf 00   ....../.'.......
          0020:  00 f2 a6 f0 18 09 04 47  49 e1 13 05 04 41 43 2d   .......GI....AC-
          0030:  33 05 04 45 41 43 33 05  04 43 55 45 49 1b f2 a6   3..EAC3..CUEI...
          0040:  f0 10 2a 02 7e 1f 97 00  e9 08 0c 00 1f 41 85 07   ..*.~........A..
          0050:  d0 41 81 f2 a7 f0 0f 81  07 06 38 0f ff 1f 00 3f   .A........8....?
          0060:  0a 04 65 6e 67 00 81 f2  a8 f0 10 81 08 06 28 01   ..eng.........(.
          0070:  ff ff 1f 00 3f 0a 04 73  70 61 00 86 f2 a9 f0 0f   ....?..spa......
          0080:  8a 01 00 97 00 e9 08 0c  00 1f 41 85 07 d0 41 c0   ..........A...A.
          0090:  f2 aa f0 08 05 04 45 54  56 31 a1 00 c0 f2 ab f0   ......ETV1......
          00a0:  09 05 04 45 54 56 31 a2  01 00 c0 f2 ac f0 08 bf   ...ETV1.........
          00b0:  06 49 6e 76 69 64 69 bf                            .Invidi.

Going to look into this furhter...

#10 Updated by Jaroslav Kysela 4 months ago

  • Status changed from New to Fixed
  • % Done changed from 0 to 100

#11 Updated by Ted Romer 4 months ago

Tried it out, all of the services successfully map to channels and all of the channels stream properly. Thanks, and thanks for the quick turnaround!

Also available in: Atom PDF