EPG for HD channels knows program times, but not titles or descriptions

Chris L · 2023-04-01T12:38:20+00:00

I have a Raspberry Pi 3 with a TV Hat installed, running a fully updated version of the latest Raspbian ("Bullseye") The TVHeadend version is the system pro...

Duncan Rix

Crikey, 5 pages lol.

I am also dyslexic and don't retain information all that well hence all ways going back to sites and "copy/past" lol.

I have had a skim through though and the interesting bit for me is that it's brought to my attention that Hauppage only list up to 20.04 and not 22.04 on there PPA instruction page so that probably enplanes my "unexpected kernel" message and perhaps my issue.

https://hauppauge.com/pages/support/support_linux.html

I'll have ago at reverting back to 20.04 next week and see if that resovles my issue. Unless of course any less time consuming ideas popup before then in this thread.

As for the other guy I probably have more experience with linux jugging buy what he's said but fault finding is not my strong point. I could do an Ubuntu walk through of the steps/sites I do to get things working which might help if he starts again.

thanks for the time so far.

DaveH

Duncan Rix wrote:
> There is a message during install of Hauppauge re the kernel not being the expected one and to reboot (All updates installed and system rebooted before Hauppauge install). Would that matter as it's all working apart from HD EPG?

For the future, if you ever get into a similar situation again, it's worth copying the error message and a bit before and after it somewhere, so that you can paste it into discussions like this one. That can be far more helpful than part-remembered bits.

Did you reboot AFTER installing the Hauppauge stuff as it suggested?

Duncan Rix

Yes your right sorry. I am rather gun ho!

And yes I did restart but the fact is that the default installed kernel is all ready up to date at this point so the more i think about it the more i think i should be using 20.04LTS.
Me being me did also try with the HWE kernel and that gave a very similar message. The revisions between the expected the the actual where very minor as in along the lines of

"install is 5.15.0-78 but expected 5.15.0-522304060810". I know I'm not really helping my self here am I lol. Interesting though I've just ssh'ed into my hts and the current kernel dose look like it's now the one it was asking for going buy the last part. It's currently 5.15.0-522304060810.

Any how I think I should probably try again with 20.04 and take note of any messages this time around so that people can better help me.

Many thanks for the time so far.

Jonas Lang

Duncan Rix wrote:
> Crikey, 5 pages lol.
>
> I am also dyslexic and don't retain information all that well hence all ways going back to sites and "copy/past" lol.
>
> I have had a skim through though and the interesting bit for me is that it's brought to my attention that Hauppage only list up to 20.04 and not 22.04 on there PPA instruction page so that probably enplanes my "unexpected kernel" message and perhaps my issue.
>
> https://hauppauge.com/pages/support/support_linux.html >
> I'll have ago at reverting back to 20.04 next week and see if that resovles my issue. Unless of course any less time consuming ideas popup before then in this thread.
>
> As for the other guy I probably have more experience with linux jugging buy what he's said but fault finding is not my strong point. I could do an Ubuntu walk through of the steps/sites I do to get things working which might help if he starts again.
>

> thanks for the time so far.

Unfortunately people for one reason or other people leave instructions non maintained and fail to use dates along the lines of “as of August 2023 this is the method used to install etc etc etc”.

As you stated the Hauppauge install instructions only mention support up to Ubuntu 21.10 https://www.hauppauge.com/pages/support/support_linux.html yet going to Ubuntu Launchpad support right up to Xenial is mentioned. https://launchpad.net/~b-rad/+archive/ubuntu/kernel+mediatree+hauppauge

I’d have tested this myself but don’t use a DVBT USB tuner and in this particular instance a Hauppauge one but do have Hauppauge PCI cards which I’ve no problem getting working.

The problem was compounded by the fact that the user I referenced in that post has little or no experience in either Linux nor TVH hence the five pages of posts.

You may be able to help that user and in the meantime solve your own issue. I’m assuming you are scanning T2 muxes successfully with your setup, something he hasn’t managed to achieve thus far. I’ll gladly help where I can in both cases.

Duncan Rix

STOP THE CLOCK I'VE GOT THE CRYSTAL!

I've done it I'm now on 20.04LTS. So many times I have done this when changing hardware around it's like muscle memory.

This time around I can report no kernel message or warnings of any kind.

So far not much in the way of EPG for HD but then I've only just enabled the uk freeview grabber and run it. I assume it's not instant so will see how it goes.
I did screen shots and such so perhaps I could do a quick and dirt how to for the other guy but I've just got to get my homeassistant VM back up first.

Chris L

Duncan Rix wrote:

> I have this same issue did you ever get it working?

No, sadly no progress here.

Delta Mike Charlie wrote:
> Could it be possible that the EPG transmitted has a language code set that your setup is not recognising? Maybe something as simple as 'ENG' vs 'eng'?
>

> You could enable tracing to a log file to see what values you are receiving.

I've tried enabling debug and tracing of

epg

and

epggrab

, and then triggering an OTA scan, but I don't see anything of interest in the log file - a couple of mentions of program events changing, but that's about it. There's so many trace options, I may well be missing the vital one.

Jonas Lang

Did you manage to tune T2 muxes.

Duncan Rix

This is where your going to loose me a little. A MUX is like the container for channels right?

Any how I mean I can watch HD channels dose that count?

When I install and setup the network I select my transmitter which is Sudbuy. I believe that got to do with MUX's no?

Duncan Rix

I'm getting the odd EPG entry for HD now which is different to when I was running 22.04 as I got nothing at all before.

But I do not get little blips in sound and slight delay in lip sinc I've noticed. From memory that's why I upgraded to 22.04 in the first place.

Those issues might be just on the desktop as I have an Libraelec running in the other room and I've not noticed it on there but I can only just hear it from where I am.

DeltaMikeCharlie

Chris L wrote:
> I've tried enabling debug and tracing of

epg

and

epggrab

Sorry for my delayed response, I live on the other side of the planet from you so I was sleeping.

I also did a quick internet search this morning and found that this issue has occurred with other types of PVR as well. There are suggestions that deleting the channel and rescanning has worked. Your scheduled recordings will be deleted and will have to be re-entered if you do that.

Is the service using DVB-T or DVB-T2? Apparently there have been some changes recently.

Do you have your 'OTA UK: Freeview' grabber enabled?

If you still want to try the trace: You need to trace 'tbl-eit' to see how the EPG gets processed.

You are obviously receiving some EPG data because you have all of the start dates/times, you just need to try to narrow down the possible reasons.

Once you have performed the OTA grab, load the log file (it could be fairly large) into your favourite text editor and search for the following: 'dtag 4D' and/or 'dtag 4E'

You should find something like this:

[  TRACE]:tbl-eit: svc='9Go! Sydney', ch='9Go! Sydney', eid=27142, tbl=50, running=0, start=2023-08-15;07:02:04(+1000), stop=2023-08-15;07:30:00(+1000), ebc=0x55da71434b90
[  TRACE]:tbl-eit: About to save rating label '0x55da70df2fc0'
[  TRACE]:tbl-eit: eit:  dtag 4D dlen 54
[  TRACE]:tbl-eit: 65 6E 67 1B 45 61 72 74 68 20 53 63 69 65 6E 63 eng.Earth Scienc   <=======
[  TRACE]:tbl-eit: 65 20 49 6E 76 65 73 74 69 67 61 74 6F 72 73 16 e Investigators.
[  TRACE]:tbl-eit: 41 20 6D 6F 75 6E 74 61 69 6E 20 74 6F 20 63 6C A mountain to cl
[  TRACE]:tbl-eit: 69 6D 62 2E 2E 2E                               imb...          
[  TRACE]:tbl-eit: eit:  dtag 4E dlen 220
[  TRACE]:tbl-eit: 00 65 6E 67 00 D6 4F 75 72 20 64 65 74 65 63 74 .eng..Our detect   <=======
[  TRACE]:tbl-eit: 69 76 65 73 20 68 61 76 65 20 6E 6F 74 69 63 65 ives have notice
[  TRACE]:tbl-eit: 64 20 74 68 61 74 20 74 68 65 20 45 61 72 74 68 d that the Earth
[  TRACE]:tbl-eit: 20 69 73 6E 27 74 20 66 6C 61 74 2E 20 42 75 74  isn't flat. But

In my example, you can see that the first three characters are 'eng'. A few lines before this, you can also see the channel name.

Go through your log looking for channels that have EPG and channels that don't and look for the language code. Are they different? Even a minor difference like 'ENG' vs 'eng' may be relevant.

Jonas Lang

Duncan Rix wrote:
> This is where your going to loose me a little. A MUX is like the container for channels right?
>
> Any how I mean I can watch HD channels dose that count?
>

> When I install and setup the network I select my transmitter which is Sudbuy. I believe that got to do with MUX's no?

Yes the mux does indeed contain the channels and yes T2 in this case refers to HD or H264.

Duncan Rix

Hi all.

Thank you all very much for all your suggestions and such. I'm a support worker and now don't have much time to put them into action.

But.... This evening I have switched kernel to HWE (still 20.04) and then issued a OTA grab and what do you know.

I now have HD EPG. It's all there now, Like Really!

Before I made the switch I looked and no EPG still so I guess it's solved for now.

Not going to have much time until mid week but If the other guy is still struggling buy then I will chime in and see if I can help.

thanks all enjoy what left of your weekends and again than you all.

Jonas Lang

You may want to let him know that you have resolved your issue and could possibly assist him so. https://tvheadend.org/d/8359-having-great-difficulty-getting-linux-to-install-and-see-win-tv-dualhd?page=5

Chris L

Delta Mike Charlie wrote:

> Sorry for my delayed response, I live on the other side of the planet from you so I was sleeping.

My turn to apologise - my email client decided to put your reply into my junk folder and it was only by random chance that I just noticed it.

> Is the service using DVB-T or DVB-T2? Apparently there have been some changes recently.

It's only HD channels that are affected. My understanding is that this means they are T2. The Edit UI for the HD mux in DVB Inputs -> Muxes lists the "Delivery system" as T2.

> Do you have your 'OTA UK: Freeview' grabber enabled?

I have 2 enabled: "OTA UK: Freeview" and "OTA EIT: EPG Grabber".

In DVB Inputs -> Muxes, I currently have most muxes set to "EIT: EPG Grabber", but the HD mux is set to "UK: Freeview". I think it was originally "EIT: EPG Grabber" like the others, but I probably changed it during my first round of trying to solve this.

> If you still want to try the trace: You need to trace 'tbl-eit' to see how the EPG gets processed.
>
> You are obviously receiving some EPG data because you have all of the start dates/times, you just need to try to narrow down the possible reasons.
>
> Once you have performed the OTA grab, load the log file (it could be fairly large) into your favourite text editor and search for the following: 'dtag 4D' and/or 'dtag 4E'
> ...

> Go through your log looking for channels that have EPG and channels that don't and look for the language code. Are they different? Even a minor difference like 'ENG' vs 'eng' may be relevant.

Searching for one of the channels that has both SD and HD variants, I can see:

Channel 4, standard definition


2023-08-26 08:51:42.940 [  TRACE]:tbl-eit: eit:  dtag 4D dlen 204
2023-08-26 08:51:42.940 [  TRACE]:tbl-eit: 65 6E 67 0C 54 68 65 20 53 69 6D 70 73 6F 6E 73 eng.The Simpsons
2023-08-26 08:51:42.940 [  TRACE]:tbl-eit: BB 54 72 69 6C 6F 67 79 20 6F 66 20 45 72 72 6F .Trilogy of Erro
2023-08-26 08:51:42.940 [  TRACE]:tbl-eit: 72 3A 20 48 6F 6D 65 72 20 67 65 74 73 20 68 69 r: Homer gets hi
2023-08-26 08:51:42.940 [  TRACE]:tbl-eit: 73 20 74 68 75 6D 62 20 73 6C 69 63 65 64 20 6F s thumb sliced o
2023-08-26 08:51:42.940 [  TRACE]:tbl-eit: 66 66 20 77 68 69 6C 65 20 74 72 79 69 6E 67 20 ff while trying 
2023-08-26 08:51:42.940 [  TRACE]:tbl-eit: 74 6F 20 73 74 65 61 6C 20 61 20 62 72 6F 77 6E to steal a brown
2023-08-26 08:51:42.940 [  TRACE]:tbl-eit: 69 65 2E 20 4C 69 73 61 20 6D 69 73 73 65 73 20 ie. Lisa misses 
2023-08-26 08:51:42.940 [  TRACE]:tbl-eit: 74 68 65 20 73 63 68 6F 6F 6C 20 62 75 73 2E 20 the school bus. 
2023-08-26 08:51:42.940 [  TRACE]:tbl-eit: 42 61 72 74 20 64 69 73 63 6F 76 65 72 73 20 46 Bart discovers F
2023-08-26 08:51:42.940 [  TRACE]:tbl-eit: 61 74 20 54 6F 6E 79 27 73 20 73 74 61 73 68 20 at Tony's stash 
2023-08-26 08:51:42.940 [  TRACE]:tbl-eit: 6F 66 20 73 6D 75 67 67 6C 65 64 20 66 69 72 65 of smuggled fire
2023-08-26 08:51:42.940 [  TRACE]:tbl-eit: 77 6F 72 6B 73 2E 20 28 53 31 32 20 45 70 31 38 works. (S12 Ep18
2023-08-26 08:51:42.940 [  TRACE]:tbl-eit: 2F 32 31 29 20 20 5B 41 44 2C 53 5D             /21)  [AD,S]    
2023-08-26 08:51:42.940 [  TRACE]:tbl-eit: svc='Channel 4', ch='Channel 4', eid=35801, tbl=50, running=0, start=2023-08-26;12:05:00(+0100), stop=2023-08-26;12:35:00(+0100), ebc=0x6645c568

Channel 4 HD , the same episode (start time and end time)


2023-08-26 08:51:47.636 [  TRACE]:tbl-eit: eit:  dtag 4D dlen 111
2023-08-26 08:51:47.636 [  TRACE]:tbl-eit: 65 6E 67 08 1F 01 07 E3 6D 64 72 60 62 1F 02 F9 eng.....mdr`b...
2023-08-26 08:51:47.636 [  TRACE]:tbl-eit: 0E D4 2D 1D BC 4D 79 8C 5B 5B 1D AE AF DB 55 25 ..-..My.[[....U%
2023-08-26 08:51:47.636 [  TRACE]:tbl-eit: D7 3A F6 2D C7 B0 03 BF B9 A9 EA 66 A5 53 89 95 .:.-.......f.S..
2023-08-26 08:51:47.636 [  TRACE]:tbl-eit: 67 FD 75 78 DF 98 FE AD C5 BF BD C5 9E 49 22 97 g.ux.........I".
2023-08-26 08:51:47.636 [  TRACE]:tbl-eit: 4A 87 AA D1 71 15 E6 E5 28 2E 45 3F 7D 13 E4 74 J...q...(.E?}..t
2023-08-26 08:51:47.636 [  TRACE]:tbl-eit: 54 A7 D8 76 29 3D 25 E7 60 7D 77 B7 E3 E2 36 01 T..v)=%.`}w...6.
2023-08-26 08:51:47.636 [  TRACE]:tbl-eit: 47 79 B8 F1 13 00 03 14 90 ED A8 6C B1 CB 80    Gy.........l... 
2023-08-26 08:51:47.636 [  TRACE]:tbl-eit: svc='Channel 4 HD', ch='Channel 4 HD', eid=35801, tbl=60, running=0, start=2023-08-26;12:05:00(+0100), stop=2023-08-26;12:35:00(+0100), ebc=0x662f2aa0

Same language marker, but the tag is much shorter and looks like nonsense.

davep

That's not nonsense, that's Huffman compression.

For Freeview HD channels the title and subtitle character strings are compressed using Huffman coding to save bandwidth (there are also commercial reasons). The '1F' byte at the start of the string marks it as compressed. Now TVH is able to decompress Huffman strings so the question is why isn't it working?

Do you have the EPG grabbers set to their default priorities - EIT EPG Grabber at 1 and UK Freeview at 7? Also have you set UK Freeview as the default grabber for ALL Freeview muxes?

Chris L

Dave Pickles wrote:

> That's not nonsense, that's Huffman compression.

Makes sense.

> Do you have the EPG grabbers set to their default priorities - EIT EPG Grabber at 1 and UK Freeview at 7? Also have you set UK Freeview as the default grabber for ALL Freeview muxes?

Freeview was set to 5, but I've now changed it to 7.

All of the SD muxes were set to EIT. Only the HD one was set to Freeview. I've now set them all to Freeview and manually triggered a grab.

The grab "completes" almost instantaneously though, with the following trace output:

2023-08-26 18:54:44.964 [ DEBUG]:epggrab: no OTA modules active for 585.833MHz in DVB-T Network, check again next time 2023-08-26 18:54:44.964 [ DEBUG]:epggrab: no OTA modules active for 626MHz in DVB-T Network, check again next time 2023-08-26 18:54:44.964 [ DEBUG]:epggrab: no OTA modules active for 578MHz in DVB-T Network, check again next time 2023-08-26 18:54:44.964 [ DEBUG]:epggrab: no OTA modules active for 562MHz in DVB-T Network, check again next time 2023-08-26 18:54:44.964 [ DEBUG]:epggrab: no OTA modules active for 538MHz in DVB-T Network, check again next time 2023-08-26 18:54:44.964 [ DEBUG]:epggrab: no OTA modules active for 602MHz in DVB-T Network, check again next time 2023-08-26 18:54:44.965 [ DEBUG]:epggrab: no OTA modules active for 554MHz in DVB-T Network, check again next time 2023-08-26 18:54:44.965 [ DEBUG]:epggrab: no OTA modules active for 522MHz in DVB-T Network, check again next time

davep

Hmmm running out of ideas...

For your Freeview Muxes, you have "EPG Module ID" set to "UK: Freeview". Do you also have "EPG Scan" set to "Manual"?

The muxes not only transmit EPG for their own channels, each one also transmits, on a slow loop, EPG for all the other channels on the network. Hence all muxes should be set the same.

DeltaMikeCharlie

I've run out of ideas, sorry. I only have access to DVB-T(1) services. Also, my testing/development system is currently not functioning.

From what I have read, the '0x1F' at the start of the tag text means that the data is encoded using the 'encoding_type_id' and the following '0x01' means that the encoding type is 1 of the 4 encoding methods that have been allocated to the BBC.

If you read along 8 bytes, you get to '0x62' (dec 98) and then another '0x1F'. This one is followed by a '0x02', also allocated to the BBC.

Apparently, the BBC 'encode' their EPG data, but not the actual programme content, for their DVB-T2 services. I have only ever seen this mechanism used to select a character set, not to encrypt data.

Chris L

Dave Pickles wrote:
> Hmmm running out of ideas...
>
> For your Freeview Muxes, you have "EPG Module ID" set to "UK: Freeview". Do you also have "EPG Scan" set to "Manual"?
>

> The muxes not only transmit EPG for their own channels, each one also transmits, on a slow loop, EPG for all the other channels on the network. Hence all muxes should be set the same.

Hmm, I hadn't appreciated the significance of the "EPG Scan" setting. They were all set to "Enable (auto)". When set to auto, does that mean the selected "EPG Module ID" is ignored?

I tried all set to "Manual", with all module IDs set to "UK: Freeview' and Status -> Subscriptions had a row for epggrab that went from Testing to Bad (and then stuck there, needing a service stop / start to clear it).

I then tried all still set to "Manual", with module IDs set to "EIT: EPG Grabber" (as that's what Ithought was being used, prior to the manual/auto revelation above) with the same "Bad" result.

Rather than leave the system with no EPG at all, I set everything back to auto again, expecting to only get the SD info back, but now suddenly my HD channels have EPG too! I believe I've put everything back exactly as it was just 12 hours ago, but something somewhere must be different. If "Enable (auto)" does mean that the selected "EPG Module ID" is ignored, than all I have done is switch every mux from auto to manual and back to auto.

Jonas Lang

You also played around with the priority of the EPG grabbers. Remember the higher the number the higher the priority.