Feature #4198

(Small?) change to fully support EIT EPG in New Zealand DVB-T

Added by Mark Cookson 9 months ago. Updated 5 months ago.

Status:AcceptedStart date:2017-01-27
Priority:NormalDue date:
Assignee:Jaroslav Kysela% Done:

100%

Category:EPG - Grabbers
Target version:-

Description

After years of only including now and next EIT information, DVB-T broadcasts in New Zealand now include a full 7 day schedule.

Unfortunately for viewers outside the Auckland region, the 7-day EIT EPG data only shows up in Tvheadend for a few channels. This is because the Auckland 7-day EPG is broadcast nationally, and EPG data for most channels does not map on to the regional multiplex channels, although the programme content is identical to that for the Auckland region.

I think that the reason is this: (quoted from http://www.freeviewnz.tv/media/1055/freeview_dtt_transmission_rules_2_1.pdf )

_
5.12 EIT Schedule actual and other
The DTT Freeview network will not Transmit EIT schedule (actual or other) information. The EPG data will be handled by an MHEG-5 EPG application, however due to the introduction of MHEG enabled PVR devices, the Freeview DVB-T Network now does carry limited CRID data in the EITSchedule actual and other tables.

To limit the EITschedule bandwidth broadcast on each multiplexer, EITschedule_actual and EITschedule_other tables are activated on Transport_streams;-
0x19 TVNZ Auckland multiplexer,
0x1d TVWorks multiplexer
0x21 Kordia multiplexer.

The TVNZ regional multiplexers transport_stream_ids 0x1a, 0x1b and 0x1c are deemed identical to transport_stream_id 0x19 since they are made up from exactly the same service_ids, event_ids and associated CID data, differing in advertorial content only. EITschedule_actual data is not activated on these multiplexers.
For a PVR device to fully populate its event information database with every Freeview services event_ids, irrespective of its current active multiplexer it must parse both the EITschedule actual and other tables. If the database includes duplicate service_ids irrespective of their transport_stream_ids it shall discard the service_id with the lesser signal quality.

_

Is there any way to make the broadcast EIT data map correctly on to the regional multiplex broadcasts?

sample.ts - TVNZ mux sample (94.3 MB) Mark Cookson, 2017-03-10 09:24

epg.png (190 KB) Mark Clarkstone, 2017-03-13 16:19

tvheadend.log (15 Bytes) Andy Gardner, 2017-03-14 14:05

tvheadend.log (15 Bytes) Andy Gardner, 2017-03-14 20:31

tvheadend.log (15.1 MB) Andy Gardner, 2017-03-14 20:36

tvheadend.log (6.08 MB) Andy Gardner, 2017-03-18 02:32

gdb.txt Magnifier (8.15 KB) Andy Gardner, 2017-03-19 15:47

tvheadend.log (5.06 MB) Andy Gardner, 2017-03-22 13:28

Associated revisions

Revision 003bebfa
Added by Jaroslav Kysela 7 months ago

eit grabber: add PIDs for New Zealand, fixes #4198

Revision 673dc8ec
Added by Jaroslav Kysela 7 months ago

eit grabber: huffman fix for NZ grabbers, fixes #4198

Revision 02be0a0a
Added by Jaroslav Kysela 7 months ago

eit: fix the huffman string decoding, fixes #4198

Revision f24bd41f
Added by Jaroslav Kysela 7 months ago

eit grabber: rework the New Zealand support (hopefully correctly now), fixes #4198

Revision 0dbc5c98
Added by Jaroslav Kysela 7 months ago

eit grabber: fix the huffman decoding, fixes #4198

Revision 1f3411cd
Added by Jaroslav Kysela 7 months ago

eit: fix nz_freesat -> freeview conversion, fixes #4198

History

#1 Updated by Andy Gardner 8 months ago

Can I add a "YES PLEASE" to this request?

tvheadend users in Auckland NZ can be up and running immediately with a 7 day EPG straight out of the box.

Those of us outside Auckland have to fiddle around with XMLTV along with an EPG file which is tricky to nail down. I'm getting close to sticking tvheadend in the "too hard" basket because of it.

I upgraded to 4.1 hoping the "EIT - skip TSID check" option would make things work but it doesn't help. I still get "epggrab: EIT: DVB Grabber - data completion timeout for 578MHz in TVNZ" and "epggrab: EIT: DVB Grabber - data completion timeout for 562MHz in Mediaworks" which are the 2 main networks in NZ.

Some way of specifying what transport stream to pull EIT data off for each MUX would be FABULOUS.

Mark Cookson wrote:

After years of only including now and next EIT information, DVB-T broadcasts in New Zealand now include a full 7 day schedule.

Unfortunately for viewers outside the Auckland region, the 7-day EIT EPG data only shows up in Tvheadend for a few channels. This is because the Auckland 7-day EPG is broadcast nationally, and EPG data for most channels does not map on to the regional multiplex channels, although the programme content is identical to that for the Auckland region.

I think that the reason is this: (quoted from http://www.freeviewnz.tv/media/1055/freeview_dtt_transmission_rules_2_1.pdf )

_
5.12 EIT Schedule actual and other
The DTT Freeview network will not Transmit EIT schedule (actual or other) information. The EPG data will be handled by an MHEG-5 EPG application, however due to the introduction of MHEG enabled PVR devices, the Freeview DVB-T Network now does carry limited CRID data in the EITSchedule actual and other tables.

To limit the EITschedule bandwidth broadcast on each multiplexer, EITschedule_actual and EITschedule_other tables are activated on Transport_streams;-
0x19 TVNZ Auckland multiplexer,
0x1d TVWorks multiplexer
0x21 Kordia multiplexer.

The TVNZ regional multiplexers transport_stream_ids 0x1a, 0x1b and 0x1c are deemed identical to transport_stream_id 0x19 since they are made up from exactly the same service_ids, event_ids and associated CID data, differing in advertorial content only. EITschedule_actual data is not activated on these multiplexers.
For a PVR device to fully populate its event information database with every Freeview services event_ids, irrespective of its current active multiplexer it must parse both the EITschedule actual and other tables. If the database includes duplicate service_ids irrespective of their transport_stream_ids it shall discard the service_id with the lesser signal quality.

_

Is there any way to make the broadcast EIT data map correctly on to the regional multiplex broadcasts?

#2 Updated by Jaroslav Kysela 8 months ago

Could you (just for a test) do this change:

diff --git a/src/epggrab/module/eit.c b/src/epggrab/module/eit.c
index 815eb3f..299fc13 100644
--- a/src/epggrab/module/eit.c
+++ b/src/epggrab/module/eit.c
@@ -770,7 +770,7 @@ static int _eit_start

   /* Viasat Baltic (0x39) */
   } else if (!strcmp("viasat_baltic", m->id)) {
-    pid = 0x39;
+    pid = 0x19;

   /* Bulsatcom 39E (0x12b) */
   } else if (!strcmp("Bulsatcom_39E", m->id)) {

And activate the 'VIASAT: Baltic' EPG grabber ? Do you see any extra EPG data from '0x19 TVNZ Auckland multiplexer' ?

#3 Updated by Mark Cookson 8 months ago

Jaroslav Kysela wrote:

Could you (just for a test) do this change:

[...]

And activate the 'VIASAT: Baltic' EPG grabber ? Do you see any extra EPG data from '0x19 TVNZ Auckland multiplexer' ?

Hi Jaroslav,

Thanks for looking at this! I currently have TVH running on a Synology NAS, so I'll give your patch a try, but it may be a few days before I get everything set up to compile and run TVH on my desktop.

Regards,

Mark

#4 Updated by Nathan Fieldhouse 8 months ago

I asked for MHEG-5 EPG support a while back got accepted as Feature #3783. I'm in the Wairarapa wonder if this would work for me two as I am using the XMLTV script, it works but it can be frustrating

#5 Updated by Mark Cookson 8 months ago

Nathan Fieldhouse wrote:

I asked for MHEG-5 EPG support a while back got accepted as Feature #3783. I'm in the Wairarapa wonder if this would work for me two as I am using the XMLTV script, it works but it can be frustrating

If I've understood other people's postings correctly, Freeview in NZ started out with the full 7 day EPG only being broadcast in MHEG-5 format, while the EIT guide was limited to Now and Next info. It was and is possible (but quite fiddly!) to extract the MHEG5 EPG data to an XMLTV file using https://sourceforge.net/projects/mheg2xmltv/

The MHEG-5 EPG is still being broadcast, but at some time in the past year or so, Freeview quietly started broadcasting the full 7 day EPG in EIT form as well. Unfortunately, they're doing it in a slightly non-standard way, which TVHeadend doesn't currently cope with (I get a full 7 day EIT EPG in Wellington for some channels such as Maori TV, but not for the TVNZ and Mediaworks channels). Other software such as NextPVR and MythTV have apparently been able to make changes so that users throughout New Zealand receive the full EIT EPG.

More info here:
http://www.geekzone.co.nz/forums.asp?forumid=126&topicid=205124

If it's possible to get the working in TVHeadend, then yes, it should be possible for users anywhere in New Zealand to get the full EPG using only the built-in EIT grabbers.

My next couple of weeks are looking very busy, so if anyone else has time to try compiling with Jaroslav's patch above, that would be wonderful!

Mark

#6 Updated by Mark Clarkstone 8 months ago

Mark Cookson wrote:

Nathan Fieldhouse wrote:

I asked for MHEG-5 EPG support a while back got accepted as Feature #3783. I'm in the Wairarapa wonder if this would work for me two as I am using the XMLTV script, it works but it can be frustrating

If I've understood other people's postings correctly, Freeview in NZ started out with the full 7 day EPG only being broadcast in MHEG-5 format, while the EIT guide was limited to Now and Next info. It was and is possible (but quite fiddly!) to extract the MHEG5 EPG data to an XMLTV file using https://sourceforge.net/projects/mheg2xmltv/

The MHEG-5 EPG is still being broadcast, but at some time in the past year or so, Freeview quietly started broadcasting the full 7 day EPG in EIT form as well. Unfortunately, they're doing it in a slightly non-standard way, which TVHeadend doesn't currently cope with (I get a full 7 day EIT EPG in Wellington for some channels such as Maori TV, but not for the TVNZ and Mediaworks channels). Other software such as NextPVR and MythTV have apparently been able to make changes so that users throughout New Zealand receive the full EIT EPG.

More info here:
http://www.geekzone.co.nz/forums.asp?forumid=126&topicid=205124

If it's possible to get the working in TVHeadend, then yes, it should be possible for users anywhere in New Zealand to get the full EPG using only the built-in EIT grabbers.

My next couple of weeks are looking very busy, so if anyone else has time to try compiling with Jaroslav's patch above, that would be wonderful!

Mark

I think what also would be really helpful here is a mux dump. This would allow Jarosav to confirm his patch is working if needed.

If either of you want to upload one, you can do so here: http://pam.exsilia.net/issues/4198 (avoiding cloudflare's 100Mb limit).

just wget a mux url & upload the resulting ts file :)

#7 Updated by Mark Cookson 7 months ago

Mark Clarkstone wrote:

I think what also would be really helpful here is a mux dump. This would allow Jarosav to confirm his patch is working if needed.

If either of you want to upload one, you can do so here: http://pam.exsilia.net/issues/4198 (avoiding cloudflare's 100Mb limit).

just wget a mux url & upload the resulting ts file :)

Hi Mark,

I didn't realise that you could grab a mux using wget! I've attached what I think should be a 30 second sample - please let me know if I've done it correctly.

Thanks,

Mark

#8 Updated by Andy Gardner 7 months ago

I was going to offer to run something (specified by someone with more knowledge than me) in dvbsnoop and send the output, but a mux dump is more useful?

I was running the Debian package as provided in the repository but downloaded the git source. Made the change and compiled it and it ran (so far so good) but isn't picking up the EIT EPG. I'm having trouble with the EPG from the other muxes so I've probably screwed up somewhere else anyway.

#9 Updated by Andy Gardner 7 months ago

Also, the spec for EIT appears to have been updated...

http://www.freeviewnz.tv/media/1216/freeview_dtt_transmission_rules_3.pdf

"Updated rules around EIT schedule population within the FreeviewNZ DVB-SI. All changes from version 2.4 have been highlighted in pink."

Also:"Strings shall be compressed using sets of static Huffman trees defined by FreeviewNZ. FreeviewNZ will use two lookup tables. Each will be a maximum of 10kilobytes in size. These tables are available electronically from FreeviewNZ."

#10 Updated by Andy Gardner 7 months ago

That newer spec also has changes to the list of transport stream that carry full EIT data:

####
To limit the EITschedule bandwidth broadcast on each multiplexer, EITschedule_actual
and EITschedule_other tables are activated on Transport_streams;-
0x19 TVNZ Auckland multiplexer,
0x1d TVWorks multiplexer
0x21 Kordia1 multiplexer
0x22 Kordia2 multiplexer
0x26 Hawke’s Bay TV Multiplexer ####

The 0x21 and 0x22 TS's are available on the transmitter here in CHCH.

#11 Updated by Andy Gardner 7 months ago

Sorry about rabbiting on.

In my case (Christchurch transmitter but applicable to entire South Island), epggrab needs to pull the EIT from the mux with TSID 0x21, and map anything listed as TSID 0x1d to 0x20 and TSID 0x19 to 0x1c

That's correct?

Is that easily done?

#12 Updated by Andy Gardner 7 months ago

Part way there.

The EPG coming off EIT was very sparse for the channels on that actual feed.

I changed the grabber we were experimenting with from viasat_baltic to uk_freeview and have got much better results. I'm guessing this is because of the Huffman encoding on the EIT data.

I added the pid line:

if (!strcmp(m->id, "uk_freeview")) {
pid= 0x22;
m = (epggrab_module_ota_t*)epggrab_module_find_by_id("eit");

and now the EPG has all the info for the channels that are on the "normal" transport streams.

We just need to be able to map EIT data from those TSID's listed above and we're good to go.

Maybe a epggrab module for each NZ region will be required?

Here's a list of my suggestions for their names (based on the formal document) and the TSID's that need to be mapped:

nz_freeview_auckland (no mapping required)
nz_freeview_central 0x0019 -> 0x001a , 0x001d -> 0x001e
nz_freeview_wellington 0x0019 -> 0x001b , 0x001d -> 0x001f
nz_freeview_south_island 0x0019 -> 0x001c , 0x001d -> 0x0020

#13 Updated by Nathan Fieldhouse 7 months ago

Where do I find the TSID's. I'm in the wairarapa so I want to check that I have the same as Wellington??

Thanks

Andy Gardner wrote:

Part way there.

The EPG coming off EIT was very sparse for the channels on that actual feed.

I changed the grabber we were experimenting with from viasat_baltic to uk_freeview and have got much better results. I'm guessing this is because of the Huffman encoding on the EIT data.

I added the pid line:

if (!strcmp(m->id, "uk_freeview")) {
pid= 0x22;
m = (epggrab_module_ota_t*)epggrab_module_find_by_id("eit");

and now the EPG has all the info for the channels that are on the "normal" transport streams.

We just need to be able to map EIT data from those TSID's listed above and we're good to go.

Maybe a epggrab module for each NZ region will be required?

Here's a list of my suggestions for their names (based on the formal document) and the TSID's that need to be mapped:

nz_freeview_auckland (no mapping required)
nz_freeview_central 0x0019 -> 0x001a , 0x001d -> 0x001e
nz_freeview_wellington 0x0019 -> 0x001b , 0x001d -> 0x001f
nz_freeview_south_island 0x0019 -> 0x001c , 0x001d -> 0x0020

#14 Updated by Andy Gardner 7 months ago

Nathan Fieldhouse wrote:

Where do I find the TSID's. I'm in the wairarapa so I want to check that I have the same as Wellington??

Go to Config -> DVB Inputs -> Muxes and the TSID should show in column #6

If you take a look at http://www.freeviewnz.tv/media/1216/freeview_dtt_transmission_rules_3.pdf (18.1.7 Cell_id) it shows a Manawatu TX at Popoiti, which is southwest of Carterton? That shows the same TSID's as Wellington.

I've struck a problem with using uk_freeview as the EIT grabber - all my recordings are set to UTC rather than our current UTC+13, so I'm getting the wrong programmes recorded. :^(

#15 Updated by Jaroslav Kysela 7 months ago

Andy Gardner wrote:

I've struck a problem with using uk_freeview as the EIT grabber - all my recordings are set to UTC rather than our current UTC+13, so I'm getting the wrong programmes recorded. :^(

You can change the time offset in the network settings - 'EIT time offset' field.

#16 Updated by Jaroslav Kysela 7 months ago

  • Status changed from New to Fixed
  • % Done changed from 0 to 100

#17 Updated by Jaroslav Kysela 7 months ago

  • Status changed from Fixed to Accepted
  • Assignee changed from Adam Sutton to Jaroslav Kysela

Could you try the above patch? It was applied to v4.1-2460-g003bebf . If I understand correctly, the 'Auckland' grabbler must be enabled and then you should enable the regional grabber which has the higher priority (thus events from these grabbers will override events from 'Auckland').

#18 Updated by Andy Gardner 7 months ago

Jaroslav Kysela wrote:

Could you try the above patch? It was applied to v4.1-2460-g003bebf . If I understand correctly, the 'Auckland' grabbler must be enabled and then you should enable the regional grabber which has the higher priority (thus events from these grabbers will override events from 'Auckland').

Hi Jaroslav,

I tried it, but I think it's missing the code to treat all those NZ EIT feeds the same as the UK_Freeview ones - they need the Huffman tables applied?

I think the bit missing is:

if (!strcmp(m->id, "uk_freeview")) {
m = (epggrab_module_ota_t*)epggrab_module_find_by_id("eit");
if (m->enabled) return -1;
}

???

Thanks,

#19 Updated by Andy Gardner 7 months ago

Jaroslav Kysela wrote:

Andy Gardner wrote:

I've struck a problem with using uk_freeview as the EIT grabber - all my recordings are set to UTC rather than our current UTC+13, so I'm getting the wrong programmes recorded. :^(

You can change the time offset in the network settings - 'EIT time offset' field.

I've got it set to "Local (server) time". It appears to be ignoring that and running on UTC.

UTC+12 and UTC+13 don't appear as options.

#20 Updated by Mark Clarkstone 7 months ago

Andy Gardner wrote:

Jaroslav Kysela wrote:

Andy Gardner wrote:

I've struck a problem with using uk_freeview as the EIT grabber - all my recordings are set to UTC rather than our current UTC+13, so I'm getting the wrong programmes recorded. :^(

You can change the time offset in the network settings - 'EIT time offset' field.

I've got it set to "Local (server) time". It appears to be ignoring that and running on UTC.

UTC+12 and UTC+13 don't appear as options.

I've created a PR that adds a note about the NZ grabbers to the help docs, and added UTC+12 & +13 to the offset list. If you want to manually add them in (instead of waiting for the PR to be merged) you can see how by looking at my PR here.

#21 Updated by Jaroslav Kysela 7 months ago

  • Status changed from Accepted to Fixed

#22 Updated by Jaroslav Kysela 7 months ago

  • Status changed from Fixed to Accepted

OK. Another round: v4.1-2463-ge664c3e . The huffman issue should be resolved and also Mark's doc changes are included.

#23 Updated by Andy Gardner 7 months ago

Jaroslav Kysela wrote:

OK. Another round: v4.1-2463-ge664c3e . The huffman issue should be resolved and also Mark's doc changes are included.

Getting loads of entries in the EPS with blank Title, which indicates the huffman issue isn't resolved as yet, unfortunately.

#24 Updated by Jaroslav Kysela 7 months ago

Could you check the value of (intptr_t)m->opaque variable in _eit_get_string_with_len() function ?

#25 Updated by Mark Clarkstone 7 months ago

Unfortunately these changes appear to have broken the Freeview (UK) grabber. I've cleared the database to be sure, and only have the Freeview grabber enabled. But I'm missing descriptions and programme titles are mixed up :(.

#26 Updated by Nathan Fieldhouse 7 months ago

The TSID's I have off Popoiti are 27 (0x001B) 31 (0x001F) 33 (0x0021) & 34 (0x0022), so which one is transmitting the guide?

Andy Gardner wrote:

Nathan Fieldhouse wrote:

Where do I find the TSID's. I'm in the wairarapa so I want to check that I have the same as Wellington??

Go to Config -> DVB Inputs -> Muxes and the TSID should show in column #6

If you take a look at http://www.freeviewnz.tv/media/1216/freeview_dtt_transmission_rules_3.pdf (18.1.7 Cell_id) it shows a Manawatu TX at Popoiti, which is southwest of Carterton? That shows the same TSID's as Wellington.

I've struck a problem with using uk_freeview as the EIT grabber - all my recordings are set to UTC rather than our current UTC+13, so I'm getting the wrong programmes recorded. :^(

#27 Updated by Jaroslav Kysela 7 months ago

  • Status changed from Accepted to Fixed

#28 Updated by Jaroslav Kysela 7 months ago

  • Status changed from Fixed to Accepted

I hope that all issues are resolved in v4.1-2467-g02be0a0 . Please, test.

#29 Updated by Andy Gardner 7 months ago

Jaroslav Kysela wrote:

I hope that all issues are resolved in v4.1-2467-g02be0a0 . Please, test.

Huffman problem is fixed. Thanks!

OK. Here's the set-up at present:

Grabbers enabled:
NZ Auckland (5)
NZ Kordia 1 (5)
NZ Kordia 2 (5)
NZ South Island (6)
NZ TVWorks South Island (6)

With EIT scan enabled on all muxes, I get:

###

Mar 15 00:17:47 dvb-t2 tvheadend17440: epggrab: ota - kick callback
Mar 15 00:17:47 dvb-t2 tvheadend17440: epggrab: no OTA modules active for 594MHz in Kordia, check again next time
Mar 15 00:17:47 dvb-t2 tvheadend17440: epggrab: no OTA modules active for 578MHz in TVNZ, check again next time
Mar 15 00:17:47 dvb-t2 tvheadend17440: epggrab: no OTA modules active for 562MHz in Mediaworks, check again next time
Mar 15 00:17:47 dvb-t2 tvheadend17440: epggrab: no OTA modules active for 530MHz in World TV, check again next time
Mar 15 00:17:47 dvb-t2 tvheadend17440: epggrab: mux stats - all 4 pending 0

###

The EPG stays empty.

If I enable the base EIT: DVB grabber as well, the EPG is filled with data for programmes on TSID 0x21, but nothing else. THe log reports:

###

Mar 15 00:26:02 dvb-t2 tvheadend17708: mpegts: 578MHz in TVNZ - tuning on Realtek RTL2832 (DVB-T) : DVB-T #2
Mar 15 00:26:03 dvb-t2 tvheadend17708: subscription: 0002: "epggrab" subscribing to mux "578MHz", weight: 4, adapter: "Realtek RTL2832 (DVB-T) : DVB-T #2", network: "TVNZ", service: "Raw PID Subscription"
Mar 15 00:26:03 dvb-t2 tvheadend17708: mpegts: 562MHz in Mediaworks - tuning on Realtek RTL2832 (DVB-T) : DVB-T #1
Mar 15 00:26:03 dvb-t2 tvheadend17708: subscription: 0003: "epggrab" subscribing to mux "562MHz", weight: 4, adapter: "Realtek RTL2832 (DVB-T) : DVB-T #1", network: "Mediaworks", service: "Raw PID Subscription
Mar 15 00:26:04 dvb-t2 tvheadend17708: tbl-eit: nz_kordia1: 578MHz in TVNZ: invalid checksum (len 452, errors 1)
Mar 15 00:26:15 dvb-t2 tvheadend17708: tbl-eit: nz_kordia1: 578MHz in TVNZ: invalid checksum (len 452, errors 41)
Mar 15 00:26:25 dvb-t2 tvheadend17708: tbl-eit: nz_kordia1: 578MHz in TVNZ: invalid checksum (len 452, errors 81)
Mar 15 00:26:35 dvb-t2 tvheadend17708: tbl-eit: nz_kordia1: 578MHz in TVNZ: invalid checksum (len 452, errors 121)

###

So the new grab modules appear to not be working quite yet?

Here's some dvbsnoop data off the local transmission., hopefully it will help you...

###

adapter 0 is tuned to Kordia1 TSID 0x21

  1. dvbsnoop -s sec 0x12 -n 1000 -adapter 0|grep Transport_stream_ID|grep -c 0019
    110
  1. dvbsnoop -s sec 0x12 -n 1000 -adapter 0|grep Transport_stream_ID|grep -c 001a
    20
  1. dvbsnoop -s sec 0x12 -n 1000 -adapter 0|grep Transport_stream_ID|grep -c 001b
    15
  1. dvbsnoop -s sec 0x12 -n 1000 -adapter 0|grep Transport_stream_ID|grep -c 001c
    20
  1. dvbsnoop -s sec 0x12 -n 1000 -adapter 0|grep Transport_stream_ID|grep -c 001d
    100
  1. dvbsnoop -s sec 0x12 -n 1000 -adapter 0|grep Transport_stream_ID|grep -c 001e
    15
  1. dvbsnoop -s sec 0x12 -n 1000 -adapter 0|grep Transport_stream_ID|grep -c 001f
    18
  1. dvbsnoop -s sec 0x12 -n 1000 -adapter 0|grep Transport_stream_ID|grep -c 0020
    20
  1. dvbsnoop -s sec 0x12 -n 1000 -adapter 0|grep Transport_stream_ID|grep -c 0021
    476
  1. dvbsnoop -s sec 0x12 -n 1000 -adapter 0|grep Transport_stream_ID|grep -c 0022
    173

adapter 1 is tuned to tvworks south island TSID 0x20

  1. dvbsnoop -s sec 0x12 -n 1000 -adapter 1|grep Transport_stream_ID|grep -c 001c
    20
  2. dvbsnoop -s sec 0x12 -n 1000 -adapter 1|grep Transport_stream_ID|grep -c 0020
    100

adapter 2 is tuned to tvnz south island TSID 0x1c

  1. dvbsnoop -s sec 0x12 -n 1000 -adapter 2|grep Transport_stream_ID|grep -c 001c
    98
  2. dvbsnoop -s sec 0x12 -n 1000 -adapter 2|grep Transport_stream_ID|grep -c 0020
    20

#30 Updated by Jaroslav Kysela 7 months ago

Provide '--trace tbl-eit' .. https://tvheadend.org/projects/tvheadend/wiki/Traces .. when you tune to one mux (channel)

#31 Updated by Andy Gardner 7 months ago

Jaroslav Kysela wrote:

Provide '--trace tbl-eit' .. https://tvheadend.org/projects/tvheadend/wiki/Traces .. when you tune to one mux (channel)

###

2017-03-15 01:03:06.573 [ TRACE]:tbl-eit: eit: pid 12 tableid 51 extraid 0000000000210578 len 11
2017-03-15 01:03:06.573 [ TRACE]:tbl-eit: eit: section 48 last 248 ver 20 (ver 20 st 2 incomp 0 comp 165)
2017-03-15 01:03:06.573 [ TRACE]:tbl-eit: eit: skip, already complete (2)
2017-03-15 01:03:06.573 [ TRACE]:tbl-eit: eit: pid 12 tableid 50 extraid 000000000021057b len 147
2017-03-15 01:03:06.573 [ TRACE]:tbl-eit: eit: section 176 last 248 ver 1 (ver 1 st 2 incomp 0 comp 165)
2017-03-15 01:03:06.573 [ TRACE]:tbl-eit: eit: skip, already complete (2)
2017-03-15 01:03:06.573 [ TRACE]:tbl-eit: eit: pid 12 tableid 51 extraid 000000000021057b len 390
2017-03-15 01:03:06.573 [ TRACE]:tbl-eit: eit: section 72 last 248 ver 17 (ver 17 st 2 incomp 0 comp 165)
2017-03-15 01:03:06.573 [ TRACE]:tbl-eit: eit: skip, already complete (2)
2017-03-15 01:03:06.573 [ TRACE]:tbl-eit: eit: pid 12 tableid 4F extraid 00000000001b04b5 len 126
2017-03-15 01:03:06.573 [ TRACE]:tbl-eit: eit: section 1 last 1 ver 19 (ver 19 st 2 incomp 0 comp 165)
2017-03-15 01:03:06.573 [ TRACE]:tbl-eit: eit: skip, already complete (2)
2017-03-15 01:03:06.573 [ TRACE]:tbl-eit: eit: pid 12 tableid 4E extraid 0000000000210579 len 261
2017-03-15 01:03:06.573 [ TRACE]:tbl-eit: eit: section 1 last 1 ver 15 (ver 15 st 2 incomp 0 comp 165)
2017-03-15 01:03:06.573 [ TRACE]:tbl-eit: eit: skip, already complete (2)
2017-03-15 01:03:06.625 [ TRACE]:tbl-eit: eit: pid 12 tableid 50 extraid 000000000021057c len 557
2017-03-15 01:03:06.625 [ TRACE]:tbl-eit: eit: section 56 last 248 ver 6 (ver 6 st 2 incomp 0 comp 165)
2017-03-15 01:03:06.625 [ TRACE]:tbl-eit: eit: skip, already complete (2)
2017-03-15 01:03:06.625 [ TRACE]:tbl-eit: eit: pid 12 tableid 4F extraid 00000000001c04b0 len 260
2017-03-15 01:03:06.625 [ TRACE]:tbl-eit: eit: section 1 last 1 ver 18 (ver 18 st 2 incomp 0 comp 165)
2017-03-15 01:03:06.625 [ TRACE]:tbl-eit: eit: skip, already complete (2)
2017-03-15 01:03:06.625 [ TRACE]:tbl-eit: eit: pid 12 tableid 62 extraid 00000000002205e1 len 220
2017-03-15 01:03:06.625 [ TRACE]:tbl-eit: eit: section 0 last 24 ver 12 (ver 12 st 2 incomp 0 comp 165)
2017-03-15 01:03:06.625 [ TRACE]:tbl-eit: eit: skip, already complete (2)
2017-03-15 01:03:06.677 [ TRACE]:tbl-eit: eit: pid 12 tableid 51 extraid 000000000021057c len 308
2017-03-15 01:03:06.677 [ TRACE]:tbl-eit: eit: section 48 last 248 ver 20 (ver 20 st 2 incomp 0 comp 165)
2017-03-15 01:03:06.677 [ TRACE]:tbl-eit: eit: skip, already complete (2)

###

Is that what you are after?

#32 Updated by Andy Gardner 7 months ago

Nathan Fieldhouse wrote:

The TSID's I have off Popoiti are 27 (0x001B) 31 (0x001F) 33 (0x0021) & 34 (0x0022), so which one is transmitting the guide?

Same as Wellington, then.

Full 7 day guide should be on 33 and 34.

#33 Updated by Jaroslav Kysela 7 months ago

Andy Gardner wrote:

Jaroslav Kysela wrote:

Provide '--trace tbl-eit' .. https://tvheadend.org/projects/tvheadend/wiki/Traces .. when you tune to one mux (channel)

Is that what you are after?

Full log, please, start with lines when you subscribe to a channel on a mux with incomplete EPG, wait for 60 seconds and attach (upload) this log to this bug.

#34 Updated by Andy Gardner 7 months ago

subscribe to a channel on a mux with incomplete EPG

Not sure what you mean by that. Do I enable a channel in the DVB Inputs -> Services tab?

#35 Updated by Mark Clarkstone 7 months ago

Andy Gardner wrote:

subscribe to a channel on a mux with incomplete EPG

Not sure what you mean by that. Do I enable a channel in the DVB Inputs -> Services tab?

While you have debugging going, play a service that is missing EPG data.

#36 Updated by Andy Gardner 7 months ago

Not sure what you mean by that. Do I enable a channel in the DVB Inputs -> Services tab?

While you have debugging going, play a service that is missing EPG data.

Ah! I understand now.

Done - just have to get the file off the other box onto here. Won't be long.

#37 Updated by Andy Gardner 7 months ago

Andy Gardner wrote:

Done - just have to get the file off the other box onto here. Won't be long.

...

#38 Updated by Mark Clarkstone 7 months ago

Andy Gardner wrote:

Andy Gardner wrote:

Done - just have to get the file off the other box onto here. Won't be long.

...

That file just contains

[object Object]

#39 Updated by Andy Gardner 7 months ago

Mark Clarkstone wrote:

That file just contains
[...]

Bizarre. It's 16Mb file before I upload it.

#40 Updated by Andy Gardner 7 months ago

3rd attempt.

#41 Updated by Jaroslav Kysela 7 months ago

  • Status changed from Accepted to Fixed

#42 Updated by Jaroslav Kysela 7 months ago

  • Status changed from Fixed to Accepted

There's just one grabber for NZ now - it enables the huffman decoding and specifies priority for the local EIT info. Both EIT and NZ Freesat may/should be enabled.

I reverted other changes, because the design was wrong (I worked with MPEG-TS PID not TSID - blame me ;-().

#43 Updated by Andy Gardner 7 months ago

Jaroslav Kysela wrote:

There's just one grabber for NZ now - it enables the huffman decoding and specifies priority for the local EIT info. Both EIT and NZ Freesat may/should be enabled.

I reverted other changes, because the design was wrong (I worked with MPEG-TS PID not TSID - blame me ;-().

I don't think we're quite there yet.

Nothing is appearing in the EPG. I have modules OTA EIT DVBGrabber and OTA NZ Freesat enable.

###
2017-03-16 03:11:12.456 [ INFO]:subscription: 0011: "epggrab" subscribing to mux "594MHz", weight: 4, adapter: "Realtek RTL2832 (DVB-T) : DVB-T #0", network: "Kordia", service: "Raw PID Subscription"
2017-03-16 03:11:12.457 [ INFO]:mpegts: 578MHz in TVNZ - tuning on Realtek RTL2832 (DVB-T) : DVB-T #2
2017-03-16 03:11:12.854 [ INFO]:subscription: 0012: "epggrab" subscribing to mux "578MHz", weight: 4, adapter: "Realtek RTL2832 (DVB-T) : DVB-T #2", network: "TVNZ", service: "Raw PID Subscription"
2017-03-16 03:11:12.854 [ INFO]:mpegts: 562MHz in Mediaworks - tuning on Realtek RTL2832 (DVB-T) : DVB-T #1
2017-03-16 03:11:13.249 [ INFO]:subscription: 0013: "epggrab" subscribing to mux "562MHz", weight: 4, adapter: "Realtek RTL2832 (DVB-T) : DVB-T #1", network: "Mediaworks", service: "Raw PID Subscription"
2017-03-16 03:11:47.456 [ DEBUG]:epggrab: grab done for 594MHz in Kordia (no data)
2017-03-16 03:11:47.456 [ INFO]:subscription: 0011: "epggrab" unsubscribing
2017-03-16 03:11:47.758 [ DEBUG]:epggrab: grab done for 578MHz in TVNZ (no data)
2017-03-16 03:11:47.758 [ INFO]:subscription: 0012: "epggrab" unsubscribing
2017-03-16 03:11:48.158 [ DEBUG]:epggrab: grab done for 562MHz in Mediaworks (no data)
2017-03-16 03:11:48.158 [ INFO]:subscription: 0013: "epggrab" unsubscribing ###

Also, I think the module should be called NZ_Freeview rather than Freesat.

Thanks!

#44 Updated by Jaroslav Kysela 7 months ago

Traces ? I renamed nz_freesat to nz_freeview in v4.1-2477-g019c946 .

#45 Updated by Andy Gardner 7 months ago

Jaroslav Kysela wrote:

Traces ? I renamed nz_freesat to nz_freeview in v4.1-2477-g019c946 .

Tracing susbsytems: tdl-eit,epggrab

This is all that shows up in the log file:

2017-03-18 13:53:44.266 [ TRACE]:epggrab: ota - kick callback
2017-03-18 13:53:44.267 [ INFO]:mpegts: 594MHz in Kordia - tuning on Realtek RTL2832 (DVB-T) : DVB-T #0
2017-03-18 13:53:44.674 [ INFO]:subscription: 022E: "epggrab" subscribing to mux "594MHz", weight: 4, adapter: "Realtek RTL2832 (DVB-T) : DVB-T #0", network: "Kordia", service: "Raw PID Subscription"
2017-03-18 13:53:44.674 [ TRACE]:epggrab: mux 594MHz in Kordia (0x7f2987f5fdd0), started
2017-03-18 13:53:44.674 [ INFO]:mpegts: 578MHz in TVNZ - tuning on Realtek RTL2832 (DVB-T) : DVB-T #2
2017-03-18 13:53:45.089 [ INFO]:subscription: 022F: "epggrab" subscribing to mux "578MHz", weight: 4, adapter: "Realtek RTL2832 (DVB-T) : DVB-T #2", network: "TVNZ", service: "Raw PID Subscription"
2017-03-18 13:53:45.089 [ TRACE]:epggrab: mux 578MHz in TVNZ (0x7f2987f6ad60), started
2017-03-18 13:53:45.089 [ INFO]:mpegts: 562MHz in Mediaworks - tuning on Realtek RTL2832 (DVB-T) : DVB-T #1
2017-03-18 13:53:45.497 [ INFO]:subscription: 0230: "epggrab" subscribing to mux "562MHz", weight: 4, adapter: "Realtek RTL2832 (DVB-T) : DVB-T #1", network: "Mediaworks", service: "Raw PID Subscription"
2017-03-18 13:53:45.497 [ TRACE]:epggrab: mux 562MHz in Mediaworks (0x7f2987f72460), started
2017-03-18 13:53:45.497 [ TRACE]:epggrab: subscription failed for 530MHz in World TV (result 200)
2017-03-18 13:53:45.497 [ TRACE]:epggrab: mux stats - all 4 pending 1
2017-03-18 13:53:45.530 [WARNING]:linuxdvb: Unable to provide UNC value.
2017-03-18 13:53:45.654 [WARNING]:linuxdvb: Unable to provide UNC value.
2017-03-18 13:53:46.056 [WARNING]:linuxdvb: Unable to provide UNC value.
2017-03-18 13:54:19.670 [ DEBUG]:epggrab: grab done for 594MHz in Kordia (no data)
2017-03-18 13:54:19.670 [ INFO]:subscription: 022E: "epggrab" unsubscribing
2017-03-18 13:54:19.673 [ TRACE]:epggrab: mux 594MHz in Kordia (0x7f2987f5fdd0) stop
2017-03-18 13:54:20.071 [ DEBUG]:epggrab: grab done for 578MHz in TVNZ (no data)
2017-03-18 13:54:20.071 [ INFO]:subscription: 022F: "epggrab" unsubscribing
2017-03-18 13:54:20.073 [ TRACE]:epggrab: mux 578MHz in TVNZ (0x7f2987f6ad60) stop
2017-03-18 13:54:20.472 [ DEBUG]:epggrab: grab done for 562MHz in Mediaworks (no data)
2017-03-18 13:54:20.472 [ INFO]:subscription: 0230: "epggrab" unsubscribing
2017-03-18 13:54:20.475 [ TRACE]:epggrab: mux 562MHz in Mediaworks (0x7f2987f72460) stop
2017-03-18 13:54:21.472 [ TRACE]:epggrab: ota - kick callback
2017-03-18 13:54:21.472 [ INFO]:mpegts: 530MHz in World TV - tuning on Realtek RTL2832 (DVB-T) : DVB-T #0
2017-03-18 13:54:21.472 [ INFO]:subscription: 0232: "epggrab" subscribing to mux "530MHz", weight: 4, adapter: "Realtek RTL2832 (DVB-T) : DVB-T #0", network: "World TV", service: "Raw PID Subscription"
2017-03-18 13:54:21.472 [ TRACE]:epggrab: mux 530MHz in World TV (0x7f2987f68cd0), started
2017-03-18 13:54:21.472 [ TRACE]:epggrab: mux stats - all 4 pending 0
2017-03-18 13:54:56.472 [ DEBUG]:epggrab: grab done for 530MHz in World TV (no data)
2017-03-18 13:54:56.472 [ INFO]:subscription: 0232: "epggrab" unsubscribing
2017-03-18 13:54:56.475 [ TRACE]:epggrab: mux 530MHz in World TV (0x7f2987f68cd0) stop

#46 Updated by Andy Gardner 7 months ago

Hang on, I see eit.c has been changed. I'll run another trace and report back.

#47 Updated by Andy Gardner 7 months ago

Andy Gardner wrote:

Hang on, I see eit.c has been changed. I'll run another trace and report back.

Done...

The huffman table issue appears to have come back.

#48 Updated by Jaroslav Kysela 7 months ago

  • Status changed from Accepted to Fixed

#49 Updated by Jaroslav Kysela 7 months ago

  • Status changed from Fixed to Accepted

Try v4.1-2482-g0dbc5c9 - the huffman decoding should be fixed.

Also, I think (judging from logs) that you split the muxes to separate networks. Don't do this. TVH looks for muxes from EIT only inside one network. Just define one DVB-T network and put all muxes into it.

If something does not work, provide traces again.

#50 Updated by Andy Gardner 7 months ago

OK. I get it. The problem is, for us old folks in NZ these used to be all separate networks so when we set stuff up from scratch we fail to realise that technically they're just one network now.

So I deleted all but one network, and loaded the predefined MUXes for Christchurch. That must be old data because they are on different frequencies now.

When I manually changed one to 594MHz everything was fine, but when I changed the 2nd one to 578MHz, I had a crash...

2017-03-19 01:14:28.068 [ ALERT]:CRASH: Signal: 11 in PRG: /usr/bin/tvheadend (4.1-2449~gda58349-dirty) [5b207419081986c4e4c604cffa656f3436912477] CWD: /
2017-03-19 01:14:28.068 [ ALERT]:CRASH: Fault address 0xa0 (Address not mapped)
2017-03-19 01:14:28.068 [ ALERT]:CRASH: Loaded libraries: linux-vdso.so.1 /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0 /lib/x86_64-linux-gnu/libz.so.1 /lib/x86_64-linux-gnu/libdl.so.2 /lib/x86_64-linux-gnu/libpthread.so.0 /lib/x86_64-linux-gnu/libm.so.6 /lib/x86_64-linux-gnu/librt.so.1 /usr/lib/x86_64-linux-gnu/libstdc++.so.6 /lib/x86_64-linux-gnu/libc.so.6 /lib64/ld-linux-x86-64.so.2 /lib/x86_64-linux-gnu/libgcc_s.so.1 /lib/x86_64-linux-gnu/libnss_compat.so.2 /lib/x86_64-linux-gnu/libnsl.so.1 /lib/x86_64-linux-gnu/libnss_nis.so.2 /lib/x86_64-linux-gnu/libnss_files.so.2
2017-03-19 01:14:28.068 [ ALERT]:CRASH: Register dump [23]: 000000000000000300000000000000106331303020646973000000000000000000007f37b801b8f000007f37b801a75000007f37cc00520000007f37cc00520000007f37b801b8f000007f3802adcac100007f37b8001d3000007f380584d7d000007f37b801b8f0000000000000000000007f37b800002000007f37d9ffa51000007f3801f3dc66000000000001020200000000000000330000000000000004000000000000000efffffffe7ffbba1100000000000000a0
2017-03-19 01:14:28.068 [ ALERT]:CRASH: STACKTRACE
2017-03-19 01:14:28.095 [ ALERT]:CRASH: ??:0 0x7f3801eb470a
2017-03-19 01:14:28.118 [ ALERT]:CRASH: ??:0 0x7f3800dee890
2017-03-19 01:14:28.141 [ ALERT]:CRASH: ??:0 0x7f3801f3dc66
2017-03-19 01:14:28.164 [ ALERT]:CRASH: ??:0 0x7f3801f3e83a
2017-03-19 01:14:28.187 [ ALERT]:CRASH: ??:0 0x7f3801f42bd3
2017-03-19 01:14:28.233 [ ALERT]:CRASH: ??:0 0x7f3801f22c9d
2017-03-19 01:14:28.257 [ ALERT]:CRASH: ??:0 0x7f3801f22e51
2017-03-19 01:14:28.280 [ ALERT]:CRASH: ??:0 0x7f3801f1cea9
2017-03-19 01:14:28.303 [ ALERT]:CRASH: ??:0 0x7f3801f1d0b6
2017-03-19 01:14:28.326 [ ALERT]:CRASH: ??:0 0x7f3801e7dd84
2017-03-19 01:14:28.349 [ ALERT]:CRASH: ??:0 0x7f3800de7064
2017-03-19 01:14:28.350 [ ALERT]:CRASH: clone+0x6d (/lib/x86_64-linux-gnu/libc.so.6)

Mar 19 01:14:28 dvb-t2 kernel: [772548.068530] tvh:mi-table30315: segfault at a0 ip 00007f3801f3dc66 sp 00007f37d9ffa510 error 4 in tvheadend[7f3801c9b000+191d000]

#51 Updated by Andy Gardner 7 months ago

If I try to have the MUXes located automatically they are not found.

If I load the predefined CHCH muxes I have to change the frequencies. They should be 530/562/578/594MHz.

When I change the frequencies for Kordia1 and Kordia2 MUXes (530 & 594MHz), everything is fine and dandy - the 7 day EPG populates and it's perfect.

When I change the frequency for the TVNZ or MediaWorks Muxes, and initiate a scan, I get the crash.

The stuff in the trace log before the crash is:

2017-03-19 01:42:10.070 [ DEBUG]:mpegts: Freeview - adding mux 578MHz in Freeview to scan queue weight 6 flags 4000
2017-03-19 01:42:10.071 [ DEBUG]:mpegts: 578MHz in Freeview - add raw service
2017-03-19 01:42:10.071 [ INFO]:mpegts: 578MHz in Freeview - tuning on Realtek RTL2832 (DVB-T) : DVB-T #2
2017-03-19 01:42:10.483 [ DEBUG]:mpegts: 578MHz in Freeview - started
2017-03-19 01:42:10.483 [ TRACE]:mpegts: table: mux 0x7f4f11c546a0 add eit 00/00 (0) pid 0012 (18)
2017-03-19 01:42:10.483 [ DEBUG]:mpegts: 578MHz in Freeview - open PID 0012 (18) [20/0x7f4ed80048e0]
2017-03-19 01:42:10.483 [ DEBUG]:mpegts: 578MHz in Freeview - open PID tables subscription [0042/0x7f4ed8002450]
2017-03-19 01:42:10.483 [ INFO]:subscription: 000B: "scan" subscribing to mux "578MHz", weight: 6, adapter: "Realtek RTL2832 (DVB-T) : DVB-T #2", network: "Freeview", service: "Raw PID Subscription"
2017-03-19 01:42:11.311 [ TRACE]:mpegts: table: mux 0x7f4f11c546a0 add pat 00/00 (0) pid 0000 (0)
2017-03-19 01:42:11.968 [ TRACE]:mpegts: input Realtek RTL2832 (DVB-T) : DVB-T #2 got 18800 bytes
2017-03-19 01:42:11.968 [ TRACE]:mpegts: 578MHz in Freeview - set tsid 001C (28)
2017-03-19 01:42:11.968 [ DEBUG]:mpegts: 578MHz in Freeview - add service 04B0 (null)
2017-03-19 01:42:11.968 [ TRACE]:mpegts: table: mux 0x7f4f11c546a0 add pmt 02/FF (2) pid 0082 (130)
2017-03-19 01:42:11.968 [ DEBUG]:mpegts: 578MHz in Freeview - open PID 0082 (130) [16/0x7f4ec00c4610]
2017-03-19 01:42:11.969 [ DEBUG]:mpegts: 578MHz in Freeview - add service 04B1 (null)
2017-03-19 01:42:11.969 [ TRACE]:mpegts: table: mux 0x7f4f11c546a0 add pmt 02/FF (2) pid 0083 (131)
2017-03-19 01:42:11.969 [ DEBUG]:mpegts: 578MHz in Freeview - open PID 0083 (131) [16/0x7f4ec00b8720]
2017-03-19 01:42:11.969 [ DEBUG]:mpegts: 578MHz in Freeview - add service 04B5 (null)
2017-03-19 01:42:11.969 [ TRACE]:mpegts: table: mux 0x7f4f11c546a0 add pmt 02/FF (2) pid 0086 (134)
2017-03-19 01:42:11.969 [ DEBUG]:mpegts: 578MHz in Freeview - open PID 0086 (134) [16/0x7f4ec00b9ba0]
2017-03-19 01:42:11.969 [ DEBUG]:mpegts: 578MHz in Freeview - add service 04B6 (null)
2017-03-19 01:42:11.969 [ TRACE]:mpegts: table: mux 0x7f4f11c546a0 add pmt 02/FF (2) pid 0084 (132)
2017-03-19 01:42:11.969 [ DEBUG]:mpegts: 578MHz in Freeview - open PID 0084 (132) [16/0x7f4ec00bb020]
2017-03-19 01:42:11.969 [ DEBUG]:mpegts: 578MHz in Freeview - add service 04B7 (null)
2017-03-19 01:42:11.969 [ TRACE]:mpegts: table: mux 0x7f4f11c546a0 add pmt 02/FF (2) pid 0085 (133)
2017-03-19 01:42:11.969 [ DEBUG]:mpegts: 578MHz in Freeview - open PID 0085 (133) [16/0x7f4ec00bc4a0]
2017-03-19 01:42:11.969 [ DEBUG]:mpegts: 578MHz in Freeview - add service 30D4 (null)
2017-03-19 01:42:11.969 [ TRACE]:mpegts: table: mux 0x7f4f11c546a0 add pmt 02/FF (2) pid 0020 (32)
2017-03-19 01:42:11.969 [ DEBUG]:mpegts: 578MHz in Freeview - open PID 0020 (32) [16/0x7f4ec00bded0]
2017-03-19 01:42:11.970 [ TRACE]:mpegts: table: mux 0x7f4f11c546a0 no fastswitch pmt 02/FF (2) pid 0020 (32)
2017-03-19 01:42:11.973 [ ALERT]:CRASH: Signal: 11 in PRG: /usr/bin/tvheadend (4.1-2449~gda58349-dirty) [5b207419081986c4e4c604cffa656f3436912477] CWD: /

#52 Updated by Jaroslav Kysela 7 months ago

Could you run tvh inside gdb and attach the backtrace? https://tvheadend.org/projects/tvheadend/wiki/Debugging#Basic-crash-debug

#53 Updated by Andy Gardner 7 months ago

Jaroslav Kysela wrote:

Could you run tvh inside gdb and attach the backtrace? https://tvheadend.org/projects/tvheadend/wiki/Debugging#Basic-crash-debug

I hope I got this right...

#54 Updated by Jaroslav Kysela 7 months ago

  • Status changed from Accepted to Fixed

#55 Updated by Jaroslav Kysela 7 months ago

  • Status changed from Fixed to Accepted

Could you try v4.1-2483-g1f3411c ?

#56 Updated by Andy Gardner 7 months ago

Jaroslav Kysela wrote:

Could you try v4.1-2483-g1f3411c ?

I did a git clone at the start but right now I'm manually updating files using wget. What command should I be using to do it properly please?

#57 Updated by Pablo Rodríguez 7 months ago

Andy Gardner wrote:

Jaroslav Kysela wrote:

Could you try v4.1-2483-g1f3411c ?

I did a git clone at the start but right now I'm manually updating files using wget. What command should I be using to do it properly please?

Yo should delete the directory "tvheadend" and then clone again. Finally, inside the folder "tvheadend" do ./Autobuild.sh

#58 Updated by Andy Gardner 7 months ago

Pablo Rodríguez wrote:

Andy Gardner wrote:

Jaroslav Kysela wrote:

Could you try v4.1-2483-g1f3411c ?

I did a git clone at the start but right now I'm manually updating files using wget. What command should I be using to do it properly please?

Yo should delete the directory "tvheadend" and then clone again. Finally, inside the folder "tvheadend" do ./Autobuild.sh

Thanks for that.

Current status: ota epg working, but only showing programme data for the services on the 2 muxes that (theoretically) carry the 7 day EIT for the entire network. I have EIT running on all 4 muxes, but no programme data shows up at all for the services on the 2 muxes that don't carry 7 day eit.

tbl-eit,epggrab debug output attached (this was being output even though I had EPG scan disabled for all muxes at the time)...

#59 Updated by Jaroslav Kysela 7 months ago

Could you a bit comment SID numbers (you can find the SID - service ID - numbers in the services grid) for the last EPG log? I need to know which SID numbers (services) are without0 EPG data or incomplete data. You can easily verify, if there are EIT entries in log for these SID numbers:

grep 'tbl-eit: eit: sid' tvheadend.log

#60 Updated by Andy Gardner 7 months ago

Here's and example of an EIT entry in the debug log that hasn't appeared in the EPG:

2017-03-23 09:56:35.495 [ TRACE]:tbl-eit: eit: sid 1206 tsid 001c onid 222a seg 01
2017-03-23 09:56:35.495 [ TRACE]:tbl-eit: eit: pid 12 tableid 4F extraid 00000000001c04b6 len 230
2017-03-23 09:56:35.495 [ TRACE]:tbl-eit: eit: section 0 last 1 ver 3 (ver 255 st 0 incomp 4 comp 0)
2017-03-23 09:56:35.495 [ TRACE]:tbl-eit: eit: new version, restart
2017-03-23 09:56:35.495 [ TRACE]:tbl-eit: 04 B6 C7 00 01 00 1C 22 2A 01 4F 01 43 E1 EA 18 ......."*.O.C...
2017-03-23 09:56:35.495 [ TRACE]:tbl-eit: 00 00 03 00 00 80 CF 54 02 20 00 4D 92 65 6E 67 .......T. .M.eng
2017-03-23 09:56:35.495 [ TRACE]:tbl-eit: 0A 05 42 72 65 61 6B 66 61 73 74 83 05 53 74 61 ..Breakfast..Sta
2017-03-23 09:56:35.495 [ TRACE]:tbl-eit: 72 74 20 79 6F 75 72 20 64 61 79 20 6F 66 66 20 rt your day off
2017-03-23 09:56:35.495 [ TRACE]:tbl-eit: 72 69 67 68 74 20 77 69 74 68 20 48 69 6C 61 72 right with Hilar
2017-03-23 09:56:35.495 [ TRACE]:tbl-eit: 79 20 42 61 72 72 79 2C 20 4A 61 63 6B 20 54 61 y Barry, Jack Ta
2017-03-23 09:56:35.495 [ TRACE]:tbl-eit: 6D 65 20 61 6E 64 20 74 68 65 20 42 72 65 61 6B me and the Break
2017-03-23 09:56:35.495 [ TRACE]:tbl-eit: 66 61 73 74 20 74 65 61 6D 2C 20 62 72 69 6E 67 fast team, bring
2017-03-23 09:56:35.495 [ TRACE]:tbl-eit: 69 6E 67 20 79 6F 75 20 74 68 65 20 6C 61 74 65 ing you the late
2017-03-23 09:56:35.495 [ TRACE]:tbl-eit: 73 74 20 69 6E 20 6E 65 77 73 2C 20 73 70 6F 72 st in news, spor
2017-03-23 09:56:35.495 [ TRACE]:tbl-eit: 74 73 20 61 6E 64 20 77 65 61 74 68 65 72 2E 55 ts and weather.U
2017-03-23 09:56:35.495 [ TRACE]:tbl-eit: 04 4E 5A 4C 00 50 06 F6 03 14 65 6E 67 50 06 F6 .NZL.P....engP..
2017-03-23 09:56:35.495 [ TRACE]:tbl-eit: 48 15 69 74 61 50 06 F5 03 01 65 6E 67 76 0B C4 H.itaP....engv..
2017-03-23 09:56:35.495 [ TRACE]:tbl-eit: 09 2F 31 30 34 36 30 31 35 30 76 0A C8 08 2F 31 ./10460150v.../1
2017-03-23 09:56:35.495 [ TRACE]:tbl-eit: 30 32 36 31 39 35 026195
2017-03-23 09:56:35.495 [ TRACE]:tbl-eit: svc='TVNZ 1 +1', ch='TVNZ 1 +1', eid= 323, tbl=4f, running=4, start=2017-03-22;18:00:00(+1300), stop=2017-03-22;21:00:00(+1300), ebc=(nil)

That program actually runs from 7am to 10am local time.

#61 Updated by Jaroslav Kysela 7 months ago

It looks like time(zone) issue again. Check, if your server settings has really correct time and timezone and try to return back the 'timezone' settings for tvh network to 'local time'.

  (7-13)+24 = 18 : start=2017-03-22;18:00:00(+1300)

#62 Updated by Andy Gardner 7 months ago

Jaroslav Kysela wrote:

It looks like time(zone) issue again. Check, if your server settings has really correct time and timezone and try to return back the 'timezone' settings for tvh network to 'local time'.

[...]

Before switching over to the dev version of tvheadend I had no problems with timezone.

#63 Updated by Jaroslav Kysela 7 months ago

The dev version allows to override the timezone but keeps the local time / UTC time functionaly. Here's the diff between 4.0 and 4.1 latest code:

-dvb_convert_date(const uint8_t *dvb_buf, int local)
+dvb_convert_date(const uint8_t *dvb_buf, int tmzone)
 {
   int i;
   int year, month, day, hour, min, sec;
@@ -433,7 +566,50 @@
   dvb_time.tm_isdst = -1;
   dvb_time.tm_wday = 0;
   dvb_time.tm_yday = 0;
-  return local ? mktime(&dvb_time) : timegm(&dvb_time);
+
+  if (tmzone == 0) /* UTC */
+    return timegm(&dvb_time);
+  if (tmzone == 1) /* Local time */
+    return mktime(&dvb_time);
+
+  /* apply offset */
+  return timegm(&dvb_time) - tmzone * 60;
+}

Enum table:

static struct strtab dvb_timezone_strtab[] = {
  { N_("UTC"),        0 },
  { N_("Local (server) time"), 1 },
....

Just try to return the time configuration back as in 4.0.

#64 Updated by Andy Gardner 7 months ago

OK. Here's the current status of the NZ Freeview EIT grabber module when run in CHCH.

If enabled only on the 562MHz (Mediaworks) mux, nothing happens
If enabled only on the 578MHz (TVNZ) mux, nothing happens
If enabled only on the 594MHz (Kordia) mux, the full EPG data for only the 594MHz Kordia mux is successfully grabbed
If enabled only on the 530MHz (WorldTV) mux, the full EPG data for only the 594MHz Kordia mux is successfully grabbed

As stated previous, the EPG data for all 4 muxes should be showing up on both the Kordia and WorldTV muxes.

Suggestions please?

#65 Updated by Jaroslav Kysela 7 months ago

Trace logs with the description?

The trace lines shows SID and TSID numbers. If you see 'svc=' line, then the EIT event is decoded and should be pushed to the database (unless the time is in past or something else is wrong). If you don't see 'svc=' line, then something is wrong and I need to know also TSID of the tuned mux (see mux grid).

Also note that tables 4E and 4F (tableid=) are for present/following events only. The full EPG info is in tables 5X and 6X.

I would start with 562Mhz, because it does not work. Watch one service (channel) from this mux when TVH is idle (no scan or EPG grab is running) and attach the logs between 'subscribe' and 'unsubscribe' operation. Give me TSID of 562Mhz mux, and describe one or more SID numbers which you see in the log but they are without EPG events.

#66 Updated by Nathan Fieldhouse 5 months ago

TvHeadend 4.2.2 has now appeared in LibreElec. I see the New Zealand grabber option but for me it didn't seem to do work at all. Is there something I can do to help contribute to this?

Also available in: Atom PDF