Project

General

Profile

Bug #5062

EPG problem with PASS profile

Added by saen acro 8 months ago. Updated 13 days ago.

Status:
Fixed
Priority:
Normal
Assignee:
Category:
EPG
Target version:
-
Start date:
2018-04-05
Due date:
% Done:

100%

Estimated time:
Found in version:
4.3.xxxx
Affected Versions:

Description

TVH1 dvb-c source EPG work perfect all channels EPG is OK.
TVH2 take streams from TVH1 via IPTV Automatic Network,
but channels don't have EPG on TVH2 ... Actually have EPG,
but it come, long time after EPG scan finish.

Same can be seen in VLC log EPG come 5+ second after channel start.
If same streams are streamed from MuMuDVB in place of TVH1,
"epg scan" take EPG on a moment.

Interconnect between TVH will solve this /already have request for this/
Or just EPG internal reader from URL. /will make such feature request if needed/

epg_trace.txt (271 KB) epg_trace.txt saen acro, 2018-04-06 16:02
epg_trace3.txt.bz2 (9.72 MB) epg_trace3.txt.bz2 saen acro, 2018-04-06 21:12
vlc.log (40.8 KB) vlc.log saen acro, 2018-04-07 21:33

Associated revisions

Revision 5f0241da (diff)
Added by Jaroslav Kysela 8 months ago

pass muxer: add rewrite NIT, fixes #5062

Revision 29140e43 (diff)
Added by Jaroslav Kysela 8 months ago

pass muxer: add more rewrites, fix NIT rewrite, fix teletext subs, fixes #5062, fixes #5075

Revision c45bfdc1 (diff)
Added by Jaroslav Kysela 2 months ago

pass muxer: rewrite also tsid/onid for eit table, fixes #5062

Revision 7e0cda03 (diff)
Added by Jaroslav Kysela 2 months ago

pass muxer: another fixed for EIT callback, issue #5062

History

#2 Updated by saen acro 8 months ago

I see this repeating message

2018-04-06 15:39:30.486 subscription: 0D3B: "scan" subscribing to mux "playlist - nova Sport", weight: 6, adapter: "IPTV #2", network: "TV Serv2", service: "Raw PID Subscription" 
2018-04-06 15:39:33.868 tbl-eit: eit: completed pid 18 table 00000000 / 00000000
2018-04-06 15:39:40.072 tbl-eit: eit: completed pid 18 table 00000000 / 00000000
2018-04-06 15:39:40.224 tbl-eit: eit: completed pid 18 table 00000000 / 00000000
2018-04-06 15:39:41.708 tbl-eit: eit: completed pid 18 table 00000000 / 00000000
2018-04-06 15:39:45.355 tbl-eit: eit: completed pid 18 table 00000000 / 00000000

will live it some more time.

#3 Updated by saen acro 8 months ago

On TVH2 I have DVB-S tunners
problematic are DVB-C from IPTV

#4 Updated by Jaroslav Kysela 8 months ago

I need to inspect traces - the standard logging does not show much for eit tables.

#5 Updated by saen acro 8 months ago

Sorry but Wiki image is outdated (!)

new log uploaded

#6 Updated by Jaroslav Kysela 8 months ago

There are EIT events in the log. Unfortunately, I cannot tell you what's wrong, because I don't know what you expect.

#7 Updated by Jaroslav Kysela 8 months ago

I suggest to configure only one test mux in the automatic network and analyze the behaviour of TVH2.

#8 Updated by saen acro 8 months ago

Im try to play via VLC from TVH2 and then from TVH1 same channel
both use PASS profile /with overwriting is impossible to be edited/
attaching log
no time stamps in vlc logs

just like pid=18 from iptv is cut or not used

#9 Updated by saen acro 8 months ago

One more detail
TVH1 send in stream only NOW/NEXT event EPG

#10 Updated by Jaroslav Kysela 8 months ago

Looking more to this: It seems that the EIT events are in EIT table 0x60 (other TS). Those events are FILTERED when you enable EIT rewrite (or SID rewrite which will implicitly enable all rewrites):

static void
pass_muxer_eit_cb(mpegts_psi_table_t *mt, const uint8_t *buf, int len)
{

....

  /* filter out the other transponders */
  if ((buf[0] < 0x50 && buf[0] != 0x4e) || buf[0] > 0x5f || len < 14)
    return;

#11 Updated by Jaroslav Kysela 8 months ago

Could you try this?

diff --git a/src/muxer/muxer_pass.c b/src/muxer/muxer_pass.c
index a6ea51dd1..d33c17826 100644
--- a/src/muxer/muxer_pass.c
+++ b/src/muxer/muxer_pass.c
@@ -240,8 +240,8 @@ pass_muxer_eit_cb(mpegts_psi_table_t *mt, const uint8_t *buf, int len)
   uint8_t *sbuf, *out;
   int olen;

-  /* filter out the other transponders */
-  if ((buf[0] < 0x50 && buf[0] != 0x4e) || buf[0] > 0x5f || len < 14)
+  /* filter out wrong tables */
+  if (buf[0] < 0x4e || buf[0] > 0x6f || len < 14)
     return;

   pm = (pass_muxer_t*)mt->mt_opaque;

#12 Updated by saen acro 8 months ago

Zero Effect, same as before :(

#13 Updated by Jaroslav Kysela 8 months ago

The traces shows the imported events nicely, so you can investigate this more yourself. The key is if EIT events are broadcasted in the mux where the service is broadcasted.

#14 Updated by saen acro 8 months ago

60sec curl download from both
command used

curl -m 60 --output tvh2.ts http://localhost:9981/stream/channelid/1826398699

Same channel but size is totally different ???

DVB-C >>> TVH1  >>>  TVH2  >>> Clients
               16Mb       21Mb (60sec)

DemuxToy TS Analyser
show lot of violations after second tvh

TVH1.TS
TVH2.TS

#15 Updated by Jaroslav Kysela 8 months ago

I do not have a time to do this analysis.

#16 Updated by saen acro 8 months ago

I'm start to be confused?
http://www.digitalekabeltelevisie.nl/dvb_inspector/ nice program work in windows and linux just need JAVA

record from TVH1

there is 0x4E (78) and 0x50 (80) events

but after stream go to TVH2

on output there is no EPG also no EPG inside

Astra send only 0x4E

and MuMuDVB send both 0x4E and 0x50

#17 Updated by Jaroslav Kysela 8 months ago

  • Status changed from New to Fixed
  • % Done changed from 0 to 100

#18 Updated by saen acro 8 months ago

Tested without success :(
No EPG on second TVH

#19 Updated by saen acro 2 months ago

Any way this to be fixed :(

#20 Updated by Jaroslav Kysela 2 months ago

Fixed in v4.3-1457-gc45bfdc1e . What about to stop crying and start coding ;-)

#21 Updated by saen acro 2 months ago

No positive result :(

Jaroslav Kysela wrote:

Fixed in v4.3-1457-gc45bfdc1e . What about to stop crying and start coding ;-)

I can do coding only on BASIC,
badly it is dead language but make "IF THEN ELSE" clear to post derivative languages ;)

#22 Updated by Jaroslav Kysela 2 months ago

On my side: TVH1 (captured from live stream):

[tsduck]$ cat ~/git/tvheadend/a.ts | tsp -P eit -o a.txt > /dev/null
[[email protected] tsduck]$ cat a.txt
Summary
-------
TS id:         1 (0x0001)
EITp/f actual: 15
EITp/f other:  0
EITs actual:   57
EITs other:    0

TS      Services  With EITp/f  With EITs  EPG days
------  --------  -----------  ---------  --------
Actual         1            1          1         0
Other          0            0          0         0

A/O  TS Id   Srv Id  Name     EITp/f  EITs  EPG days
---  ------  ------  -------  ------  ----  --------
Act  0x0001  0x0001  CT 1 HD  yes     yes          0

TVH2: The captured file as input (looped thus more events):

tsduck]$ cat ~/git/tvheadend/b.ts | tsp -P eit -o b.txt > /dev/null
[[email protected] tsduck]$ cat b.txt
Summary
-------
TS id:         1 (0x0001)
EITp/f actual: 25
EITp/f other:  0
EITs actual:   109
EITs other:    0

TS      Services  With EITp/f  With EITs  EPG days
------  --------  -----------  ---------  --------
Actual         1            1          1         0
Other          0            0          0         0

A/O  TS Id   Srv Id  Name     EITp/f  EITs  EPG days
---  ------  ------  -------  ------  ----  --------
Act  0x0001  0x0001  CT 1 HD  yes     yes          0

You must upgrade all tvh instances also wait at least 40 seconds - the new EPG detection mechanism usually waits for first 20 seconds to detect the right EIT grabber.

#23 Updated by saen acro 2 months ago

50% success now/next events is OK (0x4E (78)),
but week EPG is missing (0x50 (80))

Do I need to upload TS files?

#24 Updated by saen acro 2 months ago

both files is

# cat out1.ts | tsp -P eit -o 1.txt > /dev/null
# cat out2.ts | tsp -P eit -o 2.txt > /dev/null

# cat 1.txt
Summary
-------
TS id:         1 (0x0001)
EITp/f actual: 16
EITp/f other:  0
EITs actual:   52
EITs other:    0

TS      Services  With EITp/f  With EITs  EPG days
------  --------  -----------  ---------  --------
Actual         1            1          1         0
Other          0            0          0         0

A/O  TS Id   Srv Id  Name                  EITp/f  EITs  EPG days
---  ------  ------  --------------------  ------  ----  --------
Act  0x0001  0x0001  Discovery Channel HD  yes     yes          0

# cat 2.txt
Summary
-------
TS id:         1 (0x0001)
EITp/f actual: 48
EITp/f other:  0
EITs actual:   0
EITs other:    0

TS      Services  With EITp/f  With EITs  EPG days
------  --------  -----------  ---------  --------
Actual         1            1          0         0
Other          0            0          0         0

A/O  TS Id   Srv Id  Name                  EITp/f  EITs  EPG days
---  ------  ------  --------------------  ------  ----  --------
Act  0x0001  0x0001  Discovery Channel HD  yes     no           0

I'm guessing but, on first TVH I filter Audio subtitle streams, leaving local language only.
Is it possible this to brake EIT.

#25 Updated by Jaroslav Kysela 2 months ago

Try again code from comment # 11 (I fixed the wrong tsid/onid in EIT events recently and the current code filters out the EIT events for other transponders - your broadcaster might mark the events wrongly).

#26 Updated by saen acro 2 months ago

am not sure is it applied correctly

git diff
diff --git a/src/muxer/muxer_pass.c b/src/muxer/muxer_pass.c
index 28ec31d..37172ff 100644
--- a/src/muxer/muxer_pass.c
+++ b/src/muxer/muxer_pass.c
@@ -255,8 +255,8 @@ pass_muxer_nit_cb(mpegts_psi_table_t *mt, const uint8_t *buf, int len)
   uint8_t out[4096], *ob, dtag;
   int l, ol, lptr, dlen;

-  /* filter out the other networks */
-  if (buf[0] != 0x40)
+  /* filter out wrong tables */
+  if (buf[0] < 0x4e || buf[0] > 0x6f || len < 14)
     return;

   if (len < 10)

but result is

Summary
-------
TS id:         1 (0x0001)
EITp/f actual: 5
EITp/f other:  0
EITs actual:   0
EITs other:    0

TS      Services  With EITp/f  With EITs  EPG days
------  --------  -----------  ---------  --------
Actual         1            1          0         0
Other          0            0          0         0

A/O  TS Id   Srv Id  Name  EITp/f  EITs  EPG days
---  ------  ------  ----  ------  ----  --------
Act  0x0001  0x0001        yes     no           0

#27 Updated by saen acro 2 months ago

same channel as previous tests discovery channel hd

Summary
-------
TS id:         1 (0x0001)
EITp/f actual: 16
EITp/f other:  0
EITs actual:   16
EITs other:    0

TS      Services  With EITp/f  With EITs  EPG days
------  --------  -----------  ---------  --------
Actual         1            1          1         0
Other          0            0          0         0

A/O  TS Id   Srv Id  Name  EITp/f  EITs  EPG days
---  ------  ------  ----  ------  ----  --------
Act  0x0001  0x0001        yes     yes          0

no EPG exist

#28 Updated by Jaroslav Kysela 2 months ago

Modify pass_muxer_eit_cb() function not pass_muxer_nit_cb() with the code from comment # 11. Also, wait for few minutes to get EPG data - the complete data are broadcasted in chunks.

#29 Updated by saen acro 2 months ago

Jaroslav Kysela wrote:

Modify pass_muxer_eit_cb() function not pass_muxer_nit_cb() with the code from comment # 11. Also, wait for few minutes to get EPG data - the complete data are broadcasted in chunks.

On witch line?
https://github.com/tvheadend/tvheadend/blob/c45bfdc1ecc1287a9e4312c44d64a69dd1ff2580/src/muxer/muxer_pass.c#L339

#30 Updated by Petar Ivanov 2 months ago

#31 Updated by saen acro 2 months ago

Petar Ivanov wrote:

This lines edit: https://github.com/tvheadend/tvheadend/blob/c45bfdc1ecc1287a9e4312c44d64a69dd1ff2580/src/muxer/muxer_pass.c#L346L347

in post #26 you show how edit function pass_muxer_nit_cb

#26 is wrong is I understand

and patch from #11 is different or I can apply it

/@Petar Ivanov драсни на мейла да си трошиме сървърите по лесно и бързо ;) /

#32 Updated by Jaroslav Kysela 2 months ago

Replace just if () statement with the line from comment # 11 (minus before line means remove, plus means add):

-  /* filter out the other transponders */
-  if ((buf[0] < 0x50 && buf[0] != 0x4e) || buf[0] > 0x5f || len < 14)
+  /* filter out wrong tables */
+  if (buf[0] < 0x4e || buf[0] > 0x6f || len < 14)
     return;

https://github.com/tvheadend/tvheadend/blob/c45bfdc1ecc1287a9e4312c44d64a69dd1ff2580/src/muxer/muxer_pass.c#L346-L347

#33 Updated by saen acro 2 months ago

diff --git a/src/muxer/muxer_pass.c b/src/muxer/muxer_pass.c
index 28ec31d..bea6279 100644
--- a/src/muxer/muxer_pass.c
+++ b/src/muxer/muxer_pass.c
@@ -343,8 +343,10 @@ pass_muxer_eit_cb(mpegts_psi_table_t *mt, const uint8_t *buf, int len)
   uint8_t *sbuf, *out;
   int olen;

-  /* filter out the other transponders */
-  if ((buf[0] < 0x50 && buf[0] != 0x4e) || buf[0] > 0x5f || len < 14)
+  /* filter out wrong tables */
+  if (buf[0] < 0x4e || buf[0] > 0x6f || len < 14)
+  /* filter out the other transponders
+  if ((buf[0] < 0x50 && buf[0] != 0x4e) || buf[0] > 0x5f || len < 14) */
     return;

   pm = (pass_muxer_t*)mt->mt_opaque;

hope is correct

#34 Updated by saen acro 2 months ago

# curl -m 60 --output out3.ts http://172.31.90.254:9981/stream/channelid/1792384735
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 59.1M    0 59.1M    0     0  1009k      0 --:--:--  0:01:00 --:--:-- 1019k
curl: (28) Operation timed out after 60000 milliseconds with 62031728 bytes received

# cat out3.ts | tsp -P eit -o 4.txt > /dev/null

# cat 4.txt
Summary
-------
TS id:         1 (0x0001)
EITp/f actual: 32
EITp/f other:  0
EITs actual:   0
EITs other:    0

TS      Services  With EITp/f  With EITs  EPG days
------  --------  -----------  ---------  --------
Actual         1            1          0         0
Other          0            0          0         0

A/O  TS Id   Srv Id  Name  EITp/f  EITs  EPG days
---  ------  ------  ----  ------  ----  --------
Act  0x0001  0x0001        yes     no           0

even channel name is CUT

#35 Updated by Jaroslav Kysela 2 months ago

wait more time, 60 seconds might not be enough, try 'curl -m 300'

#36 Updated by saen acro 2 months ago

Little bit more time.

# curl -m 600 --output tvh2_.ts http://172.31.90.254:9981/stream/channelid/1792384735
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  595M    0  595M    0     0  1015k      0 --:--:--  0:09:59 --:--:-- 1016k
curl: (28) Operation timed out after 600000 milliseconds with 624207940 bytes received

# cat tvh2_.ts | tsp -P eit -o 5.txt > /dev/null

# cat 5.txt
Summary
-------
TS id:         1 (0x0001)
EITp/f actual: 600
EITp/f other:  0
EITs actual:   376
EITs other:    0

TS      Services  With EITp/f  With EITs  EPG days
------  --------  -----------  ---------  --------
Actual         1            1          1         0
Other          0            0          0         0

A/O  TS Id   Srv Id  Name  EITp/f  EITs  EPG days
---  ------  ------  ----  ------  ----  --------
Act  0x0001  0x0001        yes     yes          0

But in fact there is some wrong in this
first TVH with take from DVB-C get EPG for week in less then 2 seconds per mux
on second TVH EPG scan give up after first stream

2018-10-11 21:42:36.915 mpegts: channels - БНТ 1 in Dalekoo - tuning on IPTV #1
2018-10-11 21:42:36.916 subscription: 12C0: "epggrab" subscribing to mux "channels - БНТ 1", weight: 4, adapter: "IPTV #1", network: "Dalekoo", service: "Raw PID Subscription" 
2018-10-11 21:43:21.914 epggrab: EIT: DVB Grabber - data completion timeout for channels - БНТ 1 in Dalekoo
2018-10-11 21:43:21.914 subscription: 12C0: "epggrab" unsubscribing

#37 Updated by Jaroslav Kysela 2 months ago

saen acro wrote:

first TVH with take from DVB-C get EPG for week in less then 2 seconds per mux

I doubt. The speed must be identical.

2018-10-11 21:43:21.914 epggrab: EIT: DVB Grabber - data completion timeout for channels - БНТ 1 in Dalekoo

I tried to fix that in v4.3-1461-g7e0cda035 (inserting zero events to keep the SI tables consistent).

#38 Updated by saen acro 2 months ago

nothing changed, just messages:

2018-10-12 15:55:40.943 mpegts: channels - БНТ 1 in Dalekoo - tuning on IPTV #1
2018-10-12 15:55:40.943 subscription: 1BE8: "epggrab" subscribing to mux "channels - БНТ 1", weight: 4, adapter: "IPTV #1", network: "Dalekoo", service: "Raw PID Subscription" 
2018-10-12 15:56:20.999 tbl-eit: eit: channels - БНТ 1 in Dalekoo: invalid checksum (len 79, errors 1)
2018-10-12 15:56:25.942 epggrab: EIT: DVB Grabber - data completion timeout for channels - БНТ 1 in Dalekoo
2018-10-12 15:56:25.942 subscription: 1BE8: "epggrab" unsubscribing

#39 Updated by saen acro about 2 months ago

Oct 15 13:12:14 tvserv tvheadend[4168]: tbl-eit: eit: playlist - БНТ 1 HD in TV Serv2 Dalekoo: invalid checksum (len 166, errors 587)
Oct 15 13:12:23 tvserv tvheadend[4168]: tbl-eit: eit: playlist - nova in TV Serv2 Dalekoo: invalid checksum (len 89, errors 1667171)
Oct 15 13:12:23 tvserv tvheadend[4168]: tbl-eit: eit: playlist - bTV HD in TV Serv2 Dalekoo: invalid checksum (len 102, errors 24268)
Oct 15 13:12:26 tvserv tvheadend[4168]: tbl-eit: eit: playlist - БНТ 1 HD in TV Serv2 Dalekoo: invalid checksum (len 132, errors 590)
Oct 15 13:12:33 tvserv tvheadend[4168]: tbl-eit: eit: playlist - nova in TV Serv2 Dalekoo: invalid checksum (len 145, errors 1667791)
Oct 15 13:12:33 tvserv tvheadend[4168]: tbl-eit: eit: playlist - bTV HD in TV Serv2 Dalekoo: invalid checksum (len 95, errors 24284)
Oct 15 13:12:43 tvserv tvheadend[4168]: tbl-eit: eit: playlist - nova in TV Serv2 Dalekoo: invalid checksum (len 143, errors 1668723)
Oct 15 13:12:45 tvserv tvheadend[4168]: tbl-eit: eit: playlist - bTV HD in TV Serv2 Dalekoo: invalid checksum (len 102, errors 24309)
Oct 15 13:12:50 tvserv tvheadend[4168]: tbl-eit: eit: playlist - БНТ 1 HD in TV Serv2 Dalekoo: invalid checksum (len 166, errors 593)
Oct 15 13:12:54 tvserv tvheadend[4168]: tbl-eit: eit: playlist - nova in TV Serv2 Dalekoo: invalid checksum (len 124, errors 1669141)
Oct 15 13:12:57 tvserv tvheadend[4168]: tbl-eit: eit: playlist - bTV HD in TV Serv2 Dalekoo: invalid checksum (len 102, errors 24325)
Oct 15 13:13:00 tvserv tvheadend[4168]: tbl-eit: eit: playlist - БНТ 1 HD in TV Serv2 Dalekoo: invalid checksum (len 122, errors 596)
Oct 15 13:13:04 tvserv tvheadend[4168]: tbl-eit: eit: playlist - nova in TV Serv2 Dalekoo: invalid checksum (len 177, errors 1669603)
Oct 15 13:13:08 tvserv tvheadend[4168]: tbl-eit: eit: playlist - bTV HD in TV Serv2 Dalekoo: invalid checksum (len 63, errors 24331)
Oct 15 13:13:10 tvserv tvheadend[4168]: tbl-eit: eit: playlist - БНТ 1 HD in TV Serv2 Dalekoo: invalid checksum (len 102, errors 603)
Oct 15 13:13:14 tvserv tvheadend[4168]: tbl-eit: eit: playlist - nova in TV Serv2 Dalekoo: invalid checksum (len 169, errors 1670187)
Oct 15 13:13:19 tvserv tvheadend[4168]: tbl-eit: eit: playlist - bTV HD in TV Serv2 Dalekoo: invalid checksum (len 102, errors 24333)
Oct 15 13:13:24 tvserv tvheadend[4168]: tbl-eit: eit: playlist - nova in TV Serv2 Dalekoo: invalid checksum (len 49, errors 1670720)
Oct 15 13:13:25 tvserv tvheadend[4168]: tbl-eit: eit: playlist - БНТ 1 HD in TV Serv2 Dalekoo: invalid checksum (len 102, errors 607)
Oct 15 13:13:32 tvserv tvheadend[4168]: tbl-eit: eit: playlist - bTV HD in TV Serv2 Dalekoo: invalid checksum (len 102, errors 24337)
Oct 15 13:13:34 tvserv tvheadend[4168]: tbl-eit: eit: playlist - nova in TV Serv2 Dalekoo: invalid checksum (len 143, errors 1671222)
Oct 15 13:13:37 tvserv tvheadend[4168]: tbl-eit: eit: playlist - БНТ 1 HD in TV Serv2 Dalekoo: invalid checksum (len 85, errors 610)

There is nonstop messages as this coming from tvh

#40 Updated by saen acro 13 days ago

(x) Still not FIXED

Also available in: Atom PDF