Project

General

Profile

Bug #5995

opentv: sky uk missing guide data

Added by Thomas Bygrave 3 months ago. Updated 2 months ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
EPG - Grabbers
Target version:
-
Start date:
2021-01-17
Due date:
% Done:

0%

Estimated time:
Found in version:
4.3.0~pre+202012291955-0~built202012292042~gitaaca05cc1
Affected Versions:

Description

When I'm grabbing EPG data from opentv: sky uk guide data the EPG is only partially populated with complete guide data but some channels have the wrong guide data. I have set the EPG grabber module to be used on 11778V to opentv: sky uk


Files

Program Missing Data.PNG (41.3 KB) Program Missing Data.PNG description is missing Thomas Bygrave, 2021-01-17 14:56
EPG Timeline missing data.PNG (88.7 KB) EPG Timeline missing data.PNG timeline view on webui where season and episode data is missing Thomas Bygrave, 2021-01-17 14:56
Wrong EPG Data.PNG (134 KB) Wrong EPG Data.PNG wrong EPG entry Thomas Bygrave, 2021-01-17 14:56
packet.txt (7.67 KB) packet.txt LraiZer @ukcvs, 2021-02-07 22:52

History

#1

Updated by Dave Pickles 3 months ago

Confirmed.

My system, running current master, has tuners for DVB-T and DVB-S, and collects EPG from UK Freeview and Freesat with grabbers set to default priorities. If I enable the OpenTV Sky UK grabber with default priority, I can collect EPG for Sky channels which are not part of Freesat (for example CNN HD). However although the times and titles look correct, the descriptions are completely wrong - they are from different programmes on different channels but I can't work out where.

#2

Updated by Thomas Bygrave 3 months ago

yeah i get the same issue I also get random season and episode data attached to movie information. i think the issue is there implementation of the opentv EPG as I didn't get this issue on windows with EPG collector.
https://drive.google.com/drive/folders/18FN9_JTpbz9SwgqQvzKFBavOAcbSttP4?usp=sharing this is a link to my log file if it will help anyone work out what's wrong

#3

Updated by Dave Pickles 3 months ago

Thanks for the log which matches what I'm seeing.

I'm going to build another Tvheadend instance using just the OpenTV grabber to simplify things.

One oddity I've noticed. Taking a typical entry from your log:

2021-01-17 15:03:21.141 [  DEBUG]:opentv: find by time start 1611334800 stop 1611338400 eid 431 = 0x7f421e9faf20
2021-01-17 15:03:21.141 [  DEBUG]:opentv:     title 'Diabolical: Deadly Love'
2021-01-17 15:03:21.141 [  DEBUG]:opentv:   summary 'An international news bulletin, presented live from the newsroom.'

the title field comes before the summary. However in the code at src/epggrab/module/opentv.c the summary is output before the title.
#4

Updated by Thomas Bygrave 3 months ago

Any news on what could be the issue with the opentv module?

#5

Updated by Dave Pickles 3 months ago

I set up another TVH instance on a RPi3 using only the OpenTV grabber. That gave the same result but had several data overrun errors in the log (the "too much queued table input data (over 2MB), discarding new" error). Switching to a Pi4 gave the same errors. Installing TVH on an old laptop with a Core i3 processor fixed the overflow errors but there are still a lot of events with missing summaries and a few where the wrong summary is shown.

I haven't been able to find a description of the (proprietary, reverse-engineered) OpenTV encoding format, but from the code it seems that the title and summary fields are broadcast on separate PIDs and so the grabber must try to match them up; presumably it is this which is failing. However the code has been part of TVHeadend for many years and presumably worked in the past, and there have been no recent changes.

In short, I'm stumped.

#6

Updated by Thomas Bygrave 3 months ago

From what I've been searching PID 17 is for channel list, PIDs 48 - 55 is for titles and PIDs 64 - 71 is for summaries these where found from crossepg source code whic is used on the linux STBs and apparently they're still working so i would imagine that its the way TVheadend is handling the data incorrectly thus giving us errors in the guide data

#7

Updated by Thomas Bygrave 3 months ago

The "too much queued table input data (over 2MB), discarding new" error is possibly where it's getting stuck in a loop because my collection is timing out even when I've got the setting set to 7200

#8

Updated by LraiZer @ukcvs 2 months ago

There is a descriptor_tag 0xd0 with length 1 for certain "New:" prepended program titles in the opentv summary data. This is transmitted just before the huffman summary string and the series link id. Maybe this is causing your issue in the rlen+2 code? I currently have a similar issue of missing summaries for these "New:" programs with the built-in reader on my Linux STB, which i also need to modify this part of the code a little to read this descriptor correctly.

#9

Updated by Dave Pickles 2 months ago

Interesting, thanks. I tried going back to the last Tvheadend version which was reported working in the issues log (4.3.1605) but still hit the same problem. If there was a change in the OpenTV format that would explain it.

I then tried forking your "RadiotimesXmltvEmulator" (https://github.com/dave-p/openTVtoXML) to feed into Tvheadend. That works, but is more cumbersome to use and doesn't capture all the information.

Also available in: Atom PDF