XMLTV Not Detecting OTA Channels

Ben K

Okay...here's what I'm trying to do.

I have an IPTV Provider. They have links to an m3u playlist and xmltv file for EPG data.

Import the playlist, add a script to auto-fetch EPG data every day and copy so TVHE can read it...no problemo.

To fill in the gaps in local news and sporting events, we've also got a HDHomerun 2-channel OTA TV Tuner.

I've got this set up an ATSC-T Tuner, using us-ATSC-center-frequencies-8VSB-062009.

Scan the muxes, it finds all of the appropriate channels, etc.

Now, here's where I'm running into trouble. I cannot, no matter how I go about it, get EPG data to load for my OTA channels.

Initially, I just tried checking the Over-the-air EPG box on the TV Adapters themselves...no joy.

(It should be noted that I'm doing this in USA/Wisconsin, so AFAIK, OTA EPG is a no-go)

So, next idea was to use zap2xml to generate my local listings, then drop that in my /data folder and use tv_grab_wg to find both the IPTV and local listings .xml file. Looking at the script, it seems that should work.

But it doesn't.

SO, next idea is to write some PHP that takes the two XML files, parse them out, merge them into one file, and use THAT with tv_grab_file.

Still doesn't work. No matter what I do, I cannot seem to get these extra channels listed in my (valid) tvXML file to populate into TVHE.

For reference, I've attached the "merged" XML file I'm creating, as well as a screenshot of what I'm seeing in my channels list...

I've been working at this for like two days now, and I cannot for the life of me figure out what I'm missing or doing wrong. Any help would be greatly appreciated. :D

M. Bergmann

Are the two xmls working if you import them separately? Because the first part of your xml doesn't use wrap line, the second part does.

Can you post examples of the two xmls BEFORE merging them together (but just for one or two days to reduce the file size :) ).

Furthermore there are entries like
<programme start="20170905160000 -0400" stop="20170905163000 -0400" channel="I7.2.14807.tvguide.com">

Is this correct? I mean, you haven't named your channels "I7.2.14807" or "I55597" ??? Maybe he can't allocate these entries to the channels?

Ben K

Those channel names are auto-generated by zap2xml. I did see when trying to import the guide into Plex that it was having difficulty matching up the names when they're using the tvguide/zap2it URL names...but it didn't fail entirely.

Um, the "first part" xml, with no line-wraps, that is parsed and imported just fine. It's only the datawith line breaks that are problematic.

So, the individual file with no line breaks, from my IPTV provider, that works fine. It'sjust the data generated by Zap2XML.

What's odd is that I've seen numerous other guides that discuss using Zap2XML with no issues, so I wouldn't think that the programme referencing the channel by it's URL id would be that big of an issue.

The next step I guess would be to make my script go through all the programmes and replace the channel ID's with their display names, and see if that helps it see the stuff...

I'd post sample files, but they're literally identical to what's in the merge.

Robert Cameron

There is no need to merge the xmltv files, as Tvheadend can handle multiple xmltv files fine, as long as they come from separate grabbers. Since it looks like you're using something like tv_grab_file or similar, you can just make a copy of it, rename it, and edit to script to point to the location of your second xmltv file. Then, after your internal grabber has one once to populate the EPG Grabber Channels, assign the EPG channels from the second xmltv file to your OTA channels.

Another option is to use Schedules Direct for your OTA Channels. When creating a lineup for your OTA channels there is an option in their JSON API to upload the lineup.json file pulled from your HDHR tuner to automatically add the proper OTA lineup to your account.

In reference to the "id" attributes of the xmltv file: the "id" attribute of the programme element is not the channel name, but rather a reference to the "id" of a complementary channel element in the xmltv file. The channel's names and/or number are defined in the channel element, and the "id" attribute is used to match channels to programmes.

Ben K

Hey, thanks, that totally did the trick!

Reason I was slightly hesitant to copy the script was because I'm running TVHE in a docker container, so wanted to avoid modifying the og container if possible. But, it works, so I'll take it.

Curious...does tv_grab_wg not importall xml files in the /data/ directory? I could have sworn that I tried setting that one up to scrape all the xml files I was throwing at it, but had no luck. Interesting, regardless.

So, that should get me going...thanks again. I've got local channels 100% mapped to epg sources, and about 90% of my 500+ IPTV channels mapped. Any tips on mapping the other 10%? Would Schedules Direct work for that? A lot of them are UK/Sky TV stuff, and it doesn't look like Schedules Direct supports that? I may tinker around with the JSOn lineup you suggest, see if I can pull 100% of the epg from one source. That's be super. :D

K Shea

This information in this article, which I have mentioned before, helped me get zap2xml working with Tvheadend

https://freetoairamerica.wordpress.com/2014/12/03/some-hints-for-getting-free-to-air-satellite-channels-into-the-electronic-program-guide-in-kodi-or-xbmc-or-another-frontend/

However I had thought that the Zap2it service only covers the USA and Canada, so I am not at all sure where you'd find guide data for the UK stuff.

Quite some time back I found or wrote (don't recall which, though probably found since I am not really a programmer) a very quick and dirty Perl script to combine multiple xmltv format files into one. You would need to change the feeds-n filenames, and use more or fewer sections as required, this is just to give an idea of how it can be done. I am certain there are better ways to do this (and probably better languages to use for the purpose than Perl) but at the time I was just trying to fine a way to combine xmltv files and I didn't really care if the code was inefficient, as long as it worked. I post it here not as a shining example of how to do this, but hopefully more as an inspiration to find or create a better script to do this, if you happen to have the programming talents that I sadly lack.


#!/usr/bin/perl
use strict;

open my $OUT,'>','tv_grab_file.xmltv' or die "$!";
open my $IN ,'<','feeds-1.xml'   or die "$!";

while (<$IN>) {
  chomp;
  if ($_ ne "</tv>") {
      print $OUT "$_\n";
  }
}

close $IN;

open my $IN ,'<','feed-2.xml'   or die "$!";

my $i=0;
while (<$IN>) {
  chomp;
  if ( $i++ > 3 ) {
    if ($_ ne "</tv>") {
        print $OUT "$_\n";
    }
  }
}

close $IN;

open my $IN ,'<','feeds-3.xml'   or die "$!";

my $i=0;
while (<$IN>) {
  chomp;
  if ( $i++ > 3 ) {
    if ($_ ne "</tv>") {
        print $OUT "$_\n";
    }
  }
}

close $IN;

open my $IN ,'<','feeds-n.xml'   or die "$!";

my $i=0;
while (<$IN>) {
  chomp;
  if ( $i++ > 3 ) {
    if ($_ ne "</tv>") {
        print $OUT "$_\n";
    }
  }
}

close $IN;

open my $IN ,'<','feeds-last.xml'   or die "$!";

my $i=0;
while (<$IN>) {
  chomp;
  if ( $i++ > 3 ) {
    if ($_ ne "</tv>") {
        print $OUT "$_\n";
    }
  }
}

close $IN;
print $OUT '</tv>';
close $OUT;

G Kazaroth

What you did was awesome and congratulations on getting both to work. In case you are interested, I have an automated script (on Ubuntu) that will check and install the zap2xml portion for you correctly instead of using the manual procedure. It also updates the categories to be compatible with tvheadend in case you need that. Anyway, the link is below.

https://github.com/rocky4546/script.xmltv.tvheadend/wiki/Guide:-How-to-Setup-XMLTV-for-TVHeadEnd

Ben K

Ha! That's funny right there.

I just stumbled across your git while doing some homework, I've actually got it pulled to my test box. What I think I'm going to wind up doing is using WebGrab+Plus in conjunction with Zap2XML to grab both my OTA listings, as well as an exact list of data for my provider, as their own EPG is quite lacking.

Of course, first I need to settle on an IPTV provider that works well in the US. The first couple I've tried have been lacking in either the data they provide for tools like TVHE, or stream reliability.