EPG (XMLTV) for NZ

Bas van Rijn

Hi, I want to add an XMLTV grabber for NZ.
In NZ there is no option (anymore) to use a xmltv grabber to retrieve epg from the internet.
Instead we can retrieve the epg data through dvb-t.
The result I get is in tvheadend with my solution, is that all epg data is updated by tvheadend, except for the titles. When I click on a record (title = ".") the popup screen shows the description of the program.

Do you know how I can get around this problem and update the titles?

I will explain how I retrieve epg data:
A script in a cron job that is called weekly. This script does the following:
1. Kill tvheadend to free dvb-t multiplexers.
2. Get epg data from dvb-t (mheg). I use mheg5grab.sh script. This script utilises redbutton-download. (see http://sourceforge.net/projects/mheg2xmltv/files/) By calling mheg5grab.sh epg data is retrieved from dvb-t and stored on file.
3. Convert epg to xml. With mheg2xmltv.sh the data is converted to xml and stored in /tmp/xmltv.xml

4. Start tvheadend

Tvheadend is configured to execute "tv_grab_file_nz"
I got this script from http://code.google.com/p/tv-grab-file/. This script that I called "tv_grab_file_nz" is stored in the xmltv directory /usr/bin/.
I modified it so that it grabs the xml file from /tmp/
This is the result (in the log file) when tvheadend is restarted:
Aug 15 18:35:28 xmltv: Grabbing "tv_grab_file_nz reads /tmp/xmltv.xml file" using command "/usr/bin/tv_grab_file_nz"
Aug 15 18:35:28 xmltv: /usr/bin/tv_grab_file_nz: completed, took 0 seconds
Aug 15 18:35:28 xmltv: /usr/bin/tv_grab_file_nz: Parsing completed. XML contained 21 channels, 4264 events, 0 new events injected in EPG

Can you explain why the script says that no events are injected?

I have added a file with a small section of the xml file.

When I kill tvheadend the multiplexers are still locked. This stops my script (mheg5grab.sh) from executing.
Is there an alternative to free dvb-t multiplexers?

Joshua Cave

I have the exact same problem... channels loaded, events loaded, 0 events injected.. Hopefully we can get to the bottom of this.

Edit: this problem has come up a lot before but from what I can see it stems from not having a stop time in our EPG data, however, we do have a stop time so its not that.

Campbell Long

Hey guys,

Any update on this? I'm running XBMC & Tvheadend here in NZ and have most things configured the way I want with the exception of the EPG.

Thnx

Hein Rigolo

from what I can remember tvheadend tries to match the xmltv feed with the EPG that it received from the EIT tables. When the EIT table data is "larger" then the xmltv feed then tvheadend will keep that data.

So, if you run a latest version of tvheadend, there is a new option to disable the EIT EPG completely for certain channels. If you have an XMLTV feed then that might be used. You could try that out to see if that is a way around your problem.

Nick Rout

It is not true that there is no xmltv grabber for NZ. You can use tv_grab_xml-py

Nathan Fieldhouse

I am new to this, I dont know how to install/use the script tv_grab_xml-py, I am using OpenELEC 2.0 beta 4 with TvHeadEnd 2.1.3 and it works well bar the EPG. Can anyone give me any pointers how to use this grabber.

Thanks

Hein Rigolo

google for : tv_grab_nz-py

You will see a lot of details related to mythtv and this grabber. The concept is the same but instead of going through mythtv make sure you can run this grabber as a standalone script that will output a xmltv file. If that is working, then you have 2 options:

1) Let tvheadend run the script and insert the data directly into tvheadend

2) run the script using a cron job and then use the tv_grab_file "grabber" to load the xmltv file into tvheadend.

But first things first ... make sure you can run the tv_grab_nz-py grabber and that it outputs a xmltv file to disc.

Hein

Richard Leckinger

Hein is spot on. Here is a bit more info having just got it working using Sutton's Stable PPA 3.2~precise:

google for and download tv_grab_nz-py script
Open it with gedit and/or nano and satisfy yourself it's not malicious. Or not.
put tv_grab_nz-py into /home/hts/.xmltv
Do this because your user has rights to do stuff there, and so does tvheadend.
The tv_grab_file is already configured to look for the xmltv file there as well. Why edit things and fight it?

Tell tv_grab_nz-py what channels you want epg info about. Note this is NOT done as root:

$ ./tv_grab_nz-py --configure

Follow the prompts and say yes to all the channels you want EPG for and no to those you don't.
As at this writing, The Shopping Channel epg isn't yet integrated. Great cultural loss, I know!

Now that it is configured, ask it to write the output to a file instead of the terminal:

$ ./tv_grab_nz-py --output tv_grab_file.xmltv

Make sure you use this output file name, as the tv_grab_file script is already configured to grab a file with that name directly from the /home/hts/.xmltv folder.

Open the file with gedit/nano and familiarise yourself with the output.

google for and download tv_grab_file
I downloaded it to the same .xmltv folder for convenience.
Open it with gedit and/or nano and satisfy yourself it's not malicious. Or not.

sudo it into /usr/bin:

$ sudo cp /home/hts/.xmltv/tv_grab_file /usr/bin/tv_grab_file

Make it executable:

$ sudo chmod +x /usr/bin/tv_grab_file

Once it's in there, you need to restart tvheadened so that it sees it:

$ sudo service tvheadend restart

Use the browser to go back into tvheadend through the normal interface and configure:
Configuration>EPG Grabber>General Config> tick all three
Configuration>EPG Grabber>Internal Grabber>Module> find XMLTV: tv_grab_file is a simple grabber.....etc
Configuration>EPG Grabber>Grab interval>Set as you like, I left the default: 12 Hours
Configuration>EPG Grabber>Over the air grabbers> disable all

Configuration>EPG Grabber>External Interfaces> disable all

I also unticked the EPG boxes next to all the channels in my Configuration>TV adapters>Services tab, although I have no idea if this is really necessary.
Despite another tvheadend restart, the EIT had already built up an incomplete, crappy EPG database which prevented the shiny new xmltv file from getting loaded.
I used the suggestion from this tvheadend wiki thread:
https://www.lonelycoder.com/redmine/boards/5/topics/5937?r=5971#message-5971

..and killed off the existing EPG database and restarted tvheadend, and all was well.

"So, I stopped tvheadend, deleted /home/hts/.hts/tvheadend/epgdb.v2 (the EPG database file), and re-started the daemon again..."

Finally, I finished off with Hein's second suggestion, setting up a cron job in my own user's crontab:

$ crontab -e

I added these two lines (see man crontab for their meaning and adjust timing to suit)

50 15 /home/hts/.xmltv/tv_grab_nz-py --output /home/hts/.xmltv/tv_grab_file.xmltv 50 03 /home/hts/.xmltv/tv_grab_nz-py --output /home/hts/.xmltv/tv_grab_file.xmltv

I added the VERY necessary extra line to the crontab and saved it. (see man crontab for why an extra line is VITAL)
These times were chosen so I could see the cron job work and the output file get replaced.

It's all good and I hope spelling this out is useful to the next kiwi building their own PVR...;)

This config does NOT have the 'U' channel import error that Spencer logged here:

https://www.lonelycoder.com/redmine/boards/12/topics/4120

Adam Sutton

This seems a bit over the top. You should be able to simply run the nz script directly every 12 hours from within TVH.

If you cannot use the script directly, but can use the grab_file script then something is very wrong.

If you intend to run from a cronjob (to provide more flexibility, finer time control, script pipe lining. etc..) then I suggest you take a look at the external (unix domain socket) interfaces. This would simply some of your configuration I think.

Adam

Richard Leckinger

Adam is right of course, my post above is over the top. I wasn't able to get Hein's first recommendation to work, so I munged it into his second.

The tv_grab_nz-py script will work within Tvheadend providing you drop it into /usr/bin, make it executable and run it with --configure.

the script is not a true xmltv scraper but is calling an xml file that is scraped elsewhere. It's not entirely up to date - it's missing a few channels, but it works!

Nathan Fieldhouse

Shame the NZ XML TV server has gone. Need to find a script to extract the Mheg 5 data to XML TV file to import.

Nick Rout

tv_grab_nz-py still works, but you need to change the address of the xml feed to http://epg.org.nz/freeview.xml.gz

Nathan Fieldhouse

Thanks for that I have a guide again, been without it for months which kinda made the whole setup very hardwork.

Nick Rout

No probs. If you have troubles again this xmltv service is regularly discussed on the mythtv-nz mailing list - especially when people go into a spin cos it's not working. Archives here http://www.gossamer-threads.com/lists/mythtv/mythtvnz/