How to build a PVR in the USA using HDHomerun Extend + Raspberry Pi 3 + Kodi + Tvheadend + XMLTV + Schedulesdirect

W Beauchamp · 2016-03-25T00:21:48+00:00

3/24/2016 Goal: set up a DVR for the over-the air TV available in Chicago, Illinois USA I didn't ever find any documentation that explained completely what...

G Kazaroth

Your question is how to setup TVHeadend using zap2it.
1) Grap the zap2xml.pl file from the URL above and place in ~/xmltv/
2) Use attached .zap2xmlrc file. Set email/password and username and place in home folder ~
3) create an xmltv_default.xml file by running the command
./zap2xml.pl -D -S 3
rename resulting xmltv_default.xml to xmltv.xml
4) Now the key to making it all work... softlink the xmltv folder to where tvheadend is installed. For me it is installed in /home/hts (where the .hts folder is located.) To make the softlink type
sudo su - hts
ln -s /home/<user>/xmltv .xmltv
This should create a .xmltv folder in /home/hts pointing to where your xmltv.xml file is being generated.
5) In TVHeadend website, goto Configuration -> Channel / EPG -> EPG Grabber
6) I have the settings check "Update Channel Number". Periodically save ... = 0. Cron multiline: I have the following
0 5 0 8 0 12 *
Which pulls in the xmltv.xml file at 5am, 8am and 12 noon every day
BEFORE you click save, click on the 3 ^ in the bottom right corner. It brings up the tvheadend console window. Now click Save. If it works, you should see something like
xmltv: /usr/bin/tv_grab_file: channels tot= 55 new= 0 mod= 0
xmltv: /usr/bin/tv_grab_file: brands tot= 0 new= 0 mod= 0
xmltv: /usr/bin/tv_grab_file: seasons tot=22195 new=22195 mod=22195
xmltv: /usr/bin/tv_grab_file: episodes tot=22975 new= 9363 mod=17387
xmltv: /usr/bin/tv_grab_file: broadcasts tot=22975 new= 425 mod= 425
SUCCESS!

You now have your EPG filled with info which you can link to channels.

Additional info. First, you can use the xmltv.xml to download the icons, but I highly recommend finding good ones on the internet. The icons auto retrieved are fuzzy at best and if they are listed in the xmltv.xml, tvheadend will force their use. Just fill out the Configuration -> Channel / EPG -> Channels (icon column).
EX: "file:///home/<user>/xmltv/icons/4.1.png"

Also, my perl script I wrap around the zap2xml.pl will add more info to the description and sub-title, and map the genre to support Kodi/TVHeadend which causes pretty colors based on tv show type all over your guide based using what those apps expect.

Hope this helps.

Edward Crosby

George Kazaroth wrote:
> Your question is how to setup TVHeadend using zap2it.
> 1) Grap the zap2xml.pl file from the URL above and place in ~/xmltv/
> 2) Use attached .zap2xmlrc file. Set email/password and username and place in home folder ~
> 3) create an xmltv_default.xml file by running the command
> ./zap2xml.pl -D -S 3
> rename resulting xmltv_default.xml to xmltv.xml
> 4) Now the key to making it all work... softlink the xmltv folder to where tvheadend is installed. For me it is installed in /home/hts (where the .hts folder is located.) To make the softlink type
> sudo su - hts
> ln -s /home/<user>/xmltv .xmltv
> This should create a .xmltv folder in /home/hts pointing to where your xmltv.xml file is being generated.
> 5) In TVHeadend website, goto Configuration -> Channel / EPG -> EPG Grabber
> 6) I have the settings check "Update Channel Number". Periodically save ... = 0. Cron multiline: I have the following
> 0 5 > 0 8 > 0 12 *
> Which pulls in the xmltv.xml file at 5am, 8am and 12 noon every day
> BEFORE you click save, click on the 3 ^ in the bottom right corner. It brings up the tvheadend console window. Now click Save. If it works, you should see something like
> xmltv: /usr/bin/tv_grab_file: channels tot= 55 new= 0 mod= 0
> xmltv: /usr/bin/tv_grab_file: brands tot= 0 new= 0 mod= 0
> xmltv: /usr/bin/tv_grab_file: seasons tot=22195 new=22195 mod=22195
> xmltv: /usr/bin/tv_grab_file: episodes tot=22975 new= 9363 mod=17387
> xmltv: /usr/bin/tv_grab_file: broadcasts tot=22975 new= 425 mod= 425
> SUCCESS!
> You now have your EPG filled with info which you can link to channels.
>
> Additional info. First, you can use the xmltv.xml to download the icons, but I highly recommend finding good ones on the internet. The icons auto retrieved are fuzzy at best and if they are listed in the xmltv.xml, tvheadend will force their use. Just fill out the Configuration -> Channel / EPG -> Channels (icon column).
> EX: "file:///home/<user>/xmltv/icons/4.1.png"
> Also, my perl script I wrap around the zap2xml.pl will add more info to the description and sub-title, and map the genre to support Kodi/TVHeadend which causes pretty colors based on tv show type all over your guide based using what those apps expect.
>

> Hope this helps.

Fantastic. Thank you so much, @George Kazaroth

K Shea

George Kazaroth wrote:

> 6) I have the settings check "Update Channel Number". Periodically save ... = 0. Cron multiline: I have the following
> 0 5 > 0 8

> 0 12 > Which pulls in the xmltv.xml file at 5am, 8am and 12 noon every day I don't understand why you would do this. The listing are only updated once a day, so either you are pulling in the same* xmltv.xml file multiple times (which is kinda dumb, but harmless I guess) OR you are running the zap2xml program multiple times in a day, which is wasteful and selfish. The listings are only updated once a day, and we are using them in a manner not really intended, and if too many of us do this too many times in a day, the site operator may try to obfuscate the listings in such a way that the zap2xml program will no longer work. There is no good reason to ever run zap2xml more than once a day, and polite users will NOT run it right at the top of the hour (as they used to say in the radio business) but instead at some other random minute within a semi-random hour (early AM is probably good, or any time that's not during prime time anywhere in North America) so that not everyone is pounding their servers at once. People who think only of themselves when using this type of software are the reason some web-based service go away, since they do have to pay for server costs and data charges (probably minuscule for a single user running the software once a day, but if everyone gets greedy it could add up).

PLEASE do not hammer their servers, we'd like to see this service keep working!

G Kazaroth

Excellent point.:) Only grap the xmltv.xml once a day. I do it between 3-5am. The zap script works extremely well and you normally can count on it daily. How you run it also determines the load on the zap2it server, so lets take a look at the parameters from the .zap2xmlrc file attached above that help reduce the load. In the .zap2xmlrc, we set nsdays=0 (don't remove any cache days from the end) and ncsdays=2 (only remove 2 cache days from the front). This means that out of the 14 days that are cached, only 2 days are refreshed, so the other 12 days are reused and the server will not be contacted. On the command line, there is a "-S 3" This option causes the script to request information from the server at 3 second intervals. Without this option, the server is hammered at 100-200 requests per minute. Being nice helps everyone. ;)

So why am I pulling the same xmltv.xml 3 times a day if it is the same !#@$ file. Well, all you have to do is have TVHeadend miss a show due to not processing the file correctly, and you will do the same thing. Last issue I saw was last weekend, one week before the DST change. All shows shifted back one hour when the xmltv.xml file loaded on Sunday. So the 7pm show was going to record at 6pm. That would stink. Reloading the xmltv.xml file corrected the issue. NBC also has a nasty way of changing times 24 hours before they show. I have seen multiple times in Kodi 16 when the slots had 30 minute shows where it only partially updated and sections were left blank. I have not seen it on Kodi 17, but I have not had it long. Again, reloading multiple times fixed this issue. If you want to only load the file into TVHeadend once a day and even change Kodi from reloading "the exact same data" every 120 minutes to once a day, go for it. This type of issue is always considered very low priority and rarely fixed unless a higher issue somehow causes the fix to be implemented.

G Kazaroth

Sorry about the delay in getting the rest of the information. I was finishing up the development updates to Pandora for Kodi called Pandoki. Anyway, if you want to update the xmltv.xml to include the tvheadend genres, my project will do this. It is located at
https://github.com/rocky4546/script.xmltv.tvheadend/wiki With the files to pull by clicking on the Code tab.
Enjoy!

Edward Crosby

> Hope this helps.

So, following your direction, after step 6, from the console I see this:

2017-03-11 14:54:11.382 /usr/bin/tvgrab_na_tvmedia: grab /usr/bin/tv_grab_na_tvmedia
2017-03-11 14:54:11.386 spawn: Executing "/usr/bin/tv_grab_na_tvmedia"
2017-03-11 14:54:11.716 spawn: You need to configure the grabber by running it with --configure
2017-03-11 14:54:11.721 /usr/bin/tv_grab_na_tvmedia: no output detected
2017-03-11 14:54:11.721 /usr/bin/tv_grab_na_tvmedia: grab returned no data_

G Kazaroth

I am so sorry. It has been a while since I did the setup and I missed a step. You need to change tv_grab_na_tvmedia to use tv_grab_file. To do this on the TVHeadend website, go to
Configuration -> Channel / EPG -> EPG Grabber Modules
A list will appear. Go through it and make sure nothing is checked on the left side (just a round dot should be present instead of a round dot with a check inside)
Look for "Internal: XMLTV: tv_grab_file reads ..." and enable that. That line should now have a check on the left. Now when it runs, it should find the file.

Edward Crosby

Sorry. I may be a little dim. I'm not seeing what you are describing. All that I have is EPG Grabber not EPG Grabber Modules. Screenshot attached. I'm running 4.0.9-16~g63c1034~xenial, if that makes a difference.

G Kazaroth

OK, you are running 4.0.x and I am running 4.1.x, so we have different GUIs. I think yours does not have the module tab yet, so the grabber should be on the pull down in the internal grabber section called Modules.

Also, You need to disable the OTA Grabber section. I don't believe tvguides are provided on OTA anymore. I just clear that section out so it is empty and does not impact your service. You can also do what you want for the top check boxes, but after a bit, you may want to have control over what the channel name says on your tvguide and the icons I don't have coming from the xmltv.xml file.

Hope this helps.

Edward Crosby

So, I was trying to figure out how to get 4.1 on my Ubuntu 16.04 machine a few days ago and could never figure out how to get it. Here is my source list, I don't remember where I got this from:

deb TVHEADENDMIGRATIONSTART2855d05d34854d6fa6f4d97819c1d83dTVHEADENDMIGRATIONEND xenial main

From TVHeadend site I found this the other day:

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 379CE192D401AB61

echo "deb TVHEADENDMIGRATIONSTART4a9c5a11df6742f592d3571a7b3f0451TVHEADENDMIGRATIONEND xenial release" | sudo tee -a /etc/apt/sources.list

However, if I remember correctly while testing on a VM I attempted to use the "unstable" and never received 4.1.
Which source are you using?
And finally, @George Kazaroth, I want to truly thank you for your quick responses and assisting me on this thread. You have been incredibly helpful.
After all is completed and working for me I plan on documenting all these steps and posting to my blog for my own future reference and for anyone else looking to do what I am doing with an OTA antenna and PVR on an Ubuntu server.

G Kazaroth

deb http://apt.tvheadend.org/unstable xenial main # 16.04

I was having issues about 1 year ago and the stable version was way too slow at getting out the needed updates. I have not have many issues with the unstable version. About the only issue I have is when I reboot, the recording list disappears from tvheadend, but I have it cataloged through Emby and through the harddrive, so I really don't need it on Kodi.

I use the command sequence
sudo apt-get update
sudo apt-get dist-upgrade

sudo apt-get upgrade

Good luck

Edward Crosby

I was able to upgrade to 4.1. I do not see Internal: XMLTV: tv_grab_file in the EPG Grabber Modules tab.

G Kazaroth

Sounds like you are close. Try this
1) ls /usr/bin/tv_*
see if you have a ton of tv_grab* files. The one you want should be missing. I have 52 files.
2) tv_find_grabbers
If you have the command, it again should display a pretty long list and probably will have the tv_grab_file missing.
3) apt list | grep xmltv
This should give you a list including the packages xmltv-util and libxmltv-perl. Those should say "[installed,automatic]" if installed.
4) I believe the xmltv-util is the one you need to install with sudo apt-get install xmltv-util
Try the commands again and see if the grabbers got installed.
The files are text, so I included the file you have missing in case it might help.

Edward Crosby

Yeah, thanks for that. After long researching I had found that file was missing. I downloaded it from the github page, moved it to /usr/bin and gave it executable permissions for everyone. The issue I am at now is that I found that at the beginning of the file is that it is referencing a file that does not exist (cat ~/.xmltv/tv_grab_file.xmltv) and I haven't found anywhere how this file is supposed to be created and what is supposed to be in the .xmtltv file. I had assumed it is referencing all the EPG channel information so I commented out that line and pointed it to my .xml file that the perl script created but it still doesn't seem to be working. Here is what I did:

if (( $# < 1 ))
then
#  cat ~/.xmltv/tv_grab_file.xmltv
        cat /home/ed/xmltv/xmltv.xml
  exit 0
fi

Edward Crosby

Looks like I may have spoke too soon. When I got home from work I see that I can add EPG sources to my OTA channels.
I guess updating that file to reference my .xml file worked.

G Kazaroth

The attached screenshot is the result on Kodi 17 estuary of the zap2it xmltv with the category/genre update. You will notice the origination date, original genre and season/episode listed on top of the description. Also, the season/episode number is listed in the subtitle. The genre listed is what tvheadend needs versus the original genre. More colors are present instead of grey due to the genre update.

Edward Crosby

What's happening if my EPG is not updating? After a few days, and currently, I no longer see shows scheduled in the EPG tab.

Edward Crosby

I started a new thread for my issues.

https://tvheadend.org/d/4722-epg-for-na-ota-zap2it

« Previous Page