Synology xmltv module and zap2xml help

Added by Nick Craig 10 months ago

I have a synology nas with tvheadend from synocommunity setup and working with my hdhomerun tuner. I cannot wrap my head around how to setup the internal grabber (currently it says disabled) with zap2it epg data (I have username and account with them) . I have downloaded the tv_grab_file and the zap2xml file but really have no clue what to do with them. Where do I put them etc. I do have ssh setup. If someone would be kind enough to do some heavy typing to give me a step by step that would be awesome.


Replies (24)

RE: Synology xmltv module and zap2xml help - Added by Greg H 10 months ago

The tutorial linked above is what I used to get going on my synology box. You will have to use the other script.

"For example, in my Review of the TBS MOI+, I showed a variation of that script that runs under ash (as provided in BusyBox), since the original bash script won’t work in that environment."

To be able to run the script you will need to install perl from the synology package manager. I am also pretty sure you will need to modify the zap2xml.pl file for the version of perl that is used by synology I would need to dig up my notes.

RE: Synology xmltv module and zap2xml help - Added by Nick Craig 10 months ago

If you are able to dig them up I would be grateful. I will check out the tutorial and the review of TBD.

RE: Synology xmltv module and zap2xml help - Added by Nick Craig 10 months ago

So ...... I'm not sure I'm cut out for this. I have the tv_grab_file and the zap2xml.pl file at /volume1/Tvheadend/xmltv/ and have set both as executable and correct permissions. I edited the tv_grab_file cat to point the /volume1/TVheadend/xmltv/tv_grab_file and it doesn't show up under internal grabbers in the epg tab. Still says disabled. I also installe perl to synology. I know I have to still edit the zap2xml file perl stuff but I assume that stuff would still show up under internal grabbers.

RE: Synology xmltv module and zap2xml help - Added by Greg H 10 months ago

tv_grab_file needs to be in /usr/bin the zap2xml.pl can be left in /volume1/Tvheadend/xmltv/ tv_grab_file cat should point to the location of the xml file (I call mine xmltv.xml) created by zap2xml.pl which should default to the same location /volume1/Tvheadend/xmltv. I created a script file zap2xml.sh that executes zap2xml.pl with some options and the zap2xml.sh file is then scheduled to run using the synology DSM scheduler.

I can get a more detailed description and attache my files as examples later today.

RE: Synology xmltv module and zap2xml help - Added by K Shea 10 months ago

Nick Craig wrote:

So ...... I'm not sure I'm cut out for this. I have the tv_grab_file and the zap2xml.pl file at /volume1/Tvheadend/xmltv/ and have set both as executable and correct permissions. I edited the tv_grab_file cat to point the /volume1/TVheadend/xmltv/tv_grab_file and it doesn't show up under internal grabbers in the epg tab. Still says disabled. I also installe perl to synology. I know I have to still edit the zap2xml file perl stuff but I assume that stuff would still show up under internal grabbers.

As Greg H said the script tv_grab_file needs to be in /usr/bin but also it must be made world executable and the owner and group should be root. If you just copy it over and don't make it accessible to TVHeadend then I don't think it will appear in the list of grabbers. Remember, when things don't work as you think they should in Linux, it's often either a permissions or ownership issue, or both.

RE: Synology xmltv module and zap2xml help - Added by Nick Craig 10 months ago

Bam!! Worked like a champ. Thank you for the great writeup.

RE: Synology xmltv module and zap2xml help - Added by Greg H 9 months ago

This is how mine is set up on a Synology DS415+ DSM 6.0; Perl 5.24.0-0066; Tvheadend 4.1.2389-1

I placed tv_grab_file in /usr/bin and chmod 755 to make it readable and executable. It is owned by root and group is root.

I edited the cat line in tv_grab_file to /usr/local/zap2xml/xmltv.xml - This is where I have the zap2xml files and the created xml file xmltv.xml. This folder is chmod 777 (Prob could change that to 755) In your case you would use /volume1/Tvheadend/xmltv/ instead of /usr/local/zap2xml

In /usr/local/zap2xml is zap2xml.pl, zap2xml.sh, they are chmod 755 and owned by admin, group users. After zap2xml is run it creates the xml file xmltv.xml.

zap2xml.pl has been modified so it would run with the version of the synology perl package. All the instances of JSON have been changed to JSON::PP, I have attached a copy of the file I use.

zap2xml.sh is the script I use to run zap2xml.pl it contains command line parameters. I am currently using tvguide (-z), I was using zap but it required a second read to get the episode info as S00E00, whichever one you use be sure to set up your favorite channels to limit the file size and bandwidth used. The -o option is used to tell it what to name the file and where to put it, the -d is the number of days to get I currently am using 3 days while testing I set it to 1. The second perl line parses the file to remove empty display-name tags which were causing tvheaded to crash on import.

perl /usr/local/zap2xml/zap2xml.pl -z -u USER -p PASS -d 3 -o /usr/local/zap2xml/xmltv.xml -U -T -e

perl -ni -e 'print unless /<display-name><\/display-name>/' /usr/local/zap2xml/xmltv.xml

I have the zap2xml.sh file scheduled to run every morning in DSM using Control Panel > Task Scheduler > Create > Scheduled Task > User Defined Script - Run as root, Scheduled daily once a day at 5:30, Run command /usr/local/zap2xml/zap2xml.sh

In tvheadend I have the tv_grab_file enabled Config > Channel/EPG > EPG Modules - Enabled Checkbox and then scheduled to run at 6:30 daily Config > Channel/EPG > EPG Grabber > Cron multi-line: 30 6 * * * You can modify your schedule time and frequency as you see fit.

Hopefully this helps you get the episode grab working.

tv_grab_file (552 Bytes)

zap2xml.pl Magnifier (48.1 KB)

zap2xml.sh Magnifier (187 Bytes)

RE: Synology xmltv module and zap2xml help - Added by Joe Christl 6 months ago

zap2xml.pl has been modified so it would run with the version of the synology perl package. All the
instances of JSON have been changed to JSON::PP, I have attached a copy of the file I use.

just wanted to post to say thank you to Greg H. for the tip on getting the JSON to work in Synology.

RE: Synology xmltv module and zap2xml help - Added by D R 2 months ago

Do you guys happen to have the latest (2017-01-01) zap2xml.pl file edited for Synology?
I tried changing JSON::PP in new file but its a no go.
It runs ok with the file from Greg H.

@Can't locate JSON.pm in @INC (you may need to install the JSON module) (@INC contains: /usr/local/lib/perl5/site_perl /usr/local/share/perl5/site_perl /usr/local/lib/perl5/vendor_perl /usr/local/share/perl5/vendor_perl /usr/local/lib/perl5/core_perl /usr/local/share/perl5/core_perl .) at /usr/local/zap2xml/zap2xml.pl line 38.

See homepage for tips on installing missing modules (example: "perl -MCPAN -e shell")@

perl is installed, but apparently something is not right.

RE: Synology xmltv module and zap2xml help - Added by Greg H 2 months ago

I am using the version/file I posted. If there is a newer zap2xml.pl I can look at it and see if I can get it to work on my synology box

RE: Synology xmltv module and zap2xml help - Added by D R 2 months ago

The newest version fixes problems with za2it server not responding (skipping missing tv IDS)

Gerg H, thank you for taking the time to help.

RE: Synology xmltv module and zap2xml help - Added by Joe Christl 2 months ago

i was noticing this too, but haven't got around to updating my zap2 script. Glad I waited. :P

RE: Synology xmltv module and zap2xml help - Added by Greg H 2 months ago

I just downloaded the latest file form zap2xml.awardspace.info and modified it. I just ran it on my system and worked without any errors. I have attached the file.

zap2xml.pl Magnifier (49 KB)

RE: Synology xmltv module and zap2xml help - Added by D R 2 months ago

Thank you very much! that file worked well for me!

Beside changing JSON::PP, what else does it need to be changed?

Thank again for your help!

Found the mistake I made.... missing PP line 38. :(

RE: Synology xmltv module and zap2xml help - Added by Joe Christl 18 days ago

Hey is it just me, or has zap2xml gone offline

RE: Synology xmltv module and zap2xml help - Added by D R 18 days ago

It's not just you. Hopefully is a temporary issue.

RE: Synology xmltv module and zap2xml help - Added by K Shea 18 days ago

EDIT: Site at http://zap2xml.awardspace.info/ appears to be back up. I will leave this post up just in case it goes down again.

For now you can view a cached version of the page at the Wayback Machine: https://web.archive.org/web/20170102211055/http://zap2xml.awardspace.info:80/

The download links don't work, but those are for the original perl script, not the version modified to work with the Synology which is posted in earlier messages in this thread. In case anyone needs the original unmodified perl script (the Linux version) I have attached it to this post. This is the last released version to the best of my knowledge.

zap2xml.pl Magnifier - Unmodified zap2xml.pl file (49 KB)

RE: Synology xmltv module and zap2xml help - Added by log jam 15 days ago

I have been trying and trying to get this going for about a year now. I have had TVH running and recording on my Synology, but have been using OTA grabber which would at best give me a day's worth of guide date.

I went step by step (special thanks to Greg!) and have finally done it! I have zap working and pulling data successfully! A BIG thanks to everyone on this board!

One question I have is, I notice some sports are highlighted in blue (these are the only ones showing a genre), whereas most of the other guide data is grey. How can I modify colors to match different genres?

Thanks again all!

Logjam

RE: Synology xmltv module and zap2xml help - Added by D R 15 days ago

I use -I -D to get the extra info available on zap2it server. First run takes some time but after that is pretty smooth.

RE: Synology xmltv module and zap2xml help - Added by log jam 14 days ago

^^^
I tried your suggestion, and like you said, it took a long time. However, no additional colors showing in the guide. Any other suggestions?

Thanks!

Logjam

RE: Synology xmltv module and zap2xml help - Added by K Shea 13 days ago

It takes a long time because it has to request data for ever single show in the listings separately. I really kind of wish people wouldn't do this because I fear if too many do, it will cause them to shut down the service (or some other bad outcome). If you must do it, at least please be responsible and make sure you only request the data once a day, preferably at some totally random time (not ending in :00) in the middle of the night so their servers won't be hammered.

But that said, there is an additional script that you have to run to add in the colors, it simply translates the tags that zap2it uses to the tags that Tvheadend understands. This thread explains the problem, but (at least on the front page) doesn't provide a solution applicable to zap2it users. https://tvheadend.org/boards/12/topics/9892?r=26317 I could have sworn that someone wrote a script to do something similar for sap2it users but I can't seem to find it now. I do remember testing it quite some time back and if I recall correctly it caused some problem (maybe corrupting the .xml file so Tvheadend couldn't read it all? I honestly don't remember) but since I abandoned it rather quickly I don't recall the name of it anymore. But anyway, that's the problem.

RE: Synology xmltv module and zap2xml help - Added by log jam 13 days ago

K Shea wrote:

It takes a long time because it has to request data for ever single show in the listings separately. I really kind of wish people wouldn't do this because I fear if too many do, it will cause them to shut down the service (or some other bad outcome). If you must do it, at least please be responsible and make sure you only request the data once a day, preferably at some totally random time (not ending in :00) in the middle of the night so their servers won't be hammered.

Rest assured, I won't be hammering the server. I don't want to jeopardize a good thing! I have set my cron job for the wee small hours of the morning.

As for the additional script, I had read that thread and thought it only applied to French users, but it looks like it could be run in the US as well. Do you think it needs to be modified to run in the US, or is it good to go?

I know enough Linux to be dangerous ;)

Thanks everyone!

Logjam

RE: Synology xmltv module and zap2xml help - Added by K Shea 13 days ago

log jam wrote:

As for the additional script, I had read that thread and thought it only applied to French users, but it looks like it could be run in the US as well. Do you think it needs to be modified to run in the US, or is it good to go?

I think it probably does need to be modified, specifically to translate the tags used by zap2it and passed into the XML file by zap2xml, into the categories that Tvheadend wants to see. It's probably just script that does straight substitution of one word for another. The problem is that the program in that thread was written for use with a French service and therefore probably expects the XML file to contain French language categories.

At the heart of it, it's probably just replacing one tag with another.

I'm like you when it comes to Linux, to me almost everything in Linux seems far more complicated than it should be. I did not read all the pages of that thread, so if you have the time and go through them all, just maybe someone has modified it to work with zap2it-style categories.

But I still can't help thinking there was another script written from the get-go for use with sap2it/zap2xml, I even remember trying it out, but I simply cannot remember the name of it or where I saw it. It was likely either in this forum or the Kodi forum Tvheadend section, but I just don't remember.

EDIT: It MAY have been the Perl script on this page: https://github.com/rocky4546/script.xmltv.tvheadend/tree/master/xmltv - the package it is part of was intended to provide some kind of integration between Kodi and Tvheadend or something, but I never needed it since I already had all the necessary parts installed and working, I just wanted to see if the genre translations script worked. But I do remember that in some way it screwed things up for me, so I stopped using that script. Maybe the problem has been fixed, or maybe you could fix it if you know Perl (I'm guessing you probably don't), but I don't have high hopes that it will work for you as-is (but who knows, you might get lucky).

(1-24/24)