Running zap2xml along with tv grab file

Ian Fitch

I am running the zap2xml.pl to download an EPG. I am then loading it into Tvheadend with a tv_grab_file script. I got everything working, but I'm trying to have Tvheadend run zap2xml along with tv_grab_file. I added the command:
source /var/lib/tvheadend/scripts/run_zap2xml_ota.sh
to the tv_grab_file and it works when I run the grabber as ME, but when Tvheadend runs the grabber, it can't create the cache directory due to permission errors. I found that zap2xml.pl tries to create the cache in whatever directory the user is in. I'm guessing in the case of Tvheadend running the EPG grabber the user is hts and the directory it's in /usr/bin, but I'm not sure.
I worked around that by specifying that the cache be created in /var/lib/tvheadend/cache which I gave ownership over to hts and chmod 777. Again, it works when I run as ME, but not when Tvheadend runs it. The only message I'm getting is "/usr/bin/tv_grab_file_ota: htsmsg_xml_deserialize error Unknown syntatic element:". I think what it's actually complaining about is that nothing is actually being saved to the cache. Can someone please help me solve this or even come up with another solution? I know I can setup a cron job to pull down the EPG, but this is a much more practical solution.

Tony Wagner

Why is this more practical than cron?

Enable the EPG socket in TVHeadend, then a simple script can grab your XML and immediately pipe it in to TVH:
https://tvheadend.org/d/8981-add-epg-cron-multiple-urls/9

Although even easier than scraping is contributing $35/year to support the community via Schedules Direct.

Ian Fitch

To me it makes sense to update the EPG at the same time it is added to TVheadend as opposed to do it asynchronously.

The problem isn't getting the EPG into TVheadend. The problem is that when I run the update script via the EPG Grabber, I get all kinds of permissions issues that prevent me from caching the file. There's not enough information provided in the log for me to even troubleshoot it.

I pay for a service that provides an EPG and Zap2It works perfectly fine. I'm happy to support the community, but adding yet another subscription to my budget is not viable.

Tony Wagner

Ian Fitch To me it makes sense to update the EPG at the same time it is added to TVheadend as opposed to do it asynchronously.

That’s exactly what my link above does — a single command to grab the XML and pipe it into TVHeadend.

Ian Fitch

I see. Thanks for that.

To clarify, the problem is that the data is scraped from a website and then parsed into an XML file with a script. The part that's breaking is caching the data from the web. I don't know what's going wrong. I have run the script as hts in the terminal and it works, but when Tvheadend runs the script via the EPG grabber it just doesn't and it doesn't provide any information about what happened.

This may not be the right place to ask the question, but I'm hoping for some insight as to what's going on under the hood when Tvheadend runs the EPG grabber as that might help me diagnose the problem myself.

Tony Wagner

I understand the problem. I used to use tv_grab_file too, but trust me, the socket method I linked is truly more practical and flexible.

Is your scraper/script actually outputting full valid XML to the standard output? That is likely what the tv_grab_file script is looking for. If it is outputting anything else, like log messages etc., it is not going to work and it will also result in those syntax error messages you are seeing. If it’s writing the XML to a file, your bash script will need to read that file and echo its contents back to the standard output, again avoiding any extraneous messages that would corrupt the standard output.