Project

General

Profile

Locast interface (appliance)

Added by G Kazaroth 8 months ago

TVHeadend-Locast version 0.8.8 (7/24/2021)
Product is called cabernet for (Cable Network)

https://github.com/rocky4546/tvheadend-locast/releases
Working on getting a service that will take the Locast server stream, clean it and feed it into TVHeadend and other DVRs (Also tested on Emby and Plex). It works on the subscription and free versions. Also it runs on Linux and Windows as a service. Windows has a installer. Once installed, setup is easy with TVHeadend.
1) Setup the automatic IPTV network (recommend new URL is http://[host]:6077/Locast/channels.m3u)

Make sure to set the "Maximum # input streams". Locast has a max of 4 and tvheadend uses 2 per tuner during initial screening. Doing a force scan will create the mux and service values. Also, turn View level to Advanced and set the Re-fetch period (mins) to 1460. TVheadend has a tenancy to cause issues when channels change (Changed Services will not be mapped to channels). Doing a Force Scan every few weeks ensures control.
2) Next you can setup the grabber. I use a URL grabber written in Unix bash and is an extremely small file. It can be found in the github repo at
https://github.com/rocky4546/tvheadend-locast/tree/main/lib/tvheadend/service/Unix called tv_grab_url
Place the grabber file in the same location as the other TVHeadend tv_grab* files, change the permissions to executable and restart TVHeadend. This should allow TVHeadend to pickup the new grabber. While in the grabber list, make sure and disable any OTA grabbers. Stations no longer send this information and will only cause TVHeadend to use a tuner for scanning. Displaying the log window by clicking the three ^ in the bottom right is helpful at this time.

Have the grabber run and populate the EPG data into the EPG Grabber Channels tab. The log should show a quantity of channels were detected.
Pop over to the EPG Grabber tab and disable the OTA grabber cron. Also, update/replace the Internal grabber cron schedule using something like below. The example will pull the TV guide at 6:04am and 5:52pm. Add more if you need. It is recommended to use static cron times.

4 6 * * *
52 17 * * *

3) In the Channel view, select Map all channels.

This will tie the services, EPG and channels together, automatically. After this, re-grab the EPG data. This will populate the EPG tab with shows. I also updated the Number column with the channel number.
4) Display the TVGUIDE. This appliance has special features which maps the tvheadend genre, giving colors on tvguides. It also has enhanced guide descriptions and optional additional channel notations if you use many streams. Below is the Kodi tvguide using the pvr.hts plugin.

For Kodi, go to the settings for PVR and turn on the General setting "Use channel numbers from backend".


Replies (178)

RE: Locast interface (appliance) - Added by G Kazaroth about 2 months ago

Thank you for the information and working this. The change will help others using minimal Linux OSes. It looks like the wget executable for CoreELEC does not have the options, so I can make a change and have you pull it down to try it, or you can make the change yourself which is probably the simple answer. If you can change the line in the grabber file where the "wget" command is from

wget -q --retry-connrefused --tries=5 --timeout=20 -O - $url

to

wget -q -T 20 -O - $url

According to the Usage statement above, that should work. Let me know if you get the data dump.

RE: Locast interface (appliance) - Added by G Kazaroth about 2 months ago

Images have been updated with larger clearer images.

RE: Locast interface (appliance) - Added by Sterling Rorden about 2 months ago

G Kazaroth wrote:

Thank you for the information and working this. The change will help others using minimal Linux OSes. It looks like the wget executable for CoreELEC does not have the options, so I can make a change and have you pull it down to try it, or you can make the change yourself which is probably the simple answer. If you can change the line in the grabber file where the "wget" command is from

wget -q --retry-connrefused --tries=5 --timeout=20 -O - $url

to

wget -q -T 20 -O - $url

According to the Usage statement above, that should work. Let me know if you get the data dump.

First, thanks for your help! I don't mind doing what little I can do to help. It's nothing compared to what you've done.

I'm now just getting the following error:

CoreELEC:~/.kodi/addons/service.tvheadend42/bin # ./tv_grab_url -u http://192.168.83:6077/xmltv.xml
./tv_grab_url: line 6: can't open 1: no such file
wget: download timed out

One thing. This may sound stupid but I couldn't figure out how to directly download the file from Github. I used Windows to cut and paste the text into WinSCP text editor to create the file. I've had problems before where using Windows to copy has caused problems. Could this be the problem?

RE: Locast interface (appliance) - Added by Sterling Rorden about 2 months ago

C Island wrote:

I think you have a 'slash' instead of a period
Try this:

./tv_grab_url -u http://192.168.1.83:6077/xmltv.xml

Yea, I pasted the wrong section into the message above but I did try with the ".".

RE: Locast interface (appliance) - Added by G Kazaroth about 2 months ago

It is very possible that the editor you used added a control-M to the end of the line. I ran a test with the ctrl-M and got a WARNING in the cabernet log

WARNING: "GET /xmltv.xml%0D HTTP/1.1" 501 -

Since some windows editor add Ctrl-Ms to the end of the line, you could have this issue. Check the cabernet log to see what happened. It looks like the wget command worked, but got an unknown response from cabernet. If this is true, you probably need to learn a little vi since you are working in linux. Here is a brief list of commands once you "vi file"
:q! quits without saving
:qw writes updates and then quits
ctrl-G goto end of file
shift-A goto end of line
ESC exit editor mode
x deletes current character
i starts insert mode
dd deletes current line
yy copies current line
p pastes line from yy

RE: Locast interface (appliance) - Added by G Kazaroth about 2 months ago

BTW, I do see the error ./tv_grab_url: line 6: can't open 1: no such file
but the script runs past it, so ignoring it for now. The result of "(( $# < 1 ))" is 1 which means true with the if statement. Not sure why it is confused with the bash format and stating the error. If you just run the ./tv_grab_url without the -u, then the result of the expression is 0 and the print statements should appear.

The best answer on the internet is that I need to add semicolons into the script to make it work. I will look further into this issue.

RE: Locast interface (appliance) - Added by Sterling Rorden about 2 months ago

G Kazaroth wrote:

It is very possible that the editor you used added a control-M to the end of the line. I ran a test with the ctrl-M and got a WARNING in the cabernet log

WARNING: "GET /xmltv.xml%0D HTTP/1.1" 501 -

Since some windows editor add Ctrl-Ms to the end of the line, you could have this issue. Check the cabernet log to see what happened. It looks like the wget command worked, but got an unknown response from cabernet. If this is true, you probably need to learn a little vi since you are working in linux. Here is a brief list of commands once you "vi file"
:q! quits without saving
:qw writes updates and then quits
ctrl-G goto end of file
shift-A goto end of line
ESC exit editor mode
x deletes current character
i starts insert mode
dd deletes current line
yy copies current line
p pastes line from yy

I opened the cabernet.log file in the data/logs folder but it is empty.

I tried re-running the internal EGP grabbers to see what it would do. It got stuck in a loop and kept repeating the following information in the Tvheadend log:

2021-06-06 17:49:57.869 subscription: 0025: No input source available for subscription "127.0.0.1 [ | Kodi Media Center ]" to channel "29.1L ION"
2021-06-06 17:49:59.869 subscription: 0025: No input source available for subscription "127.0.0.1 [ | Kodi Media Center ]" to channel "29.1L ION"
2021-06-06 17:50:00.653 http: 127.0.0.1: HTTP/1.1 HEAD /imagecache/15 -- 404
2021-06-06 17:50:01.904 subscription: 0025: No input source available for subscription "127.0.0.1 [ | Kodi Media Center ]" to channel "29.1L ION"
2021-06-06 17:50:03.635 http: 127.0.0.1: HTTP/1.1 HEAD /imagecache/19 -- 404
2021-06-06 17:50:03.904 subscription: 0025: No input source available for subscription "127.0.0.1 [ | Kodi Media Center ]" to channel "29.1L ION"
2021-06-06 17:50:05.910 subscription: 0025: No input source available for subscription "127.0.0.1 [ | Kodi Media Center ]" to channel "29.1L ION"
2021-06-06 17:50:06.645 http: 127.0.0.1: HTTP/1.1 HEAD /imagecache/15 -- 404
2021-06-06 17:50:07.933 subscription: 0025: No input source available for subscription "127.0.0.1 [ | Kodi Media Center ]" to channel "29.1L ION"
2021-06-06 17:50:09.741 http: 127.0.0.1: HTTP/1.1 HEAD /imagecache/19 -- 404
2021-06-06 17:50:09.933 subscription: 0025: No input source available for subscription "127.0.0.1 [ | Kodi Media Center ]" to channel "29.1L ION"
2021-06-06 17:50:11.933 subscription: 0025: No input source available for subscription "127.0.0.1 [ | Kodi Media Center ]" to channel "29.1L ION"
2021-06-06 17:50:12.665 http: 127.0.0.1: HTTP/1.1 HEAD /imagecache/15 -- 404
2021-06-06 17:50:13.933 subscription: 0025: No input source available for subscription "127.0.0.1 [ | Kodi Media Center ]" to channel "29.1L ION"
2021-06-06 17:38:10.310 subscription: 0025: No input source available for subscription "127.0.0.1 [ | Kodi Media Center ]" to channel "29.1L ION"
2021-06-06 17:38:12.299 http: 127.0.0.1: HTTP/1.1 HEAD /imagecache/15 -- 404 ....

I opened the tv_grab_url file in vi but I don't know how to tell if something is wrong.

RE: Locast interface (appliance) - Added by G Kazaroth about 2 months ago

The cabernet.log in the data/logs folder is suppose to be empty unless you turn on file logging which is disabled by default. You can either go over to the cabernet website, go to Settings > Logging > File Log Handler and enable that or find out where the system logs are. Then restart cabernet and it will start logging there as well as to where ever your system logs go.

The claim from the errors in your tvheadend log is that the channel 29.1L does not have a service tied to it. You can check under Configuration > Channels/EPG > Channels and see whether it has a service listed. You can also play that channel on that tab by clicking on the > button on the far left. In my case, it brought up VLC and played the channel. You can also play the video from the TVHeadend EPG tab by clicking on the TV icon on the far left.

Associated with vi, if you have the issue you will see a bunch of ^M appearing on the end of lines. Those are the issues.

I went ahead and added all the semicolons in the tv_grab_url file and pushed out a new version 0.8.6a.

RE: Locast interface (appliance) - Added by Sean Micklem about 2 months ago

Just a couple of general thoughts here worth mentioning, although they may not be all that helpful, but on the off chance...

First, don't just assume that CoreELEC includes bash. Some lightweight, single purpose distributions are built around BusyBox or something similar and only include the sh shell. sh is rather limited compared to bash, so a script written for bash may or may not run in sh (or whatever shell CoreELEC includes). I would enter which bash at a command prompt; if bash is installed it should show a path such as /usr/bin/bash on the following line. If nothing is printed, that means bash is not installed, or is not in your path, and that means some scripts will break.

Second, if you have a file and you are not sure whether it contains Windows-style line endings, you can run the dos2unix command on it. I believe the proper syntax is simply dos2unix filename but I would run man dos2unix to be certain. dos2unix is included in many Linux distributions, or can be added from a distribution's repositories, and you can run which dos2unix to see if you already have it. I think it may even be included in Windows now, but I don't use Windows at all, so can't be certain of that. It's a lot easier than manually searching for ^M characters or something similar (also, if you are like me and prefer nano to vi, you don't have to learn a different editor).

RE: Locast interface (appliance) - Added by Sterling Rorden about 2 months ago

Good progress.

I copied tv_grab_url v 0.8.6a over and ran dos2unix on it (CoreELEC has both bash and dos2unix).

I still get the line 6 no such file error when I re-run the internal EPG grabbers. The cabernet log has the following at the same time:

2021-06-07 07:27:24,629-INFO:web_handler [192.168.1.83] "GET /xmltv.xml HTTP/1.1" 200 -

However, the EPG is populating and things seem to work correctly. I can view shows, record shows, set timers, etc.

My channel numbers still start with 10000 in the Kodi Tvheand client and don't corelate with the actual channel numbers, but I can work around that if necessary.

I'm assuming the best way to get cabernet to run on system restart is to create an autostart.sh file with the correct command, correct?

Thanks guys for all your help!

RE: Locast interface (appliance) - Added by G Kazaroth about 2 months ago

I think I have answers.

1) channels in Kodi EPG have 10000 numbers.
Follow step 3 and add channel numbers to the number column in TVHeadend

2) You may have OTA grabbers enabled.
I noticed in your log that tvheadend is running scans. This is normally associated with OTA grabbers. In the old days, stations would broadcast tvguide data over the channel itself and could be obtained directly from the station instead of going to zap2it or the like. This is no longer happening and it just causes the tuner to use up streams. Recommend disabling all OTA grabbers in the "EPG Grabber Modules" tab.

3) Still getting the line 6 no such file error.
I agree with Sean. Based on some tests it looks like you do not have BASH shell installed on CoreELEC at the location in the script. type "which bash" and see if it is located at /bin/bash. This is where the script expects to find the executable. If you find it elsewhere, copying it to the /bin location will correct the issue.

4) On a new install of tvheadend, it is requesting xmltv.xml updates every minute
This seems to be the tvheadend cron causing the issue.

1 Go to the EPG Grabber tab
2 Disable the OTA grabber
3 Update the Internal Grabber to have 2 or more individual grabs. One in the morning around 6am (6:04 in example) and one in the evening around 6pm (5:52 in example). Each line represents a separate cron job time to request the xmltv.xml. Remember to save changes!

4 6 * * *
52 17 * * *

RE: Locast interface (appliance) - Added by Sterling Rorden about 2 months ago

G Kazaroth wrote:

I think I have answers.

1) channels in Kodi EPG have 10000 numbers.
Follow step 3 and add channel numbers to the number column in TVHeadend

2) You may have OTA grabbers enabled.
I noticed in your log that tvheadend is running scans. This is normally associated with OTA grabbers. In the old days, stations would broadcast tvguide data over the channel itself and could be obtained directly from the station instead of going to zap2it or the like. This is no longer happening and it just causes the tuner to use up streams. Recommend disabling all OTA grabbers in the "EPG Grabber Modules" tab.

3) Still getting the line 6 no such file error.
I agree with Sean. Based on some tests it looks like you do not have BASH shell installed on CoreELEC at the location in the script. type "which bash" and see if it is located at /bin/bash. This is where the script expects to find the executable. If you find it elsewhere, copying it to the /bin location will correct the issue.

1) I'm assuming you mean manually add the channel numbers (don't mind - just want to be clear). However, I have a question. What would happen if I use the same channel numbers as the corresponding OTA channels and record the same show both OTA and from Locast? I want to mainly use OTA for recordings but have Locast be a backup incase my reception drops.

2) I currently have an OTA grabber running as low priority but will disable it.

3) The command "which bash" returns the following:
/usr/bin/bash
Can I just change the first line in the script to "#!/usr/bin/bash"?

RE: Locast interface (appliance) - Added by C Island about 2 months ago

Sterling

I believe CoreELEC uses busybox. The 'which' command does not show it is all. If you do a 'ls -al' command you will see that it uses busybox. Here is the output from a CoreELEC box I have.

CoreELEC:~/.kodi/addons # which bash
/usr/bin/bash
CoreELEC:~/.kodi/addons # ls -al /usr/bin/bash
lrwxrwxrwx    1 root     root             7 Jun  6 02:42 /usr/bin/bash -> busybox

BTW: I deleted my previous comments about using a python based tv_grab_url because it does not work on CoreELEC (the script would need to be converted to python3)

RE: Locast interface (appliance) - Added by G Kazaroth about 2 months ago

Looks like we are moving in the right direction...
1) The number column does require a number only and cannot use letters. If you use the same number, then Kodi will place the channels next to each other in the list (which is a good thing). Locast will still have the channel number in the channel name, so you will know which is the Locast channel. The ZAP2EPG may not add the channel number to the channel name, but I found this to be deficient since there are numerous channels with the same name (such as COMET). To fix this, I added the channel number to the channel name. I do this even with the Zap2It xmltv.xml file.

2) yes, you can update the top line of the file to point to the correct location of bash. If you could try something first. Try changing the line to a more portable text

#!/usr/bin/env bash

and see if that works. If so, I will make the change.

During my testing of a clean tvheadend install, I did also find that I had made a mistake on settings in the Network area. All of this is going into the configuration notes, but I need to let you know the change. If you edit the Network in TVHeadend and change the view level to at least Advanced, you will see "Re-fetch period (mins)" and is set to 60. There is no need to grab channels every hour, so the recommendation is to make it 1-2 days. One day would be 1440, but I would recommend something like 1460 or slightly over 1 day. I will update the default update rate in cabernet to 1 day and that should make it all work.

RE: Locast interface (appliance) - Added by G Kazaroth about 2 months ago

As for your question on how to setup a service on CoreELEC, have no idea, but what I did to create the cabernet service was to determine how tvheadend was started and basically use that to create the cabernet file. I would recommend the same approach.

RE: Locast interface (appliance) - Added by C Island about 2 months ago

TVHeadend is implemented as a Kodi addon in Kodi. Writing a Kodi addon can be somewhat complicated for someone who has not written a Kodi service previously.

The good news is that CoreELEC has 'systemd' that can be used to create your own services which is pretty straightforward.

The directory containing systemd unit files is /storage/.config/system.d. There are some example unit files in that directory.

You can find info about how to create a service using systemd all over the web. Here is an example of a 'how to' https://www.linode.com/docs/guides/start-service-at-boot/

RE: Locast interface (appliance) - Added by G Kazaroth about 2 months ago

WOW, that is an eye opener. Since TVHeadend is written in C++, it would take a ton of effort to wrap it into a python-based Kodi. I saw a video on the install and configuration. It now makes some sense why he calls it Kodi TVHeadend EPG and it also made sense why people would use edit4ever Kodi addons. Although it plays to the question whether cabernet could be wrapped into Kodi, there are more important updates to make before we get there. Thanks for the clarification!

RE: Locast interface (appliance) - Added by Sterling Rorden about 2 months ago

G Kazaroth wrote:

Looks like we are moving in the right direction...
1) The number column does require a number only and cannot use letters. If you use the same number, then Kodi will place the channels next to each other in the list (which is a good thing). Locast will still have the channel number in the channel name, so you will know which is the Locast channel. The ZAP2EPG may not add the channel number to the channel name, but I found this to be deficient since there are numerous channels with the same name (such as COMET). To fix this, I added the channel number to the channel name. I do this even with the Zap2It xmltv.xml file.

2) yes, you can update the top line of the file to point to the correct location of bash. If you could try something first. Try changing the line to a more portable text
[...]
and see if that works. If so, I will make the change.

During my testing of a clean tvheadend install, I did also find that I had made a mistake on settings in the Network area. All of this is going into the configuration notes, but I need to let you know the change. If you edit the Network in TVHeadend and change the view level to at least Advanced, you will see "Re-fetch period (mins)" and is set to 60. There is no need to grab channels every hour, so the recommendation is to make it 1-2 days. One day would be 1440, but I would recommend something like 1460 or slightly over 1 day. I will update the default update rate in cabernet to 1 day and that should make it all work.

1) Thanks, I'll go with your suggestion.

2) I tried both "#!/usr/bin/env bash" and "#!/usr/bin/bash" as the first line in the tv_grab_url file. Both still have the line 6 error in TVHeadend. C Island confirmed CorELEC uses busybox but I don't understand the implications of this.

I changed the fetch period to 1460.

RE: Locast interface (appliance) - Added by Sterling Rorden about 2 months ago

C Island wrote:

TVHeadend is implemented as a Kodi addon in Kodi. Writing a Kodi addon can be somewhat complicated for someone who has not written a Kodi service previously.

The good news is that CoreELEC has 'systemd' that can be used to create your own services which is pretty straightforward.

The directory containing systemd unit files is /storage/.config/system.d. There are some example unit files in that directory.

You can find info about how to create a service using systemd all over the web. Here is an example of a 'how to' https://www.linode.com/docs/guides/start-service-at-boot/

Systemd looks like a little bit of a learning curve for me, which I'm willing to do if necessary.
Is there a problem with using the Python3 tvh_main.py in the autostart.sh file? I don't understand the difference between a service and using autostart.sh.

RE: Locast interface (appliance) - Added by C Island about 2 months ago

autostart.sh should work as well. I have used systemd directly but not autostart.sh. I think autostart.sh is a simple service run by systemd so it should work.....

RE: Locast interface (appliance) - Added by G Kazaroth about 2 months ago

Hi Sterling, I will try to convert the script to Bourne shell and see if that will work. Give me some time and I will get back to you.

Sorry, but I had to let everyone know. I checked edit4ever tv_grab* file that you are using and he uses #!/bin/bash in the top line. However, his script has no bash specific lines in it; it is all bourne shell.

RE: Locast interface (appliance) - Added by Sterling Rorden about 2 months ago

G Kazaroth wrote:

Hi Sterling, I will try to convert the script to Bourne shell and see if that will work. Give me some time and I will get back to you.

Sorry, but I had to let everyone know. I checked edit4ever tv_grab* file that you are using and he uses #!/bin/bash in the top line. However, his script has no bash specific lines in it; it is all bourne shell.

Thanks!

RE: Locast interface (appliance) - Added by G Kazaroth about 2 months ago

0.8.6b has the updated script written in Bourne Shell. Give it a try and let me know.

RE: Locast interface (appliance) - Added by Sterling Rorden about 2 months ago

G Kazaroth wrote:

0.8.6b has the updated script written in Bourne Shell. Give it a try and let me know.

That appears to have worked! No errors and a list of updated channels when I forced a re-run of the internal grabbers.

Also, for any CoreELEC user that comes across this thread in the future, the following autostart.sh script seems to work when put in the /storage/.config folder (I unzipped cabernet to a folder called tvheadend-locast in storage):

#!/bin/sh
(
sleep 10
python3 /storage/tvheadend-locast/tvh_main.py
) &

RE: Locast interface (appliance) - Added by G Kazaroth about 2 months ago

File with notes added to baseline. Thank you for the help.

(126-150/178)