DaddyLive, PlutoTV, XUMO, M3U/XMLTV, SamsungTV, Plex, TVGuide interfaces (appliance)

Sean Micklem

G Kazaroth wrote:
> @Sean Micklem
> Since the config interface is a plugin, it was easy to add flags for the locast provider to disable channel and program icons in the xml file if you want. Modded the xml generator to make the output pretty print. Thanks for letting me know what version of python you have, but would have like to know the hardware and OS in use.
>

> Still working on making the channel list, plugin capable. Once that is complete, will push out the next update to github.

Thanks. The OS is Debian 10 (Buster) but I can't really tell you anything about the hardware because I didn't build the system myself, I only installed the operating system and software, and that was several months ago.

Sean Micklem

When you say "Here is what is in this update", what update are you talking about? When I go to https://github.com/rocky4546/tvheadend-locast/releases the last release I see is 0.7.5c from 29 days ago.

I see you are still working on it so maybe you are referring to a beta version? I'm happy to wait for the next full release and am looking forward to it, but just trying to understand what you mean here. Glad to see that you are making progress in any case!

Sean Micklem

G Kazaroth wrote:

> It is very possible Locast may not like both running at the same time, so you are on your own if you use multiple sites. I did it to get the instance software working.

Many thanks for your work on this. I haven't got around to trying it yet, have been busy today, but I do have one question about the above. What if you made two Locast accounts and used a different one for each city? In other words, instead of

[locast]
login-username = j@hotmail.com

Could you do

[locast_dallas]
login-username = j@hotmail.com
login-password = MyFinePassword

label = Locast Dallas

[locast_denver]
login-username = k@hotmail.com
login-password = AnotherFinePassword

label = Locast Denver

Or would that not work?

Sean Micklem

G Kazaroth wrote:

> 0.8.1 published with the ability as defined above for locast authentication.

Well I gave it a quick test this morning but when I sent the m3u list to VLC it could not tune any channels and I got errors like this on the display when running it using "python3 tvh_main.py":


Exception happened during processing of request from (redacted)
Traceback (most recent call last):
  File "/usr/lib/python3.7/socketserver.py", line 316, in _handle_request_noblock
    self.process_request(request, client_address)
  File "/usr/lib/python3.7/socketserver.py", line 347, in process_request
    self.finish_request(request, client_address)
  File "/usr/lib/python3.7/socketserver.py", line 360, in finish_request
    self.RequestHandlerClass(request, client_address, self)
  File "/home/test/tvheadend-locast/lib/clients/web_tuner.py", line 168, in __init__
    super(CustomHttpHandler, self).__init__(*args, **kwargs)
  File "/home/test/tvheadend-locast/lib/clients/web_tuner.py", line 55, in __init__
    super().__init__(*args)
  File "/usr/lib/python3.7/socketserver.py", line 720, in __init__
    self.handle()
  File "/usr/lib/python3.7/http/server.py", line 426, in handle
    self.handle_one_request()
  File "/usr/lib/python3.7/http/server.py", line 414, in handle_one_request
    method()
  File "/home/test/tvheadend-locast/lib/clients/web_tuner.py", line 79, in do_GET
    self.do_tuning(sid, query_data['name'], query_data['instance'])
  File "/home/test/tvheadend-locast/lib/clients/web_tuner.py", line 131, in do_tuning
    self.internal_proxy.stream_direct(station_list[sid], self.wfile)
  File "/home/test/tvheadend-locast/lib/streams/internal_proxy.py", line 57, in stream_direct
    stream_uri = self.get_stream_uri(_channel_dict)
  File "/home/test/tvheadend-locast/lib/streams/stream.py", line 62, in get_stream_uri
    .plugin_obj.get_channel_uri(_channel_dict['uid'], _channel_dict['instance'])
  File "/home/test/tvheadend-locast/plugins/provider_video_locast/lib/locast.py", line 56, in get_channel_uri
    return self.locast_instances[_instance].get_channel_uri(sid)
  File "/home/test/tvheadend-locast/plugins/provider_video_locast/lib/locast_instance.py", line 69, in get_channel_uri
    return self.channels.get_channel_uri(sid)
  File "/home/test/tvheadend-locast/lib/common/decorators.py", line 44, in wrapper_func
    return f(self, *args, **kwargs)
  File "/home/test/tvheadend-locast/lib/common/decorators.py", line 29, in wrapper_func
    return f(self, *args, **kwargs)
  File "/home/test/tvheadend-locast/plugins/provider_video_locast/lib/channels.py", line 128, in get_channel_uri
    'authorization': 'Bearer ' + self.locast.auth.token,
TypeError: can only concatenate str (not "NoneType") to str

This is what my config.ini looks like, I have replaced the city name with "city" and anything else that might be unique to my system I xxx'd out, I just wanted you to see the general format I have in case I removed something I shouldn't have:


[locast_city]
login-username = xxx@xxx.xxx
login-password = xxx
label = Locast city
overrides-zipcode = XXXXX
dma = XXX
city = city
m3u-group_hdtv = Locast city HDTV
m3u-group_sdtv = Locast city SDTV
epg-suffix = X

[epg]
epg_suffix = X
epg_channel_icon = False
epg_program_icon = False

[loggers]
keys = root

[logger_root]
level = WARNING
handlers = loghandler

[handlers]
keys = loghandler

[formatters]
keys = extend,simple

[handler_loghandler]
class = StreamHandler
formatter = extend
args = (sys.stdout,)

[formatter_extend]
format = %(asctime)s-%(levelname)s:%(module)s %(message)s

[formatter_simple]
format = %(levelname)s:%(module)s %(message)s

[main]
uuid = xxx

[hdhomerun]
hdhr_id = xxx

That's about all I can do with it for right now but just wanted to let you know about the errors.

EDIT: P.S. Don't know if this has anything to do with it but the m3u file still has tvg-logo links in it. Also my suspicion is that the link in the m3u file may be wrong, the format it is in now is http://ipaddress:port/Locast/watch/channelid, that /Locast was not in previous versions but anyway I am wondering if the instance name should be in there somewhere or something??? Anyway there is no video stream there, apparently, and VLC tried several channels (when one failed it would go to the next) and none worked.

P.P.S. Also wasn't paying attention before but when I first start it up it shows a bunch of warnings (data redacted same as above):

2021-05-09 12:33:53,041-WARNING:user_config DEFN None for section:locast_city key:login-username value:xxx@xxx.xxx
2021-05-09 12:33:53,042-WARNING:user_config DEFN None for section:locast_city key:login-password value:xxx
2021-05-09 12:33:53,042-WARNING:user_config DEFN None for section:locast_city key:label value:Locast city
2021-05-09 12:33:53,042-WARNING:user_config DEFN None for section:locast_city key:overrides-zipcode value:XXXXX
2021-05-09 12:33:53,043-WARNING:user_config DEFN None for section:locast_city key:dma value:XXX
2021-05-09 12:33:53,043-WARNING:user_config DEFN None for section:locast_city key:city value:city
2021-05-09 12:33:53,043-WARNING:user_config DEFN None for section:locast_city key:m3u-group_hdtv value:Locast city HDTV
2021-05-09 12:33:53,044-WARNING:user_config DEFN None for section:locast_city key:m3u-group_sdtv value:Locast city SDTV
2021-05-09 12:33:53,044-WARNING:user_config DEFN None for section:locast_city key:epg-suffix value:X
2021-05-09 12:33:53,044-WARNING:user_config DEFN None for section:epg key:epg_suffix value:X
2021-05-09 12:33:53,045-WARNING:user_config DEFN None for section:locast key:enabled value:False

Sean Micklem

Thank you! I downloaded 0.8.1b and it is working now with one instance, VLC plays all the channels I tried with no problems. I will try to do some more tests tonight or tomorrow but so far so good. I did remove that second epg-suffix line, thanks for the heads up on that!

EDIT: Added another instance today (May 10); it seems to be working great. Thanks much! I assume (hope) you are working on the channel editor now; that's the one more thing I'd really like to see is the ability to exclude unwanted channels. However what you have accomplished so far is very impressive!

Sean Micklem

Well first of all it really does look great, I especially enjoy the springtime theme you have chosen. I tend not to say much about appearance anymore because many years ago I was talking about a piece of software and said that the interface was well designed, well it turned out that was the one part of the software that the main author hadn't written and he got a little bent out of shape because that was the first thing I'd mentioned, and not how well it worked. So since then, commenting on appearance is typically something I tend to leave for others. Anyway just wanted to say that I updated to 0.8.3 and it works fine, then I tried adding a third instance and that worked until I clicked on the link for the xmltv.xml file in my browser, then I saw this in the terminal window

2021-05-11 17:21:54,679-WARNING:web_handler [127.0.0.1] "GET /pages/channels.html?area=general HTTP/1.1" 404 -
2021-05-11 17:21:56,710-WARNING:web_handler [127.0.0.1] "GET /pages/channels.html?area=general HTTP/1.1" 404 -

2021-05-11 17:25:36,287-WARNING:web_handler [127.0.0.1] "GET /web/index.html HTTP/1.1" 404 -

And then the system froze up and I had to reboot! Went back down to two instances and then it worked again, so my suspicion is that it is a memory issue of some kind. Normally I would never need three instances anyway so just was testing it, but I am wondering if creating that combined xmltv.xml file uses quite a bit of memory. I do realize that I could get the xmltv.xml file for each instance separately (which I assume would take less memory) so this is really not a big deal. Anyway as long as I stick to two instances everything I have tried seems to work, so thanks for the update!

By the way I likely won't have time to test anything else until the end of the week, but I should be able to do more this weekend, hopefully.

Sean Micklem

Well I tried the new versions today, the channel editor appears to be working great now. For future reference (when you run out of other things to work on) it might be nice to have a couple of toggle buttons that would enable/disable all HD, or all SD channels, and also maybe an "invert selection" button that would invert the checked status of all channels. But those are very minor things because once you've gone though and selected the channels you want or don't want, you probably won't need to do it again any time soon.

The bigger issue I am seeing with the new version is that when I send the channel list to VLC, for some reason VLC skips over a lot of channels rather than playing them. I don't know if this is a VLC issue or an issue in the software but I do know that every time it happens I see a couple of lines like this:

2021-05-15 17:59:18,323-WARNING:web_tuner Unknown channel id ###

2021-05-15 17:59:18,323-WARNING:web_handler [ip address ] "GET /Locast/watch/### HTTP/1.1" 501 -

Where ### is the channel ID. I don't recall seeing this with previous versions. I am testing with three instances now and the problem strikes random channels on all instances. My belief is that something is not waiting long enough for a channel to "lock in" because when it skips a channel it does so very quickly, and often will quickly skip the next three or four channels in the list (give or take) before finding one it likes. Sometimes if you go back and click on a channel again it will play it, or it may take several tries, or it may not work at all. I have only tested this in VLC so far, it is quite possible that this might not be an issue in Tvheadend but I need to make some changes to the backend configuration before I can test that. Have you ever encountered this in VLC?

Other than that, using three instances doesn't appear to be a problem now, and I was able to get a xmltv.xml file with only my selected channels without crashing the system, so either that is fixed, or just reducing the number of channels causes it to not use so much memory.

Quick question, when I upgraded to the 0.8.4a version this afternoon I lost my edited channels list. Normally what I do with a new version (in Linux) is unzip the source file into a new directory, then copy the config file from the current directory to the new one, then rename the current and new directories so that the new one has the same name as the previous (tvheadend-locast). This has always worked in the past but today it lost the editing I had done on the channels list. I suspect what I should have done is copy the data directory to the new version's directory but didn't know if doing that would mess anything up since you had told us to delete the contents of that directory before installing 0.8.4. But if I had copied or moved the data directory over would that have preserved my channels list without causing any other issues?

Thank you again for all you are doing on this!
<<

Sean Micklem

I was not saying I was able to edit the channels list in the past, I was saying that I have always used the same method of upgrading in the past, creating a new directory and copying my settings from the old one and then renaming both. When I did both upgrades today I did remove the [loggers] section. All I was asking is, when doing an upgrade how do I not lose any channel edits I have made? Of course this was not a problem in the past because you could not do channel edits. It was only a problem when I went from 0.8.4 to 0.8.4a.

As for those 501 errors, why they are so confusing to me is that I am clicking on the link that goes to http://ip:6077/channels.m3u (the one that reads "channels.m3u: M3U formatted channel list") and my web browser asks me if I want to download it or open with VLC (it downloads in either case). I say open with VLC and that is where VLC is getting the channel list from. If I open http://ip:6077/Locast/lineup.json it is giving me the same list, just in a different format. I can look at the errors that VLC is throwing and see that they contain the same URLs that are in the lineup.json file. If I cut it back to a single instance the problem goes away. You may not be seeing it because you are testing two same instances that only have different enable and disable settings. I was testing using three different instances with different logins and also specifying different overrides in each instance. The channel lineups were all mixed together in the channels lists (sorted by channel number rather than by instance) so it was trying to quickly switch from one lineup to another (because when VLC fails it immediately tries the next item in the list; I REALLY wish there were a way to shut that off, it is one thing I hate about VLC!) but still that should not have been an issue because each instance had its own login. But as I say if I went back a single instance then it seems to work fine and not skip channels (by the way it also lost all my channel edits again when I did that, but that's not my main concern). So I am not sure what is going on.

EDIT: I guess that for now you really have to go in and delete the files in the data directory if you are removing instances, otherwise some data from the removed instances remains and it tends to confuse the program (for example you still see tabs for the removed instances).

Sean Micklem

I happened to look and saw you had released 0.8.4b, and it fixes the issue I mentioned in my previous posts. All the channels in all instances are working reliably now, did not see a single skip. Great job! And thanks much!!

EDIT: I had this page open in my browser so did not see your two posts above before I posted this, but thanks also for that information, glad you were able to track it down. I'm not too worried about the issue with deleting instances since once I get it configured the way I want, I doubt I will ever change it or remove an instance. It definitely won't be a frequent occurrence.

K Shea

I have a couple of questions/comments about your top post.

You say, "Make sure to set the number of tuners to 1. Locast does not like the speed of channel changes during initial scanning. Doing a force scan will create the mux and service values." I assume by that you mean the value of the "Maximum # input streams " setting. By default it is 0 and you are correct, if you leave it there most of the created muxes will fail until you manually force scan them. Setting that value to 1 limits Tvheadend to scanning one at a time. But my question is, after you have done the initial scan, should you set it back to 0? Or whatever the limit is for the number of simultaneous tuners Locast will support, and if so do you happen to know what that number is? I am pretty sure that the "2" in your screenshot is not right because if I recall correctly Locast allows you to use four or five tuners simultaneously per account, but I don't know which is correct. Also just a suggestion, in the Advanced section of the Network settings I would set the "Re-fetch period (mins) " to something other than zero, because if you don't it will re-fetch your channel list every minute, which is quite unnecessary once everything is configured. I set it to 720 so it will re-fetch every 12 hours, but might go to something even higher once all the configuration is finished.

Also, you say, "Next you can setup the grabber. I use a URL grabber which is an extremely small file" but you don't mention what the name of the small file is or where to get it, and the screenshot doesn't have enough resolution to see it. You may want to mention where that small file can be obtained, if it is still available on the web. Otherwise there is always the option to use wget or curl to get the xml file where you need it, and then use tv_grab_file or a socket-based grabber or whatever you would normally use to import the file into Tvheadend, and those actions could be combined in a small shell script.

K Shea

The only thing I am not sure about is where you say: Also, I set the Network discovery in the table to "Disable".

In the version of TVH that I have there is no "Network discovery" option in the "Edit Network" dialogue. There is the "Re-fetch period (mins)", which taking your advice I have set to 43200 which is 30 days, and while in there I set the "Maximum # input streams" to 4 -- thank you for the explanations on those. But I do not see anything called "Network discovery" or anything similar that would take a non-numeric value such as Disable. The only thing I found that even came close was in the screen that came up when I clicked on Help...

>IPTV Automatic Network - Don't Probe Services >
> > Tvheadend will by default probe each playlist entry for service information. Some service providers do not allow such probing & will deny (or rate limit) access, leading to scan failures.
> >
> > To create services without probing, Service ID must be set (usually to 1) and the Scan after creation check box un-ticked.
> >

> > Note, the above two settings are only visible with the view level set to Expert.

But I think that is only for when the network is first created. I believe I am running a build of TVH from 2019, it is version 4.2.something, so it is possible that setting may only be in newer versions. The reason I have not updated is that I have been burned by updates in the past that caused things to stop working, so I usually only update TVH right after updating Ubuntu, and right now I'm still on Ubuntu 18.04. I figure that as long as it is working reliably, there is no good reason to update.

(...several hours later...) I know someone that is running a 4.3 version of TVH and I checked with them and they can't see a "Network discovery" option either, except in their ATSC network, and I can see it there in mine as well in the "Advanced Settings" section. But it just does not appear for the Locast network, which was created as an "IPTV Automatic Network" as shown in the top post.

K Shea

Oh, I thought it was supposed to be in popup you see when you add or edit a network. I see it now, and apparently it was set to Disabled by default, and I did not change it. Thanks for explaining that.

Tvheadend is funny, it is the only software I have ever used that doesn't suggest that regular users stay with the stable version, but instead suggests they use the beta or whatever they call it. The only reason I am reluctant to do that is that several years ago I had it set to auto update every time a new release came out (in other words I had not commented out their repository in my apt sources list as I do now), and there was an update that caused Tvheadend to stop working! And when you have people who are using the software as their primary means of watching and recording TV, well let's just say I heard about it immediately and with great emphasis! So now as long as I am not having issues I don't let it update anymore unless I am also updating the operating system, and the only reason I do it then is because typically I can swap out a hard drive and put the new OS version on a totally different drive, and that way if I have major problems I can go back to the previous version simply by swapping the old hard drive back.

I would probably be a whole lot less reluctant if Ubuntu didn't make it so difficult to go back to a previous version of a program, especially with programs installed using apt. They just assume that no one would ever have a reason to go back a version so they don't give you a way to save a package in the same way that MacOS or Windows does. But the other issue is that as far as I can tell, Tvheadend itself offers no way to backup and restore a configuration from within the GUI. So even if you could drop back a version, it might not work if any databases have been updated to a different format. I know there is a way you can save a directory or directories prior to an upgrade and then restore them to transfer your old settings to a new version of Tvheadend because I have the procedure to do that written down somewhere. But they don't make it easy, especially if Linux isn't your primary operating system. And going in the reverse direction might not work if any settings or databases were changed by the new version.

By way of comparison, I usually don't fear upgrading software on a Mac because if something goes wrong, or even if I just hate the new version of a program for some reason, I can easily restore the old one and its configuration from my Time Machine backup. But Linux is a whole different animal, and not particularly friendly toward those who don't use it regularly and know all its secrets. And it doesn't even have anything exactly like Time Machine, particularly if you are running a server only with no Gnome desktop or whatever. So with Linux my general philosophy is, "if it ain't broke don't fix it!" - if it weren't for the necessity of getting critical security fixes I would not even do apt updates/upgrades!

Now that you have explained the issue I know that if I do run into the problem of having more channels recording than there are available tuners, upgrading Tvheadend to the latest 4.3 version might be the way to fix it. I just hope that if I do wind up doing that it doesn't break anything! But Ubuntu 18.04 is still supported until April 2023 so I am not in any big hurry.

Sean Micklem

So far, everything appears to be working great!!! Question about instances: If I add another instance that is the same as my first instance so as to have additional tuners available, in my config.ini do I just duplicate the entire [locast_city] section and change only the username and password? I assume I'd need to tweak the instance name slightly also, for example [locast_city_2]? And if I do that will it use the channel selections I have already defined for that city's lineup? In other words what I am asking is some assurance that it won't re-enable channels that I have disabled or add duplicates of channels.

You had said it does load balancing in this case. So consider two possible scenarios:

First, you have set Tvheadend to record two programs from theSAME channel, back to back. Normally Tvheadend overlaps the recordings by padding the start and end times a little, but at least in the case of ATSC it usually uses the same tuner. I am assuming that in the case of using your program it would be the same and that Tvheadend would use the same mux and the same incoming stream for both recordings, correct? So in that case, if I am correct, only one tuner and stream would be used for both recordings, even though they overlap.

Second, you have set Tvheadend to record two programs fromDIFFERENT channels from the same city, back to back. Again there will be overlap but in this case I am assuming that it will obviously need to use two tuners and two streams. But if you have two instances for the same city, I am assuming that due to load balancing it would request the first program using the login for the first instance and the second program using the login for the second instance, correct?

Sean Micklem

Well first of all I am sorry to hear about your hard drive but it was good that you had a backup!

I installed 0.8.4c and added a second instance for the city by putting a 2 after the name (so, I have [locast_city] and [locast_city2]) with different login credentials, and upped the Client -> concurrent_listeners and locast providers -> tuner counts. And what I found was what when requesting the channel list it did work as expected, in other words there were only the channels I wanted and no duplicates. But in the Channel Editor it does show all the channels twice, however for the second instance instead of the first column being red or green it is yellow, and all of those channels are checked as enabled including the ones I had previously disabled in the original instance. So, I went through and unchecked the ones I did not want in the second instance, and now they show as pink rather than red. So I assume that if the first column is yellow or pink then data for that channel is not included in the channel listings, either because it's a duplicate (yellow) or it's a duplicateand it is disabled (pink).

The sorting by column does work and was actually quite helpful in checking that I had disabled all the duplicate channels that I didn't want, but also seeing all the channels that were SD vs. HD by sorting the metadata column. It all seems to be working quite well, so thanks again!

Sean Micklem

Well I am sorry to say that things did not go well with 0.8.4e!

First of all it would no longer start after a system reboot so I had to launch it manually. But when I attempted to view the xmltv.xml file it threw an error that said:

XML Parsing Error: no root element found Location: http://ipaddress:6077/xmltv.xml Line Number 1, Column 1:

And in the terminal output:

$ python3 tvh_main.py 2021-05-22 04:08:27,158-WARNING:main ######################################### 2021-05-22 04:08:27,159-WARNING:main MIT License, Copyright (C) 2021 ROCKY4546 Traceback (most recent call last): File "tvh_main.py", line 15, in <module> main.main(script_dir) File "/home/hts/locast/tvheadend-locast/lib/main.py", line 88, in main plugins.refresh_channels() File "/home/hts/locast/tvheadend-locast/lib/plugins/plugin_handler.py", line 90, in refresh_channels self.call_function(plugin, 'refresh_channels') File "/home/hts/locast/tvheadend-locast/lib/plugins/plugin_handler.py", line 104, in call_function call_f(*args) File "/home/hts/locast/tvheadend-locast/plugins/provider_video_locast/lib/locast.py", line 41, in refresh_channels instance.refresh_channels() File "/home/hts/locast/tvheadend-locast/plugins/provider_video_locast/lib/locast_instance.py", line 64, in refresh_channels self.channels.refresh_channels() File "/home/hts/locast/tvheadend-locast/plugins/provider_video_locast/lib/channels.py", line 64, in refresh_channels ch_dict = self.get_locast_channels() File "/home/hts/locast/tvheadend-locast/lib/common/decorators.py", line 44, in wrapper_func return f(self, *args, **kwargs) File "/home/hts/locast/tvheadend-locast/lib/common/decorators.py", line 29, in wrapper_func return f(self, *args, **kwargs) File "/home/hts/locast/tvheadend-locast/plugins/provider_video_locast/lib/channels.py", line 101, in get_locast_channels if 'videoProperties' in locast_channel['listings'][0]: IndexError: list index out of range hts@server:~/locast/tvheadend-locast$ ---------------------------------------- Exception happened during processing of request from ('127.0.0.1', 48964) Traceback (most recent call last): File "/usr/lib/python3.7/socketserver.py", line 316, in _handle_request_noblock self.process_request(request, client_address) File "/usr/lib/python3.7/socketserver.py", line 347, in process_request self.finish_request(request, client_address) File "/usr/lib/python3.7/socketserver.py", line 360, in finish_request self.RequestHandlerClass(request, client_address, self) File "/home/hts/locast/tvheadend-locast/lib/clients/web_admin.py", line 189, in __init__ super(CustomWebAdminHttpHandler, self).__init__(*args, **kwargs) File "/home/hts/locast/tvheadend-locast/lib/clients/web_admin.py", line 77, in __init__ super().__init__(*args) File "/usr/lib/python3.7/socketserver.py", line 720, in __init__ self.handle() File "/usr/lib/python3.7/http/server.py", line 426, in handle self.handle_one_request() File "/usr/lib/python3.7/http/server.py", line 414, in handle_one_request method() File "/home/hts/locast/tvheadend-locast/lib/clients/web_admin.py", line 89, in do_GET if getrequest.call_url(self, self.content_path): File "/home/hts/locast/tvheadend-locast/lib/common/decorators.py", line 75, in call_url self.url2func[_name](_tuner, *args, **kwargs) File "/home/hts/locast/tvheadend-locast/lib/clients/channels/channels_html.py", line 27, in get_channels_html html = channels_html.get() File "/home/hts/locast/tvheadend-locast/lib/clients/channels/channels_html.py", line 40, in get return ''.join([self.header, self.body]) File "/home/hts/locast/tvheadend-locast/lib/clients/channels/channels_html.py", line 57, in header list(self.tab_names.keys())[0], '")</script>' ]) IndexError: list index out of range ---------------------------------------- ---------------------------------------- Exception happened during processing of request from ('127.0.0.1', 48968) Traceback (most recent call last): File "/usr/lib/python3.7/socketserver.py", line 316, in _handle_request_noblock self.process_request(request, client_address) File "/usr/lib/python3.7/socketserver.py", line 347, in process_request self.finish_request(request, client_address) File "/usr/lib/python3.7/socketserver.py", line 360, in finish_request self.RequestHandlerClass(request, client_address, self) File "/home/hts/locast/tvheadend-locast/lib/clients/web_admin.py", line 189, in __init__ super(CustomWebAdminHttpHandler, self).__init__(*args, **kwargs) File "/home/hts/locast/tvheadend-locast/lib/clients/web_admin.py", line 77, in __init__ super().__init__(*args) File "/usr/lib/python3.7/socketserver.py", line 720, in __init__ self.handle() File "/usr/lib/python3.7/http/server.py", line 426, in handle self.handle_one_request() File "/usr/lib/python3.7/http/server.py", line 414, in handle_one_request method() File "/home/hts/locast/tvheadend-locast/lib/clients/web_admin.py", line 89, in do_GET if getrequest.call_url(self, self.content_path): File "/home/hts/locast/tvheadend-locast/lib/common/decorators.py", line 75, in call_url self.url2func[_name](_tuner, *args, **kwargs) File "/home/hts/locast/tvheadend-locast/lib/clients/channels/channels_html.py", line 27, in get_channels_html html = channels_html.get() File "/home/hts/locast/tvheadend-locast/lib/clients/channels/channels_html.py", line 40, in get return ''.join([self.header, self.body]) File "/home/hts/locast/tvheadend-locast/lib/clients/channels/channels_html.py", line 57, in header list(self.tab_names.keys())[0], '")</script>' ]) IndexError: list index out of range ---------------------------------------- 2021-05-22 04:09:41,890-WARNING:epg EPG Days to collect is too high. 2021-06-01 has no data ---------------------------------------- Exception happened during processing of request from ('127.0.0.1', 48998) Traceback (most recent call last): File "/usr/lib/python3.7/socketserver.py", line 316, in _handle_request_noblock self.process_request(request, client_address) File "/usr/lib/python3.7/socketserver.py", line 347, in process_request self.finish_request(request, client_address) File "/usr/lib/python3.7/socketserver.py", line 360, in finish_request self.RequestHandlerClass(request, client_address, self) File "/home/hts/locast/tvheadend-locast/lib/clients/web_admin.py", line 189, in __init__ super(CustomWebAdminHttpHandler, self).__init__(*args, **kwargs) File "/home/hts/locast/tvheadend-locast/lib/clients/web_admin.py", line 77, in __init__ super().__init__(*args) File "/usr/lib/python3.7/socketserver.py", line 720, in __init__ self.handle() File "/usr/lib/python3.7/http/server.py", line 426, in handle self.handle_one_request() File "/usr/lib/python3.7/http/server.py", line 414, in handle_one_request method() File "/home/hts/locast/tvheadend-locast/lib/clients/web_admin.py", line 89, in do_GET if getrequest.call_url(self, self.content_path): File "/home/hts/locast/tvheadend-locast/lib/common/decorators.py", line 75, in call_url self.url2func[_name](_tuner, *args, **kwargs) File "/home/hts/locast/tvheadend-locast/lib/clients/epg2xml.py", line 38, in xmltv_xml reply_str = epg.get_epg_xml(_tuner) File "/home/hts/locast/tvheadend-locast/lib/clients/epg2xml.py", line 65, in get_epg_xml self.plugins.refresh_epg(self.namespace, self.instance) File "/home/hts/locast/tvheadend-locast/lib/plugins/plugin_handler.py", line 98, in refresh_epg self.call_function(plugin, 'refresh_epg') File "/home/hts/locast/tvheadend-locast/lib/plugins/plugin_handler.py", line 104, in call_function call_f(*args) File "/home/hts/locast/tvheadend-locast/plugins/provider_video_locast/lib/locast.py", line 48, in refresh_epg instance.refresh_epg() File "/home/hts/locast/tvheadend-locast/plugins/provider_video_locast/lib/locast_instance.py", line 74, in refresh_epg self.epg.refresh_epg() File "/home/hts/locast/tvheadend-locast/plugins/provider_video_locast/lib/epg.py", line 56, in refresh_epg self.refresh_programs(epg_day, True) File "/home/hts/locast/tvheadend-locast/plugins/provider_video_locast/lib/epg.py", line 111, in refresh_programs for ch_data in json_data: TypeError: 'NoneType' object is not iterable ----------------------------------------

One other thing I noticed was the lineup.xml file was not right, it was all run together like all the line breaks were missing or something.

Weird thing is that after going back to the 0.8.4c version, it started at bootup and everything seems to work except now I can't see the XML guide data there anymore. in the browser it just sits there for a few seconds and then says:

The connection was reset The connection to the server was reset while the page was loading.

Can't explain why the problems all of a sudden but just wanted to let you know.

Sean Micklem

Well I tried the 0.8.4f version, specifically I expanded the zip file, renamed the resulting folder to tvheadend-locast after copying in my config.ini and the data directory from the previous version, although I then removed all the db files except channels.db as you suggested. I then attempted to do

sudo systemctl restart locast.service

but then I could not access the web interface. So I did

python3 tvh_main.py

and then it started up, but with warnings:

$ python3 tvh_main.py 2021-05-23 00:51:20,508-WARNING:main ######################################### 2021-05-23 00:51:20,509-WARNING:main MIT License, Copyright (C) 2021 ROCKY4546 2021-05-23 00:51:33,077-WARNING:epg EPG Days to collect is too high. 2021-06-01 has no data 2021-05-23 00:51:33,855-WARNING:epg EPG Days to collect is too high. 2021-06-02 has no data 2021-05-23 00:51:39,667-WARNING:epg EPG Days to collect is too high. 2021-06-01 has no data 2021-05-23 00:51:39,781-WARNING:epg EPG Days to collect is too high. 2021-06-02 has no data 2021-05-23 00:51:45,938-WARNING:epg EPG Days to collect is too high. 2021-06-01 has no data 2021-05-23 00:51:46,588-WARNING:epg EPG Days to collect is too high. 2021-06-02 has no data 2021-05-23 00:51:53,498-WARNING:epg EPG Days to collect is too high. 2021-06-01 has no data 2021-05-23 00:51:53,599-WARNING:epg EPG Days to collect is too high. 2021-06-02 has no data 2021-05-23 00:52:04,675-WARNING:epg EPG Days to collect is too high. 2021-06-02 has no data Traceback (most recent call last): File "tvh_main.py", line 15, in <module> main.main(script_dir) File "/home/hts/locast/tvheadend-locast/lib/main.py", line 111, in main plugins.refresh_epg() File "/home/hts/locast/tvheadend-locast/lib/plugins/plugin_handler.py", line 98, in refresh_epg self.call_function(plugin, 'refresh_epg') File "/home/hts/locast/tvheadend-locast/lib/plugins/plugin_handler.py", line 104, in call_function call_f(*args) File "/home/hts/locast/tvheadend-locast/plugins/provider_video_locast/lib/locast.py", line 48, in refresh_epg instance.refresh_epg() File "/home/hts/locast/tvheadend-locast/plugins/provider_video_locast/lib/locast_instance.py", line 74, in refresh_epg self.epg.refresh_epg() File "/home/hts/locast/tvheadend-locast/plugins/provider_video_locast/lib/epg.py", line 56, in refresh_epg self.refresh_programs(epg_day, True) File "/home/hts/locast/tvheadend-locast/plugins/provider_video_locast/lib/epg.py", line 118, in refresh_programs self.db.save_program_list(self.locast_instance.locast.name, self.instance, _day, program_list) File "/home/hts/locast/tvheadend-locast/lib/db/db_epg.py", line 82, in save_program_list json.dumps(_prog_list))) File "/home/hts/locast/tvheadend-locast/lib/db/db.py", line 64, in add DB.conn[self.db_name][threading.get_ident()].commit() sqlite3.OperationalError: database is locked

BUT even so I could once again access the xmltv.xml file in my browser, so that is working again. And it recreated the missing databases, so I used control-C to exit and then ran

sudo systemctl restart locast.service

again and now it appears to be working. So I suspect maybe the missing databases may have had something to do with it not restarting the first time. Or something, but it does appear to be working great again now. But after a reboot it would not work again, and even manually entering

sudo systemctl restart locast.service

no longer seems to work.

That led me to another issue. There is a log file on the system named /var/log/daemon.log and about once a minute a set of new entries like this are added:


May 23 01:33:36 server systemd[1]: locast.service: Service RestartSec=54s expired, scheduling restart.
May 23 01:33:36 server systemd[1]: locast.service: Scheduled restart job, restart counter is at 50.
May 23 01:33:36 server systemd[1]: Stopped Locast Service.
May 23 01:33:36 server systemd[1]: Started Locast Service.
May 23 01:33:37 server locast[26630]: ERROR: Config file missing None, Exiting...
May 23 01:33:37 server systemd[1]: locast.service: Main process exited, code=exited, status=1/FAILURE
May 23 01:33:37 server systemd[1]: locast.service: Failed with result 'exit-code'.

This confuses me because there is a config.ini file in the same directory as tvh_main.py and locast certainly appears to be finding in and using it, so why is it filling this log file with these messages, and how do I make it stop? Also these message appear to be duplicated to /var/log/syslog. After a reboot I see a similar issue, it says this:


May 23 01:44:11 server locast[350]: ERROR: Config file missing None, Exiting...
May 23 01:44:11 server systemd[1]: locast.service: Main process exited, code=exited, status=1/FAILURE
May 23 01:44:11 server systemd[1]: locast.service: Failed with result 'exit-code'.

If I start over from scratch (fresh install of 0.8.4f with the ONLY thing copied over being the config.ini file) then it still fails after a reboot but the log messages are slightly different:


May 23 02:09:54 server systemd[1]: locast.service: Service RestartSec=54s expired, scheduling restart.
May 23 02:09:54 server systemd[1]: locast.service: Scheduled restart job, restart counter is at 2.
May 23 02:09:54 server systemd[1]: Stopped Locast Service.
May 23 02:09:54 server systemd[1]: Started Locast Service.
May 23 02:09:55 server locast[1400]: ERROR: Config file missing None, Exiting...
May 23 02:09:55 server systemd[1]: locast.service: Main process exited, code=exited, status=1/FAILURE
May 23 02:09:55 server systemd[1]: locast.service: Failed with result 'exit-code'.

If I then go into the tvheadend-locast directory and run

python3 tvh_main.py

that runs normally and with no error or warning messages, but of course my edited channel list is gone if I do it that way, however it is showing all my instances so it is definitely finding the config.ini file. If I control-c out and reboot at that point, it still shows the same three error lines as on the previous reboot.

Basically all these issues seem to come from trying to run it as a service. I finally got fed up and did

sudo systemctl disable locast.service

and

sudo systemctl stop locast.service

and all the messages stopped. For now I have just put a line in the crontab file that reads

@reboot cd /home/hts/locast/tvheadend-locast/ && /usr/bin/python3 /home/hts/locast/tvheadend-locast/tvh_main.py &

Note that if I don't add that cd /home/hts/locast/tvheadend-locast/ at the beginning then it gives me the same error about the config file missing, so that may be a clue as to why that error keeps happening! I should mentioned that I had copied the config file to /home/hts/locast/ at one point to see if maybe it was looking for it there, but that didn't help. Anyway I hope this is not a tough nut to crack, but anyway it is running, just as a background task started at bootup and not as a service, which I suppose to real Linux devotees is some kind of sin but for now at least it works!

Sean Micklem

Thanks again! I am not sure if your first comment was intended to be a compliment or a polite way of saying I'm being a pest, but for whatever reason I have sometimes been that one guy that would quite accidentally uncover a bug that seems to affect no one else in the universe. Many years ago I was testing an online communications service and I found a bug that turned out to be a hash collision, whatever that is, but I was told that there was a less than one in a million chance that someone would trigger it, and yet I did.

I did have the number of days set to collect to 11 but changed it to 10 and still get errors when I start it manually:


$ python3 tvh_main.py
2021-05-24 07:35:16,529-WARNING:main #########################################
2021-05-24 07:35:16,530-WARNING:main MIT License, Copyright (C) 2021 ROCKY4546
2021-05-24 07:35:19,845-WARNING:epg EPG Days to collect is too high.  2021-06-01 has no data
2021-05-24 07:35:19,965-WARNING:epg EPG Days to collect is too high.  2021-06-02 has no data

(those last two lines repeat a few times)

Still no luck in starting it as a service but the errors in the log are different:


May 24 07:32:31 server systemd[1]: locast.service: Service RestartSec=54s expired, scheduling restart.
May 24 07:32:31 server systemd[1]: locast.service: Scheduled restart job, restart counter is at 1.
May 24 07:32:31 server systemd[1]: Stopped Locast Service.
May 24 07:32:31 server systemd[1]: Started Locast Service.
May 24 07:32:32 server locast[1305]: Traceback (most recent call last):
May 24 07:32:32 server locast[1305]:   File "/home/hts/locast/tvheadend-locast/tvh_main.py", line 21, in <module>
May 24 07:32:32 server locast[1305]:     main.main(script_dir)
May 24 07:32:32 server locast[1305]:   File "/home/hts/locast/tvheadend-locast/lib/main.py", line 75, in main
May 24 07:32:32 server locast[1305]:     config_obj = user_config.get_config(script_dir, opersystem, args)
May 24 07:32:32 server locast[1305]:   File "/home/hts/locast/tvheadend-locast/lib/config/user_config.py", line 36, in get_config
May 24 07:32:32 server locast[1305]:     return TVHUserConfig(script_dir, opersystem, args)
May 24 07:32:32 server locast[1305]:   File "/home/hts/locast/tvheadend-locast/lib/config/user_config.py", line 62, in __init__
May 24 07:32:32 server locast[1305]:     self.import_config(config_file)
May 24 07:32:32 server locast[1305]:   File "/home/hts/locast/tvheadend-locast/lib/config/user_config.py", line 100, in import_config
May 24 07:32:32 server locast[1305]:     utils.logging_setup(self.data['paths'])
May 24 07:32:32 server locast[1305]:   File "/home/hts/locast/tvheadend-locast/lib/common/utils.py", line 51, in logging_setup
May 24 07:32:32 server locast[1305]:     os.makedirs('data/logs')
May 24 07:32:32 server locast[1305]:   File "/usr/lib/python3.7/os.py", line 211, in makedirs
May 24 07:32:32 server locast[1305]:     makedirs(head, exist_ok=exist_ok)
May 24 07:32:32 server locast[1305]:   File "/usr/lib/python3.7/os.py", line 221, in makedirs
May 24 07:32:32 server locast[1305]:     mkdir(name, mode)
May 24 07:32:32 server locast[1305]: PermissionError: [Errno 13] Permission denied: 'data'
May 24 07:32:32 server systemd[1]: locast.service: Main process exited, code=exited, status=1/FAILURE
May 24 07:32:32 server systemd[1]: locast.service: Failed with result 'exit-code'.

But, still can start it using the crontab entry. So I hope that means something to you. I don't see anything weird about the permission settings on the data directory but maybe that is not the data being referred to in that error?

Sean Micklem

One more issue, for some reason today the channel editor insists on re-enabling many channels that I had disabled!!

Sean Micklem

G Kazaroth wrote:

> OK, I was right. It needed to create a data/logs folder and could not due to the permissions on the data folder. The data folder in general needs to be write accessible by the user of the app. Either change the owner/group or change the permissions and that should fix the issue.

Okay, this is what I have tried. I completely blew away my previous tvheadend-locast folder and then expanded the 0.8.4g.zip file to create the folder inside, which I then renamed to tvheadend-locast. I copied over the config-ini file but removed everything other than the configurations for the instances, so it would regenerate everything including the data directory. I had to redo my channel edits, but then I enabled it as a service again and after a reboot it still would not start:


May 24 10:12:16 server locast[354]: Traceback (most recent call last):
May 24 10:12:16 server locast[354]:   File "/home/hts/locast/tvheadend-locast/tvh_main.py", line 21, in <module>
May 24 10:12:16 server locast[354]:     main.main(script_dir)
May 24 10:12:16 server locast[354]:   File "/home/hts/locast/tvheadend-locast/lib/main.py", line 75, in main
May 24 10:12:16 server locast[354]:     config_obj = user_config.get_config(script_dir, opersystem, args)
May 24 10:12:16 server locast[354]:   File "/home/hts/locast/tvheadend-locast/lib/config/user_config.py", line 36, in get_config
May 24 10:12:16 server locast[354]:     return TVHUserConfig(script_dir, opersystem, args)
May 24 10:12:16 server locast[354]:   File "/home/hts/locast/tvheadend-locast/lib/config/user_config.py", line 62, in __init__
May 24 10:12:16 server locast[354]:     self.import_config(config_file)
May 24 10:12:16 server locast[354]:   File "/home/hts/locast/tvheadend-locast/lib/config/user_config.py", line 100, in import_config
May 24 10:12:16 server locast[354]:     utils.logging_setup(self.data['paths'])
May 24 10:12:16 server locast[354]:   File "/home/hts/locast/tvheadend-locast/lib/common/utils.py", line 51, in logging_setup
May 24 10:12:16 server locast[354]:     os.makedirs('data/logs')
May 24 10:12:16 server locast[354]:   File "/usr/lib/python3.7/os.py", line 211, in makedirs
May 24 10:12:16 server locast[354]:     makedirs(head, exist_ok=exist_ok)
May 24 10:12:16 server locast[354]:   File "/usr/lib/python3.7/os.py", line 221, in makedirs
May 24 10:12:16 server locast[354]:     mkdir(name, mode)
May 24 10:12:16 server locast[354]: PermissionError: [Errno 13] Permission denied: 'data'
May 24 10:12:16 server systemd[1]: locast.service: Main process exited, code=exited, status=1/FAILURE
May 24 10:12:16 server systemd[1]: locast.service: Failed with result 'exit-code'.

But nevertheless it keeps retrying and fills up the log with these:


May 24 10:14:58 server systemd[1]: locast.service: Service RestartSec=54s expired, scheduling restart.
May 24 10:14:58 server systemd[1]: locast.service: Scheduled restart job, restart counter is at 3.
May 24 10:14:58 server systemd[1]: Stopped Locast Service.
May 24 10:14:58 server systemd[1]: Started Locast Service.
May 24 10:14:59 server locast[1438]: Traceback (most recent call last):
May 24 10:14:59 server locast[1438]:   File "/home/hts/locast/tvheadend-locast/tvh_main.py", line 21, in <module>
May 24 10:14:59 server locast[1438]:     main.main(script_dir)
May 24 10:14:59 server locast[1438]:   File "/home/hts/locast/tvheadend-locast/lib/main.py", line 75, in main
May 24 10:14:59 server locast[1438]:     config_obj = user_config.get_config(script_dir, opersystem, args)
May 24 10:14:59 server locast[1438]:   File "/home/hts/locast/tvheadend-locast/lib/config/user_config.py", line 36, in get_config
May 24 10:14:59 server locast[1438]:     return TVHUserConfig(script_dir, opersystem, args)
May 24 10:14:59 server locast[1438]:   File "/home/hts/locast/tvheadend-locast/lib/config/user_config.py", line 62, in __init__
May 24 10:14:59 server locast[1438]:     self.import_config(config_file)
May 24 10:14:59 server locast[1438]:   File "/home/hts/locast/tvheadend-locast/lib/config/user_config.py", line 100, in import_config
May 24 10:14:59 server locast[1438]:     utils.logging_setup(self.data['paths'])
May 24 10:14:59 server locast[1438]:   File "/home/hts/locast/tvheadend-locast/lib/common/utils.py", line 51, in logging_setup
May 24 10:14:59 server locast[1438]:     os.makedirs('data/logs')
May 24 10:14:59 server locast[1438]:   File "/usr/lib/python3.7/os.py", line 211, in makedirs
May 24 10:14:59 server locast[1438]:     makedirs(head, exist_ok=exist_ok)
May 24 10:14:59 server locast[1438]:   File "/usr/lib/python3.7/os.py", line 221, in makedirs
May 24 10:14:59 server locast[1438]:     mkdir(name, mode)
May 24 10:14:59 server locast[1438]: PermissionError: [Errno 13] Permission denied: 'data'
May 24 10:14:59 server systemd[1]: locast.service: Main process exited, code=exited, status=1/FAILURE
May 24 10:14:59 server systemd[1]: locast.service: Failed with result 'exit-code'.

I have no idea what directory may be causing the issue, especially since the data directory was completely regenerated. Starting it from a cron job still works and then it did keep my channel edits after a reboot so no idea why I had problems with that. If I dols -l you can see there is nothing weird about the permissions or ownership:

$ ls -l /home/hts/locast/tvheadend-locast total 88 drwxr-xr-x 4 hts hts 4096 May 23 09:50 build -rwxr-xr-x 1 hts hts 4146 May 23 09:50 CHANGELOG.md -rwxr-xr-x 1 hts hts 3485 May 24 10:19 config.ini -rwxr-xr-x 1 hts hts 1554 May 23 09:50 CONTRIBUTING.md drwxr-xr-x 5 hts hts 4096 May 24 09:15 data -rwxr-xr-x 1 hts hts 182 May 23 09:50 docker-compose.yml -rwxr-xr-x 1 hts hts 321 May 23 09:50 Dockerfile -rwxr-xr-x 1 hts hts 304 May 23 09:50 Dockerfile_l2p -rwxr-xr-x 1 hts hts 304 May 23 09:50 Dockerfile_tvh -rwxr-xr-x 1 hts hts 518 May 23 09:50 Dockerfile_tvh_crypt.alpine -rwxr-xr-x 1 hts hts 849 May 23 09:50 Dockerfile_tvh_crypt.slim-buster drwxr-xr-x 14 hts hts 4096 May 24 09:15 lib -rwxr-xr-x 1 hts hts 1095 May 23 09:50 LICENSE drwxr-xr-x 4 hts hts 4096 May 23 09:50 misc drwxr-xr-x 4 hts hts 4096 May 24 09:15 plugins -rwxr-xr-x 1 hts hts 9982 May 23 09:50 README.md -rw-r--r-- 1 hts hts 22 May 23 09:50 requirements.txt -rwxr-xr-x 1 hts hts 2996 May 23 09:50 TVHEADEND.md -rwxr-xr-x 1 hts hts 824 May 23 09:50 tvh_main.py

$ ls -l /home/hts/locast/tvheadend-locast/data total 12 drwxr-xr-x 3 hts hts 4096 May 24 09:15 cache drwxr-xr-x 2 hts hts 4096 May 24 10:19 db drwxr-xr-x 2 hts hts 4096 May 24 09:15 logs

$ ls -l /home/hts/locast/tvheadend-locast/data/logs total 0 -rw-r--r-- 1 hts hts 0 May 24 09:15 cabernet.log

I wish I knew what "data" it is unable to access when trying to run as a service. Would it make any difference what the permissions are on the service file itself? Right now the owner is set to hts but it is world executable:

$ ls -l /lib/systemd/system/locast* -rwxr-xr-x 1 hts hts 954 May 23 09:50 /lib/systemd/system/locast.service

I know previously I had set the user/group to root/root but I copied it back over again today as part of trying to refresh everything, and leaving it as hts/hts doesn't seem to make any difference.

I am totally stumped, does any of this make sense to you?

Sean Micklem

Version 0.8.4j appears to be working great, starts as a service and does not leave error messages in the log file! I very much appreciate all your hard work on this, thanks again!

« Previous Page Next Page »