DaddyLive, PlutoTV, XUMO, M3U/XMLTV, SamsungTV, Plex, TVGuide interfaces (appliance)

G Kazaroth

Those logs had exceptions in them. You indicated that no exceptions were found. Since there are exceptions, I will take a look and see what is the cause.

Sterling Rorden

G Kazaroth wrote:

> Those logs had exceptions in them. You indicated that no exceptions were found. Since there are exceptions, I will take a look and see what is the cause.

Thanks. I guess didn't understand what was meant by exceptions. I just looked for errors around the time I was having problems.

K Shea

For some reason I am having a problem with the EPG not sending program data for any of the m3u channels. If I disable the EPG for everything but one m3u provider and then look at the xmltv.xml file, I see all the channels listed between channel tags, but there are no programme tags at all in the file. But, if I enable PlutoTV then I do get program listings for that. I have checked that the XMLTV File URL's are correct, and I figure Cabernet must be getting them or else it would not be able to get the channels to put between the channel tags.

I know at one time it used to bring in program data for those m3u instances and I am a little confused as to why it's not working now.

EDIT: Figured it out. For some reason the scheduler had lost its EPG timers for the m3u instances (but not for PlutoTV) and it hadn't updated the guide data in several days (probably not since the last reboot).

G Kazaroth

I guess I need to add a conditional popup on deleting schedule tasks. Most likely was caused by hitting the trash instead of the play button.

G Kazaroth

To Sterling:

An exception is not an error. An error is an issue that has been processed, appropriately. An exception can be an issue that is either processed or not processed. Exceptions are found in the log by searching for the word exception. In your log, you have processed and unprocessed exceptions. None are considered errors.

As an example, you could have a FileNotFoundException, but it may be expected at times, so you log it as INFO. Once it is logged as INFO, it is a processed exception. Unprocessed (or unexpected) exceptions happen all the time. This is caused by unknown operator conditions that the author was not expecting. So, when an unprocessed exception occurs, authors are very interested and normally will patch the code to handle the condition.

As I mentioned, I will be looking into the unprocessed exception that was found. As for the connection refused, I did find some interesting info. It looks like you have the network refresh set to 60 minutes (called Re-fetch period). What you are saying is you believe the channel listing is changing every hour to have different channels. In reality, if channels change weekly, I would be surprised. Although this setting probably does not cause the connection refused, it is probably something you should set to a MUCH higher value. 1440 min is what I use. You are basically banging on all channels every hour looking for new channels (that rarely change). Run a search on "web_handler" in the log to see what is being requested, if you want. The xmltv.xml is successfully being executed twice each time. (TVH sends 2 requests as expected) The xmltv.xml requests took about 60 seconds according to the log to complete. Check the bottom of the tv_grab_url file and see if the -T option is greater that 60 seconds, but I believe it should be.

Sterling Rorden

G Kazaroth wrote:
> To Sterling:
> An exception is not an error. An error is an issue that has been processed, appropriately. An exception can be an issue that is either processed or not processed. Exceptions are found in the log by searching for the word exception. In your log, you have processed and unprocessed exceptions. None are considered errors.
>
> As an example, you could have a FileNotFoundException, but it may be expected at times, so you log it as INFO. Once it is logged as INFO, it is a processed exception. Unprocessed (or unexpected) exceptions happen all the time. This is caused by unknown operator conditions that the author was not expecting. So, when an unprocessed exception occurs, authors are very interested and normally will patch the code to handle the condition.
>

> As I mentioned, I will be looking into the unprocessed exception that was found. As for the connection refused, I did find some interesting info. It looks like you have the network refresh set to 60 minutes (called Re-fetch period). What you are saying is you believe the channel listing is changing every hour to have different channels. In reality, if channels change weekly, I would be surprised. Although this setting probably does not cause the connection refused, it is probably something you should set to a MUCH higher value. 1440 min is what I use. You are basically banging on all channels every hour looking for new channels (that rarely change). Run a search on "web_handler" in the log to see what is being requested, if you want. The xmltv.xml is successfully being executed twice each time. (TVH sends 2 requests as expected) The xmltv.xml requests took about 60 seconds according to the log to complete. Check the bottom of the tv_grab_url file and see if the -T option is greater that 60 seconds, but I believe it should be.

OK, I understand what you mean by "exception" now.

I changed Re-fetch to 1440 (I had left it at the default) for Pluto TV and USTVGO.

The -T option appears to be set to 60 seconds:

wget -q -T 60 -O - ${url}

Should I increase it? I've noticed that the grab can take longer than 60 seconds but it still completes.

Thanks

G Kazaroth

Cabernet does chunk the xmltv.xml output in instance/day chunks, so it should send something to the client fairly often, but if it somehow hangs due to I/O buffering or whatever, wget will terminate the connection after 60 seconds of dead space. Per the man page on wget:
> "if, at any point in the download, no data is received for more than the specified number of seconds, reading fails and the download is restarted."
You can make a decision on what timeout you want to use. It was just another possible reason you could be having issues.

Sean Micklem

Got a question for you. Up until recently, if you went to https://i.mjh.nz/PBS/ and used theall.m3u8 file as a source in VLC, it would load up a whole list of dozens of PBS stations from across the country, and you could play any of them. And of course you could select any of the streams from that m3u8 file that you might have had an interest in, and add it to your own custom list of m3u streams for use with Cabernet.

But sometime recently things changed, for one thing the list of stations shrank considerably, but even the ones that remain mostly are unplayable. The only exception I could find that would still play was the one from Nebraska Public Media, and who knows how long that one will continue working.

However there is a new m3u8 link on that page at https://i.mjh.nz/PBS/kodi.m3u8 and at first I could not understand what it was doing (it doesn't work in VLC except for that same Nebraska feed) so I went to the Kodi forum and found this thread: https://forum.kodi.tv/showthread.php?tid=357548 - If you look at message #4 in that thread (https://forum.kodi.tv/showthread.php?tid=357548&pid=3071833#pid3071833) it says this:

> But yes, there were changes at PBS - 95% of the streams are now using Widevine DRM.

> Add-on 0.1.5 added support for these.

> https://i.mjh.nz/PBS/all.m3u8 now only has the non DRM streams

> There is a new https://i.mjh.nz/PBS/kodi.m3u8 which includes the DRM streams.

> But will only work on players that support Widevine and the #KODIPROP tags.

So using that addon you could watch any of those streams live in Kodi, but not using Tvheadend.

What I am curious about is if you understand what he's doing in his Kodi addon to support these streams, and if there would be any easy way to add that same ability to Cabernet. Personally I am not really interested in having every one of those PBS stations appear in the Tvheadend guide, but there were a few select ones that I had some interest in.

G Kazaroth

There are 3 types of streams: Open, Encrypted and DRM. Open is by definition a clear stream that needs no additional processing. Encrypted is one where the provider gives the http URL to the decryption key in the m3u8 file so we can decrypt it. DRM provides a unique url string that informs DRM knowledgeable software how to obtain the key needed to decrypt the stream. The ones I looked into required a login to obtain the key, so Cabernet will log that it cannot DRM the stream if it finds one.

Since Kodi is providing one type of DRM, I will take a look. My understanding is Kodi uses OS DRM libraries to obtain the key.

Sean Micklem

Thanks. I had a feeling that if Kodi could do it, Cabernet just might be able to do it as well.

Sean Micklem

All of a sudden I was getting no guide data. The http://localhost:6077/xmltv.xml file only contained:


HTTP/1.0 501 Not Implemented
Server: BaseHTTP/0.6 Python/3.7.3
Date: Thu, 30 Dec 2021 10:33:48 GMT
Content-type: text/html

<html>
        <head></head>
        <body>
            <h2>501 - Server Error</h2>
        </body>
    </html>

If I tried to get an individual instance, for example http://localhost:6077/plutotv/xmltv.xml that worked. So I unchecked "XMLTV Enabled" and saved the change for ALL the providersexcept PlutoTV, same problem - http://localhost:6077/xmltv.xml returned the error, while http://localhost:6077/plutotv/xmltv.xml worked fine.

Turned out I had deleted a m3u instance from the config file but forgot to go into Data Mgmt and clear the data for that instance there. So, when Cabernet was trying to collect EPG data, it was throwing errors like this:


Dec 30 05:59:34 systemname locast[2935]: 2021-12-30 05:59:34,487-ERROR:web_admin UNEXPECTED EXCEPTION on GET='m3u_custom2'
Dec 30 05:59:34 systemname locast[2935]: Traceback (most recent call last):
Dec 30 05:59:34 systemname locast[2935]:   File "/home/user/cabernet/lib/clients/web_admin.py", line 104, in do_GET
Dec 30 05:59:34 systemname locast[2935]:     elif getrequest.call_url(self, self.content_path):
Dec 30 05:59:34 systemname locast[2935]:   File "/home/user/cabernet/lib/common/decorators.py", line 228, in call_url
Dec 30 05:59:34 systemname locast[2935]:     self.url2func[_name](_webserver, *args, **kwargs)
Dec 30 05:59:34 systemname locast[2935]:   File "/home/user/cabernet/lib/clients/epg2xml.py", line 40, in xmltv_xml
Dec 30 05:59:34 systemname locast[2935]:     epg.get_epg_xml(_webserver)
Dec 30 05:59:34 systemname locast[2935]:   File "/home/user/cabernet/lib/clients/epg2xml.py", line 87, in get_epg_xml
Dec 30 05:59:34 systemname locast[2935]:     self.gen_channel_xml(xml_out, channel_list)
Dec 30 05:59:34 systemname locast[2935]:   File "/home/user/cabernet/lib/clients/epg2xml.py", line 172, in gen_channel_xml
Dec 30 05:59:34 systemname locast[2935]:     if not self.config[config_section]['enabled']:
Dec 30 05:59:34 systemname locast[2935]: KeyError: 'm3u_custom2'

I note that the program is still identifying as locast in the log so you might want to think about changing that to cabernet.

On another note, I appreciate that you can disable "Channel # in Name" but kind of wish that option were available on a per-instance basis. The reason is I'd like to turn it off, but don't want to have to deal with having several hundred channels all getting renamed at once. I'd prefer to deal with one instance at a time. Also, I wish there was a way on a per-instance basis to append a string to each channel NAME (not the number), for example to append " (PlutoTV)" after all the PlutoTV channels. I realize that can be done manually in the channel editor, but that's kind of a pain if you are doing several hundred channels, and it's a greater pain if you ever decide you want to change that suffix for some reason. The point is that I'd like to get rid of channel numbers as part of the channel name entirely (but preferably not all at once), but still have a way to know which streaming service any given channel is coming from.

And speaking of the channel editor, one thing that would be useful is a button that downloads the icons of allcurrently displayed andenabled channels in the largest available size, giving them the name of the channel (but keeping the original extension such as .png). These files could then be loaded into a program such as GIMP for additional processing. For example, one thing I hate is when they leave a big transparent border around an icon, which has the effect of making it look smaller than all the other icons in Kodi. In such a case I can bring it into GIMP and do a "Zealous Crop" to remove the border but it is a pain to have to download each such icon manually, which is why it would be nice to be able to download all of them en masse.

None of this is really urgent, it's just things that I have been thinking for a while that it would sure be easier if Cabernet could do this or that. So those are just suggestions in case you feel like tweaking Cabernet a bit more.

G Kazaroth

On your exception, the key error indicates that you have plugin: M3U; instance: custom2; in your database, but you have removed it from your config.ini file.

As you mentioned, to clean up the database, you need to remove the m3u/custom2 instance. If you go to Data Mgmt/Delete Instance, you should find the M3U:custom2 instance in the list. Remove that and you should be good.

Sean Micklem

G Kazaroth wrote:
> On your exception, the key error indicates that you have plugin: M3U; instance: custom2; in your database, but you have removed it from your config.ini file.
>

> As you mentioned, to clean up the database, you need to remove the m3u/custom2 instance. If you go to Data Mgmt/Delete Instance, you should find the M3U:custom2 instance in the list. Remove that and you should be good.

Yeah, I had figured that out, I had just thought that maybe Cabernet could handle that situation a little more gracefully, or at least print some kind of warning on the home screen or something. I would guess that most users don't delete instances often, so when you do it is easy to forget that you also need to go in and remove the data for that instance. All I knew initially was that I wasn't getting guide data and I had no idea why until I checked the log (which was not my first course of action because usually I never read log files, and also I always forget which log file Cabernet puts its entries in).

Not a huge deal, the problem is fixed. I guess I don't know if you consider Cabernet finished and have moved on to other things, or if you still feel like polishing it a bit more. I know at one point you had talked about putting up a web page for it but then we never heard any more about that. One thing I have been thinking for a while now is that it would be good to have some kind of wiki where you could have a page corresponding to each settings page, and also for each streaming service (including the m3u ones) with the suggested settings for each one. I always feel like I am flying blind whenever I have to adjust settings, even though many of them do have mouseover tooltips. The settings that I always feel the most uncertain about in are the provider instance tabs. For example, there is the "Enable PTS/DTS Resync" setting. What I have discovered is if you enable it on streams that don't require it, they may not be playable (the stream will never start playing). But the opposite is also true; if you don't enable it on a stream that does require it, it will either not play at all or you might get video but no audio. And there is just no way to tell ahead of time which streams require it and which don't. I wound up having to create two lists and instances for individual feeds, one for the channels that don't need PTS/DTS Resync and another for those that do. If I add a channel I try it in one list first and if it doesn't work there, then in the other.

I totally disabled Xumo because I never could figure out good settings for that; any time I tried to record something it would either record the wrong show or it would record for just a few minutes and quit, and since most of their channels are also present in other services and their guide data was often minimal or incomplete, I just decided that Xumo wasn't worth the hassle.

Please don't get the idea that I am trying to load you down with a bunch of projects if you are ready to move on to something else. I only mention this stuff in case you happen to be looking for something to do over the winter months, but also because these notes might be helpful to me or someone else at some point. I just know it's been over a month since the last update was released and I didn't know if you had moved on to other projects or if you've just been occupied with other things over the holidays or what. In any case, no pressure, for the most part Cabernet is working quite well now, and that's the main thing.

Also (year end reflection here) I know it must have been pretty demoralizing when Locast abruptly shut down (and then those other few unofficial Locast-like streams did the same), I know that was your original purpose for the program and I think we were all shocked by how quickly Locast just folded after that terrible decision by a judge that was probably born when horses and buggies and telegrams were still a thing. Locast had always led us to believe that they had the law on their side and that they would fight any adverse legal decision vigorously, but then when they had one bad ruling against them they folded like a tent in a hurricane, without even making any kind of appeal. My guess is they decided that if they could not strongly encourage people to donate by interrupting programming, the service would never be financially viable. But still I don't think anyone expected them to just up and shut everything down that quickly. I also think if they hadn't relied on that donation nag everyone would have been a lot more sympathetic toward them and might even have donated to a legal defense fund for them, but who knows.

Sterling Rorden

G Kazaroth wrote:
> Cabernet does chunk the xmltv.xml output in instance/day chunks, so it should send something to the client fairly often, but if it somehow hangs due to I/O buffering or whatever, wget will terminate the connection after 60 seconds of dead space. Per the man page on wget:
> > "if, at any point in the download, no data is received for more than the specified number of seconds, reading fails and the download is restarted."

> You can make a decision on what timeout you want to use. It was just another possible reason you could be having issues.

I changed -T to 120 but the problem is still the same. The problem occurs consistently every 2 days sometimes successfully running one more time after 48 hours has expired. It seems like a token is expiring or something like that.

Sterling Rorden

Sterling Rorden wrote:
> G Kazaroth wrote:
> > Cabernet does chunk the xmltv.xml output in instance/day chunks, so it should send something to the client fairly often, but if it somehow hangs due to I/O buffering or whatever, wget will terminate the connection after 60 seconds of dead space. Per the man page on wget:
> > > "if, at any point in the download, no data is received for more than the specified number of seconds, reading fails and the download is restarted."
> > You can make a decision on what timeout you want to use. It was just another possible reason you could be having issues.
>

> I changed -T to 120 but the problem is still the same. The problem occurs consistently every 2 days sometimes sometimes successfully running one more time after 48 hours has expired. It seems like a token is expiring or something like that.

Some more information:

If I SSH into the box and issue a Telnet ping to 6077 I get a connection refused.
If I ping 9981 on the same box (Tvheadend) I get a response that the connection is open.
If I try to access Cabernet via the web access I get a connection refused response from my browser.
I can access the Tvheadend web interface on the same machine.

However, I can tell from the Cabernet log that Cabernet is still active.

Is there a way to automatically restart Cabernet on a schedule? I could give that a try.

G Kazaroth

On USTVGO, I am getting a 404 error for the url https://ustvgo.tv/data.php

Looking into the issue.

They have a new API and web site. It's going to take a little time to get it updated, so in the meantime, I recommend disabling the plugin.

K Shea

On Reddit someone had posted this:

> Just use ustvgo_to_m3u by Benmoose39. Not an “Addon” so it needs an initial configuration but works great with Kodi. When USTVGO changed something yesterday and it stopped working it was fixed same day. Before someone asks for help, he has a discord Channel

I assume they are talking about this: https://github.com/benmoose39/ustvgo_to_m3u

So if you get stuck, I don't know if you would be able to see what changes he made to his program in the last few days (not real familiar with how Github works) but if you can then you might be able to determine what needs to be changed in Cabernet. Hope this helps.

G Kazaroth

I think I have it working. They shutdown the backend api so they can control which channels require their VPN to stream. This way they get a monthly subscription for pay to view channels. If USTVGO requires a VPN that you do not have, Cabernet will remove those channels from the list. To see the list of channels with and without VPN, check out.
> https://github.com/benmoose39/ustvgo_to_m3u/blob/main/ustvgo_channel_info.txt

Those with VPN at the end will require a VPN to stream.

It may come out today, if testing gets done.

K Shea

Thanks, it seems to be working fine on the three or four channels I tested it on. And it does appear to be removing the channels that require a VPN (I don't have one) from Cabernet's channel list.

But I saw another issue when I went into Cabernet to do the update - the main page showed a M3U tuner in use that I know was not being used by Tvheadend. This worries me that it may not always be releasing channels when Tvheadend disconnects. Is it possible to set a maximum stream timer that will automatically disconnect a tuner that's been in continuous use for longer than a specified amount of time? For example I'd probably want to set it for six hours because I cannot conceive of a circumstance where a tuner should be in continuous use for that long (that is long enough for two movies back to back on most channels).

I'm aware that I could set up a cron job to do a forced restart of Cabernet in the middle of the night, only reasons I don't like that idea are because first of all it could actually be PVR'ing something at that time, and also Cabernet might be in the middle of some process involving the database during that time (not that I really think that would cause a problem, because Ithink it would probably start that process over when it comes back up).

G Kazaroth

If you want to do a restart, you should use the scheduler and not some cronjob that kills processes. The scheduler will cause a clean shutdown and restart, so any database issue would be mitigated.

I am not one to use band-aids to fix an issue, so if there is a tuner showing on the home page, it would be nice to fix it. The issue is, we need more info.
# Is the tuner actually running or is the state wrong? If no INFO logging is going on, then the tuner is not really running.
# Is there more than three python3 tvh_main.py processes running? When no tuners are in use, there should only be 3 processes. If more than 3 are there and no logs show an active tuner, then the process is not terminating properly.

# Are there any exceptions in the log? This can help to resolve how a tuner is still active while terminated.

So, the next time you see a tuner process still present, take a look and see if you can find any info.
Thanks

« Previous Page Next Page »