Tvheadend

I had this working running from command line in debian but after installing and removing some other software on the same machine it seems to be failing now.

When I go to get http://IP:6077/channels.m3u in the browser I get:

501 - Server Error

The console on host running the script generates the error:

UNEXPECTED EXCEPTION on GET='stream_url'
Traceback (most recent call last):
  File "/cabernet/lib/clients/web_admin.py", line 104, in
    elif getrequest.call_url(self, self.content_path):
  File "/cabernet/lib/common/decorators.py", line 228, in call_url
    self.url2func[_name](_webserver, *args, **kwargs)
  File "/cabernet/lib/clients/channels/channels.py", line 42, in channels_m3u
    _webserver.do_mime_response(200, 'audio/x-mpegurl', get_channels_m3u(
  File "/cabernet/lib/clients/channels/channels.py", line 100, in get_channels_m3u
    uri = sid_data['json']['stream_url']
KeyError: 'stream_url

Wondering if I removed some dependency while removing software.

Running V 0.9.7.5

Thanks

I am so glad to be passed the previous trolling and spinning. Thank you for giving the problem. The issue is the stream_url key is not in the database record for the plugin you wanted to get.

This indicates you have a corrupted database file. You have two choices, either delete the database files from data/db folder or you can go to Data Mgmt and request the "Reset Channel Data" for all plugins. Then you can regenerate the channel data by either going to the schedule data and requesting a channel update or restarting the app.

If you are still having issues, I would look at what is happening during the initial pull of channel data from the provider.

You may also need to determine which plugin data is having the issue. You can use the URLs
http://IP:6077/[pluginname]/channels.m3u
or
http://IP:6077/[pluginname]/[instancename]/channels.m3u
to pull just the one instance/plugin. Example:
http://IP:6077/plutotv/channels.m3u
Instance name is what is found in the config.ini. [plutotv_instance] has a name of "instance".

I only use xumo. No other instances are installed.

deleting or renaming the channels.db database does not work.

Once the database is rebuilt, it just returns to the same error.

Hi

So I have an issue trying to record from the Popflix channel that is available via SamsungTVPlus (gb.m3u8). I use cabernet via TVHeadend. The program will record for upto an hour and then cabernet will stop the stream. TVHeadend however thinks that the recording goes to completion (see attached screenshot).

I see the following in the cabernet logs:

2022-06-10 09:28:29,713-DEBUG:m3u8_queue Removed https://af4689b464b143c89c08aab63ce4718d.mediatailor.us-east-1.amazonaws.com/v1/segment/04fd913bb278d8775298c26fdca9d9841f37601f/Samsung-gb_Popflix/16f7ff09-7474-43ab-9032-f1eafb7f02e1/2/438670 from play queue 23794
2022-06-10 09:28:29,714-DEBUG:m3u8_queue Removed https://af4689b464b143c89c08aab63ce4718d.mediatailor.us-east-1.amazonaws.com/v1/segment/04fd913bb278d8775298c26fdca9d9841f37601f/Samsung-gb_Popflix/16f7ff09-7474-43ab-9032-f1eafb7f02e1/2/438671 from play queue 23794
2022-06-10 09:28:29,714-DEBUG:m3u8_queue Removed https://signatureentertainment-popflix-1-gb.samsung.wurl.tv/56497864/hls-v2/3000-00005.ts?e=1654850589&st=5baabb968750476720cbcb17bf15d45a from play queue 23794
2022-06-10 09:28:29,716-DEBUG:m3u8_queue Added https://signatureentertainment-popflix-1-gb.samsung.wurl.tv/56497864/hls-v2/3000-00007.ts?e=1654850601&st=edad55f7babbc4f59ff759581447d4eb to play queue 23794
2022-06-10 09:28:29,993-ERROR:m3u8_queue UNEXPECTED EXCEPTION M3U8Queue=('https://signatureentertainment-popflix-1-gb.samsung.wurl.tv/56497864/hls-v2/3000-00006.ts?e=1654850595&st=e30ea9315670e57a2086ac09ff8eb224', datetime.datetime(2022, 6, 10, 8, 28, 11, 790000, tzinfo=datetime.timezone.utc))
Traceback (most recent call last):
File "/home/pi/cabernet/lib/streams/m3u8_queue.py", line 106, in run
self.process_m3u8_item(queue_item)
File "/home/pi/cabernet/lib/streams/m3u8_queue.py", line 249, in process_m3u8_item
PLAY_LIST[uri_dt]['played'] = True
KeyError: ('https://signatureentertainment-popflix-1-gb.samsung.wurl.tv/56497864/hls-v2/3000-00006.ts?e=1654850595&st=e30ea9315670e57a2086ac09ff8eb224', datetime.datetime(2022, 6, 10, 8, 28, 11, 790000, tzinfo=datetime.timezone.utc))
2022-06-10 09:28:31,776-NOTICE:web_tuner Provider Connection Closed, ch_id=GBAJ400045P5

Any ideas what is going on?

Thanks.

Issue is a bug which did not have a key "played" in a hash table in the current playlist table that was expected. I am looking into the issue.

Hello everyone, this is a brilliant program and I decided to mess around with it again. Just a couple of questions:

I tried the usatvgo setup and got it to work. The problem that I have is that it takes about 20 sec to load into Kodi or vlc. The error that I get is " H264 @ #256 Continuity counter error (total 1)" and then the channel loads in. It buffs a few times and then straightens itself out if I pause the stream for a few seconds. Another problem is if a timer is set to record. It is taking too long for the channel to start and tvheadend will keep on timing out while trying to record.

Any tips on how to correct this?

Thanks Lisa~~

Sorry about being absent. I have been working on a kitchen remodel (which is a lot more work than I thought). Due to supply chain issues, I have some time, so getting back into it for a short. As for Lisa's issue, there is very little info provided, but the most likely issue is that you are having IP or TCP packet collisions causing retries. This is a guess, but it could be your router/Wifi or cabling. If you are on Wifi, I would try to direct connect the box to the router and see if the issue persists. If none of this info helps, we are going to need the Cabernet logs to understand what is going on (at either the INFO or DEBUG level). Continuity errors can easily occur at the start of the stream. It is caused by the DTS for the audio or video to not be in prefect sync. It has to be in sync to 93,000th of a second to not get it. I have also seen it during show-to-ad or ad-to-show transitions. If you get one that says the continuity is extremely large, then that could be an issue.

Thank you for your response. I was curious why this was happening and the error is sporadic. I do have a good internet speed ranging 1 gig internet speed and everything is directly wired.

I have a subscription to philo and fubotv. Is there a plug-in that I can use with these paid services to make them work with cabinet?

Thanks
Lisa

As indicated in my last post a couple of weeks ago I had this working then just started getting 501 errors from cabernet when trying to get the playlist.

As I run this from command line I deleted the cabenet folder which apparently contains all cabernet data as well and copied the 0.9.7.5 latest source.

I created a config.ini file with the following contents:

[plutotv_default]
label = PlutoTV Instance

[xumo_default]
label = Xumo Instance

[ustvgo_default]
label = USTVGo Instance

# /usr/local/bin/python3 /root/cabernet/tvh_main.py

I then went to Cabernet web interface and enabled Pluto xumo and ustvgo and their instances. Now back on terminal, I hit ctrl+c to shut down cabernet

then restarted cabernet with the same command as above.

The following was then seen as output

/usr/local/lib/python3.8/site-packages/cryptography/hazmat/bindings/openssl/binding.py:197: CryptographyDeprecationWarning: OpenSSL version 1.1.0 is no longer supported by the OpenSSL project, please upgrade. The next release of cryptography will be the last to support compiling with OpenSSL 1.1.0.
  warnings.warn(
2022-06-21 15:58:09,100-WARNING:main #########################################
2022-06-21 15:58:09,101-WARNING:main MIT License, Copyright (C) 2021 ROCKY4546
2022-06-21 15:58:55,527-WARNING:db Update request ignored, retrying 4, no such column: False
2022-06-21 15:58:56,058-WARNING:db Update request ignored, retrying 3, no such column: False
2022-06-21 15:58:56,400-WARNING:db Update request ignored, retrying 2, no such column: False
2022-06-21 15:58:57,031-WARNING:db Update request ignored, retrying 1, no such column: False
2022-06-21 15:58:57,543-WARNING:db Update request ignored, retrying 0, no such column: False

I hope the above shows up correctly as I do not understand how to post code here , apparently I need to define a specific kind of code, so trying the "pre" option

In short I never see any channels listed in channels.m3u now and think maybe I missed a step, but not sure what I am doing wrong at this point.

Thanks

You have the log set to Warning, so not much info is being provided. The database warnings indicate one of your DB files is corrupted and should probably be deleted and the app restarted. Increasing the logging level will show you which database is causing the issue.

Robert, I have confirmed that the issue with Popflix was not associated with the exception you saw, but rather the ads from amazonaws. Once the ads finishes playing and the program resumes, the stream seems to have issues. I am running a number of different configuration to see if I can get it to play through the ads and keep going.

I also did review the exception in the code and it is not obvious as to the cause. My best guess is the date time stamp for the stream changed for the 6 second segment in mid-stream, but there is no evidence of this. I have yet to reproduce the exception, but the stream issue with ads is easily reproduced.

As Indicated in my previous post, I already deleted and re-installed, and in fact I have done that twice now installing to two different disks thinking that perhaps a disk was failing thus causing corruption but clearly that is not the issue.

So you did not say your suggestion of the log levels only that I had it at a level that was not useful. This is one of the things I find confusing in your software also with the use of "expert" and "advanced", where expert is usually more than advanced yours seems reversed. So I have set it at debug but who knows maybe I need to be at the other extreme? I would think if I wanted to "debug" the program that is where it should be but I have nothing in the log after zapping the old data again restarting and waiting several minutes. So it seems very confusing to me.

After setting at "debug' Level and waiting a log while then requesting channels.m3u, I get:

2022-06-24 01:34:06,063-DEBUG:decorators Registering GET URL: /api/channels
2022-06-24 01:34:06,064-DEBUG:decorators Registering GET URL: /api/channels_form
2022-06-24 01:34:06,066-DEBUG:decorators Registering GET URL: /discover.json
2022-06-24 01:34:06,068-DEBUG:decorators Registering GET URL: /device.xml
2022-06-24 01:34:06,070-DEBUG:decorators Registering GET URL: /lineup_status.json
2022-06-24 01:34:06,076-DEBUG:decorators Registering GET URL: /api/configform
2022-06-24 01:34:06,078-DEBUG:decorators Registering GET URL: /tunerstatus
2022-06-24 01:34:06,080-DEBUG:decorators Registering GET URL: /api/upgrade
2022-06-24 01:34:06,082-DEBUG:decorators Registering GET URL: /config.json
2022-06-24 01:34:06,084-DEBUG:decorators Registering GET URL: /api/schedulehtml
2022-06-24 01:34:06,086-DEBUG:decorators Registering GET URL: /api/scheduler
2022-06-24 01:34:06,087-DEBUG:decorators Registering GET URL: /api/restart
2022-06-24 01:34:06,089-DEBUG:decorators Registering POST URL: /api/data_mgmt
2022-06-24 01:34:06,091-DEBUG:decorators Registering POST URL: /api/channels_form
2022-06-24 01:34:06,092-DEBUG:decorators Registering POST URL: /lineup.post
2022-06-24 01:34:06,094-DEBUG:decorators Registering POST URL: /api/configform
2022-06-24 01:34:06,095-DEBUG:decorators Registering POST URL: /api/schedulehtml
2022-06-24 01:34:06,097-DEBUG:decorators Registering GET URL: /html/
2022-06-24 01:34:06,098-DEBUG:decorators Registering GET URL: /images/
2022-06-24 01:34:06,100-DEBUG:decorators Registering GET URL: /modules/
2022-06-24 01:34:06,102-DEBUG:decorators Registering GET URL: /temp/
2022-06-24 01:34:06,106-NOTICE:main Starting streaming tuner website on 10.0.0.250:5004
2022-06-24 01:34:06,104-INFO:web_handler WebAdminHttpHandler Now listening for requests. Number of listeners=8
2022-06-24 01:34:06,123-DEBUG:web_tuner TunerHttpHandler Implementing 3 tuners for M3U
2022-06-24 01:34:06,139-DEBUG:web_tuner TunerHttpHandler Implementing 3 tuners for PlutoTV
2022-06-24 01:34:06,143-DEBUG:web_tuner TunerHttpHandler Implementing 4 tuners for USTVGO
2022-06-24 01:34:06,161-DEBUG:web_tuner TunerHttpHandler Implementing 3 tuners for XUMO
2022-06-24 01:34:06,196-INFO:web_handler TunerHttpHandler Now listening for requests. Number of listeners=13
2022-06-24 01:34:06,322-NOTICE:scheduler Running inline task Channels:Refresh USTVGO Channels
2022-06-24 01:34:06,334-DEBUG:plugin_channels Channel data still new for USTVGO default, not refreshing
2022-06-24 01:34:06,384-NOTICE:main Cabernet is now online.
2022-06-24 01:34:06,729-NOTICE:scheduler Running threaded task Applications:Check for Updates
2022-06-24 01:34:06,767-NOTICE:scheduler Running threaded task EPG:Refresh USTVGO EPG
2022-06-24 01:34:06,785-DEBUG:plugin_epg EPG still new for USTVGO default, not refreshing
2022-06-24 01:38:03,757-ERROR:web_admin UNEXPECTED EXCEPTION on GET='stream_url'
Traceback (most recent call last):
  File "/media/Buffer/cabernet/lib/clients/web_admin.py", line 104, in do_GET
    elif getrequest.call_url(self, self.content_path):
  File "/media/Buffer/cabernet/lib/common/decorators.py", line 228, in call_url
    self.url2func[_name](_webserver, *args, **kwargs)
  File "/media/Buffer/cabernet/lib/clients/channels/channels.py", line 42, in channels_m3u
    _webserver.do_mime_response(200, 'audio/x-mpegurl', get_channels_m3u(
  File "/media/Buffer/cabernet/lib/clients/channels/channels.py", line 100, in get_channels_m3u
    uri = sid_data['json']['stream_url']
KeyError: 'stream_url'
2022-06-24 01:38:03,761-WARNING:web_handler [10.0.0.183] "GET /channels.m3u HTTP/1.1" 501 -

That is shortly after clearing all databases and once the database is rebuilt , just the same 501 error again. If that is the wrong level of logging please tell me which level of logging to use. There was little more in the log really, but I do see that somewhere it retained the epg data .. No idea where that is but apparently not in a database.

How do I set up an m3u instance?

I did it like this but on the channels display I get a Partial or broken tab for it on channels editor, so it must not be right.

[m3u_default]
label = M3U Instance

Thanks

That looks fine in the config.ini. You need to restart to have it applied. Then go into the web site and configure the "M3U Instance". It has no idea where to get anything, so you will need to configure it before it will work. Once configured, enabled it and restart the app. It should come up. The log is normally very helpful at the INFO level when setting up M3U streams.

Ok that is what I have done a few times now with a few different m3u files and different file locations.

After doing it all again, my entry for m3u looks like this

[m3u_default]
label = M3U Instance
channel-m3u_file = http://127.0.0.1:8866/playlists/my.m3u
channel-m3u_file_type = m3u
channel-start_ch_num = 50
epg-xmltv_file = http://127.0.0.1:8866/my.xml
epg-xmltv_file_type = xml
player-stream_type = m3u8redirect

This results in a deformed tab with no text - see attached image
Log attached. I do see an error in the log but that must be where the guide data comes from for USTVgo???

I will take a look with your configuration, but it is interesting that you are using the m3u8direct process that assume Cabernet has nothing to do and just passes the M3U8 data directly to the client DVR. So, many of the parameters do nothing in Cabernet.

Thank you!

Please keep in mind , it has never worked anyway so that redirect setting is irrelevant while no channels appear anyhow. I am not planning on using it in that mode, just showing what I currently have.

If you mean to imply that setting is somehow inhibiting the tab from showing, then that is different . Changing that setting does not correct the issue.

M3uRedirect is not the default setting. I am sure, because I tried changing several options to see if that would correct the issue. I am certain that is one of the settings I changed.

Is there nothing documented about using the m3u file? I have seen statements here saying it can be done but I have not seen anything specific on how to make it work; config.ini entry, etc.
I think using Cabernet in the ffmpeg mode could resolve an issue I have with one provider.

*UPDATE*

This has just become a moot point as MacOSX is now giving me the same error I was getting on the RaspberryPi¶

501 - Server Error

It is not possible that this error is a quirk with my RaspberryPi as I previously thought. The OSX worked for only a couple of days much like the Pi.

As I use NextPVR , NextPVR seems to hold in a Cache the last playlist it was able to download (rather than generate an error if it tries to download it anew), so there is no sign that it has failed.

When I moved Cabernet to OSX a couple of days ago it was working fine with USTVGo and all the USTVGo channels still play on NextPVR despite the fact that the channels.m3u list is now giving it an 501 error. I would suspect that TVHeadend might behave the same way , which might mean that other uses have this issue and do not know it because they have made no change. They would only notice it if trying to access channels.m3u directly, or making a change like adding channels to Cabernet.

In both cases , Raspberry Pi and OSX I am running from native OS command line Python 3.8.0 on Pi running Raspbian Stretch, and Python 3.9.12 on Mac OS 10.15 Catalina

So a question to other users here. ...Can You access your http://IPADDRESS:6077/channels.m3u file from a browser?

It seems, at least I have convinced myself, that I have identified two different bugs here.

I use cabernet with tvheadend. It is set up just like it says at the top of the page. I use it mainly with the tvheadend pvr client on kodi. Everything works fine. For quite some time now. I also got it working with jellyfin and plex by using antennas. It's all running from Ubuntu running on a laptop.

Tekno M it looks like you are using output from NextPVR and using it as an input to cabernet. Not sure why you would do that. The program recording function is part of NextPVR. So seems like you would want to use Cabernet as an input to NextPVR. Good luck. But I suspect you are doing it backwards.

Here is the m3u portion of my config.ini

[m3u]
enabled = True

[m3u_samsumgtv]
label = M3U SamsungTV
enabled = True
channel-m3u_file = https://i.mjh.nz/SamsungTVPlus/us.m3u8
epg-xmltv_file = https://i.mjh.nz/SamsungTVPlus/us.xml
player-enable_pts_resync = True
channel-import_groups = True
epg-min_refresh_rate = 3600

[m3u_plextv]
label = M3U PlexTV
channel-m3u_file = https://i.mjh.nz/Plex/us.m3u8
epg-xmltv_file = https://i.mjh.nz/Plex/us.xml
enabled = True
channel-group_name = plex
player-enable_pts_resync = True
channel-import_groups = True
epg-min_refresh_rate = 3600

[m3u_stirr]
label = M3U Stirr
enabled = True
channel-m3u_file = https://i.mjh.nz/Stirr/all.m3u8
epg-xmltv_file = https://i.mjh.nz/Stirr/all.xml
player-enable_pts_resync = True
channel-import_groups = True
epg-min_refresh_rate = 3600

[m3u_canada]
label = M3U Canada
enabled = True
channel-m3u_file = https://i.mjh.nz/SamsungTVPlus/ca.m3u8
epg-xmltv_file = https://i.mjh.nz/SamsungTVPlus/ca.xml
player-enable_pts_resync = True
channel-import_groups = True
epg-min_refresh_rate = 3600

JohnTocher , Is this directed to me?

If so, all I can say is, good for you!

It does not mean we use it in the same way . In fact the docs at the beginning do not touch on the subject of how to configure m3u files nor USTVGo. Yes they do mention it, but it ends there. Also from my experience it does not "cure" the problems with Pluto TV and Vevo channels no matter where you get them. I did run TVHeadend but found that it would not scan many of my sources and I could not get them working in TVH . All you need to do is check the forums here and you can see this is a long standing BUG with TVH. Yes a BUG because as others have pointed out even the work arounds do not always work. It would be great if TVH did not have this channel scanning issue, but it does and makes it unusable for many sources.

Have you tried to access your channels.m3u file directly from a web browser? Even though I can no longer download it with a browser NextPVR is happy with the last copy it got and the links in that file are fixed so they keep working even if your TVHeadend can not get that file again (silently fail) that is why I asked if you can access your channels.m3u list.

John I guess maybe we posted at the same time. No I do not do what you are saying. Cabernet is an INPUT (meaning Cabernet's output feeds to NextPVR) to NextPVR . I have no idea where you got that idea from but there is no "loop" . Cabernet feeds next PVR . In fact I think in the logs I uploaded there is only one source in Cabernet, and that is USTVGo. I have tried to add an m3u list (that was probably originating from a script on localhost:8183), but it has never worked and probably shows up somewhere on the logs. From what I have read on the NextPVR forums m3u file output has been disabled in Next PVR V5 (which this is) so what you are saying is not even possible.

And thank you for that real world m3u entries. Now when I get through the other bug (continued data corruption even on a completely different OS) I can try that. There is little sense in wasting time to do it all again on OSX when I was already through the same exact data corruption issue on Linux (even with different playlist sources on Cabernet).

Honestly see if you can access your channels.m3u list! Just because TVH is "working" does not mean Cabernet continues producing channels.m3u. TVH will just use the old one in a cache or database somewhere if it can't get it again. THats what Happens on NextPVR . Load up a browser and try http://IPaddress:port/channels.m3u where IP address and port are those of Cabernet. If you or anyone else here sees a "501" error then they have the same problem, even though their set-up is working (for now) .

Tekno M, yes talking to you, afaik 8866 is the NextPVR port, I don't know what you are trying to do. So I am guessing because you did not say.

channel-m3u_file = http://127.0.0.1:8866/playlists/my.m3u
channel-m3u_file_type = m3u
channel-start_ch_num = 50
epg-xmltv_file = http://127.0.0.1:8866/my.xml

why are you using these localhost addresses as inputs to cabernet?

BTW, that "malformed tab" looks exactly the same on my setup. I wouldn't take it as an indication of data corruption.

Documentation is like sex --some is better than none. As far as I can tell G Kazaroth has a life beyond doing tech support for software he was kind enough to share with us.

He has my thanks.

John Toucher

If you take a look at my posts again you will see I have TWO DIFFERENT and UNRELATED issues

1) when trying to add m3u source Malformed tab and no m3u channels appear

2) On two different operating systems now (Raspberry Pi Raspbian Stretch and OSX 10.15) running the script from Python command prompt BOTH systems suffered from IDENTICAL data corruption resulting a "501" error when I access channels.m3u . Deleting all data has sometimes worked to allow me to get an m3u file again but inevitably fails again . In some cases I had to freshly install the code. ..Did you try as I asked to access your channels.m3u?

Port 8866 is the port of next PVR, however you need to understand I have a script generating a file and placing it in a custom folder under the NextPVr web server wwwroot . That particular script "FrndlyTV for channels" has its own web server integrated however it generates data on the fly and comes out very slow, in fact too slow for most things like NextPVR or TVHeadend, so I wget it using a cron every couple of hours. I only use the existing Web server / port of NextPVR to avoid having to run a different web server. I do understand that with cabernet I can use a direct file location but keep in mind I am experimenting with other solutions at the same time and writing it to a web server ensures I can access it if I choose to retry TVHeadend or TVMosaic or one of the others that I have tested over the last few months. In essence, that file, although being served up by the NextPVR web server is generaated by a cron that access the FrndlyTV for Channels script (via its integrate web server) . It does not originate from NextPVR is only "served up" by nextPVRs web server.

I think you should not be so quick to deny my problem, and yes I am trying to let the dev know of a bug. I appreciate his work too and maybe you are mis-reading me again just like you have mis-read my posts. Although I do not consider myself a "developer", I wrote and maintain a couple of PHP scrips that share scanners between computers over the eSCL protocol. I certainly like getting feedback, not that I want it to become a full time job but if someone tells me of what they believe is a bug I take it at face value. I actually appreciate hearing from people, as I think most other do too.

Now I appreciate seeing your M3u entries. do not get me wrong, but I feel like I am p*ssing into the wind here. Have I detailed the two separate issues enough and elaborated enough so that you can get your head around it all now? Did you even bother to do the simple test to see if YOUR install has the same bug (bug #2)? I can not believe that others do not have the issue that I have seem. Cabernet generates and exposes a static playlist so if at some point that playlist is no longer available by download, the old one, in a cache works perfectly well as you PVR hists up Cabernet for the stream then Cabernet proxies or redirects . My point being you could go a long time without knowing about this 501 error.

Do I WANT to delete the data knowing I will probably again only return to the same problem later? Probably not. Can I work with the other problem (#1) while I still have the playlist (#2) problem? Simple answer is no. If I leave things as-is then NextPVR is happy with just the USTVGo channels, although I can not see the frndlyTV channels on NextPVR for now.