Some channels in a mux are not being found - those which are not broadcasting

Martin Underwood

(sorry, title should say "not broadcasting - typo!")

Several years ago I installed an older version (4.2.x, IIRC) of TVHeadend on a Raspberry Pi and I'm updating that Pi to a newer OS (RasPiOS Bookworm) and the lastest TVHeadend 4.3-2375_g653bd0400 by installing to a different SD card system drive (so I can flip between the two while I'm setting everything up). All the DVB-USB tuners and the .fw firmware files are identical.

I've noticed that two channels are missing when I scan mux PSB3 (the high-definition T2 mux) in the UK. These are BBC Three HD and BBC Four HD. All other channels in that mux are found and can be played or recorded from. Is it siginificant that these channels are not currently broadcasting? They share bandwidth with CBBC HD and CBeebies HD and switch over at 19:00. They are definititely present in the list of available channels - if I go to the mux Configuration | DVB Inputs | Muxes and press the > play button for that mux, VLC starts and I can see them in the Playback | Programmes list, though obviously nothing is displayed if I select either of those channels.

Is this new behaviour, because I don't remember having this problem with the older version of the software. Does TVHeadend now only see channels if they are broadcasting at the time of mux scan? Do I need to rescan after 19:00 when those two channels are broadcasting (and CBBC/CBeebies which have been found are no longer broadcasting).

Unfortunately the debug log doesn't describe the precise channels that are found during a mux scan:

2024-11-22 14:53:06.102 mpegts: 530MHz in Terrestrial - tuning on Hauppauge WinTV Dual (1) (DVB-T2) (Silicon Labs Si2168) #0

2024-11-22 14:53:06.102 subscription: 0038: "scan" subscribing to mux "530MHz", weight: 6, adapter: "Hauppauge WinTV Dual (1) (DVB-T2) (Silicon Labs Si2168) #0", network: "Terrestrial", service: "Raw PID Subscription"

2024-11-22 14:53:14.704 mpegts: 530MHz in Terrestrial scan complete

2024-11-22 14:53:14.704 subscription: 0038: "scan" unsubscribing

Martin Underwood

When I say that I don't see that channels, I mean that they are missing from the Configuration | DVB Inputs | Services list, sorted by mux.

Chris G

Try setting "hide" to "None".

Does that make them appear?

Martin Underwood

Chris G

Yes it did. I didn't know about that option. I thought it was odd that BBC Four wasn't listed either for terrestrial or satellite. I'll try to remember that option. I hope it's "sticky" and I don't have to change it every time...

There are a lot of options (eg which columns in the services, channels, programmes to be recorded, programmes that have been recorded) on various screens which ought to be configurable once and for all at the server end, rather than having to be configured (and usually customisations get forgotten after a few days) at the client (browser) end.

Chris G

The option doesn't help you/us… It just lets us see the services exist.
Unfortunately, someone decided that services which don't have an active video or audio track should be disabled.
Even if you do get it mapped, at some point when you tune or try recording one of these services, it will fail. “No adapter available” or similar.

I think this is the 4th or so time I'm saying this on this forum:
If a service is referenced by the EIT tables or whatever (no sure the technical terms), it should exist and be enabled, regardless of what tracks it has or doesn't have.
As soon as a service is no longer referenced by the network, delete it completely. There's no need for a bunch of ghost channels to stick around, even if they are disabled.
Basically, just work like a regular TV.
I think someone was trying to be too clever.

Martin Underwood

Chris G

In answer to my implied question, yes it was necessary to re-scan once BBC Four started transmitting, preumably in order to associate the EIT table entry with the video and audio stream IDs. Had to do that for terrestrial PSB1 (BBC Four SD), PSB3 (BBC Four HD) and the relevant satellite mux (10817, IIRC) that contains BBC Four HD.

Definitely a retrograde step. None of that faffing around was necessary with the older version that I used. I wonder if that older version is still available for download as a .deb package...

TVH is a great program. but it does suffer from developers who aren't responsive to bug reports (or deny that it is a bug). I remember submitting an enhancement request for TVH to verify when you were adding a recording to be scheduled whether it would conflict with any exisitng recordings, bearing in mind any pre-/post-padding that you may have added to recordings (I have a blanket default of 5 before and 10 after) and the number of tuners that you have. The developers' response seemd to be "because this is not technically/logicaly possible in an obscure set of circumstances, we're not even goign to consider doing it for the majority of cases, and you are a blithering idiot for even suggesting it." Dedicated PVRs can do it, and other packages such as NextPVR can do it, so why can't TVH?

Oh, and that "Hide" option isn't sticky: you have to set it to "None" every time you visit the Services page :-(

DeltaMikeCharlie

Martin Underwood I remember submitting an enhancement request for TVH to verify when you were adding a recording to be scheduled whether it would conflict with any exisitng recordings,

This is on my 'to do' list, but it is not as easy as it sounds when you take into account the way that the internal TVH data structure are arranged as well as the variety of priority and other parameters.

https://tvheadend.org/d/8369-my-list-of-feature-ideas

For a simple system with a single tuner, the solution is relatively straight forward. However, when looking at mode complex systems with multiple tuners of multiple types with the potential for a single channel to be obtained from multiple sources, then the solution also becomes more complex.

Chris G

Like I said, at some point, the channels will stop tuning.
I think this happens next time a scan occurs when the services aren't broadcasting.

There are a lot of “Sky Sport” channels on 19.2E.
They only broadcast when they are needed, depending on how many games are on that day, etc.
Because of how TVH works, I can't schedule recordings on them because the channels won't tune when TVH inevitably disables the parent.

There are no devs as such, just people submitting pull requests to fix their own issues.
If you can't implement something yourself, it's very unlikely to happen.

Martin Underwood

Chris G Do you happen to know the release at which the "channels stop tuning because they weren't transmitting" bug/feature was introduced, so I know how far back I need to go to find a version of a .deb that doesn't have the bug? I'd say that it is a major bug that doesn't take into account part-time channels. Maybe the UK is unusual in having those.

Maybe I need to abandon my plans to upgrade my Pi to Debian 12 (Bookworm) from Debian 10, in order to keep my old version of TVH which works perfectly. It all started when I planned move my weather station monitoring software from an older Pi to the TVH Pi, to free up that older Pi, and this needed an upgrade of operating system on the newer Pi4. Then the fun began...

Chris G

I have no idea when it was changed, sorry.
It's not just the UK, though. Sky on 19.2E is German and has the same issues, even more in fact. I'm sure there are loads.
I think someone said the reason behind the change was to hide all the “dummy” channels some providers use.
EG: A “YouTube” service that, when “tuned”, launches the YouTube app on the provider's STB, but isn't actually a channel at all.

Martin Underwood

DeltaMikeCharlie

Yes, I can imagine that it's more difficult with several tuners. I suppose one way is to allocate a tuner to a scheduled recording when the recording is set up (allocating in prioroty order) and then to try to allocate a tuner as each subsequent recording is set up. Obviously at the time of actually recording some allocation occurs, in order that a conflicting recording can be said to have failed because there are no free tuners, so maybe that same code can be used to perform a "what if" test at the time of scheduling - to say "if I wanted to record right now, with this set of recordings scheduled, would I run out of tuners?". OK, I'm sure it's a lot more complicated than that... ;-)

If you are looking for other things to add to your "to do list", my suggestions would be:

Allow the administrator to configure which columns are displayed by default in the list of services, the list of channels, the list of scheduled recordings and the list of programmes that have been recorded. The current architecture displays everything and ends up with a lot of the columns that are too narrow for their contents. Some fields are not relevant - eg ATSC and minor local channel in DVB land, and DVB-related things in ATSC-land, and it would be nice for the admin to turn off the display of them. The mechanism (cookies?) which stores the user's preferences for displayed/not-displayed and column width is unreliable and I find that all too often it resets to display everything. Maybe find a way of automatically sizing the column widths to the longest data in that column and allow the page to scroll sideways.
The automatic deletion (if that really is what happens) of services which are present in the mux tables but which happen not to be transmitting at the time of a scan - that sounds so serious as to be a bug. At the very least, there needs to be a way of turning the behaviour on/off. It means that channels such as BBC Four which only transmit between 1900 and 0800 (and share bandwidth with CBBC 0800-1900) are at risk of being deleted if a scan happens to run at hte wrong time. It didn't happen with 4.2.8 which I have on my old config, so someone has taken the conscious decision to introduce the feature, in the misguided belief that it is actually a good idea rather than a spectacularly bad one.

I had a quick look at NextPVR as an alternative to TVHeadend. Nowhere near as configurable in terms of being able to edit lists of muxes (the config tables supplied with the operating system are bound to be out of date!) and to merge the same channel on terrestrial and satellite (to implement "if BBC One is wanted, try satellite first, if that's busy, try one or other of the terretrial tuners"). Very poor compared with TVH. I hated TVH's lack of a graphical EPG (channels on one axis, time on the other) when I first started using it, but I like it now. It's got a great deal going for it. It would be a shame if this auto-deletion of services that aren't transmitting (if that's what it is) spoiled an otherwise good product.

DeltaMikeCharlie

Martin Underwood The automatic deletion (if that really is what happens) of services which are present in the mux tables but which happen not to be transmitting at the time of a scan - that sounds so serious as to be a bug.

I don't know enough about this 'feature' to comment seriously. Perhaps a flag could be put on the mux or the network to enable/disable automatic deletion. It may already be there, I would have to do a lot more research. I do recall seeing a 'last seen' field somewhere, perhaps that field is related.

Martin Underwood Yes, I can imagine that it's more difficult with several tuners. I suppose one way is to allocate a tuner to a scheduled recording when the recording is set up (allocating in prioroty order) and then to try to allocate a tuner as each subsequent recording is set up.

My Python model built a portfolio, sorted by start date/time, of upcoming recordings by combining the scheduled recordings and then merging/rolling forward the TVH timers.

The model then 'executes' the contents of the portfolio taking into account priorities, etc. Before allocating a tuner, the model first looks for tuners that were already in use for a service on the same mux. Once there were no more free tuners, a collision occurred.

There can also be collisions of different severity. If a collision happens in the pre/post padding period, it may not be as serious as a collision that happens in the middle of the programme being recorded.

Combining this into TVH will be an issue, I was thinking that the PVR module should have a new 'recording conflict' page. Each 'conflict' will then have a drop-down that contains the recordings that will collide and it will be up to the user to resolve the issue.

Another totally manual option would be to display a gantt chart of upcomming recordings. The user could manually detect overlapping recordings and take action.

davep

Martin Underwood The mechanism (cookies?) which stores the user's preferences for displayed/not-displayed and column width is unreliable and I find that all too often it resets to display everything.

Cookies are used to store the display layout, but the expiration period is only 7 days by default. The maximum (set in Configuration -> Base) is 2047 days.

Martin Underwood The automatic deletion (if that really is what happens) of services which are present in the mux tables but which happen not to be transmitting at the time of a scan - that sounds so serious as to be a bug.

I don't believe that a mux scan ever deletes services - that's what Services -> Maintenance -> Remove Unseen Services is for. I've never seen part-time services such as BBC4 disappear.

DeltaMikeCharlie My Python model built a portfolio, sorted by start date/time, of upcoming recordings by combining the scheduled recordings and then merging/rolling forward the TVH timers.

The model then 'executes' the contents of the portfolio taking into account priorities, etc. Before allocating a tuner, the model first looks for tuners that were already in use for a service on the same mux. Once there were no more free tuners, a collision occurred.

TVHadmin does this. The major problem is that Tvheadend is non-deterministic; if two timers are due to start at the same time it is not possible to know which one will fire first.

Chris G

TVH is brilliant.
It's just let down by a couple of things.
Off the top of my head:

Disabling services that don't have a video or audio track.
Disabling rather than deleting unreferenced services.
No support for T2-MI streams (I believe this is DVB-T2 over DVB-S).
Scanning an IPTV playlist shouldn't scan each channel inside it, just refetch/reload the playlist.

Chris G

davep

I don't think BBC Four is a good example because I think most of the time, there is something on the channel?
I for sure have had it not tune in the past because the parent had been disabled, but it's tuning and broadcasting right now.

If you can receive 19.2E, Sky DE is a very good package to observe the unwanted behaviour discussed in this topic.
There are a load of Sky Sport channels that sit there not broadcasting unless there are enough games on to warrant firing them up.

I have to hide None to see them all:

See how a lot of them have no CAID?
That's because the last time TVH scanned them there was nothing broadcasting, so disabled them.

You might think, so what?
But if I schedule a recording or try to tune a channel whose parent has been disabled, it will fail to tune, even if the provider has fired up the service since last time TVH checked.

What we want is for TVH to not disable services.
“Disabling” is something that only the user should be able to do.
Furthermore, I suggest that as soon as a mux stops referencing a service, TVH should delete that service completely.

davep

Chris G There is another column in the Services page "Automatic Checking" which by default is set to "Auto check enabled". The documentation isn't clear on how this works but it is possible to edit the field (on a per-channel basis unfortunately) to "Auto check disabled". Does changing this field fix your issue?

Are you using Bouquets? Another possibility is that the broadcaster is removing the services from the bouquet when they are not being used.

Chris G

davep Are you using Bouquets? Another possibility is that the broadcaster is removing the services from the bouquet when they are not being used.

I am using bouquets, a mix of network bouquets, OTA bouquets and E2 bouquets from the web.
I think the reason the parent disabled services are getting mapped at all is because the bouquets ignore the fact that the parents are disabled and map them anyway.
100% the bouquets are not altered when the channels aren't airing.
The particular one I use for Sky DE is hosted on GitHub, so I can see the history.
You can see here that parent disabled services are being mapped by the bouquet (I do no manual mapping):

Most of these services on the right are parent disabled by TVH, but still successfully map to channels.
These channels appear in Kodi with OTA EPG data and I can schedule recordings, however the channels will not tune and recordings will fail too.

davep There is another column in the Services page "Automatic Checking" which by default is set to "Auto check enabled". The documentation isn't clear on how this works but it is possible to edit the field (on a per-channel basis unfortunately) to "Auto check disabled". Does changing this field fix your issue?

I just tried setting that option on a service who's “parent” is disabled but is actually broadcasting, and it had no effect.
Kodi still responds, “No free adapter available”.
In the TVH log, I see “restarting channel Sky Sport Bundesliga 1” over and over.
If I try playing the service, rather than the channel, I get “restarting service 19.2E/11914H/Sky Sport Bundesliga 1” over and over.

After rescanning the mux (setting it to “Active” and waiting a few seconds for it to complete), “Sky Sport Bundesliga 1” is no longer parent disabled (because it's broadcasting right now) and plays without issue, both the service and the channel.
This service will stay parent enabled now until the next time TVH sees it not broadcasting (which is shortly).

You can see how this is a huge problem for a lot of the community.
I happen to know that most people (particularly German users) still use 4.2 for this reason.
The Sky DE iCAM patch for TVH was made for 4.2 before 4.3, because that's what the community is using.
To be clear, this isn't a German only issue, it's any provider who turns channels off when they are not needed.
Germany just has a lot of them, and they happen to be popular channels that people actually watch (sports).
And because of issues like this, 4.2 is the only “supported” version of TVH by various Kodi/distro devs (OSMC etc).

Moving forward, I want to see if I can (hopefully with some help) make a pull request to add CAID filtering to bouquet mapping.
I understand Python and looking at the bouquet code, I could kinda see what was happening, even though I don't know C at all.
But this issue seems much deeper and way above my skill level.

davep

Chris G After rescanning the mux (setting it to “Active” and waiting a few seconds for it to complete), “Sky Sport Bundesliga 1” is no longer parent disabled (because it's broadcasting right now) and plays without issue, both the service and the channel.
This service will stay parent enabled now until the next time TVH sees it not broadcasting (which is shortly).

If you set the service "Automatic Checking" field to "disabled" while it is on air, does that prevent the service from being disabled?

Also what is the value of "service type" in the Services display for those services?

Chris G

davep If you set the service "Automatic Checking" field to "disabled" while it is on air, does that prevent the service from being disabled?

No idea… I'd have to wait for the channel to broadcast again…. Could be days or more.
I appreciate the suggestion, but this isn't a solution anyone wants, even if it works.

davep Also what is the value of "service type" in the Services display for those services?

All service types appear to be correct, whether the parent is disabled or not.
25 is HDTV, right?

davep

Chris G No idea… I'd have to wait for the channel to broadcast again…. Could be days or more.
I appreciate the suggestion, but this isn't a solution anyone wants, even if it works.

Maybe not, but it helps to narrow down the reason for the behaviour.

Chris G All service types appear to be correct, whether the parent is disabled or not.
25 is HDTV, right?

OK that's another theory disproved...

Do you have anything in the Tvheadend logs? Maybe entries like "disabling service ... (missing in PAT/SDT)"

Do you know if there are any services on 23.2E which auto-disable in this way?

Chris G

davep Maybe not, but it helps to narrow down the reason for the behaviour.

I appreciate that.
Now, if it was on a network level, I think that would be perfectly acceptable.
This is all assuming it actually fixes the issue.

davep Do you have anything in the Tvheadend logs? Maybe entries like "disabling service ... (missing in PAT/SDT)"

I have seen these messages in the past when doing network scans

davep Do you know if there are any services on 23.2E which auto-disable in this way?

Sorry, I don't know of any.
I mainly use 19.2E for my sports, then 23.5E