Detecting Scheduled Recording Conflicts

DeltaMikeCharlie

Currently, TVH will allow unlimited scheduling of concurrent recordings. If there are insufficient tuners to service all scheduled recordings, some recordings will be missed resulting in (what I am referring to as) a 'Scheduling Conflict'.

There is an external tool that can help identify these conflicts, however, I would like to explore the possibility of incorporating this functionality directly into TVH and alerting users immediately.

dave-p/TVHadmin-JS

Even though a recording is set for a specified 'Channel', TVH navigates links from Channels -> Services -> Muxes -> Networks -> Adapters (tuners) before the recording can actually take place.

https://docs.tvheadend.org/documentation/setup/concepts#applicationtvheadend-fundamentals

Because a 'Mux' can support multiple concurrent recordings, detecting future scheduling conflicts will involve not only examining overlapping recordings, but also which muxes are involved and which tuners can service them.

Recordings for IPTV channels can also complicate matters because they do not have 'adapters' but can be restricted by the number of concurrent streams at the 'Network' level.

'Timers' can also complicate matters because only the next resulting recording is scheduled at any time. Consequently, a Scheduling Conflict involving a timer may not arise until the day that the recording is due. In order to include timers in scheduling conflict detection, every future instance of a timer will need to be calculated and taken into account.

The least predictable recording feature is the 'AutoRec' because these respond to external changes in the the form of EPG updates potentially resulting in a recording being scheduled at very short notice.

TVH already has a priority assignment system for recordings, services and adapters, however, should multiple recordings have the same priority settings, the recording that is eventually missed can not be precisely determined in advance.

Because it is not possible to determine exactly which scheduled recording will fail in advance, my proposal is a solution involving the creation of 'Conflict Groups'. Every overlapping scheduled recording that is involved in a conflict will be tagged and grouped together. The user will then be required to resolve the conflict manually by editing one of the members of the group. In future, it may also be possible to use the EPG CRID to look ahead and find alternate occurrences of a conflicting recording. If this is the case, then the conflict could be resolved automatically.

Thoughts and comments are welcome.

Cutter

DeltaMikeCharlie should multiple recordings have the same priority settings, the recording that is eventually missed can not be precisely determined in advance.

Isn't the failing recording the last to start ?

DeltaMikeCharlie The user will then be required to resolve the conflict manually

I agree it would be very useful to be able to detect conflicts in advance. But IMHO the manual conflict resolving process should be optional for the sake of simplicity. What I mean is that tvheadend should warn about conflicts and offer the option to resolve them, but always take a default action if the user does not take action. It should not complicate the process of scheduling recordings, and should remain compatible with existing software that did not yet implement the new conflict resolving system.

davep

(I'm the author of TVHadmin.)

At present, if two recordings are scheduled to start at the same time with the same service preferences, it is indeterminate which recording will grab which input device. This could be avoided, when creating a recording, by checking if any existing recording has the same start time; if so then the new recording should start one second later.

DeltaMikeCharlie

davep it is indeterminate which recording will grab which input device.

Agreed. That's why I'm proposing producing a conflict group and then allowing the user to resolve the situation manually.

DeltaMikeCharlie

@Cutter - Thanks for your feedback.

Cutter Isn't the failing recording the last to start ?

If they both have EXACTLY the same start time, which one is the last to start and how can that be predicted?

Cutter but always take a default action if the user does not take action

What should this 'default action' be?

If another broadcast of a conflicting programme can be found using the CRID, that would be easy. However, if there are no other suitable broadcasts, the user will have to choose which recording is more important to them.

Cutter

DeltaMikeCharlie What should this 'default action' be?

I don't know but the rule should be clear and predictable.
For example, existing recording schedules could have priority over newly created ones to prevent newly scheduled recordings to "break" existing ones.
It would be more complicated for Autorecordings because as you said, the EPG is not entirely predictable. But if a conflict appears in the EPG for existing autorecs, we can't always expect the use to notice or take action in time, and resolving the conflict somewhat arbitrarily is better than recording nothing.

Chris G

I always imagined just having a "!" in the details column here:

DeltaMikeCharlie

TVH already has a Unicode 'Warning' ⚠ emoji somewhere. I was going to reuse that.

DeltaMikeCharlie

Cutter I don't know but the rule should be clear and predictable.

I agree.

Cutter For example, existing recording schedules could have priority over newly created ones to prevent newly scheduled recordings to "break" existing ones.

There is no way to determine, apart from user-assigned priorities, that a new recording is more or less important than any existing recording. If scheduled recording age were to be considered to be a resolution mechanism, there should be a parameter to indicate to favour oldest or favour newest.

Cutter

DeltaMikeCharlie that a new recording is more or less important than any existing recording.

I mean by default. Of the course the user could resolve the conflict differently. But while no conscious action has been taken by the user, tvheadend should prioritize the existing recording schedules over newer ones, because what has been scheduled should be set in stone and not subject to conflicts with yet to be scheduled recordings (again, by default).

DeltaMikeCharlie

It occurred to me that scheduling conflicts can vary in severity. Consider the following 'Typical' recording profile and some conflict scenarios as follows:

|--WarmUp--|----Padding----|------Programme------|----Padding----|
A    |-------------------------Recording-------------------------|
B                        |---------Recording------------|
C                            |---Recording---|

I propose the following scale of severity:

0 = No conflict.
1 = Part of the warm-up period will be missed.
2 = 33% or less of either padding will be missed.
3 = 66% or less of either padding will be missed.
4 = More than 66% of either padding will be missed.
5 = 25% or less of the target programme will be missed.
6 = 50% or less of the target programme will be missed.
7 = 75% or less of the target programme will be missed.
8 = More than 75% of the target programme will be missed.
9 = Recording interrupted but restarted. (Started OK but stopped by a higher priority recording, restarted at a later time.)
10 = Recording never started.

Using the examples above, 'A' would be severity 1, 'B' would be severity 4 and 'C' would be severity 5.

Each recording in a conflict group will have its own severity value and the group will be assigned the highest severity value of any recording within that group.

Perhaps a 0-10 scale is overly complicated. Perhaps something simpler would suffice:

Conflict 'A' could be considered 'Trivial' because only the warm-up period is impinged upon and the padding and programme are fully covered.

Conflict 'B' could be considered 'Minor' because although the padding is impinged upon, the whole programme is covered.

Conflict 'C' could be considered to be 'Major' because part of the target programme has been missed.

Regardless of the scale used, the background colour, or other design feature, of any warning icon could be used to indicate the severity of the conflict.

Cutter

I like the idea of different levels of severities depending on what phase (warmup / padding / programme) of the recording is missed.

Just a thought: measuring severity by the proportion of a recording that is missed would assign higher severity to short recordings.

DeltaMikeCharlie

I've been doing some work on this new feature and I thought that it was time to present my progress and seek some feedback.

Definition

A 'Scheduling Conflict' occurs when there are more scheduled recordings than there are resources to service them. Currently, TVH will automatically resolve this situation by not recording one or more items.

Method

All scheduled DVR Entries plus forward projections for enabled TimerRec entries are merged into a single test portfolio.
The resources (adaptors) required to service the portfolio are assessed, including pseudo-adaptors for IPTV recordings.
The portfolio is processed chronologically to identify instances where more recordings exist than available adaptors, or adaptors capable of reuse (same mux) have exhausted their PID capacity.
All of the current recordings-in-progress, plus the unsuccessful recordings at that time are assembled into a 'Conflict 'Group'.
Where a higher priority recording interrupts a lower priority recording, a scheduling conflict is also registered.

Why Conflict Groups?

Once recording priorities have been taken into account, because of the way that TVH manages its recordings, it is not always possible to predict with 100% accuracy which of a cohort of recordings starting at the same time will be unsuccessful. When an unsuccessful recording is detected, all overlapping recordings are added to a 'Conflict Group'.

1 [   AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA    ] 2
1 [BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB] 2
1 [      CCCCCCC
                               DDDDDDD  ] 2

Assuming that the system has 2 adaptors and each recording is on a different mux, the example above shows 2 conflict groups. A recording can also belong to multiple conflicts groups.

Activation

There is a global setting that determine if TVH looks for scheduling conflicts. This setting is off by default.

Q1 Should this feature be OFF by default (user opts-in) or ON by default (user opts-out)?

Under normal circumstances, TVH will check for scheduling conflicts when one of the following object types is created/edited/deleted: DVR Entry, TimeRec, Channel, Service, Mux, Network, Adaptor.

If a recording is currently underway or there is a streaming subscription in progress, only DVR Entry and TimeRec objects will trigger checking scheduling conflicts. This may seem strange, but during recording/streaming, many of the above object type are constantly being updated with usage statistics, etc. This leads to almost continuous scheduling conflict checking.

Notification - WebUI

If a DVR Entry is part of a scheduling conflict group:

The status icon will be changed;
The conflict groups will be shown in their own column; and
A message will be displayed in the details window.

If a DVR Entry is also linked to an EPG event:

The status icon will be changed; and
A message will be displayed in the details window.

If a TimeRec is part of a conflict group:

The conflict groups will be shown in their own column; and
A message will be displayed in the details window.

Notification – JSON API

The existing DVR Entry and TimeRec JSON API calls have been updated to include 3 new conflict fields:

Conflict flag;
List of conflict groups; and
An audit trail of the projected events.

New JSON API calls have been added to:

Manually initiate a scheduling conflict check; and
Obtain a list of conflict groups, their members and audit trails.

After a check for scheduling conflicts, any DVR Entry or TimeRec that is a member of a conflict group, or was until recently a member of a conflict group, will have an update message broadcast.

EPG events linked to any of the above entries will also have an update message broadcast.

Notification – HTSP API

A conflict flag has been added to DVR and TimeRec objects; and
Notifications described in the JSON API section will also be sent.

In addition to these, it is possible for the user to provide a custom prefix and suffix to add to the recording and EPG title fields to indicate the presence of a scheduling conflict. Depending on the client system, this feature could be used temporarily until support for scheduling conflict indication is added or permanently if necessary.

Here is a sample of what that could look like in Kodi.
EPG Grid

'Timers' List

Future Work (Potentially Phase II)

Automatic Conflict Resolution: Given sufficiently accurate EPG data, TVH should be able to identify alternate screenings and select those in order to avoid a conflict. TVH already has the ability to do this for missed recordings, so hopefully utilising that mechanism should be possible.
WebUI 'Scheduling Conflicts' Tab: A single screen on the WebUI for managing scheduling conflicts is required.
Severity Levels: Some of the base work on this is complete but not yet provided to the user.

Conclusion

I'm not sure how much longer it will take me to release this feature, however, any feedback or questions on my current development would be appreciated. I should be able to implement simple stuff immediately, more complex suggestions may take some time or be deferred to the dreaded 'Phase II'.

DeltaMikeCharlie

I have been working on providing a consolidated view of the current scheduling conflicts in the WebUI. To this end, I have created a new 'Conflicts' sub-tab under the main 'DVR' tab.

This new tab provides a tree view of the scheduling conflicts to assist the user in assessing the cause of the conflict and determining the appropriate action to take. I also plan to have some help text available here.

Each conflict group can be viewed in 2 ways, 'By Entry / Time' and 'By Time'.

'By Entry / Time' will show each member of the collision group (either a DVR entry or a timerec) with an expandable list of chronological events for each entry.

'By Time' will expand to show a chronological list of all events for all entries involved in the conflict.

Eventually, I hope to be able to click on one of the chronological entries and take the user directly to editing that entry.

DeltaMikeCharlie

I am seeking opinions.

Scheduling conflict 'Groups' can overlap forming a 'Cluster'. In the example below, there are 2 groups, both groups contain 'Long Low' and 'Long Normal'. However, 'Short start high' and 'Short end high' cause independent conflicts and therefore form 2 separate groups because they are never themselves active concurrently, only with other entries.

Does it make more sense to?:

1) Show the Conflicts Groups plus the resulting Cluster.
2) Clusters only where applicable. (Hide the composite groups)

DaveH

DeltaMikeCharlie I'd think just show the clusters only where applicable.
It simplifies the display, and the solution is the same to both groups - delay Long Low to some repeated broadcast. Or maybe I'm not understanding the issue?

DaveH

DaveH edit to add - Maybe it would make the display clearer if you used a different colour to indicate the periods of conflict (e.g. a red background or foreground)?

DeltaMikeCharlie

DaveH I'd think just show the clusters only where applicable.

Thanks, that's the way I was thinking too.

DaveH if you used a different colour to indicate the periods of conflict

Do you mean where the bars already overlap? Something like this, but with better colours?

The idea of using a Gantt chart layout was so that the overlap would be clearly visible, as opposed chronological list of times where the user would have to wade through a table of text. The darker coloured bars indicate the main schedule whereas the light colours on the sides indicate the padding. The dashed lines put each entry into context for the entire conflict group.

There can also be situations where a conflict occurs where one recording is scheduled for the middle of another recording.

I will need to think about this some more, however for now, I am not convinced. Maybe a more subtle colour?

Or maybe a less prominent conflict bar?

DaveH

DeltaMikeCharlie Do you mean where the bars already overlap? Something like this, but with better colours?

Yes, exactly. I don't particularly mind what colours; just something to highlight the times. I think the Gantt chart is a good idea. The conflict bar is probably strong enough as well. It's just I didn't think of it 🙂
I have a DVR (Manhattan) that automatically suggests rescheduling conflicts where there's a +1 channel or a repeated broadcast at some other time. I've no idea how difficult it is to detect such situations.

DeltaMikeCharlie

DaveH I have a DVR (Manhattan) that automatically suggests rescheduling conflicts

This is on my 'To Do' list and will probably end up in Phase II.

Most EPG items have a CRID (content identifier) that uniquely identifies a movie or an episode within a series. It should be possible to search for matching CRIDs at other times and schedule those events instead.

This could turn into an endless loop because the new recording could cause a new conflict whilst resolving the first conflict. In trying to automatically fix the second conflict, the first conflict returns, and so on.

Also, selecting a channel from a service on another mux could also resolve a conflict in some cases. A lower priority IPTV service, for example, could be used to resolve a conflict between DVB-T recordings. One channel can be sourced from multiple services. The current system only evaluates the highest priority service and does not consider falling back to a lower priority service.

It may be safer to indicate to the user that these alternatives exist and let them select a manual resolution.

This needs some more thought.

DaveH

DeltaMikeCharlie It may be safer to indicate to the user that these alternatives exist and let them select a manual resolution.

Yes, that's what the DVR does. It presents it as an option, alongside other choices such as cancelling one of the recordings.