Download and restart lead to missing file - Bug?

Digard

I'm new to tvheadend, so I'm not sure if this is a bug? Though I can reproduce this behaviour:
Files in Finished Recordings, once Downloaded, show up as Removed Recordings without me having removed those; with "file missing". The files are still there, however, until I actually Delete the record.

It doesn't bug me too much, since a 'downloaded' file may be removed by a software too clever by heart; but the statement of "file missing" is definitely wrong.

davep

Some more information would be useful; what is your setup and what do you mean by "downloaded"?

Digard

davep With pleasure.
What do you mean with 'Setup'?
"Downloaded" was with capitals, meaning in the web interface. I usually select from "Electronic Program Guide", "Record", and then from "Finished Recordings", "Download".
After a restart of the box, the result is what I wrote; so far 100 percent reproduceable. The record is gone under "Finished Recordings" and pops up under "Removed Recordings", with "file missing".

davep

OK thanks.

Do you have anything set under Configuration -> Recording -> DVR Profiles -> (your profile) -> Automatically Delete Played Recordings?

Digard

davep
No. I have set this to keep forever.

davep

I tried to reproduce this but failed.

I recorded a short section of programme (about 5 minutes so 50Mb). I then used the "Download" button to copy the file to another computer, then restarted tvheadend. The recording was still in the "Finished Recordings" tab as expected.

Digard

davep
I can reproduce this here, restarting the PC (reboot). I agree, I can't imagine a difference between restarting the service and restarting the box from the point of tvheadend, though here there is one. Maybe timing of mounting vs starting tvheadend at boot?
I'll keep watching it more closely and report back, thanks so far!

Digard

Digard
Okay, I had some time, and can confirm 100 % that this happens here. How can I continue debugging this?
For the time being - since restart tvheadend alone doesn't produce this bug - I assume it could be the mount delay. Could it be that the system service tvheadend starts before the external drive becomes available?:

$ df -h
Filesystem      Size  Used Avail Use% Mounted on
...
/dev/sdc2       6.8G  4.6G  1.8G  72% /
...
...
/dev/sdc1       537M  6.1M  531M   2% /boot/efi
/dev/sdb2       4.6T  2.8T  1.9T  61% /media/me/...
/dev/sda1       224G   67G  158G  30% /var/lib/tvheadend/recordings
$ ...

The files noted as "missing" are available, untouched, and the recordings pop up under "Removed Recordings". It would at least be logical, though wrong, if tvheadend didn't find the recordings on the not yet mounted drive /dev/sda1, to consider the recordings as willfully removed.

davep

Digard Could it be that the system service tvheadend starts before the external drive becomes available?:

That seems very possible. The fact that you have shortened the output of "df -h" above suggests you have a lot of drives. However tvheadend is supposed to be started after locally-attached drives have been mounted and networking is available.

What version of tvheadend are you using and where did you get it? Also what operating system are you using and on what hardware?

Digard

davep
Nope. Not more. Just removed the 'artificial' drives, Ubuntu 22.04 server. Ex thin client on SSD, running gerbera and tvheadend as media center.
Tvheadend 4.3.0~pre+202406250002-0~built202406250547~git128d6861f~ubuntu22.04.1, mamarley

DeltaMikeCharlie

I can recreate this at will simply by renaming a recording file.

If the recording file has the name that matched the property in the object, then it shows as 'finished'. If the recording file has another name, then it shows as 'removed'. TVH needs to be restarted to show the changed state.

@Digard - Try adding a delay before the TVH service starts to allow time for the external drive to moount.

This thread discusses delaying startup so that a tuner card can become ready.
https://tvheadend.org/d/8672-dvb-s-device-not-showing-up
It is the same concept.

Digard

DeltaMikeCharlie
Thanks, though this is why I'm here. I want to learn, and try to improve FOSS, that I have used since last century. Of course, I had thought of adding a delay myself. But I think that this behaviour is wrong. The database must not 'auto update' just because some files exist or don't.
How would a service know what else does come up? If the Finished Recordings show a "missing file" temporarily after reboot, that'd be okay, AFAIAC. We don't expect miracles. But then, some seconds later, the display should be updated.

Digard

DeltaMikeCharlie

DeltaMikeCharlie @Digard - Try adding a delay before the TVH service starts to allow time for the external drive to moount.

This thread discusses delaying startup so that a tuner card can become ready.
https://tvheadend.org/d/8672-dvb-s-device-not-showing-up
It is the same concept.

Yes, thanks. By now I have added ExecStartPre=/bin/sleep 30 to tvheadend.service, and that works as expected.
The underlying logic is somewhat astonishing, though. It looks like tvheadend at startup tries to list the physical files, and compare the result with the database content. Differences are noted as "Failed Recordings", which is not necessarily a logical conclusion.
(What would a "failed recording" be after all? So far this list has always been empty here. I'd rather expect/suggest to list a recording that could not be taken (eventually not completely) because of lack of tuners or disk space or whatnot. It doesn't)

DaveH

Digard Of course, I had thought of adding a delay myself. But I think that this behaviour is wrong. The database must not 'auto update' just because some files exist or don't.
How would a service know what else does come up? If the Finished Recordings show a "missing file" temporarily after reboot, that'd be okay, AFAIAC. We don't expect miracles. But then, some seconds later, the display should be updated.

You're proposing adding considerable complexity (event-driven code is not easy) to cover a case that won't affect the vast majority of users. I'm sure patches are welcome, but do consider how to test the facility comprehensively. 🙂

davep

Tvheadend scans the recordings directory at startup, and flags any recordings which are not present as "removed" in the UI. However there is no change made to tvheadend's internal data structures, so if TVH is restarted after making the missing files available again they will reappear in the "finished" recordings table.

I don't understand why in your case tvheadend seems to be starting before all the drives are available; there was a patch applied in 2019 to address this issue and your install package has this patch. Adding a manual delay may be your best option.

Digard

davep
Yes, agree. Except with the "Removed recordings". Actually, though that's correct, one (only me?) would rather expect to see either "Finished Recordings" where the file is (temporarily) missing, or a "Failed Recordings", because the file is missing. Preferably the former. I wouldn't expect to see a physical ssh-su-rm as "Removed Recordings". Especially since the file is still there, and takes up space.

Event-driven (kernel flag, etc.) should still be on the RFE list, since (unintentionally) unplugging a USB should preferably be reflected in the display.

Jonas Lang

I really do suggest that anyone who uses TVH regularly and wants to learn a little bit more about its inner workings builds/compiles it from source themselves.

I don’t see any of these errors contained in my version of TVH installed on Ubuntu 22.04 lts using the current master. In fact I’d go as far as saying it’s working flawlessly these days. That and plus the fact you get to exploit the full GPU capabilities of your processor.

Friedhofs Blond

HTS Tvheadend 4.3-2370~g28de5c092 auf Debian 12.

Guten Morgen,
ich habe TVH auf einem kleinen Dell-Wyse via deb installiert.
Die TV-Records liegen auf einem NAS (DS920) was nur 20cm neben dem Wyse steht.
Nach einem Power On oder Reboot habe ich genau den gleichen Fehler.
Wenn ich den TVH-Dienst neu starte, sind alle Aufnahmen wieder da.
Ich versuche jetzt die Lösung den TVH-Dienst verzögernd zu starten.
Grüße

Good morning,
I installed TVH on a small Dell Wyse via deb.
The TV recordings are on a NAS (DS920) which is only 20cm from the Wyse.
After a power on or reboot I have exactly the same error.
If I restart the TVH service, all the recordings are back.
I am now trying the solution of delaying the start of the TVH service.
Greetings

Friedhofs Blond

Moin,
service 10sec verzögern - dann klappts:

DeltaMikeCharlie

Friedhofs Blond service 10sec verzögern - dann klappts:

Good to hear that a 10 second startup delay solved your problem.

Thanks for sharing your solution.

DaveH

Most people here speak English.