Create .nfo files when creating recordings.

A L

Is there any way that TVHeadend can save the EPG, associated with a recording, to a separate .nfo that can be saved in the same location as the .ts file?

I assume that this must be possible with the post processing part of TVH, but have no idea how to go about this.

The reason I ask is that I am trying to get way from watching TV Shows via the TVHeadend Recording folder. I would prefer to watch everything via Kodi's TV viewer instead - much less messy that way.

I am finding that, for some reason, lots of my recordings are not getting up by the TVDB scraper and wondered whether I could just do away with it, instead using the EPG data from Schedules Direct, which is really detailed and accurate.

Em Smith

Yes, I did it for a while but it becomes tedious to do it perfectly. Unfortunately I can't easily share the scripts since they have a lot of hard-coded scraper credentials and are a bit ugly as I found more and more annoyances.

I found the problem with the scraper is that they often have completely made-up episode numbers that don't correspond to the actual episodes, especially for specials. So, my OTA EPG for the channel that made the programme tells me it is S6E2, but the first person to insert details in the scraper insists it is S0E14, and then locks it to prevent any discussion/updates since they've already named their file S0E14.

Similarly, movies can be a year-or-two out since they used to be released at different times in different countries until, was it Hulk that got the really bad reviews and made studios decide to change so they release at roughly the same time everywhere so people in other countries don't bypass bad movies?

With the %U (uuid) option in commands in tvh, it should be easier to write nowadays. For basic nfo, you could basically get the record from tvh based on uuid (similar to python fanart code in tvh), extract title, description, etc. (perhaps using jq or xslt), and then write to the nfo file, with exact format depending on whether it was EP or MV. The other tedious bit is that episodes also need a tvshow.nfo, but I think that can be minimal.

A L

Thanks Em - sounds rather complicated (I really am still not very good with Linux).

I wonder if I should change tact and try something else.

I totally agree about TVDB - is a wonderful idea if you are cataloguing all your favourite shows and films and have the time to go over everything.

However, as you say, random stuff that you record just doesn't seem to nicely fit in with the scraper. I have been using the Missing Movie Scanner add-on and it has found masses of recordings that have never been added at all, for one reason or another.

Furthermore I have increasingly found that lots of new series that I record have to have the year in brackets in the title. I guess that this is inevitable - as more and more programs get made, the likelihood of a duplication increases.

This has meant that lots of new series get scraped as older, completely unrelated, series.

To tell you the truth, at the moment, I am watching everything via file mode in Kodi (i.e. not library mode and not via the recording folder in the TVHeadend add-on).

My biggest bugbear with this is the watched status tick on the folders - folders seemed to get ticked, when none of the videos inside have been ticked and then, conversely, when all the files inside are ticked, the folder itself sometimes does not produce a tick.

Not the worst issue in the world but still very irritating - I either think that there is a new episode of a series (and get disappointed to find that there isn't) or I don't watch new episodes because there is no way of me knowing if from the folder.

May I ask what strategy you are using?

Em Smith

Have you tried adding a Kodi file source of "pvr://" (one of the options in advanced IIRC). That then gives you your file hierarchy but is using htsp (TVHeadend protocol) to actually play the files and you get proper watched status since it is from tvh. You can then long-click (or press C for custom menu) to change the scraper for "good" directories and they'll get scraped in to your library; others you can just browse.

I've tried a number of strategies but now just use LiveTV.

There was a time last year when htsp would constantly crash for me, so I added code to tvh to generate "scraper friendly names" so I could browse the file hierarchy directly and later started scraping to the library. But I then realized that scraping often doesn't work.

So I wrote a bunch of nfo generators, and spent quite a while improving them with extra SD information and tvdb/tmdb lookups to get full information since I could do better fuzzy matching (this movie says 1999 but tmdb says 1998, this episode says S5E4 but the title matches S5E3). But, there were still some annoyances in how Kodi worked for me, especially with deleting watching programmes.

So then I added code for artwork/fanart to my SD grabber, tvh and Kodi plugin. And so now I'm back full circle at using the standard "LiveTV" in Kodi, but at least having decent artwork, apart from the fact some skins decide to ignore my artwork and scrape their own incorrect one (even in LiveTV) which is very annoying.

One reason I prefer the LiveTV to scraping is because my grabber adds things like age ratings of "U" or "Adult language", and that is useful to me when looking for films to watch when there is a mixed age group in the room since 99% of my programmes are recorded via generic rules ("movies after 1990 rated 4* or higher", "films starring Tom Cruise"), hence I have no idea what most programmes are until I read the description! Most skins I've tried don't show age ratings, but do tell me the show is stereo.

I had added to tvh a "sub-directory by age rating" but I had things like some episodes of old Top Gear rated 18 with others as PG, or some episodes of a cookery show as 15! So, it proved too annoying.

A L

Thanks for that.

I added "pvr://" as a source and was pretty chuffed when three folders showed up - one for channels, one for frequencies and one for recordings.

I was hoping that every file in my recordings folder would be in there (even those that were old and not logged by TVHeadend), but clicking on the recording folder did nothing - i.e. it wouldn't open. Why do you think that is?

I then noticed that recordings was an option in my sources (recordings/tv/active) - I tried adding this as a source but this just gave me the same files as the recording folder.

Unless I am doing something wrong, I am not sure what more I can do.

I can try importing every single recording back into TVHeadend, which I am told is very lengthy or perhaps find a way of hacking Kodi itself, to try and get the folder tick back in file mode (as mentioned above). This used to happen in previous versions of Kodi (I am sure it was there in Helix), so there must be a way of getting it back.

Em Smith

That I don't know. I've used pvr on a number of different machines and versions and it works, but I don't have the very latest versions, mine are a few months old.

I go "Videos->Files->Add. Browse button. Recordings." This then shows "pvr://recordings/tv/active", which sounds like what you have. Inside that I see "tvmovies" and "tvshows", but I click "ok" to get the top level directory added as a source. You can then press "c" (or long press if your remote supports it) to change content type on the sub-directories to allow them to be scanned. It seems a bit temperamental sometimes and so sometimes setting shows removes movies, but eventually it will work.

The only thing I can think of is maybe it is a tvh permission problem? But, I don't know why your LiveTV would work if such a problem existed.

The main reason to do it this way (rather than directly browse the filesystem) is because the file tick should be synched back with tvheadend (since it is using htsp instead of direct file reads), so it may workaround your problems with ticks.

You mention you have lots of files not imported in tvh?

A L

Sorry - yes,I record to single hard drive that has only two folders, "TV" and "Movie".

I have two presets set up in TVHeadend, so that the default is TV but if I know that a recording is a Movie, I choose the other one.

I have to reinstall TVH a number of times over the last few years and each time I do, I have to start from scratch - i.e. TVH only logs the recordings made since that install, so there are masses of recordings in the recording folder that TVH is completely unaware of.

Em Smith

Since you're using Schedules Direct, if you also use tvh dev branch (4.3) then you can use the "$q" format specifier to automatically split tvshows and tvmovies, so no need to do it manually. The autorec rules can also "match if unique" which will use the Schedules Direct unique identifier for duplicates.

(If you use my unsupported version of the grabber then you also get artwork and it will prefer tvdb season/episode numbering if SD has specific knowledge of a difference; but take a backup of your .xmltv directory first since mine modifies the database which means you won't be able to re-run the official version without restoring the db).

Unfortunately it's a bit difficult to import the old files if you have lost the original dvr/log entries. I think the filenames have some specific format so you can't just generate a new unique id filename and say 'filename: "abc"' in it.

A L

Right - I see.

I was hoping that all files that happened to be in the recording path (regardless of whether they were in the TVH database or not) would appear, but sadly not - as you say, trying to add these files back into TVH is just not going to be worth it.

Maybe I should just accept that I have to have two sets of file, those in my TVH Recording folder and those in my Library.

Out of interest - how stable is 4.3?

What you have mentioned above sounds amazing, but I am slightly worried about upgrading from 4.2 if it is too buggy.

Em Smith

I've not upgraded my 4.3 for a while due to lack of time, but it's usually stable, but sometimes might have a week of less reliability where some fixes might aggravate something else that was broken but had always seemed to work.

Once you get a version that works then stick with it! Don't go chasing the latest and greatest version unless you see some feature you want, and always take a quick peek at the bug reports before upgrading just in case.

Features which might be of use to you are the "$q"
https://github.com/tvheadend/tvheadend/blob/master/docs/property/pathname.md
; "record if unique" option in autorec (so just works regardless of if it is a movie or episode); "mark as previously recorded" (so if your autorec matches a movie you've seen then you can flag it so it isn't recorded again); save epg after xmltv import; and artwork poster/fanart passed to Kodi (v18+/Leia+). But, there are thousands of behind the scenes improvements.

But, ensure you backup your dvr/log before trying to run it since the format of entries has changed so recordings won't be backward compatible (can't be imported if you later downgrade).

A L

Thanks Em - that is really sensible advice.

I am always excited about new features but, as you say, the most important thing is a stable PVR system that just works (plus I really want to avoid irritating my wife and son with missed recordings).

I think I will wait for the next stable version - the things you have mentioned above are going to be really useful to me.

Thanks so much for you help with this (and another thread you have been helping me with).

You clearly have spent a lot of time with your answers and I just wanted to say that it is very much appreciated.

Em Smith

Thanks, much appreciated.

Don't let me put you off trying the 4.3 though since it is usually stable.

Since you're moving machine, you could test it for a few days by setting up your stable machine as a "sat>ip server" and use your other machine as a "sat>ip client". So your tuners from stable would then be shared with your other machine, and you can record the same shows on both machines (since the tuner mux would be shared and could record on both machines).

So on stable machine you'd go config->general->satip server, change port number from 0 to 9983 to enable the server; update "exported dvb-s tuners" to 2; then go to config->dvb->network and set "satip source" to 1 for your dvb-s network. Then on the other machine you'd start with a copy of your original machine's config and should be able to just config->general->satip server->discover and they should show up as adapters which you map to your "dvb-s" network. Sounds complicated, but it's not too difficult when you have the UI in front of you.

Might be more time/effort than it's worth, but it means you can run both systems for a few days to see if it is stable/worth the risk.

A L

A rather intriguing development.

I have come accross this page:

https://kodi.wiki/view/Video_file_tagging

which suggests that you can now put files into Kodi's libraries, scraping from metadata contained in the file.

I know that .ts files cannot include metadata, but .mkv files can.

If I start saving my files as MKVs and check the box for "Tag files with metadata", this might just work.

It could be the perfect solution that we have been looking for - I use your version of the Schedules Direct grabber. If I could have this data in my library, it would be the perfect solution.

Em Smith

As Sir Humphrey might say, that's a very courageous decision :)

I've not used mkv tagging for a while, but it used to be recommended to record ts and then post-process to mkv if required. I occasionally would have missing audio/subtitles with mkv, possibly because the channel stream can "reconfigure", so you'd have a 4:3 mp2 programme followed by a 16:9 aac/audio described programme (that you wanted to record), but tvh had already decided the streams to use so would ignore the aac stream.

Looks like it might be fixed though
https://tvheadend.org/issues/4882
.

I'm not sure if the files are tagged with artwork and don't know if it uses the exact same tags that Kodi now reads since their wiki suggests they don't use the iTunes tags that most software uses. But it's certainly worth a try.

Em Smith

I looked a little in to the Kodi scraping and my initial thought is that it wouldn't solve your problem of having episodes not scraped or scraped incorrectly.

I looked more closely at the wiki since I couldn't determine how it would work with episodes (which need tvshow info when using separate nfo files) vs. the easier case of movies which only really have title+year+plot, and perhaps "movie set" info.

The wiki said:
> There are three more tags that should be filled in for TV Shows: Show, Season and Episode Number. At the moment these tags do not seem to be of any practical use in Kodi, but should nevertheless be filled out as this may change.

It still wasn't clear, so I looked at the original Kodi PR
https://github.com/xbmc/xbmc/pull/13106/commits/47bb0b69bb07c9a6499e00dcbcabbee66a98ef29
and the latest version of the code in xbmc/video/tags/VideoTagLoaderFFmpeg.cpp.

Acasual inspection (of avtag->key) suggests it parses "title", "director", "date_released", "title", "composer", "genre", "date", "description", "synopsis", "track", "album", "artist"; and a couple of other places where is parses artwork, "kodi-metadata", "kodi-override-metadata", and a couple of places where it uses a different context for "IMDBURL" or "TMDBURL".

Perhaps it parses season and episode and associated show details elsewhere (so you know an episode is part of a specific series).

I could easily be wrong though since I haven't done much research, just a quick look.

Perhaps you could just add tags to one of your own existing files and give it a try?

One of my concerns is that the metadata option seems to be an "all or nothing" button (rather than per-directory so you can enable it on file hierarchy you know is good).

I'm also not sure what happens if you have, say, a title in the mp4 but no plot or artwork. This probably means you don't get any art. And since they don't parse cast, it means the one feature I like of displaying actor thumbnails would presumably be lost.

It seems the best tag to add would be the TMDBURL or IMDBURL and then, perhaps, it would scrape the extra info it needed for both episodes and movies.

Em Smith

For your missing recordings, you could try the attached script to re-create a basic dummy dvr/log so you are not missing so many items from your reinstall.

Download the script, then "chmod 755 tvhimport" to make it executable. Backup.

Run it with the name of a media file and it will print to the screen what it would generate in the logfile (so you can check it). Only fields you're probably interested in are title and episode. We assume files are roughly named "title - SxxEyy.ts" or "title (yyyy).ts".

Run it with "./tvhimport --dvr-log-dir /path/to/hts/dvr/log file_to_import.ts" and it will generate a file for you in that directory. You can specify a wildcard such as *.ts. It's not a good idea to include files that already have an existing dvr/log.

I'd probably output to a temporary directory first, check a few files to see if they look correct, then chown/chmod/"mv -i" a couple of the files to the correct location, reboot tvheadend and see if it works in the UI/Kodi.

Seems to work on tvh4.3, but some fields were changed in 4.3 so I don't know if it works on tvh4.2. Obviously many of the fields are dummy since the information has been lost. I can't guarantee it won't mess something up since you're not really meant to mess around with internals like this, but you pays your money, you takes your chances.

Joe User

Maybe it would be better to use the api to "import" the new dvr log entries?
From this thread ([[https://tvheadend.org/d/6128-help-with-unusual-setup-needed-two-th-servers-sharing-recordings-but-no-settings?r=34637#message-34637]]

here is a partial script which adds the "conf=" prefix to a dvr/log file and then uses curl to create the dvr entry and test if it worked.

# Use logfile to create json data for import dir="/path/to/recordings/json/" # note important trailing / filename=${file##*/} # get just filename printf "conf=" | cat - "$logfile" > "$dir$filename".import.json #wait to make sure the network file is written sleep 5 # Create recording entry on play2 curl -q --data @"$dir$filename".import.json 'http://user:pass@wetek_play2:9981/api/dvr/entry/create' > "$dir$filename".reply if grep -q uuid "$dir$filename".reply ;then echo "worked" mv "$dir$filename".import.json "$dir"$filename"import.json.done else echo "failed" fi

Definitely use a temp directory to create the entries for import. (do not touch the dvr/log directory)

This way you know which imported ok and you do not have to stop and restart tvheadend...

Sorry, I do not have time to write a full script, but the important parts are there (add "conf=", curl command and test for result).

Em Smith

That's very good advice.

I think that in the off-chance we create the same uuid as an existing entry then we'd get an error back from the api and could just re-run the generation to create a new uuid.

Joe User

Using the api, you do not need to generate any uuid (it is not in the conf ) - tvheadend will create the uuid when it creates the dvr log entry (and file).
A temporary file you create is only used to upload the data into tvheadend, which itself will create the real file in the dvr/log dir.

Em Smith

Newer version attached that uses the api directly, with "--dry-run" to test it out without doing an actual import.