Rcorded TV show file naming issues with TVDB

Mike Gideons

I'm using TVHeadEnd (TVHE) for my DVR functionality. The only pertinent configurations/settings i can seem to find in the TVHE server web interface for file naming is "Make Subdirectories Per Title" and "Include Episode In Filename". When enabled, i get a subdirectory which is named the same name as the show (this is compatible with TVDB conventions), and then i get a file name such as Scorpion.E37.ts (this seems to NOT be compatible with TVDB naming conventions per the info at http://kodi.wiki/view/Naming_video_files/TV_shows#Shows_without_seasons as the 'p' is missing, as in ep37 instead of E37). So, i enable the use absolute ordering (single season) setting in the TVDB addon. In this event, The TVDB addon doesn't seem to find anything. Using the Kodi naming convention as specified in the aforementioned link, it would seem that the file name should be Scorpion.ep37.ts (note the additional character, 'p'). However, after renaming the file, the TVDB addon still doesn't find anything, that is, the file doesn't show up in the library and no metadata is listed.

But if i go to thetvdb.org and find that episode, i find that it is actually season 2 episode 15. So i name the file Scorpion.s02e15.ts and what do you know, TVDB addon finds it and loads the metadata and i can now see the episode in the library.

So, here are my questions:
1) does anybody know of a setting that i'm missing to get the TVDB addon to use the absolute episode number (preferably in the format that TVHE uses, which is an 'E' followed by the absolute episode number)?
2) does anybody know if the extra character 'p' in the file name is really needed for absolute episode numbering, that is, is anything.E37.ts just as valid as anything.ep37.ts?
3) where is the best place for documentation on how to use the post processing functionality of TVHE? (What i'm looking to do is pass the name of the file that was just created from the recording to a script)

Steve Black

I had the same experience with absolute episode numbers. I would have to guess that Kodi has a bug preventing it from recognizing such as documented in their wiki, although, I never tried change the 'E' from uppercase to lowere, just added the 'p'.

I DID discover that Zap2It provides s00e00 numbering in their feed, which SchedulesDirect (non-JSON) does not. Together with being free, I was persuaded to switch. However, even then, some broadcasts are still limited to absolute numbering. (Carol Burnett and Friends on MeTV, for example.) My methodology involves moving and renaming my recording files with filebot which gets me around this issue. But of course, it adds its own issues - like breaking the TVH recording logs.

Anyway, that leads me finally to an answer to your third question, since I am successfully using the Post Processing feature. The Wiki page here has a link to the documentation, where you will find the Post Processing interface documented under Web Interface Guide->Configuration Tab->Recording->Digital Video Recorder Profiles. (Direct link: http://docs.tvheadend.org/webui/config_dvr/) There is a list of format strings you can include among the arguments to your script or command, and there is an example command line using some of those strings. In my case I wrote a script that accepts a few of those strings as well as a couple of arguments that identify what I want to do based on the particular recording profile.

I hope this helps you out.

Mike Gideons

Very helpful. I will check out the info in the link you provided. My next step is to get a command line / batch type of tool that can compress these recorded shows. I've read some stuff on HandbrakeCLI and filebot so it's good to know you've had success with one of those. As for breaking the TVHE recording logs, yeah, that's what led me down this path to begin with. At some point (i think i know when/how) the log file got corrupted and i thought i lost all my shows. Then i found that the file were still there, just not linked with TVHE. That's what led me to create the video library in the first place.

One possible way around the issue of staying sync'd with TVHE recording log file would be for the post processing script to not only rename (and possibly compress) the file but to create a symbolic link with the old file name to keep it sync'd with TVHE. I could then use TVHE to delete watched/unwanted shows (cleaning up the log file in the process), which would really just be deleting a link. I could then setup a cron job to compare the links with the files and if the link of the same base name is missing, it will delete the file that it was linked to, in essence deleting the file in the library. The next time the library is updated (maybe do that daily with a cron job too), it will see the file is missing and remove the item from the library (or at least it should).

Steve Black

Curious to hear your results with symbolic links. I never tried that but tried recreating the original file as zero length with touch. That did not work. It SEEMS like TVH sees the file go away and considers the recording broken by reason of "file missing". Note however, that the log file does remain in place and TVH continues to manage it for both duplicate avoidance ad retention time. That means that you if you manage to retain the original file's base name (by symlink or any other means) you can grep for that file name among the log files and delete the recording file (and link if necessary) if no matching log file is found. I have a script that does this successfully, though not with renamed files, and I have not gotten it to work under cron for some reason.

Here is my current methodology:
* I have two profiles: eps-perm and eps-temp. eps-perm specifies a very long retention period to allow duplicate detection for shows I want to keep permanently. eps-temp has a short retention to provide a small number of rotating episodes
* both profiles record to the same folder, but each one passes an argument to post processing that the script interprets to control the destination folder
* The script assembles the necessary information based on arguments and calls filebot to do the renaming - permanent shows move to one folder and temporaries to another
* Kodi has TV Show sources defined for each of these folders
* I use the XBMC Library Auto Update addon to periodically refresh the library with newly added content. (The addon includes a clean function to remove missing content but that is not working for me right now, due, I think, to some corruption in my library. Takes hours to run and blocks Kodi with a dialog.)

Steve Black

I forgot to mention: on reason for my methodology is that I do NOT want recordings to show up in the PVR section of Kodi. I want them only in the TV Shows page, where there is complete show and episode information available. I find it more informative as well as less confusing than having two different places to watch tv shows.

Mike Gideons

@Steve, well i finally got around to starting my script and i'm having some difficulty. Per the site at the link you provided %p will give you the episode info to pass to your post processing script. I also use %b and %f and they seem to be ok. However, my script shows "%p" for my episode info; as if literally the string %p was passed to my script and not some string of the form S02.E07 (for example). Are you using %p?

Steve Black

I have never used %p, and I think it was added fairly recently. You might be using an older version of TVH that does not support it. Or, perhaps the doc has gotten ahead of the current production version, which is 4.0.8.

I have not needed this because filebot reads the version number & episode title directly from the filename, and changes both as needed based on TVDB results. Settings to control the file naming are included further down in the same doc page.

I don't know if this will help or hurt you, but I will try to do a little clean-up on my script and then share it here.

Steve Black

@Mike

Here (finally) is my post-recording script. I just completed changes that update the TVH recording log with the new name of the file after moving it. This still leaves TVH thinking the recording is broken, but it allows for finding the log file based on the recording file name. This way an automated script can walk through recording files and identify that the log file is missing and that the recording should be deleted to correspond with the retention rule in the TVH profile.

[EDIT: Code replaced with some changes made necessary by a raft of overdue system upgrades. Also upgraded to tvheadend 4.1 (unstable) and found that quotes are now needed around each command line argument to avoid problems with special characters in filenames, etc.]


#!/bin/bash

#
# Constants
#
TVRENAME_FMT_DATE="{n})/{n}.{airdate}.{t}"
TVRENAME_DB_DATE="TheTVDB"
TVRENAME_FMT_EPISODE="{n} ({y})/{'Season '+s}/{n}.{s00e00}.{t}"
TVRENAME_DB_EPISODE="TheTVDB"
TVRENAME_FMT_MOVIE="{n}/{n}"
TVRENAME_DB_MOVIE="TheMovieDB"
TVH_LOG_HOME="/home/hts/.hts/tvheadend/dvr/log"

#
# Argument variables
#
sSourceFile="$1"
sSourcePath="${1%/*}"
sBaseName="$2"
sShowName="$3"
sError="$4"
sLibType="$5"
sRetention="$6"

#
# Compensate for known EPG/TVDB discrepancies
#
case "$sShowName" in
    "Sonny and Cher Comedy Hour")   sShowName="Sonny Cher Comedy Hour";;
    #*)
esac

#
# Internal variables
#
sTmpLog="$sSourceFile.log"

##############################################################################
#
# logMsg() - write log messages to output locations
#
logMsg() {
    msg="$1"
    echo -e "`date +'%b %d %H:%M:%S'`: $msg"
    echo -e "`date +'%b %d %H:%M:%S'`: $msg">>"$sTmpLog"
}

##############################################################################
##############################################################################
## MAIN
##############################################################################
##############################################################################
logMsg "******************** TVHPostRecord.bash started"

case "$sRetention" in
    "perm") sDest="/media/NetStore3/Recorded TV/perm";;
    "temp") sDest="/media/NetStore3/Recorded TV/temp";;
         *) sDest="/media/NetStore3/Recorded TV/temp";;
esac

case "$sLibType" in
    "episode")  sFmt="$sDest/$TVRENAME_FMT_EPISODE"; sDB="$TVRENAME_DB_EPISODE";;
       "date")  sFmt="$sDest/$TVRENAME_FMT_DATE"; sDB="$TVRENAME_DB_DATE";;
      "movie")  sFmt="$sDest/$TVRENAME_FMT_MOVIE"; sDB="$TVRENAME_DB_MOVIE";;
            *)  sFmt="$sDest/$TVRENAME_FMT_EPISODE"; sDB="$TVRENAME_DB_EPISODE";;
esac
#logMsg "          sFmt : $sFmt"
#logMsg "           sDB : $sDB"

logMsg "   sSourceFile : $sSourceFile"
#logMsg "   sSourcePath : $sSourcePath"
#logMsg "     sBaseName : $sBaseName"
logMsg "     sShowName : $sShowName"
logMsg "        sError : $sError"

#
# Rename file if recording is successful
#
if [[ "$sError" == "OK" ]]
then
    # Set the command and save as a script for later execution in case of failure here
    sRenameCmd="-rename \"$sSourceFile\" -non-strict --action move --format \"$sFmt\" --q \"$sShowName\" --db $sDB"
    logMsg "     RenameCmd : filebot $sRenameCmd"
    echo "filebot $sRenameCmd">"$sSourceFile.RenameCmd"
    chmod 774 "$sSourceFile.RenameCmd"

    # Execute the rename (would be nice to use the variable here, but haven't made my way around all the quotes)
    filebot -rename "$sSourceFile" -non-strict --action move --format "$sFmt" --q "$sShowName" --db "$sDB">"$sSourceFile.fbmsg" 2>&1
    rc=$?

    # Process the rename results
    logMsg "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
    while read sLine
    do
        if ! [ -z "$sLine" ]; then logMsg "$sLine"; fi 
        #then 
        #    logMsg "$sLine"
        #fi
        case "${sLine:7:6}" in
             "Rename")  sNewFile="${sLine##*[}";    sNewFile="${sNewFile%]*}";;
            "Skipped")  sSkipMsg="$sLine";;
            #*)          ;;
        esac
    done<"$sSourceFile.fbmsg"
    logMsg "~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~"
    logMsg "     Rename RC : $rc"

    #
    # Deal with any problems with renaming
    #
    if ! [ -f "$sSourceFile" ]
    then
        # If the source file is gone, get clean up artifacts
        rm "$sSourceFile.RenameCmd"
        rm "$sSourceFile.fbmsg"

        # Update the recording log with the new name
        sRecordingLog=`fgrep -l "$sSourceFile" $TVH_LOG_HOME/*`
        if [ -f "$sRecordingLog" ]
        then
            sed -i "s/${sSourceFile//\//\\\/}/${sNewFile//\//\\\/}/" "$sRecordingLog"
            if [ $? -eq 0 ]
            then
                logMsg "    Log Update : $sRecordingLog updated to \"$sNewFile\""
            else
                logMsg "    Log Update : Edit failed with rc: $?"
            fi
        else
            logMsg "    Log Update : Recording log not found for \"$sSourceFile\""
        fi
    else
        # Report error, or cleanup after name collision
        if [ $rc -ne 0 ]
        then
            logMsg "!!!! Rename Error"
        else
            logMsg "$sSkipMsg"
            rm "$sSourceFile"
            rm "$sSourceFile.RenameCmd"
            rm "$sSourceFile.fbmsg"
        fi
    fi
fi

logMsg "******************** TVHPostRecord.bash ended"

#
# If all is well, clean up the temporary post-processing log
#
if ! [ -f "$sSourceFile" ]
then
    rm "$sTmpLog"
fi

Steve Black

Just rebooted and found that TVH now recognizes those updated log files as valid and moved them from Failed back to Finished. Not exactly GOOD news, but certainly a positive.