Project

General

Profile

Re-Import Existing Recordings

Added by John Jones over 1 year ago

I'm running HTS Tvheadend-testing 4.4.20181215 from http://syno.dierkse.nl/ on a Synology NAS.

It stopped working a little while ago and I recently had a chance to get in there and try to fix it. I backed up the entire /volume1/\@appstore/tvheadend-testing/var/ folder and uninstalled it, then reinstalled it, but it didn't work. Turns out the reason everything stopped working was that the update to FFMPEG my Synology prompted me to do broke the symlink to the application binary that allowed the AUS Freeview playlist to work. I reinstalled, deleted the broken symlink, re-created it, and it all just fired up again. Great!

Problem is, however, none of my previous library of recordings is showing up. The file path and setup I had for recordings is the same, and I've tested to see that the file structure and paths are all exactly the same as the previous installation.

I tried copying the files that were in the previous var/dvr/config and var/dvr/log directories over, but this did nothing. I've seen some mentions of scripts in some other logs, but I really have no idea what people are talking about in those posts, and none of the paths look the same as my Synology installation, so don't really know how to apply their fixes to my problem. I don't even really know how to run a Python script in a Synology SSH session.

Can someone please help me get my extensive collection of previous recordings to reappear in my library? Do I have any options here?

Thank-you!

MJJ


Replies (71)

RE: Re-Import Existing Recordings - Added by Hiro Protagonist over 1 year ago

I'm assuming you tried restarting TVheadend after copying your config and log directories. If not, do that now, if you tried that without success, read on.

Does your log directory contain logs for the recordings you have?

An easy way to check [using paths appropriate for your system] is:

grep filename /path/to/dvr/log/*

If you get a hit, then there is a log file for that recording. Try for a few different recordings.

If you have log files, then things can be fixed, we can still fix it without the log files, but metadata will be lost.

Secondly, what do your recording paths look like? For instance if your recordings are in /foo/bar, and you had recordings of a program called "Dispatches", would it be stored in a directory called /foo/bar/Dispatches ?

In this thread https://tvheadend.org/boards/5/topics/41062 I posted my tvhimport.sh script

If you can answer yes to the above 2 questions, then that script will work for you. It assumes recordings are in place, and reinstates the recording in TVheadend. To use it you need to know the full path to the recording file, and the name of the log file.

Before starting, you must edit the LOGPATH variable in the script and set it to the absolute path to your /whatever/dvr/log directory.

An example - of course to use it you'll need to use paths appropriate for your setup:

grep Maigret2020-02-04.mkv .hts/tvheadend/dvr/log/*
.hts/tvheadend/dvr/log/c70d16a17d0042e1767714ecb80006dc: "filename": "/var/spool/pvr/Maigret/Maigret2020-02-04.mkv",

tvhimport.sh /var/spool/pvr/Maigret/Maigret2020-02-04.mkv -l c70d16a17d0042e1767714ecb80006dc

If you don't have a logfile for a recording, you can still import it by just passing the full path to the filename and skipping the -l <logfilename>, but it will import without metadata.

RE: Re-Import Existing Recordings - Added by John Jones over 1 year ago

Hi Hiro,

Thanks again for coming to the rescue of us TVHeadend strugglers.

I have log files. I don't know how to run your script, however.

Are you saying that I have to run it manually for each of the hundreds of files in my recorded program folder?

How do I run the script? Is there some sort of program I need to install into the Synology NAS with the SSH terminal? I really don’t know where to start with this... What's a grep?

Thanks,

MRR

RE: Re-Import Existing Recordings - Added by Hiro Protagonist over 1 year ago

John Jones wrote:

Hi Hiro,

Thanks again for coming to the rescue of us TVHeadend strugglers.

I have log files. I don't know how to run your script, however.

The example above it pretty explicit.

Are you saying that I have to run it manually for each of the hundreds of files in my recorded program folder?

Yes, and No. If the script works for you it is possible to write a wrapper script that runs it for every file in your recordings folder. It doesn't sound like you'd be up for that though.

How do I run the script? Is there some sort of program I need to install into the Synology NAS with the SSH terminal? I really don’t know where to start with this... What's a grep?

All you should need to install is the script itself, with execute permissions. You should find grep is already installed on any minimally useful unix-like system. Why not try grep out on the commandline and see?

RE: Re-Import Existing Recordings - Added by Hiro Protagonist over 1 year ago

Hiro Protagonist wrote:

Yes, and No. If the script works for you it is possible to write a wrapper script that runs it for every file in your recordings folder. It doesn't sound like you'd be up for that though.

I've written a script that can do this for you. However in order to make it work on your system I need to know:

The absolute path to your dvr/log directory, and the absolute path to your recordings directory.

Absolute paths start with a /

RE: Re-Import Existing Recordings - Added by John Jones over 1 year ago

A million thanks, Hiro. Really sorry you have to go through all this because of my ignorance and ineptitude with unix, but I only dive in when I absolutely have to and forget 99% of what I learn within a day or so of learning it when it comes to this stuff. Googling just brings up other people talking about stuff as if we all understand what they're talking about, so it gets so frustrating that I don't know what to do.

/volume1/@appstore/tvheadend-testing/var/dvr/log/ is the path, but there's a strange thing about the "at" symbol that means when I'm cd-ing into that path, I have to type /volume1/\@appstore/tvheadend-testing/var/dvr/log to get in there.

Can you please explain how to install the script with execute permissions? Where do I put it? What's the command I have to type to set the permissions correctly? What do I type to make it go?

Thank-you again and again.

RE: Re-Import Existing Recordings - Added by Hiro Protagonist over 1 year ago

John Jones wrote:

/volume1/@appstore/tvheadend-testing/var/dvr/log/ is the path, but there's a strange thing about the "at" symbol that means when I'm cd-ing into that path, I have to type /volume1/\@appstore/tvheadend-testing/var/dvr/log to get in there.

I need the path to your recordings as well.

RE: Re-Import Existing Recordings - Added by Hiro Protagonist over 1 year ago

I've attached a script which will do test run on your system.

This script won't change anything, so is safe to run. It will print some output - if you can copy & post the output, I'll be able to see if it's working as expected, and then we can go onto something more serious. This version of the script is deliberately limited to stop after scanning 5 recordings, as we don't need to do more than that for a test.

Copy the script onto your system, you can put it in your home directory.
To use it, you first need to make it executable - type:

chmod +x importall.sh

Then to run it type:

./importall.sh

RE: Re-Import Existing Recordings - Added by John Jones over 1 year ago

Scanning logs and recordings
grep: /volume1/@appstore/tvheadend-testing/var/dvr/log//dvr/log/*: No such file or directory
Preparing to import Gogglebox with no log
grep: /volume1/@appstore/tvheadend-testing/var/dvr/log//dvr/log/*: No such file or directory
Preparing to import Gogglebox with no log
grep: /volume1/@appstore/tvheadend-testing/var/dvr/log//dvr/log/*: No such file or directory
Preparing to import Australia-TEN with no log
grep: /volume1/@appstore/tvheadend-testing/var/dvr/log//dvr/log/*: No such file or directory
Preparing to import HD2020-02-2720-30.ts with no log
grep: /volume1/@appstore/tvheadend-testing/var/dvr/log//dvr/log/*: No such file or directory
Preparing to import Gogglebox with no log
grep: /volume1/@appstore/tvheadend-testing/var/dvr/log//dvr/log/*: No such file or directory
Preparing to import Gogglebox with no log

RE: Re-Import Existing Recordings - Added by John Jones over 1 year ago

Edited your script to remove the /dvr/log from the CONFIGDIR variable setting and uploaded it, ran again and got this now -

Scanning logs and recordings
basename: extra operand ‘/volume1/@appstore/tvheadend-testing/var//dvr/log/19dfdeb1ef88c91df245936d35b2c141’
Try 'basename --help' for more information.
Preparing to import Gogglebox with log
basename: extra operand ‘/volume1/@appstore/tvheadend-testing/var//dvr/log/19dfdeb1ef88c91df245936d35b2c141’
Try 'basename --help' for more information.
Preparing to import Gogglebox with log
basename: extra operand ‘/volume1/@appstore/tvheadend-testing/var//dvr/log/19dfdeb1ef88c91df245936d35b2c141’
Try 'basename --help' for more information.
Preparing to import Australia-TEN with log
Preparing to import HD2020-02-2720-30.ts with log 77200c696a6cd8c24e95fb4795a26868
basename: extra operand ‘/volume1/@appstore/tvheadend-testing/var//dvr/log/19dfdeb1ef88c91df245936d35b2c141’
Try 'basename --help' for more information.
Preparing to import Gogglebox with log
basename: extra operand ‘/volume1/@appstore/tvheadend-testing/var//dvr/log/19dfdeb1ef88c91df245936d35b2c141’
Try 'basename --help' for more information.
Preparing to import Gogglebox with log

RE: Re-Import Existing Recordings - Added by John Jones over 1 year ago

Ok, removed the extraneous forward slash from the path. I get this now -

Scanning logs and recordings
basename: extra operand ‘/volume1/@appstore/tvheadend-testing/var/dvr/log/19dfdeb1ef88c91df245936d35b2c141’
Try 'basename --help' for more information.
Preparing to import Gogglebox with log
basename: extra operand ‘/volume1/@appstore/tvheadend-testing/var/dvr/log/19dfdeb1ef88c91df245936d35b2c141’
Try 'basename --help' for more information.
Preparing to import Gogglebox with log
basename: extra operand ‘/volume1/@appstore/tvheadend-testing/var/dvr/log/19dfdeb1ef88c91df245936d35b2c141’
Try 'basename --help' for more information.
Preparing to import Australia-TEN with log
Preparing to import HD2020-02-2720-30.ts with log 77200c696a6cd8c24e95fb4795a26868
basename: extra operand ‘/volume1/@appstore/tvheadend-testing/var/dvr/log/19dfdeb1ef88c91df245936d35b2c141’
Try 'basename --help' for more information.
Preparing to import Gogglebox with log
basename: extra operand ‘/volume1/@appstore/tvheadend-testing/var/dvr/log/19dfdeb1ef88c91df245936d35b2c141’
Try 'basename --help' for more information.
Preparing to import Gogglebox with log

RE: Re-Import Existing Recordings - Added by John Jones over 1 year ago

This is how it looks to get the result above -

#!/bin/bash #
  1. Modify to suit
    CONFIGDIR='/volume1/\@appstore/tvheadend-testing/var'
    RECORDINGS='/volume1/TV/'

echo "Scanning logs and recordings"

COUNT=0

for FILE in $(find $RECORDINGS -name "*.ts" -o -name "*.mkv") ; do
LOGFILE=$(grep ${FILE} $CONFIGDIR/dvr/log/* | cut -d ':' -f 1)
if [ "$LOGFILE" ]
then
echo "Preparing to import $(basename $FILE) with log $(basename $LOGFILE)"
#tvhimport.sh "${FILE}" -l $LOGFILE
else
echo "Preparing to import $(basename $FILE) with no log"
#tvhimport.sh $COMMAND "${FILE}"
fi
let COUNT++
if [ $COUNT -gt 5 ]; then
exit 0
fi
done

RE: Re-Import Existing Recordings - Added by Hiro Protagonist over 1 year ago

Sorry about the configdir - my bad.

I'm a bit unsure about the problem with basename - the same string works fine at my end.

I'm also unsure why we're seeing stuff like Gogglebox and Australia-TEN, when the file command is looking for .ts and .mkv files. I've modified it to ignore directories.

Try my updated script.

RE: Re-Import Existing Recordings - Added by John Jones over 1 year ago

The recordings are organised into program series directories.

Here's the result with this new version -

Scanning logs and recordings
basename: extra operand ‘/volume1/\\@appstore/tvheadend-testing/var/dvr/log/19dfdeb1ef88c91df245936d35b2c141’
Try 'basename --help' for more information.
Preparing to import Gogglebox with log
basename: extra operand ‘/volume1/\\@appstore/tvheadend-testing/var/dvr/log/19dfdeb1ef88c91df245936d35b2c141’
Try 'basename --help' for more information.
Preparing to import Gogglebox with log
basename: extra operand ‘/volume1/\\@appstore/tvheadend-testing/var/dvr/log/19dfdeb1ef88c91df245936d35b2c141’
Try 'basename --help' for more information.
Preparing to import Australia-TEN with log
Preparing to import HD2020-02-2720-30.ts with log 77200c696a6cd8c24e95fb4795a26868
basename: extra operand ‘/volume1/\\@appstore/tvheadend-testing/var/dvr/log/19dfdeb1ef88c91df245936d35b2c141’
Try 'basename --help' for more information.
Preparing to import Gogglebox with log
basename: extra operand ‘/volume1/\\@appstore/tvheadend-testing/var/dvr/log/19dfdeb1ef88c91df245936d35b2c141’
Try 'basename --help' for more information.
Preparing to import Gogglebox with log

RE: Re-Import Existing Recordings - Added by Hiro Protagonist over 1 year ago

Hi,

There's something weird going on here, I've just removed the basename calls so I can see the raw paths.

RE: Re-Import Existing Recordings - Added by John Jones over 1 year ago

Wow... Ok. Here's the result -

Scanning logs and recordings
Preparing to import /volume1/TV/Gogglebox with log /volume1/@appstore/tvheadend-testing/var/dvr/log/0cebefe544356b3550b319f68feb4721
/volume1/@appstore/tvheadend-testing/var/dvr/log/144353f755d9857868b4596252192782
/volume1/@appstore/tvheadend-testing/var/dvr/log/19dfdeb1ef88c91df245936d35b2c141
/volume1/@appstore/tvheadend-testing/var/dvr/log/2c53bd38a6ce71217c33cec6a9c854eb
/volume1/@appstore/tvheadend-testing/var/dvr/log/77200c696a6cd8c24e95fb4795a26868
/volume1/@appstore/tvheadend-testing/var/dvr/log/85308a99cb82e413274eedf50f37d2de
/volume1/@appstore/tvheadend-testing/var/dvr/log/e7e63685d83d098d8525c72fd4aa8f1f
Preparing to import Australia/Gogglebox with log /volume1/@appstore/tvheadend-testing/var/dvr/log/0cebefe544356b3550b319f68feb4721
/volume1/@appstore/tvheadend-testing/var/dvr/log/144353f755d9857868b4596252192782
/volume1/@appstore/tvheadend-testing/var/dvr/log/19dfdeb1ef88c91df245936d35b2c141
/volume1/@appstore/tvheadend-testing/var/dvr/log/2c53bd38a6ce71217c33cec6a9c854eb
/volume1/@appstore/tvheadend-testing/var/dvr/log/77200c696a6cd8c24e95fb4795a26868
/volume1/@appstore/tvheadend-testing/var/dvr/log/85308a99cb82e413274eedf50f37d2de
/volume1/@appstore/tvheadend-testing/var/dvr/log/e7e63685d83d098d8525c72fd4aa8f1f
Preparing to import Australia-TEN with log /volume1/@appstore/tvheadend-testing/var/dvr/log/0cebefe544356b3550b319f68feb4721
/volume1/@appstore/tvheadend-testing/var/dvr/log/144353f755d9857868b4596252192782
/volume1/@appstore/tvheadend-testing/var/dvr/log/19dfdeb1ef88c91df245936d35b2c141
/volume1/@appstore/tvheadend-testing/var/dvr/log/2c53bd38a6ce71217c33cec6a9c854eb
/volume1/@appstore/tvheadend-testing/var/dvr/log/77200c696a6cd8c24e95fb4795a26868
/volume1/@appstore/tvheadend-testing/var/dvr/log/85308a99cb82e413274eedf50f37d2de
/volume1/@appstore/tvheadend-testing/var/dvr/log/e7e63685d83d098d8525c72fd4aa8f1f
Preparing to import HD2020-02-2720-30.ts with log /volume1/@appstore/tvheadend-testing/var/dvr/log/77200c696a6cd8c24e95fb4795a26868
Preparing to import /volume1/TV/Gogglebox with log /volume1/@appstore/tvheadend-testing/var/dvr/log/0cebefe544356b3550b319f68feb4721
/volume1/@appstore/tvheadend-testing/var/dvr/log/144353f755d9857868b4596252192782
/volume1/@appstore/tvheadend-testing/var/dvr/log/19dfdeb1ef88c91df245936d35b2c141
/volume1/@appstore/tvheadend-testing/var/dvr/log/2c53bd38a6ce71217c33cec6a9c854eb
/volume1/@appstore/tvheadend-testing/var/dvr/log/77200c696a6cd8c24e95fb4795a26868
/volume1/@appstore/tvheadend-testing/var/dvr/log/85308a99cb82e413274eedf50f37d2de
/volume1/@appstore/tvheadend-testing/var/dvr/log/e7e63685d83d098d8525c72fd4aa8f1f
Preparing to import Australia/Gogglebox with log /volume1/@appstore/tvheadend-testing/var/dvr/log/0cebefe544356b3550b319f68feb4721
/volume1/@appstore/tvheadend-testing/var/dvr/log/144353f755d9857868b4596252192782
/volume1/@appstore/tvheadend-testing/var/dvr/log/19dfdeb1ef88c91df245936d35b2c141
/volume1/@appstore/tvheadend-testing/var/dvr/log/2c53bd38a6ce71217c33cec6a9c854eb
/volume1/@appstore/tvheadend-testing/var/dvr/log/77200c696a6cd8c24e95fb4795a26868
/volume1/@appstore/tvheadend-testing/var/dvr/log/85308a99cb82e413274eedf50f37d2de
/volume1/@appstore/tvheadend-testing/var/dvr/log/e7e63685d83d098d8525c72fd4aa8f1f

RE: Re-Import Existing Recordings - Added by Hiro Protagonist over 1 year ago

OK, so it look like 'find' on your system is very basic, and doesn't support most of the normal options. I'll update taking that into account.

RE: Re-Import Existing Recordings - Added by John Jones over 1 year ago

No logs this time... All I get is this -

Scanning logs and recordings

RE: Re-Import Existing Recordings - Added by John Jones over 1 year ago

My system is running -

Linux HPDSN40L 3.10.102 #15217 SMP Fri Nov 10 20:09:05 CST 2017 x86_64 GNU/Linux synology_broadwell_3617xs

RE: Re-Import Existing Recordings - Added by Hiro Protagonist over 1 year ago

OK, that's even more limiting ... are all your recordings .ts files or do you have some .mkv?

RE: Re-Import Existing Recordings - Added by John Jones over 1 year ago

There's some of each. I can't remember why...

RE: Re-Import Existing Recordings - Added by John Jones over 1 year ago

I could mount those directories with my Mac and you might have a better chance of getting a script to do more of what you need it to do with that? Or I have an Ubuntu 16 VM I could use... ?

RE: Re-Import Existing Recordings - Added by Hiro Protagonist over 1 year ago

OK, had to do a bit of clunky logic to get around the limitations, try again...

RE: Re-Import Existing Recordings - Added by John Jones over 1 year ago

Scanning logs and recordings
./importall.sh: line 23: [: =: unary operator expected
./importall.sh: line 23: [: =: unary operator expected
./importall.sh: line 23: [: =: unary operator expected
./importall.sh: line 23: [: =: unary operator expected
./importall.sh: line 23: [: =: unary operator expected
./importall.sh: line 23: [: =: unary operator expected
Preparing to import HD2020-02-2720-30.ts with log 77200c696a6cd8c24e95fb4795a26868
./importall.sh: line 23: [: =: unary operator expected
./importall.sh: line 23: [: =: unary operator expected
./importall.sh: line 23: [: =: unary operator expected
./importall.sh: line 23: [: =: unary operator expected
./importall.sh: line 23: [: =: unary operator expected
./importall.sh: line 23: [: =: unary operator expected
Preparing to import HD2020-03-0520-30.ts with log 0cebefe544356b3550b319f68feb4721
./importall.sh: line 23: [: =: unary operator expected
./importall.sh: line 23: [: =: unary operator expected
./importall.sh: line 23: [: =: unary operator expected
./importall.sh: line 23: [: =: unary operator expected
./importall.sh: line 23: [: =: unary operator expected
./importall.sh: line 23: [: =: unary operator expected
Preparing to import HD2020-03-1220-30.ts with log 144353f755d9857868b4596252192782
./importall.sh: line 23: [: =: unary operator expected
./importall.sh: line 23: [: =: unary operator expected
./importall.sh: line 23: [: =: unary operator expected
./importall.sh: line 23: [: =: unary operator expected
./importall.sh: line 23: [: =: unary operator expected
./importall.sh: line 23: [: =: unary operator expected
Preparing to import HD2020-03-1920-30.ts with log 85308a99cb82e413274eedf50f37d2de
./importall.sh: line 23: [: =: unary operator expected
./importall.sh: line 23: [: =: unary operator expected
./importall.sh: line 23: [: =: unary operator expected
./importall.sh: line 23: [: =: unary operator expected
./importall.sh: line 23: [: =: unary operator expected
./importall.sh: line 23: [: =: unary operator expected
Preparing to import HD2020-04-1620-52.mkv with log e7e63685d83d098d8525c72fd4aa8f1f
./importall.sh: line 23: [: =: unary operator expected
./importall.sh: line 23: [: =: unary operator expected
./importall.sh: line 23: [: =: unary operator expected
./importall.sh: line 23: [: =: unary operator expected
./importall.sh: line 23: [: =: unary operator expected
./importall.sh: line 23: [: =: unary operator expected
Preparing to import HD2020-04-2321-45.mkv with log 19dfdeb1ef88c91df245936d35b2c141

(1-25/71)