Project

General

Profile

[SOLVED] Importing recordings ?

Added by Hairy Wookiee almost 2 years ago

Hi

Is there a way to import a bunch of recordings into Tvheadend 4.2 so that they will show up with the rest of the recordings ?

The reason I as is I used to run Tvh & Kodi on openelec but the harddisc died, however I have the .mkv recordings (they were on another drive) and I've managed to rescue the /dvr/log/ files but the paths to the recordings on my new setup are different so when I copy the rescued files into my new setup they all show up as "Failed recordings"

IIRC the old recordings were done under 3.9something and I'm now running 4.2

thanks


Replies (16)

RE: Importing recordings ? - Added by Em Smith almost 2 years ago

If you have the dvr/log then you can use symlinks to fix up the paths.

Assume your old recordings were on a 1tb drive and the path was /mnt/1tb/rec and your new files are on a 2tb drive on /mnt/2tb/rec.

So you've moved /mnt/1tb/rec/*mkv to /mnt/2tb/rec/. Removed the old 1tb drive. Now you want to create a symlink such as:

cd /mnt
rmdir 1tb
ln -s 2tb 1tb

Now you can access the files via /mnt/1tb or /mnt/2tb.

So when the dvr/log points to /mnt/1tb is will traverse the symlink and find them in /mnt/2tb.

Alternatively you can fix up the paths in dvr/log. So, when tvheadend is not running you can run:

perl -pibak -e 's!/mnt/1tb!/mnt/2tb!' dvr/log/*

Which means substitute (replace) /mnt/1tb with /mnt/2tb in dvr/log files and rename the originals to have an extension of .bak. If the files look ok then remove the .bak files and restart tvheadend.

RE: Importing recordings ? - Added by Em Smith almost 2 years ago

This will show you which dvr/log files refer to files that can not be found:

grep filename dvr/log/* | awk -F\" '{print $4}' | while read f; do ls "$f" > /dev/null; done 2>&1

It will output lines saying "cannot access xxxx".

RE: Importing recordings ? - Added by Hairy Wookiee almost 2 years ago

Em Smith wrote:

Alternatively you can fix up the paths in dvr/log. So, when tvheadend is not running you can run:
[...]
Which means substitute (replace) /mnt/1tb with /mnt/2tb in dvr/log files and rename the originals to have an extension of .bak. If the files look ok then remove the .bak files and restart tvheadend.

Hi
I tried that, it correctly rewrites the path in the file to the new location but I still end up with all the old recordings showing up in "failed"

I also ran the other command you listed

grep filename dvr/log/* | awk -F\" '{print $4}' | while read f; do ls "$f" > /dev/null; done 2>&1

Which does list the files that can't be found, is there an easy way for it to list the name of the dvr/log file that it relates to ?

RE: Importing recordings ? - Added by Hairy Wookiee almost 2 years ago

Hairy Wookiee wrote:

Em Smith wrote:

Alternatively you can fix up the paths in dvr/log. So, when tvheadend is not running you can run:
[...]
Which means substitute (replace) /mnt/1tb with /mnt/2tb in dvr/log files and rename the originals to have an extension of .bak. If the files look ok then remove the .bak files and restart tvheadend.

Hi
I tried that, it correctly rewrites the path in the file to the new location but I still end up with all the old recordings showing up in "failed recordings" tab.

I also ran the other command you listed

grep filename dvr/log/* | awk -F\" '{print $4}' | while read f; do ls "$f" > /dev/null; done 2>&1

Which does list the files that can't be found, is there an easy way for it to list the name of the dvr/log file that it relates to ?

RE: Importing recordings ? - Added by Em Smith almost 2 years ago

Try this:

grep filename dvr/log/*  | while read l; do f=`echo $l | awk -F\" '{print $4}'`; ls -lasd "$f" > /dev/null 2>&1  || echo "Not found: $l"; done

It will show the filename and the dvr log filename for any files that can not be found.

If you change the "|| echo" to "&& echo" it will show you the files that can be found in case that is also useful to you.

RE: Importing recordings ? - Added by Em Smith almost 2 years ago

When you start tvheadend it will log "dvr: unable to stat file 'xxxx'" for each filename that it cannot find. I'm not sure where OpenElec puts the logs, maybe /var/log/syslog or underneath the tvheadend directory.

The only other thing I can think of that may help is that perhaps it is file permissions on the directory if the old entries are in a different sub-directory?

The dvr log entries have a "fileremoved" flag in them but that should be 0 for your case since that flag is slightly different to the "file cannot be found" case. But, perhaps best to double-check a couple of files. (grep fileremoved dvr/log/*)

Otherwise, unfortunately, it's a case of checking a file that works and one that doesn't and see if there is something that seems majorly wrong.

RE: Importing recordings ? - Added by Hairy Wookiee almost 2 years ago

Well I tried your last command and it listed the files perfectly so I now know which to delete which I have now done.

However the modified dvr/log files still don't add the "old" recordings into Tvh, they just end up listed in the "Failed Recordings" tab and I cannot play them.

Looking at the individual files for recordings made with Tvh 4.2 and those of my old setup there is quite a difference between the content of the dvr/log files. I can post an example of both if it would help sort out this problem.

RE: Importing recordings ? - Added by Em Smith almost 2 years ago

If you can post an example and what difference you noticed then I'll take a look.

RE: Importing recordings ? - Added by Em Smith almost 2 years ago

And (before I forget) it might be worth taking a backup before the next lot of edits just in case. So if you have recordings under /storage/rec then this will create a backup in that directory of all files in the dvr/log.

tar cvf /storage/rec/backup.tar dvr/log

You can restore them (but won't delete any newly added files) using this in the same directory you ran the original tar:

tar pxvf /storage/rec/backup.tar

RE: Importing recordings ? - Added by Hairy Wookiee almost 2 years ago

thanks for the heads up about backing up the files, but I did that before I started all this by using "cp", I've a lot of storage space on this machine so I didn't bother anything fancier.

Here's a NEW recording as done by Tvh 4.2

{
    "enabled": true,
    "start": 1510693200,
    "start_extra": 0,
    "stop": 1510696800,
    "stop_extra": 0,
    "channel": "c21ce739589fe95431a87819c5551dec",
    "channelname": "4Music",
    "title": {
        "eng": "8 Out of 10 Cats Best Bits" 
    },
    "subtitle": {
        "eng": "S17, E11. Compilation edition with Jimmy Carr, team captains Sean Lock and Jon Richardson, and guests including Jo Brand, David O'Doherty, Katherine Ryan, Jerry Springer, and Roisin Conaty." 
    },
    "description": {
        "eng": "S17, E11. Compilation edition with Jimmy Carr, team captains Sean Lock and Jon Richardson, and guests including Jo Brand, David O'Doherty, Katherine Ryan, Jerry Springer, and Roisin Conaty." 
    },
    "pri": 2,
    "retention": 0,
    "removal": 0,
    "playposition": 0,
    "playcount": 0,
    "config_name": "9fd12d0886d1e62684a33493c13c2cab",
    "creator": "192.168.0.128",
    "errorcode": 0,
    "errors": 1,
    "data_errors": 0,
    "dvb_eid": 0,
    "noresched": true,
    "norerecord": false,
    "fileremoved": 0,
    "autorec": "bac31816545721d1f9d43651b49979ea",
    "timerec": "",
    "parent": "",
    "child": "",
    "content_type": 3,
    "broadcast": 0,
    "comment": "Auto recording: Created from EPG query",
    "files": [
        {
            "filename": "/mnt/3tb/video-files/8 Out of 10 Cats Best Bits/8 Out of 10 Cats Best Bits.mkv",
            "info": [
                {
                    "type": "MPEG2VIDEO",
                    "width": 704,
                    "height": 576,
                    "duration": 3600,
                    "aspect_num": 16,
                    "aspect_den": 9
                },
                {
                    "type": "MPEG2AUDIO",
                    "language": "nar",
                    "audio_type": 0,
                    "audio_version": 2
                },
                {
                    "type": "MPEG2AUDIO",
                    "language": "eng",
                    "audio_type": 0,
                    "audio_version": 2
                },
                {
                    "type": "DVBSUB",
                    "language": "eng",
                    "composition_id": 2,
                    "ancillary_id": 2
                },
                {
                    "type": "TEXTSUB",
                    "language": "eng",
                    "composition_id": 0,
                    "ancillary_id": 0
                }
            ],
            "start": 1510693080,
            "stop": 1510695949
        },
        {
            "filename": "/mnt/3tb/video-files/8 Out of 10 Cats Best Bits/8 Out of 10 Cats Best Bits-2.mkv",
            "info": [
                {
                    "type": "MPEG2VIDEO",
                    "width": 704,
                    "height": 576,
                    "duration": 3600,
                    "aspect_num": 16,
                    "aspect_den": 9
                },
                {
                    "type": "MPEG2AUDIO",
                    "language": "nar",
                    "audio_type": 0,
                    "audio_version": 2
                },
                {
                    "type": "MPEG2AUDIO",
                    "language": "eng",
                    "audio_type": 0,
                    "audio_version": 2
                },
                {
                    "type": "DVBSUB",
                    "language": "eng",
                    "composition_id": 2,
                    "ancillary_id": 2
                },
                {
                    "type": "TEXTSUB",
                    "language": "eng",
                    "composition_id": 0,
                    "ancillary_id": 0
                }
            ],
            "start": 1510696049,
            "stop": 1510697100
        }
    ]
}

And here is one of the old recording files I got from my previous install with a corrected path.

{
        "enabled": true,
        "start": 1450306800,
        "start_extra": 0,
        "stop": 1450314900,
        "stop_extra": 0,
        "channel": "5be62462f0d82696e56a2425d611e3ec",
        "channelname": "BBC Four HD",
        "title": {
                "eng": "2001: A Space Odyssey" 
        },
        "subtitle": {
                "eng": "Classic sci-fi in which an apparently intelligent monolith is discovered on the moon. Also in HD. [1968] [AD,S]" 
        },
        "description": {
                "eng": "Classic sci-fi in which a monolith is discovered on the moon. Astronauts following its radio signal are drawn into a struggle for sur$
        },
        "pri": 2,
        "retention": 0,
        "container": -1,
        "config_name": "b1a1fe764192d816d91026a63372077a",
        "filename": "/mnt/3tb/video-files/old-recordings/2001: A Space Odyssey/2001: A Space Odyssey.mkv",
        "errorcode": 0,
        "errors": 0,
        "data_errors": 0,
        "dvb_eid": 45400,
        "noresched": false,
        "autorec": "",
        "timerec": "",
        "content_type": 1,
        "broadcast": 5406682
}

RE: Importing recordings ? - Added by Em Smith almost 2 years ago

OK. So it looks like you're missing the file section.

The attached ugly code will add it back in. It doesn't use any libraries since OpenElec might not have them installed.

So it takes one dvr/log file as input and outputs the new version to stdout. So we send it to a file, diff to check it looks ok, if so move it over the old version.

For example for file dvr/log/aaaaaaaa:

./tvhdvr dvr/log/aaaaaaaa > /tmp/new_version
diff dvr/log/aaaaaaaa /tmp/new_version
mv /tmp/new_version dvr/log/aaaaaaaa

If that works then we can write a script to do them all.

tvhdvr (731 Bytes) tvhdvr

RE: Importing recordings ? - Added by Em Smith almost 2 years ago

Assuming all the files are in old-recordings then the wrapper would be something like:

grep -l /mnt/3tb/video-files/old-recordings/ dvr/log/* | while read f; do ./tvhdvr $f > $f.new && mv $f.new $f; done

(Though you'd have to put the correct path to where you saved tvhdvr instead of ./tvhdvr)

RE: Importing recordings ? - Added by Hairy Wookiee almost 2 years ago

That did the trick, thanks for the help.

RE: Importing recordings ? - Added by Em Smith almost 2 years ago

That's fantastic news. Glad it worked.

I don't know why tvheadend didn't do this file upgrade automatically. It looks like the list of files was added a couple of years ago so it's surprising no-one had a problem when they upgraded to 4.2.

RE: [SOLVED] Importing recordings ? - Added by Ed Llew over 1 year ago

Great info, thank you.
It helped me move over my recordings (log files) from TvHeadEnd 4.0.x to 4.2.x , in a LibreElec+RaspberryPi setup, when I upgraded to the most recent LibreElec.

I did need to make one small addition to the tvhdvr script, included here in case it helps the next guy out. My situation requires the "owner" line also with my user name. This may have to do with user/permission settings somewhere, I don't know:

if ($filename && $start && $stop) {
print <&lt;EOF;
"files": [ {
"filename" : "$filename",
"start": $start
"stop": $stop
}
],
"owner": "johndoe",
EOF
    (1-16/16)