AutoRec a specific Season

Ben Crundwell · 2016-08-21T23:18:28+00:00

I've searched through and can't appear to find any information, but apologise if the question has already been asked: I am transferring over from a Windows...

Ben Crundwell

Yeah that's working now. I've updated GitHub. Try using the following formatting:

(Big Bang Theory).*(S09)

Enjoy :)

A L

This thread is exactly what I am after but I cannot quite get it to work for me.

At the moment, I want to record every episode of Wheeler Dealers from Season 12 onwards.

What would I do to set up a timer for this?

Fretzke Biber

Bump!

I got the same problem as AL.

Take this case as example:


Title         | Subtitle
----------------------------------------------
The Simpsons  | Episode title yada yada (1990)
The Simpsons  | Episode title yada yada (1991)
The Simpsons  | Episode title yada yada (2018)

Now I'd like to not record all 30 years of episodes, but only the current ones. But a search string like
(Simpsons).*(2018)

with fulltext checked turns up an empty result. Seems whatever I do my regexps can only match against titleOR subtitle, but never a pattern in both combined.

Any ideas?

Em Smith

Regular expressions are applied per field so do not combine title, subtitle and description.

The reason it doesn't work for AL is that you use a different grabber. Ben said he has his own variant of tv_grab_sd (not the sqlite version you use).

You can either use his, or you could try my unsupported fork of tv_grab_zz_sdjson_sqlite on github
https://github.com/azlm8t/tv_grab_zz_sdjson_sqlite

Just copy the tv_grab_az_sdjson_sqlite file (yes, "az" not "zz" in the filename to avoid conflict) in to the same directory as your other grabbers (maybe /usr/local/bin) ensuring it is executable (sudo chmod 755 /usr/local/bin/tv_grab_az_sdjson_sqlite).

If you use mine then you need to:

# Backup your current .xmltv configuration from ~/.xmltv (since my version patches the DB to download artwork details);
# Change the grabber to tv_grab_az_sdjson_sqlite (look for it in "path") in tvheadend's configuration->channel/epg->epg grabber modules, and add extra arguments of "--update-description-with-title" or "--update-description-with-all" (without the quotes) which has more description fixups such as adding year, age rating and stars. I'd also add "--artwork-max-width=720" to avoid slow artwork downloads so it becomes "--update-description-with-all --artwork-max-width=720 --merge-split=5";
# ???

# Profit.

With the "all" option, the description is changed to be something similar to:



NCIS - S07E07 - Endgame.

S7/15 E7/24. (2009-11-10). (TV-14). Vance's family...

So this can be searched by a fulltext regex such as "^NCIS.*S07" or "^NCIS.S07E0?". Using "^NCIS.*2009" does not work (since it is not on one line), but with pcre2 (in newer Tvheadend versions) something like "(*CR)NCIS.*2009" will work, as will, for example, "(*CR):NEW:LIVE:.*Cricket" to detect live cricket matches.

Due to merging of SD hierarchy of data, sometimes you get bad data such as "S3/2" (season 3 of 2 seasons). And, of course, descriptions can vary over time, so "S3/4" may become "S3/5" in a few years' time, so dup matching on descriptions will fail.

The date in the example is the first broadcast date of the episode, but for generic shows such as news it is often the date of the first ever episode.

Mine does a few things slightly different to the "zz" grabber, such as using ttvdb episode numbers (which sometimes differ from SD episode numbers, but does Kodi mismatches happen less frequently); flagging "premiere"; removing five minute "news" programmes (with --merge-split=5), and some other minor features.

It works for me. YMMV. If it doesn't work then you can revert back to your other version.

Mischa Mischock

It is also possible to run this script on libreelec?
i put the "az" file to the others tv_grab files, "/storage/.kodi/addons/service.tvheadend44/bin/tv_grab_az_sdjson_sqlite"

It will not be recognized after a reboot..

To use the script, do I need an account at schedulesdirect.org?
or i can also attach my own xml file, from webgrabplus?

Em Smith

Yes, this only would only work with SchedulesDirect so needs an account with them.

I think LibreElec used to only have tv_grab_file and didn't (doesn't?) ship with perl (which this and most other grabbers use) so I'm not sure if it works on LibreElec.

Mischa Mischock

@Em Smith, can you please provide a finished xml (epg) file, I would like to compare it with webgrabplus and maybe i can implement it there so that regexp recognizes it, thanks

Em Smith

Sure.

There's nothing special about my epg, it just puts "title - season/episode - subtitle" on one line in the description.

So an example in the client looks like this:

NCIS - S07E19 - Guilty Pleasure. S7/15 E19/24. (2010-04-06). (TV-14). When the team members link the death of a Navy officer to a call girl, they are forced to ask Holly Snow for help to find the culprit. Cast: Mark Harmon, Michael Weatherly, David McCallum, Pauley Perrette, Sean Murray, Cote de Pablo, Rocky Carroll, Dina Meyer, Adam Kaufman, Jason London, Taylor Cole, Jillian Bach, Bruce Nozick, Hayley Chase, Cliff Weissman, Kallee Brookes, Chris Sheffield, Jeff Wolfe. Categories: Action, Adventure, Crime drama, Episode, Mystery, Series, Show.

The description is from this subset of the file at the end of the post (my whole file is too big to include).

So, each "programme" has a "desc" tag which contains the description. I include some extra information in the description since most skins don't show the information I want to see (such as age rating).

The season/episode number is taken from the "episode-num" tag, but be careful since there are different formats for episode-num so some (such as xmltv_ns) has "6 / 15" to mean "S7/15" and "18 / 24" to mean "E19/24".


 <programme channel="I52335.json.schedulesdirect.org" start="20180809190000 +0000" stop="20180809200000 +0000">
  <title>NCIS</title>
  <sub-title>Guilty Pleasure</sub-title>
  <desc lang="en-GB">NCIS - S07E19 - Guilty Pleasure.

S7/15 E19/24. (2010-04-06). (TV-14). When the team members link the death of a Navy officer to a call girl, they are forced to ask Holly Snow for help to find the culprit.

Cast: Mark Harmon, Michael Weatherly, David McCallum, Pauley Perrette, Sean Murray, Cote de Pablo, Rocky Carroll, Dina Meyer, Adam Kaufman, Jason London, Taylor Cole, Jillian Bach, Bruce Nozick, Hayley Chase, Cliff Weissman, Kallee Brookes, Chris Sheffield, Jeff Wolfe.

Categories: Action, Adventure, Crime drama, Episode, Mystery, Series, Show.
&lt;img src="https://s3.amazonaws.com/schedulesdirect/assets/p11857845_b_h13_ab.jpg" width="120"/&gt;
&lt;img src="https://json.schedulesdirect.org/20141201/image/assets/p8059916_e_h9_ab.jpg" width="120"/&gt;</desc><credits><director>James Whitmore Jr.</director><actor role="NCIS Special Agent Leroy Jethro Gibbs">Mark Harmon</actor><actor role="NCIS Special Agent Anthony &quot;Tony&quot; Dinozzo">Michael Weatherly</actor><actor role="Dr. Donald &quot;Ducky&quot; Mallard">David McCallum</actor><actor role="Abby Sciuto">Pauley Perrette</actor><actor role="NCIS Special Agent Timothy McGee">Sean Murray</actor><actor role="Ziva David">Cote de Pablo</actor><actor role="NCIS Director Leon Vance">Rocky Carroll</actor><writer>Reed Steiner</writer><writer>Christopher J. Waild</writer><producer>Donald P. Bellisario</producer><guest>Dina Meyer</guest><guest>Adam Kaufman</guest><guest>Jason London</guest><guest>Taylor Cole</guest><guest>Jillian Bach</guest><guest>Bruce Nozick</guest><guest>Hayley Chase</guest><guest>Cliff Weissman</guest><guest>Kallee Brookes</guest><guest>Chris Sheffield</guest><guest>Jeff Wolfe</guest></credits>

  <date>2010-04-06</date>
  <category>series</category>
  <category>Crime drama</category>
  <category>Action</category>
  <category>Adventure</category>
  <category>Mystery</category>
  <category>Series</category>
  <category>Episode</category>
  <category>Show</category>
  <category>2000s</category>
  <category>Colleague</category>
  <category>Crime</category>
  <category>Government agents</category>
  <category>Headquarters</category>
  <category>Homicide</category>
  <category>Investigation</category>
  <category>Naval base</category>
  <category>Naval officer</category>
  <category>Navy</category>
  <category>Special agent</category>
  <category>Washington, D.C.</category>
  <length units="seconds">2640</length>
  <icon src="https://s3.amazonaws.com/schedulesdirect/assets/p11857845_b_h13_ab.jpg" width="480" height="270" />
  <episode-num system="dd_progid">EP01260840.0154</episode-num>
  <episode-num system="xmltv_ns"> 6 / 15 . 18 / 24 .  </episode-num>
  <episode-num system="thetvdb.com">series/72108</episode-num>
  <video></video>
  <audio>
   <stereo>stereo</stereo>
  </audio>
  <previously-shown start="20100406 +0000" />
  <rating system="VCHIP">
   <value>TV-14</value>
  </rating>
  <rating system="CHVRS">
   <value>14+</value>
  </rating>
  <rating system="ClassInd">
   <value>16</value>
  </rating>
  <rating system="FPB">
   <value>16</value>
  </rating>
  <rating system="MEKU">
   <value>K16</value>
  </rating>
  <rating system="RÃ©gie du cinÃ©ma">
   <value>13+</value>
  </rating>
 </programme>

HTH.

A L

Wow - thank you, Em Smith.

This development looks absolutely amazing.

Other than the fact that I was unable to specify which episodes to record, two things about my system remain problematic:

1. A news break, in the middle of a film, would often mean that I would only record the first half of it.

2. I like to watch recordings from "TV Series" in Kodi, rather than from the Recordings folder in the plugin. However, it is not uncommon for the Schedules Direct episode information to differ from that of the TVDB, so I either get informed that the wrong episode is there or (if the episode is not in the TVDB), I won't know it has been recorded at all.

Having just read your posts, your grabber seems to be absolutely perfect for me!

I am still pretty much a beginner, when it comes to Linux, so I still have to be spoon fed instructions, when it comes to these sorts of things.

I should have no problem in adding the new grabber and making it executable.

Em Smith wrote:

> # Backup your current .xmltv configuration from ~/.xmltv (since my version patches the DB to download artwork details);
> # Change the grabber to tv_grab_az_sdjson_sqlite (look for it in "path") in tvheadend's configuration->channel/epg->epg grabber modules, and add extra arguments of "--update-description-with-title" or "--update-description-with-all" (without the quotes) which has more description fixups such as adding year, age rating and stars. I'd also add "--artwork-max-width=720" to avoid slow artwork downloads so it becomes "--update-description-with-all --artwork-max-width=720 --merge-split=5";
> # ???

> # Profit.

Could you possibly clarify why I need to backup my current .xmltv configuration, please? Is this strictly necessary?

What are the next steps, ??? and Profit?

Sorry if I am being really thick with this, I still find programming with Linux a bit of a struggle :) .

Em Smith

Sorry, "???, Profit" is an old "joke" where if you give a list of things, the last two steps are always that. It comes from bad business plans where you only know the first two steps but you know you'll make a profit.

No need to backup your .xmltv config (actually the SchedulesDirect.DB inside the .xmltv directory), but it just allows you to revert to the "zz" grabber if you don't like the "az" grabber since the "az" makes some changes to the DB which won't be compatible if you later revert.

So (for others) you'd do something like:


cd ~/.xmltv &&                                      (or wherever the .xmltv is on your system)
cp -p SchedulesDirect.DB SchedulesDirect.DB.backup  (miss out the "-p" if your system doesn't have it).

dirname `which tv_grab_zz_sdjson_sqlite`     (Note these are backquotes, maybe it's left of your "1" key)
cd /usr/local/bin     (or wherever the "zz" grabber is)

sudo wget https://raw.githubusercontent.com/azlm8t/tv_grab_zz_sdjson_sqlite/master/tv_grab_az_sdjson_sqlite &&
sudo chmod 755 tv_grab_az_sdjson_sqlite

I think that's everything on the shell. And of course, update the settings inside tvheadend itself.

Then to revert from "az" to "zz":

cd ~/.xmltv && mv SchedulesDirect.DB.backup SchedulesDirect.DB

A L

Sorry - I did think it may well have been a joke, but then you never know with Linux commands :)

This is perfect - I am going to have a go at this, when I get in tonight.

Rather than blindly type stuff into the command line, I try and understand everything I do so I can try and remember/learn if for the future.

May I just check with you, what I think will be going on with the above, please?

1. [code] cd ~/.xmltv && [/code] -change directory to wherever your XMLTV folder is kept.

2. [code] cp -p SchedulesDirect.DB SchedulesDirect.DB.backup [/code] -make a copy of the current database in the same folder, but don't change datestamp.

3. [code] dirname `which tv_grab_zz_sdjson_sqlite` -no idea.

4. [code] cd /usr/local/bin -change directory to bin.

Is that broadly correct?

I have tried looking up dirname with which but cannot seem to find what this command does.

Em Smith

Yes, broadly correct.

The "-p" also preserves permissions (that you can read/write to the file) just in case you have a funny setup where your files are owned by one person but you have someone else run the grabber. Surprisingly common.

The "which" says "where is the program called tv_grab_zz_sdjson_sqlite". So it might print /usr/bin/tv_grab_zz_sdjson_sqlite, or /usr/local, or /storage/.kodi/somewhere. The dirname is just to print out the directory name just to make it easier to read so you know to where you need to change directory to store your executable.

Effectively you would normally just combine the two commands in to one if you knew the file existed:


cd $(dirname `which tv_grab_zz_sdjson_sqlite`)

The command to lookup commands is "man" such as "man dirname" or "man 1 dirname" (since commands are in section 1 of the manual just in case of a bad setup where you get "dirname" from the developer manual in section 3).

And if the "cd ~/.xmltv" fails then it could be that it's on some LibreElec or some place odd. So you could possibly use find to find the directory:

find /storage/.kodi -type d -name .xmltv -print

Oh, and you probably have to restart tvheadend since it only finds grabbers at startup.

Em Smith

Oh, and the "profit" is basically from South Park S2E17.

So a local example could be:

# Purchase failing DIY chain on the other side of the planet for a high price;
# Spend a fortune to refit stores to remove all high profit items and sell only low profit margin items for prices far higher than your competitors;
# ???
# Profit.

Allegedly.

So the "???" is where you imagine the board presenting their plan and you wonder what that stage could possibly be and why they couldn't see the plan would fail.

A L

Thanks very much for that - you really know your stuff. I am getting a Linux lesson at the same time! :)

I followed the prior instructions and I am not sure that it has worked for me - I must have forgotten to do something (or messed something up).

The screenshot shows my current EPG date for Casualty and you can see the grab log below - what do you think I am doing wrong?

Em Smith

Perhaps you forgot to update "extra arguments" on the epg grabber tab to contain "--update-description-with-all --artwork-max-width=720 --merge-split=5" (without the quotes)?

Mischa Mischock

this working on webgrabplus @em smith, and i have the old tvh server 4.2.6 has no PCRE on this..

the trick is, the title, season, episode stand in the description on first line, see Screenshot 1.jpg..

"^Mythbusters - Die Wissensjäger - S13" for only recording S13 but u must make fulltext on, see Screenshot 2.jpg

----------------

to Webgrabplus..

It must be a provider, that provides you with episode information..

on Webgrabplus.config, you should turn on "rex"<postprocess run="y" grab="y">rex</postprocess>

make a fresh rex.config file and add this lines to..

---
> <?xml version="1.0" encoding="utf-8"?>
>
> <settings>
>
> <filename>\path\to\yours\rex.guide.xml</filename>
>
> <title>'title'</title>
> <desc>'title'{ - 'episode'}{ - 'subtitle'.\n}'description'</desc>
> <star-rating></star-rating>
>
> </settings>

---

special thanks to the webgrabplus support, Blackbear199
<<

Em Smith

Fantastisch, thanks for sharing the webgrab+ config. I'm glad you got it working, and the webgrab config looks easy to expand.

Myguess is that "desc" should be:


<desc>'title'{ - 'episode'}{ - 'subtitle'}{.\n}'description'</desc>

(With .\n outside subtitle). So if your episode does not have a subtitle then you still have a newline before description. Just my guess though since I've never run it and perhaps all your programmes have a subtitle and the original version will always work.

A L

Em Smith wrote:

> Perhaps you forgot to update "extra arguments" on the epg grabber tab to contain "--update-description-with-all --artwork-max-width=720 --merge-split=5" (without the quotes)?

No - I updated that too, as per the attached.

How odd.

A L

Further quick update.

As per the attached, it seems that the new grabber is in fact working but I am only getting the updated EPG data for new programs added to the database (from 30/8/18 onwards).

Is there any way to remove all the programs from the database and grab from scratch?

Em Smith

That is very strange. I've never had that problem since all Tvheadend data gets overwritten if it is modified (at least in newer versions of Tvheadend). I can't think what would cause that.

Tvheadend stores a copy of its epg inside the tvheadend directory. I'm not sure where it is on your system, but you may be able to find it with this:


find /storage/.kodi ~/.hts -name epgdb.v2 -print

You'd have to shutdown tvheadend completely (not sure how you'd do that, maybe "service tvheadend stop" or "systemctl stop tvheadend" and then use "start" later to restart it), then remove the file ("rm epgdb.v2") or preferably move it out the way with "mv epgdb.v2 epgdb.v2.old". Then restart tvheadend and re-run the grabber.

For the grabber there is the "--force-download" option to force it to remove its data. It should only be used once otherwise it puts a strain on the SD servers, but you can add that to the "extra arguments".

One other thing, I noticed you don't have channel icons in the information panel. If you go in to Tvheadend epg grabber (where the schedule is) and tick the "update icon" button and save and re-run grabber. Then, if that doesn't work, then you may also have to go in to the configuration->channel/epg->epg grabber channels (may be called something different on your version since things were renamed), highlight the ones with XMLTV, hit edit, then update "channel update options" to include icons, press the tick box next to it then save and re-run grabber.

« Previous Page Next Page »