XMLTV, Kazer & French categories

Stephane Chauveau · 2013-11-27T15:47:07+00:00

SEE THE REPLY POSTS BELOW FOR AN UPDATED SCRIPT THAT CAN PROCESS ANY INPUT LANGUAGE. The following information are mostly intended for french users of www.k...

David jrm

The system is pretty new and as far as I know it doesn't have clock issues at all... what is clear is that pearl reinstallation fix the issue.. I have ran the script several times with different files and it works perfectly always :)

Will take note of the above if I see further issues, but from now it is up & working :)

Thx

Stephane Chauveau

For Nicolas,

The easiest way to log the errors is to redirect the error output stream (number 2) to a file.

For example, you can run the script as follow

/usr/local/bin/category-filter.pl 2> /tmp/category-filter.log

or if you want to APPEND to the log file, use a double >> instead

/usr/local/bin/category-filter.pl 2>> /tmp/category-filter.log

The alternative is to modify the perl script itself.

Add the following line at the beginning of the script to open the log file in append mode:

open(LOG, ">>", "/tmp/category-filter.log") or die "Can't open LOG file: $!";

Then clone the 'print STDERR' line and replace 'STDERR' by 'LOG':

print STDERR "Warning: Unmanaged category: '$a'\n" ;

print LOG "Warning: Unmanaged category: '$a'\n" ;

If you do not want to repeat the same error hundreds or thousands of times, you can memorize the wrong categories as follow to emit a single error for each.

At the beginning of the script, create an empty map:

my %BAD ;

Then modify the prints to STDERR and LOG as follow

if ( ! exists $BAD{$a} ) {
print STDERR "Warning: Unmanaged category: '$a'\n" ;
print LOG "Warning: Unmanaged category: '$a'\n" ;
# Record in BAD map so next error won't produce a message
$BAD{$a} = 1 ;

}

Stephane Chauveau

I made a syntax error in the open command. I edited the previous post but no email is sent so I repeat the change here

open(LOG, ">>", "/tmp/category-filter.log") or die "Can't open LOG file: $!";

Nicolas Rioja

Thank you very much Stephane.

I really appreciate your help.

Best regards.

Pablo R.

Could I use this script with multiple languages at once, for example English and French at the same time? And how?

Thanks. :)

Stephane Chauveau

I am not sure to understand what you mean by "multiple languages at once".

Do you mean that the XML file contains <category> tags with different lang values?

If so, it should be possible to modify the the script to take the lang value into account. The attached file is a quick attempt.

The idea with those modifications is that the myfilter function now receives the language (as defined by the lang attribute in the <category> tag) and the value. The entries in %REPLACE should now be prefixed

with the "language:"

Remark: Be aware that $PRE now contains 2 hidden levels of level of "( )" so in the $line regex, $1 is $PRE, $3 is the lang value, $4 is the category name and $5 is the $POST value.

I also modified the myfilter function to ignore missing entries for the 'en' language since most of them are likely to remain identical. If you prefer to have an error message for all missing entries, comment those 2 lines and all known English categories in %REPLACE.

I also assumed that the lang attribute may be omitted for 'en' but I do not have any examples so that may not work as expected.

Stephane Chauveau

the attached version does not add any new features but should be able to process multiple <category> tags per line.

This is currently not needed but one day, Kazer way very well decide to reduce the size of their XML by removing unnecessary line breaks.

Juan García

If anyone are interested, I create a bash script to process the categories caught from the plus.es site for the Astra 19.2 and Movistar+.

This is util to have the categories in tvheadend.

#!/bin/bash
MOVIE=$"Movie/Drama"
THRILLER=$"Detective/Thriller"
ADVENTURE=$"Adventure/Western/War"
SF=$"Sciencefiction/Fantasy/Horror"
COMEDY=$"Comedy"
SOAP=$"Soap/Melodrama/Folkloric"
ROMANCE=$"Romance"
HISTORICAL=$"Serious/Classical/Religious/Historicalmovie/Drama"

XXX=$"Adultmovie/Drama"

NEWS=$"News/Currentaffairs"
WEATHER=$"News/Weatherreport"
NEWS_MAGAZINE=$"Newsmagazine"
DOCUMENTARY=$"Documentary"
DEBATE=$"Discussion/Interview/Debate"

INTERVIEW=$DEBATE;

SHOW=$"Show/Gameshow"
GAME=$"Gameshow/Quiz/Contest"
VARIETY=$"Varietyshow"

TALKSHOW=$"Talkshow"

SPORT=$"Sports"
SPORT_SPECIAL=$"Specialevents(OlympicGames;WorldCup;etc.)"
SPORT_MAGAZINE=$"Sportsmagazines"
FOOTBALL=$"Football/Soccer"
TENNIS=$"Tennis/Squash"
SPORT_TEAM=$"Teamsports(excludingfootball)"
ATHLETICS=$"Athletics"
SPORT_MOTOR=$"Motorsport"

SPORT_WATER=$"Watersport"

KIDS=$"Children's/Youthprogrammes"
KIDS_0_5=$"Pre-schoolchildren'sprogrammes"
KIDS_6_14=$"Entertainmentprogrammesfor6to14"
KIDS_10_16=$"Entertainmentprogrammesfor10to16"
EDUCATIONAL=$"Informational/Educational/Schoolprogrammes"

CARTOON=$"Cartoons/Puppets"

MUSIC=$"Music/Ballet/Dance"
ROCK_POP=$"Rock/Pop"
CLASSICAL=$"Seriousmusic/Classicalmusic"
FOLK=$"Folk/Traditionalmusic"
JAZZ=$"Jazz"

OPERA=$"Musical/Opera"

CULTURE=$"Arts/Culture(withoutmusic)"
PERFORMING=$"Performingarts"
FINE_ARTS=$"Finearts"
RELIGION=$"Religion"
POPULAR_ART=$"Popularculture/Traditionalarts"
LITERATURE=$"Literature"
FILM=$"Film/Cinema"
EXPERIMENTAL_FILM=$"Experimentalfilm/Video"

BROADCASTING=$"Broadcasting/Press"

SOCIAL=$"Social/Politicalissues/Economics"
MAGAZINE=$"Magazines/Reports/Documentary"
ECONOMIC=$"Economics/Socialadvisory"

VIP=$"Remarkablepeople"

SCIENCE=$"Education/Science/Factualtopics"
NATURE=$"Nature/Animals/Environment"
TECHNOLOGY=$"Technology/Naturalsciences"
DIOLOGY=$TECHNOLOGY
MEDECINE=$"Medicine/Physiology/Psychology"
FOREIGN=$"Foreigncountries/Expeditions"
SPIRITUAL=$"Social/Spiritualsciences"
FURTHER_EDUCATION=$"Furthereducation"

LANGUAGES=$"Languages"

HOBBIES=$"Leisurehobbies"
TRAVEL=$"Tourism/Travel"
HANDICRAF=$"Handicraft"
MOTORING=$"Motoring"
FITNESS=$"Fitnessandhealth"
COOKING=$"Cooking"
SHOPPING=$"Advertisement/Shopping"

GARDENING=$"Gardening"

declare -A ARRAY
ARRAY["Actualidad"]=$MAGAZINE
ARRAY["Automovilismo"]=$SPORT_MOTOR
ARRAY["Ciencia ficción"]=$SF
ARRAY["Ciencia y tecnología"]=$TECHNOLOGY
ARRAY["Cine"]=$MOVIE
ARRAY["Cocina"]=$COOKING
ARRAY["Concursos"]=$GAME
ARRAY["Corazón"]=$VIP
ARRAY["Cortometraje"]=$EXPERIMENTAL_FILM
ARRAY["Cultura"]=$CULTURE
ARRAY["Cultural/Educativo"]=$EDUCATIONAL
ARRAY["Curso educativo"]=$FURTHER_EDUCATION
ARRAY["Debates"]=$DEBATE
ARRAY["Deportes"]=$SPORT
ARRAY["Divulgativo"]=$MAGAZINE
ARRAY["Docu-Drama"]=$DOCUMENTARY
ARRAY["Documental"]=$DOCUMENTARY
ARRAY["Documental Actualidad"]=$DOCUMENTARY
ARRAY["Documental Antropológico"]=$DOCUMENTARY
ARRAY["Documental Arte y Espectáculos"]=$FINE_ARTS
ARRAY["Documental Ciencia y tecnología"]=$SCIENCE
ARRAY["Documentales"]=$DOCUMENTARY
ARRAY["Documental Historia"]=$DOCUMENTARY
ARRAY["Documental Naturaleza y Animales"]=$NATURE
ARRAY["Documental Viaje"]=$FOREIGN
ARRAY["Drama"]=$SOAP
ARRAY["El Tiempo"]=$WEATHER
ARRAY["Entretenimiento"]=$SHOW
ARRAY["Entrevistas"]=$INTERVIEW
ARRAY["Humor"]=$SHOW
ARRAY["Infantil"]=$KIDS
ARRAY["Información"]=$NEWS
ARRAY["Literatura"]=$LITERATURE
ARRAY["Magacín"]=$MAGAZINE
ARRAY["Moda"]=$MAGAZINE
ARRAY["Música"]=$MUSIC
ARRAY["Noticias"]=$NEWS
ARRAY["Ocio y Aficiones"]=$HOBBIES
ARRAY["Otros"]=$VARIETY
ARRAY["Pesca"]=$SPORT_WATER
ARRAY["Policíacas"]=$THRILLER
ARRAY["Política"]=$THRILLER
ARRAY["RealityShow"]=$SHOW
ARRAY["Religiosos"]=$RELIGION
ARRAY["Reportaje Actualidad"]=$SOCIAL
ARRAY["Reportajes"]=$SOCIAL
ARRAY["Saludybelleza"]=$MEDECINE
ARRAY["Series"]=$FILM
#ARRAY["Sinclasificar"]=$
ARRAY["Telefilme"]=$FILM
ARRAY["Telenovela"]=$FILM
ARRAY["Televisión"]=$VARIETY
ARRAY["Toros"]=$SPORT

ARRAY["Viajes"]=$TRAVEL

#echo "${ARRAY["Viajes"]}"

REGEX_PRE=$'<category[^>]*>'

REGEX_POST=$'</category>'

for K in "${!ARRAY[@]}"; do
#echo $K --- ${ARRAY[$K]}
REGEX=$REGEX_PRE$K$REGEX_POST
SUSTI='<category lang="en">'${ARRAY[$K]}'</category>'
#echo $REGEX
#echo $SUSTI
sed -r -i "s|$REGEX|$SUSTI|" $1
done

Renato Moscardini

I have an issue with Kazer grabber after the upgrade to synology DSM 6.0
1) unzip command does no more exist in DSM 6. I had to use 7z
2) After this script modification all seems ok. When I launch the grabber (script) manually there is no apparant issue. I receive the xml content to output as expected.
3) The grabber in TvHeadEnd configuration template is green. It seems to be correctly found by tvheadend.

4) Sadly there is no EPG update, why ? How can I debug this.

Note : In Synology DSM 6.0 the grabber (script) is replicated in /bin even if the copy is on /usr/bin. So we find twice the grabber in configuration Template, but only one is activated /usr/bin as it was on DSM 5.0

For info I strictly used this tutorial to install the grabber and it worked just fine till the upgrade to DSM 6.0

http://www.inrepublica.fr/blog/2013/06/11/un-vrai-guide-des-programmes-avec-tvheadend-tnt-fr/

Could someone help ?

thierry castelot

Hi Renato,

i had problems with kazer and dsm 6 too, After numerous searches and trial, I modified a script which works with the categories. it works with http://xmltv.dtdns.net/alacarte/. just create a account and pick your channels.

modify your account name at line 15 and put this script into usr/bin and restart tvheadend.

Renato Moscardini

cool Thierry,
Many thanks for your answer.
I will implement your solution and see.

Renato Moscardini

Yes Thierry, -> it works just as expected.
Many thanks for the answer.

thierry castelot

Your welcome :)

i'm currently working to make categories more accurate, i'll post result when it will be finished.

thierry castelot

this one works with Kazer, replace xxxxxxxxx by your user hash.

Renato Moscardini

Nice to know.
It works with xml file but not with the zip one, strange.
Anyway many thanks for your efforts.
For the time being I will stay to alacare and see the reliability in the long run.

thierry castelot

As promised, this one works with the subcategories, like "comédie" "thriller", etc..instead of "movie/drama".

I also corrected the grab so that it appears as tv_grab_alacarte.
<<

james Bond

many thanks for this : better guide that Kazer and categories :)

However I have some questions :

I have placed the script in /usr/bin
chmod 755

restarted Tvheadend

When Tvheadend restarts I can clearly see the script working (at least during 10 mins) using "htop".

But when I go to "configuration/EPG Grabber Modules" inside html interface I don't see tv_grab_alacarte :(

Do you have any idea?

Renato Moscardini

1) Have you subscribed to alacarte ?
2) Have you changed in the script the YORNAME.xml by your login name of alacarte acount (case sensitive).

3) In Tvheadend Under config/EPG modules, is the internal grabber "Alacarte" activated and in green

If all this is ok, I do not see
Try to restart all the server/NAS, not only tvheadend. It seems restarting only tvheadend is not enough, at least on my NAS.

james Bond

1) Have you subscribed to alacarte ?

yes

2) Have you changed in the script the YORNAME.xml by your login name of alacarte acount (case sensitive).

yes

3) In Tvheadend Under config/EPG modules, is the internal grabber "Alacarte" activated and in green

That is the problem : I don't see "Alacarte"

I did already tried to restart Tvheadend, reboot : same result.
The script runs, I can see /tmp/grab.txt is generated
But no "Alacarte" EPG module inside the interface.

Renato Moscardini

@James,
In fact I discovered that I used the previous version of the script.

The last one seems to have an issue. It hangs somewhere and display some "sed" errors.

So in the wait time, use the tv_grab_kazer frome the Thierry previous message and rename it to tv_grab_alacarte.

« Previous Page Next Page »