Project

General

Profile

Sky Auto Channel Numbering

Added by Jonathan Kempson almost 9 years ago

So I may have bitten off far more than I can chew here but...

I have been experimenting with the idea of region-aware automatic channel numbering for Sky UK that matches the official channel number layout. I know that you can get TVH to get 95% of the way there but I find that I'm having to more often than not deal with new/old channels being added/removed (and I also fancied the challenge!).

So far I have written a Perl scraper to pull all the data from this URL into a MySQL db:

http://en.kingofsat.net/pack-skydigital.php

I also have another script which pulls all the data from the Sky EPG json backend into a separate table in my db:

http://tv.sky.com/channel/index

This URL can be appended with a specific region code to return a bespoke list of channels, there might need to be some manual intervention here but it should be able to figure out the vast majority of channels:

4101-19"Atherstone HD" 
4097-19"Atherstone SD" 
4101-12"Border England HD" 
4097-12"Border England SD" 
4102-36"Border Scotland HD" 
4098-36"Border Scotland SD" 
4103-65"Brighton HD" 
4099-65"Brighton SD" 
4101-3"Central Midlands HD" 
4097-3"Central Midlands SD" 
4104-34"Channel Isles HD" 
4100-34"Channel Isles SD" 
4101-20"East Midlands HD" 
4097-20"East Midlands SD" 
4101-2"Essex HD" 
4097-2"Essex SD" 
4101-24"Gloucester HD" 
4097-24"Gloucester SD" 
4102-35"Grampian HD" 
4098-35"Grampian SD" 
4101-7"Granada HD" 
4097-7"Granada SD" 
4103-70"Henley On Thames HD" 
4099-70"Henley On Thames SD" 
4103-43"HTV Wales HD" 
4099-43"HTV Wales SD" 
4101-4"HTV West HD" 
4097-4"HTV West SD" 
4103-63"HTV West / Thames Valley HD" 
4099-63"HTV West / Thames Valley SD" 
4101-29"Humber HD" 
4097-29"Humber SD" 
4101-1"London HD" 
4097-1"London SD" 
4101-18"London / Essex HD" 
4097-18"London / Essex SD" 
4103-66"London / Thames Valley HD" 
4099-66"London / Thames Valley SD" 
4103-64"London Kent HD" 
4099-64"London Kent SD" 
4101-11"Meridian East HD" 
4097-11"Meridian East SD" 
4103-68"Meridian North HD" 
4099-68"Meridian North SD" 
4101-5"Meridian South HD" 
4097-5"Meridian South SD" 
4101-10"Meridian South East HD" 
4097-10"Meridian South East SD" 
4103-45"Merseyside HD" 
4099-45"Merseyside SD" 
4101-21"Norfolk HD" 
4097-21"Norfolk SD" 
4103-62"North East Midlands HD" 
4099-62"North East Midlands SD" 
4101-8"North West Yorkshire HD" 
4097-8"North West Yorkshire SD" 
4101-26"North Yorkshire HD" 
4097-26"North Yorkshire SD" 
4104-33"Northern Ireland HD" 
4100-33"Northern Ireland SD" 
4103-71"Oxford HD" 
4099-71"Oxford SD" 
4104-50"Republic of Ireland HD" 
4100-50"Republic of Ireland SD" 
4103-41"Ridge Hill HD" 
4099-41"Ridge Hill SD" 
4103-61"Scarborough HD" 
4099-61"Scarborough SD" 
4102-37"Scottish East HD" 
4098-37"Scottish East SD" 
4102-38"Scottish West HD" 
4098-38"Scottish West SD" 
4103-60"Sheffield HD" 
4099-60"Sheffield SD" 
4101-28"South Lakeland HD" 
4097-28"South Lakeland SD" 
4103-72"South Yorkshire HD" 
4099-72"South Yorkshire SD" 
4103-69"Tees HD" 
4099-69"Tees SD" 
4101-9"Thames Valley HD" 
4097-9"Thames Valley SD" 
4101-27"Tring HD" 
4097-27"Tring SD" 
4101-13"Tyne HD" 
4097-13"Tyne SD" 
4101-25"West Anglia HD" 
4097-25"West Anglia SD" 
4103-67"West Dorset HD" 
4099-67"West Dorset SD" 
4101-6"Westcountry HD" 
4097-6"Westcountry SD" 

e.g. http://tv.sky.com/channel/index/4101-24

This is where I currently am.

So I'm thinking my next step should be to do some fuzzy string matching on the channel names and marry up the channels to each data source.

Once this was done I could lookup any channel from from the Sky EPG json feed and know the associated Service ID (from the kingofsat scrape). For example, I would know channel 101 (in my region) would be BBC One Scotland HD and also know the Service ID to go with it.

The final stag (and this is the one I'm totally stumped on) would be how I then create a custom channel list for TVH. I've been delving around in the channel, channels, dvbmuxes, dvbtransport etc folders but I'm getting a bit overwhelmed with the easiest way to achieve what I want.


Replies (142)

RE: Sky Auto Channel Numbering - Added by Geoff Geoff over 8 years ago

its probably something i havent done

here my config as well - and done what was suggested in the posts above

use URI::Escape;
use List::Util qw(first);

my $data = "4097";
my $region = "10";

my $tag_channels = 1;
my $tvh_user = "geoff";
my $tvh_pass = "*****";
my $tvh_ip = "localhost";
my $tvh_proto = "http";
my $tvh_port = "9981";
my $tvh_tag = "AutoChan";
my $icons_enabled = 1;

RE: Sky Auto Channel Numbering - Added by Geoff Geoff over 8 years ago

tried everything... no luck :( spend a few hours on it... everything else is working ok on it - so don't really want to start again.

perl ./autobouquets.pl
malformed JSON string, neither array, object, number, string or atom, at character offset 0 (before "(end of string)") at /usr/share/perl5/JSON.pm line 171.

any other ideas or advise?

RE: Sky Auto Channel Numbering - Added by Jonathan Kempson over 8 years ago

Hey Geoff,

It sounds like something is broken when trying to access the tvheadend API. What happens if you go to the following URL?

http://USERNAME:[email protected]_IP:9981/api/channeltag/list?enum=true

Obviously replace USERNAME, PASSWORD and TVH_IP with relevant values.

Also, do you see this error immediately after running the script or is there a short pause?

RE: Sky Auto Channel Numbering - Added by Geoff Geoff over 8 years ago

i download a file straight away, and in that file is

{"entries": [{"key": "3985a1ac8fc061c8674efa654151b443","val": "TV channels"},{"key": "b3a195dc16259139ccc597461b7cd3af","val": "AutoChan"},{"key": "5cc8791947b2f21e06dc703cb7322b81","val": "SDTV"},{"key": "1794f9f4f6525b39eef79e861b0db5ac","val": "HDTV"},{"key": "a832692d815afc9aac55aab9d77d4f54","val": "ARQIVA"},{"key": "a75bb843ea884768d69671398e21e45c","val": "Radio"},{"key": "bade3384dafb31fb6ef20d7490426dd1","val": "XBMC"}]}

AutoChan - was one i manually created to help the script along a bit.

RE: Sky Auto Channel Numbering - Added by Jonathan Kempson over 8 years ago

Thanks Geoff, that rules out quite a bit at least!

I've updated the script to give a bit more verbose output. If you can re-download the script and let me know what gets output?

RE: Sky Auto Channel Numbering - Added by Geoff Geoff over 8 years ago

thank you - will download now.

RE: Sky Auto Channel Numbering - Added by Geoff Geoff over 8 years ago

[email protected]:/home/hts/.hts/tvheadend# perl ./autobouquets.pl
malformed JSON string, neither array, object, number, string or atom, at character offset 0 (before "(end of string)") at /usr/share/perl5/JSON.pm line 171.

RE: Sky Auto Channel Numbering - Added by Jonathan Kempson about 8 years ago

Sorry for the late response Geoff. Can you run
sudo dvbsnoop -nph -n 500 0x11
for me?
Does it output anything? No need to paste here if it does, if its not working you won't see anything.

RE: Sky Auto Channel Numbering - Added by Geoff Geoff about 8 years ago

Hi no need to apologise understand its a hobby - yes reports pages of information.

RE: Sky Auto Channel Numbering - Added by Mark Clarkstone about 8 years ago

The script no longer works with newer versions of Tvheadend btw. :(

RE: Sky Auto Channel Numbering - Added by Geoff Geoff about 8 years ago

thats a shame it was one of the best scripts!

RE: Sky Auto Channel Numbering - Added by Jonathan Kempson about 8 years ago

Hi guys,

I wanted to develop things a bit further but in light of it not working at all with the latest version of TVH I'll upload the latest version of my script "as is".

It now has the feature to be given an SD bouquet and an associated HD bouquet. Each channel in the HD bouquet will be tested and only overwrite an SD channel if it can be accessed.

To do this set $data_sd and $region_sd to your SD bouquet values and $data_hd and $region_hd to the corresponding HD bouquet.

E.g. I'm in East Scotland so my settings are:

my $data_sd   = "4098";
my $region_sd = "37";

my $data_hd   = "4102";
my $region_hd = "37";

Very experimental and very poorly implemented at the moment so if you don't understand my poor description of this feature, aren't interested or it doesn't work (very likely!) just leave $data_hd and $region_hd blank and set $data_sd and $region_sd to whatever you need.

Let me know how things go.

PS Geoff, sorry for not getting back to you. Maybe give this version a go? Let me know the output.

RE: Sky Auto Channel Numbering - Added by Geoff Geoff about 8 years ago

thanks will try tomorrow. cheers

RE: Sky Auto Channel Numbering - Added by Geoff Geoff about 8 years ago

[email protected]:~/.hts/tvheadend$ perl ./autobouquets.pl
Global symbol "@sid_adds" requires explicit package name at ./autobouquets.pl line 133.
Execution of ./autobouquets.pl aborted due to compilation errors.

RE: Sky Auto Channel Numbering - Added by Jonathan Kempson about 8 years ago

Try again Geoff (re-download script) :)

RE: Sky Auto Channel Numbering - Added by Geoff Geoff about 8 years ago

sorry...

[email protected]:~/.hts/tvheadend$ perl ./autobouquets.pl
Started
Get Channel Tags...
malformed JSON string, neither array, object, number, string or atom, at character offset 0 (before "(end of string)") at /usr/share/perl5/JSON.pm line 171.

RE: Sky Auto Channel Numbering - Added by Jonathan Kempson about 8 years ago

and again? obvs remove password if sensitive!

RE: Sky Auto Channel Numbering - Added by Geoff Geoff about 8 years ago

Started
http://geoff:[email protected]:9981
Get Channel Tags...
malformed JSON string, neither array, object, number, string or atom, at character offset 0 (before "(end of string)") at /usr/share/perl5/JSON.pm line 171.

RE: Sky Auto Channel Numbering - Added by Jonathan Kempson about 8 years ago

and does that URL work if you paste it in a browser?

RE: Sky Auto Channel Numbering - Added by Jonathan Kempson about 8 years ago

Have you tried using the actual IP address of your TVH machine e.g. 192.168.1.2

RE: Sky Auto Channel Numbering - Added by Geoff Geoff about 8 years ago

yes

[email protected]:~/.hts/tvheadend$ perl ./autobouquets.pl
Started
http://geoff:[email protected]:9981
Get Channel Tags...
malformed JSON string, neither array, object, number, string or atom, at character offset 0 (before "(end of string)") at /usr/share/perl5/JSON.pm line 171.

and using url on firefox works. see attached however... there seems a 2nd login now with latest and click on the login at top says you are about to login using the username geoff click ok and logs in.

RE: Sky Auto Channel Numbering - Added by Jonathan Kempson about 8 years ago

So it looks like an authentication issue with TVH. I've never seen the "no verified access" message before. Mine always says "Logged in as jon"..

My script fails at a very early point where it tries to speak to TVH. The error you're getting happens to me when it can't access TVH, usually due to bad config although I'm sure you've checked all that!

Just for testing, could you try temporarily changing your password to something really easy with no special characters e.g. abc123, then try again.

RE: Sky Auto Channel Numbering - Added by Duncan Henry about 8 years ago

I'm having a similar problem. I get the following output.

% perl ~/bin/autobouquets.pl
Started
http://auto:[email protected]:9981
Get Channel Tags...
malformed JSON string, neither array, object, number, string or atom, at character offset 0 (before "(end of string)") at /home/aitch/bin/autobouquets.pl line 317.

RE: Sky Auto Channel Numbering - Added by Jason Bell about 8 years ago

Could anyone tell me where I can find the region numbers from
To enter into the perl script
Many thanks

RE: Sky Auto Channel Numbering - Added by Rob Rob about 8 years ago

....uhhhhh the OP...maybe...!!!?

(76-100/142)