Project

General

Profile

TVH 4.3 Help with tvhmeta scraping fails, log errors.

Added by James Roberts about 1 year ago

Currently running TVH 4.3-1789 on Ubuntu 18.04

Trying to get scraping to work for recordings. I have checked "fetch artwork" in recording profile and added my API keys to additional command line options (--tmdb-key xxxxxxxxxxxxxx --tvdb-key xxxxxxxxxxxxxx)

However, the scraper fails apparently with "401 unauthorized" (see below)

Am I not specifying the API keys properly or...? Maybe missing some python modules?

Apr 24 08:32:01 jaaxx-server tvheadend[945]: spawn: 2019-04-24 08:32:01,635:INFO:tvhmeta:149:Sending grid=1&list=uuid%2Cimage%2Cfanart_image%2Ctitle%2Ccopyright_year%2Cepisode_disp%2Curi&uuid=bbe8fe0b5f9130f3188ed3e56f587f35 to http://localhost:9981/api/idnode/load
Apr 24 08:32:01 jaaxx-server tvheadend[945]: spawn: Traceback (most recent call last):
Apr 24 08:32:01 jaaxx-server tvheadend[945]: spawn: File "/usr/bin/tvhmeta", line 457, in <module>
Apr 24 08:32:01 jaaxx-server tvheadend[945]: spawn: process(sys.argv[1:])
Apr 24 08:32:01 jaaxx-server tvheadend[945]: spawn: File "/usr/bin/tvhmeta", line 452, in process
Apr 24 08:32:01 jaaxx-server tvheadend[945]: spawn: tvhmeta.fetch_and_persist_artwork(opts.uuid, opts.force_refresh, opts.modules_movie, opts.modules_tv)
Apr 24 08:32:01 jaaxx-server tvheadend[945]: spawn: File "/usr/bin/tvhmeta", line 224, in fetch_and_persist_artwork
Apr 24 08:32:01 jaaxx-server tvheadend[945]: spawn: recjson = self.request("api/idnode/load", data)
Apr 24 08:32:01 jaaxx-server tvheadend[945]: spawn: File "/usr/bin/tvhmeta", line 150, in request
Apr 24 08:32:01 jaaxx-server tvheadend[945]: spawn: req = urlopen(full_url, data=bytearray(data, 'utf-8'))
Apr 24 08:32:01 jaaxx-server tvheadend[945]: spawn: File "/usr/lib/python2.7/urllib.py", line 89, in urlopen
Apr 24 08:32:01 jaaxx-server tvheadend[945]: spawn: return opener.open(url, data)
Apr 24 08:32:01 jaaxx-server tvheadend[945]: spawn: File "/usr/lib/python2.7/urllib.py", line 215, in open
Apr 24 08:32:01 jaaxx-server tvheadend[945]: spawn: return getattr(self, name)(url, data)
Apr 24 08:32:01 jaaxx-server tvheadend[945]: spawn: File "/usr/lib/python2.7/urllib.py", line 366, in open_http
Apr 24 08:32:01 jaaxx-server tvheadend[945]: spawn: return self.http_error(url, fp, errcode, errmsg, headers, data)
Apr 24 08:32:01 jaaxx-server tvheadend[945]: spawn: File "/usr/lib/python2.7/urllib.py", line 379, in http_error
Apr 24 08:32:01 jaaxx-server tvheadend[945]: spawn: result = method(url, fp, errcode, errmsg, headers, data)
Apr 24 08:32:01 jaaxx-server tvheadend[945]: spawn: File "/usr/lib/python2.7/urllib.py", line 701, in http_error_401
Apr 24 08:32:01 jaaxx-server tvheadend[945]: spawn: errcode, errmsg, headers)
Apr 24 08:32:01 jaaxx-server tvheadend[945]: spawn: File "/usr/lib/python2.7/urllib.py", line 386, in http_error_default
Apr 24 08:32:01 jaaxx-server tvheadend[945]: spawn: raise IOError, ('http error', errcode, errmsg, headers)
Apr 24 08:32:01 jaaxx-server tvheadend[945]: spawn: IOError: ('http error', 401, 'Unauthorized', <httplib.HTTPMessage instance at 0x7f2d897e8ef0>)


Replies (5)

RE: TVH 4.3 Help with tvhmeta scraping fails, log errors. - Added by Em Smith about 1 year ago

Unfortunately I can't see from the log what is wrong. I'd suspect either keys not being valid since it seems to work for me, or perhaps your Tvheadend setup is password protected and you need to pass a "--user" and "--password" option to gain access?

If neither of those solve the problem then can you add "--debug" to your options?

Do you know where the files "tv_meta_tvdb.py" and "tv_meta_tmdb.py" are installed (they vary on different systems)?

If so, you could possibly run it manually to also verify what is happening using something similar to:

tv_meta_tvdb.py --tvdb-key abcdef --title "Fawlty Towers" --debug

and
tv_meta_tmdb.py --tmdb-key abcdef --title "Fawlty Towers" --debug

You're basically looking for a final line which says "fanart" and "poster" and has a URL. That would verify that your keys are valid and it's all installed correctly.

RE: TVH 4.3 Help with tvhmeta scraping fails, log errors. - Added by James Roberts about 1 year ago

Both commands work fine from the command line and yes, I do have a user and password set for tvheadend.

So, how do I pass user and password? Can I do it from recording config the same way I am sending the api keys? Or can I create a * user with the appropriate rights for the query? What rights?

RE: TVH 4.3 Help with tvhmeta scraping fails, log errors. - Added by Sammy B about 2 months ago

I'm getting the same errors - did you ever find the solution? thanks
I can also run tv_meta_tmdb.py and tv_meta_tvdb.py by themselves successfully.

2020-04-07 11:07:56.712 spawn: 2020-04-07 11:07:56,712:INFO:tvhmeta:452:Got movie modules: [tv_meta_tmdb] tv modules [tv_meta_tmdb,tv_meta_tvdb]
2020-04-07 11:07:56.713 spawn: 2020-04-07 11:07:56,713:INFO:tvhmeta:149:Sending grid=1&list=uuid%2Cimage%2Cfanart_image%2Ctitle%2Ccopyright_year%2Cepisode_disp%2Curi&uuid=94cca03d38740f1dee9640b96bf9cba4 to http://localhost:9981/api/idnode/load
2020-04-07 11:07:56.717 spawn: Traceback (most recent call last):
2020-04-07 11:07:56.717 spawn:   File "/usr/bin/tvhmeta", line 458, in 
2020-04-07 11:07:56.717 spawn:     process(sys.argv[1:])
2020-04-07 11:07:56.717 spawn:   File "/usr/bin/tvhmeta", line 453, in process
2020-04-07 11:07:56.717 spawn:     tvhmeta.fetch_and_persist_artwork(opts.uuid, opts.force_refresh, opts.modules_movie, opts.modules_tv)
2020-04-07 11:07:56.717 spawn:   File "/usr/bin/tvhmeta", line 225, in fetch_and_persist_artwork
2020-04-07 11:07:56.717 spawn:     recjson = self.request("api/idnode/load", data)
2020-04-07 11:07:56.717 spawn:   File "/usr/bin/tvhmeta", line 150, in request
2020-04-07 11:07:56.717 spawn:     req = urlopen(full_url, data=bytearray(data, 'utf-8'))
2020-04-07 11:07:56.717 spawn:   File "/usr/lib/python2.7/urllib.py", line 89, in urlopen
2020-04-07 11:07:56.717 spawn:     return opener.open(url, data)
2020-04-07 11:07:56.717 spawn:   File "/usr/lib/python2.7/urllib.py", line 217, in open
2020-04-07 11:07:56.717 spawn:     return getattr(self, name)(url, data)
2020-04-07 11:07:56.718 spawn:   File "/usr/lib/python2.7/urllib.py", line 368, in open_http
2020-04-07 11:07:56.718 spawn:     return self.http_error(url, fp, errcode, errmsg, headers, data)
2020-04-07 11:07:56.718 spawn:   File "/usr/lib/python2.7/urllib.py", line 381, in http_error
2020-04-07 11:07:56.718 spawn:     result = method(url, fp, errcode, errmsg, headers, data)
2020-04-07 11:07:56.718 spawn:   File "/usr/lib/python2.7/urllib.py", line 703, in http_error_401
2020-04-07 11:07:56.718 spawn:     errcode, errmsg, headers)
2020-04-07 11:07:56.718 spawn:   File "/usr/lib/python2.7/urllib.py", line 388, in http_error_default
2020-04-07 11:07:56.718 spawn:     raise IOError, ('http error', errcode, errmsg, headers)
2020-04-07 11:07:56.718 spawn: IOError: ('http error', 401, 'Unauthorized', )

RE: TVH 4.3 Help with tvhmeta scraping fails, log errors. - Added by Em Smith about 2 months ago

I didn't see the reply from James, but it looks like Sammy has the same problem.

So, yes, you need to pass '--user abc --password def" in the same place in Tvheadend configuration; or re-add the default admin user (named "*") without a password. This is because the script needs to load and save details for the recordings so needs to be able to login to the Tvheadend server.

After you change the configuration in Tvheadend, then in the logs you should see your "http://localhost:9981/api/idnode/load" line change to "http://[email protected]:localhost:9981/api/idnode/load". You can probably try using "curl" or "wget" on that url to test it out quickly until you no longer get a 401 error, then Tvheadend should just work for new recordings.

Starting a new recording should then do the lookups successfully.

To test from a command line, the "tvhmeta" script is the one that Tvheadend uses that (effecively) calls tv_meta_tmdb.py. So, once you have the user and password configured you should be able to run that from the command line to test it out, rather than waiting for a new recording.

So, from the first example where you had logged "grid=1&list=uuid%2Cimage%2Cfanart_image%2Ctitle%2Ccopyright_year%2Cepisode_disp%2Curi&uuid=bbe8fe0b5f9130f3188ed3e56f587f35 to http://localhost:9981/api/idnode/load", you would extract the bit after "uuid=" and use that as an argument to the script as "--uuid=bbe8fe0b5f9130f3188ed3e56f587f35"

So, in this case you would run "tvhmeta --uuid=be8fe0b5f9130f3188ed3e56f587f35 --user=abc --password=def --tmdb-key=ghi --tvdb-key=jkl --debug", and it should all work.

Of course,, the uuid you use should be from a recent recording since the be8fe0b5f9130f3188ed3e56f587f35 recording would probably have expired by now.

RE: TVH 4.3 Help with tvhmeta scraping fails, log errors. - Added by Sammy B about 2 months ago

Thanks for the reply
I tried curl http://username:[email protected]:9981/api/idnode/load with my login details and I just kept getting 401.
I then tried adding a user with name * without a password and that got it working.

Thanks for the help

    (1-5/5)