I'm running tvheadend in a docker container, using the docker-compose template from https://github.com/linuxserver/docker-tvheadend. I'd been ignoring failing tvhmeta runs for a while & eventually decided to take a look.
What I found was that when a username and password are supplied, urlopen() can't grok the URL that tvhmeta constructs for the tvh api - eg http://user:pass@hostname:portnum/url, and instead treats 'user:pass@hostname' as the whole hostname. I'm certain this is the case as I saw exactly this kind of failing lookup requests in my DNS server logs. Here's what the error looks like in the tvheadend logs:
2023-01-06 15:14:17.989 spawn: Traceback (most recent call last):
2023-01-06 15:14:17.989 spawn: File "/usr/lib/python3.9/urllib/request.py", line 1346, in do_open
2023-01-06 15:14:17.990 spawn: h.request(req.get_method(), req.selector, req.data, headers,
2023-01-06 15:14:17.990 spawn: File "/usr/lib/python3.9/http/client.py", line 1285, in request
2023-01-06 15:14:17.991 spawn: self._send_request(method, url, body, headers, encode_chunked)
2023-01-06 15:14:17.991 spawn: File "/usr/lib/python3.9/http/client.py", line 1331, in _send_request
2023-01-06 15:14:17.991 spawn: self.endheaders(body, encode_chunked=encode_chunked)
2023-01-06 15:14:17.991 spawn: File "/usr/lib/python3.9/http/client.py", line 1280, in endheaders
2023-01-06 15:14:17.992 spawn: self._send_output(message_body, encode_chunked=encode_chunked)
2023-01-06 15:14:17.992 spawn: File "/usr/lib/python3.9/http/client.py", line 1040, in _send_output
2023-01-06 15:14:17.992 spawn: self.send(msg)
2023-01-06 15:14:17.992 spawn: File "/usr/lib/python3.9/http/client.py", line 980, in send
2023-01-06 15:14:17.993 spawn: self.connect()
2023-01-06 15:14:17.993 spawn: File "/usr/lib/python3.9/http/client.py", line 946, in connect
2023-01-06 15:14:17.993 spawn: self.sock = self._create_connection(
2023-01-06 15:14:17.993 spawn: File "/usr/lib/python3.9/socket.py", line 823, in create_connection
2023-01-06 15:14:17.994 spawn: for res in getaddrinfo(host, port, 0, SOCK_STREAM):
2023-01-06 15:14:17.994 spawn: File "/usr/lib/python3.9/socket.py", line 954, in getaddrinfo
2023-01-06 15:14:17.994 spawn: for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
2023-01-06 15:14:17.994 spawn: socket.gaierror: [Errno -3] Try again
2023-01-06 15:14:17.994 spawn: During handling of the above exception, another exception occurred:
2023-01-06 15:14:17.994 spawn: Traceback (most recent call last):
2023-01-06 15:14:17.994 spawn: File "/usr/bin/tvhmeta", line 458, in
2023-01-06 15:14:17.994 spawn: process(sys.argv[1:])
2023-01-06 15:14:17.994 spawn: File "/usr/bin/tvhmeta", line 453, in process
2023-01-06 15:14:17.994 spawn: tvhmeta.fetch_and_persist_artwork(opts.uuid, opts.force_refresh, opts.modules_movie, opts.modules_tv)
2023-01-06 15:14:17.994 spawn: File "/usr/bin/tvhmeta", line 225, in fetch_and_persist_artwork
2023-01-06 15:14:17.995 spawn: recjson = self.request("api/idnode/load", data)
2023-01-06 15:14:17.995 spawn: File "/usr/bin/tvhmeta", line 150, in request
2023-01-06 15:14:17.995 spawn: req = urlopen(full_url, data=bytearray(data, 'utf-8'))
2023-01-06 15:14:17.995 spawn: File "/usr/lib/python3.9/urllib/request.py", line 214, in urlopen
2023-01-06 15:14:17.995 spawn: return opener.open(url, data, timeout)
2023-01-06 15:14:17.995 spawn: File "/usr/lib/python3.9/urllib/request.py", line 517, in open
2023-01-06 15:14:17.995 spawn: response = self._open(req, data)
2023-01-06 15:14:17.995 spawn: File "/usr/lib/python3.9/urllib/request.py", line 534, in _open
2023-01-06 15:14:17.996 spawn: result = self._call_chain(self.handle_open, protocol, protocol +
2023-01-06 15:14:17.996 spawn: File "/usr/lib/python3.9/urllib/request.py", line 494, in _call_chain
2023-01-06 15:14:17.996 spawn: result = func(*args)
2023-01-06 15:14:17.996 spawn: File "/usr/lib/python3.9/urllib/request.py", line 1375, in http_open
2023-01-06 15:14:17.996 spawn: return self.do_open(http.client.HTTPConnection, req)
2023-01-06 15:14:17.996 spawn: File "/usr/lib/python3.9/urllib/request.py", line 1349, in do_open
2023-01-06 15:14:17.997 spawn: raise URLError(err)
2023-01-06 15:14:17.997 spawn: urllib.error.URLError:
2023-01-06 15:15:00.161 spawn: Done
For reference, this is one such tvhmeta invocation:
# tvhmeta --force-refresh --host localhost --user my-username --password my-password --tmdb-key my-tmdb-key --tvdb-key my-tvdb-key --uuid f5be8a96ea70376a0cf298c0ca873b21
I've written-up a replacement request() function along the lines of the urlopen documentation. I'm no regular python programmer but it appears to work, at least in the authenticated situation:
def request(self, url, data):
"""Send a request for data to Tvheadend."""
full_url = "http://{}:{}/{}".format(self.host,self.port,url)
req = urllib.request.Request(full_url, method="POST", data=bytearray(data, 'utf-8'))
if self.user is not None and self.password is not None:
base64string = base64.b64encode(bytes('{}:{}'.format(self.user, self.password), 'ascii'))
req.add_header('Authorization', 'Basic {}'.format(base64string.decode('utf-8')))
logging.info("Sending %s to %s" % (data, full_url))
ret = urllib.request.urlopen(req).read().decode('UTF-8', errors='replace');
logging.debug("Received: %s", ret)
return ret
Note there's also an
import base64
declaration among the other imports at the top of the tvhmeta script. CAVEAT, in the current docker image python is at version 3.9, so I don't know if the current function might work/have worked OK in earlier versions.
So back to the question in $TITLE, has this ever worked? Bonus question, if it indeed currently doesn't work, does the above look like a viable fix, or is there a simpler and more obvious fix?
