How to upload a file via http with authorization in python 3.0, working with errors?

Question

I have a script that I would like to continue to use, but it looks like I need to find some workaround for the error in Python 3 or return to version 2.6, and therefore you will have to downgrade other scripts as well ...

Hopefully someone here has already managed to find a workaround.

The problem is that due to new changes in Python 3.0 regarding bytes and strings, not all library files are apparently tested.

I have a script that loads a page from a web server. This script passed the username and password as part of the url in python 2.6, but in Python 3.0 this no longer works.

For example, this:

import urllib.request; url = "http://username:password@server/file"; urllib.request.urlretrieve(url, "temp.dat"); 

fails with this exception:

 Traceback (most recent call last): File "C:\Temp\test.py", line 5, in <module> urllib.request.urlretrieve(url, "test.html"); File "C:\Python30\lib\urllib\request.py", line 134, in urlretrieve return _urlopener.retrieve(url, filename, reporthook, data) File "C:\Python30\lib\urllib\request.py", line 1476, in retrieve fp = self.open(url, data) File "C:\Python30\lib\urllib\request.py", line 1444, in open return getattr(self, name)(url) File "C:\Python30\lib\urllib\request.py", line 1618, in open_http return self._open_generic_http(http.client.HTTPConnection, url, data) File "C:\Python30\lib\urllib\request.py", line 1576, in _open_generic_http auth = base64.b64encode(user_passwd).strip() File "C:\Python30\lib\base64.py", line 56, in b64encode raise TypeError("expected bytes, not %s" % s.__class__.__name__) TypeError: expected bytes, not str 

Apparently base64-encoding now needs bytes and prints a string, and thus urlretrieve (or some code in it) that builds a username string: password and tries to make base64-encode for easy authorization fails .

If I try to use urlopen instead, like this:

 import urllib.request; url = "http://username:password@server/file"; f = urllib.request.urlopen(url); contents = f.read(); 

Then this failure is eliminated:

 Traceback (most recent call last): File "C:\Temp\test.py", line 5, in <module> f = urllib.request.urlopen(url); File "C:\Python30\lib\urllib\request.py", line 122, in urlopen return _opener.open(url, data, timeout) File "C:\Python30\lib\urllib\request.py", line 359, in open response = self._open(req, data) File "C:\Python30\lib\urllib\request.py", line 377, in _open '_open', req) File "C:\Python30\lib\urllib\request.py", line 337, in _call_chain result = func(*args) File "C:\Python30\lib\urllib\request.py", line 1082, in http_open return self.do_open(http.client.HTTPConnection, req) File "C:\Python30\lib\urllib\request.py", line 1051, in do_open h = http_class(host, timeout=req.timeout) # will parse host:port File "C:\Python30\lib\http\client.py", line 620, in __init__ self._set_hostport(host, port) File "C:\Python30\lib\http\client.py", line 632, in _set_hostport raise InvalidURL("nonnumeric port: '%s'" % host[i+1:]) http.client.InvalidURL: nonnumeric port: 'password@server' 

Apparently, parsing the URLs in this "next url search library" doesn't know what to do with the username and passwords in the url.

What other options do I have?

Answer 1

Straight from Py3k docs: http://docs.python.org/dev/py3k/library/urllib.request.html#examples

 import urllib.request # Create an OpenerDirector with support for Basic HTTP Authentication... auth_handler = urllib.request.HTTPBasicAuthHandler() auth_handler.add_password(realm='PDQ Application', uri='https://mahler:8092/site-updates.py', user='klem', passwd='kadidd!ehopper') opener = urllib.request.build_opener(auth_handler) # ...and install it globally so it can be used with urlopen. urllib.request.install_opener(opener) urllib.request.urlopen('http://www.example.com/login.html') 

Answer 2

My advice would be to maintain your branch 2. * as your production branch until you can sort the contents of 3.0.

I'm going to wait a while before moving on to Python 3.0. It seems that a lot of people are in a hurry, but I just want everything to be sorted out, and a decent selection of third-party libraries. It may take a year, it may take 18 months, but the pressure on the “update” is really low for me.