Oddites

Question

I bang my head against the wall with this. I tried every example, reading every last bit that I can find on the Internet, about basic http authorization with urllib2, but I can't figure out what causes my specific error.

Adding to the frustration is that the code works for one page, but not for another. Entrance to www.mysite.com/adm is absolutely smooth. It does not authenticate the problem. However, if I change the address to "http://mysite.com/adm/items.php?n=201105&c=200", I get this error:

<h4 align="center" class="teal">Add/Edit Items</h4> <p><strong>Client:</strong> </p><p><strong>Event:</strong> </p><p class="error">Not enough information to complete this task</p> <p class="error">This is a fatal error so I am exiting now.</p> 

A google search resulted in zero information about this error.

The administrator is a page with a set of frames, I'm not sure that this is generally relevant.

Here is the current code:

 import urllib2, urllib import sys import re import base64 from urlparse import urlparse theurl = 'http://xxxxxmedia.com/adm/items.php?n=201105&c=200' username = 'XXXX' password = 'XXXX' passman = urllib2.HTTPPasswordMgrWithDefaultRealm() passman.add_password(None, theurl,username,password) authhandler = urllib2.HTTPBasicAuthHandler(passman) opener = urllib2.build_opener(authhandler) urllib2.install_opener(opener) pagehandle = urllib2.urlopen(theurl) url = 'http://xxxxxxxmedia.com/adm/items.php?n=201105&c=200' values = {'AvAudioCD': 1, 'AvAudioCDDiscount': 00, 'AvAudioCDPrice': 50, 'ProductName': 'python test', 'frmSubmit': 'Submit' } #opener2 = urllib2.build_opener(urllib2.HTTPCookieProcessor()) data = urllib.urlencode(values) req = urllib2.Request(url, data) response = urllib2.urlopen(req) 

This is only one of the many versions I have tried. I followed each example from the Urllib2 Missing Manual, but still got the same error.

Can someone point out what I'm doing wrong?

Answer 1

About a year ago, I went through the same process and documented how I solved the problem. Direct and easy authentication method and standard. Choose what you see fit.

Python authentication

In the description .

Answer 2

Launch a similar problem today. I use basic authentication on a website that I am developing and I could not authenticate users.

Here are a few things you can use to debug your problem:

• I used slumber.in and httplib2 for testing purposes. I ran both from the ipython shell to find out what answers I get.
• Slumber actually uses httplib2 under the covers, so they act similarly. I used tcpdump and later tcpflow (which displays information in a much more readable form) to see what was actually sent and received. If you want to use the graphical interface, see Postings or alternatives.
• I tested my site with curl, and when I used curl with my username / password, it worked correctly and showed the requested page. But nap and httplib2 still didn't work.
• I checked my site and browserspy.dk to see what were the differences. It is important that the website of browsers works for basic authentication, and my website did not, so I could compare them. I read in many places that you need to send HTTP 401 Not Authorized so that the browser or tool you use can send the username / password that you provided. But what I did not know, you also needed the WWW-Authenticate field in the header. So this is the missing part.
• What made this whole situation fuzzy was during testing. I would see that httplib2 sends the main authentication headers with most requests (tcpflow will show this). It turns out that the library does not send username and password authentication on first request. If the response contains "Status 401" and "WWW-Authenticate", the credentials are sent on the second request and all requests to this domain from now on.

So, to summarize, your application may be correct, but you can not return the standard headers and status code so that the client can send credentials. Use debugging tools to find what to eat. In addition, there is a debugging mode for httplib2, just set httplib2.debuglevel=1 so that debugging information is printed on standard output. This is much more useful than using tcpdump because it is at a higher level.

Hope this helps someone.

Answer 3

From the HTML you submitted, it still thinks that you have authenticated successfully, but encounter an error after processing the POST request. I tried your URL and failed authentication, I got a standard 401 page.

In any case, I suggest that you try to run your code again and perform the same operation manually in Firefox, only this time with Wireshark to capture the exchange. You can get the full text of the HTTP request and response in both cases and compare the differences. In most cases, this will lead you to the source of the error you receive.

Answer 4

I also found that passman stuff doesn't work (sometimes?). Adding a base64 header / password in accordance with this answer https://stackoverflow.com/a/4646262/2326288 really worked for me. I access the jenkins url like this: http: /// job // lastCompletedBuild / testR eport / api / python

This works for me:

 import urllib2 import base64 baseurl="http://jenkinsurl" username=... password=... url="%s/job/jobname/lastCompletedBuild/testReport/api/python" % baseurl base64string = base64.encodestring('%s:%s' % (username, password)).replace('\n', '') request = urllib2.Request(url) request.add_header("Authorization", "Basic %s" % base64string) result = urllib2.urlopen(request) data = result.read() 

This does not work for me, error 403 every time:

 import urllib2 baseurl="http://jenkinsurl" username=... password=... ##urllib2.HTTPError: HTTP Error 403: Forbidden passman = urllib2.HTTPPasswordMgrWithDefaultRealm() passman.add_password(None, url, username,password) urllib2.install_opener(urllib2.build_opener(urllib2.HTTPBasicAuthHandler(passman))) req = urllib2.Request(url) result = urllib2.urlopen(req) data = result.read()