Authenticated HTTP POST with XML payload using Python urllib2

Question

I am trying to send a POST message with a purely XML payload (I think) using urllib2 in IronPython. However, every time I send it, it returns error code 400 (failed request).

I'm actually trying to mimic Boxee to remove a queue point call for which the actual data packets look like this (from WireShark):

POST /action/add HTTP/1.1 User-Agent: curl/7.16.3 (Windows build 7600; en-US; beta) boxee/0.9.21.11487 Host: app.boxee.tv Accept: */* Accept-Encoding: deflate, gzip Cookie: boxee_ping_version=9; X-Mapping-oompknoc=76D730BC9E858725098BF13AEFE32EB5; boxee_app=e01e36e85d368d4112fe4d1b6587b1fd Connection: keep-alive Content-Type: text/xml Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7 Accept-Language: en-us,en;q=0.5 Keep-Alive: 300 Connection: keep-alive Content-Length: 53 <message type="dequeue" referral="3102296"></message> 

I am using the following python code to send a POST:

 def PostProtectedPage(theurl, username, password, postdata): req = urllib2.Request(theurl, data=postdata) req.add_header('Content-Type', 'text/xml') try: handle = urllib2.urlopen(req) except IOError, e: # here we are assuming we fail pass else: # If we don't fail then the page isn't protected print "This page isn't protected by authentication." sys.exit(1) if not hasattr(e, 'code') or e.code != 401: # we got an error - but not a 401 error print "This page isn't protected by authentication." print 'But we failed for another reason.' sys.exit(1) authline = e.headers.get('www-authenticate', '') # this gets the www-authenticat line from the headers - which has the authentication scheme and realm in it if not authline: print 'A 401 error without an authentication response header - very weird.' sys.exit(1) authobj = re.compile(r'''(?:\s*www-authenticate\s*:)?\s*(\w*)\s+realm=['"](\w+)['"]''', re.IGNORECASE) # this regular expression is used to extract scheme and realm matchobj = authobj.match(authline) if not matchobj: # if the authline isn't matched by the regular expression then something is wrong print 'The authentication line is badly formed.' sys.exit(1) scheme = matchobj.group(1) realm = matchobj.group(2) if scheme.lower() != 'basic': print 'This example only works with BASIC authentication.' sys.exit(1) base64string = base64.encodestring('%s:%s' % (username, password))[:-1] authheader = "Basic %s" % base64string req.add_header("Authorization", authheader) try: handle = urllib2.urlopen(req) except IOError, e: # here we shouldn't fail if the username/password is right print "It looks like the username or password is wrong." print e sys.exit(1) thepage = handle.read() return thepage 

However, whenever I run this, it returns Error 400 (Bad Request)
I know that authentication is correct because I use it elsewhere to retrieve the queue (and I can’t imagine that it is not being used, otherwise now, in which account will the change be applied?)

Looking at network capture, can I just skip adding some headers to the request? Probably something simple, but I just don't know enough about python or HTTP requests to find out what.

Edit : BTW, I call the code as follows (it is really dynamic, but this is the main idea):

 PostProtectedPage("http://app.boxee.tv/action/add", "user", "pass", "<message type=\"dequeue\" referral=\"3102296\"></message>") 

Answer 1

This worked fine for me:

 curl -v -A 'curl/7.16.3 (Windows build 7600; en-US; beta) boxee/0.9.21.11487' \ -H 'Content-Type: text/xml' -u "USER:PASS" \ --data '<message type="dequeue" referral="12573293"></message>' \ 'http://app.boxee.tv/action/add' 

But I get a 400 Bad Request if I try to remove a referral ID that is not currently in the queue. If you use the same referral ID that was found at Wireshark, this is very likely to happen for you. Use

 wget -nv -m -nd --user=USER --password=PASS http://app.boxee.tv/api/get_queue 

is in the queue to make sure that you are trying to delete.