Python request cookie authentication

Question

I am trying to simulate a user action on a site programmatically using the Python requests API. To do this programmatically, the request must have user / password authentication, as well as pass several invisible objects in the form of cookies in the header. To get NVP, I initially make a dummy request and the server returns cookies to me. I get the necessary values ​​from these cookies and use this to send the actual request. But the request failed, and the server complains that I have not logged in. But if I use the cookie value from my browser, the request will succeed.

Fiction request to programmatically retrieve JSESSIONID, glide_user, and glide_user_session parameters in a cookie

 response = requests.get('http://example.com/make_dummy_get',auth=('username','pasword')) cookie_params = response.cookies.items() 

below is the actual request

 headers = { 'Host': 'example.com' ,'Connection': 'keep-alive' ,'Content-Length': 113 ,'Cache-Control': 'max-age=0' ,'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8' ,'Origin': 'example.com' ,'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36' ,'Content-Type': 'application/x-www-form-urlencoded' ,'Referer': 'www.example.com/asdas/' ,'Accept-Encoding': 'gzip,deflate,sdch' ,'Accept-Language': 'en-US,en;q=0.8' ,'Cookie': 'JSESSIONID=B6F7371A11825472CAB0366A4DCDD8EFB; glide_user="SC:Z3Vlc3Q=:b890b38b7f000001121dbe81a08c413ca5"; glide_user_session="SC:Z3Vlc3Q=:b890b38b7f000001121dbe81a08c413ca5"' } form_data = { 'param1': 'value1' ,'param2': 'value2' ,'param3': 'value3' } res = requests.post('http://example.com/make_post_request',auth=('username','pasword'),data=form_data,headers = headers) 

It seems to me that the session created by my dummy request for some reason closes and therefore the second request is rejected and the html response says that I have to log in to get access to the requested resource.

I did the same exercise with the Java apache HttpClient and ended up with the same problem. What am I missing here to make the request successful without any problems with registration or authentication?

Answer 1

First you must use the Session object from the queries. This will manage the cookies (and prepare them for you), so you do not need to create a cookie header for yourself.

 s = requests.Session() s.get('http://example.com/make_dummy_get',auth=('username','pasword')) print(s.cookies) 

Next, I need strong to advise you to stop installing the following headers:

• Host
• Content-Length
• Content-Type
• Cookie

All four of these headers will be created by requests for you. A Cookie header will be generated using a CookieJar that uses Session . Content-Length and Content-Type will be evaluated while requests prepare the body.

Also, if you try to use cookies for authentication, the server is likely to get confused because you also pass auth=('username', 'password') in the second request. This generates an authorization header, so you send the Cookie header and the Authorization header. The server considers this rather suspicious and, in truth, refuses to accept your request as authenticated.