Geek Answers Handbook

Python requests don't work with google engine

I am trying to get a page using python requests with this code in views.py :

 s = requests.Session() r = s.get("https://www.23andme.com/")

I get an error Exceeded 30 redirects. My app is the Django app for Google App Engine. I got this code to work on the python console and on the django server on pythonanywhere.com, but for some reason it does not work on the Google engine. What could be the reason for this? Thanks

Edit: there seems to be another problem with the Requests module in my application. I have this code to add a letter to the mailchimp list:

 m = mailchimp.Mailchimp(MAILCHIMP_API_KEY) list_response = m.lists.list()

but if with an error HTTPS/SSL is required

+6
source share
2 answers

According to this discussion, the problem is netrc in the Google App Engine. I was able to solve this problem using an earlier version of the queries ( 1.2.3 in my case, but others may work too.)

Edit: According to this, respond, Requests 2.1.0 should also work.

+2
source

Try installing urllib3 (and other things, see below).

See, historically there have been many problems for queries with the Google engine (and see question # 498 ). They were mostly solved with urllib3 support for GAE, which came with v1.3 . It was a long time ago (the current version is 1.7), so this is probably not a problem, however, during the initial installation of requests, it includes urllib3 in a folder called packages and, possibly, it does not include all of it.

I also tried to find the source code for requests and found this interesting:

 # Attempt to enable urllib3 SNI support, if possible try: from requests.packages.urllib3.contrib import pyopenssl pyopenssl.inject_into_urllib3() except ImportError: pass

Going deeper, the contrib package includes a pyopenssl.py script that requires:

SSL with SNI support for Python 2.

The following packages are required for this:

  • pyOpenSSL (verified since 0.13)
  • ndg-httpsclient (checked since 0.3.2)
  • pyasn1 (verified from 0.1.6)

So to summarize:

  • Install urllib3 and the other SSL packages mentioned above, then try running this request that you make again and see if anything has changed. I assume this will (at least) help with mailchimp as it also complains about SSL / HTTPS issues.

  • If this does not work, try using urllib3 api instead of requests to accomplish the same task and see if it works. If so, the problem is specifically related to packaged urllib3 that uses requests , which may require some fixing.

     import urllib3 http = urllib3.PoolManager() r = http.request('GET', 'https://www.23andme.com/')

Sorry, this is not a definite solution, I hope one of my suggestions will help. Update me regarding the progress that I will try and help me as much as possible.

+5
source

More articles:


All Articles