App Engine API API is slower than native python

Question

I am using the AE socket API and I have performed the following test:

class TestHandler(webapp.RequestHandler): def get(self): size = 1024 * 4 start = datetime.datetime.now() for _ in range(10): sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.connect(('some.ip.here', 12345)) sock.send('some dummy data') data = '' newData = 'dummy' while newData != '' and not newData.endswith('\r\n'): newData = sock.recv(size) #print newData data += newData print '.' sock.close() #print 'timespent:' + str(datetime.datetime.now() - start) logging.info('timespent:' + str(datetime.datetime.now() - start)) 

• App Engine: 0: 00: 04.022290 and 0: 00: 04.209410
• Python local environment: 0: 00: 00.509000 and 0: 00: 00.511000

I started doing some tests after I noticed that a single request takes from 400 ms to 500 ms. The server responsible for the request is located on the Google GCE instance. The request is processed in a timeframe, which averages 0: 00: 00.0005 sec. Therefore, most of the delay is connected to the network. I think a good response time will be 100 ms. I could even do it with 200 ms if I knew that this would require some hard stuff. But I do not see that it should be otherwise, with the exception of some safety things. But this will prohibit some features that do not slow down.

Can anyone explain why the difference is so great?

Answer 1

I spent quite a bit of time trying to reproduce this problem with specific , publicly accessible sites, and it seems like I just can't.

My version of your code (so enriched that it actually works and shows the results) ...:

 import datetime import logging import socket import webapp2 class TestHandler(webapp2.RequestHandler): def get(self): size = 1024 * 4 start = datetime.datetime.now() for _ in range(10): sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.connect(('216.58.216.4', 80)) sock.send('GET / HTTP/1.0\n\r\n\r') data = '' newData = 'dummy' while newData != '' and not newData.endswith('\r\n'): newData = sock.recv(size) #print newData data += newData print '.' sock.close() msg = 'timespent:{} for:{}'.format( datetime.datetime.now() - start, len(data)) logging.info(msg) self.response.write('<p>{}</p>'.format(msg)) app = webapp2.WSGIApplication([ ('/timit', TestHandler), ], debug=True) 

This IP number is one for www.google.com (it would be better to use the host name www.google.com in its place, but you specified the IP number so I tried it), and the interaction, of course, just turns out its home page with ancient HTTP protocol 1.0.

The results vary, but pretty typical is what I got now:

 timespent:0:00:02.748760 for:52823 

I get about the same number as locally with dev_appserver.py , since I download it and run from <myappid>.appspot.com .

So, I made the version completely independent of GAE:

 import datetime import logging import socket def doit(): size = 1024 * 4 start = datetime.datetime.now() for _ in range(10): sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) sock.connect(('216.58.216.4', 80)) sock.send('GET / HTTP/1.0\n\r\n\r') data = '' newData = 'dummy' while newData != '' and not newData.endswith('\r\n'): newData = sock.recv(size) #print newData data += newData print '.' sock.close() msg = 'timespent:{} for:{}'.format( datetime.datetime.now() - start, len(data)) logging.info(msg) print('<p>{}</p>'.format(msg)) doit() 

and guess what - running it, after a bunch of dots, shows me very comparable

 <p>timespent:0:00:02.421856 for:52853</p> 

Therefore, I do not know what to do next. Maybe there was a problem a year ago when you made your observation, and now it has disappeared; perhaps this has something to do with the geographic location of the specific IP address you are connecting to (next to your local machine, far from where your GAE application works); who knows - without additional information and specific IP addresses, and, in fact, this is a complete, blind guess about what you observed.

In my case, my local machine where I am currently sitting is in Sunnyvale, California, and I’m sure that the two Google services in the game (www.google.com and myapp.appspot.com) in terms of network latency, cannot be like this far from here (my pings are on average 50 or 60 ms).

will have access to some type of closure for this, the highest of the voted "unanswered" questions related to the Google application ...! -)