How can I optimize this Google App Engine code?

Question

I'm relatively new to the python world, but that seems very straight forward.

Google yells at me that this code needs to be optimized:

class AddLinks(webapp.RequestHandler): def post(self): # Hash the textarea input to generate pseudo-unique value hash = md5.new(self.request.get('links')).hexdigest() # Seperate the input by line allLinks = self.request.get('links').splitlines() # For each line in the input, add to the database for x in allLinks: newGroup = LinkGrouping() newGroup.reference = hash newGroup.link = x newGroup.put() # testing vs live #baseURL = 'http://localhost:8080' baseURL = 'http://linkabyss.appspot.com' # Build template parameters template_values = { 'all_links': allLinks, 'base_url': baseURL, 'reference': hash, } # Output the template path = os.path.join(os.path.dirname(__file__), 'addLinks.html') self.response.out.write(template.render(path, template_values)) 

The control panel tells me that this uses a ton of processor.

Where to look for improvements?

Answer 1

The main overhead here is the multiple individual inserts in the data warehouse. If you can, save the links as a whole, as Andre suggests. You can always split array references and store it in ListProperty.

If you need an entity for each link, try the following:

 # For each line in the input, add to the database groups = [] for x in allLinks: newGroup = LinkGrouping() newGroup.reference = hash newGroup.link = x groups.append(newGroup) db.put(groups) 

This will reduce the number of accesses to the data warehouse to one, and these are circular transitions that really kill your high processor cap.

Answer 2

It looks pretty tense.

I see one thing that can make a slight improvement. Your calling is "self.request.get ('links')" twice.

So adding:

 unsplitlinks = self.request.get('links') 

And links, "unsplitlinks" can help.

In addition, the loop is the only area that I see that will be the goal of optimization. Is it possible to prepare the data and then add it to db at the same time, instead of doing a db add for each link? (I assume the .put () command adds a link to the database)

Answer 3

You can significantly reduce the interaction between your application and the database by simply saving the full self.request.get('links') in the text box in the database.

• only one put() per post(self)
• the hash is not saved n times (for each link, which does not make sense and is actually a waste of space).

And you save yourself by parsing the text field when someone actually calls the page ....

Answer 4

How often is it called? It doesn't look so bad ... especially after deleting a duplicate request.

Answer 5

Can I request a ListProperty method?

Sort of

 SELECT * FROM LinkGrouping WHERE links.contains('http://www.google.com') 

I have plans for the future where I need this functionality.

I definitely implement a single db.put () to reduce usage.

Answer 6

no / you cannot use something like "links.contains (' http://www.google.com ')" GQL does not support this