How to manipulate files in google app engine datastore

Question

My problem revolves around a user uploading a text file to my application. I need to get this file and process it using my application before saving it to the data store. From the small number that I read, I understand that loading the user goes directly to the data store as blots, and it is normal if I can then get this file, perform operations on it (which means changing the data inside), and then rewrite it back to the data warehouse. All these operations must be performed by the application. Unfortunately, from the datastore documentation, http://code.google.com/appengine/docs/python/blobstore/overview.html the application cannot directly create blob in the data store. This is my main headache. I just need a way to create a new blob / file in the data store from my application without any user interaction.

Answer 1

blobstore != datastore .

You can read and write data to the datastore as much as you want, as long as your data is <1MB using db.BlobProperty in your organization.

As Wooble comments, the new File API allows you to write to blobstore , but if you do not incrementally type text into the blobstore file using tasks or something like the mapreduce library, you are still limited by the limit of calling the 1MB API for reading / writing.

Answer 2

Thank you for your help. After many sleepless nights, 3 books with application engines and lots of Google, I found the answer. Here is the code (it should be very clear):

 from __future__ import with_statement from google.appengine.api import files from google.appengine.ext import blobstore from google.appengine.ext import webapp from google.appengine.ext.webapp import util class MainHandler(webapp.RequestHandler): def get(self): self.response.out.write('Hello WOrld') form=''' <form action="/" method="POST" enctype="multipart/form-data"> Upload File:<input type="file" name="file"><br/> <input type="submit"></form>''' self.response.out.write(form) blob_key="w0MC_7MnZ6DyZFvGjgdgrg==" blob_info=blobstore.BlobInfo.get(blob_key) start=0 end=blobstore.MAX_BLOB_FETCH_SIZE-1 read_content=blobstore.fetch_data(blob_key, start, end) self.response.out.write(read_content) def post(self): self.response.out.write('Posting...') content=self.request.get('file') #self.response.out.write(content) #print content file_name=files.blobstore.create(mime_type='application/octet-stream') with files.open(file_name, 'a') as f: f.write(content) files.finalize(file_name) blob_key=files.blobstore.get_blob_key(file_name) print "Blob Key=" print blob_key def main(): application=webapp.WSGIApplication([('/', MainHandler)],debug=True) util.run_wsgi_app(application) if __name__=='__main__': main()