Using Mongo with FLASK and python

Question

I am trying to learn python, mongodb and flask, and I use the VERY EXCELLENT blog by Miguel Greenberg, which provides an excellent set of tutorials on blog.miguelgrinberg.com

I got a small RESTful server working fine, but now I want to pull stuff from mongo, not mysql

I can pull the mongo record using the code below, but I'm struggling to display it.

I used the arrows in the code below to show where I am fighting, the lack of experience that I think. Any thoughts would be appreciated.

#!flask/bin/python from flask import Flask, jsonify, abort, make_response, url_for from pymongo import MongoClient # connect to mongo database hosted on AWS # the script expects the host name to be in /etc/hosts file ''' Set up global variables here ''' mongo_server = "mongo_api" mongo_port = "27017" mongo_user = "admin" mongo_passwd = ":mysecretpassword@" connect_string = "mongodb://"+ mongo_user + mongo_passwd + mongo_server + ":" + mongo_port app = Flask(__name__) @app.errorhandler(404) def not_found(error): return make_response(jsonify( { 'error': 'Notfound' } ), 404) def make_public_page(page): new_page = {} for field in page: if field == 'id': new_page['uri'] = url_for('get_page', page_id = page['id'], _external = True) else: new_page[field] = page[field] return new_page @app.route('/api/v1.0/pages/<int:page_id>',methods = ['GET']) def get_page(page_id): ''' Can connect otherwise exit with message ''' try: connection = MongoClient(connect_string) # equal to > show dbs except: exit("Error: Unable to connect to the database") # exit with an error ''' connect to database and pull back collections ''' db = connection.test_database # equal to > use test_database pages = db.pages page = pages.find_one({"id": int(page_id)}) <------ this pulls back a document if page == None: <---- if a null set comes back then this works great abort(404) return jsonify( { 'page' : make_public_page(page[0])} ) <- error says its not json if __name__ == '__main__': app.run(debug = True) 

Any help is appreciated, page [0] is code that just doesn't work. I get

TypeError: ObjectId ('527e17c538320915e9893f17') is not serializable JSON

Thank you in advance

BTW Can't recommend Miguel mega-tutorial as a place to start building things

Answer 1

First of all, find_one will return one dictionary or None if there is no corresponding element in the collection. So I think that page[0] equivalent to getting the page dictionary value for key 0

If the returned documents contain an ObjectId as _id , you cannot just use jsonify , because, for example, ObjectId not serializable JSON. You can use something like this:

 jsonify({ 'page': make_public_page({k:v for k, v in page.items() if k != '_id'})) 

or you can just delete _id by calling page.pop('_id')

You can also use bson.json_util . It contains tools for converting between BSON and JSON.

 from flask import Response from bson import json_util 

And then replace jsonify with something similar to this:

 return Response( json_util.dumps({'page' : make_public_page(page)}), mimetype='application/json' ) 

Edit

If you need a short and dirty way to solve the problem, you can do it as follows:

 from bson import json_util, ObjectId import json #Lets create some dummy document to prove it will work page = {'foo': ObjectId(), 'bar': [ObjectId(), ObjectId()]} #Dump loaded BSON to valid JSON string and reload it as dict page_sanitized = json.loads(json_util.dumps(page)) 

Answer 2

you can set the default encoder:

 import json from bson.objectid import ObjectId def newEncoder(o): if type(o) == ObjectId: return str(o) return o.__str__ .... return json.dumps(list(somecollection.find(expr)) , default=newEncoder ) 

or you can subclass json.encoder.JSONEncoder

Answer 3

From what I see in your code, it seems you are not using your own Mongo identifiers (which are stored in the _id key), but instead you are generating your own integer identifiers that you store in the id key. Is it correct?

The problem with your code is that the Mongo _id key is in the dict objects that you send to make_public_page() and they cannot be serialized in JSON.

You can solve this problem by skipping the _id key:

 def make_public_page(page): new_page = {} for field in page: if field == 'id': new_page['uri'] = url_for('get_page', page_id = page['id'], _external = True) elif field != '_id': new_page[field] = page[field] return new_page 

As a side note, I think it's best not to invent your own identifiers and just use the ones that Mongo had.