Migrations in mongoengine: InvalidId

Question

I work with mongoengine and try to do a simple migration. I have a field that I would like to transfer from StringField to ReferenceField for another object. I planned to do the migration manually, first creating a new object based on the string that comes from the old StringField, and then explicitly setting it.

The problem is that I can’t even access one of the top-level documents after changing the field type. Do I need to create a "dummy" field in my document class class as a placeholder for a new field? It seems to me like nothing, so I guess there is a better way?

Here is the error, because the field exiting the database (StringField) does not match the reference field.

/usr/lib/python2.7/site-packages/mongoengine/queryset/base.pyc in __getitem__(self, key) 149 return queryset._get_as_pymongo(queryset._cursor.next()) 150 return queryset._document._from_son(queryset._cursor[key], --> 151 _auto_dereference=self._auto_dereference) 152 raise AttributeError 153 /usr/lib/python2.7/site-packages/mongoengine/base/document.pyc in _from_son(cls, son, _auto_dereference) 568 try: 569 data[field_name] = (value if value is None --> 570 else field.to_python(value)) 571 if field_name != field.db_field: 572 del data[field.db_field] /usr/lib/python2.7/site-packages/mongoengine/fields.pyc in to_python(self, value) 935 not isinstance(value, (DBRef, Document, EmbeddedDocument))): 936 collection = self.document_type._get_collection_name() --> 937 value = DBRef(collection, self.document_type.id.to_python(value)) 938 return value 939 /usr/lib/python2.7/site-packages/mongoengine/base/fields.pyc in to_python(self, value) 390 def to_python(self, value): 391 if not isinstance(value, ObjectId): --> 392 value = ObjectId(value) 393 return value 394 /usr/lib/python2.7/site-packages/bson/objectid.pyc in __init__(self, oid) 88 self.__generate() 89 else: ---> 90 self.__validate(oid) 91 92 @classmethod /usr/lib/python2.7/site-packages/bson/objectid.pyc in __validate(self, oid) 192 raise InvalidId("%s is not a valid ObjectId" % oid) 193 else: --> 194 raise InvalidId("%s is not a valid ObjectId" % oid) 195 else: 196 raise TypeError("id must be an instance of (%s, %s, ObjectId), " InvalidId: Was Dirty: a2111fe89383bb562738b81c2b63fe78e877ed32 is not a valid ObjectId 

Answer 1

I always wrap things manually using an intermediate collection or an intermediate field in the same collection, but this example does not make you look like it. Stack overflow hates external links, so I include the example below. By the way, be careful with this part of "drop_collection"!

 import unittest from mongoengine import * class Test(unittest.TestCase): def setUp(self): conn = connect(db='mongoenginetest') def create_old_data(self): class Person(Document): name = StringField() age = FloatField() # Save as string meta = {'allow_inheritance': True} Person.drop_collection() Person(name="Rozza", age=35.00).save() rozza = Person._get_collection().find_one() self.assertEqual(35.00, rozza['age']) def test_migration(self): self.create_old_data() class Person(Document): name = StringField() age = StringField() meta = {'allow_inheritance': True} for p in Person.objects: p.age = "%s" % p.age p.save() rozza = Person.objects.first() self.assertEqual("35.0", rozza.age) if __name__ == '__main__': unittest.main() 

Answer 2

Two options that I would suggest for your migration script:

• Using DynamicField in the field you need to migrate should allow you to read ObjectIds and save back DBRefs
• Performing migration to pymongo directly (the pymongo collection is available through Person._get_collection ()) and cyclic moving elements, updating and saving