Retroactive Google Cloud Storage Indexing

Question

There are many properties in my model that I currently do not need to index, but I can assume that I would like to be indexed at some unknown point in the future. If I explicitly set indexed=False for the property now, but changed my mind along the way, will Datastore automatically restore all indexes at that moment, including for previously recorded data? Are there any other implications for this approach?

Answer 1

No, changing indexed = True for indexed = False (and vice versa) will only affect entities written after this point to the data store. Here is the documentation that talks about this, and the corresponding paragraph:

Similarly, changing a property from indexed to non-indexed applies only to objects subsequently recorded in the data warehouse. Index entries for any existing objects with this property will continue to exist until the entities are updated or deleted. To avoid unwanted results, you should clear your code of all requests that filter or sort by property (now unindexed).

If you later decide to start indexing properties, you will have to go through your entities and rearrange them in the data warehouse.

Note, however, that changing a property from unindexed to indexed does not affect any existing entities that might have been created before the change. Filtering queries by property does not return such existing objects, because entities were not written to the query index when they were created. To make entities available for future queries, you must rewrite them in the data warehouse so that they are entered into the corresponding indexes. That is, you must do the following for each such existing object:

Get (get) an object from the data store.

Write (put) the object back to the data warehouse.

Answer 2

To index the properties of existing objects (according to the documentation):

• Get (get) an object from the data store.
• Write (put) the object back to the data warehouse.

didn't work for me. I used the appengine-mapreduce library and wrote MapOnlyMapper<Entity, Void> using the DatastoreMutationPool to index all existing objects in the data store.

Suppose the name property has not been indexed, and I want to index it in all existing entities. I had to do this:

 @Override public void map(Entity value) { String property = "name"; Object existingValue = value.getProperty(property); value.setIndexedProperty(property, existingValue); datastoreMutationPool.put(value); } 

Essentially, you will need to set the property as an indexed property using setIndexedProperty(prop, value) , and then save (place) the object.

I know that it’s very late to lay out the answer. I thought I could help someone who can deal with this problem.