How to synchronize CoreData and the REST web service asynchronously and at the same time correctly propagate any REST errors to the user interface

Question

Hey, I'm currently working on a model layer for our application.

Some of the requirements are as follows:

• It should work on iPhone OS 3.0 +.
• The source of our data is a RESTful Rails application.
• We must cache data locally using Core Data.
• Client code (our user interface controllers) should have as little knowledge as possible about any network capabilities and should request / update the model using the kernel data API.

I tested a WWDC10 session 117 to create a server-driven user experience, spent some time checking Objective Resource , Core Resource, and RestfulCoreData .

The Objective Resource frame does not apply to master data per se and is simply an implementation of a REST client. Core Resource and RestfulCoreData all assume that you are talking to Core Data in your code, and they solve all nuts and bolts in the background at the model level.

Everything looks fine so far, and initially I, although either Core Resource or RestfulCoreData will cover all of the above requirements, but ... There are a couple of things that none of them seem to be able to solve correctly:

• The main thread should not be blocked when saving local updates to the server.
• If the save operation failed, the error should be propagated in the user interface, and no changes should be saved in the local master data store.

The main resource performs the issuance of all its requests to the server when calling - (BOOL)save:(NSError **)error in the context of the managed object and, therefore, can somehow provide the correct instance of NSError of basic requests to the server. But it blocks the calling thread until the save operation completes. Failure.

RestfulCoreData keeps your -save: messages intact and does not represent an additional wait time for the client thread. It simply watches for NSManagedObjectContextDidSaveNotification , and then issues the appropriate requests to the server in the notification handler. But in this way, the -save: call always succeeds (well, considering that Core Data is in order with the changes saved), and the client code that actually called it has no way of knowing that saving might not extend to the server from - for some 404 or 421 or some kind of error occurred on the server side. And even more, the local storage becomes for updating data, but the server never knows about the changes. Failure.

So, I am looking for a possible solution / general practice to solve all these problems:

• I do not want the calling thread to block every call to -save: during network requests.
• I want to somehow get notifications in the user interface that some synchronization operation went wrong.
• I want the actual storage of kernel data to fail if server requests were not executed.

Any ideas?

Answer 1

You really should take a look at RestKit ( http://restkit.org ) for this use case. It is designed to solve the problems of modeling and synchronizing remote JSON resources with a local cache supporting Core Data. It supports offline operation completely from the cache when there is no network available. All synchronization takes place in the background thread (network access, payload parsing and context merging of the managed entity), and there is a rich set of delegate methods for you to know what is happening.

Answer 2

There are three main components:

• UI action and saving changes in CoreData li>
• Saving changes to the server
• UI update with server response

NSOperation + NSOperationQueue helps streamline network requests. The delegate protocol will help your user interface classes understand the state of network requests, for example:

 @protocol NetworkOperationDelegate - (void)operation:(NSOperation *)op willSendRequest:(NSURLRequest *)request forChangedEntityWithId:(NSManagedObjectID *)entity; - (void)operation:(NSOperation *)op didSuccessfullySendRequest:(NSURLRequest *)request forChangedEntityWithId:(NSManagedObjectID *)entity; - (void)operation:(NSOperation *)op encounteredAnError:(NSError *)error afterSendingRequest:(NSURLRequest *)request forChangedEntityWithId:(NSManagedObjectID *)entity; @end 

The format of the protocol, of course, will depend on your specific use case, but essentially what you create is the mechanism by which changes can be “pushed” to your server.

It follows that the UI loop to consider, in order to keep your code clean, would be nice to call save: and the changes would be automatically redirected to the server. You can use NSManagedObjectContextDidSave notifications for this.

 - (void)managedObjectContextDidSave:(NSNotification *)saveNotification { NSArray *inserted = [[saveNotification userInfo] valueForKey:NSInsertedObjects]; for (NSManagedObject *obj in inserted) { //create a new NSOperation for this entity which will invoke the appropraite rest api //add to operation queue } //do the same thing for deleted and updated objects } 

The estimated overhead for inserting network operations should be fairly low, however, if it creates a noticeable delay in the user interface, you can simply extract the object identifiers from the storage notifications and create operations in the background thread.

If your REST API supports batch processing, you can even send the entire array immediately, and then notify the user interface that synchronized several objects.

The only problem that I foresee, and for which there is no “real” solution, is that the user does not want to wait until their changes are redirected to the server in order to be able to make more changes. The only good paradigm I came across is that you allow the user to continue editing objects and editing their edits together, i.e. Do not click on each save notification.

Answer 3

This becomes a synchronization problem and cannot be solved. Here's what I would do: in your iPhone UI, use one context, and then using a different context (and another stream), load the data from your web service. After all this has completed the synchronization / import processes recommended below, then update your interface after everything is imported correctly. If everything happens badly while accessing the network, simply discard the changes in a context other than the interface. This is a ton of work, but I think this is the best way to get closer to it.

Master data: import data efficiently

Master Data: Change Management

Master data: multithreading with master data

Answer 4

You need a callback function that will be launched in another thread (the one where the actual interaction with the server takes place), and then puts the result code / result information in semi-global data that will be periodically checked by the user interface thread. Make sure that the wirting of the number that serves as the flag is atomic or you will have a race state - let's say if your error response is 32 bytes, you need an int (which should have atomic acces) and then you save that int in the off / false / not-ready state until your large data block is written, and only then write “true” to flip the switch so that it says.

For correlated savings on the client side, you just need to save this data or not save it until you get OK from the server, make sure you have the kinnf option for the rollback option - say, a way to remove the server crash.

Beware that it will never be 100% safe if you do not complete the full two-step commit procedure (saving or deleting the client may fail after a signal from the server of the server), but it will cost you 2 trips to the server on (it may cost you 4 if your only rollback option is removed).

Ideally, you do the entire blocking version of the operation in a separate thread, but for this you will need 4.0.