Simple loading of kernel data is very slow

Question

my iPhone application has a Words object with the attributes word , length and language . Both are indexed:

I copied cdatamodel and the database to a separate importer application, where it got pre-populated with approximately 400 kilograms of words in different languages. I checked the import by looking at the SQLite file, and then copied the pre-populated database back to the iPhone project.

At first, I thought that the predicate (simple) is a problem. But even after removing the predicate from the query, it takes a very long time to complete:

 2011-09-01 09:26:38.945 MyApp[3474:3c07] Start 2011-09-01 09:26:58.120 MyApp[3474:3c07] End 

This is what my code looks like:

 // Get word NSLog(@"Start"); NSFetchRequest *fetchRequest = [[NSFetchRequest alloc] init]; NSEntityDescription *entity = [NSEntityDescription entityForName:@"Words" inManagedObjectContext:appDelegate.managedObjectContext]; [fetchRequest setEntity:entity]; NSError *error = nil; NSArray *fetchedObjects = [appDelegate.managedObjectContext executeFetchRequest:fetchRequest error:&error]; if (fetchedObjects == nil) { //... error handling code } [fetchRequest release]; NSLog(@"End"); return fetchedObjects; 

Is the number of records in the database a problem for master data?

---

EDIT : As pointed out by gbbrückmann and jrturton, it is nice to set fetchBatchSize . But the receiving time is still unsatisfactory:

• 2 seconds with a set of predicates:

  NSPredicate * predicate = [NSPredicate predicateWithFormat: @ "length ==% d And the language is BEGINSWITH% @", wordLength, lng]; [fetchRequest setPredicate: predicate];

• 7 seconds with a set batch size:

  [fetchRequest setFetchBatchSize: 1];

• 1 second indicating both the predicate and batch size

Is there another bottleneck?

Answer 1

Since you are not limiting the result set in any way, retrieving 400,000 objects will immediately be a burden for Core Data. There are several ways to improve performance:

Changing the selection request fetchBatchSize limits the number of objects that the selection will store in memory at a time. This feature is completely transparent to your application, so it is definitely worth a try.

If you do not need full-fledged objects, you can consider changing the query resultType to a more appropriate value. Especially if you are only interested in certain values ​​of an object, using NSDictionaryResultType is a good idea.

Finally, the fetchLimit and fetchOffset properties allow you to limit the range of results if you want to control batch processing yourself. It is a good idea if your processing of each of the result objects uses a lot of memory, because you can wrap each batch in NSAutoreleasePool (just do not be tempted to create an autoplay pool for each result object).

I think 1 sec. it can be as fast as in your case, even if you resorted to a simple Sqlite database. The only further optimization I can think of is to use one table in the language (instead of putting words from all languages ​​in one table). This, of course, will only work with Sqlite, unless you define separate entities for all languages, i.e. e. Take your Words object as is and make it abstract. Then add subtitles like EnglishWord etc. Objects from different objects are stored in separate tables. Thus, in combination with the fetchBatchSize and predicate parameters, this should be done similarly to the Sqlite approach with separate tables for all languages.

Answer 2

You do BEGINSWITH - this is not a very fast operation! However, there are a finite number of languages, so emum will probably help.

Have a language_id field that indexed the integer and uses it in your predicate. You can still save the name of the language and return it as part of the extracted object, just do not search on it :)

---

PS You can enable SQL debugging by adding "-com.apple.CoreData.SQLDebug 1" as the argument passed at startup (configure this in your schema) - this can help you see what SQL does behind the scenes.

(see this question for more details)

Answer 3

This will get your complete 400 kilobyte database into memory, which seems to be a lot. You can explore NSFetchRequest

 setFetchBatchSize 

which stops the framework returning full objects for everything in your query of the selection, assuming that you do not need every returned object that will be retrieved from the store in the first instance.