Django: Quickly get many multi-user field identifiers

Question

I have the following model diagram in Django (with Postgres).

class A(Models.model): related = models.ManyToManyField("self", null=True) 

Given the QuerySet for A, I would like to return a dictionary matching each instance of A in QuerySet with a list of id its related instances as quickly as possible.

I can, of course, iterate over each A and request the corresponding field, but is there a better way?

Answer 1

According to three cases. You can use the values_list method to get only the results, and from this result, get only the identifier of your related instances. I use the pk field for my filter because I do not know your schema, but you can use anything, there just has to be a QuerySet .

 >>> result = A.objects.filter(pk=1) >>> result.values('related__id') [{'id': 2}, {'id': 3}] >>> result.values_list('related__id') [(2,), (3,)] >>> result.values_list('related__id', flat=True) [2, 3] 

Answer 2

You can get pretty close like this:

 qs = A.objects.prefetch_related(Prefetch( 'related', queryset=A.objects.only('pk'), to_attr='related_insts')).in_bulk(my_list_of_pks) 

This will give a mapping from the pks of the current object to the instance itself, so you can iterate as follows:

 for pk, inst in qs.iteritems(): related_ids = (related.pk for related in inst.related_insts) 

Or, if an instance is specified, you can perform a quick search like this:

 related_ids = (related.pk for related in qs[instance.pk]). 

This method matches instance identifiers with corresponding identifiers (indirectly) since you specifically requested a dictionary. If you are not doing a search, you may need the following:

 qs = A.objects.prefetch_related(Prefetch( 'related', queryset=A.objects.only('pk'), to_attr='related_insts')).filter(pk__in=my_list_of_pks) for inst in qs: related_ids = (related.pk for related in inst.related_insts) 

You can take note of using only to get pks out of db. There is an open, open ticket allowing the use of values and (I suppose) values_list in Prefetch requests. This will allow you to do the following.

 qs = A.objects.prefetch_related(Prefetch( 'related', queryset=A.objects.values_list('pk', flat=True), to_attr='related_ids')).filter(pk__in=my_list_of_pks) for inst in qs: related_ids = inst.related_ids 

Of course, you could optimize, for example, using qs.only('related_insts') in the main set of queries, but make sure that you do nothing with these instances - they are essentially just expensive containers for storing your related_funds.

I believe that this is the best that is available at the moment (without user requests). To achieve exactly what you want, two things are needed:

• The above function is implemented
• values_list is created to work with Prefetch to_attr , as is done for annotations.

With these two things (and a continuation of the above example), you can do the following to get exactly what you requested:

 d = qs.values_list('related_ids', flat=True).in_bulk() for pk, related_pks in d: print 'Containing Objects %s' % pk print 'Related objects %s' % related_pks # And lookups print 'Object %d has related objects %s' % (20, d[20]) 

I have explained some details explaining things, but this should be pretty clear from the documentation. If you need any clarification, feel free!