Geek Answers Handbook

Django REST Framework: Slow UI to browse due to large linked table

I have a model in my API that has a foreign key for a table with tens of thousands of records. When I look at a page with detailed information about the model in the user interface that I am viewing, loading the page takes forever because it tries to fill out the foreign key drop-down list with tens of thousands of entries for the HTML form for the PUT command.

Is there any way around this? I think my best solution would be to prevent the browser interface from displaying this field and thereby prevent a slow load. Users can update the field directly using the PUT api request.

Thanks.

+8
user-interface django-rest-framework
source share
4 answers

Take a look at using an autocomplete widget or switch to using a dumb text field widget.

Autocomplete documentation here: http://www.django-rest-framework.org/topics/browsable-api/#autocomplete

+4
source share

You can force TextInput with a simple:

from django.forms import widgets ... class YourSerializer(serializers.ModelSerializer): param = serializers.PrimaryKeyRelatedField( widget=widgets.TextInput )

Or after the correct autocomplete_light configuration:

 import autocomplete_light ... class YourSerializer(serializers.ModelSerializer): paramOne = serializers.PrimaryKeyRelatedField( widget=autocomplete_light.ChoiceWidget('RelatedModelAutocomplete') ) paramMany = serializers.PrimaryKeyRelatedField( widget=autocomplete_light.MultipleChoiceWidget('RelatedModelAutocomplete') )

Filter the results returned by autocomplete_light this part of the documentation .

+1
source share

Please note that you can disable the HTML form and save the raw data in json format:

 class BrowsableAPIRendererWithoutForms(BrowsableAPIRenderer): """Renders the browsable api, but excludes the forms.""" def get_rendered_html_form(self, data, view, method, request): return None

and in settings.py:

 REST_FRAMEWORK = { 'DEFAULT_RENDERER_CLASSES': ( 'rest_framework.renderers.JSONRenderer', 'application.api.renderers.BrowsableAPIRendererWithoutForms', ), }

this speeds up the process, and you can still send messages from the user interface.

+1
source share

This is a very good question for no obvious problem. The assumptions you make about learning Django and the related DRF plugins when reading the official documentation will create a conceptual model that is simply not true. I am saying here that Django, explicitly designed for relational databases, does not do it quickly out of the box!

problem

The reason for slow Django / DRF when querying a model that contains one-to-many relationships in the ORM world is known as the N + 1 ( N + 1 , N + 1 ) problem and is especially noticeable when ORM uses lazy loading - Django uses lazy loading !!!

example

Suppose you have a model that looks like this: the reader has many books . Now you would like to get all the books with the title that the hardcore reader read. In Django, you accomplish this by interacting with ORM in this way.

 # First Query: Assume this one query returns 100 readers. > readers = Reader.objects.filter(type='hardcore') # Constitutive Queries > titles = [reader.book.title for reader in readers]

Under the hood. The first Reader.objects.filter(type='hardcore') statement will create a single SQL query similar to this one. We assume that it will return 100 records.

 SELECT * FROM "reader" WHERE "reader"."type" = "hardcore";

Next, for each reader [reader.book.title for reader in readers] you should get related books. It will look something like this in SQL.

 SELECT * FROM "book" WHERE "book"."id" = 1; SELECT * FROM "book" WHERE "book"."id" = 2; ... SELECT * FROM "book" WHERE "book"."id" = N;

What you have left is 1, select 100 readers, and N - get books -where N is the number of books. Thus, in total you have N + 1 database queries.

The consequence of this behavior is 101 database queries, which ultimately leads to extremely long loading times of a small amount of data and slows down Django!

Decision

The solution is easy, but not obvious. The following white papers for Django or DRF do not cover the issue. In the end, you follow best practices and end up with slow application.

To solve the problem of slow loading, you will have to upload your data to Django. This usually means using the appropriate prefetch_related () or select_related () method to build the SQL INNER JOIN on models / tables and retrieve all your data in just 2 queries instead of 101.

Related Readings

0
source share

More articles:


All Articles