Geek Answers Handbook

Django REST Database Caching Error

TL DR

I am looking for a way to clear the cache after a request or completely disable it when running tests. The Django REST Framework seems to cache the results, and I need a way around this.

Long version and code

Well, that turned out to be very strange as I kept checking it. In the end, I got it to work, but I really do not like my workaround and in the name of knowledge, I have to find out why this is happening and how to solve this problem.

So, I have an APITestCase class declared as follows:

class UserTests(APITestCase):

Inside this class, I have a test function for my user-list view, since I have a custom set of queries depending on permissions. To clarify the situation:

  • the superuser can get a list of all users (4 instances returned),
  • employees cannot see superusers (3 copies returned),
  • ordinary users can get only 1 result, their own user (1 instance is returned)

The version of the test function that works:

 def test_user_querysets(self): url = reverse('user-list') # Creating a user user = User(username='user', password=self.password) user.set_password(self.password) user.save() # Creating a second user user2 = User(username='user2', password=self.password) user2.set_password(self.password) user2.save() # Creating a staff user staff_user = User(username='staff_user', password=self.password, is_staff=True) staff_user.set_password(self.password) staff_user.save() # Creating a superuser superuser = User(username='superuser', password=self.password, is_staff=True, is_superuser=True) superuser.set_password(self.password) superuser.save() # SUPERUSER self.client.logout() self.client.login(username=superuser.username, password=self.password) response = self.client.get(url) # HTTP_200_OK self.assertEqual(response.status_code, status.HTTP_200_OK) # All users contained in list self.assertEqual(response.data['extras']['total_results'], 4) # STAFF USER self.client.logout() self.client.login(username=staff_user.username, password=self.password) response = self.client.get(url) # HTTP_200_OK self.assertEqual(response.status_code, status.HTTP_200_OK) # Superuser cannot be contained in list self.assertEqual(response.data['extras']['total_results'], 3) # REGULAR USER self.client.logout() self.client.login(username=user2.username, password=self.password) response = self.client.get(url) # HTTP_200_OK self.assertEqual(response.status_code, status.HTTP_200_OK) # Only 1 user can be returned self.assertEqual(response.data['extras']['total_results'], 1) # User returned is current user self.assertEqual(response.data['users'][0]['username'], user2.username)

As you can see, I am testing user rights in this order: superuser, staff, regular user. And it works, so ...

The funny thing:

If I change the order of the tests and start with the regular user, state, superuser, the tests fail. The response from the first request gets caching, and then I get the same answer when I log in as user-user, so the number of results is again 1.

Version that does not work:

it is exactly the same as before, only tests are performed in the reverse order

 def test_user_querysets(self): url = reverse('user-list') # Creating a user user = User(username='user', password=self.password) user.set_password(self.password) user.save() # Creating a second user user2 = User(username='user2', password=self.password) user2.set_password(self.password) user2.save() # Creating a staff user staff_user = User(username='staff_user', password=self.password, is_staff=True) staff_user.set_password(self.password) staff_user.save() # Creating a superuser superuser = User(username='superuser', password=self.password, is_staff=True, is_superuser=True) superuser.set_password(self.password) superuser.save() # REGULAR USER self.client.logout() self.client.login(username=user2.username, password=self.password) response = self.client.get(url) # HTTP_200_OK self.assertEqual(response.status_code, status.HTTP_200_OK) # Only 1 user can be returned self.assertEqual(response.data['extras']['total_results'], 1) # User returned is current user self.assertEqual(response.data['users'][0]['username'], user2.username) # STAFF USER self.client.logout() self.client.login(username=staff_user.username, password=self.password) response = self.client.get(url) # HTTP_200_OK self.assertEqual(response.status_code, status.HTTP_200_OK) # Superuser cannot be contained in list self.assertEqual(response.data['extras']['total_results'], 3) # SUPERUSER self.client.logout() self.client.login(username=superuser.username, password=self.password) response = self.client.get(url) # HTTP_200_OK self.assertEqual(response.status_code, status.HTTP_200_OK) # All users contained in list self.assertEqual(response.data['extras']['total_results'], 4)

I work in python 2.7 with the following package versions:

 Django==1.8.6 djangorestframework==3.3.1 Markdown==2.6.4 MySQL-python==1.2.5 wheel==0.24.0

UPDATE

I use django cache by default, that is, I did not add anything to the cache in django settings.

As suggested, I tried to disable the default Django cache:

 CACHES = { 'default': { 'BACKEND': 'django.core.cache.backends.dummy.DummyCache', } } REST_FRAMEWORK = { 'DEFAULT_AUTHENTICATION_CLASSES': ( 'rest_framework.authentication.SessionAuthentication', ) }

The problem is.

Even if I don't think the problem is here, this is my UserViewSet:

api.py (important part)

 class UserViewSet( mixins.RetrieveModelMixin, mixins.UpdateModelMixin, mixins.ListModelMixin, viewsets.GenericViewSet ): queryset = User.objects.all() serializer_class = UserExpenseSerializer permission_classes = (IsAuthenticated, ) allowed_methods = ('GET', 'PATCH', 'OPTIONS', 'HEAD') def get_serializer_class(self): if self.action == 'retrieve': return UserExpenseSerializer return UserSerializer def get_queryset(self): if(self.action == 'list'): return User.objects.all() if self.request.user.is_superuser: return User.objects.all() if self.request.user.is_staff: return User.objects.exclude(is_superuser=True) return User.objects.filter(pk = self.request.user.id) def list(self, request): filter_obj = UsersFilter(self.request) users = filter_obj.do_query() extras = filter_obj.get_extras() serializer = UserSerializer(users, context={'request' : request}, many=True) return Response({'users' : serializer.data, 'extras' : extras}, views.status.HTTP_200_OK)

filters.py

 class UsersFilter: offset = 0 limit = 50 count = 0 total_pages = 0 filter_params = {} def __init__(self, request): if not request.user.is_superuser: self.filter_params['is_superuser'] = False if (not request.user.is_superuser and not request.user.is_staff): self.filter_params['pk'] = request.user.id # Read query params rpp = request.query_params.get('rpp') or 50 page = request.query_params.get('page') or 1 search_string = request.query_params.get('search') # Validate self.rpp = int(rpp) or 50 self.page = int(page) or 1 # Set filter set_if_not_none(self.filter_params, 'username__contains', search_string) # Count total results self.count = User.objects.filter(**self.filter_params).count() self.total_pages = int(self.count / self.rpp) + 1 # Set limits self.offset = (self.page - 1) * self.rpp self.limit = self.page * self.rpp def get_filter_params(self): return self.filter_params def get_offset(self): return self.offset def get_limit(self): return self.limit def do_query(self): users = User.objects.filter(**self.filter_params)[self.offset:self.limit] return users def get_query_info(self): query_info = { 'total_results' : self.count, 'results_per_page' : self.rpp, 'current_page' : self.page, 'total_pages' : self.total_pages } return query_info

UPDATE 2

As Linovia noted, the problem was not in the cache or any other DRF problem, but in the filter. Here's a fixed filter class:

 class UsersFilter: def __init__(self, request): self.filter_params = {} self.offset = 0 self.limit = 50 self.count = 0 self.total_pages = 0 self.extras = {} if not request.user.is_superuser: # and so long...
+6
source share
2 answers

In fact, you are creating a new user, which 2 users must make, and you state the length against 3. You are not going to work even without caching.

Edit: So you really have a problem because you are using mutables at the class level.

Here is the evil code:

 class UsersFilter: filter_params = {} def __init__(self, request): if not request.user.is_superuser: self.filter_params['is_superuser'] = False

In fact, it should be:

 class UsersFilter: def __init__(self, request): filter_params = {} if not request.user.is_superuser: self.filter_params['is_superuser'] = False

Otherwise, UsersFilter.filter_params will be stored from one request to another and will never be reset. See http://www.toptal.com/python/python-class-attributes-an-overly-thorough-guide for more on this.

+3
source

You can disable caching in debug mode by adding it to your settings.py

 if DEBUG: CACHES = { 'default': { 'BACKEND': 'django.core.cache.backends.dummy.DummyCache', } }

https://docs.djangoproject.com/en/dev/topics/cache/?from=olddocs/#dummy-caching-for-development

You can then disable caching by switching DEBUG to settings.py or by separately creating / testing and / deploying settings.py files.

If you want to avoid individual files or switches, you can set DEBUG to true for specific tests using the override_settings decorator:

 from django.test.utils import override_settings from django.test import TestCase from django.conf import settings class MyTest(TestCase): @override_settings(DEBUG=True) def test_debug(self): self.assertTrue(settings.DEBUG)
0
source

More articles:


All Articles