Are python "global" (modular) variable threads local?

Question

I would like to use the local in-memory cache for a value from the database that will not change during the request / response cycle, but receives a call hundreds (potentially thousands) times. My limited understanding is that using a "global" / modular variable is one way to implement this type of cache.

eg:.

#somefile.py foo = None def get_foo(request): global foo if not foo: foo = get_foo_from_db(request.blah) return foo 

I am wondering if using this type is “global” thread safe in python, and therefore it may be convenient for me that get_foo_from_db () will be called exactly once in a django request / response cycle (using either server or gunicorn + GEvent). Do I understand correctly? This thing is called enough that even using memcached to store the value will be a bottleneck (I profile it when we talk).

Answer 1

No, you are mistaken for two reasons.

First, the use of "threads" is a bit vague here. Depending on how its server is configured, Django can be served either by threads, or by processes, or both (see the mod_wsgi documentation for a full discussion). If there is one thread for each process, you can guarantee that only one instance of the module will be available for each process. But it depends a lot on this configuration.

However, it is still not the case that during the request / response cycle there will be “exactly one” call to this function. This is due to the fact that the lifetime of the process is not completely associated with this cycle. The process will last for several requests, so the variable will be saved for all these requests.

Answer 2

No, access to global variables is not thread safe. Threads do not get their own copy of globals, global common to threads.

The code:

 if not foo: foo = get_foo_from_db(request.blah) 

compiles several python bytecode statements:

  2 0 LOAD_FAST 1 (foo) 3 POP_JUMP_IF_TRUE 24 3 6 LOAD_GLOBAL 0 (get_foo_from_db) 9 LOAD_FAST 0 (request) 12 LOAD_ATTR 1 (blah) 15 CALL_FUNCTION 1 18 STORE_FAST 1 (foo) 21 JUMP_FORWARD 0 (to 24) 

A thread switch may occur after each bytecode execution, so another thread may change foo after checking it.