Geek Answers Handbook

Werkzeug and class state with Flask: how are class member variables reloaded when the class is not reinitialized?

I am trying to write a flask extension that should retain some information between requests. This works fine when I start Werkzeug using a single process, but when I start several processes, I get some strange behavior that I don't understand. Take this simple application as an example:

from flask import Flask app = Flask(__name__) class Counter(object): def __init__(self, app): print('initializing a Counter object') self.app = app self.value = 0 def increment(self): self.value += 1 print('Just incremented, current value is ', self.value) counter = Counter(app) @app.route('/') def index(): for i in range(4): counter.increment() return 'index' if __name__ == '__main__': #scenario 1 - single process #app.run() #scenario 2 - threaded #app.run(threaded=True) #scenario 3 - two processes app.run(processes=2)

For the first two scenarios, it works exactly as I expected: the Counter object is initialized once, and then it grows with each request for the '/' route. When I run it with the third script (process transfer = 2), I get this as output:

  initializing a Counter object * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit) Just incremented, current value is 1 Just incremented, current value is 2 Just incremented, current value is 3 Just incremented, current value is 4 127.0.0.1 - - [30/Aug/2015 09:47:25] "GET / HTTP/1.1" 200 - Just incremented, current value is 1 Just incremented, current value is 2 Just incremented, current value is 3 Just incremented, current value is 4 127.0.0.1 - - [30/Aug/2015 09:47:26] "GET / HTTP/1.1" 200 - Just incremented, current value is 1 Just incremented, current value is 2 Just incremented, current value is 3 Just incremented, current value is 4 127.0.0.1 - - [30/Aug/2015 09:47:27] "GET / HTTP/1.1" 200 -

Counter.value seems to return a state to it immediately after initialization without actually re-initializing. Can anyone shed light on what Werkzeug is doing internally for this to happen? I would also love to know if there is a way to do as I would naively expect (two processes, each with its own Counter instance). Thanks!

+5
source share
1 answer

In the first example (one thread), only one Counter , so it works.

The second example (multiple threads), threads are generated to process each request. They share memory with one Counter , which was created before they appear, so their increment from each increases the same.

The last example (several processes), processes are generated to process each request. Flask dev server uses fork : each child sees the same starting point (the counter is already initialized), but increases their own address space , which disappears when the request ends.

 import os class Counter: def __init__(self): print('init') self.value = 0 def increment(self): self.value += 1 print('inc -> {}'.format(self.value)) counter = Counter() def multi(): if not os.fork(): # child starts with copy of parent memory for _ in range(3): # increments three times counter.increment() # child is done os._exit(0) # three processes run for _ in range(3): multi() 
 init inc -> 1 inc -> 2 inc -> 3 inc -> 1 inc -> 2 inc -> 3 inc -> 1 inc -> 2 inc -> 3

Use a database or other external storage to store global state through processes using before_ and after_request . Please note that this is not so simple, since you will need to save the counter value from each request in streaming mode so that two streams do not overwrite the value at the same time.

 req 1 starts, gets stored value = 4 req 2 starts, gets stored value = 4 req 1 increments, value = 8 req 1 saves, value = 8 req 2 increments, value = 8 req 2 saves, value = 8 but should = 12
+4
source

More articles:


All Articles