Flask + gevent - SSE time with nginx + uwsgi

Question

I am writing a webapp based on Flask, gevent and Redis that uses Server Sent Events.

I looked at a few questions about StackOverflow and advanced google search, but did not find a suitable answer that works for me, so here I am asking for help to the community.

The problem with the nginx + uwsgi production stack: the browser regularly receives updates (and updates as expected) for about 30 seconds. After that, the connection time ends, and the browser no longer receives updates until the page is manually reloaded.

Since all this works fine on the local host, with a standard flash development server (the connection continues after 30 minutes of inactivity), I am sure that the problem is with the uwsgi / nginx configuration. I tried all the nginx / uwsgi settings that I could think of, but nothing, it doesn't sync after a few seconds.

Does anyone have a key?

Here are some codes and configs.

nginx related performance settings:

location / { include uwsgi_params; uwsgi_pass unix:/tmp/myapp.sock; uwsgi_param UWSGI_PYHOME /srv/www/myapp/venv; uwsgi_param UWSGI_CHDIR /srv/www/myapp; uwsgi_param UWSGI_MODULE run; uwsgi_param UWSGI_CALLABLE app; uwsgi_buffering off; proxy_set_header Connection ''; proxy_http_version 1.1; chunked_transfer_encoding off; proxy_cache off; } 

Uwsgi performance settings

 [uwsgi] base = /srv/www/myapp app = run home = %(base)/venv pythonpath = %(base) socket = /tmp/%n.sock gevent = 100 module = %(app) callable = app logto = /srv/www/myapp-logs/uwsgi_%n.log 

this is the javascript that the template runs to subscribe to the channel (for now, the template just refreshes the entire page when the server pushes some data)

 <script type="text/javascript"> var eventOutputContainer = document.getElementById("event"); var evtSrc = new EventSource("/movers/monitor"); evtSrc.onmessage = function(e) { console.log(e.data); location.reload(); //eventOutputContainer.innerHTML = e.data; }; </script> 

This is the code I use to return streaming data.

 from myapp import redislist from flask import Response, Blueprint, stream_with_context movers = Blueprint('movers', __name__, url_prefix='/movers') r = redislist['r'] @movers.route("/monitor") def stream_movers(): def gen(): pubsub = r.pubsub() pubsub.subscribe('movers-real-time') for event in pubsub.listen(): if event['type'] == 'message': yield 'retry: 10000\n\ndata: %s\n\n' % event['data'] return Response(stream_with_context(gen()), direct_passthrough=True, mimetype="text/event-stream") 

and finally, the application runs as follows (DEBUG is True on the local host)

 from myapp import app from gevent.wsgi import WSGIServer if __name__ == '__main__': DEBUG = True if app.config['DEBUG'] else False if DEBUG: app.run(debug=DEBUG, threaded=True) app.debug = True server = WSGIServer(("", 5000), app) server.serve_forever() else: server = WSGIServer("", app) server.serve_forever() 

Answer 1

after long hours of working with nginx log files and the firefox js console, it turned out that the configurations shown in the question are fine.

The problem was reloading the page, this action kills and reinitializes the connection, so the retry command has no effect.

After removing this command, SSE updates work like a charm even after a long period of inactivity.

Now the question is why this worked for a simpler stack of development environment :-)

EDIT

indeed, after a few days the connection still does not work. I took some temporary measures and found out that the time interval is variable between 30 seconds and several minutes of inactivity.

My conclusion is that the above stack is good, and that is the amazon EC2 connection, which expires after some time of inactivity of the variable, since I still use the microinstance.

The latest fix is ​​the following JS snippet:

  evtSrc.onerror = function(e) { location.reload(); } 

page reload When the connection is dropped (regardless of the reason). Reboots are not expected to occur when the server sends events frequently.