Mongolian communications have never been released - Django and Mongoengine, working on a machine gun with gevent

Question

I have a django application using mongoengine running on gunicorn with working gevents. Under load, the number of Mongo compounds rises to about 3 thousand and never returns. Even after the load test is completed, the number of mongo joints remains constant. Restarting machine guns releases connections.

Package versions

gunicorn==0.17.4 mongoengine==0.8.7 pymongo==2.7 

mongodb 2.6.0

I have mongoengine connection settings in an environment-specific django settings file:

 MONGO_DATABASES = { 'default': { 'DB': '****', 'HOST': ***********:27017', 'PORT': 27017 } } from gevent import monkey monkey.patch_all() from mongoengine import connect connect(MONGO_DATABASES['default']['DB'], host=MONGO_DATABASES['default']['HOST'], port=MONGO_DATABASES['default']['PORT'], max_pool_size=100) 

Is there something I need to do to make sure that unused connections will eventually be released?

Thanks,

Arc

Answer 1

The function you are looking for is a configuration parameter called maxIdleTimeMS , which is designed to clean up after load surges (among other things).

According to the docs:

maxIdleTimeMS - the maximum time in milliseconds that a socket can sit idle before closing and discarding. Useful for cleaning after loading spike.

Unfortunately, this option is currently (March 2015) not available in the pymongo driver (which is used by mongoengine under the hood), but it works! See the relevant JIRA ticket (and don't forget to raise a question!). It should be available with pymongo 3.1 in a few months.