Dynamically load python source code

Question

I am currently playing with Flask and I cannot understand how the debugging mechanism works. To be more precise, when I save the python file with my application, I do not need to restart the server, it will load automatically when I make a request. So my question is, how does a running program know that it has been modified and responds to these changes?

Answer 1

The checkbox uses Werkzug, which is the basis of the run_with_reloader function (found in serving.py ) ... which itself uses the restart_with_reloader and reloader_loop functions created earlier in the same file.

run_with_reloader starts another python process (starts Werkzug again with all the same arguments that you passed to the first), and these new processes use the thread module to create a new thread or subprocess that runs your server function. Then it starts reloader_loop and waits.

reloader_loop simply reloader_loop over all the modules that have been imported and gets the last modified dates. Then, at certain intervals (the default is 1 s), he again checks all the files to see if they have been changed. If they are, the currently running (subordinate) Werkzug process is terminated (terminated) by exit code 3. After it is complete, the thread or subprocess that it starts (which actually does the job) is terminated. The master process checks to see if the exit code was 3. If so, it starts a new slave subprocess, as before. Otherwise, it exits with the same exit code.

Here is the code for reference:

 def reloader_loop(extra_files=None, interval=1): """When this function is run from the main thread, it will force other threads to exit when any modules currently loaded change. Copyright notice. This function is based on the autoreload.py from the CherryPy trac which originated from WSGIKit which is now dead. :param extra_files: a list of additional files it should watch. """ def iter_module_files(): for module in sys.modules.values(): filename = getattr(module, '__file__', None) if filename: old = None while not os.path.isfile(filename): old = filename filename = os.path.dirname(filename) if filename == old: break else: if filename[-4:] in ('.pyc', '.pyo'): filename = filename[:-1] yield filename mtimes = {} while 1: for filename in chain(iter_module_files(), extra_files or ()): try: mtime = os.stat(filename).st_mtime except OSError: continue old_time = mtimes.get(filename) if old_time is None: mtimes[filename] = mtime continue elif mtime > old_time: _log('info', ' * Detected change in %r, reloading' % filename) sys.exit(3) time.sleep(interval) def restart_with_reloader(): """Spawn a new Python interpreter with the same arguments as this one, but running the reloader thread. """ while 1: _log('info', ' * Restarting with reloader...') args = [sys.executable] + sys.argv new_environ = os.environ.copy() new_environ['WERKZEUG_RUN_MAIN'] = 'true' # a weird bug on windows. sometimes unicode strings end up in the # environment and subprocess.call does not like this, encode them # to latin1 and continue. if os.name == 'nt': for key, value in new_environ.iteritems(): if isinstance(value, unicode): new_environ[key] = value.encode('iso-8859-1') exit_code = subprocess.call(args, env=new_environ) if exit_code != 3: return exit_code def run_with_reloader(main_func, extra_files=None, interval=1): """Run the given function in an independent python interpreter.""" if os.environ.get('WERKZEUG_RUN_MAIN') == 'true': thread.start_new_thread(main_func, ()) try: reloader_loop(extra_files, interval) except KeyboardInterrupt: return try: sys.exit(restart_with_reloader()) except KeyboardInterrupt: pass 

Answer 2

The built-in reload() function can do this for you. This function is probably behind what Flask does to reload the code (noticing that it has somehow changed on disk).

Question How do I unload (reload) a Python module? has more details about this.