Import a python module over the Internet / multiple protocols or dynamically create a module

Question

Is it possible to use the Python import module over the Internet using http ( s ), ftp , smb or any other protocol? If so, how? If not, why?

I assume that Python uses more than one protocol (reading the file system) and allows it to use others. Yes, I agree that it will be much faster, but some optimizations and large future bandwidths will certainly balance it.

eg:.

 import site site.addsitedir("https://bitbucket.org/zzzeek/sqlalchemy/src/e8167548429b9d4937caaa09740ffe9bdab1ef61/lib") import sqlalchemy import sqlalchemy.engine 

Answer 1

Basically, yes, but all the tools built into any support go through the file system.

To do this, you will need to download the source from anywhere, compile it using compile and exec using the __dict__ new module. See below.

I left actually exciting text from the internet and parsing uris etc. as an exercise for the reader (for beginners: I suggest using requests )

In pep 302, this will be the implementation of the loader.load_module function (the parameters are different). You can learn more about this in this import statement.

 import imp modulesource = 'a=1;b=2' #load from internet or wherever def makemodule(modulesource,sourcestr='http://some/url/or/whatever',modname=None): #if loading from the internet, you'd probably want to parse the uri, # and use the last part as the modulename. It'll come up in tracebacks # and the like. if not modname: modname = 'newmodulename' #must be exec mode # every module needs a source to be identified, can be any value # but if loading from the internet, you'd use the URI codeobj = compile(modulesource, sourcestr, 'exec') newmodule = imp.new_module(modname) exec(codeobj,newmodule.__dict__) return newmodule newmodule = makemodule(modulesource) print(newmodule.a) 

At this point, newmodule already a module object in scope, so you do not need to import it or anything else.

 modulesource = ''' a = 'foo' def myfun(astr): return a + astr ''' newmod = makemodule(modulesource) print(newmod.myfun('bat')) 

Ideal here: http://ideone.com/dXGziO

Tested with python 2, should work with python 3 (uses text compatible printing, uses function-like syntax).

Answer 2

Another version,

I like this answer. when using it, I simplified it a bit - it looks like the look of javascript , including more than HTTP .

This is the result:

 import os import imp import requests def import_cdn(uri, name=None): if not name: name = os.path.basename(uri).lower().rstrip('.py') r = requests.get(uri) r.raise_for_status() codeobj = compile(r.content, uri, 'exec') module = imp.new_module(name) exec (codeobj, module.__dict__) return module 

Application:

 redisdl = import_cdn("https://raw.githubusercontent.com/p/redis-dump-load/master/redisdl.py") # Regular usage of the dynamic included library json_text = redisdl.dumps(host='127.0.0.1') 

• Tip - put the import_cdn function in a shared library so you can reuse this little function
• Keep in mind that it will not succeed if you are not connected to this file via http

Answer 3

This seems to be a precedent for a self-starting import hook. See PEP 302 exactly how they work.

Essentially, you will need to provide a crawler object, which in turn provides a loader object. I do not understand this process at first glance (otherwise I would have become more explicit), but PEP contains all the necessary details for the implementation of this material.

Answer 4

Take a look at: [ https://github.com/Asmeble/Languages/raw/Documentation-of-5/13/2018-%22What-you-think-is-left-for-indifference-has-yet-to-shape -or-be-shaped% 22 / Python3.6 / URL_import.py] I use it to import code from my github to develop projects elsewhere.

Although it would be useful for game developers or web application designers, updating the code base will not take much time, even for installation or removal.

Answer 5

Like glglgl , this import hook was implemented for Python2 and Python3 in a module called httpimport . It uses a custom search / download object to search for resources using HTTP / S.

In addition, the import_cdn function in the answer of Yosef Harush is almost equally implemented in the httpimport github_repo and bitbucket_repo .

@ Marcin's answer contains a good part of the httpimport bootloader class code.