Python late binding - dynamically locating locales in a scope

Question

i has an m_chain function that refers to two functions bind and unit that are not defined. I want to wrap this function in some context that provides definitions for these functions - you can represent them as interfaces for which I want to dynamically provide an implementation.

 def m_chain(*fns): """what this function does is not relevant to the question""" def m_chain_link(chain_expr, step): return lambda v: bind(chain_expr(v), step) return reduce(m_chain_link, fns, unit) 

In Clojure, this is done using macros. What are some elegant ways to do this in python? I thought:

• Polymorphism
• : turn m_chain into a method related to self.bind and self.unit , the implementation of which is provided by a subclass
• the implementation of the with interface, so I can change the environment map and then clear when I finish
• changing the m_chain signature to accept a unit and bind as arguments
• requiring the use of m_chain should be wrapped with a decorator that will do something or not - not sure if that even makes sense

Ideally, I don’t want to change m_chain at all, I want to use the definition as is, and all the options listed above require a change in the definition. This is very important because there are other m_ * functions that relate to additional functions that will be provided at runtime.

What is the best way for me to structure this so that I can pass the bind and unit implementations well? It’s important that the final use of m_chain is very easy to use, despite its complicated implementation.

edit: here is another approach that works, which is ugly, like all hell, because it requires m_chain to execute a function with no arguments. but this is a minimal working example.

 def domonad(monad, cmf): bind = monad['bind']; unit = monad['unit'] return cmf() identity_m = { 'bind':lambda v,f:f(v), 'unit':lambda v:v } maybe_m = { 'bind':lambda v,f:f(v) if v else None, 'unit':lambda v:v } >>> domonad(identity_m, lambda: m_chain(lambda x: 2*x, lambda x:2*x)(2)) 8 >>> domonad(maybe_m, lambda: m_chain(lambda x: None, lambda x:2*x)(2)) None 

Answer 1

Ok, here is my final answer to this question.

You should be able to re-perform some functions at least for a while. Your hacking, backing up the .__globals__ and pasting into new values, is ugly: slow, not thread safe, and specific to CPython. I thought about this, and there is no Pythonic solution that works this way.

In Python, you can double-check any function, but you have to do it explicitly, and some functions are not a good idea for re-checking. For example, I like the built-in all() and any() , and I think it would be scary if you could secretly double-check them, and that would not be obvious.

You want some functions to be rearranged, and I don't think you need all of them to be rearranged. Thus, it would be entirely advisable to tag related functions. The obvious and Pythonic way of doing this is to make them functions of a class method, which we can call Monad . You can use the standard variable name m for Monad instances, and then when someone tries to read and understand their code, they will know that a function with a name like m.unit() can potentially be reconnected through some other Monad instance instance.

It will be pure Python and fully portable if you follow these rules:

• All functions must be connected in a monad. If you reference m.bind() , then "bind" should appear in the .__dict__ instance of Monad .
• Functions using Monad must accept the argument name m= or, for functions that use the *args function, they must accept the **kwargs argument and check it against a key named "m" .

Here is an example of what I mean.

 class Monad(object): def __init__(self, *args, **kwargs): # init from each arg. Try three things: # 0) if it has a ".__dict__" attribute, update from that. # 1) if it looks like a key/value tuple, insert value for key. # 2) else, just see if the whole thing is a dict or similar. # Other instances of class Monad() will be handled by (0) for x in args: if hasattr("__dict__", x): self.__dict__.update(x.__dict__) else: try: key, value = x self.__dict__[key] = value except TypeError: self.__dict__.update(x) self.__dict__.update(kwargs) def __identity(x): return x def __callt(v, f): return f(v) def __callt_maybe(v, f): if v: return f(v) else: return None m_identity = Monad(bind=__callt, unit=__identity) m_maybe = Monad(bind=__callt_maybe, unit=__identity) def m_chain(*fns, **kwargs): """what this function does is not relevant to the question""" m = kwargs.get("m", m_identity) def m_chain_link(chain_expr, step): return lambda v: m.bind(chain_expr(v), step) return reduce(m_chain_link, fns, m.unit) print(m_chain(lambda x: 2*x, lambda x:2*x, m=m_identity)(2)) # prints 8 print(m_chain(lambda x: None, lambda x:2*x, m=m_maybe)(2)) # prints None 

The above is clean, Pythonic, and should work just as well as IronPython, Jython or PyPy, as it does in CPython. Inside m_chain() expression m = kwargs.get("m", m_identity) tries to read the given monad argument; if not found, the monad is set to m_identity .

But you may need more. You may want the Monad class to support only optional function name overrides; and you may want to stick with just CPython. Here is a more complicated version above. In this version, when the expression m.some_name() is evaluated, if the Monad m instance does not have the name some_name associated with its .__dict__ , it will look for some_name in the caller’s locales and in globals() .

In this case, the expression m.some_name() means that " m may override some_name , but not necessarily; some_name may not be in m , in which case some_name will be checked as if it were not a prefix m. ". The magic is in the .__getattr__() function, which uses sys._getframe() to view the caller in the locales. .__getattr__() is only called when the local search fails, so we know that the Monad instance does not have the name associated with .__dict__ ; so look at the locals belonging to the caller using sys._getframe(1).f_locals ; otherwise, look at globals() . Just paste this into the Monad class definition in the source code above.

 def __getattr__(self, name): # if __getattr__() is being called, locals() were already checked d = sys._getframe(1).f_locals if name in d: return d[name] d = globals() if name in d: return d[name] mesg = "name '%s' not found in monad, locals, or globals" % name raise NameError, mesg 

Answer 2

In Python, you can write all the code you want that relates to what does not exist; To be specific, you can write code that refers to names that have no values ​​attached to them. And you can compile this code. The only problem will occur at runtime if the names still don't have values ​​attached to them.

Here is an example of code that you can run tested under Python 2 and Python 3.

 def my_func(a, b): return foo(a) + bar(b) try: my_func(1, 2) except NameError: print("didn't work") # name "foo" not bound # bind name "foo" as a function def foo(a): return a**2 # bind name "bar" as a function def bar(b): return b * 3 print(my_func(1, 2)) # prints 7 

If you don't want the names to be just anchored in the local namespace, but you want to be able to fine-tune them for each function, I think the best practice in Python would be to use named arguments. You can always close function arguments and return a new function object as follows:

 def my_func_factory(foo, bar): def my_func(a, b): return foo(a) + bar(b) return my_func my_func0 = my_func_factory(lambda x: 2*x, lambda x:2*x) print(my_func0(1, 2)) # prints 6 

EDIT: Here is your example modified using the above idea.

 def domonad(monad, *cmf): def m_chain(fns, bind=monad['bind'], unit=monad['unit']): """what this function does is not relevant to the question""" def m_chain_link(chain_expr, step): return lambda v: bind(chain_expr(v), step) return reduce(m_chain_link, fns, unit) return m_chain(cmf) identity_m = { 'bind':lambda v,f:f(v), 'unit':lambda v:v } maybe_m = { 'bind':lambda v,f:f(v) if v else None, 'unit':lambda v:v } print(domonad(identity_m, lambda x: 2*x, lambda x:2*x)(2)) # prints 8 print(domonad(maybe_m, lambda x: None, lambda x:2*x)(2)) # prints None 

Please let me know how this works for you.

EDIT: Alright, another version after your comment. You can write arbitrary m_ functions following this pattern: they check kwargs for the "monad" key. This should be specified as a named argument; There is no way to pass it as a positional argument because of the *fns argument, which collects all the arguments into a list. I provided default values ​​for bind() and unit() if they are not defined in the monad, or the monad is not provided; they probably don't do what you want, so replace them with something better.

 def m_chain(*fns, **kwargs): """what this function does is not relevant to the question""" def bind(v, f): # default bind if not in monad return f(v), def unit(v): # default unit if not in monad return v if "monad" in kwargs: monad = kwargs["monad"] bind = monad.get("bind", bind) unit = monad.get("unit", unit) def m_chain_link(chain_expr, step): return lambda v: bind(chain_expr(v), step) return reduce(m_chain_link, fns, unit) def domonad(fn, *fns, **kwargs): return fn(*fns, **kwargs) identity_m = { 'bind':lambda v,f:f(v), 'unit':lambda v:v } maybe_m = { 'bind':lambda v,f:f(v) if v else None, 'unit':lambda v:v } print(domonad(m_chain, lambda x: 2*x, lambda x:2*x, monad=identity_m)(2)) print(domonad(m_chain, lambda x: None, lambda x:2*x, monad=maybe_m)(2)) 

Answer 3

this is how i did it. I don’t know if this is a good idea. but it allows me to write my m_ * functions, completely independent of the unit / bind implementation, and also completely independent of any implementation details of the monads method in python. the right things are only in the lexical sphere.

 class monad: """Effectively, put the monad definition in lexical scope. Can't modify the execution environment `globals()` directly, because after globals().clear() you can't do anything. """ def __init__(self, monad): self.monad = monad self.oldglobals = {} def __enter__(self): for k in self.monad: if k in globals(): self.oldglobals[k]=globals()[k] globals()[k]=self.monad[k] def __exit__(self, type, value, traceback): """careful to distinguish between None and undefined. remove the values we added, then restore the old value only if it ever existed""" for k in self.monad: del globals()[k] for k in self.oldglobals: globals()[k]=self.oldglobals[k] def m_chain(*fns): """returns a function of one argument which performs the monadic composition of fns.""" def m_chain_link(chain_expr, step): return lambda v: bind(chain_expr(v), step) return reduce(m_chain_link, fns, unit) identity_m = { 'bind':lambda v,f:f(v), 'unit':lambda v:v } with monad(identity_m): assert m_chain(lambda x:2*x, lambda x:2*x)(2) == 8 maybe_m = { 'bind':lambda v,f:f(v) if v else None, 'unit':lambda v:v } with monad(maybe_m): assert m_chain(lambda x:2*x, lambda x:2*x)(2) == 8 assert m_chain(lambda x:None, lambda x:2*x)(2) == None error_m = { 'bind':lambda mv, mf: mf(mv[0]) if mv[0] else mv, 'unit':lambda v: (v, None) } with monad(error_m): success = lambda val: unit(val) failure = lambda err: (None, err) assert m_chain(lambda x:success(2*x), lambda x:success(2*x))(2) == (8, None) assert m_chain(lambda x:failure("error"), lambda x:success(2*x))(2) == (None, "error") assert m_chain(lambda x:success(2*x), lambda x:failure("error"))(2) == (None, "error") from itertools import chain def flatten(listOfLists): "Flatten one level of nesting" return list(chain.from_iterable(listOfLists)) list_m = { 'unit': lambda v: [v], 'bind': lambda mv, mf: flatten(map(mf, mv)) } def chessboard(): ranks = list("abcdefgh") files = list("12345678") with monad(list_m): return bind(ranks, lambda rank: bind(files, lambda file: unit((rank, file)))) assert len(chessboard()) == 64 assert chessboard()[:3] == [('a', '1'), ('a', '2'), ('a', '3')] 

Answer 4

Python is already limited. There is no need to do any work:

 def m_chain(*args): return bind(args[0]) sourcemodulename = 'foo' sourcemodule = __import__(sourcemodulename) bind = sourcemodule.bind print m_chain(3)