How to create a metaclass?

Question

I have a general understanding of metaclasses. These are the classes on which class objects are based (since classes are objects in Python). But someone can explain (with code) how this is done during creation.

Answer 1

There are (at this point) two key methods in the metaclass:

• __prepare__ and
• __new__

__prepare__ allows you to create a custom mapping (such as OrderedDict ) that will be used as the namespace during class creation. You must return an instance of any namespace that you select. If you do not implement __prepare__ , the usual dict .

__new__ is responsible for actually creating / modifying the final class.

The bare bones, the do-nothing-extra metaclass will look like this:

 class Meta(type): def __prepare__(metaclass, cls, bases): return dict() def __new__(metacls, cls, bases, clsdict): return super().__new__(metacls, cls, bases, clsdict) 

A simple example:

Suppose your attributes need some simple validation code. As always, it must be int or str . Without a metaclass, your class will look something like this:

 class Person: weight = ValidateType('weight', int) age = ValidateType('age', int) name = ValidateType('name', str) 

As you can see, you need to repeat the attribute name twice. This makes typos possible along with annoying errors.

A simple metaclass can solve this problem:

 class Person(metaclass=Validator): weight = ValidateType(int) age = ValidateType(int) name = ValidateType(str) 

This will look like a metaclass (not using __prepare__ , since it is not needed):

 class Validator(type): def __new__(metacls, cls, bases, clsdict): # search clsdict looking for ValidateType descriptors for name, attr in clsdict.items(): if isinstance(attr, ValidateType): attr.name = name attr.attr = '_' + name # create final class and return it return super().__new__(metacls, cls, bases, clsdict) 

Run Example:

 p = Person() p.weight = 9 print(p.weight) p.weight = '9' 

gives:

 9 Traceback (most recent call last): File "simple_meta.py", line 36, in <module> p.weight = '9' File "simple_meta.py", line 24, in __set__ (self.name, self.type, value)) TypeError: weight must be of type(s) <class 'int'> (got '9') 

---

Notes

This example is simple enough, it can also be executed using the class decorator, but the actual metaclass will probably do much more.

In Python 2.x, the __prepare__ method __prepare__ not exist, and the class specifies its metaclass with the class variable __metaclass__ = ... , for example:

 class Person(object): __metaclass__ = ValidateType 

Class "ValidateType" for reference:

 class ValidateType: def __init__(self, type): self.name = None # will be set by metaclass self.attr = None # will be set by metaclass self.type = type def __get__(self, inst, cls): if inst is None: return self else: return inst.__dict__[self.attr] def __set__(self, inst, value): if not isinstance(value, self.type): raise TypeError('%s must be of type(s) %s (got %r)' % (self.name, self.type, value)) else: inst.__dict__[self.attr] = value 

Answer 2

I just wrote a fully commented metaclass example. This is in Python 2.7. I am sharing this and I hope that it will help you better understand the __new__ , __init__ , __call__ , __dict__ and the concept of limited / unlimited in Python, as well as the use of metaclasses.

The problem with the metaclass, I feel, is that it has too many places where you can do the same things or similar, but with some slight differences . Therefore, my comments and test cases mainly emphasize where to write what, what goes to where at certain points, and what is available for a particular object.

The example tries to build a factory class while maintaining the correct class definitions.

 from pprint import pprint from types import DictType class FactoryMeta(type): """ Factory Metaclass """ # @ Anything "static" (bounded to the classes rather than the instances) # goes in here. Or use "@classmethod" decorator to bound it to meta. # @ Note that these members won't be visible to instances, you have to # manually add them to the instances in metaclass' __call__ if you wish # to access them through a instance directly (see below). extra = "default extra" count = 0 def clsVar(cls): print "Class member 'var': " + str(cls.var) @classmethod def metaVar(meta): print "Metaclass member 'var': " + str(meta.var) def __new__(meta, name, bases, dict): # @ Metaclass' __new__ serves as a bi-functional slot capable for # initiating the classes as well as alternating the meta. # @ Suggestion is putting majority of the class initialization code # in __init__, as you can directly reference to cls there; saving # here for anything you want to dynamically added to the meta (such # as shared variables or lazily GC'd temps). # @ Any changes here to dict will be visible to the new class and their # future instances, but won't affect the metaclass. While changes # directly through meta will be visible to all (unless you override # it later). dict['new_elem'] = "effective" meta.var = "Change made to %s by metaclass' __new__" % str(meta) meta.count += 1 print "================================================================" print " Metaclass __new__ (creates class objects)" print "----------------------------------------------------------------" print "Bounded to object: " + str(meta) print "Bounded object __dict__: " pprint(DictType(meta.__dict__), depth = 1) print "----------------------------------------------------------------" print "Parameter 'name': " + str(name) print "Parameter 'bases': " + str(bases) print "Parameter 'dict': " pprint(dict, depth = 1) print "\n" return super(FactoryMeta, meta).__new__(meta, name, bases, dict) def __init__(cls, name, bases, dict): # @ Metaclass' __init__ is the standard slot for class initialization. # Classes' common variables should mainly goes in here. # @ Any changes here to dict won't actually affect anything. While # changes directly through cls will be visible to the created class # and its future instances. Metaclass remains untouched. dict['init_elem'] = "defective" cls.var = "Change made to %s by metaclass' __init__" % str(cls) print "================================================================" print " Metaclass __init__ (initiates class objects)" print "----------------------------------------------------------------" print "Bounded to object: " + str(cls) print "Bounded object __dict__: " pprint(DictType(cls.__dict__), depth = 1) print "----------------------------------------------------------------" print "Parameter 'name': " + str(name) print "Parameter 'bases': " + str(bases) print "Parameter 'dict': " pprint(dict, depth = 1) print "\n" return super(FactoryMeta, cls).__init__(name, bases, dict) def __call__(cls, *args): # @ Metaclass' __call__ gets called when a class name is used as a # callable function to create an instance. It is called before the # class' __new__. # @ Instance initialization code can be put in here, although it # is bounded to "cls" rather than instance "self". This provides # a slot similar to the class' __new__, where cls' members can be # altered and get copied to the instances. # @ Any changes here through cls will be visible to the class and its # instances. Metaclass remains unchanged. cls.var = "Change made to %s by metaclass' __call__" % str(cls) # @ "Static" methods defined in the meta which cannot be seen through # instances by default can be manually assigned with an access point # here. This is a way to create shared methods between different # instances of the same metaclass. cls.metaVar = FactoryMeta.metaVar print "================================================================" print " Metaclass __call__ (initiates instance objects)" print "----------------------------------------------------------------" print "Bounded to object: " + str(cls) print "Bounded object __dict__: " pprint(DictType(cls.__dict__), depth = 1) print "\n" return super(FactoryMeta, cls).__call__(*args) class Factory(object): """ Factory Class """ # @ Anything declared here goes into the "dict" argument in the metaclass' # __new__ and __init__ methods. This provides a chance to pre-set the # member variables desired by the two methods, before they get run. # @ This also overrides the default values declared in the meta. __metaclass__ = FactoryMeta extra = "overridng extra" def selfVar(self): print "Instance member 'var': " + str(self.var) @classmethod def classFactory(cls, name, bases, dict): # @ With a factory method embedded, the Factory class can act like a # "class incubator" for generating other new classes. # @ The dict parameter here will later be passed to the metaclass' # __new__ and __init__, so it is the right place for setting up # member variables desired by these two methods. dict['class_id'] = cls.__metaclass__.count # An ID starts from 0. # @ Note that this dict is for the *factory product classes*. Using # metaclass as callable is another way of writing class definition, # with the flexibility of employing dynamically generated members # in this dict. # @ Class' member methods can be added dynamically by using the exec # keyword on dict. exec(cls.extra, dict) exec(dict['another_func'], dict) return cls.__metaclass__(name + ("_%02d" % dict['class_id']), bases, dict) def __new__(cls, function): # @ Class' __new__ "creates" the instances. # @ This won't affect the metaclass. But it does alter the class' member # as it is bounded to cls. cls.extra = function print "================================================================" print " Class' __new__ (\"creates\" instance objects)" print "----------------------------------------------------------------" print "Bounded to object: " + str(cls) print "Bounded object __dict__: " pprint(DictType(cls.__dict__), depth = 1) print "----------------------------------------------------------------" print "Parameter 'function': \n" + str(function) print "\n" return super(Factory, cls).__new__(cls) def __init__(self, function, *args, **kwargs): # @ Class' __init__ initializes the instances. # @ Changes through self here (normally) won't affect the class or the # metaclass; they are only visible locally to the instances. # @ However, here you have another chance to make "static" things # visible to the instances, "locally". self.classFactory = self.__class__.classFactory print "================================================================" print " Class' __init__ (initiates instance objects)" print "----------------------------------------------------------------" print "Bounded to object: " + str(self) print "Bounded object __dict__: " pprint(DictType(self.__dict__), depth = 1) print "----------------------------------------------------------------" print "Parameter 'function': \n" + str(function) print "\n" return super(Factory, self).__init__(*args, **kwargs) # @ The metaclass' __new__ and __init__ will be run at this point, where the # (manual) class definition hitting its end. # @ Note that if you have already defined everything well in a metaclass, the # class definition can go dummy with simply a class name and a "pass". # @ Moreover, if you use class factories extensively, your only use of a # manually defined class would be to define the incubator class. 

The result is as follows (for a better demonstration):

 ================================================================ Metaclass __new__ (creates class objects) ---------------------------------------------------------------- Bounded to object: <class '__main__.FactoryMeta'> Bounded object __dict__: { ..., 'clsVar': <function clsVar at 0x00000000029BC828>, 'count': 1, 'extra': 'default extra', 'metaVar': <classmethod object at 0x00000000029B4B28>, 'var': "Change made to <class '__main__.FactoryMeta'> by metaclass' __new__"} ---------------------------------------------------------------- Parameter 'name': Factory Parameter 'bases': (<type 'object'>,) Parameter 'dict': { ..., 'classFactory': <classmethod object at 0x00000000029B4DC8>, 'extra': 'overridng extra', 'new_elem': 'effective', 'selfVar': <function selfVar at 0x00000000029BC6D8>} ================================================================ Metaclass __init__ (initiates class objects) ---------------------------------------------------------------- Bounded to object: <class '__main__.Factory'> Bounded object __dict__: { ..., 'classFactory': <classmethod object at 0x00000000029B4DC8>, 'extra': 'overridng extra', 'new_elem': 'effective', 'selfVar': <function selfVar at 0x00000000029BC6D8>, 'var': "Change made to <class '__main__.Factory'> by metaclass' __init__"} ---------------------------------------------------------------- Parameter 'name': Factory Parameter 'bases': (<type 'object'>,) Parameter 'dict': { ..., 'classFactory': <classmethod object at 0x00000000029B4DC8>, 'extra': 'overridng extra', 'init_elem': 'defective', 'new_elem': 'effective', 'selfVar': <function selfVar at 0x00000000029BC6D8>} 

The calling sequence is the metaclass' __new__ , then its __init__ . __call__ will not be called at this time.

And if we create an instance,

 func1 = ( "def printElems(self):\n" " print \"Member new_elem: \" + self.new_elem\n" " print \"Member init_elem: \" + self.init_elem\n" ) factory = Factory(func1) 

Output:

 ================================================================ Metaclass __call__ (initiates instance objects) ---------------------------------------------------------------- Bounded to object: <class '__main__.Factory'> Bounded object __dict__: { ..., 'classFactory': <classmethod object at 0x00000000029B4DC8>, 'extra': 'overridng extra', 'metaVar': <bound method type.metaVar of <class '__main__.FactoryMeta'>>, 'new_elem': 'effective', 'selfVar': <function selfVar at 0x00000000029BC6D8>, 'var': "Change made to <class '__main__.Factory'> by metaclass' __call__"} ================================================================ Class' __new__ ("creates" instance objects) ---------------------------------------------------------------- Bounded to object: <class '__main__.Factory'> Bounded object __dict__: { ..., 'classFactory': <classmethod object at 0x00000000029B4DC8>, 'extra': 'def printElems(self):\n print "Member new_elem: " + self.new_elem\n print "Member init_elem: " + self.init_elem\n', 'metaVar': <bound method type.metaVar of <class '__main__.FactoryMeta'>>, 'new_elem': 'effective', 'selfVar': <function selfVar at 0x00000000029BC6D8>, 'var': "Change made to <class '__main__.Factory'> by metaclass' __call__"} ---------------------------------------------------------------- Parameter 'function': def printElems(self): print "Member new_elem: " + self.new_elem print "Member init_elem: " + self.init_elem ================================================================ Class' __init__ (initiates instance objects) ---------------------------------------------------------------- Bounded to object: <__main__.Factory object at 0x00000000029BB7B8> Bounded object __dict__: {'classFactory': <bound method FactoryMeta.classFactory of <class '__main__.Factory'>>} ---------------------------------------------------------------- Parameter 'function': def printElems(self): print "Member new_elem: " + self.new_elem print "Member init_elem: " + self.init_elem 

The metaclass ' __call__ first called, then class' __new__ and __init__ .

By comparing the printed elements of each object, you can find out when and where they were added or changed, just as I commented on the code.

I also run the following test cases:

 factory.clsVar() # Will raise exception Factory.clsVar() factory.metaVar() factory.selfVar() func2 = ( "@classmethod\n" "def printClassID(cls):\n" " print \"Class ID: %02d\" % cls.class_id\n" ) ProductClass1 = factory.classFactory("ProductClass", (object, ), { 'another_func': func2 }) product = ProductClass1() product.printClassID() product.printElems() # Will raise exception ProductClass2 = Factory.classFactory("ProductClass", (Factory, ), { 'another_func': "pass" }) ProductClass2.printClassID() # Will raise exception ProductClass3 = ProductClass2.classFactory("ProductClass", (object, ), { 'another_func': func2 }) 

What you can run yourself to see how it works.

Please note that I intentionally left the names of dynamically generated classes different from the names of the variables to which they were assigned. This means which names actually act.

One more note: I put “static” in quotation marks, which I call a concept similar to C ++, not a Python decorator. Traditionally, I am a C ++ programmer, so I still like to think in my own way.