Most Pythonic way to provide global configuration variables in config.py?

Question

In my endless search for overcomplicating simple things, I study the most “Putin's” way of providing global configuration variables inside the typical “ config.py ” found in Python egg packages.

The traditional way (aah, good ol '#define!) Is as follows:

MYSQL_PORT = 3306 MYSQL_DATABASE = 'mydb' MYSQL_DATABASE_TABLES = ['tb_users', 'tb_groups'] 

Therefore, global variables are imported in one of the following ways:

 from config import * dbname = MYSQL_DATABASE for table in MYSQL_DATABASE_TABLES: print table 

or

 import config dbname = config.MYSQL_DATABASE assert(isinstance(config.MYSQL_PORT, int)) 

This makes sense, but can sometimes be a little messy, especially when you are trying to remember the names of certain variables. In addition, providing the 'configuration of an object as attributes can be more flexible. So, taking the lead from the bpython config.py configuration file, I came up with:

 class Struct(object): def __init__(self, *args): self.__header__ = str(args[0]) if args else None def __repr__(self): if self.__header__ is None: return super(Struct, self).__repr__() return self.__header__ def next(self): """ Fake iteration functionality. """ raise StopIteration def __iter__(self): """ Fake iteration functionality. We skip magic attribues and Structs, and return the rest. """ ks = self.__dict__.keys() for k in ks: if not k.startswith('__') and not isinstance(k, Struct): yield getattr(self, k) def __len__(self): """ Don't count magic attributes or Structs. """ ks = self.__dict__.keys() return len([k for k in ks if not k.startswith('__')\ and not isinstance(k, Struct)]) 

and "config.py", which imports the class and reads as follows:

 from _config import Struct as Section mysql = Section("MySQL specific configuration") mysql.user = 'root' mysql.pass = 'secret' mysql.host = 'localhost' mysql.port = 3306 mysql.database = 'mydb' mysql.tables = Section("Tables for 'mydb'") mysql.tables.users = 'tb_users' mysql.tables.groups = 'tb_groups' 

and used this way:

 from sqlalchemy import MetaData, Table import config as CONFIG assert(isinstance(CONFIG.mysql.port, int)) mdata = MetaData( "mysql://%s:%s@%s:%d/%s" % ( CONFIG.mysql.user, CONFIG.mysql.pass, CONFIG.mysql.host, CONFIG.mysql.port, CONFIG.mysql.database, ) ) tables = [] for name in CONFIG.mysql.tables: tables.append(Table(name, mdata, autoload=True)) 

What seems to be a more readable, expressive, and flexible way to store and retrieve global variables within a package.

Best known idea? What is the best practice for dealing with these situations? What is your way of storing and choosing global names and variables inside your package?

Answer 1

I did it once. I ended up finding my simplified basicconfig.py suited to my needs. You can walk in the namespace with other objects for reference if you need to. You can also pass additional default values ​​from your code. It also maps the attribute style and display syntax to the same configuration object.

Answer 2

How to use only built-in types:

 config = { "mysql": { "user": "root", "pass": "secret", "tables": { "users": "tb_users" } # etc } } 

You will get access to the following values:

 config["mysql"]["tables"]["users"] 

---

If you are willing to sacrifice the potential for evaluating expressions within your configuration tree, you can use YAML and end up with a more readable configuration file as follows:

 mysql: - user: root - pass: secret - tables: - users: tb_users 

and use a library like PyYAML to simplify the analysis and access to the configuration file

Answer 3

Similar to blubb request. I like the built-in types. I suggest creating them using lambda functions if you can. Like this:

 mkDict = lambda passwd, hair, name: {'passwd':passwd, 'hair':hair, 'name':name} #Col Names: Password Hair Color Real Name config = {'st3v3' : mkDict('password', 'blonde', 'Steve Booker'), 'blubb' : mkDict('12345678', 'black', 'Bubb Ohaal'), 'suprM' : mkDict('kryptonite', 'black', 'Clark Kent'), #... } 

Yes, now you do not need to copy so much. With comments, it is also easier to compare and read data later.

Answer 4

How about using classes?

 # config.py class MYSQL: PORT = 3306 DATABASE = 'mydb' DATABASE_TABLES = ['tb_users', 'tb_groups'] # main.py from config import MYSQL print(MYSQL.PORT) # 3306 

Answer 5

I like this solution for small applications:

 class App: __conf = { "username": "", "password": "", "MYSQL_PORT": 3306, "MYSQL_DATABASE": 'mydb', "MYSQL_DATABASE_TABLES": ['tb_users', 'tb_groups'] } __setters = ["username", "password"] @staticmethod def config(name): return App.__conf[name] @staticmethod def set(name, value): if name in App.__setters: App.__conf[name] = value else: raise NameError("Name not accepted in set() method") 

And then the following is used:

 if __name__ == "__main__": # from config import App App.config("MYSQL_PORT") # return 3306 App.set("username", "hi") # set new username value App.config("username") # return "hi" App.set("MYSQL_PORT", "abc") # this raises NameError 

.. you will like it because:

• uses class variables (no object to bypass / missing a single element),
• uses encapsulated built-in types and looks like () a method call on the App ,
• has control over the individual immutability configuration; mutable global variables are the worst kind of global variables.
• supports regular and well-named access / readability in source code
• is a simple class, but provides structured access , an alternative is to use @property , but this requires more processing code variable for each element and is based on objects.
• requires minimal changes to add new configuration items and establish its variability.

- Edit - : For large applications, storing values ​​in a YAML file (i.e. properties) and reading, which as immutable data, is the best approach (i.e. blubb / ohaal answer ). For small applications, the solution above is simpler.

Answer 6

A small variation of the Husky idea that I use. Create a file called "globals" (or whatever you like), and then define several classes in it:

 #globals.py class dbinfo : # for database globals username = 'abcd' password = 'xyz' class runtime : debug = False output = 'stdio' 

Then, if you have two code files c1.py and c2.py, both can have a top

 import globals as gl 

Now all the code can access and set the values ​​as such:

 gl.runtime.debug = False print(gl.dbinfo.username) 

People forget that classes exist even if no object is ever created that is a member of this class. And variables in the class that are not preceded by "I". shared in all instances of the class, even if they are not. After "debugging" is modified by any code, all other code will see the change.

When importing it as gl, you can have several files and variables that allow you to get and set values ​​between code files, functions, etc., but without the risk of a namespace collision.

This lacks some clever error checking of other approaches, but is simple and easy to follow.

Answer 7

please check IPython configuration system implemented via traitsts to force input of the type you do manually.

Cut and paste here to follow SO guidelines not only for removing links, as the contents of links change over time.

trace documentation

Here are the basic requirements that we would like our configuration system to have:

Support for hierarchical configuration information.

Full integration with parsers of command line parameters. Often you want to read a configuration file, but then override some values ​​with command line options. Our configuration system automates this process and allows you to associate each command line option with a specific attribute in the configuration hierarchy that will be canceled.

Configuration files, which themselves are valid Python code. It does a lot. First, it becomes possible to put logic into your configuration files, which sets attributes based on your operating system, network settings, Python version, etc. Secondly, Python has a super simple syntax for accessing hierarchical data structures, namely regular access to attributes (Foo. Bar.Bam.name). Third, using Python makes it easy for users to import configuration attributes from one configuration file to another. Fourth, although Python is dynamically typed, it has types that can be checked at runtime. Thus, 1 in the configuration file is an integer of '1, and' 1 'is a string.

A fully automated method for obtaining configuration information for classes that need it at runtime. Writing code that runs a configuration hierarchy to retrieve a specific attribute is painful. When you have complex configuration information with hundreds of attributes, it makes you cry.

Validation and validation of types that do not require the entire configuration hierarchy to be specified statically before running. Python is a very dynamic language, and you do not always know everything that you need to configure when you run the program.

To achieve this, they basically define 3 classes of objects and their relationship to each other:

1) Configuration - mainly ChainMap / basic dict with some improvements for merging.

2) Custom - the base class for a subclass of all the things you want to customize.

3) Application - an object that is created to perform a specific application function or your main application for single-purpose software.

According to them:

Application: Application

An application is a process that performs a specific task. The most obvious application is the ipython command-line tool. Each application reads one or more configuration files and one set of command-line options, and then creates the main configuration object for the application. This configuration object is then passed to the custom objects created by the application. These custom objects implement the actual logic of the application and know how to configure themselves based on the configuration object.

Applications always have a log attribute that is configured by Logger. This allows a centralized logging configuration for each application. Configurable: customizable

Configurable is a regular Python class that serves as the base class for all the main classes in the application. The custom base class is lightweight and does just one thing.

This Configurable is a subclass of HasTraits that knows how to configure itself. Class-level values ​​with config = True metadata become values ​​that can be configured from the command line and configuration files.

Developers create custom subclasses that implement the entire application logic. Each of these subclasses has its own configuration information that governs instantiation.