ConfigParser and String interpolation with env variable

Question

I am a bit out of python syntax and I have a problem reading an .ini file with interpolated values.

this is my ini file:

 [DEFAULT] home=$HOME test_home=$home [test] test_1=$test_home/foo.csv test_2=$test_home/bar.csv 

These lines

 from ConfigParser import SafeConfigParser parser = SafeConfigParser() parser.read('config.ini') print parser.get('test', 'test_1') 

displays

 $test_home/foo.csv 

while i expect

 /Users/nkint/foo.csv 

EDIT:

I suggested that the $ syntax is implicitly included in what is called string interpolation (referring to manual ):

In addition to its core functionality, SafeConfigParser supports interpolation. This means that values ​​can contain formatting strings that refer to other values ​​in the same section or to values ​​in a special DEFAULT section.

But I am wrong. How to deal with this case?

Answer 1

First of all, according to the documentation, you should use %(test_home)s to interpolate test_home . In addition, the key is case insensitive, and you cannot use the HOME and HOME keys. Finally, you can use SafeConfigParser(os.environ) to account for your environment.

 from ConfigParser import SafeConfigParser import os parser = SafeConfigParser(os.environ) parser.read('config.ini') 

Where config.ini is

 [DEFAULT] test_home=%(HOME)s [test] test_1=%(test_home)s/foo.csv test_2=%(test_home)s/bar.csv 

Answer 2

You can write custom interpolation in the case of Python 3:

 import configparser import os class EnvInterpolation(configparser.BasicInterpolation): """Interpolation which expands environment variables in values.""" def before_get(self, parser, section, option, value, defaults): return os.path.expandvars(value) cfg = """ [section1] key = value my_path = $PATH """ config = configparser.ConfigParser(interpolation=EnvInterpolation()) config.read_string(cfg) print(config['section1']['my_path']) 

Answer 3

If you want to deploy some environment variables, you can do this using os.path.expandvars before StringIO stream:

 import ConfigParser import os import StringIO with open('config.ini', 'r') as cfg_file: cfg_txt = os.path.expandvars(cfg_file.read()) config = ConfigParser.ConfigParser() config.readfp(StringIO.StringIO(cfg_txt)) 

Answer 4

ConfigParser.get values are strings, even if you set the values ​​as an integer or True. But in ConfigParser there are getint , getfloat and getboolean .

settings.ini

 [default] home=/home/user/app tmp=%(home)s/tmp log=%(home)s/log sleep=10 debug=True 

configuration reader

 >>> from ConfigParser import SafeConfigParser >>> parser = SafeConfigParser() >>> parser.read('/home/user/app/settings.ini') >>> parser.get('defaut', 'home') '/home/user/app' >>> parser.get('defaut', 'tmp') '/home/user/app/tmp' >>> parser.getint('defaut', 'sleep') 10 >>> parser.getboolean('defaut', 'debug') True 

Edit

Indeed, you can get the name values ​​as environment var if you initialize SafeConfigParser with os.environ . Thanks for Michelle's answer.

Answer 5

The trick to correctly substituting environment variables is to use the $ {} syntax for environment variables:

 [DEFAULT] test_home=${HOME} [test] test_1=%(test_home)s/foo.csv test_2=%(test_home)s/bar.csv 

Answer 6

It seems that in the latest version 3.5.0 ConfigParser did not read env variables, so I ended up providing custom interpolation based on BasicInterpolation .

 class EnvInterpolation(BasicInterpolation): """Interpolation as implemented in the classic ConfigParser, plus it checks if the variable is provided as an environment one in uppercase. """ def _interpolate_some(self, parser, option, accum, rest, section, map, depth): rawval = parser.get(section, option, raw=True, fallback=rest) if depth > MAX_INTERPOLATION_DEPTH: raise InterpolationDepthError(option, section, rawval) while rest: p = rest.find("%") if p < 0: accum.append(rest) return if p > 0: accum.append(rest[:p]) rest = rest[p:] # p is no longer used c = rest[1:2] if c == "%": accum.append("%") rest = rest[2:] elif c == "(": m = self._KEYCRE.match(rest) if m is None: raise InterpolationSyntaxError(option, section, "bad interpolation variable reference %r" % rest) var = parser.optionxform(m.group(1)) rest = rest[m.end():] try: v = os.environ.get(var.upper()) if v is None: v = map[var] except KeyError: raise InterpolationMissingOptionError(option, section, rawval, var) from None if "%" in v: self._interpolate_some(parser, option, accum, v, section, map, depth + 1) else: accum.append(v) else: raise InterpolationSyntaxError( option, section, "'%%' must be followed by '%%' or '(', " "found: %r" % (rest,)) 

The difference between BasicInterpolation and EnvInterpolation is:

  v = os.environ.get(var.upper()) if v is None: v = map[var] 

where I try to find var in the environment before checking on map .

Answer 7

Pretty late, but maybe it can help someone else look for the same answers that I recently received. In addition, one of the comments was on how to get environment variables and values ​​from other sections. This is how I deal with the conversion of environment variables and multi-section tags when reading from an INI file.

INI FILE:

 [PKG] # <VARIABLE_NAME>=<VAR/PATH> PKG_TAG = Q1_RC1 [DELIVERY_DIRS] # <DIR_VARIABLE>=<PATH> NEW_DELIVERY_DIR=${DEL_PATH}\ProjectName_${PKG:PKG_TAG}_DELIVERY 

A Python class that uses ExtendedInterpolation so you can use formatting like ${PKG:PKG_TAG} . I am adding the ability to convert Windows environment variables when I read in INI into a string using the built-in os.path.expandvars() such as ${DEL_PATH} above.

 import os from configparser import ConfigParser, ExtendedInterpolation class ConfigParser(object): def __init__(self): """ initialize the file parser with ExtendedInterpolation to use ${Section:option} format [Section] option=variable """ self.config_parser = ConfigParser(interpolation=ExtendedInterpolation()) def read_ini_file(self, file='./config.ini'): """ Parses in the passed in INI file and converts any Windows environ vars. :param file: INI file to parse :return: void """ # Expands Windows environment variable paths with open(file, 'r') as cfg_file: cfg_txt = os.path.expandvars(cfg_file.read()) # Parses the expanded config string self.config_parser.read_string(cfg_txt) def get_config_items_by_section(self, section): """ Retrieves the configurations for a particular section :param section: INI file section :return: a list of name, value pairs for the options in the section """ return self.config_parser.items(section) def get_config_val(self, section, option): """ Get an option value for the named section. :param section: INI section :param option: option tag for desired value :return: Value of option tag """ return self.config_parser.get(section, option) @staticmethod def get_date(): """ Sets up a date formatted string. :return: Date string """ return datetime.now().strftime("%Y%b%d") def prepend_date_to_var(self, sect, option): """ Function that allows the ability to prepend a date to a section variable. :param sect: INI section to look for variable :param option: INI search variable under INI section :return: Void - Date is prepended to variable string in INI """ if self.config_parser.get(sect, option): var = self.config_parser.get(sect, option) var_with_date = var + '_' + self.get_date() self.config_parser.set(sect, option, var_with_date)