Like a "source" file in python script

Question

I have a text file /etc/default/foo that contains one line:

 FOO="/path/to/foo" 

In my python script, I need to reference the FOO variable.

What is the easiest way to "pass" the file /etc/default/foo to my python script, like in bash?

 . /etc/default/foo 

Answer 1

You can use execfile :

 execfile("/etc/default/foo") 

But please keep in mind that this will evaluate the contents of the file as is in your program source. This is a potential security risk if you cannot fully trust the source.

It also means that the file must be valid python syntax (your given example file).

Answer 2

If you know for sure that it contains only VAR="QUOTED STRING" style variables, for example:

 FOO="some value" 

Then you can simply do this:

 >>> with open('foo.sysconfig') as fd: ... exec(fd.read()) 

What you need:

 >>> FOO 'some value' 

(This is actually the same as the execfile() solution in another answer.)

This method has significant safety implications; if instead of FOO="some value" your file contains:

 os.system("rm -rf /") 

Then you would be in Trouble.

Alternatively, you can do this:

 >>> with open('foo.sysconfig') as fd: ... settings = {var: shlex.split(value) for var, value in [line.split('=', 1) for line in fd]} 

Which gives you the settings dictionary, which has:

 >>> settings {'FOO': ['some value']} 

This line of settings = {...} uses dictionary understanding . You can do the same thing on multiple lines with a for loop, etc.

And, of course, if the file contains a shell-type variable extension, for example ${somevar:-value_if_not_set} , then this will not work (unless you write your own shell variable parser).

Answer 3

To give a different approach, note that if your source file is configured as

 export FOO=/path/to/foo 

You can do source /etc/default/foo; python myprogram.py source /etc/default/foo; python myprogram.py (or . /etc/default/foo; python myprogram.py ) and inside myprogram.py all the values ​​exported to the sourced file are visible in os.environ, for example

 import os os.environ["FOO"] 

Answer 4

Keep in mind that if you have a text file with this content that has .py as the file extension, you can always:

 import mytextfile print(mytestfile.FOO) 

Of course, this assumes the text file is syntactically correct with respect to Python. In the project I was working on, we did something similar to this. Turned some text files into Python files. Awful, but maybe worth considering.

Answer 5

There are several ways to do such things.

• You really can import file as a module if the data contained in it matches the python syntax. But any file in question is a .py in the same directory as your script, or you must use imp (or importlib , depending on your version) as here .

• Another solution (which has my preferences) could be to use a data format that any python library can use (the JSON example comes to my mind).

/ etc / default / foo:

 {"FOO":"path/to/foo"} 

And in your Python code:

 import json with open('/etc/default/foo') as file: data = json.load(file) FOO = data["FOO"] ## ... file.close() 

This way you do not run the risk of executing some undefined code ...

You have a choice, depending on what you prefer. If your data file is auto-generated using some script, it might be easier to save a simple syntax like FOO="path/to/foo" and use imp .

Hope this helps!

Answer 6

Decision

Here is my approach: parse the bash file itself and process only the variable destination lines, such as:

 FOO="/path/to/foo" 

Here is the code:

 import shlex def parse_shell_var(line): """ Parse such lines as: FOO="My variable foo" :return: a tuple of var name and var value, such as ('FOO', 'My variable foo') """ return shlex.split(line, posix=True)[0].split('=', 1) if __name__ == '__main__': with open('shell_vars.sh') as f: shell_vars = dict(parse_shell_var(line) for line in f if '=' in line) print(shell_vars) 

How it works

Take a look at this snippet:

  shell_vars = dict(parse_shell_var(line) for line in f if '=' in line) 

This line is repeated through the lines in the shell script, processes only those lines that have an equal sign (not a reliable way to define variables, but the easiest). Then run these lines in the parse_shell_var function, which uses shlex.split to correctly handle quotes (or lack thereof). Finally, the parts are compiled into a dictionary. The result of this script is:

 {'MOO': '/dont/have/a/cow', 'FOO': 'my variable foo', 'BAR': 'My variable bar'} 

Here is the contents of shell_vars.sh:

 FOO='my variable foo' BAR="My variable bar" MOO=/dont/have/a/cow echo $FOO 

Discussion

This approach has several advantages:

• It does not execute a shell (either in bash or Python), which avoids any side effect
• Therefore, it is safe to use even if the source of the shell script is unknown.
• It correctly processes values ​​with or without quotation marks

This approach is not ideal, it has several limitations:

• The method of detecting variable assignment (by searching for the presence of an equal sign) is primitive and not accurate. There are ways to better define these lines, but this is a topic for another day.

• It incorrectly parses values ​​built on other variables or commands. This means that it will not work with strings such as:

   FOO=$BAR FOO=$(pwd)