At what point does the configuration file become a programming language?

Question

I pondered the configuration files and their relationship to the code for a while, and depending on the day and direction of the wind, my opinions seem to change. More and more, although I keep returning to the awareness that I had when learning Lisp: there are few differences between data and code. In configuration files, this looks doubly. When I looked at the right light, the Perl script is slightly larger than the configuration file for perl. This, as a rule, has rather serious consequences for tasks such as QA and labor departments, for example, who should be responsible for changing configuration files.

Creep from a configuration file into a full-fledged language is usually slow and, apparently, due to the desire to have a common system. Most projects seem to start with small configuration items, such as where to write logs, where to look for data, usernames and passwords, etc. But then they begin to grow: functions begin to be turned on or off, the time and order of operations begin to be controlled, and, inevitably, someone wants to start adding logic to it (for example, use 10 if the machine is X and 15 if the machine is Y) . At some point, the configuration file becomes a domain-specific language and is poorly written.

Now that I’ve been on stage, here are my questions:

• What is the true purpose of the file configuration?
• Should I try to keep the configuration files simple?
• Who should be responsible for changes in them (developers, users, administrators, etc.)?
• If they are under source control (see question 3)?

As I said, my answers to these questions are constantly changing, but now I think:

• to allow non programmers to change large pieces of behavior quickly
• yes, everything that is not coarse grainy should be in code
• users should be responsible for the configuration files, and programmers should be responsible for the configuration layer between the configuration files and the code, which gives finer-grained control of the application
• no, but the finer middle layer should be

Answer 1

Very interesting questions!

I tend to limit my configuration files to the very simple "key = value" because I totally agree with you that configuration files can quickly become full-blown programs. For example, anyone who has ever tried to "tune" OpenSER knows what you're talking about: this is not a configuration, this is a (painful) programming.

When you need your application to be very “customizable” in ways that you cannot imagine today, then you really need a plugin system. You need to develop the application in such a way that someone else can encode the new plugin and connect it to your application in the future.

So, to answer your questions:

• What is the true purpose of the configuration file?

  I would say that the people who installed your application could twist some deployment-related parameters, such as the host name, number of threads, the names of the plugins you need and the deployment parameters for these plugins (check the FreeRadius configuration for an example of this principle) and etc. Definitely not the place to express business logic.

• Should an attempt be made to keep configuration files simple?

  Sure. As you said, the “programming” in the configuration file is terrible. I think this should be avoided.

• Who should be responsible for making changes to them (developers, users, administrators, etc.)?

  In general, I would say the administrators who deploy the application.

• Should they be controlled by the source (see question 3)?

  I usually don’t control the source files of the configuration files themselves, but I control the source code of the template configuration file with all the parameters and their default values ​​and comments describing what they do. For example, if the configuration file is named database.conf , I usually control the source file called database.conf.template . Now, of course, I'm talking about what I do as a developer. As an administrator, I may want to set the initial parameters that I selected for each installation. For example, we delete several hundred servers remotely, and we need to track their configurations: we decided to do this using source control.

---

Edit: Although I believe that the above value applies to most applications, there are always exceptions, of course. For example, your application may allow its users to dynamically configure complex rules. Most email clients allow users to define rules for managing their emails (for example, "all emails coming from" john doe "and not having me in the To: field should be dropped"). Another example is an application that allows the user to define a new comprehensive business proposal. You can also think of applications such as Cognos that allow their users to create sophisticated database reports. The email client is likely to offer the user a simple interface for defining rules, and this will create a complex configuration file (or even, perhaps, some code). On the other hand, the user configuration for commercial offers can be stored in a database in a structured way (neither a simple key structure = value, nor part of the code). And some other applications may even allow the user to code in python or VB, or some other language that supports automation. In other words ... your mileage may vary.

Answer 2

Ok You will have some users who need a really simple configuration, you should give them to them. At the same time, you will have constant requests “Can you add this? How to do this in the configuration file?”, I don’t understand why you cannot support both groups.

The project I'm working on is using Lua for its configuration file. Lua is a scripting language, and it works well in this script. There is an example of our default configuration .

You will notice that these are basically keywords = values, where the value can be any of Lua's built-in types. The hardest part is that there are lists, and they are not very complex (this is just a matter of syntax).

Now I'm just waiting for someone to ask how to set their server port to a random value every time they start it ...

Answer 3

I recently worked on a project, and I realized that I want to have conventions inside my configuration file, which was previously quite simple in form:

 key = val key2 = val name = `hostname`
key = val key2 = val name = `hostname` 

I did not want to write a mini-language, because if I did not do this very carefully, I could not allow the flexibility that would be useful.

Instead, I decided that I would have two forms:

• If the file started with "#!" and was executable, I analyzed the result of its launch.

• Otherwise, I would read it as-is

This means that now I can let people write “configuration files” that look like this:

 #!/usr/bin/perl if ( -x /bin/foo ) { print <<EOF; foo=me bar=you EOF } else { print <<EOF; foo=bar bar=foo EOF } 

Thus, I get the power of a dynamic configuration file if the user wants to use it, and the simplicity is not to write my own mini-language.

Answer 4

Each (fairly durable) configuration of the configuration file ultimately becomes a programming language. Due to all the consequences that you describe, it is wise for a configuration file designer to understand that she is the author of a programming language and is appropriately planned so that she does not expose future users to a bad heritage.

Answer 5

I have a different philosophy about configuration files. The data on how the application should be executed is still data and therefore belongs to the data store, and not to the code (the IMO configuration file is code). If end users need to be able to modify data, the application must provide an interface for this.

I use only configuration files to point to data stores.

Answer 6

You can turn to computational theory to determine what is considered a programming language. If your Turing Complete configuration file format, then it is reasonably considered a programming language. By this definition, the file format for describing Sokoban levels is considered a programming language (see here ). There are other difficulty levels below Turing Complete, which can also be taken into account, for example Regular Grammars and Pushdown Automata .

Another way to look at this is that many configuration files can only mark up data, while the correct programming language should be able to implement algorithms. For example, JSON is a configuration file format, while ECMA Script is a programming language.

Answer 7

Here are my thoughts:

• To easily change application behavior in runtime. These may be programmers or non-programmers, depending on the needs. This may be during development, but I often see configuration files as a way to make the program more flexible at any time.

• Yes. I think configuration files should be as simple as possible, given the limitation that you may need various options to control the different types of behavior of your runtime environment. I prefer to group configuration settings and simplify them as much as possible.

• Depends on what and why the change is occurring. If users will change it, you must close the front panel to hide them from the details. The same often applies to non-developers in general.

• I often control the default configuration, but have the ability to override this for each system for the actual runtime.

As for adding logic to the configuration file - I would avoid this. I think it’s better to just include the configuration file in your application logic. In my experience, behavior in configuration files leads to a lack of maintainability and understanding. I prefer to keep configuration files as simple as possible.

Answer 8

It depends on what you agree with other developers on the team. You use configuration files in the same way as configuration files, or you create a Model Driven application.

Symptoms of a configuration file that become a programming language:

• name = value pairs begin to depend on each other
• do you feel the need for flow control (e.g. if (this) than this)
• documentation for the configuration file becomes necessary for further development (instead of using the application)
• before reading the value from config, it needs to have some context (i.e. the values ​​depended on something from the external configuration file)

Answer 9

Configuration files invariably come close to becoming ugly, illogical "full-fledged programming languages." Developing good programming languages ​​requires art and skill, and customization languages ​​translated into a programming language are terrifying.

A good approach is to use a well-developed language like python or ruby, and use it to create

Answer 10

I believe that your question is very important, given the transition to "free interfaces". Many developers have seen the light of day about customized XML applications. Using XML can be very verbose and difficult to edit correctly (especially if a schema is not provided). The presence of a free interface allows the developer to configure the application in a domain-specific language using some key-value pairs from the plain text configuration file (or, possibly, command line parameters). It also makes it easy to configure and configure new instances of the application for testing or something else.

Here are my answers to your question:

• What is the true purpose of the file configuration?

A configuration file is a method that allows the user to customize the behavior of his program at runtime.

• Should I try to keep the configuration files simple?

Ideally, I would think that the configuration files, at least, should be supplemented by a free interface for setting up the program (this is useful in many respects). If you need a configuration file, it should be very simple, nothing but key-value pairs.

• Who should be responsible for changes in them (developers, users, administrators, etc.)?

I think the answer to this depends on your organization. This should be the responsibility of the person deploying the software to ensure its proper configuration.

• If they are under source control (see question 3)?

I will steal this answer from someone else :) I like the idea of ​​saving the configuration of the template in the source control and changing it for each user. Most likely, one developer configuration file is another developer nightmare, so it's best to leave things that differ from the user outside the source control. Having a template is also a good way to let the person deploying the application (or other developers) see exactly which values ​​are valid for the configuration file.

Answer 11

I tend to agree with the premise of this question. I avoid getting into trouble by predicting in advance that this will happen, and so never turn my own configuration system upside down.

• Either I'm using the operating system configuration configuration (e.g. plist or gconf or something suitable)
• Or a simple flat file that can be processed by something like an INI parser on a shelf.
• Bite a bullet and plug in a lightweight language parser, usually lua, sometimes tcl to an application,
• Or store data in SQLite or a similar relational database.

And come to terms with living with any decision I made, or, if I can’t, refactoring, use one of the above options, which is best suited for the application.

The point, in fact, there is no reason to use a home configuration solution. Firstly, it’s more difficult for your users to learn the new configuration format of the application. For others, you benefit from all bug fixes and updates that are provided free of charge when using a turnkey solution. Finally, the creep function is set to rest, because, in fact, you actually can’t just add another function without performing a major overhaul, because the configuration system is actually not in your hands.

Answer 12

Yes, the configuration files should be simple. They should not contain any “logic” - consider them as a list of expressions in if statements, and not conditional statements in general.

They are there to allow the user to decide which of the options encoded in the application should be used, so do not try to complicate them, this will ultimately be self-destructing - you can write simple configuration files to control how the original configuration file should be configured otherwise case!

Answer 13

One of the goals of Oslo at Microsoft is to resolve (although not requiring) a solution to this problem.

• The application will come with models of any new components that it includes. He will also use existing models. For example, it can include a web service, so it can reuse the system model of a web service.
• Models will include metadata describing them, including sufficient information for the tools to access them, text or graphic.
• Parts of the models will match the “configuration”

This means that the equivalent of today's configuration files can be rich enough to support textual and graphical editing of their configuration. The graphical tool will be provided with Oslo (codenamed Quadrant).

Answer 14

I will be an adversary and will only give him the language when he embodies more than XML can represent; or when XML is considered a language.

Alternatively, most configuration files can be considered classes, but only with properties and without methods. And without methods, I don’t think it is a language.

Ultimately, “language” is a squishy abstraction, but yes, the edges are ambiguous.

Answer 15

The code of our applications becomes less important ... There is a scenario, there are all kinds of attributes that determine the behavior of classes, methods, arguments and method properties. Users can define database triggers and database restrictions. There can be very complex configuration files. Sometimes a user can define XSLT styles to control inputs and outputs because our systems must be open (SOA). And there are things like BizzTalk, which also needs a complex configuration. Users can define complex workflows.

We need to write better code to work with this complex environment, so the code of our applications is becoming more important ...

Answer 16

I saw python programs where the configuration file is the code. If you do not need to do anything (conditional, etc.), It is not much different from other configuration styles. for example I could create a config.py with things like:

 num_threads = 13 hostname = 'myhost' 

and the only burden for the user, compared to (say) INI files, is that they should be placed “around the lines”. No doubt you could do the same in other interpreted languages. This gives you an unlimited opportunity to complicate your configuration file if necessary, at the risk of scaring your users.

Answer 17

I am a big fan of using python programs as configuration files, especially for daemons. I like to stick to making the daemon completely empty from the configuration other than the "configuration port". Then the python program connects to the daemon and continues to create objects in the daemon and connects them together to create the desired configuration. After everything is configured, the daemon can be left to run on it. Of course, the advantages are that you get a full-fledged programming language for writing your configuration files, and since you already have a way to talk to a daemon from another program, you can use it to debug and get statistics. The main disadvantage is dealing with messages from another program arriving at any time.