Workflow for supporting different versions of the codebase for different versions of Python

Question

I am developing an open source application called GarlicSim .

So far, I have developed it only for Python 2.6. It does not seem to work in any other version.

I decided it was important to create versions that would support other versions of Python. I think I will make a version for 2.5, 3.1, and possibly 2.4.

So, I have a few questions:

• What would be a good way to organize the folder structure of my repo to include these different versions?
• What would be a good way to “merge” the changes I make in one version of the code to other versions? I know how to do merges in my SCM (which is git), but these are folders that are all in the same repo, and I want to do a merge between them. Of course, it is possible to have a repo for each version, but I think this is not a good idea.

Does anyone have any suggestions?

Answer 1

You need separate branches for individual versions only in the rarest cases. You mention context managers, and they are beautiful, and it suck, so as not to use them, and you are right. But for Python 2.4 you do not have to use them. So suck. Therefore, if you want to support Python 2.4, you will have to write a version without context managers. But this one will work under Python 2.6, so it makes no sense to have separate versions there.

As for Python 3, which has a separate branch, there is a solution, but usually not the best. To support Python 3, there is something called 2to3 that converts your Python 2 code to Python 3 code. This is not ideal, so quite often you will have to change Python 2 code to create beautiful Python 3 code, but Python 2 code anyway tends to get better.

When using Distribute (supported by fork setuptools), you can do this conversation automatically during installation. This way you don't need to have a separate branch even for Python 3. See http://bitbucket.org/tarek/distribute/src/tip/docs/python3.txt for documents on this.

As Paul McGuire writes, it’s even possible to support Python 3 and Python 2 with the same code without using 2to3, but I would not recommend it if you want to support anything else except 2.6 and 3.x. You get too many of these ugly special hacks. Version 2.6 has enough advanced compatibility with Python 3 so that you can write decent code and support both Python 2.6 and 3.x, but not Python 2.5 and 3.x.

Answer 2

I would try to maintain one branch to cover all python 2.4-2.6

The differences are not so great, because if you need to write a bunch of additional code for 2.4 in order to do something that is easy to do in version 2.6, it will work less for you in the long run to use version 2.4 for 2.5 and 2.6.

Python 3 should have a different branch, you should still try to save as much code as possible.

Answer 3

If your code does not depend too much on runtime performance in exception handlers, you can even leave without a separate branch for Py3. I managed to save one version of pyparsing for all my versions of Py2.x, although I had to adhere to the "lowest common denominator" approach, which means that I should refuse to use some constructs, such as generator expressions, and your point, context managers. I use dicts instead of sets, and all my generator expressions get wrapped like lists, so they will still work, returning to Python 2.3. I have a block at the top of my code that takes care of several 2vs3 issues (contributed by pyparsing user Robert A Clark):

_PY3K = sys.version_info[0] > 2 if _PY3K: _MAX_INT = sys.maxsize basestring = str unichr = chr unicode = str _str2dict = set alphas = string.ascii_lowercase + string.ascii_uppercase else: _MAX_INT = sys.maxint range = xrange def _str2dict(strg): return dict( [(c,0) for c in strg] ) alphas = string.lowercase + string.uppercase 

The biggest difficulty I ran into was the incompatible syntax for throwing exceptions, which was introduced in Py3, changing from

 except exceptiontype,varname: 

to

 except exceptionType as varname: 

Of course, if you really don't need an exception variable, you can simply write:

 except exceptionType: 

and this will work on Py2 or Py3. But if you need to access the exception, you can find cross-compatible syntax like:

 except exceptionType: exceptionvar = sys.exc_info()[1] 

This is a minor punishment at runtime, which makes it unusable in some places in pyparsing, so I still have to maintain separate versions of Py2 and Py3. To merge the source, I use the WinMerge utility, which I find very useful for synchronizing source code directories.

So, although I keep two versions of my code, some of these unification methods help me keep the differences to an absolute incompatible minimum.

Answer 4

In the end, I decided to have 4 different forks for my project, for 2.4, 2.5, 2.6 and 3.1. My top priority is 2.6, and I don't want to compromise the elegance of this code for 2.4. Thus, ugly compatibility hacks will be in lower versions, and not in higher versions.