good question. this looks like something specific to the mcrypt / libmcrypt program.
I could not find any documents, so I looked at the source for libmcrypt. which contains two modules, one for blowfish, and one for blowfish-compat. when I look at them, the only difference that I see (warning: I'm a software engineer, but not a cryptography specialist) is that the logic for byte order is replaced ( ifdef WORDS_BIGENDIAN is replaced by ifndef WORDS_BIGENDIAN - pay attention to "p" )
therefore, I assume that it is designed to decode data on large-end machines that were encoded on machines with a small limb, or vice versa. or perhaps there is some convention that the code should follow the content, but some libraries break it, and this compensates.
update yeah! and knowing that the search engine for "blowfish-compat big-endian" appears, which seems like confirmation. see http://www.spinics.net/lists/crypto/msg00175.html - which considers an incorrect implementation, which forced to order the opposite.
so, in short, your data was incorrectly encoded. "compat" mode reproduces the error so that they can be decoded.
given that it looks like you're out of luck on the python front end if you can't find the python interface for mcrypt. http://labix.org/python-mcrypt looks like it can work (pypi page - http://pypi.python.org/pypi/python-mcrypt ).
(this was one of the funniest answers to provide for a long time: o)
oh, and I got the source http://sourceforge.net/projects/mcrypt/ , following the "view all files" link under the download button (the button loads mcrypt, not libmcrypt).
source share