The most efficient way to load formatted binaries in Python

Question

I have binaries no larger than 20 MB that have a header section and then a data section containing uchars sequences. I have Numpy, SciPy, etc., And each library has different ways to load data. Any suggestions on the most effective methods I should use?

Answer 1

A struct should work in the header section, and a numpy memmap would be effective for a data section if you would still manipulate it in numpy. There is no need to emphasize what is incompatible here. Both methods are compatible, just use the appropriate tool for each job.

Answer 2

Use a struct module or perhaps a custom module written in C if performance is critical.

Answer 3

bdec seems promising.

Answer 4

I found that array.fromfileis the fastest method for uniform data.