Python: how to store a multicore numpy array in PyTables?

Question

How can I put a numpy multidimensional array into an HDF5 file using PyTables?

From what I can say, I cannot put an array field in a pytables table.

I also need to save some information about this array and do the math on it.

Any suggestions?

Answer 1

There may be an easier way, but this is how you will do it, as far as I know:

import numpy as np import tables # Generate some data x = np.random.random((100,100,100)) # Store "x" in a chunked array... f = tables.open_file('test.hdf', 'w') atom = tables.Atom.from_dtype(x.dtype) ds = f.createCArray(f.root, 'somename', atom, x.shape) ds[:] = x f.close() 

If you want to specify the compression used, take a look at tables.Filters . tables.Filters . for example

 import numpy as np import tables # Generate some data x = np.random.random((100,100,100)) # Store "x" in a chunked array with level 5 BLOSC compression... f = tables.open_file('test.hdf', 'w') atom = tables.Atom.from_dtype(x.dtype) filters = tables.Filters(complib='blosc', complevel=5) ds = f.createCArray(f.root, 'somename', atom, x.shape, filters=filters) ds[:] = x f.close() 

There is probably an easier way for many of them ... I have not used pytables for anything other than tabular data for a long time.

Note: in pytables 3.0 f.createCArray been renamed f.create_carray . It can also take an array directly, without specifying atom ,

 f.create_carray('/', 'somename', obj=x, filters=filters)