How should python dictionaries be used in pytables?

Question

pytables does not support python language dictionaries. The way I came up with was to create a form data structure:

tables_dict = { 'key' : tables.StringCol(itemsize=40), 'value' : tables.Int32Col(), } 

(note that I guarantee that the keys are up to 40 characters long) and then create the table using this structure:

 file_handle.createTable('/', 'dictionary', tables_dict) 

and then fill it in:

 file_handle.dictionary.append(dictionary.items()) 

and get the data using:

 dict(file_handle.dictionary.read()) 

This works fine, but reading the dictionary back is very slow. I think the problem is that the read() function causes the entire dictionary to load into memory, which should not be really necessary. Is there a better way to do this?

Answer 1

You can ask PyTables to search inside the table, as well as create an index in the key column to speed it up.

To create an index:

 table.cols.key.createIndex() 

Request values ​​where key is equal to search_key variable:

 [row['value'] for row in table.where('key == search_key')] 

http://pytables.github.com/usersguide/optimization.html#searchoptim