Converting a dictionary with lists for values ​​to a data framework

Question

I spent some time looking through SO and it looks like I have a unique problem.

I have a dictionary that looks like this:

dict={ 123: [2,4], 234: [6,8], ... } 

I want to convert this dictionary, which has lists for values ​​into a data frame of three columns, as shown below:

 time, value1, value2 123, 2, 4 234, 6, 8 ... 

I can run:

 pandas.DataFrame(dict) 

but this begets the following:

 123, 234, ... 2, 6, ... 4, 8, ... 

Perhaps a simple fix, but I'm still collecting pandas

Answer 1

You can either pre-process the data, as levi suggests, or you can transpose the data frame after creating it.

 testdict={ 123: [2,4], 234: [6,8], 456: [10, 12] } df = pd.DataFrame(testdict) df = df.transpose() print(df) # 0 1 # 123 2 4 # 234 6 8 

Answer 2

It may be interesting that the Roger Fan pandas.DataFrame(dict) method is actually quite slow if you have a ton of indexes. A faster way is to simply pre-process the data into separate lists, and then create a DataFrame from these lists. (Perhaps this was explained in levi's answer, but now it's gone.)

For example, consider this dictionary dict1 , where each value is a list. In particular, dict1[i] = [ i*10, i*100] (to simplify the verification of the final data block).

 keys = range(1000) values = zip(np.arange(1000)*10, np.arange(1000)*100) dict1 = dict(zip(keys, values)) 

The pandas method takes about 30 times more time. For example.

 t = time.time() test1 = pd.DataFrame(dict1).transpose() print time.time() - t 0.118762016296 

against

 t = time.time() keys = [] list1 = [] list2 = [] for k in dict1: keys.append(k) list1.append(dict1[k][0]) list2.append(dict1[k][1]) test2 = pd.DataFrame({'element1': list1, 'element2': list2}, index=keys) print time.time() - t 0.00310587882996