What a quick (non-cyclic) way to apply dict to ndarray (which means using elements as keys and replacing values)

Question

I'm passing through now

new_data = [transform_dict[pt] for pt in line] for line in data] 

But it is too slow. I tried looking for a suitable numpy method, but did not find anything myself. Are there any matrix implementations for this kind of thing?

Answer 1

convert the dictionary to an array and use np.take :

 Na=1000 #array Nd = 10**4 #dict data=randint(0,Nd,(Na,Na)) dic=dict(zip(range(Nd),randint(0,Nd,Nd))) dicarray=np.array(list(dic.values())) 

This is usually much faster:

 In [3]: %timeit np.array([[dic[x] for x in line] for line in data]) 1 loops, best of 3: 2.27 s per loop In [4]: %timeit dicarray.take(data) 10 loops, best of 3: 24.4 ms per loop 

Answer 2

I think you need something that uses numpy compiled code. But the problem is that there is no such code to access dictionary values. All this went through dd[k] or dd.get(key) .

dd.items() creates a list of key pairs and values. Perhaps you can turn this into an array and play numpy games with them.

pandas may have some ways to efficiently turn a dictionary into a data framework.

How big is this dictionary? What are the keys and meanings?