How can I speed up this two-line code?

Question

I need to speed up the following code:

for i in range(0, 2**N): output[i] = f(np.array(map(int, bin(i)[2:].zfill(N)))) 

N is around 30 , so the code is very slow (on my laptop it takes about 33 hours). The argument of the function f() is the binary representation of the index i , and f() can be an arbitrary vector function. I am not an expert, but in order to speed up the code, I decided to get rid of the for loop, which means that I need to vectorize the f() argument. In other words, I need to create a matrix with binary representations of numbers from 0 to 2**N This can be achieved with the following code:

 list(itertools.product([0, 1], repeat=N)) 

which I found in this link . However, it seems to me that itertools very slow, and obviously it takes up a lot of memory, since 2**30 is about one billion.

Do you have any suggestion to speed up this code? Thanks in advance.

Answer 1

Always profile:

 >>> timeit.timeit("for i in range(0, 2**N): numpy.array(map(int, bin(i)[2:].zfill(N)))", "import numpy; N=5", number=100000) 26.472519159317017 >>> timeit.timeit("for t in itertools.product((0, 1), repeat=N): numpy.array(t)", "import numpy, itertools; N=5", number=100000) 6.129688024520874 

You can see that the itertools.product method itertools.product significantly faster since it does not need to mess around with strings.

The problem may be that most of the time is spent on the function f .

Another solution would be to take f an integer and use it as a binary field.