How to copy data from a numpy array to another

Question

What is the fastest way to copy data from array b to array a without changing the address of array a. I need this because the external library (PyFFTW) uses a pointer to my array, which cannot change.

For example:

a = numpy.empty(n, dtype=complex) for i in xrange(a.size): a[i] = b[i] 

Can this be done without a loop?

Answer 1

I believe

 a = numpy.empty_like (b) a[:] = b 

quickly make a deep copy. As Funsi mentions, recent versions of numpy also have a copyto function.

Answer 2

numpy version 1.7 has a numpy.copyto function that does what you are looking for:

numpy.copyto (dst, src)

Copying values ​​from one array to another,> translation as needed.

See: http://docs.scipy.org/doc/numpy-dev/reference/generated/numpy.copyto.html

Answer 3

 a = numpy.array(b) 

even faster than the proposed solutions before numpy v1.6, and makes a copy of the array. However, I could not test it against copyto (a, b), since I do not have the latest version of numpy.

Answer 4

To answer your question, I played around with some options and profiled them.

Conclusion: to copy data from a numpy array to another, if possible, use one of the built-in functions numpy numpy.array(src) or numpy.copyto(dst, src) .

(But always choose later if dst memory is already allocated to reuse memory. See Profiling at the end of the post.)

profiling setting

 import timeit import numpy as np import pandas as pd from IPython.display import display def profile_this(methods, setup='', niter=10 ** 4, p_globals=None, **kwargs): if p_globals is not None: print('globals: {0}, tested {1:.0e} times'.format(p_globals, niter)) timings = np.array([timeit.timeit(method, setup=setup, number=niter, globals=p_globals, **kwargs) for method in methods]) ranking = np.argsort(timings) timings = np.array(timings)[ranking] methods = np.array(methods)[ranking] speedups = np.amax(timings) / timings pd.set_option('html', False) data = {'time (s)': timings, 'speedup': ['{:.2f}x'.format(s) if 1 != s else '' for s in speedups], 'methods': methods} data_frame = pd.DataFrame(data, columns=['time (s)', 'speedup', 'methods']) display(data_frame) print() 

profiling code

 setup = '''import numpy as np; x = np.random.random(n)''' methods = ( '''y = np.zeros(n, dtype=x.dtype); y[:] = x''', '''y = np.zeros_like(x); y[:] = x''', '''y = np.empty(n, dtype=x.dtype); y[:] = x''', '''y = np.empty_like(x); y[:] = x''', '''y = np.copy(x)''', '''y = x.astype(x.dtype)''', '''y = 1*x''', '''y = np.empty_like(x); np.copyto(y, x)''', '''y = np.empty_like(x); np.copyto(y, x, casting='no')''', '''y = np.empty(n)\nfor i in range(x.size):\n\ty[i] = x[i]''' ) for n, it in ((2, 6), (3, 6), (3.8, 6), (4, 6), (5, 5), (6, 4.5)): profile_this(methods[:-1:] if n > 2 else methods, setup, niter=int(10 ** it), p_globals={'n': int(10 ** n)}) 

results for Windows 7 on Intel i7, CPython v3.5.0, numpy v1.10.1 processors.

 globals: {'n': 100}, tested 1e+06 times time (s) speedup methods 0 0.386908 33.76xy = np.array(x) 1 0.496475 26.31xy = x.astype(x.dtype) 2 0.567027 23.03xy = np.empty_like(x); np.copyto(y, x) 3 0.666129 19.61xy = np.empty_like(x); y[:] = x 4 0.967086 13.51xy = 1*x 5 1.067240 12.24xy = np.empty_like(x); np.copyto(y, x, casting=... 6 1.235198 10.57xy = np.copy(x) 7 1.624535 8.04xy = np.zeros(n, dtype=x.dtype); y[:] = x 8 1.626120 8.03xy = np.empty(n, dtype=x.dtype); y[:] = x 9 3.569372 3.66xy = np.zeros_like(x); y[:] = x 10 13.061154 y = np.empty(n)\nfor i in range(x.size):\n\ty[... globals: {'n': 1000}, tested 1e+06 times time (s) speedup methods 0 0.666237 6.10xy = x.astype(x.dtype) 1 0.740594 5.49xy = np.empty_like(x); np.copyto(y, x) 2 0.755246 5.39xy = np.array(x) 3 1.043631 3.90xy = np.empty_like(x); y[:] = x 4 1.398793 2.91xy = 1*x 5 1.434299 2.84xy = np.empty_like(x); np.copyto(y, x, casting=... 6 1.544769 2.63xy = np.copy(x) 7 1.873119 2.17xy = np.empty(n, dtype=x.dtype); y[:] = x 8 2.355593 1.73xy = np.zeros(n, dtype=x.dtype); y[:] = x 9 4.067133 y = np.zeros_like(x); y[:] = x globals: {'n': 6309}, tested 1e+06 times time (s) speedup methods 0 2.338428 3.05xy = np.array(x) 1 2.466636 2.89xy = x.astype(x.dtype) 2 2.561535 2.78xy = np.empty_like(x); np.copyto(y, x) 3 2.603601 2.74xy = np.empty_like(x); y[:] = x 4 3.005610 2.37xy = np.empty_like(x); np.copyto(y, x, casting=... 5 3.215863 2.22xy = np.copy(x) 6 3.249763 2.19xy = 1*x 7 3.661599 1.95xy = np.empty(n, dtype=x.dtype); y[:] = x 8 6.344077 1.12xy = np.zeros(n, dtype=x.dtype); y[:] = x 9 7.133050 y = np.zeros_like(x); y[:] = x globals: {'n': 10000}, tested 1e+06 times time (s) speedup methods 0 3.421806 2.82xy = np.array(x) 1 3.569501 2.71xy = x.astype(x.dtype) 2 3.618747 2.67xy = np.empty_like(x); np.copyto(y, x) 3 3.708604 2.61xy = np.empty_like(x); y[:] = x 4 4.150505 2.33xy = np.empty_like(x); np.copyto(y, x, casting=... 5 4.402126 2.19xy = np.copy(x) 6 4.917966 1.96xy = np.empty(n, dtype=x.dtype); y[:] = x 7 4.941269 1.96xy = 1*x 8 8.925884 1.08xy = np.zeros(n, dtype=x.dtype); y[:] = x 9 9.661437 y = np.zeros_like(x); y[:] = x globals: {'n': 100000}, tested 1e+05 times time (s) speedup methods 0 3.858588 2.63xy = x.astype(x.dtype) 1 3.873989 2.62xy = np.array(x) 2 3.896584 2.60xy = np.empty_like(x); np.copyto(y, x) 3 3.919729 2.58xy = np.empty_like(x); np.copyto(y, x, casting=... 4 3.948563 2.57xy = np.empty_like(x); y[:] = x 5 4.000521 2.53xy = np.copy(x) 6 4.087255 2.48xy = np.empty(n, dtype=x.dtype); y[:] = x 7 4.803606 2.11xy = 1*x 8 6.723291 1.51xy = np.zeros_like(x); y[:] = x 9 10.131983 y = np.zeros(n, dtype=x.dtype); y[:] = x globals: {'n': 1000000}, tested 3e+04 times time (s) speedup methods 0 85.625484 1.24xy = np.empty_like(x); y[:] = x 1 85.693316 1.24xy = np.empty_like(x); np.copyto(y, x) 2 85.790064 1.24xy = np.empty_like(x); np.copyto(y, x, casting=... 3 86.342230 1.23xy = np.empty(n, dtype=x.dtype); y[:] = x 4 86.954862 1.22xy = np.zeros(n, dtype=x.dtype); y[:] = x 5 89.503368 1.18xy = np.array(x) 6 91.986177 1.15xy = 1*x 7 95.216021 1.11xy = np.copy(x) 8 100.524358 1.05xy = x.astype(x.dtype) 9 106.045746 y = np.zeros_like(x); y[:] = x 

---

Also see Results for the profiling option, where y = np.empty_like(x) memory is already pre-allocated during the copying of the value, since y = np.empty_like(x) is part of the configuration:

 globals: {'n': 100}, tested 1e+06 times time (s) speedup methods 0 0.328492 2.33x np.copyto(y, x) 1 0.384043 1.99xy = np.array(x) 2 0.405529 1.89xy[:] = x 3 0.764625 np.copyto(y, x, casting='no') globals: {'n': 1000}, tested 1e+06 times time (s) speedup methods 0 0.453094 1.95x np.copyto(y, x) 1 0.537594 1.64xy[:] = x 2 0.770695 1.15xy = np.array(x) 3 0.884261 np.copyto(y, x, casting='no') globals: {'n': 6309}, tested 1e+06 times time (s) speedup methods 0 2.125426 1.20x np.copyto(y, x) 1 2.182111 1.17xy[:] = x 2 2.364018 1.08xy = np.array(x) 3 2.553323 np.copyto(y, x, casting='no') globals: {'n': 10000}, tested 1e+06 times time (s) speedup methods 0 3.196402 1.13x np.copyto(y, x) 1 3.523396 1.02xy[:] = x 2 3.531007 1.02xy = np.array(x) 3 3.597598 np.copyto(y, x, casting='no') globals: {'n': 100000}, tested 1e+05 times time (s) speedup methods 0 3.862123 1.01x np.copyto(y, x) 1 3.863693 1.01xy = np.array(x) 2 3.873194 1.01xy[:] = x 3 3.909018 np.copyto(y, x, casting='no') 

Answer 5

you can easily use:

 b = 1*a 

This is the fastest way, but there are also some problems. If you do not directly determine dtype of a and also do not check dtype of b , you may run into problems. For example:

 a = np.arange(10) # dtype = int64 b = 1*a # dtype = int64 a = np.arange(10.) # dtype = float64 b = 1*a # dtype = float64 a = np.arange(10) # dtype = int64 b = 1. * a # dtype = float64 

I hope I could clarify the situation clearly. Sometimes you will have a data type change in just one small operation.

Answer 6

There are many different things you can do:

 a=np.copy(b) a=np.array(b) # Does exactly the same as np.copy a[:]=b # a needs to be preallocated a=b[np.arange(b.shape[0])] a=copy.deepcopy(b) 

Things that don't work

 a=b a=b[:] # This have given my code bugs 

Answer 7

Why not use

 a = 0 + b 

I think this is similar to the previous multiplication, but could be simpler.