How do I use the Numpy vstack method?

Question

First, here is the relevant piece of code:

stokes_list = np.zeros(shape=(numrows,1024)) # 'numrows' defined earlier for i in range(numrows): epoch_name = y['filename'][i] # 'y' is an array from earlier os.system('pdv -t {0} > temp.txt '.format(epoch_name)) # 'pdv' is a command from another piece of software - here I copy the output into a temporary file stokes_line = np.genfromtxt('temp.txt', usecols=3, dtype=[('stokesI','float')], skip_header=1) stokes_list = np.vstack((stokes_line,stokes_line)) 

So basically, every time the stokes_line code, stokes_line extracts one of the columns (fourth) from the temp.txt file, and I want it to add a line to stokes_list every time.

For example, if the first stokes_line

 1.1 2.2 3.3 

and second

 4.4 5.5 6.6 

then stokes_list will be

 1.1 2.2 3.3 4.4 5.5 6.6 

and will continue to grow ...

This does not work at the moment, because I think the line:

 stokes_list = np.vstack((stokes_line,stokes_line)) 

not correct. This is only the addition of 2 lists - which makes sense since I have only 2 arguments. Basically, I would like to know how I keep packing again and again.

Any help would be greatly appreciated!
If necessary, here is an example temp.txt file format:

 File: t091110_065921.SFTC Src: J1903+0925 Nsub: 1 Nch: 1 Npol: 4 Nbin: 1024 RMS: 0.00118753 0 0 0 0.00148099 -0.00143755 0.000931365 -0.00296775 0 0 1 0.000647476 -0.000896698 0.000171287 0.00218597 0 0 2 0.000704697 -0.00052846 -0.000603842 -0.000868739 0 0 3 0.000773361 -0.00234724 -0.0004112 0.00358033 0 0 4 0.00101559 -0.000691062 0.000196023 -0.000163109 0 0 5 -0.000220367 -0.000944024 0.000181002 -0.00268215 0 0 6 0.000311783 0.00191545 -0.00143816 -0.00213856 

Answer 1

vstack ing is not good again and again because it copies all arrays.

Create a regular Python list , .append , and then pass it all to np.vstack to create a new array once.

 stokes_list = [] for i in xrange(numrows): ... stokes_line = ... stokes_list.append(stokes_line) big_stokes = np.vstack(stokes_list) 

Answer 2

You already know the final size of the stokes_list array since you know numrows . So it doesn't seem like you need to grow the array (which is very inefficient). You can simply assign the correct line at each iteration. Just replace your last line with:

 stokes_list[i] = stokes_line 

By the way, about your idle line, I think you meant:

 stokes_list = np.vstack((stokes_list, stokes_line)) 

where you replace stokes_list its new value.