Providing a drive to display the table

accumarray uses two rows of indices to create a matrix with elements at the location of valid index pairs with the value assigned by the specified function, for example:

 A = [11:20]; B = flipud([11:20]); C = 1:10; datamatrix = accumarray([AB],C); 

In this way, the datamatrix will be a 20x20 matrix with values. If the values ​​of A and B , however, are very large, this will lead to most of the empty matrix with a small batch of data in the far corner. To get around this, you can set accumarray to issparse :

 sparsedatamatrix = accumarray([AB],C,[],@sum,[],true); 

This will save a lot of memory if min(A) and / or min(B) / are very large.

My problem, however, is that I have a Mx7 matrix with M~1e8 on which I want to collect funds from columns three to seven based on indexing in the first two columns and standard deviation. The third column is also based on the third:

 result = accumarray([data(:,1) data(:,2)],data(:,3),[],@std); 

I want to save this back to a table structured as [XYZ std RGBI] , where X and Y are the indices, Z is the average height of this pixel, R , G , B and I are the average values ​​(color and intensity) per pixel, and std - standard deviation of heights (i.e. roughness). Using issparse does not help in this case, since I am converting my matrices from accumarray using repmat .

The purpose of this code is to evaluate the height, roughness, color, and intensity of a piece of land from a point cloud. I rounded the coordinates in X and Y to create a grid and now I need these averages on the cell of the cell, but it is displayed as a β€œtable” (not the MATLAB data type, but a 2D array, which is not the default output signal).

So, in conclusion, with the question:

Is there a way for an accumarray or similar function to output this table without an intermediate (potentially very large) matrix?

Code below:

 Xmax = max(Originaldata(:,1)); Ymax = max(Originaldata(:,2)); X_avg_grid=(Edgelength:Edgelength:Xmax*Edgelength)+Xorig; TestSet = zeros(Xmax*Ymax,9); xx = [1:length(X_avg_grid)]'; %#ok<*NBRAK> TestSet(:,1) = repmat(xx,Ymax,1); ll = 0:Xmax:Xmax*Ymax; for jj = 1:Ymax TestSet(ll(jj)+1:ll(jj+1),2) = jj; end for ll = 1:7 if ll == 2 tempdat = accumarray([Originaldata(:,1) Originaldata(:,2)],Originaldata(:,3),[],@std); tempdat = reshape(tempdat,[],1); TestSet(:,ll+2) = tempdat; elseif ll == 7 tempdat = accumarray([Originaldata(:,1) Originaldata(:,2)],1); tempdat = reshape(tempdat,[],1); TestSet(:,ll+2) = tempdat; elseif ll == 1 tempdat = accumarray([Originaldata(:,1) Originaldata(:,2)],Originaldata(:,3),[],@mean); tempdat = reshape(tempdat,[],1); TestSet(:,ll+2) = tempdat; else tempdat = accumarray([Originaldata(:,1) Originaldata(:,2)],Originaldata(:,ll+1),[],@mean); tempdat = reshape(tempdat,[],1); TestSet(:,ll+2) = tempdat; end end TestSet = TestSet(~(TestSet(:,9)==0),:); 

The ninth column here is just the number of dots per cell.

 Originaldata = 19 36 2.20500360107422 31488 31488 31488 31611 20 37 2.26400360107422 33792 33792 34304 33924 20 37 2.20000360107422 33536 33536 34048 33667 19 36 2.20500360107422 34560 34560 34560 34695 20 36 2.23300360107422 32512 32512 33024 32639 21 38 2.22000360107422 31744 31488 33024 31611 21 37 2.20400360107422 32512 32768 33792 32896 21 37 2.24800360107422 29696 29440 30720 29555 21 38 2.34800360107422 32768 32768 32768 32639 21 37 2.23000360107422 33024 33024 33536 33153 

Thus, all points on the same X, Y (for example, [19 36] or [21 37] ) are averaged (height, RGB, intensity in this order), and the standard deviation is also desirable in the third column: / p>

 Result = 19 36 2.2050036 0.00 33024 33024 33024 33153 21 37 2.227336934 0.02212088 31744 31744 32682.66 31868 

etc. for the rest of the data.

I updated my code to the latest version. This slightly reduced the amount of memory, since now the function creates grids one by one, and not all at the same time. However, the code works in parallel, so eight more simultaneous grids are created, so the solution will still be evaluated.

+6
source share
3 answers

Sketch of a solution using linear indices and a two-dimensional sparse matrix

 lind = Originaldata(:,1) + max( Originaldata(:,1) ) * ( Originaldata(:,2) - 1 ); daccum(7,:) = accumarray( lind, 1, [], @sum, [], true ); %// start with last one to pre-allocate all daccum daccum(1,:) = accumarray( lind, Originaldata(:,3), [], @mean, [], true ); daccum(2,:) = accumarray( lind, Originaldata(:,3), [], @std, [], true ); daccum(3,:) = accumarray( lind, Originaldata(:,4), [], @mean, [], true ); daccum(4,:) = accumarray( lind, Originaldata(:,5), [], @mean, [], true ); daccum(5,:) = accumarray( lind, Originaldata(:,6), [], @mean, [], true ); daccum(6,:) = accumarray( lind, Originaldata(:,7), [], @mean, [], true ); 

Now you can get only what you need

 inter = [Originaldata(:,1), Originaldata(:,2), full( daccum(:,lind) )' ]; 
+3
source

You can first use unique with the 'rows' option to find the indices of the unique X and Y coordinate pairs, use these indices as an index in your accumarray calls accumarray (you will have to name it separately for each column, since accumarray does not process matrix inputs) :

 [xyPairs, ~, index] = unique(Originaldata(:, 1:2), 'rows'); nPairs = max(index); Result = [xyPairs ... accumarray(index, Originaldata(:, 3), [nPairs 1], @mean) ... accumarray(index, Originaldata(:, 3), [nPairs 1], @std) ... accumarray(index, Originaldata(:, 4), [nPairs 1], @mean) ... accumarray(index, Originaldata(:, 5), [nPairs 1], @mean) ... accumarray(index, Originaldata(:, 6), [nPairs 1], @mean) ... accumarray(index, Originaldata(:, 7), [nPairs 1], @mean) ... accumarray(index, ones(size(index)), [nPairs 1], @sum)]; 
+2
source

You can pre-process the data.

One thing you can achieve this way is to remove unwanted lines (e.g. those that have two or fewer cases), so you don't have to deal with the standard deviation of 0 :

 %// Count occurences: combined_coord = Originaldata(:,1)*1E6+Originaldata(:,2); %// "concatenating" the coords [C,~,ic] = unique(combined_coord); occurences = [C accumarray(ic,1)]; %// Find all points that have <=2 occurences: coords_to_remove = occurences((occurences(:,2)<=2),1); %// Find valid lines: valid_lns = ~sum(bsxfun(@eq,combined_coord,coords_to_remove'),2); %' %// Filter original data: new_data = Originaldata(valid_lns,:); 
+1
source

All Articles