Matlab - signal removal

Question

I have a data vector that contains integers in the range -20 20.

Anther is a graph with values:

This is a sample of 96 elements from vector data. Most elements are in the range of -2, 2, as can be seen from the above graph.

I want to exclude noise from the data. I want to exclude peaks with low amplitude and keep the peak of high amplitude, namely peaks similar to those with index 74.

Basically, I just want to increase the contrast between high amplitude highs and low amplitude peaks, and if low amplitude peaks could be eliminated.

Could you suggest me a way to do this?

I tried the mapstd function, but the problem is that it also normalizes the peak with high amplitude.

I thought about using the wavelet transform toolkit, but I don’t know exactly how to recover the data from the wavelet decomposition coefficients.

Can you recommend me a way to do this?

Answer 1

If this is for demonstration purposes only and you are not really going to use these scaled values ​​for anything, I sometimes like to increase the contrast as follows:

 % your data is in variable 'a' plot(a.*abs(a)/max(abs(a))) 

edit: since we post images, here are my (before / after):

Answer 2

One way to detect outliers is to use the rule of three standard deviations . Example:

 %# some random data resembling yours x = randn(100,1); x(75) = -14; subplot(211), plot(x) %# tone down the noisy points mu = mean(x); sd = std(x); Z = 3; idx = ( abs(x-mu) > Z*sd ); %# outliers x(idx) = Z*sd .* sign(x(idx)); %# cap values at 3*STD(X) subplot(212), plot(x) 

---

EDIT:

It seems I misunderstood the purpose here. If you want to do the opposite, perhaps something like this:

 %# some random data resembling yours x = randn(100,1); x(75) = -14; x(25) = 20; subplot(211), plot(x) %# zero out everything but the high peaks mu = mean(x); sd = std(x); Z = 3; x( abs(x-mu) < Z*sd ) = 0; subplot(212), plot(x) 

Answer 3

You can try the broken window filter. If x is your current pattern, the filter will look something like this:

 k = [LLLLLL 0 0 0 x 0 0 0 RRRRRR] 

For each sample x, you compare the band of surrounding samples on the left (L) and the range of surrounding samples on the right. If your samples are positive and negative (like yours), you should take abs. at first. Then you divide sample x by the average of these surrounding samples.

 y[n] = x[n] / mean(abs(x([LR]))) 

Each time you do this, the peaks are amplified and the noise is smoothed out. You can do more than one pass to increase the effect. It is somewhat sensitive to the choice of the width of these stripes, but it can work. For example:

Two passes:

Answer 4

In fact, you need some kind of compression to scale your data, that is: values ​​between -2 and 2 are scaled by a certain factor, and everything else is scaled by another factor. A rough way to accomplish such a thing is to put all the small values ​​to zero, i.e.

 x = randn(1,100)/2; x(50) = 20; x(25) = -15; % just generating some data threshold = 2; smallValues = (abs(x) <= threshold); y = x; y(smallValues) = 0; figure; plot(x,'DisplayName','x'); hold on; plot(y,'r','DisplayName','y'); legend show; 

Please do not do this, this is a very non-linear operation (for example, when you need peaks of 2.1 and 1.9, they will produce a completely different behavior: one will be deleted, the other will be saved). Therefore, for display, this may be all that you need, for further processing it may depend on what you are trying to do.

Answer 5

To eliminate low amplitude peaks, you will equate the entire low amplitude signal with noise and ignore it.

If you have any a priori knowledge, just use it.

if your signal is a, then

 a(abs(a)<X) = 0 

where X is the maximum expected size of your noise.

If you want to get imagination and find it on the fly, then use kmeans from 3. This is in the statistics toolbar, here:

http://www.mathworks.com/help/toolbox/stats/kmeans.html

Alternatively, you can use the Otsu method for absolute data values ​​and use the backslash.

Please note that these and all other methods that I saw in this thread assume that you are doing post-processing. If you do this processing in real time, everything should change.