R filter () associated with NA

Question

I'm trying to implement a Chebyshev filter to smooth out time series, but, unfortunately, there are NS in the data series.

For example,

t <- seq(0, 1, len = 100) x <- c(sin(2*pi*t*2.3) + 0.25*rnorm(length(t)),NA, cos(2*pi*t*2.3) + 0.25*rnorm(length(t))) 

I use the Chebyshev filter: cf1 = cheby1(5, 3, 1/44, type = "low")

I am trying to filter out time series excluding NA, but not mess up the order / position. So, I already tried na.rm=T , but there seems to be no such argument. Then

 z <- filter(cf1, x) # apply filter 

Thanks guys.

Answer 1

You can delete NA using the compelete.cases function. You may also consider imputing missing data. Check out the mtsdi or Amelia II packages.

EDIT:

Here is a solution with Rcpp. This can be useful since speed is important:

 require(inline) require(Rcpp) t <- seq(0, 1, len = 100) set.seed(7337) x <- c(sin(2*pi*t*2.3) + 0.25*rnorm(length(t)),NA, cos(2*pi*t*2.3) + 0.25*rnorm(length(t))) NAs <- x x2 <- x[!is.na(x)] #do something to x2 src <- ' Rcpp::NumericVector vecX(vx); Rcpp::NumericVector vecNA(vNA); int j = 0; //counter for vx for (int i=0;i<vecNA.size();i++) { if (!(R_IsNA(vecNA[i]))) { //replace and update j vecNA[i] = vecX[j]; j++; } } return Rcpp::wrap(vecNA); ' fun <- cxxfunction(signature(vx="numeric", vNA="numeric"), src,plugin="Rcpp") if (identical(x,fun(x2,NAs))) print("worked") # [1] "worked" 

Answer 2

Try using x <- x[!is.na(x)] to remove NA, then run the filter.

Answer 3

I don't know if ts objects can have missing values, but if you just want to reinsert NA values, you can use ?insert from R.utils . There may be a better way to do this.

 install.packages(c('R.utils', 'signal')) require(R.utils) require(signal) t <- seq(0, 1, len = 100) set.seed(7337) x <- c(sin(2*pi*t*2.3) + 0.25*rnorm(length(t)), NA, NA, cos(2*pi*t*2.3) + 0.25*rnorm(length(t)), NA) cf1 = cheby1(5, 3, 1/44, type = "low") xex <- na.omit(x) z <- filter(cf1, xex) # apply z <- as.numeric(z) for (m in attributes(xex)$na.action) { z <- insert(z, ats = m, values = NA) } all.equal(is.na(z), is.na(x)) ?insert 

Answer 4

Here is a function that you can use to filter the signal using NA in it. NAs are ignored, not replaced by zero.

Then you can specify the maximum percentage of weight that the NS can take at any point in the filtered signal. If at a certain point there is too much NA (and too little actual data), the filtered signal itself will be set to NA.

 # This function applies a filter to a time series with potentially missing data filter_with_NA <- function(x, window_length=12, # will be applied centrally myfilter=rep(1/window_length,window_length), # a boxcar filter by default max_percentage_NA=25) # which percentage of weight created by NA should not be exceeded { # make the signal longer at both sides signal <- c(rep(NA,window_length),x,rep(NA,window_length)) # see where data are present and not NA present <- is.finite(signal) # replace the NA values by zero signal[!is.finite(signal)] <- 0 # apply the filter filtered_signal <- as.numeric(filter(signal,myfilter, sides=2)) # find out which percentage of the filtered signal was created by non-NA values # this is easy because the filter is linear original_weight <- as.numeric(filter(present,myfilter, sides=2)) # where this is lower than one, the signal is now artificially smaller # because we added zeros - compensate that filtered_signal <- filtered_signal / original_weight # but where there are too few values present, discard the signal filtered_signal[100*(1-original_weight) > max_percentage_NA] <- NA # cut away the padding to left and right which we previously inserted filtered_signal <- filtered_signal[((window_length+1):(window_length+length(x)))] return(filtered_signal) }