Geek Answers Handbook

Is there an efficient way to parallelize mapply?

I have many lines, and on each line I calculate the uniroot of a nonlinear function. I have a Ubuntu quad-core computer that has not stopped working my code for two days. No wonder I'm looking for ways to speed up the process :-)

After some research, I noticed that only one core is currently being used, and parallelization is what needs to be done. Digging deeper, I came to the conclusion (maybe wrong?) That the package foreachis not really designed for my problem, because there is too much overhead (see, for example, https://stackoverflow.com/a/3/165/). ) A good alternative for Unix machines is multicore. In particular, the function pvecappears to be most effective after checking the help page.

However, if I understand correctly, this function takes only one vector and splits it accordingly. I need a function that can be paralyzed, but accepts multiple vectors (or instead data.frame), like a function mapply. Is there something I missed?

Here is a small example of what I want to do: (Please note that I am including an example here plyrbecause it can be an alternative to the base function mapplyand has a parallelism parameter. Slower in my implementation and internally, it calls foreachfor parallelization, so I think that this will not help. Is that right?)

library(plyr)
library(foreach)
n <- 10000
df <- data.frame(P   = rnorm(n, mean=100, sd=10),
                 B0  = rnorm(n, mean=40,  sd=5),
                 CF1 = rnorm(n, mean=30,  sd=10),
                 CF2 = rnorm(n, mean=30,  sd=5),
                 CF3 = rnorm(n, mean=90,  sd=8))

get_uniroot <- function(P, B0, CF1, CF2, CF3) {

  uniroot(function(x) {-P + B0 + CF1/x + CF2/x^2 + CF3/x^3}, 
          lower = 1,
          upper = 10,
          tol   = 0.00001)$root

}

system.time(x1 <- mapply(get_uniroot, df$P, df$B0, df$CF1, df$CF2, df$CF3))
   #user  system elapsed 
   #0.91    0.00    0.90 
system.time(x2 <- mdply(df, get_uniroot))
   #user  system elapsed 
   #5.85    0.00    5.85
system.time(x3 <- foreach(P=df$P, B0=df$B0, CF1=df$CF1, CF2=df$CF2, CF3=df$CF3, .combine = "c") %do% {
    get_uniroot(P, B0, CF1, CF2, CF3)})
   #user  system elapsed 
  # 10.30    0.00   10.36
all.equal(x1, x2$V1) #TRUE
all.equal(x1, x3)    #TRUE

, chanapple Ryan Thompson SO ( doMC, , .), . , foreach, , , , , .

#chunkapply(get_uniroot, list(P=df$P, B0=df$B0, CF1=df$CF1, CF2=df$CF2, CF3=df$CF3))
#Error in { : task 1 failed - "invalid function value in 'zeroin'"

PS: , tol, , uniroot. tol .

+5
3

parallel, R 2.14 . mclapply :

dfm <- as.matrix(df)
result <- mclapply(seq_len(nrow(dfm)),
          function(x) do.call(get_uniroot,as.list(dfm[x,])),
          mc.cores=4L
          )
unlist(result)

mapply, .

...

, . , , , , . uniroot . mapply, mclapply. :

ncores <- 4
id <- floor(
        quantile(0:nrow(df),
                 1-(0:ncores)/ncores
        )
      )
idm <- embed(id,2)

mapply_uniroot <- function(id){
  tmp <- df[(id[1]+1):id[2],]
  mapply(get_uniroot, tmp$P, tmp$B0, tmp$CF1, tmp$CF2, tmp$CF3)
}
result <-mclapply(nrow(idm):1,
                  function(x) mapply_uniroot(idm[x,]),
                  mc.cores=ncores)
final <- unlist(result)

, df , , mapply . , :

> x1 <- mapply(get_uniroot, df$P, df$B0, df$CF1, df$CF2, df$CF3)
> all.equal(final,x1)
[1] TRUE
+6

, "" . ,

bisect <-
    function(f, interval, ..., lower=min(interval), upper=max(interval),
             f.lower=f(lower, ...), f.upper=f(upper, ...), maxiter=20)
{
    nrow <- length(f.lower)
    bounds <- matrix(c(lower, upper), nrow, 2, byrow=TRUE)
    for (i in seq_len(maxiter)) {
        ## move lower or upper bound to mid-point, preserving opposite signs
        mid <- rowSums(bounds) / 2
        updt <- ifelse(f(mid, ...) > 0, 0L, nrow) + seq_len(nrow)
        bounds[updt] <- mid
    }
    rowSums(bounds) / 2
}


> system.time(x2 <- with(df, {
+     f <- function(x, PB0, CF1, CF2, CF3)
+         PB0 + CF1/x + CF2/x^2 + CF3/x^3
+     bisect(f, c(1, 10), PB0, CF1, CF2, CF3)
+ }))
   user  system elapsed 
  0.180   0.000   0.181 
> range(x1 - x2)
[1] -6.282406e-06  6.658593e-06

1.3s uniroot . P B0 , .

+/- diff(interval) * (.5 ^ maxiter) . ( , ?uniroot), ( ).

+3

this is an old topic, but now you have a parallel::mcmapplydoc here . do not forget to set mc.coresin the parameters. I usually use mc.cores=parallel::detectCores()-1one cpu to be freed up for OS operations.

x4 <- mcmapply(get_uniroot, df$P, df$B0, df$CF1, df$CF2, df$CF3,mc.cores=parallel::detectCores()-1)

+1
source

More articles:


All Articles