The problem of defining the area when sfApply is used inside the function (snowfall package - R)

Question

Let me add one more problem of determining the region R, this time with a snowfall package. If I define a function in my global environment, and I try to use it later in sfApply () inside another function, my first function is no longer found:

#Runnable code. Don't forget to stop the cluster with sfStop() require(snowfall) sfInit(parallel=TRUE,cpus=3) func1 <- function(x){ y <- x+1 y } func2 <- function(x){ y <- sfApply(x,2,function(i) func1(i) ) y } y <- matrix(1:10,ncol=2) func2(y) sfStop() 

This gives:

 > func2(y) Error in checkForRemoteErrors(val) : 2 nodes produced errors; first error: could not find function "func1" 

If I use the function most of all inside another function, it works. It also works when I use sfApply () in a global environment. The fact is that I do not want to embed the func1 function inside this function2, since this can lead to the fact that func1 is defined many times (func2 is used in a loop-like structure).

I tried already simplifying the code to get rid of the double loop, but this is completely impossible due to the nature of the problem. Any ideas?

Answer 1

I think you want sfExport(func1) , although I'm not sure if you need to do this in .GlobalEnv or inside func2 . Hope this helps ...

 > y <- matrix(1:10,ncol=2) > sfExport(list=list("func1")) > func2(y) [,1] [,2] [1,] 2 7 [2,] 3 8 [3,] 4 9 [4,] 5 10 [5,] 6 11 

Answer 2

Thinks that now you are confusing parallel computing. You are invoking new R sessions --- and usually your responsibility is to recreate your environment on the nodes.

An alternative would be to use foreach and others. In the foreach docs (or iterator?) There are examples that show just that. Oh, see, and Josh already recommended the same thing.