How to solve the problem "overflow of the security stack" in R Studio

Question

I am trying to create a model with the glmnet package, but I get the following error when I run the following line:

#library('glmnet') x = model.matrix(response ~ ., data = acgh_frame[,c(3:ncol(acgh_frame))]) Error: protect(): protection stack overflow 

I know this is due to my many variables (26k +) in the dataframe. When I use fewer variables, the error is not displayed. I know how to solve this on the R command line, but I need to stay in R studio, so I want to fix it from R Studio. So how do I do this?

Answer 1

@ Ansjovis86

You can specify ppsize as a command line argument for Rstudio

 rstudio.exe --max-ppsize=5000000 

You can also set the expression parameter via .Rprofile or at run time using the options(expressions = 5e5) command options(expressions = 5e5) .

 > options(expressions = 5e5) >?options 

...

expressions:

sets a limit on the number of nested expressions to be evaluated. Valid values: 25 ... 500000 by default 5000. If you increase it, you can also start R with a large stack of protection; see --max-ppsize in memory. Also note that you can call segfault from C, and in the OS, where possible, you can increase it. Once the limit is reached, an error is issued. The current estimated number can be found by calling Cstack_info .

 Cstack_info() - to determine current setting.s 

Answer 2

The root cause is the model.matrix function, which will 1) use a lot of memory; and 2) throwing this error is not big enough. columns.

Try using my glmnetUtils package, which will work around both of these problems. Instead of building a model matrix at a time, it does it terminally; nor does he try to evaluate huge formulas. It is much faster and does not run the risk of blowing up the stack.

 install.packages("glmnetUtils") library(glmnetUtils) glmnet(response ~ ., data = acgh_frame[3:ncol(acgh_frame)])