Implement LIME for modeling h2o in R

Question

I want to implement LIME in a model created using h2o (deep learning) in R. To use the data in the model, I created h2oFrames and converted it back to a dataframe before using it in LIME (lime function because the LIME explanation function cannot recognize h2oFrame). Here I can run the function

The next step is to use the explanation function for the test data to generate explanations. Here R raises an error for using a data frame as well as h2oFrame.

This is an error that occurs when using a data frame:

Error in chk.H2OFrame(x) : must be an H2OFrame 

This is the error that occurs when using h2oframe:

 Error in UseMethod("permute_cases") : no applicable method for 'permute_cases' applied to an object of class "H2OFrame" 

 if(!require(pacman)) install.packages("pacman") pacman::p_load(h2o, lime, data.table, e1071) data(iris) h2o.init( nthreads = -1 ) h2o.no_progress() # Split up the data set iris <- as.h2o(iris) split <- h2o.splitFrame( iris, c(0.6, 0.2), seed = 1234 ) iris_train <- h2o.assign( split[[1]], "train" ) # 60% iris_valid <- h2o.assign( split[[2]], "valid" ) # 20% iris_test <- h2o.assign( split[[3]], "test" ) # 20% output <- 'Species' input <- setdiff(names(iris),output) model_dl_1 <- h2o.deeplearning( model_id = "dl_1", training_frame = iris_train, validation_frame = iris_valid, x = input, y = output, hidden = c(32, 32, 32), epochs = 10, # hopefully converges earlier... score_validation_samples = 10000, stopping_rounds = 5, stopping_tolerance = 0.01 ) pred1 <- h2o.predict(model_dl_1, iris_test) list(dimension = dim(pred1), pred1$predict) #convert to df from h2ofdataframe train_org<-as.data.frame(iris_train) #converting train h2oframe to dataframe sapply(train_org,class) #checking the class of train_org test_df <- as.data.frame(iris_test) #converting test data h2oFrame to dataframe test_sample <- test_df[1:1,] #works #lime is used to get explain on the train data explain <- lime(train_org, model_dl_1, bin_continuous = FALSE, n_bins = 5, n_permutations = 1000) # Explain new observation explanation <- explain(test_sample, n_labels = 1, n_features = 1) h2o.shutdown(prompt=F) 

Can someone help me find a solution or a way to use the LIME explanation function using the appropriate dataFrame

Answer 1

In the lime package under the hood, two functions predict_model() and model_type() , which must be configured for any models that are not currently supported.

For your specific example, here is what you need to do.

Step 1 Configure the generic model_type function for models of the H2OMultinomialModel class. All you do here is tell lime what type of model you want to perform, such as “classification” or “regression”.

 model_type.H2OMultinomialModel <- function(x, ...) { # Function tells lime() what model type we are dealing with # 'classification', 'regression', 'survival', 'clustering', 'multilabel', etc # # x is our h2o model return("classification") } 

Step 2 Set up the universal function predict_model for models of class H2OMultinomialModel . The key point here is the understanding that lime requires a classification probability rather than a prediction (it took me a little time to figure it out and it must deal with the variable lime:::output_type(explaination) ).

 predict_model.H2OMultinomialModel <- function(x, newdata, type, ...) { # Function performs prediction and returns dataframe with Response # # x is h2o model # newdata is data frame # type is only setup for data frame pred <- h2o.predict(x, as.h2o(newdata)) # return classification probabilities only return(as.data.frame(pred[,-1])) } 

After properly installing these functions, you can run the lime scripts.

 # Lime is used to get explain on the train data explainer <- lime(train_org, model_dl_1, bin_continuous = FALSE, n_bins = 5, n_permutations = 1000) # Explain new observation explanation <- explain(test_sample, explainer, n_labels = 1, n_features = 1)