Is there a way to configure a neural network with multiple hidden layers using the mlp method in a caret package?

Question

The mlp method in the caret package calls the mlp function in RSNNS . In the RSNNS package RSNNS I can configure as many hidden layers in the neural network as possible by setting the size parameter, for example.

 data(iris) #shuffle the vector iris <- iris[sample(1:nrow(iris),length(1:nrow(iris))),1:ncol(iris)] irisValues <- iris[,1:4] irisTargets <- decodeClassLabels(iris[,5]) #irisTargets <- decodeClassLabels(iris[,5], valTrue=0.9, valFalse=0.1) iris <- splitForTrainingAndTest(irisValues, irisTargets, ratio=0.15) iris <- normTrainingAndTestSet(iris) model <- mlp(iris$inputsTrain, iris$targetsTrain, size=c(5,7), learnFuncParams=c(0.1), maxit=50, inputsTest=iris$inputsTest, targetsTest=iris$targetsTest) 

A neural network will be created with two hidden layers of 5 and 7 nodes, respectively. I want to use the caret package because it has the functionality to search for parameters / models, as well as parallel implementations for the cluster. In caret , when I look at a method, it can be configured with only one size parameter, for example.

 data(iris) mlpGrid <- data.frame(.size=3) model2<-caret::train(Species~. , iris, method='mlp', tuneGrid=mlpGrid) 

Installs a neural network with a single hidden layer of 3 node.

I tried adding other columns to mlpGrid and such, but caret doesn't seem to allow adding a second (or more) hidden layer.

Answer 1

You must use the " mlpML " method set to "mlp". It uses the mlp function from RSNNS, but you can determine the number of neurons per hidden layer separately. For example, the following code should do the job. You define your custom grid with the definition of your layers, each layer ( 1 , 2 and 3 ) and the number of neurons per layer.

 mlp_grid = expand.grid(layer1 = 10, layer2 = 10, layer3 = 10) mlp_fit = caret::train(x = train_x, y = train_y, method = "mlpML", preProc = c('center', 'scale', 'knnImpute', 'pca'), trControl = trainControl(method = "cv", verboseIter = TRUE, returnData = FALSE), tuneGrid = mlp_grid) 

Given verboseIter=TRUE it shows that the values ​​were actually applied

 + Fold01: layer1=10, layer2=10, layer3=10 + Fold02: layer1=10, layer2=10, layer3=10 + Fold03: layer1=10, layer2=10, layer3=10 ... 

Answer 2

The short answer is that I don’t think Caret supports networks with multiple hidden layers using the mlp method.

If you are configured to use mlp (or it needs to call RSNSS), you might think of combining the networks together, as when applying the output of the first to the inputs of the second, simulating another hidden layer. This, obviously, would be harder to implement and lose many of the optimizations that make you want to use Caret in the first place.

Caret, on the other hand, offers an alternative that uses the neuralnet package (method = 'neuralnet). This will allow you to use Caret with a neural network with several hidden layers. However, the documentation shows that only 3 hidden layers are supported.