I have a very noisy dataset with 2000 observations and 42 functions (financial data), and I'm doing binary classification. Here I set up the network using h2o.gridand providing a validation set. I have set epochs=1000, and I intend to stop training when the error in the misclassification does not improve at> = 1% for 5 scoring events ( stopping_rounds=5, stopping_tolerance=0.01). I am interested to know what is the value for epochs, which minimizes the verification error.
hyper_params = list(rho = c(0.9,0.95,0.99),
epsilon = 10^(c(-10, -8, -6, -4)),
hidden=list(c(64, 64)),
activation=c("Tanh", "Rectifier", "RectifierWithDropout"))
grid = h2o.grid("deeplearning", x = predictors, y = response,
training_frame = tempTrain, validation_frame = tempValid,
grid_id="h2oGrid10", hyper_params = hyper_params,
adaptive_rate = TRUE, stopping_metric="misclassification",
variable_importances = TRUE, epochs = 1000,
stopping_rounds=5, stopping_tolerance=0.01, max_w2 = 20)
In accordance with this issue, the solution should be as follows:
gridErr = h2o.getGrid("h2oGrid10", sort_by="err", decreasing=FALSE)
best_model = h2o.getModel(gridErr@model_ids[[1]])
solution = rev(best_model@model$scoring_history$epochs)[1]
Where solution=1000. In any case, checking scoring_history, we observe the following result, which is rather ambiguous.
cbind(best_model@model$scoring_history$epochs,
+ best_model@model$scoring_history$validation_classification_error)
[,1] [,2]
[1,] 0 NaN
[2,] 10 0.4971347
[3,] 160 0.4813754
[4,] 320 0.4770774
[5,] 490 0.4799427
[6,] 660 0.4727794
[7,] 840 0.4713467
[8,] 1000 0.4727794
[9,] 1000 0.4713467
, , 840 1000 . , , . , , stopping_rounds=5 stopping_tolerance=0.01, , - . , , (.. 1,2,... 10 160,...)?
EDIT: 8 . , . , train_samples_per_iteration, . ?