I saved a random forest model in a file using Weka 3.7.9, and now I'm trying to evaluate it against another (very large) set (on some large machines in Amazon EC2). I use the following command line:
> java -server -Xmx60000m -cp weka.jar weka.classifiers.Evaluation weka.classifiers.trees.RandomForest -T test.arff -l random-forest.model -i -no-cv
However, the only conclusion I have is something like this:
=== Error on test data === Correctly Classified Instances 3252532 80.0686 % Incorrectly Classified Instances 809651 19.9314 % Kappa statistic 0.2884 Mean absolute error 0.2539 Root mean squared error 0.3608 Coverage of cases (0.95 level) 98.7413 % Total Number of Instances 4062183
While I also look at something like this:
=== Detailed Accuracy By Class === TP Rate FP Rate Precision Recall F-Measure MCC ROC Area PRC Area Class 0.804 0.295 0.731 0.804 0.766 0.512 0.826 0.803 buyer 0.705 0.196 0.783 0.705 0.742 0.512 0.826 0.798 non-buyer Weighted Avg. 0.755 0.245 0.757 0.755 0.754 0.512 0.826 0.801 === Confusion Matrix === ab <-- classified as 61728 15004 | a = buyer 22662 54066 | b = non-buyer
Please note that even if I run the full training method again, for example:
> java -Xmx60000m -cp weka.jar weka.classifiers.Evaluation weka.classifiers.trees.RandomForest -t train.arff -T test.arff -I 10 -K 0 -S 1 -num-slots 8 -d random-forest.model -i -no-cv
I still do not show the confusion matrix for test data (prepared data only).
source share