Why are the probabilities and response in ksvm in R not consistent?

Question

I use ksvm from the ksvm package in R to predict probabilities using the type="probabilities" option in predict.ksvm . However, I believe that sometimes using predict(model,observation,type="r") does not give the class with the highest probability given by predict(model,observation,type="p") .

Example:

 > predict(model,observation,type="r") [1] A Levels: AB > predict(model,observation,type="p") AB [1,] 0.21 0.79 

Is this the correct behavior or error? If this is the correct behavior, how can I evaluate the most likely class of probabilities?

---

Try to reproduce an example:

 library(kernlab) set.seed(1000) # Generate fake data n <- 1000 x <- rnorm(n) p <- 1 / (1 + exp(-10*x)) y <- factor(rbinom(n, 1, p)) dat <- data.frame(x, y) tmp <- split(dat, dat$y) # Create unequal sizes in the groups (helps illustrate the problem) newdat <- rbind(tmp[[1]][1:100,], tmp[[2]][1:10,]) # Fit the model using radial kernal (default) out <- ksvm(y ~ x, data = newdat, prob.model = T) # Create some testing points near the boundary testdat <- data.frame(x = seq(.09, .12, .01)) # Get predictions using both methods responsepreds <- predict(out, newdata = testdat, type = "r") probpreds <- predict(out, testdat, type = "p") results <- data.frame(x = testdat, response = responsepreds, Px0 = probpreds[,1], Px1 = probpreds[,2]) 

Result:

 > results x response Px0 Px1 1 0.09 0 0.7199018 0.2800982 2 0.10 0 0.6988079 0.3011921 3 0.11 1 0.6824685 0.3175315 4 0.12 1 0.6717304 0.3282696 

Answer 1

If you look at the decision matrix and voices, they seem to correspond more to the answers:

 > predict(out, newdata = testdat, type = "response") [1] 0 0 1 1 Levels: 0 1 > predict(out, newdata = testdat, type = "decision") [,1] [1,] -0.07077917 [2,] -0.01762016 [3,] 0.02210974 [4,] 0.04762563 > predict(out, newdata = testdat, type = "votes") [,1] [,2] [,3] [,4] [1,] 1 1 0 0 [2,] 0 0 1 1 > predict(out, newdata = testdat, type = "prob") 0 1 [1,] 0.7198132 0.2801868 [2,] 0.6987129 0.3012871 [3,] 0.6823679 0.3176321 [4,] 0.6716249 0.3283751 

Link to kernlab ( ?predict.ksvm ) help pages on paper Probability estimates for classifying several classes using TF Wu, CJ Lin and RC Van pairing.

Section 7.3 states that decisions and probabilities may vary:

... We explain why probability-based results and decision-based methods can be so different. For some problems, the parameters chosen by δDV are very different from the parameters, the other five rules. In the waveform, with some parameters, all probability-based methods give much higher accuracy for checking cross-references than δDV. We observe, for example, the values ​​of the solution for validating the set are in [0.73, 0.97] and [0.93, 1.02] for data in two classes; therefore, all data in the validation sets is classified as in the same class and the error is high. In contrast, probability-based methods correspond to solution values ​​by a sigmoid function, which may be better to separate the two classes by cutting the solution value to about 0.95. This observation shed some light on the difference between probability-based and decision-based methods ...

I do not know enough about these methods to understand the problem, but maybe you are doing it. It seems that there are different forecasting methods with probabilities and some other method, and type=response corresponds to a different method than that used for forecasting probabilities.