Yes, your research points in the right direction: the same class definition loaded by different class loaders is considered by the JVM as two different classes. So casting between the two fails with ClassCastException.
I think the difference is simply because the game has two different classes of tokens. This should be so, since classes loaded by different loaders can be different versions of the same class. It is known that a class token for each class is unique (within the same loadload class as it is). It will open a can of worms if the JVM starts comparing class tokens by their various attributes, and not by physical equality ( ==).
source
share