How reliable are the standards running in a virtual machine, as opposed to real equipment?
Let me analyze a specific situation. Suppose we want to compare the performance impact of a recent code change. For simplicity, assume that the workload is completely CPU related (although IO and mixed workloads are also related). Suppose that the machine runs under VirtualBox, because it is the best;)
Suppose we measured the source code and the new code, and the new code was 5% faster (when testing on a virtual machine). Can we say with confidence that on real equipment it will also be 5% faster?
And an even more important part, suppose the new code is 3% slower. Can we be completely sure that on real equipment it will be 3% or less slower, but definitely not worse than 3%?
UPDATE: I am most interested in the results of your battle. I.e. can you witness a case when a code that was 10% slower in a VM ran 5% faster on real hardware or vice versa? Or is it always consistent (i.e. if it is faster / slower in a VM, it is always proportionally faster / slower on a real machine)? Mine is still more or less consistent; at least it always goes in one direction.
source
share