How can I be sure that the compiler is not optimizing my performance test?

Question

I have a class that does some laborious calculations. I am trying to run a test:

int numValues = 1000000; Random random = new Random(); startMeasuringTime(); double result; for (int i = 0; i < numValues; i++) { result = calculatorInstance.doSomeTimeConsumingCalculationsOn(random.nextDouble()); } stopMeasuringTime(); 

I use random values, so the compiler will not optimize the calculations a million times. But what about the results? The compiler sees that it is no longer in use and does not make the call (but can it see any side effects that the method call may have?)

I don’t want to put the results somewhere (in a file, array or in System.out), because I think this will slow down the test with work that I don’t want to measure. Or create an OutOfMemoryError.

Thanks in advance.

EDIT: slightly changed the title

Answer 1

But what about the results? The compiler sees that it is no longer in use and does not make the call (but can it see any side effects that the method call may have?)

It depends. If the JIT compiler can detect that a method call has no side effects, it has the right to optimize it. Moreover, the value of the result is not used. In this case, you can simply measure calls to random.nextDouble() ... or, possibly, to an empty loop.

To be sure that it cannot be optimized, you should probably write it as follows:

 int numValues = 1000000; Random random = new Random(); startMeasuringTime(); double result; for (int i = 0; i < numValues; i++) { result = result + calculatorInstance.doSomeCalculationsOn(random.nextDouble()); } stopMeasuringTime(); System.err.println(result); // Force result to be computed. 

(I assume that calculating the time takes dependent on the argument ...)

---

You also need to consider the JVM workout; those. run this control code several times in the JVM until the measured time has stabilized.

---

Saying that the compiler is "too optimized" is wrong. The compiler really does its job correctly. In any case, an error in the code; that is, "nothing useful."

Answer 2

Make sure that the result is used in some way, for example, by summing and printing at the end. Summing up is a good choice because adding is a very cheap operation.

Answer 3

The compiler does not actually do much optimization (other than calculating the values ​​of constant expressions), since it was found that the JVM does a much better job of optimization.

This is a modern JVM that discovers that code can be embedded (i.e., inserted directly into the calling code instead of making a method call), and it is particularly suitable for empty code, deleting a method call and replacing it - tada - no code. It works very fast, but not very well measured.

In addition, the JVM cannot optimize calls, so your concern about passing different arguments to a forced evaluation is not required. If your method is nontrivial, it will be called.

But your concern for micro tests showing the wrong thing is valid. A much better approach to get an idea of ​​performance is to run a real run with an attached profiler (in JDK 6 there is a simple one in jvisualvm).

What do you need to know?

Answer 4

If in doubt, look at the byte code of this test class. If the compiler "optimized" this call, then you will not find a call to this method.

thod is a small disassembler that comes with jdk. Upload the output to a file, open this file with a standard editor and try to find the name of the method. You do not need to understand the entire byte code if you just want to check if any method is called or used.

---

Did a quick test:

 public static main(String[] args) { for (int i = 0; i < 1000000; i++) { double result = doSomething(Math.random()); } } public double doSomething(double random) { return random * random; } 

For this class, the byte code contains the lines

 invokestatic #22; // Method doSomething:(D)D dstore_2 

which definitely reports that the method is being called and the result is stored in a local variable.

But it is still possible that the virtual machine detects an unused local variable and that the compiler "at the exact moment" excludes the call when compiling real machine code.