I am experimenting with Gradle's ability to run tests in parallel. The main setup I found is the maxParallelForks Test property. I expected the behavior of this setting to be similar to how Executors.newFixedThreadPool execute tests. Namely, a fixed number of threads (processes in the case of Gradle) are executed simultaneously; whenever one thread finishes, a new one is activated in the pool.
However, the behavior of Gradle is fundamentally different in a less optimal way. It seems that Gradle divides the test classes into a number equal to the maxParallelForks of the groups, and then Gradle spawns a process for each group and allows these processes to run in parallel. The problem with this strategy is obvious: it cannot dynamically adjust execution depending on the time it takes for the test class.
For example, suppose you have 5 classes and maxParallelForks is 2. In five classes, there are slow classes and the rest are relatively fast. An ideal strategy would be to let one process run slow and the other fast. However, what Gradle does is group slow together with one or two fast and creates two processes to execute two groups of classes, which, of course, is less optimal than the ideal case.
Here is a simple demo.
Slow class:
class DemoTest { @Test void one() { Thread.sleep( 5000 ) println System.getProperty('org.gradle.test.worker') + ": " + new Date().format('HH:mm:ss') assert 1 == 1 } @Test void two() { Thread.sleep( 5000 ) println System.getProperty('org.gradle.test.worker') + ": " + new Date().format('HH:mm:ss') assert 1 == 1 } }
Quick classes (DemoTest2-4, with the same class):
class DemoTest2 { @Test void one() { Thread.sleep( 1000 ) println System.getProperty('org.gradle.test.worker') + ": " + new Date().format('HH:mm:ss') assert 1 == 1 } @Test void two() { Thread.sleep( 1000 ) println System.getProperty('org.gradle.test.worker') + ": " + new Date().format('HH:mm:ss') assert 1 == 1 } }
All classes are in the junit package, which happens to coincide with the name of a well-known test environment junit
Here is a possible conclusion:
junit.DemoTest2 > one STANDARD_OUT 2: 14:54:00 junit.DemoTest2 > two STANDARD_OUT 2: 14:54:01 junit.DemoTest4 > one STANDARD_OUT 2: 14:54:02 junit.DemoTest4 > two STANDARD_OUT 2: 14:54:03 junit.DemoTest > one STANDARD_OUT 3: 14:54:04 junit.DemoTest > two STANDARD_OUT 3: 14:54:09 junit.DemoTest3 > one STANDARD_OUT 3: 14:54:10 junit.DemoTest3 > two STANDARD_OUT 3: 14:54:11 junit.DemoTest5 > one STANDARD_OUT 3: 14:54:12 junit.DemoTest5 > two STANDARD_OUT 3: 14:54:13
As you can see, the slow DemoTest class DemoTest grouped with two fast classes. The total runtime is about 13 seconds, which would be 10 seconds if the fast classes were grouped together.
So, is there an easy way to optimize this behavior in Gradle without resorting to a custom running JUnit?
Many thanks.