Your processor has 8 cores, not threads. This actually means that only 8 things can work at any time. This does not mean that you are limited to only 8 threads.
When a thread synchronously opens a connection to a URL, it will often sleep while it waits for the remote server to return to it. Although this thread has slept, other threads may do the work. If you have 500 threads and all 500 are asleep, you are not using any of your processor cores.
On the flip side, if you have 500 threads and all 500 threads want to do something, they cannot all work right away. There is a special tool to handle this scenario. Processors (or, most likely, the operating system or some combination thereof) have a scheduler that determines which threads will actively work on the processor at any given time. There are many different rules, and sometimes random activity, that controls the work of these planners. This may explain why in the above example, thread 3 always seems to end first. Perhaps the scheduler prefers thread 3 because it was the last thread to be scheduled by the main thread, sometimes it is impossible to predict the behavior.
Now answer the question about performance. If opening a connection never included a dream, then it would not matter if you process things synchronously or asynchronously, you will not be able to get a performance boost above 8 threads. In fact, a lot of the time spent opening a connection is spent sleeping. The difference between asynchronous and synchronous is how to handle the time spent sleeping. Theoretically, you should have almost equal performance between them.
With a multi-threaded model, you simply create more threads than there are kernels. When the threads fall into sleep, they allow other threads to work. This can sometimes be easier to handle because you do not need to write any planning or interaction between threads.
With an asynchronous model, you only create one thread per core. If this thread needs to sleep, it does not sleep, but in fact it should have a code to switch to the next connection. For example, suppose there are three steps when opening a connection (A, B, C):
while (!connectionsList.isEmpty()) { for(Connection connection : connectionsList) { if connection.getState() == READY_FOR_A { connection.stepA(); //this method should return immediately and the connection //should go into the waiting state for some time before going //into the READY_FOR_B state } if connection.getState() == READY_FOR_B { connection.stepB(); //same immediate return behavior as above } if connection.getState() == READY_FOR_C { connection.stepC(); //same immediate return behavior as above } if connection.getState() == WAITING { //Do nothing, skip over } if connection.getState() == FINISHED { connectionsList.remove(connection); } } }
Please note that in no case does the thread sleep, so it makes no sense to have more threads than your kernel. Ultimately, the question of whether to use a synchronous approach or an asynchronous approach is a matter of personal preference. Only at absolute extremes will there be differences in performance between them, and you will need to spend a long time profiling to get to the point where this is a bottleneck in your application.
It looks like you are creating a lot of threads and not getting a performance boost. There may be several reasons for this.
- It is possible that you made the connection really awake, in which case I would not expect performance to increase in 8 threads. I do not think that's possible.
- It is possible that all threads share a common resource. In this case, other threads cannot work because the sleeping thread has a shared resource. Is there any object that shares all threads? Does this object have any synchronized methods?
- Perhaps you have your own synchronization. This may create the problem mentioned above.
- Perhaps each thread should do some installation / distribution work that wins the advantage you get using multiple threads.
If I were you, I would use the JVisualVM tool to profile your application when working with a small number of threads (20). JVisualVM has a beautiful color flow chart that will show when threads are running, blocking or sleeping. This will help you understand the thread / kernel relationship, as you will see that the number of threads running is less than the number of cores you have. In addition, if you see a lot of blocked threads, this can lead you to your bottleneck (if you see that many blocked threads use JVisualVM to dump the thread at this point in time and see what the threads are blocked on).