How do you "ignore" java.util.concurrent.Future objects?

Question

Can you spot a mistake? This will raise java.lang.OutOfMemoryError .

 import java.util.concurrent.ExecutorService; import java.util.concurrent.Executors; public class TestTheads { public static void main(String[] args) { ExecutorService executorService = Executors.newFixedThreadPool(1); while(true) { executorService.submit(new Runnable() { public void run() { try { Thread.sleep(10); } catch (InterruptedException e) { } } }); } } } 

The error is that I call executorService.submit() instead of executorService.execute() because submit() returns a Future object that I ignore. With execute() this program will work forever.

However, it is not always possible to have a execute() method, for example, when using ScheduledExecutorService :

 public static void main(String[] args) { // this will FAIL because I ignore the ScheduledFuture object ScheduledExecutorService executorService = Executors.newScheduledThreadPool(2); while(true) { executorService.scheduleWithFixedDelay(new Runnable() { public void run() { try { Thread.sleep(10); } catch (InterruptedException e) { } } }, 1, 1, TimeUnit.SECONDS); } } 

What needs to be done with tasks that return nothing, only compute?

Any ideas would be appreciated!

EDIT : ThreadPoolExecutor purge() looked promising, but it only clears canceled tasks.

Answer 1

The returned Future object categorically refers to the ExecutorService only before it is executed. (This is actually an instance of FutureTask that your Runnable delegates.) Once it is executed, it will collect garbage because the caller does not reference it. In other words, the memory issue has nothing to do with Future processing.

If you run out of memory, this is because the work queue has millions of tasks in the queue. As in any queue, if the average consumption rate does not exceed the average production rate, the queue will be filled. The contents of the queue consume memory.

Use a limited queue that will efficiently throttle the task in the queue or get more memory.

This code will work forever:

  ExecutorService executorService = Executors.newFixedThreadPool(1); while(true) { executorService.submit(new Runnable() { public void run() { try { Thread.sleep(10); } catch (InterruptedException e) { } } }); Thread.sleep(12); } 

The difference is not in processing the resulting Future instances, but in the fact that the tasks are queued at the speed with which they can be processed.

Answer 2

This is not so much the futures that come back, this is your problem. The problem is that for every Runnable you submit, the ExecutorService will save them so that it can handle later. Each time you call the submit method using Runnable (or Future), the ExecutorService will push that runnable to the work queue. Runnable will sit there until Thread can select Runnable from the queue (later). If all worker threads are busy, then the ExecutorService will simply put the runnable in the specified queue.

So your problem is that you have only one thread trying to pull out a queue that is endlessly added to another thread. Adding it is much faster than a worker thread can handle each Runnable.

Edit: the code sample that I gave as nos did not execute, in fact, throws a RejectedExecutionException, so the throttling mechanism would have to be slightly different if you chose.

As for the best solution, as I mentioned in my comment; If you expect to fill out the ExecutorService so that the worker threads cannot keep up with the queue, you can serialize and deserialize the requests as they arrive (by building my own ThreadPoolExecutor), but I would make sure such a case is absolutely necessary.

Keep in mind that after the work has been done, they will be discarded and garbage collected. Therefore, if you make one Future per second, and it is fulfilled in a second, then the Future will be deleted and you will not have a memory problem. But if you make one Future second, and streams make the Future every 3 seconds, it will draw and render.

Edit: I have profiled a bunch of the program you are working in, and that is the problem. FutureTask created by ExecutorService sits in the work queue until the workflow selects it

 Class Name | Shallow Heap | Retained Heap | Percentage ------------------------------------------------------------------------------------------------------------ java.util.concurrent.ThreadPoolExecutor @ 0x78513c5a0 | 104 | 2,051,298,872 | 99.99% |- java.util.concurrent.LinkedBlockingQueue @ 0x785140598 | 80 | 2,051,298,216 | 99.99% | |- java.util.concurrent.LinkedBlockingQueue$Node @ 0x785142dd8| 32 | 2,051,297,696 | 99.99% ------------------------------------------------------------------------------------------------------------ 

Heap analysis goes a bit, there are a lot of LinkedBlockingQueue $ Node, as you can imagine