How to manage memory when using ExecutorService?

Question

I have a large set of tasks that need to be done with multithreading .

I use ExecutorService to send a large number of Callable objects , and each Callable object contains several resource that are needed to execute.

Problem

These tasks are so many that the Callable objects presented in the ExecutorService take up too much memory, then the heap exhausted.

I want to know when the JVM free up space for these Callable objects ?

It does this right after the task is completed, and how can I manage the memory used for the ExecutorService ?

I want to control the ExecutorService view, for example, when there is not enough memory, and then block sending, some of the completed tasks and free space are performed. Could it be?

---

All the code is too complicated, I will attach the main frame of my code.

 public class MyCallable implements Callable<Result> { ... // Some members public MyCallable(...) { ... } @Override public Result call() throws Exception { ... // Doing the task } } class MyFutureTask extends FutureTask<Result> { ... // Some members public MyFutureTask(...) { super(new MyCallable(...)); } @Override public void done() { Result result = get(); ... // Something to do immediately after the task is done } } // Here is the code to add tasks ExecutorService executor = Executors.newFixedThreadPool(threadPoolSize); while (...) { ... // Reading some data from files executor.submit(new MatchFutureTask(...)); } executor.shutdown(); 

Answer 1

If you use ExecutorService, you must pass the BlockingQueue to the constructor, right? I assume that you are using ThreadPoolExecutor. Use a fixed-bandwidth ArrayBlockingQueue and pass ThreadPoolExecutor.CallerRunsPolicy to the ThreadPoolExecutor constructor. This will quite effectively limit the feed rate if you set MaximumPoolSize to some reasonable value.

To answer your first question, the virtual machine will free the memory “some time after”, there are no more references to objects. If the Callable is autonomous in terms of state, after it is completed, the VM must collect garbage before you receive an error from memory.

Answer 2

The Runtime.getRuntime().freeMemory() method tells you how much free memory the JVM has. You can write a ThreadFactory implementation that checks this value before creating a new thread.

Answer 3

It was believed that using Executors.newCachedThreadPool () reduces memory by 30% - 60%