ExecutorService slows down, scares my computer

Question

I am writing a parser for a site, it has many pages (I call them IndexPages). Each page has many links (from 300 to 400 links in IndexPage). I use Java ExecutorServiceto call 12 Callablesat the same time in the same IndexPage. Each Callablesimply launches an HTTP request on one link and performs some actions on parsing and storing db. When the first IndexPage is completed, the program moves to the second IndexPage until the next IndexPage is found.

When working, it seems good, I can keep track of work / work flows well. Each parsing / storage of links takes from 1 to 2 seconds.

, Callable (/) . , , 10 , Callable ( , ). , .

:

ExecutorService executorService = Executors.newFixedThreadPool(12);    
String indexUrl = // Set initial (1st page) IndexPage
while(true)
{
  String nextPage = // parse next page in the indexUrl

  Set<Callable<Void>> callables = new HashSet<>();
  for(String url : getUrls(indexUrl))
  {
    Callable callable = new ParserCallable(url , … and some DAOs);
    callables.add(callable);
  } 

  try {
    executorService.invokeAll(callables);
  } catch (InterruptedException e) {
    e.printStackTrace();
  }

  if (nextPage == null) 
    break;

  indexUrl = nextPage;
} // true
executorService.shutdown();

. , ​​? , ?

CPU/Memory/Heap .

, FYI.

================ >

ExecutorService ForkJoinPool:

ForkJoinPool pool=new ForkJoinPool(12);
String indexUrl = // Set initial (1st page) IndexPage
while(true)
{
  Set<Callable<Void>> callables = new HashSet<>();
  for(String url : for(String url : getUrls(indexUrl)))
  {
    Callable callable = new ParserCallable(url , DAOs...);
    callables.add(callable);
  }
  pool.invokeAll(callables);

  String nextPage = // parse next page in this indexUrl
  if (nextPage == null)
    break;

  indexUrl = nextPage;
} // true

, ExecutorService. ExecutorService 2 , , ForkJoinPool 3 , Callable ( 1 5,6 10 ). , , , ( ).

, ( ) GregorianCalendar, Date SimpleDateFormat . . .

Answer 1

. ExecutorService.invokeAll Callable List List, . , ExecutorService.submit, Callable.

Answer 2

, Callable , "Caller" ParserCallable. , , Runnable.

Callable.call(), BackTask , .

Runnable,

ExecutorService executor = Executors.newFixedThreadPool(12);
for(String url : getUrls(indexUrl))  {
  Runnable worker = new ParserRunnable(url , … and some DAOs);
  executor.execute(worker);
}

class ParserRunnable implements Runnable{
}

Answer 3

, 40 , ~ 300 URL-, ~ 12 000 Callables? , , Callables, HTTPConnections .

, Callable . , . , HTTP-, , 12 000 .

. 40 . , ~ 300 , .