Multi: Threading - Is this Right?

Question

Experts -

I need advice in the following scenario.

I have a configuration file with a list of tasks. Each task can have zero, one or more dependencies. I wanted to complete these tasks in parallel [right now they are being done sequentially]

The idea is to have a main program for reading the configuration file and loading all tasks. Read the individual tasks and pass them on to the executor [the called person], who will complete the task and return the results to the Future. When a task is sent to the executor (thread), it will track that its dependencies end first and perform its own task.

Is this the right approach? Are there any other more efficient approaches using java 1.5 features?

Answer 1

Sounds good, but beware of Thread starvation deadlock . Basically, do not use a limited thread pool.

Here is an example illustrating this problem.
http://www.javaconcurrencyinpractice.com/listings/ThreadDeadlock.java

In addition, if you have, for example, a federated database connection, you may run into a problem. 10 threads can be blocked, holding the entire joint connection, waiting for the 11th thread that the 11th pooling cannot receive, because it is no more.

Answer 2

What you described is the Java5 / 6 approach.

Just make sure your tasks / Callables are thread safe / do not share state.

Answer 3

Consider using ValueFuture (from guava ) to communicate between tasks. Add a listener to each ValueFuture that submits a new task to the ExecutorService only when all its inputs are available. This way, you will never have an end-to-end thread executing a task, waiting for another task to complete.

Answer 4

Database activity, archiving and rule processing - ideal use for Ant (or even Maven)

Do not invent this. Just use Ant.

Write Ant Tasks for each task. Define dependencies in XML using Ant dependency rules.

Turn Ant freely to do dependency-based tasks. You can start with the "executable" Ant Tasks ( http://ant.apache.org/manual/tasksoverview.html#exec ), and not define your own subclass of the task.

Ant understands parallel tasks. You do not need to do anything other than declare tasks parallel. http://ant.apache.org/manual/Tasks/parallel.html

Do not invent new software for this. Use Ant.

Answer 5

In fact, you do want multithreading when the cost of running multiple processes is expensive.

Which, frankly, is almost something that spins a copy of the JVM.

To answer an OP, ANT can run tasks in parallel, but NOT targets in parallel. Thus, even if you have a build.xml file that you want to modify for parallel operation, you will have to make a possible basic rewrite. (I think it was a rather strange design decision made by the ANT developers since you already got your dependency graph described by the targets themselves.)