Geek Answers Handbook

General Purpose Distributed Distribution Library for Java

I am looking for some general library for planning many tasks. The library should provide the ability to split tasks into cluster nodes, perform load balancing and fault tolerance, therefore, if some nodes are omitted, tasks for the node should be distributed among the remaining nodes.

I looked at Hadoop - but it looks like it will work well for map reduction tasks. In my case, the tasks are just the senders of notifications, checkers for the state of the object, etc.

Quartz seems great - but it's unclear how good it is when it comes to sending events to nodes.

Any other options?

+7
source share
4 answers

Sounds like a precedent for Akka .

+7
source

I agree with Christian that Akka looks better. Quartz is great for what it does, but its main building block is Job, which must be completed. This does not help you decompose your work into distributed components.

If all your tasks can be broken down into tasks, then Quartz can help you plan them, what he does best. But if the task needs to be decomposed into subtasks, you will need to use a different infrastructure.

Another option might be a spring package depending on your needs.

+3
source

You can try ProActive , a workflow scheduler that integrates well with Java (written in Java). It supports the listed functions. This is an open source project that is part of OW2.

The following functions are available:

  • Workflows i.e. a set of tasks with dependencies and constructs such as loop / if / replication
  • Tasks are performed on nodes, i.e. agents working on different machines.
  • Tasks can be restarted automatically after node failure

[disclaimer] I am part of Activeeon, a professional support company ProActive [/ disclaimer]

+1
source

Or you can also try Quartz , it can handle almost everything you want :)

0
source

More articles:


All Articles