Geek Answers Handbook

Java batch job in a cluster environment

We have a cluster with two JBOSS nodes. We have a batch task that loads all user data from the active directory into the database. This work is done every day. Previously, this was performed in a non-cluster environment, and therefore we developed it as a singleton. Now we have a cluster environment, and I do not know what is the best way to achieve the same result. I want the batch job to run only once a day. We use spring and hibernation, and I watched the spring package. I could not get a short answer to my question.

Can someone please tell me if you run the package in a cluster environment? What would be the best solution in this scenario?

+4
source share
3 answers

We implemented this by starting and starting work from the outside through MQ (an HTTP request will also work to get started). The scheduler places the message in the queue, and even if we have "n" nodes that are listening to the queue, one node will receive the message and start the task based on its contents. You can do this with HTTP too.

The real "solution" for this is to schedule the batch job from the outside, rather than through the internal cron trigger. The actual trigger mechanism is secondary to this.

+5
source

Consider also https://github.com/willschipp/spring-batch-cluster , which has

  • batch job repository entry
  • HA for a batch in the cluster (automatic stop and recovery after completing tasks)
+1
source

In general, it is sometimes advisable to externalize / isolate batch jobs from transactional systems, so they do not interfere with availability or performance. At the same time, if there are good reasons for implementing a batch job in a cluster application (simplicity, code reuse, etc.), then in addition to the already mentioned solutions, ShedLock is an excellent option.

Take a look at the blog post about batch jobs in clustered environments .

0
source

More articles:


All Articles