Saving JPA using multiple threads

Question

I have a problem while trying to save objects using multiple threads .

Details:

Suppose I have a PaymentOrder object that has a list of PaymentGroup relationships (from one to many), and PaymentGroup contains a list of CreditTransfer (From one to another).

Since the number of CreditTransfer large (in lakhs), I grouped it based on PaymentGroup (based on some business logic) and creating WORKER flows (one stream for each payment group) to form PaymentOrder objects and commit to the database.

The problem is that each workflow creates each of PaymentOrder (which contains a unique set of PaymentGroup s).

The primary key for all permissions is generated automatically.

So, there are three tables: 1. PAYMENT_ORDER_MASTER, 2. PAYMENT_GROUPS, 3. CREDIT_TRANSFERS, all are mapped from one to large.

Because of this, when the second thread tries to save its group in the database, the infrastructure tries to keep the same PaymentOrder that was committed by the previous thread, the transaction failed due to some other unique field restrictions ( PaymentOrder ).

Ideally, this should be 1..n..m ( PaymentOrder → PaymentGroup --> CreditTransfer`)

What I need to achieve is if there is no PaymentOrder entry in the PaymentOrder , enter the entry, if it is there, do not enter the entry in PAYMENT_ORDER_MASTER , but only in PAYMENT_GROUPS and CREDIT_TRANSFERS .

How can I prevent this problem by maintaining the logic split-master-payment-order-using-groups and multiple threads?

Answer 1

You have options.
1) Primitive, but simple, catch the key violation error at the end and try to paste without parents again. Assuming your parents are truly unique, you know that the other stream was just made by the parents ... go on with the children. This may not work well compared to other options, but maybe you are getting the pop you need. If you had a high% of parents with one child, this would work well.

2) Change the level of read consistency. This is a specific provider, but sometimes you can read uncommitted transactions. This will help you see how other threads work before committing. This is not perfect, you still have to do # 1, as another thread may sneak after reading. But it can increase your throughput at the expense of more complexity. It may not be possible, on the basis of the RDBMS (or maybe it can happen, but only at the database level, to ruin other applications!)

3) Implement a work queue with a single-threaded consumer. If the main expensive work of the program is up to the level of persistence, you can "insert" your threads into the work queue, where the keys are not used. Then pull one thread out of the work queue and continue. A work queue can be in memory, in another table, or in a specific vendor location (Weblogic Queue, Oracle AQ, etc.). If the main work of the program is before saving, you parallelize THAT and return to one thread in the inserts. You can even work with consumers in batch insert mode. Sweeeeeeeet.

4) Relax your restrictions. Who really cares if there are two parents for the same child with the same information? I'm just asking. If you don’t need faster updates for parent information, and you can modify your readers to understand this, it may work well. This will not give you an “A” in the DB design class, but if it works .....

5) Enter the deadlock table. I hate this solution, but it really works - write that it works on the parent "x" and no one can (and commit) its first transaction. As a rule, this leads to the same problem (and others - clearing records later, etc.), but it can work when child inserts are slow and single-line insert is fast. You will still have collisions, but less.

Answer 2

Sleep mode is not thread safe. The JDBC connections that underlie Hibernate are not thread safe. Consider the multithreading of your business logic so that each thread uses its own Hibernate session and JDBC connection. Using the thread pool, you can further improve your code by adding the ability to throttle the number of simultaneous threads.