JMS Queue Split. Integration of enterprises. Apache camel

Question

I have a third-party application that puts some messages in the JMS queue. I also have an application that reads messages from this queue. Depending on the type of message, I save this message in the database or send it to a third-party service. In addition, we should not exceed the limit of fixed calls per second, so as not to overload the third-party.

There are currently two solutions being used for my use.

The first is to ask a third party to send some custom headers so that the JMS consumer can filter messages using JMS Selectors. Thus, in this case, we will be able to create two consumers, the first one will be able to read messages and save them in the database, and the second will use the throttling / polling mechanism to send messages to a third party at a certain load, But this approach will not work for me, because third parties are required to add these custom headers. Something like this in Camel:

from("jms:queue?selector=firstSelector") .bean(dbSaver); from("jms:queue?selector=secondSelector") .throttle(10) .bean(httpClient); 

The second is to create two more JMS queues and a processor that will separate the messages between these queues. Then comes the same logic as in the first decision. However, this means that you need to add 2 additional JMS queues. In the Camel:

 from("jms:parentQueue") .choice() .when(body().contains(...)) .to("jms:fistChildQueue") .otherwise() .to("jms:secondChildQueue") .end() from("jms:fistChildQueue") .bean(dbSaver); from("jms:secondChildQueue") .throttle(10) .bean(httpClient); 

Also, I was thinking about using two in-memory queues instead of JMS queues. However, in this case, if there are a lot of messages in the JMS queue, we will easily run into memory problems.

Can anyone suggest architectural design for this use case? It would be great to see him in the style of a camel route.

Answer 1

1. Do you really need a queue for a thread in the database? You could have a bean (dbSaver) on the first route or abstract it on a "direct" route instead of a route using jms. So you have two queues instead of three.

2. The second approach. If you have control over the database, you can write a message of the second type to another table. Then, the sql consumer can poll the records and delete them, since it consumes them, and passes them to the http service. However, the table acts as "collapses its own Q". Probably more work for a small payback, so perhaps the second turn is better.

3. Finally, I am wondering if you can reuse the same queue. I see an option that will allow you to write back in the same queue. You can add a title and write a specific message. This may seem confusing, and an error can create an endless loop.

If you are already using JPA, this can be done a little easier using the camel-jpa component. As a consumer, he acts, reads, and deletes records (default behavior). I do not think that the SQL / JDBC components have something like this out of the box.