Is Hibernate Good For Batch Processing? How about memory usage?

Question

I have a daily batch process that involves selecting a large number of records and formatting the file to send to an external system. I also need to mark these entries as sent so that they are not sent tomorrow.

In my naive JDBC way, I had to prepare and execute an instruction, and then start a loop through a set of records. Since I am only going forward through a set of records, it is not necessary for my application server to store the entire set of results in memory at once. Record groups can be transferred from the database server.

Now let's say I use sleep mode. Don't I get a bunch of objects representing the whole set of results right away in memory?

Answer 1

Hibernate also iterates over a set of results, so only one row is stored in memory. This is the default value. If it loads eagerly, you should say it like that.

Reasons to use Hibernate:

• Someone was Creative with Column Names (PRXFC0315.XXFZZCC12)
• The database design is still in motion and / or you need one place where the column names are mapped to Java.
• Anyway you are using hibernate
• You have complex queries and you do not own SQL

Reasons not to use Hibernate:

• The rest of the application is pure JDBC
• You don't need the power of hibernate
• You have complex queries and you are fluent in SQL
• You need a specific function of your DB to execute SQL

Answer 2

Hibernate offers some options for saving a session.

You can use Query.scroll (), Criteria.scroll () to scroll through JDBC. You can use Session.evict (object object) to remove objects from the session. You can use StatelessSession to suppress a dirty check. And there are a few more performance optimizations, see the Hibernate Documentation.

Answer 3

Sleep mode, since any ORM structure is designed to develop and support systems based on the principle of object-oriented programming. But most databases are relational rather than object-oriented, so in any case, ORM is always a compromise between convenient OOP programming and optimized / most efficient access to the database.

I would not use ORM for specific isolated tasks, but rather as a general architectural choice for the application persistence layer.

Answer 4

In my opinion, I would use NOT Hibernate, since it makes your application much larger and less maintainable, and you cannot optimize the generated sql scripts in a quick way. In addition, you can use all the SQL functionality supported by the JDBC bridge and not limited only to the functionality of sleep mode. Another thing is that you also have limitations that come with each layer of legacy code.

But in the end, this is a philosophical question, and you should do it the way it fits, you think best.

Answer 5

If you have performance issues, then use the JDBC code.

There are a number of well-known pure SQL optimizations that would be very difficult to do in Hibernate.

Select only the columns you are using! (No "select *").

Keep SQl as simple as possible. for example Do not include small reference tables, such as currency codes in the connection. Instead, load the currency table into memory and define the currency descriptions using the program search.

Depending on the minor override of the SQL DBMS, where predicates can significantly affect performance.

If you update / insert, make only every 100 - 1000 updates. those. Do not do every unit of work, but keep a counter so that you do less often.

Take advantage of the aggregate features of your database. If you want to get the total values ​​for the DEPT code, then do it in SQL with "SUM (quantity) ... GROUP BY DEPT".