Geek Answers Handbook

Java Heap Memory Error

I get this error:

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space at com.mysql.jdbc.MysqlIO.nextRowFast(MysqlIO.java:1585) at com.mysql.jdbc.MysqlIO.nextRow(MysqlIO.java:1409) at com.mysql.jdbc.MysqlIO.readSingleRowSet(MysqlIO.java:2886) at com.mysql.jdbc.MysqlIO.getResultSet(MysqlIO.java:476) at com.mysql.jdbc.MysqlIO.readResultsForQueryOrUpdate(MysqlIO.java:2581) at com.mysql.jdbc.MysqlIO.readAllResults(MysqlIO.java:1757) at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2171) at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2562) at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2512) at com.mysql.jdbc.StatementImpl.executeQuery(StatementImpl.java:1476) at DBase.connect(automateExport.java:31) at automateExport.main(automateExport.java:10)

I tried to increase the heap memory space by opening the eclipse.ini file and then changing

 -Xms 256m and -Xmx 512m

But it did not help. I tried 512 m and 1024 m, but it ended up giving an error: the JVM could not start, and the eclipse did not open.

I tried to do the same on the cmd line:

 java -Xms 256m and -Xmx 512m

as well as eclipse -vmargs -Xms 256m and -Xmx 512m But still no help. I basically create a JDBC connection to query the database for a very large set of records. Please help me.

+4
source share
6 answers

When you request the return of very large sets, the mysql driver by default loads the entire result set into memory before giving you control. It seems that you simply do not have enough memory, so increase the memory, for example. -Xmx1024M

Another alternative might be to include streaming results in your MySQL queries - this prevents the entire ResultSet from loading into memory at once. This can be done by doing

 stmt = conn.createStatement(java.sql.ResultSet.TYPE_FORWARD_ONLY, java.sql.ResultSet.CONCUR_READ_ONLY);//These are defaults though,I believe stmt.setFetchSize(Integer.MIN_VALUE);

(Note that nothing but Integer.MIN_VALUE affects the MySQL driver)

+8
source

You do not have to worry too much about -Xms - that the initial heap size at startup.

-Xmx - maximum heap size. Try increasing it until you get an OutOfMemory exception more or until you finish the memory.

If you run out of memory, you cannot immediately load the entire ResultSet, and you will need to apply the approaches that others have talked about.

+1
source

Try splitting your data into multiple result sets. Think about what you want to do with the data as soon as you return from the database. The result set is too large to fit on the heap.

0
source

Should the entire query result be in memory? If so, then you should buy more RAM. If not, breaking up the problem would be the right solution. You can even let the database consolidate data for you in many cases.

If your solution to this problem does not scale well with the amount of data, you will return to this problem, even increasing the amount of RAM. Therefore, a really good solution is to avoid the situation altogether.

0
source

Is Lazy Load applicable? Can you use an upper limit to bind the results of a piecewise query?

0
source

I found that the MySql driver has a memory leak. I use it with tomcat in setting up a connection pool. Loads of JDBCResultSet and StatementImpl objects are on the heap. Setting maxAge in the data source configuration (context.xml) in tomcat helped.

0
source

More articles:


All Articles