ObjectInputStream from a file causing a memory leak

Question

I have a huge file with a list of objects written by ObjectOutputStream, one by one.

for (Object obj : currentList){ oos.writeUnshared(obj); } 

Now I want to read this file using an ObjectInputStream. However, I need to read several files at the same time, so I can not read the entire file in memory. However, using an ObjectInputStream causes an "Error from memory" error. From what I read, this is because the ObjectInputStream has a memory leak and maintains references to read objects even after they are returned.

How can I ask ObjectInputStream not to support the link to all its reading?

Answer 1

A possible solution is to call the reset () method on ObjectOutputStream: "This will ignore the state of any objects already written to the stream. The reset state will be the same as the new ObjectOutputStream. The current point in the stream is marked as reset, so the corresponding ObjectInputStream will be reset to that same point. " (extracted from java documentation) Performing a reset on an ObjectOutputStream also resets the state of ObjectInputStream.

I assume you can also control your ObjectOutputStreams objects?

Answer 2

When you use writeUnshared on the write side, you have already completed one half of the job. If you now use readUnshared on the input side rather than readObject , ObjectInputStream will not support object references.

You can use the following program to check the behavior:

 package lib.io; import java.awt.Button; import java.io.*; import java.lang.ref.Reference; import java.lang.ref.ReferenceQueue; import java.lang.ref.WeakReference; import java.util.concurrent.ConcurrentHashMap; public class ObjectInputStreamReferences { public static void main(String[] args) throws IOException, ClassNotFoundException { final int numObjects=1000; Serializable s=new Button(); ByteArrayOutputStream os=new ByteArrayOutputStream(); try( ObjectOutputStream oos=new ObjectOutputStream(os) ) { for(int i=0; i<numObjects; i++) oos.writeUnshared(s); } final ConcurrentHashMap<WeakReference<?>, Object> map =new ConcurrentHashMap<>(); final ReferenceQueue<Object> q=new ReferenceQueue<>(); new Thread(new Runnable() { public void run() { reportCollections(map, q); } }).start(); try(ObjectInputStream ois= new ObjectInputStream(new ByteArrayInputStream(os.toByteArray()))) { for(int i=0; i<numObjects; i++) { Object o=ois.readUnshared(); map.put(new WeakReference<>(o,q), ""); o=null; System.gc();Thread.yield(); } } System.exit(0); } static void reportCollections( ConcurrentHashMap<WeakReference<?>, Object> map, ReferenceQueue<?> q) { for(;;) try { Reference<?> removed = q.remove(); System.out.println("one object collected"); map.remove(removed); } catch(InterruptedException ex){} } }