How to save list items on disk instead of memory in Java

Question

I am looking for a data structure, the same as ArrayList in Java, which stores elements on disk instead of memory. Does such a data structure have Java? Thanks

I want to have a dynamic structure that stores elements in memory, and when its size exceeds a certain value, save new elements on disk until the size is lower than the value.

Answer 1

You can do it yourself: Serialize ArrayList to disk.

Make sure the contents of the list are serializable, i.e. objects in the list must implement the Serializable interface.

then
to write to a file:

 ObjectOutputStream oos = new ObjectOutputStream(new FileOutputStream(fileName)); oos.writeObject(list); oos.flush(); oos.close(); 

To read from a file:

 ObjectInputStream ois = new ObjectInputStream(new FileInputStream(fileName)); List<YourClass> list = ois.readObject(); ois.close() 

Answer 2

If you often need to change the data in this ArrayList by changing the disk image, then why not think of something like http://www.hibernate.org/ ? Well, this is a lot more complicated than an ArrayList, but it also gives more features.

Answer 3

Just to complete the answer set :)

XStream

XStream is a simple library for serializing objects into XML and back again.

 Person joe = new Person("Joe", "Walnes"); joe.setPhone(new PhoneNumber(123, "1234-456")); joe.setFax(new PhoneNumber(123, "9999-999")); 

Now, to convert it to XML, all you have to do is make a simple XStream call:

 String xml = xstream.toXML(joe); 

The resulting XML is as follows:

 <person> <firstname>Joe</firstname> <lastname>Walnes</lastname> <phone> <code>123</code> <number>1234-456</number> </phone> <fax> <code>123</code> <number>9999-999</number> </fax> </person> 

Answer 4

Check out http://jakarta.apache.org/jcs/ . This allows you to control objects on the disc and drum. Another possible solution might use http://code.google.com/p/pcollections/

Answer 5

MapDB ( mapdb.org ) is a library that stores Java objects on disk in different collections: sets, maps, queues.

It supports caching, so you can have the most frequently used items in memory.

Answer 6

Another way to preserve material that has the advantage that language and transport are neutral is to use Protocol Buffers .

Answer 7

see https://dzone.com/articles/a-filebasedcollection-in-java-for-big-collections

 try(FileBasedCollection<Integer> fbc = new FileBasedCollection<>()) { for(int i = 1; i < 1000_000; i++) { fbc.add(i); } long sum = 0; try(FileBasedIterator<Integer> iter = fbc.iterator()) { while(iter.hasNext()) { Integer i = iter.next(); sum += i; } } catch (IOException e) { // TODO Auto-generated catch block e.printStackTrace(); } System.out.println("sum: " + sum); }