Java Compressed Data Library

Question

Is there a third-party Java library that provides a collection that compresses its composite objects? I was looking for her, but came up. Wouldn't such a construction be useful for large (read cards with several concerts) or some such thing? Are they sure there is a penalty for performance when accessing and storing, but for long-lasting rarely found links, this seems reasonable, no?

Answer 1

Use cases for this are mainly used for in-memory databases, so you should probably study them.

If you did this, you would probably have to basically serialize arbitrary java objects into bytes, and then reflect them back into classes. It can also use a database in memory - there is no real difference that I can see anyway, other than this java, probably a bit of a high level for this kind of thing.

Note that this is actually somewhat specific to java - maybe in C you might have a library that will grab memory and just compress it without doing any fancy stuff, but since Java does not have access to memory it makes it a little more complicated ...

Answer 2

MapDB apparently implements Java collection style maps and can do transparent compression (see http://www.mapdb.org/apidocs/org/mapdb/DBMaker.html # compressionEnable () ).

I think it is intended for storage on disks or for storage outside the heap (see http://www.mapdb.org/apidocs/org/mapdb/DBMaker.html # newDirectMemoryDB () ), so you can choose between a disk block or a memory block that does not contain garbage.

Answer 3

It is unlikely that such a collection will not be very useful in most situations.

Data structures are typically designed to provide high performance for a set of specific usage patterns. Adding compression will simply add overhead and slow down their use in common use cases. In particular, note that the most efficient compression algorithms use backlinks to previously seen data. This is usually incompatible (i.e., Impossible to effectively implement) with random access patterns expected from collection classes, and is also incompatible with the possibility of mutation of parts of the collection.

Of course, compression is great for sequentially accessing large chunks of data and processing very large amounts of data, which we need to shuffle between slower storage and main memory. But we already have great tools for the named file systems and databases :-)