How to prevent the simultaneous loading of an uncached value multiple times?

Question

How to prevent loading a value that is not in the cache multiple times at the same time, in an efficient way?

A typical use of the cache is the following pseudo-code:

Object get(Object key) { Object value = cache.get(key); if (value == null) { value = loadFromService(key); cache.set(key,value); } return value; } 

Problem: before loading the value from the service (Database, WebService, RemoteEJB or something else), the second call can be made at the same time, which will start the value again.

For example, when I cache all the elements for user X, and this user is often browsed and has many elements, there is a high probability of simultaneously invoking a load of all its elements, which leads to a large load on the server.

I could make the get function synchronized , but this will make other search queries wait, which doesn't make much sense. I can create a new lock for each key, but I don’t know whether it is worth managing so many locks in Java (this part depends on the language, the reason I marked it as java ).

Or is there another approach I could use? If so, what would be most effective?

Answer 1

Do not reinvent the wheel, use guava LoadingCache or a vendor memorandum .

If you use Ehcache, read about read-through , this is the template you are asking for. You must implement the CacheEntryFactory interface to instruct the cache on how to read objects when the cache misses, and you must wrap the Ehcache instance with the SelfPopulatingCache instance.

Answer 2

Something you can do in the general case is to use the hash code of the object.

You can have an array of locks that are used based on hashCode to reduce the chance of collisions. Or as a hack, you can use the fact that bytes with automatic shorts always return the same objects.

 Object get(Object key) { Object value = cache.get(key); if (value == null) { // every possible Byte is cached by the JLS. Byte b = Byte.valueOf((byte) key.hashCode()); synchronized (b) { value = cache.get(key); if (value == null) { value = loadFromService(key); cache.set(key, value); } } } return value; } 

Answer 3

During the download, insert an intermediate object on the map instead of the result to indicate that the download has started but not finished. Below java.util.concurrent.FutureTask is used for an intermediate object:

 Object get(final Object key) throws Exception { boolean doRun = false; Object value; synchronized (cache) { value = cache.get(key); if (value == null) { value = new FutureTask(new Callable() { @Override public Object call() throws Exception { Object loadedValue = loadFromService(key); synchronized (cache) {cache.put(key, loadedValue);}; return loadedValue; } }); cache.put(key, value); doRun=true; } } if (value instanceof FutureTask) { FutureTask task = (FutureTask) value; if (doRun) { task.run(); } return task.get(); } return value; }`