Search for Teracotta and Hibernate

Question

Does anyone have experience using Terracotta with Hibernate Search to satisfy application requests?

If yes:

• What is the size of the "update object" can it work? (Like performance)

• What performance do Queries have?

• Is it possible to use Hibernate Search terracotta without even having a support database to satisfy all the “queries” in memory?

Answer 1

I am the technical director of Terracotta. I spent some time last month looking at Hibernate Search. It is not built to be transparent with Terracotta. That's why in a nutshell: Hibernate has custom JMS replication of Lucene indexes in the JVM.

The basic idea in the Search is that communicating with a local disk under lucene works very well, while fragmenting or splitting Lucene indexes over the network introduces sooo much latency, because Lucene seems bad when it is not a Lucene error at all. To this end, HIbernate Search does not rely on JBossCache or any partitioning / caching schemes in memory, and instead relies on JMS and each local JVM to provide the latest indexing in the cluster with simultaneous low latency. Then the beauty of Hibernate Search is that standard Hibernate queries and much more can be launched through Hibernate with these natural language indexes on each machine.

At Terracotta, it turned out that we had a similar idea with Emmanuel and the SearchableMap product was built on top of Compass. Each machine gets its own Compass storage, and the storage is configured to distribute locally to disk. Terracotta is used to create multi-master recording capabilities where any JVM can add to the index, and delta is sent via Terracotta for replay / reuse locally on each disc. It works the same as Hibernate Search, but with DSO as the network protocol instead of JMS and without using Hibernate interfaces, but instead with Compass interfaces.

I think we will support Hibernate Search w / help from JBoss (they will need to separate JMS impl as pluggable) by the end of the year.

Now right to your questions:

1.Updates / sec in Hibernate or SearchableMap should be high enough, because both send only delta. In the case of Hibernate, this is a feature of our JMS provider. In Terracotta, it scales by simply adding more Terracotta Servers to the array.

2. Query performance in both cases is very fast. Local memory performance in most cases. And if you need to go to disk from disk, it turns out that most OSs do an excellent job and can respond to requests faster than any network-based cluster work.

3. It will be, I think, when we make JBoss abandon his JMS assumptions, etc.

Greetings

- Ari

Answer 2

As people on the Hibernate forums continue to link to this post, I feel the need to indicate that although Ari comments on where it is right at the beginning of 2009, we are developing and improving a lot.

Hibernate Search provides a set of backend feeds out of the box, like the already mentioned JMS, and a more recent addition using JGroups, but we also easily hooked up alternative implementations or redefined some of them.

In addition to using a custom backend, now since version 4 replaces the entire strategy, and instead of changing the implementation of the backend, you can use the IndexManager, which follows a different design and does not use the backend at all; at this time we have only two IndexManager, but we are working on other alternatives; again, the idea is to provide good implementations for the most common

It has an Infinispan-based database for distributing the index very quickly across nodes, and it must be direct in order to contribute based on Terracotta or any other clustering technology. New solutions will appear.