Geek Answers Handbook

Difference between FSDirectory and MMap Directory?

Can someone explain to me what is the difference between FSDirectory and MMapDirectory? I want to stretch my cache. I read that this may be useful, but I cannot find how it will be useful to warm up the cache. Explain to me if you have any ideas. Even pointers are welcome.

Lucene documentation says virtual memory is used to speed up index search in MMAP.

How is acceleration achieved and what happens if my indexes are large so that they do not match my virtual memory>

+7
source share
2 answers

MMapDirectory is one of the specific subclasses of the abstract FSDirectory class. It uses memory mapped files to access information in the index.

Other parameters: SimpleFSDirectory and NIOFSDirectory , which use different methods. You should look at the documentation for FSDirectory for a brief explanation of all three. As you will see there, FSDirectory.open(File) tries to choose the best implementation for your environment.

In my own experience, I did not notice a significant performance difference between NIOFSDirectory and MMapFSDirectory , but you should do some performance testing using your own data and hardware settings.

In the case of using MMapFSDirectory , virtual memory and index size can only be a problem on a 32-bit machine ( if your indexes are more than 2 ^ 48 bits = 32 TB ).

+7
source

If your indexes do not fit into virtual memory, you probably would be better off using FSDirectory. The problem is that using MMapDirectory when it does not fit in virtual memory is equivalent to using FSDirectory and using the OS caching algorithm (the OS caching algorithm is more likely to be better than what you can use for code). (Equivalent, because in both cases only parts of the index will be in physical memory at a time.)

But as "martin" said above, you need to do your own performance testing.

+1
source

More articles:


All Articles