Geek Answers Handbook

Fastest full text search today?

spoiler:
This is just another Lucene vs Sphinx vs what,
I saw that all the other threads were almost two years old, so I decided to start again.

Here is the requirement:

data size: no more than 10 GB.
ranks: nearly billions
indexing should be fast
the search should be below 0 ms [ok, joke ... laugh ... but keep it as low as possible]

In the modern world, what / what / how do I do this?

edit: I did some time on lucene, and it took 5 minutes to index the 1.8 GB data.
the search is pretty fast if I don't do a * . a * takes 400 ~ 500 ms.
My biggest concern is indexing, which takes loooonnngnggg time and loads of resources!

+6
sqlite search lucene sphinx fts3
source share
3 answers

I have no experience except with Lucene - this is pretty much the standard indexing solution, so don’t think you might be wrong.

10 GB is not much data. You can quickly index it - or save it to an SSD for extra speed. And of course, save your entire index in RAM (which Lucene supports) for super-fast searches.

+2
source share

Please check the Lucene wiki tips for improving Lucene indexing speed . This is pretty eloquent. All in all, Lucene is pretty fast (used for real-time searches.) Tips will help you figure out if you are missing something "obvious".

0
source share

My biggest concern is indexing, which taking time loooonnnnggg, and lots of resources !!

Take a look at Lusql , we used it once, FWIW 100 GBdata from mysql on a decent machine took no more than an hour to index the file system (NTFS)

Now, if u adds an SSD or any superfast disk technique, you can significantly reduce it.

0
source share

More articles:


All Articles