How to optimize the solr index

Question

How to optimize the solr index. I want to optimize my indexing solr in order to try to change its indexing in the solrconfig.xml file, but I want to check that they are optimized and for what purpose they are related to index optimization.

Answer 1

I believe this is the easiest way to optimize the Solr index. In my context, “optimization” means joining all segments of an index.

curl http://localhost:8983/solr/<core_name>/update -F stream.body=' <optimize />' 

Answer 2

Before starting work, check the size of the corresponding kernel.

Open terminal 1:

 watch -n 10 "du -sh /path to core/data/*" 

Open terminal 2 and do:

 curl http://hostname:8980/solr/<core>/update?optimize=true 

Instead of "kernel", update your own kernel name.

You can see that the size of the kernel will gradually increase by about two times compared with your indexed data and will suddenly decrease. It will take time, depends on your solr data.

For example, 50G indexed data increases by almost 90G and ends to optimize 25G data. And usually for this amount of data it will take 30-45 minutes.

Why doesn't my index directory get smaller (immediately) when deleting documents? force a merger? optimize?

Answer 3

You need to pass optimize=true to update the solr request to optimize solr.

http: // [HostName]: [port] / solr / update? optimize = true

Answer 4

There are various ways to optimize the index. You can run one of the basic solr scripts: http://wiki.apache.org/solr/SolrOperationsTools#optimize

You can also set optimize=true to (full) import or when adding new data. ... or just call commit using optimize=true

Perhaps this may be interesting for your needs: http://wiki.apache.org/solr/UpdateXmlMessages#A.22commit.22_and_.22optimize.22

Answer 5

To test how many changes you optimize indexing, simply write your own index and add randomly generated content. Add a large number of documents (500,000 or 1,000,000) and measure the time it takes.

In accordance with the above articles, I made a custom indexer for myself, and I set out to optimize the time required to index documents by 80%.