Geek Answers Handbook

SOLR slave makes a full copy because it was unable to remove the unused dir index

When the slave replicates the index, it creates an index directory with a name format like index.timestamp The next time it replicates, it tries to clean and create a new folder with a new timestamp, but in my case it does not happen, and every time I I see this warning, and Slave launches fullCopy

Unable to cleanup unused lucene index files so we must do a full copy instead

Can anyone why a slave cannot clear unused index files.

thanks

+7
lucene solr solrj
source share
1 answer

It is very difficult to answer without knowing your version of solr.

From my experience, solr copies index files that are different (newer than a timestamp). However, if your indexes are combined, then it will run a full copy.

Here are some related tickets

https://issues.apache.org/jira/browse/SOLR-6640

** update 11/27/2017 **

this is an important part in the 5x branches,

https://github.com/apache/lucene-solr/blob/branch_5x/solr/core/src/java/org/apache/solr/handler/IndexFetcher.java#L398-L418

 try { IndexWriter indexWriter = writer.get(); int c = 0; indexWriter.deleteUnusedFiles(); while (hasUnusedFiles(indexDir, commit)) { indexWriter.deleteUnusedFiles(); LOG.info("Sleeping for 1000ms to wait for unused lucene index files to be delete-able"); Thread.sleep(1000); c++; if (c >= 30) { LOG.warn("IndexFetcher unable to cleanup unused lucene index files so we must do a full copy instead"); isFullCopyNeeded = true; break; } } if (c > 0) { LOG.info("IndexFetcher slept for " + (c * 1000) + "ms for unused lucene index files to be delete-able"); } } finally { writer.decref(); }

So, you have over 30 unused index files, and this will trigger a warning and a full copy. I would try to optimize or combine the indexes from the wizard and see if fullCopy is replicated.

+3
source share

More articles:


All Articles