Geek Answers Handbook

Solr Distributed Component Testing

I developed a custom Solr search component for which I need to write unit tests. As I saw in the code of other Solr components, writing unit tests to Solr is done by extending the SolrTestCaseJ4 class . Unfortunately, SolrTestCaseJ4 does not apply to testing in a distributed setup, and my custom component only works in that setup. In fact, my component deliberately returns empty answers, if not in a distributed setting.

I am trying to come up with a way to use the BaseDistributedSearchTestCase class to test my component. The problem with BaseDistributedSearchTestCase is that how this works will not solve my problem. When using BaseDistributedSearchTestCase, you define one testing method in which you index all documents and perform some queries. Test execution executes queries both in distributed configuration and in one basic configuration. He then compares the responses of each parameter to verify their equality. I cannot explicitly state anything in this thread.

How to write unit tests for a distributed Solr component?

+4
source share
1 answer

In Solr 4.7, the MiniSolrCloudCluster class has been added, which actually "expands" locally (and if you want to use only ram or in the temporary directory) a full test cluster with zookeeper, shards, etc. for your tests.

Here you can find jira: https://issues.apache.org/jira/browse/SOLR-5865

I successfully used it to run tests on distributed solr components, taking an example from Solr tests, as follows:

    private static MiniSolrCloudCluster miniCluster;
    private static CloudSolrServer cloudSolrServer;

    @BeforeClass
    public static void setup () throws Exception {
        miniCluster = new MiniSolrCloudCluster (2, null, new File ("src / main / solr / solr.xml"), null, null);
        uploadConfigToZk("src/main/solr/content/conf/", "content");

        // override settings in the solrconfig include
        System.setProperty("solr.tests.maxBufferedDocs", "100000");
        System.setProperty("solr.tests.maxIndexingThreads", "-1");
        System.setProperty("solr.tests.ramBufferSizeMB", "100");
        // use non-test classes so RandomizedRunner isn't necessary
        System.setProperty("solr.tests.mergeScheduler", "org.apache.lucene.index.ConcurrentMergeScheduler");
        System.setProperty("solr.directoryFactory", "solr.RAMDirectoryFactory");        

        cloudSolrServer = new CloudSolrServer(miniCluster.getZkServer().getZkAddress(), false);
        cloudSolrServer.setRequestWriter(new RequestWriter());
        cloudSolrServer.setParser(new XMLResponseParser());
        cloudSolrServer.setDefaultCollection("content");
        cloudSolrServer.setParallelUpdates(false);
        cloudSolrServer.connect();

        createCollection(cloudSolrServer, "content", 2, 1, "content");

    }

    protected static void uploadConfigToZk(String configDir, String configName) throws Exception {
        SolrZkClient zkClient = null;
        try {
            zkClient = new SolrZkClient(miniCluster.getZkServer().getZkAddress(), 10000, 45000, null);
            uploadConfigFileToZk(zkClient, configName, "solrconfig.xml", new File(configDir, "solrconfig.xml"));
            uploadConfigFileToZk(zkClient, configName, "schema.xml", new File(configDir, "schema.xml"));
            uploadConfigFileToZk(zkClient, configName, "stopwords_en.txt", new File(configDir, "stopwords_en.txt"));
            uploadConfigFileToZk(zkClient, configName, "stopwords_it.txt", new File(configDir, "stopwords_it.txt"));

            System.out.println(zkClient.getChildren(ZkController.CONFIGS_ZKNODE + "/" + configName, null, true));
        } finally {
            if (zkClient != null)
                zkClient.close();
        }
    }

    protected static void uploadConfigFileToZk(SolrZkClient zkClient, String configName, String nameInZk, File file) throws Exception {
        zkClient.makePath(ZkController.CONFIGS_ZKNODE + "/" + configName + "/" + nameInZk, file, false, true);
    }

    @AfterClass
    public static void shutDown() throws Exception {
        miniCluster.shutdown();
    }

    protected static NamedList createCollection(CloudSolrServer server, String name, int numShards, int replicationFactor, String configName) throws Exception {
        ModifiableSolrParams modParams = new ModifiableSolrParams();
        modParams.set(CoreAdminParams.ACTION, CollectionAction.CREATE.name());
        modParams.set("name", name);
        modParams.set("numShards", numShards);
        modParams.set("replicationFactor", replicationFactor);
        modParams.set("collection.configName", configName);
        QueryRequest request = new QueryRequest(modParams);
        request.setPath("/admin/collections");
        return server.request(request);
    }

    @Test
    public void test() throws Exception {
      // Do you stuff here using cloudSolrServer as a normal solrServer
    }
+4

More articles:


All Articles