Geek Answers Handbook

Invalid characters for searching text in lucene

On my IndexController I have

    public function buildAction()
    {

    $index = Zend_Search_Lucene::create(APPLICATION_PATH . '/indexes');     

    foreach ($this->pages as $p) {
        $doc = new Zend_Search_Lucene_Document();

        $doc->addField(Zend_Search_Lucene_Field::unIndexed('page_id', $p['page_id']));

        $doc->addField(Zend_Search_Lucene_Field::text('page_name', $p['page_name']));

        $doc->addField(Zend_Search_Lucene_Field::text('page_headline', $p['page_headline']));

        $doc->addField(Zend_Search_Lucene_Field::text('page_content', $p['page_content']));


        $index->addDocument($doc);
    }
    $index->optimize();
    $this->view->indexSize = $index->numDocs();
    }

and i get an error

[Tue Jan 18 16:23:32 2011] [error] [client 127.0.0.1] PHP Notice:  iconv(): Detected an illegal character in input string in /usr/share/php/libzend-framework-php/Zend/Search/Lucene/Analysis/Analyzer/Common/Text.php on line 58
[Tue Jan 18 16:23:32 2011] [error] [client 127.0.0.1] PHP Notice:  iconv(): Detected an illegal character in input string in /usr/share/php/libzend-framework-php/Zend/Search/Lucene/Field.php on line 222
[Tue Jan 18 16:23:32 2011] [error] [client 127.0.0.1] PHP Notice:  iconv(): Detected an illegal character in input string in /usr/share/php/libzend-framework-php/Zend/Search/Lucene/Analysis/Analyzer/Common/Text.php on line 58
[Tue Jan 18 16:23:32 2011] [error] [client 127.0.0.1] PHP Notice:  iconv(): Detected an illegal character in input string in /usr/share/php/libzend-framework-php/Zend/Search/Lucene/Field.php on line 222
[Tue Jan 18 16:23:32 2011] [error] [client 127.0.0.1] PHP Notice:  iconv(): Detected an illegal character in input string in /usr/share/php/libzend-framework-php/Zend/Search/Lucene/Analysis/Analyzer/Common/Text.php on line 58
[Tue Jan 18 16:23:32 2011] [error] [client 127.0.0.1] PHP Notice:  iconv(): Detected an illegal character in input string in /usr/share/php/libzend-framework-php/Zend/Search/Lucene/Field.php on line 222

and variable

$this->pages

contains an array of text copied from wikipedia, and I get an error for the characters - (not -) and ö, for which I get an error (I believe). I got a similar question in the Lucen foreign chars problem which does not explain what to do. Please, I would be grateful if I knew where and what to do, as well as a little explanation.

UPDATE :: Iconv

 iconv support          enabled
 iconv implementation   glibc
 iconv library version  2.12.1
+5
source share
2 answers

Try adding this to your loading tray:

Zend_Search_Lucene_Search_QueryParser::setDefaultEncoding('utf-8');
Zend_Search_Lucene_Analysis_Analyzer::setDefault(
    new Zend_Search_Lucene_Analysis_Analyzer_Common_Utf8_CaseInsensitive ()
);
+10
source

bootsrap,

$doc->addField(Zend_Search_Lucene_Field::text('page_name', $p['page_name'], 'UTF-8'));
+4

More articles:


All Articles