Are there any silver bullets there to search for average text data (hundreds of gigabytes)? It really doesn't matter if it is commercial or open source. I have to add that I need it to be in C ++ or C.
Apache Lucene is a widely used indexing mechanism, not sure if it qualifies as a silver bullet or not. I saw this in conjunction with Carrot2 for clustering and index search.
xapian.
Solr may also be an option. It is also a Lucene based engine.
http://lucene.apache.org/solr/