Geek Answers Handbook

Was Sunspot-solr eating too much memory? What is a convenient way to reduce memory usage?

So, it seems Sunspot-Solr eats a lot of memory. And probably due to my negligence as a programmer who uses it correctly.

First of all, I do not allow the sunspot to be reviewed on its own.

searchable :auto_index => false do

Only this, I believe, prevents him from reindexing. Instead, I run the reindex process on the cron tab, which starts once a day in the early morning.

The reason I started doing this was because, as the application scaled, the data seemed to be sitting so tight on the machine that it would take quite a while to load the main page.

What am I doing wrong to cause the error below? And what can I do better?

Error in the question:

 ActionView::TemplateError (Solr Response: Lock_obtain_timed_out_NativeFSLocktmpindexlucenede61b2c77401967646cf8916982a09a0writelock__orgapachelucenestoreLockObtainFailedException_Lock_obtain_timed_out_NativeFSLocktmpindexlucenede61b2c77401967646cf8916982a09a0writelock__at_orgapachelucenestoreLockobtainLockjava85__at_orgapacheluceneindexIndexWriterinitIndexWriterjava1545__at_orgapacheluceneindexIndexWriterinitIndexWriterjava1402__at_orgapachesolrupdateSolrIndexWriterinitSolrIndexWriterjava190__at_orgapachesolrupdateUpdateHandlercreateMainIndexWriterUpdateHandlerjava98__at_orgapachesolrupdateDirectUpdateHandler2openWriterDirectUpdateHandler2java173__at_orgapachesolrupdateDirectUpdateHandler2addDocDirectUpdateHandler2java220__at_orgapachesolrupdateprocessorRunUpdateProcessorprocessAddRunUpdateProcessorFactoryjava61__at_orgapachesolrhandlerXMLLoaderprocessUpdateXMLLoaderjava139__at_orgapachesolrhandlerXMLLoaderloadXMLLoaderjava69__at_orgapachesolrhandlerContentStreamHandlerBasehandleRequestBodyContentStreamHandlerBasejava54__at_orgapachesolrhandlerRequestHandlerBasehandleRequestRequestHandlerBasejava131__at_orgapachesolrcoreSolrCoreexecuteSolrCorejava1316__at_orgapachesolrservletSolrDispatchFilterexecuteSolrDispatchFilterjava338__at_orgapachesolrservletSolrDispatchFilterdoFilterSolrDispatchFilterjava241__at_orgmortbayjettyservletServletHandler$CachedChaindoFilterServletHandlerjava1089__at_orgmortbayjettyservletServletHandlerhandleServletHandlerjava365__at_orgmortbayjettysecuritySecurityHandlerhandleSecurityHandlerjava216__at_orgmortbayjettyservletSessionHandlerhandleSessionHandlerjava181__at_orgmortbayjettyhandlerContextHandlerhandleContextHandlerjava712__at_orgmortbayjettywebappWebAppContexthandleWebAppContextjava405__at_orgmortbayjettyhandlerContextHandlerCollectionhandleContextHandlerCollectionjava211__at_orgmortbayjettyhandlerHand) on line #24 of app/views/main/_main_nav.html.haml: 21: %br 22: community calendars 23: 24: - if (current_user.blank? || current_user.card_signup.blank?) 25: %li 26: - if current_user.blank? 27: = link_to 'Get Your HQcard', signup_path, :title => "Signup for your free HQcard and redeem local deals and promotions." rsolr (0.12.1) [v] lib/rsolr/connection/requestable.rb:39:in `request' rsolr (0.12.1) [v] lib/rsolr/client.rb:34:in `request_without_rails_logging' /usr/lib/ruby/gems/1.8/gems/sunspot_rails-1.2.1/lib/sunspot/rails/solr_logging.rb:25:in `request' /usr/lib/ruby/gems/1.8/gems/sunspot_rails-1.2.1/lib/sunspot/rails/solr_logging.rb:24:in `request' rsolr (0.12.1) [v] lib/rsolr/client.rb:22:in `update' rsolr (0.12.1) [v] lib/rsolr/client.rb:46:in `add' sunspot (1.2.1) lib/sunspot/indexer.rb:101:in `add_documents' sunspot (1.2.1) lib/sunspot/indexer.rb:26:in `add' sunspot (1.2.1) lib/sunspot/session.rb:91:in `index' sunspot (1.2.1) lib/sunspot/session_proxy/abstract_session_proxy.rb:11:in `index' sunspot (1.2.1) lib/sunspot.rb:175:in `index' /usr/lib/ruby/gems/1.8/gems/sunspot_rails-1.2.1/lib/sunspot/rails/searchable.rb:349:in `solr_index' /usr/lib/ruby/gems/1.8/gems/sunspot_rails-1.2.1/lib/sunspot/rails/searchable.rb:405:in `maybe_auto_index' vendor/gems/binarylogic-authlogic-2.1.1/lib/authlogic/acts_as_authentic/session_maintenance.rb:73:in `save_without_session_maintenance' vendor/gems/binarylogic-authlogic-2.1.1/lib/authlogic/session/callbacks.rb:83:in `save_record' vendor/gems/binarylogic-authlogic-2.1.1/lib/authlogic/session/priority_record.rb:30:in `save_record' vendor/gems/binarylogic-authlogic-2.1.1/lib/authlogic/session/persistence.rb:60:in `persisting?' vendor/gems/binarylogic-authlogic-2.1.1/lib/authlogic/session/persistence.rb:39:in `find' app/controllers/application_controller.rb:23:in `current_user_session' app/controllers/application_controller.rb:28:in `current_user' (eval):2:in `send' (eval):2:in `current_user' app/views/main/_main_nav.html.haml:24:in `_run_haml_app47views47main47_main_nav46html46haml_locals_main_nav_object' haml (2.2.2) [v] lib/haml/helpers/action_view_mods.rb:11:in `render' haml (2.2.2) [v] lib/haml/helpers.rb:96:in `non_haml' haml (2.2.2) [v] lib/haml/helpers/action_view_mods.rb:11:in `render' app/views/main/index.html.haml:2:in `_run_haml_app47views47main47index46html46haml' haml (2.2.2) [v] lib/haml/helpers/action_view_mods.rb:13:in `render' haml (2.2.2) [v] lib/haml/helpers/action_view_mods.rb:13:in `render' haml (2.2.2) [v] rails/./lib/sass/plugin/rails.rb:19:in `process' lib/flash_session_cookie_middleware.rb:14:in `call' vendor/gems/hoptoad_notifier-2.2.2/lib/hoptoad_notifier/rack.rb:27:in `call' : ActionView::TemplateError (Solr Response: Lock_obtain_timed_out_NativeFSLocktmpindexlucenede61b2c77401967646cf8916982a09a0writelock__orgapachelucenestoreLockObtainFailedException_Lock_obtain_timed_out_NativeFSLocktmpindexlucenede61b2c77401967646cf8916982a09a0writelock__at_orgapachelucenestoreLockobtainLockjava85__at_orgapacheluceneindexIndexWriterinitIndexWriterjava1545__at_orgapacheluceneindexIndexWriterinitIndexWriterjava1402__at_orgapachesolrupdateSolrIndexWriterinitSolrIndexWriterjava190__at_orgapachesolrupdateUpdateHandlercreateMainIndexWriterUpdateHandlerjava98__at_orgapachesolrupdateDirectUpdateHandler2openWriterDirectUpdateHandler2java173__at_orgapachesolrupdateDirectUpdateHandler2addDocDirectUpdateHandler2java220__at_orgapachesolrupdateprocessorRunUpdateProcessorprocessAddRunUpdateProcessorFactoryjava61__at_orgapachesolrhandlerXMLLoaderprocessUpdateXMLLoaderjava139__at_orgapachesolrhandlerXMLLoaderloadXMLLoaderjava69__at_orgapachesolrhandlerContentStreamHandlerBasehandleRequestBodyContentStreamHandlerBasejava54__at_orgapachesolrhandlerRequestHandlerBasehandleRequestRequestHandlerBasejava131__at_orgapachesolrcoreSolrCoreexecuteSolrCorejava1316__at_orgapachesolrservletSolrDispatchFilterexecuteSolrDispatchFilterjava338__at_orgapachesolrservletSolrDispatchFilterdoFilterSolrDispatchFilterjava241__at_orgmortbayjettyservletServletHandler$CachedChaindoFilterServletHandlerjava1089__at_orgmortbayjettyservletServletHandlerhandleServletHandlerjava365__at_orgmortbayjettysecuritySecurityHandlerhandleSecurityHandlerjava216__at_orgmortbayjettyservletSessionHandlerhandleSessionHandlerjava181__at_orgmortbayjettyhandlerContextHandlerhandleContextHandlerjava712__at_orgmortbayjettywebappWebAppContexthandleWebAppContextjava405__at_orgmortbayjettyhandlerContextHandlerCollectionhandleContextHandlerCollectionjava211__at_orgmortbayjettyhandlerHand) on line #24 of app/views/main/_main_nav.html.haml: 21: %br 22: community calendars 23: 24: - if (current_user.blank? || current_user.card_signup.blank?) 25: %li 26: - if current_user.blank? 27: = link_to 'Get Your HQcard', signup_path, :title => "Signup for your free HQcard and redeem local deals and promotions." rsolr (0.12.1) [v] lib/rsolr/connection/requestable.rb:39:in `request' rsolr (0.12.1) [v] lib/rsolr/client.rb:34:in `request_without_rails_logging' /usr/lib/ruby/gems/1.8/gems/sunspot_rails-1.2.1/lib/sunspot/rails/solr_logging.rb:25:in `request' /usr/lib/ruby/gems/1.8/gems/sunspot_rails-1.2.1/lib/sunspot/rails/solr_logging.rb:24:in `request' rsolr (0.12.1) [v] lib/rsolr/client.rb:22:in `update' rsolr (0.12.1) [v] lib/rsolr/client.rb:46:in `add' sunspot (1.2.1) lib/sunspot/indexer.rb:101:in `add_documents' sunspot (1.2.1) lib/sunspot/indexer.rb:26:in `add' sunspot (1.2.1) lib/sunspot/session.rb:91:in `index' sunspot (1.2.1) lib/sunspot/session_proxy/abstract_session_proxy.rb:11:in `index' sunspot (1.2.1) lib/sunspot.rb:175:in `index' /usr/lib/ruby/gems/1.8/gems/sunspot_rails-1.2.1/lib/sunspot/rails/searchable.rb:349:in `solr_index' /usr/lib/ruby/gems/1.8/gems/sunspot_rails-1.2.1/lib/sunspot/rails/searchable.rb:405:in `maybe_auto_index' vendor/gems/binarylogic-authlogic-2.1.1/lib/authlogic/acts_as_authentic/session_maintenance.rb:73:in `save_without_session_maintenance' vendor/gems/binarylogic-authlogic-2.1.1/lib/authlogic/session/callbacks.rb:83:in `save_record' vendor/gems/binarylogic-authlogic-2.1.1/lib/authlogic/session/priority_record.rb:30:in `save_record' vendor/gems/binarylogic-authlogic-2.1.1/lib/authlogic/session/persistence.rb:60:in `persisting?' vendor/gems/binarylogic-authlogic-2.1.1/lib/authlogic/session/persistence.rb:39:in `find' app/controllers/application_controller.rb:23:in `current_user_session' app/controllers/application_controller.rb:28:in `current_user' (eval):2:in `send' (eval):2:in `current_user' app/views/main/_main_nav.html.haml:24:in `_run_haml_app47views47main47_main_nav46html46haml_locals_main_nav_object' haml (2.2.2) [v] lib/haml/helpers/action_view_mods.rb:11:in `render' haml (2.2.2) [v] lib/haml/helpers.rb:96:in `non_haml' haml (2.2.2) [v] lib/haml/helpers/action_view_mods.rb:11:in `render' app/views/main/index.html.haml:2:in `_run_haml_app47views47main47index46html46haml' haml (2.2.2) [v] lib/haml/helpers/action_view_mods.rb:13:in `render' haml (2.2.2) [v] lib/haml/helpers/action_view_mods.rb:13:in `render' haml (2.2.2) [v] rails/./lib/sass/plugin/rails.rb:19:in `process' lib/flash_session_cookie_middleware.rb:14:in `call' vendor/gems/hoptoad_notifier-2.2.2/lib/hoptoad_notifier/rack.rb:27:in `call'
+4
source share
3 answers

You can specify the minimum and maximum memory allocation according to the current environment in the config / sunspot.yml file, for example.

 development: solr: min_memory: 512M max_memory: 1G solr_jar: /path/to/start.jar log_level: DEBUG

Documentation: http://outoftime.github.com/sunspot/rails/docs/classes/Sunspot/Rails/Configuration.html

+12
source

I canceled the execution of the Lucene engine and forced it to use the 64-bit JVM, which allowed it to get full access to all the memory available on my machine ...

Why use a rake 32-bit JVM on 64-bit windows outside of me ...

Now I use sunspot: solr run script to execute manually:

 cd \RailsInstaller\Ruby1.9.3\lib\ruby\gems\1.9.1\gems\sunspot_solr-2.1.0\solr\

java -d64 -Djetty.port = 8982 -Dsolr.data.dir = C: / [your-project-dir] / solr / data / development -Dsolr.solr.home = solr -Djava.util.logging.config.file = C: / Users / [user- name] /AppData/Local/Temp/logging.properties20140117 -jar start.jar

0
source

max_memory and min_memory have been changed to memory at https://github.com/sunspot/sunspot/commit/74ea3881872398bd14cf6e96481826141ee82c93

So the current answers should be:

 development: solr: memory: 1G solr_jar: /path/to/start.jar log_level: DEBUG
0
source

More articles:


All Articles