Running Rails 3 with PostgreSQL,
I have a migration, updating millions of small records.
Record.find_each do |r| r.doing_incredibly_complex_stuff r.save! puts "#{r.id} updated" end
Since I believe that ActiveRecord wraps such updates in a transaction, the “commit” time is very long, and the received memory is HUGE, and each record was printed on the screen in the above example.
So, can I run this find_each outside of a transaction — although it is completely safe — to save a lot of time and memory on “commit”?
View ActiveRecord :: Base.without_transaction do ...; the end, I think :-)
OR: Am I mistaken, the migrations are not wrapped in transactions, and the time I see is just statements about updating SQL?
EDIT: Trere doesn't seem to be a link to transactions, here is the stack trace I got when I abort the migration, when everything was printed on the screen and the RAM decreased from 500 MB to ~ 30 MB:
IRB::Abort: abort then interrupt!! from /Users/clement/.rvm/gems/ ruby-1.9.2-p136@gemset /gems/activesupport-3.0.4/lib/active_support/whiny_nil.rb:46:in `call' from /Users/clement/.rvm/gems/ ruby-1.9.2-p136@gemset /gems/activesupport-3.0.4/lib/active_support/whiny_nil.rb:46:in `method_missing' from /Users/clement/.rvm/gems/ ruby-1.9.2-p136@gemset /gems/activerecord-3.0.4/lib/active_record/connection_adapters/postgresql_adapter.rb:978:in `flatten' from /Users/clement/.rvm/gems/ ruby-1.9.2-p136@gemset /gems/activerecord-3.0.4/lib/active_record/connection_adapters/postgresql_adapter.rb:978:in `block in select' from /Users/clement/.rvm/gems/ ruby-1.9.2-p136@gemset /gems/activerecord-3.0.4/lib/active_record/connection_adapters/postgresql_adapter.rb:977:in `map' from /Users/clement/.rvm/gems/ ruby-1.9.2-p136@gemset /gems/activerecord-3.0.4/lib/active_record/connection_adapters/postgresql_adapter.rb:977:in `select' from /Users/clement/.rvm/gems/ ruby-1.9.2-p136@gemset /gems/activerecord-3.0.4/lib/active_record/connection_adapters/abstract/database_statements.rb:7:in `select_all' from /Users/clement/.rvm/gems/ ruby-1.9.2-p136@gemset /gems/activerecord-3.0.4/lib/active_record/connection_adapters/abstract/query_cache.rb:56:in `select_all' from /Users/clement/.rvm/gems/ ruby-1.9.2-p136@gemset /gems/activerecord-3.0.4/lib/active_record/base.rb:467:in `find_by_sql' from /Users/clement/.rvm/gems/ ruby-1.9.2-p136@gemset /gems/activerecord-3.0.4/lib/active_record/relation.rb:64:in `to_a' from /Users/clement/.rvm/gems/ ruby-1.9.2-p136@gemset /gems/activerecord-3.0.4/lib/active_record/relation.rb:356:in `inspect' from /Users/clement/.rvm/gems/ ruby-1.9.2-p136@gemset /gems/railties-3.0.4/lib/rails/commands/console.rb:44:in `start' from /Users/clement/.rvm/gems/ ruby-1.9.2-p136@gemset /gems/railties-3.0.4/lib/rails/commands/console.rb:8:in `start' from /Users/clement/.rvm/gems/ ruby-1.9.2-p136@gemset /gems/railties-3.0.4/lib/rails/commands.rb:23:in `<top (required)>' from script/rails:6:in `require' from script/rails:6:in `<main>'
EDIT (2): Wow. It turned out that it was a very long time because find_each returns all the elements after iterating it.
I tried:
Record.tap do |record_class| record_class.find_each do |r| r.doing_incredibly_complex_stuff r.save! puts "#{r.id} updated" end end => Record(id: integer, ...)
So, he instantly gave away the console as expected. :)
But then I still see strange behavior: RAM does not free. Instead, as soon as I got out of the condition, the RAM is still sinking ...
Maybe my solution with a tap does not satisfy? Still a massive choice? How to avoid mass selection after find_each?
Thanks!