Memory Spread When Creating Many New Objects

Question

When I run this and then watch the memory consumption of my ruby ​​process in OSX Activity Monitor, the memory grows by about 3 MB / s.

If I delete the transaction, it will reduce the memory consumption by about half, but still the amount of memory continues to grow. I have a problem with my production application where Heroku kills a process due to memory consumption.

Is there any way to do this below so as not to increase memory? If I comment out the .save line, then this is fine, but of course this is not a solution.

 ActiveRecord::Base.transaction do 10000000.times do |time| puts "---- #{time} ----" a = Activity.new(:name => "#{time} Activity") a.save!(:validate => false) a = nil end end 

I run this with delayed_job .

Answer 1

The string a = nil not needed, and you can delete it.

You create many objects every time you loop - two lines, two hashes and an Activity object, so I'm not surprised that you use memory heavily, especially when you loop 10 million times! There seems to be no more efficient way to write this code.

The only way to reduce memory usage is to manually run the garbage collector every x iterations. Most likely, Ruby GC is not aggressive enough. However, you do not want to call it at every iteration, as this will drastically slow down your code. Perhaps you can use every 100 iterations as a starting point and jump from there. You will need to profile and verify what is most effective.

The documentation for the GC is here .

Answer 2

I know this is an ancient problem, but I have to offer a different radical approach:

 ActiveRecord::Base.transaction do 10000000.times do |time| puts "---- #{time} ----" sql = <<SQL INSERT INTO activities ("name") VALUES ("#{time}") SQL Activity.connection.execute(sql) end end 

The fact is that if the insertion is so simple and you have already skipped any ActiveModel check, there is no reason to create an instance of the activerecord object in the first place. This will usually not hurt, but since it hurts you, I think that you will use a lot less memory in this way.