How can I tune RSpec to test performance sideways,

Question

We use RSpec in the rails project for unit testing. I would like to set up some performance tests in RSpec, but do it in such a way as not to disturb the "usual" functions and devices.

Ideally, I could mark my performance specifications in a certain way so that they don't start by default. Then, when I point out, in order to explicitly run these specifications, it will load a different set of devices (it makes sense to perform performance testing with a much larger and more “production” data set).

Is it possible? It seems like it should be.

Has anyone set up something like this? How did you do that?

Answer 1

I managed to get what I was looking for through the following:

# Exclude :performance tagged specs by default config.filter_run_excluding :performance => true # When we're running a performance test load the test fixures: config.before(:all, :performance => true) do # load performance fixtures require 'active_record/fixtures' ActiveRecord::Fixtures.reset_cache ActiveRecord::Fixtures.create_fixtures('spec/perf_fixtures', File.basename("products.yml", '.*')) ActiveRecord::Fixtures.create_fixtures('spec/perf_fixtures', File.basename("ingredients.yml", '.*')) end # define an rspec helper for takes_less_than require 'benchmark' RSpec::Matchers.define :take_less_than do |n| chain :seconds do; end match do |block| @elapsed = Benchmark.realtime do block.call end @elapsed <= n end end # example of a performance test describe Api::ProductsController, "API Products controller", :performance do it "should fetch all the products reasonably quickly" do expect do get :index, :format => :json end.to take_less_than(60).seconds end end 

But I tend to agree with Marnen that this is not the best idea for performance testing.

Answer 2

If you want to do performance testing, why not run New Relic or something with a production snapshot? I think you do not need different specifications.

Answer 3

I created an Ruby gem rspec test to write performance tests in RSpec. There are many expectations for testing speed, resource utilization and scalability.

For example, to check how fast your code works:

 expect { ... }.to perform_under(60).ms 

Or compare with another implementation:

 expect { ... }.to perform_faster_than { ... }.at_least(5).times 

Or check the computational complexity:

 expect { ... }.to perform_logarithmic.in_range(8, 100_000) 

Or see how many objects are selected:

 expect { _a = [Object.new] _b = {Object.new => 'foo'} }.to perform_allocation({Array => 1, Object => 2}).objects