Spring versus Zeus performance issue: Spring doesn't run tests much faster after first run?

Question

So, I am trying to improve the test execution time and follow some of the excellent recommendations (especially Railscasts 412 ).

Zeus is working fine (except for the oddity that requires you to zeus test test to run the Minitest package.)

Here is the time from zeus:

Initial level (rake test):

 $ time rake test Run options: --seed 62848 # Running tests: ........ Finished tests in 0.117071s, 68.3346 tests/s, 68.3346 assertions/s. 8 tests, 8 assertions, 0 failures, 0 errors, 0 skips real 0m5.771s user 0m4.477s sys 0m0.872s 

And with Zeus:

 $ time zeus test test Run options: --seed 10325 # Running tests: ........ Finished tests in 0.126365s, 63.3087 tests/s, 63.3087 assertions/s. 8 tests, 8 assertions, 0 failures, 0 errors, 0 skips real 0m0.765s user 0m0.209s sys 0m0.030s 

Good. 5.7 s to 0.7 s

Now, with spring, I just don't see an improvement after the first run. I run Spring gem 1.1.0beta4, installing as described (adding to the Gemfile and then running bundle exec spring binstub --all ).

And here is what I have:

Initial level, 1st performance (rake test):

 time bin/rake test Run options: --seed 49915 # Running tests: ........ Finished tests in 0.108359s, 73.8287 tests/s, 73.8287 assertions/s. 8 tests, 8 assertions, 0 failures, 0 errors, 0 skips real 0m6.265s user 0m0.072s sys 0m0.020s 

Confirm that Spring is running:

 $ spring status Spring is running: 2990 spring server | gdc4 | started 43 secs ago 2991 spring app | gdc4 | started 43 secs ago | test mode 

And run the tests again ...

 $ time bin/rake test Run options: --seed 27416 # Running tests: ........ Finished tests in 0.132677s, 60.2968 tests/s, 60.2968 assertions/s. 8 tests, 8 assertions, 0 failures, 0 errors, 0 skips real 0m3.885s user 0m0.069s sys 0m0.014s 

So, a little acceleration, but still almost 4 seconds to complete. Meanwhile, by the way, the rail console MUST load a ton faster than w / Spring .... <1 second.

So my question is: why can Zeus get an acceleration of the order of magnitude according to the test results (5.7s → 0.7s), but Spring is not?

Running Rails 4.0.2, Ruby 2.0.0, btw.

Answer 1

You cannot run

rake test

you need to run

spring rake test

Example:

time rake -T

10.737s

time rake -T

10.811s # no improvement as you noted

spring time rake -T

18.468s

spring time rake -T

1.140s # huge improvement

Answer 2

I played a little with spring and did not compare times with zeus (which I prefer), but I assume that you still see the slow ruby ​​startup time of spring itself. Zeus has nothing of what is written in Go.

Answer 3

Here I see the same difference. Starting the rail server, for example: with zeus <1s with spring ~ 4s

Answer 4

Perhaps because zeus test does not run db:reset .

In one of my applications, Spring runs the tests in 5-6 seconds, whereas Zeus runs them in about 0.5 s, and this is definitely due to the fact that Spring reloads the database every time.