Lamps and selenium and rails (oh my?)

Question

What data do you use in Selenium tests for Rails applications? Are you loading from fixtures? Use existing dev db? Use separate (non-fixed) db?

I consider my options here. I have a Rails application with a large set of Selenium tests that runs on a modified version of Selenium Grid. Part of the process, right now, is loading a large set of fixtures, once, before launching the test suite. This is a lot of data. Most of them provide information exported from our production database. When I installed it initially, I exported data to yaml from Oracle.

Now, in some report tables, a schema has changed, so of course I need to restore the binding data. There are so many that you do not need to edit files manually. But it seems to be inefficient to recover with every change in the circuit - not to mention that this is another step that needs to be remembered. Is there a better way?

EDIT: I originally intended to load fixtures before each test and unload them after each test, for example, regular Rails tests. But it takes about 15 minutes to collect data due to reporting data. There are 200+ tests, and the kit runs every 12 hours. I cannae bend the space-time commander!

EDIT 2: I also agree that having this large set of fixtures is a bad smell. I'm not sure how to do this, because reports combine a lot of data, and most selenium tests are that they check reports.

Even if it's a small data set, though ... this is another set that will be coordinated with schema changes. (We have a separate, smaller set for integrated unit tests, functional and [Rails].)

Which brings me back to my original question - are there any other options besides doing it manually, or remembering to regenerate them every time?

Answer 1

If you can, then the best thing you can do is to have a system in which each Selenium test gets its own data state (i.e. DB tables, dumped and recreated, reloaded data, and flush caches). This is easier said than done, and, as a rule, this is possible only if the project is planned from the very beginning.

The next best thing is to have a consistent database state for each test / run set. This is not so pleasant, since now there is a high probability that some tests will depend on the success of previously performed tests, which makes it difficult to identify true errors and false negatives.

In the worst case, IMO should use a static database, in which each test run mutates the date. This almost always leads to problems and is usually the "smell of a project." The key to making this the “right way” (again, IMO) must be vigilant about any state / circuit change and capture it as part of an automated testing / build process.

Rails does a great job with Migrations, so use them! Not knowing your situation, I generally doubt the need to run Selenium tests against binding a complete database. Most databases can (or should) be distilled to less than 1 MB for automatic testing, which makes automatic circuit migrations and reset data much more efficient.

The only time I saw a “real” reason for massive databases for Selenium tests is that the database itself contains large pieces of “logical data” in which data affects the application flow (I think: a data-driven interface).

Answer 2

I think you are asking here two questions that are intertwined, so if I break this:

• You want to quickly get test data into and out of your database, and fixtures do not do this for you.
• You were burned by changing the scheme, and you want to make sure that everything you do does not require eight iterations of the thematic "messing with test data ... more" :)

You have a couple of alternatives that I skipped below. Since you mentioned Oracle, I use Oracle technologies here, but the same is true for other database platforms (like Postgresql):

• Rake tesks that invoke PL / SQL scripts to generate data is a disgusting terrible evil idea, don't do this unless there is another option. I did this in one project that needs to be downloaded in billions of lines for some tests of infrastructure architecture. I still jerk about it.
• Get your database in dump format. For fast binary dumps, check out the exp / imp and data pump utilities. This will allow you to quickly configure and disable your database. Of course, on the rails project that I was working on, we used rake tasks for the exp / imp database, which had about 300 thousand records in less than a minute. Also check out SQLLoader, which is a logical dump utility, as its logical one is slower and requires that you have control scripts to help SQLLoader understand dumps. However, the advantage of a logical dump is that you can run conversion scripts on them to massage the data into the latter format. Unfortunately, although such tools, all of these tools are very sensitive to changes in the circuit.
• Use a plugin like Machinist or Factory Girl to make data generation more enjoyable. You still incur a fine for using ActiveRecord to set up your database, but these fake object generators will help you stay close to you, and there is much less hassle for them than fixtures.
• Combine approaches 2 and 3. What happens here is that you do some test data with Machinst. You export this test data to a dump, and then reload the dump during each test run. When a schema change updates Machinist configuration and relay.

Hope this helps.

Answer 3

I am currently working on a project with a huge set of Selenium tests - in fact, it was written for Selenium Grid - and our tests use a small amount of reference data (although we do not use Rails YAML lights) and object factories for one-time data needed for specific tests .

Alternatively, in many of the ThoughtWorks Rails projects I've been to, we have written validation scripts that include a number of prefixes - for example, running tests before resolving a commit. One thing you can try is to write (or configure) a similar checkin script that will check for circuit changes and reload reference data as needed.

See Paul Gross rake commit tasks on Github.