Designing a stress testing framework

Question

I am doing a kind of integration / stress test on a very large product (I think the size of the operating system), and recently my team and I discussed ways to better organize test loads. So far, we have been pleased that all of our (custom) workload applications were run in a series of batch type jobs, each of which is a single stress test. Now that we are at a point where the average test run includes more than 100 workloads running on 13 systems, we think it's time to create something more advanced.

I saw a lot there about unit unit tests, but very little for higher level voltage tests. Does anyone know of a general (or unusual) way to solve the problem of managing a large number of workloads?

At the moment, we would like to keep a database of each individual workload and provide an interface for mixing and match them with test packages depending on what stress we need on a given day, but we do not have examples of a better way to do more advanced things, such like the stress ranking that each individual workload is placed in the system.

What are my fellow stress testers on big products? For us, a few handwritten scripts just won't cut anymore.

Answer 1

My own experience was initially on the IIS platform with WCAT, and then with JMeter and Selenium.

WCAT and JMeter allow you to go through the website as a route, fill out forms, etc. and write the process as a script. Then the script can be played separately or simulate multiple clients and multiple threads. Randomize playback to simulate lumpy and unpredictable use, etc.

Scripts can be edited and can be written manually as soon as you know what you're going on. WCAT allows you to play log files, and also allows you to simulate use in the real world.

Both of the above are installed on a PC or server.

Selenium - FireFox, , .

, , . . Expcet , .

Answer 2

. . Visual Studio Tester, , . -, SQL-, ASP.NET .. , . , . . , . , !:) , VS Load Agents ( ). , . MS , . , , . , 3000 - 2 . , .

, Windows. , $6K , .