Geek Answers Handbook

Python script download download webpage

I want to do a test load for a webpage. I want to do this in python with multiple threads. The first POST request is to log in to the user (set cookies).

Then I need to know how many users are simultaneously executing the same POST request. Therefore, I think of spawning threads in which requests will be executed in a loop.

I have a couple of questions: 1. Is it possible to run 1000 - 1500 requests simultaneously with the processor? I mean, will he slow down the system so that it is not reliable? 2. What about bandwidth limitations? How good is the channel for this test to be reliable?

The server hosting the test site - Amazon EC2 script will be launched from another server (Amazon too).

Thanks!

+4
source share
2 answers

cPython does not take advantage of multiple cores when running multiple threads. This means that basically you will have only one core performing the test task.

There are special tools to accomplish what you want to do. Let me suggest two:

FunkLoad is a functional and downloadable web tester written in Python, the main uses of which are:

  • Functional testing of web projects and, therefore, regression testing.
  • Performance Testing: By downloading the web application and monitoring, your servers will help you identify bottlenecks by giving a detailed performance measurement report.
  • Download a testing tool to identify errors that do not overlap with surface testing, such as volume testing or durability testing.
  • Stress testing tool to suppress web application resources and test application recovery capabilities.
  • Writing web agents by writing any repetitive task on the web, for example, if the site is live.

Tsung is an open source distributed load testing tool for open source

Tsung's goal is to simulate users to test scalability and performance based on IP client / server applications. You can use it to conduct stress and stress tests of your servers. Many protocols have been implemented and tested, and this can be easily expanded. WebDAV, LDAP and MySQL support recently (experimental).

It can be distributed across several client machines and is able to simulate hundreds of thousands of virtual users at the same time (or even millions if you have enough equipment ...).

If you decide to write your own tool, you probably want to use the Python multiprocessing module , as it will allow you to use multiple cores. You should also take a look at Twisted , as this will allow you to easily handle multiple sockets with a limited number of threads. This would be much better than spawning a new thread for each socket.

You are working with Amazon EC2, so I would recommend using Tsung. You can rent a dozen multicore servers for several hours and run very heavy load tests with Tsung. It scales very well in this configuration.

Regarding bandwidth, this is usually not a problem, but it depends on the application. When performing a load test, you will have to carefully monitor all of your resources.

+5
source

too many variables. 1000 at the same time ... no. in the same second ... maybe. bandwidth could very well be a bottleneck. this is best allowed by experimentation.

+1
source

More articles:


All Articles