Multithreaded performance for calling a web service

Question

Here is my sample program for web server server and client side. I ran into a strnage performance problem, which even if I increase the number of threads for calling web services, performance does not improve. At the same time, CPU / memory / network consumption from the task manager performance panel is small. I wonder what the bottleneck is and how to improve it?

(my testing experience, double the number of threads will almost double the total response time)

Client side:

class Program { static Service1[] clients = null; static Thread[] threads = null; static void ThreadJob (object index) { // query 1000 times for (int i = 0; i < 100; i++) { clients[(int)index].HelloWorld(); } } static void Main(string[] args) { Console.WriteLine("Specify number of threads: "); int number = Int32.Parse(Console.ReadLine()); clients = new Service1[number]; threads = new Thread[number]; for (int i = 0; i < number; i++) { clients [i] = new Service1(); ParameterizedThreadStart starter = new ParameterizedThreadStart(ThreadJob); threads[i] = new Thread(starter); } DateTime begin = DateTime.Now; for (int i = 0; i < number; i++) { threads[i].Start(i); } for (int i = 0; i < number; i++) { threads[i].Join(); } Console.WriteLine("Total elapsed time (s): " + (DateTime.Now - begin).TotalSeconds); return; } } 

Server side:

  [WebMethod] public double HelloWorld() { return new Random().NextDouble(); } 

thanks in advance George

Answer 1

Although you are creating a multi-threaded client, keep in mind that .NET has a configurable bottleneck of two simultaneous calls to the same host. This is by design. Please note that this is on the client, not the server.

Try setting up the app.config file on the client:

 <system.net> <connectionManagement> <add address="*" maxconnection="20″ /> </connectionManagement></system.net> 

There is additional information about this:

Answer 2

My experience, as a rule, is that the problem is blocking: I had a massive parallel server that spent more time switching context than doing work.

So, check your memory and process counters in perfmon, if you look at the context switches and your high ones (more than 4000 per second), then you have problems.

You can also check the statistics of your memory on the server - if it spends all its time swapping or just creates and frees lines, it will also stall.

Finally, check the disk I / O for the same reason as above.

The permission is to remove the locks or hold them for at least time. Our problem was solved by eliminating the COM BSTR dependency and their global blocking, you will find that C # has many similar synchronization bottlenecks (designed for the safe functioning of your code). I saw a performance drop when I moved a simple C # application from single-core to multi-core.

If you cannot remove the locks, the best option is to create not so many threads. Use the thread pool to let the processor complete one task before starting another.

Answer 3

I do not think that you are actually facing a bottleneck.

Have you tried what I suggested?

Your idea is to add more threads to improve performance, because you expect all your threads to work perfectly in parallel. This is why you assume that doubling the number of threads should not double the total testing time.

Your service takes a fraction of a second to return, and your threads will not all work at the same instant on the client.

So, your threads do not actually work completely in parallel, as you expected, and the expected results.

Answer 4

You do not see a performance gain because it is not. One line of code in your service (below) is probably executed in any case without a context switch.

 return new Random().NextDouble(); 

The overhead associated with calling a web service is higher than the work you do inside it. If you have significant work inside the service (database calls, browsing, file access, etc.), you can see some increase in performance. Just parallelizing a task will not automatically speed up.

Jason

Answer 5

Of course, adding Sleep will not improve performance.

But the test point must be tested with a variable number of threads. So, save Sleep in your WebMethod.

And try now with 5, 10, 20 threads.

If you have no other problems with your code, then the increase in time should not be linear, as before.

You understand that in your test, when you double the number of threads, you double the amount of work done. Therefore, if your threads are not running in parallel, then of course you will see a linear increase in overall time ...

I conducted a simple test using your client code (with sleep in the service). For 5 threads, I saw a total time of about 53 seconds. And for 10 threads, 62 seconds. Thus, for 2x the number of calls in the web service, it took only 17% more time. Is that what you expect, no?

Answer 6

Well, in this case, you do not balance your work between the selected n.º threads ... Each thread you create will perform the same Job. Therefore, if you create n threads and you have limited parallel processing capacity, performance naturally decreases. Another opinion is that I noticed that the required Job is a relatively fast operation for 100 iterations, and even if you plan to divide this task into several threads, you need to consider that the time it takes to switch context, creating / deleting threads will be an important factor in over time.

Answer 7

As Bruno mentioned, your web method is a very fast operation. As an experiment, try making sure your HelloWorld method takes a little longer. Throw in Thread.Sleep (1000) before returning a random double. This will make it more likely that your service is actually forced to process requests in parallel. Then try your client with a different number of threads and see how performance is different.

Answer 8

Try using some kind of processor task instead of Thread.Sleep. In fact, a combined approach is the best.

Sleep will simply pass the stream of time to another stream.

Answer 9

IIS AppPool Maximum Workflows is set to 1 by default. For some reason, each workflow is limited to 10 service calls at a time. My asynchronous WCF server function is disabled (10 * 1000); only. This is what happens when Maximum Worker Processes = 1 http://s4.postimg.org/4qc26cc65/image.png

as an alternative

http://i.imgur.com/C5FPbpQ.png?1

(First post on SO, I need to combine all the images into one image.)

In this test, the client makes 48 asynchronous WCF WS calls (using 16 processes). Ideally, it will take about 10 seconds (Sleep (10000)), but it takes 52 seconds. You can see 5 horizontal lines in the perfmon image (above the link) (using perfmon to monitor the current web services connections on the server). Each horizontal line lasts 10 seconds (which makes a dream (10,000)). There are 5 horizontal lines, since the server processes 10 calls each time and then closes 10 connections (this happens 5 times to process 48 calls). Completing all calls took 52 seconds.

After setting Maximum Workflows = 2 (in the same picture above), this time there are 3 horizontal lines, because the server processes 20 calls each time and then closes 20 connections (this happens 3 times to process 48 calls). Took 33 seconds.

After setting Maximum Workflows = 3 (in the same picture above), this time there are 2 horizontal lines, because the server processes 30 calls each time. (occurs 2 times to handle 48 calls) Took 24 seconds.

After setting the maximum workflows = 8 (in the same picture above) This time there is 1 horizontal line, because the server processes 80 calls each time. (occurs once to process 48 calls) Took 14 seconds.

If you do not like this situation, your parallel (asynchronous or streaming) client calls will be queued for 10 seconds on the server, then all your streaming calls (> 10) will not be processed by the server in parallel.

PS: I used Windows 8 x64 with IIS 8.5. The limit of 10 simultaneous requests is for Windows operating systems. Server OSs do not have this limitation according to another post in SO (I cannot give a link because of rep <10).