A web service that can handle 1000 concurrent users with a response in 25 milliseconds

Question

Our client is developing WCF, which can withstand users with a 1-2k parallel site, and the response should be about 25 milliseconds.

This service reads a couple of columns from the database and will be consumed by different providers.

Can you suggest any architecture or any additional efforts that I need to make when developing. And how do we calculate the server hardware configuration to handle it.

Thanks in advance.

Answer 1

Hardly possible. You need a network connection, a service, activation of business logic, a database connection (another network connection), a database request. Because of the simultaneous users of 2000, several application servers are required: the network connection is affected by load balancing. I can not imagine the network infrastructure and HW, which should be able to complete such an operation within 25 ms for simultaneous users of 2000. Such a requirement is unrealistic.

I think if you just try to run a database query from your computer to a remote database, you will see that even such a simple task will not be completed in 25 ms.

Answer 2

A few principles:

• Test early on, check often.
• Successful systems get more traffic
• Reliability is usually important.
• Caching is often the key to performance

To develop. Create a simple system right now. Even if the business logic is very simplified, if it is a web service and database access, you can test it. Test with one user. What do you see? Where does the time go? As you develop the system by adding to the actual code, continue to do this test. Reasons: a). right now you know that 25ms is even achievable. b) You will find any code changes that immediately impair performance. Now test with a lot of users, which degradation patterns do you click? It begins to give you and indicate your capabilities in the palm of your hand.

I suspect the result will be that one machine does not cut it for you. And even if this happens, if you succeed, you will get more traffic. Therefore, plan to use multiple servers.

And in any case, for reliability reasons, you need more than one server. And all sorts of interesting implementation details fall out when you cannot accept a single server - for example. you no longer have Singleton; -)

In most cases, we get good performance using the cache. Will many users request the same data? Can you cache it? Are there any updates to consider? In this case, do you need a distributed caching system with cluster revocation? This multiserver case reappears.

Answer 3

Why do you need WCF?

Could you switch as much of this service as possible to the static service and cache search?

If I understand your question, 1000 users will beat your site and execute queries in your database. You should definitely look into connection pools on your WCF connections, but it is best to avoid doing a database lookup altogether and make your site return data from the cache.

I would also see why you couldn’t just directly connect to the database for your searches, do you really need the WCF service in the first place?

Take a look at memcached.