Server 500: Too Many Pending Secure Chains

Question

I have a fairly simple WCF web service hosted on IIS Express (ultimately full IIS) using .Net 3.5. The service method is rather uninteresting.

[ServiceContract] public class MySvc { [OperationContract] public Stuff MyMethod(string input) { Stuff result = DoSomething(); return result; } } 

The service configuration is also quite general:

 <system.serviceModel> <services> <service behaviorConfiguration="MySvcBehavior" name="MySvc"> <endpoint address="" binding="wsHttpBinding" contract="MySvc"> <identity> <dns value="localhost"/> </identity> </endpoint> <endpoint address="mex" binding="mexHttpBinding" contract="IMetadataExchange"/> </service> </services> <behaviors> <serviceBehaviors> <behavior name="MySvcBehavior"> <serviceMetadata httpGetEnabled="true"/> <serviceDebug includeExceptionDetailInFaults="false"/> </behavior> </serviceBehaviors> </behaviors> </system.serviceModel> 

The service is used in code in an ASPX application. There is a service link leading to some equally uninteresting code.

 MySvcClient svc = new MySvcClient(); Stuff result = svc.MyMethod("foo"); 

So far, this is one request at a time, everything works fine, and the client code gets the expected result. Yay

The problem arises when I do very primitive stress testing. I load the ASPX client page into a browser and then hold down the F5 key. Watching the IIS Express window, at first the results are returned as status 200, but after a few minutes I start to see status 500. At this point, the service will only respond with status 500 until I restart IIS Express. (Based on an expectation of about 10 minutes.)

Setting a breakpoint in the client code, I see that the full return message is "There are too many pending secure conversations on the server. Please try again later."

Setting a breakpoint in the server code, I believe that my code is not even called. Thus, it does not work somewhere between the call and the actual start of my code.

My searches on the Internet were not very promising, basically leading to the same suggestion of writing a custom binding to override the maxPendingSessions property and the thread starting with “Someone told me that there is a [unnamed] configuration file file”, which then leads to a broken link claiming that Microsoft recognized this as a mistake.

The maxPendingSessions property link mentions a limit of 128 connections with a timeout of two minutes, and I can, of course, see where my testing method will interrupt some connections. Is this the expected result of an erroneous testing methodology? Or can something be done in the configuration to improve this?

Answer 1

It looks like you have zillion open connections on the server - due to the fact that the client side does not use "use" to connect.

Answer 2

I would first suggest checking this on full IIS on a server class machine. I would not trust IIS Express on the client OS for performance testing.
If it is still bad, then look at throttling to limit maxConcurrentCalls. http://msdn.microsoft.com/en-us/library/vstudio/ms735114(v=vs.90).aspx

Answer 3

There are too many connections to the destination server. There are several ways to close the connection correctly. It has already been mentioned that there should always be a using statement around the WCF proxy.

In addition, there was (is there?) An error requiring a request from you when opening a proxy. When creating a proxy server, it gives the proxy about two minutes to make a request to the server. If so, this invalidates the timer, allowing the proxy to be cleared. If no request has been made, the proxy server still waits for a timeout before the connection is actually closed. Perhaps this time not so, but added for completeness.

One way around this is to use caching (ASP.NET). System.Web.Caching.Cache is a good candidate for this. It allows you to cache the results for a given (eventually moving) period. Thus, only one request is created, say, 2 minutes. This makes the / webservice website highly scalable. For example, if the number of requests per minute is 1000, only one connection is used for the same service call.

Another way to do this, if the request is expensive and not worth the wait, make a request in advance and cache the response (ala cron, task scheduler or just another thread). This will allow the page for quick searches.

In addition, the server must be configured correctly. There is no need to have 128 concurrent connections open from one client. Try to limit this, for example, to 8 connections per client. This allows multiple clients to be processed simultaneously. Also try setting a decent timeout. Waiting 5 minutes for the client to close its connection does not actually increase throughput.

Also consider asynchronous message processing . It uses the time during which the service must wait for I / O to process additional connections. I do not see what the WCF service does, so this may not be relevant.

Answer 4

For those who recommend using the statement: you should not use the using statement to close clients.

See http://msdn.microsoft.com/en-us/library/aa355056.aspx

Instead of closing clients, you should use the following snippet:

 try { ... client.Close(); } catch (CommunicationException e) { ... client.Abort(); } catch (TimeoutException e) { ... client.Abort(); } catch (Exception e) { ... client.Abort(); throw; } 

Or just a vanilla trick if you prefer not to cache explicit exceptions. Sometimes you want to catch obvious exceptions, because if the communication object is not damaged, you can repeat the operation that makes the best use of resources.

To your problem of waiting conversations on the server, this will become less common after your client closes properly, however you can still see it if your server-side code has a long operation and the clients are in the queue. In this case, to increase the MaxPendingChannels value, you must either use the user binding in the client and receiver side configuration files, or modify the binding on the receiver side programmatically before starting the service. The default value of MaxPendingChannels is actually 4 if you are using .NET 3.5. The value is specified as 128 for .NET 3.0. In my opinion, both are low.

See http://social.msdn.microsoft.com/Forums/en-US/wcf/thread/745b95d2-1480-4618-89f5-1339d6ee228d