How to clear outgoing message buffers on a server?

Question

I wrote a service using PollingDuplexHttpBinding , which has a Silverllight client that uses it. My Service basically has a certain number of clients sending their data (often every second, and the data is quite large, each call is about 5 KB) for this service, as well as listening to new data sent by other clients to the service to be sent to them, it’s very similar to chat room architecture.

The problem that I notice is that when clients connect to the service via the Internet, after a few minutes the response of the service slows down and the responses become lagging. I came to the conclusion that when the bandwidth of the service host is reached (Internet download speed on the server is about 15 KB / s), messages sent by other clients are buffered and processed accordingly when there is available bandwidth. I am wondering how can I limit the capture of this buffer that the service uses to store received messages from clients? It is not so critical that my clients receive all the data, but rather receive the latest data sent by others, so a real-time connection is what I am looking for due to guaranteed delivery.

In short, I want to be able to clear the queue / buffer in the service whenever it is full, or a certain cap is reached, and again fill it with received calls to get rid of the delay. How can I do it? Is the MaxBufferSize property what I need to reduce on the service side as well as the client side? Or do I need to encode this function in my service? Any ideas?

Thanks.

EDIT:

Here is my service architecture:

 //the service [ServiceContract(Namespace = "", CallbackContract = typeof(INewsNotification))] [AspNetCompatibilityRequirements(RequirementsMode = AspNetCompatibilityRequirementsMode.Allowed)] [ServiceBehavior(ConcurrencyMode = ConcurrencyMode.Multiple, InstanceContextMode = InstanceContextMode.Single)] public class NewsService { private static Dictionary<IChatNotification, string> clients = new Dictionary<IChatNotification, string>(); private ReaderWriterLockSlim subscribersLock = new ReaderWriterLockSlim(); [OperationContract(IsOneWay = true)] public void PublishNotifications(byte[] data) { try { subscribersLock.EnterReadLock(); List<INewsNotification> removeList = new List<INewsNotification>(); lock (clients) { foreach (var subscriber in clients) { if (OperationContext.Current.GetCallbackChannel<IChatNotification>() == subscriber.Key) { continue; } try { subscriber.Key.BeginOnNotificationSend(data, GetCurrentUser(), onNotifyCompletedNotificationSend, subscriber.Key); } catch (CommunicationObjectAbortedException) { removeList.Add(subscriber.Key); } catch (CommunicationException) { removeList.Add(subscriber.Key); } catch (ObjectDisposedException) { removeList.Add(subscriber.Key); } } } foreach (var item in removeList) { clients.Remove(item); } } finally { subscribersLock.ExitReadLock(); } } } //the callback contract [ServiceContract] public interface INewsNotification { [OperationContract(IsOneWay = true, AsyncPattern = true)] IAsyncResult BeginOnNotificationSend(byte[] data, string username, AsyncCallback callback, object asyncState); void EndOnNotificationSend(IAsyncResult result); } 

Service Configuration:

  <system.serviceModel> <extensions> <bindingExtensions> <add name="pollingDuplex" type="System.ServiceModel.Configuration.PollingDuplexHttpBindingCollectionElement, System.ServiceModel.PollingDuplex, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" /> </bindingExtensions> </extensions> <behaviors> <serviceBehaviors> <behavior name=""> <serviceMetadata httpGetEnabled="true" /> <serviceThrottling maxConcurrentSessions="2147483647" /> <serviceDebug includeExceptionDetailInFaults="true" /> </behavior> </serviceBehaviors> </behaviors> <bindings> <pollingDuplex> <binding name="myPollingDuplex" duplexMode="SingleMessagePerPoll" maxOutputDelay="00:00:00" inactivityTimeout="02:00:00" serverPollTimeout="00:55:00" sendTimeout="02:00:00" openTimeout="02:00:00" maxBufferSize="10000" maxReceivedMessageSize="10000" maxBufferPoolSize="1000"/> </pollingDuplex> </bindings> <serviceHostingEnvironment aspNetCompatibilityEnabled="true" multipleSiteBindingsEnabled="true" /> <services> <service name="NewsNotificationService.Web.NewsService"> <endpoint address="" binding="pollingDuplex" bindingConfiguration="myPollingDuplex" contract="NewsNotificationService.Web.NewsService" /> <endpoint address="mex" binding="mexHttpBinding" contract="IMetadataExchange" /> </service> </services> </system.serviceModel> <system.webServer> <directoryBrowse enabled="true" /> </system.webServer> </configuration> 

The client will usually call the service between periods from 500ms to 1000ms, for example:

_client.PublishNotificationAsync(byte[] data);

and the callback will notify the client of notifications sent by other clients:

 void client_NotifyNewsReceived(object sender, NewsServiceProxy.OnNewsSendReceivedEventArgs e) { e.Usernamer//WHich client published the data e.data//contents of the notification } 

Thus, when the number of clients increases, and the speed of loading the host nodes of the service on the Internet is limited, messages sent by the service to subscribers receive a buffer somewhere and are processed in the queue, which is the reason for the problem, I do not know where these messages are buffered. The service works fine on the local network because the server has a download speed equal to its download speed (for 100 KB / s incoming calls, it sends 100 KB / s notifications). Where are these messages buffered? And how can I clear this buffer?

I did something experimental to try to check if messages are being buffered in the service, I tried calling this method on the client, but it always returns 0, even when one client is still in the process of receiving notifications that someone else shipped 4-5 minutes ago:

 [OperationContract(IsOneWay = false)] public int GetQueuedMessages() { return OperationContext.Current.OutgoingMessageHeaders.Count(); } 

Answer 1

I did some math for your situation.

• message size = 100 KB
• download channel = 15 KB / s
• Channel load = 100 KB / s
• The customer calls the service 1-2 times per second

the client will usually call the service between periods from 500 ms to 1000 ms

Is it correct?

For one client, your message-only download traffic will be 100-200 KB / s, and this is just the message body. More with a headline and more with security enabled.

Messages are collected for an asynchronous call. Therefore, if we have 3 clients, and each sent message callback contains 2 messages for each client. 4 clients - 3 messages in each callback.

For 3 clients, this will be 200-400 KB / s in the download channel.

Messages appear to be too large for the declared bandwidth.

Check if you can:

• Reduce message size. I do not know the nature of your business, so I can not give advice here.

• Use compression for messages or traffic.

• Increase network bandwidth. Without this, even the perfect solution would have a very high latency. You can spend days and weeks optimizing your code, and even if you use the network solution correctly, it will still be slow.

I know this sounds like Captain Obvious, but some questions just don't have the right answer unless you change the question.

After completing the above steps, follow the steps with ServiceThrottlingBehavior in conjunction with custom code that will manage the callback.

ServiceThrottlingBehavior rejects requests if the boundary is reached.

http://msdn.microsoft.com/en-us/library/ms735114(v=vs.100).aspx

This is a trivial pattern. Real numbers should be determined specifically for your environment.

 <serviceThrottling maxConcurrentCalls="1" maxConcurrentSessions="1" maxConcurrentInstances="1" /> 

Update:

A big mistake on my part in the question, each call is 5 KB / s,

Even with 5 KB, 15 KB / s is not enough. Let traffic be calculated for only 3 users. 3 Incoming message per second. The system should now use duplex to notify users of what others have sent. Each user should receive messages from their partners. Only 3 posts. One (which belongs to the sender) can be skipped so that each user receives 2 messages. 3 users will receive 10 KB (5 KB + 5 KB = one message 10 KB) = 30 KB. One message per second will make 30 Kb / s in the download channel for 3 users.

Where are these messages buffered? And how can I clear this buffer?

It depends on how you place your service. If it is a self-service, it is not buffered at all. You have a code that tries to send a message to the recipient, but it is very slow because the channel is flooded. There may be some buffering with IIS, but it is not recommended to practice it from the outside.

The correct solution is throttled. With a bandwidth limit, you should not try to send all messages to all clients. Instead, you should, for example, limit the number of threads sending messages. Thus, from the 3 users above, instead of sending three messages in parallel, you can send them sequentially or decide that only one user will receive an update in this round and the next in the next round.

Thus, the general idea is not to send everything as soon as you have data, but to send only the amount of data that you can afford, and control it using the number of streams or the response rate.

Answer 2

MaxBufferSize will not help you with this problem. You will have to encode it yourself. I do not know any existing solution / framework. This sounds like an interesting issue. You can start by maintaining a Queue<Message> for each connected client, and when you click on this queue (or when the client calls dequeue) you can overestimate what to send Message .

UPDATE: firstly, I would forget about trying to do this from the client side and in the configuration, you will have to code it yourself

Here I can see where you are sending to your customers:

  subscriber.Key.BeginOnNotificationSend(data, GetCurrentUser(), onNotifyCompletedNotificationSend, subscriber.Key); 

so that instead of pushing these notifications asynchronously to each client, you should insert them into Queue<byte[]> . Each client connection will have its own queue, and you should probably create a class dedicated to each client connection:

note that this code will not compile out of the box and probably has some logical errors, use only as a guide

 public class ClientConnection { private INewsNotification _callback; private Queue<byte> _queue = new Queue<byte>(); private object _lock = new object(); private bool _isSending public ClientConnection(INewsNotification callBack) { _callback=callback; } public void Enqueue(byte[] message) { lock(_lock) { //what happens here depends on what you want to do. //Do you want to only send the latest message? //if yes, you can just clear out the queue and put the new message in there, //or you could keep the most recent 5 messages. if(_queue.Count > 0) _queue.Clear(); _queue.Enqueue(message); if(!_isSending) BeginSendMessage(); } } private void BeginSendMessage() { _isSending=true; _callback.BeginOnNotificationSend(_queue.Dequeue(), GetCurrentUser(), EndCallbackClient, subscriber.Key); } private void EndCallbackClient(IAsyncResult ar) { _callback.EndReceive(ar); lock(_lock) { _isSending=false; if(_queue.Count > 0) BeginSendMessage();//more messages to send } } } 

Imagine a scenario where one message is forwarded to a client and when another 9 messages are sent, it calls ClientConnection.Enqueue . When the first message is completed, it checks the queue, which should contain only the last (9th message)