SignalR Owin Self-Host on Linux / Mono SocketException when clients lose connection

Question

I am running a very simple signalr server hosted through Owin on an ubuntu 14.04 server, mono 3.2.8. (code below).

Connect / Disconnect works fine both on a remote Windows server and when deploying bits on a Linux server. But when the client dies unexpectedly, and does not tell signalr that it is disconnecting, this is when I get an infinite SocketException only on the linux server. The Windows server shuts down the client after about 30 seconds or so, but the linux server outputs a socket pivot (also lower) every 10 seconds or so forever.

How to make the linux server behave like a Windows server when running the same code, disconnect the user after the set timeout and not cause sockets?

Server Code:

using System; using System.Threading.Tasks; using Microsoft.AspNet.SignalR; using Owin; namespace signalrtestserver { class Program { static void Main(string[] args) { var uri = System.Configuration.ConfigurationManager.AppSettings["startup_uri"] ?? "http://*:7890"; using (Microsoft.Owin.Hosting.WebApp.Start<Startup>(uri)) { Console.WriteLine(string.Format("Server started on {0}. Press enter to close.", uri)); Console.ReadLine(); } } } class Startup { static Hub hub; public void Configuration(IAppBuilder app) { app.UseCors(Microsoft.Owin.Cors.CorsOptions.AllowAll); var configuration = new HubConfiguration(); configuration.EnableDetailedErrors = true; app.MapSignalR("/signalr", configuration); hub = new MyHub(); } } public class MyHub : Hub { public override Task OnConnected() { Console.WriteLine(Context.ConnectionId + " connected"); return base.OnConnected(); } public override Task OnDisconnected() { Console.WriteLine(Context.ConnectionId + " disconnected"); return base.OnDisconnected(); } public override Task OnReconnected() { Console.WriteLine(Context.ConnectionId + " reconnected"); return base.OnReconnected(); } } } 

Client code:

 using System; using System.Net; using Microsoft.AspNet.SignalR.Client; namespace signalrconnection { class Program { static void Main(string[] args) { var uri = System.Configuration.ConfigurationManager.AppSettings["signalr_uri"] ?? "http://localhost:7890/signalr"; ServicePointManager.DefaultConnectionLimit = 10; var hubConnection = new HubConnection(uri, false); hubConnection.StateChanged += stateChange => Console.WriteLine(string.Format("SignalR {0} >> {1} ({2})", stateChange.OldState, stateChange.NewState, hubConnection.Transport == null ? "<<null>>" : hubConnection.Transport.Name)); var hubProxy = hubConnection.CreateHubProxy("MyHub"); hubConnection.Start(); Console.WriteLine("Press enter to die..."); Console.ReadLine(); //hubConnection.Dispose(); //uncomment this to simulate a graceful disconnect which works on both windows and linux } } } 

Unsuccessful one-block exception:

 {path-to-project}/Microsoft.AspNet.SignalR.Core.dll Error : 0 : SignalR exception thrown by Task: System.AggregateException: One or more errors occured ---> System.IO.IOException: Write failure ---> System.Net.Sockets.SocketException: Connection reset by peer at System.Net.Sockets.Socket.Send (System.Byte[] buf, Int32 offset, Int32 size, SocketFlags flags) [0x00000] in <filename unknown>:0 at System.Net.Sockets.NetworkStream.Write (System.Byte[] buffer, Int32 offset, Int32 size) [0x00000] in <filename unknown>:0 --- End of inner exception stack trace --- at System.Net.Sockets.NetworkStream.Write (System.Byte[] buffer, Int32 offset, Int32 size) [0x00000] in <filename unknown>:0 at System.Net.ResponseStream.InternalWrite (System.Byte[] buffer, Int32 offset, Int32 count) [0x00000] in <filename unknown>:0 at System.Net.ResponseStream.Write (System.Byte[] buffer, Int32 offset, Int32 count) [0x00000] in <filename unknown>:0 at Microsoft.Owin.Host.HttpListener.RequestProcessing.ExceptionFilterStream.Write (System.Byte[] buffer, Int32 offset, Int32 count) [0x00000] in <filename unknown>:0 --- End of inner exception stack trace --- --> (Inner exception 0) System.IO.IOException: Write failure ---> System.Net.Sockets.SocketException: Connection reset by peer at System.Net.Sockets.Socket.Send (System.Byte[] buf, Int32 offset, Int32 size, SocketFlags flags) [0x00000] in <filename unknown>:0 at System.Net.Sockets.NetworkStream.Write (System.Byte[] buffer, Int32 offset, Int32 size) [0x00000] in <filename unknown>:0 --- End of inner exception stack trace --- at System.Net.Sockets.NetworkStream.Write (System.Byte[] buffer, Int32 offset, Int32 size) [0x00000] in <filename unknown>:0 at System.Net.ResponseStream.InternalWrite (System.Byte[] buffer, Int32 offset, Int32 count) [0x00000] in <filename unknown>:0 at System.Net.ResponseStream.Write (System.Byte[] buffer, Int32 offset, Int32 count) [0x00000] in <filename unknown>:0 at Microsoft.Owin.Host.HttpListener.RequestProcessing.ExceptionFilterStream.Write (System.Byte[] buffer, Int32 offset, Int32 count) [0x00000] in <filename unknown>:0 

Any help would be greatly appreciated. Thanks in advance!

Answer 1

The problem is this line of static Hub hub and this hub = new MyHub() in your autoload method.

You do not need to explicitly instantiate the Hub class, and you do not need to maintain a link. A hub class is created with every request to the server. See the http://www.asp.net/signalr/overview/signalr-20/hubs-api/hubs-api-guide-server#transience section for the lifetime of the hub object .

Answer 2

This means that whenever a shutdown occurs, Mono on Linux throws a different exception than MS.NET on Windows. SignalR was implemented by people who most likely used MS.NET, and therefore SignalR should expect a certain exception and be religious in this regard.

The best way to fix this is to debug the code included in the SignalR implementation (when running on Linux with Mono) to see where the exception is located and compare with what happens in SignalR under MS.NET on Windows. Then create minimal test information about the difference and send the error to http://bugzilla.xamarin.com/ , most likely it will fix it in the near future (or you can assign it to me and I will look at it because I have interest to run SignalR under mono).

Answer 3

I had the same issue on a Raspberry Pi with a simple self-serve demo Owin application. Requiring a “welcome” page from a single browser while holding F5 was enough to kill the host in 15 seconds. I haven’t run automatic stress tests yet, as my “consumer acceptance” test failed immediately.

The reliable solution I found is to create a custom OwinMiddleware to catch and slow down the socket exception:

 class CustomExceptionMiddleware : OwinMiddleware { public CustomExceptionMiddleware(OwinMiddleware next) : base(next) { } public override async Task Invoke(IOwinContext context) { try { await Next.Invoke(context); } catch (IOException ex) { Console.WriteLine(ex.Message); } } } 

To use middleware, you need to update the Owin launch:

 using Owin; public class Startup { public void Configuration(IAppBuilder app) { app.Use<OwnExceptionMiddleware>() .UseNancy(); } } 

He was inspired by http://dhickey.ie/2014/02/bubbling-exceptions-in-nancy-up-the-owin-pipeline/

Here is my exception for reference:

 Unhandled Exception: System.IO.IOException: Write failure ---> System.Net.Sockets.SocketException: The socket has been shut down at System.Net.Sockets.Socket.Send (System.Byte[] buf, Int32 offset, Int32 size, SocketFlags flags) [0x00000] in <filename unknown>:0 at System.Net.Sockets.NetworkStream.Write (System.Byte[] buffer, Int32 offset, Int32 size) [0x00000] in <filename unknown>:0 --- End of inner exception stack trace --- at System.Net.Sockets.NetworkStream.Write (System.Byte[] buffer, Int32 offset, Int32 size) [0x00000] in <filename unknown>:0 at System.Net.ResponseStream.InternalWrite (System.Byte[] buffer, Int32 offset, Int32 count) [0x00000] in <filename unknown>:0 at System.Net.ResponseStream.Close () [0x00000] in <filename unknown>:0 at System.Net.HttpConnection.Close (Boolean force_close) [0x00000] in <filename unknown>:0 at System.Net.HttpListenerResponse.Close (Boolean force) [0x00000] in <filename unknown>:0 at System.Net.HttpListenerResponse.Abort () [0x00000] in <filename unknown>:0 at Microsoft.Owin.Host.HttpListener.RequestProcessing.OwinHttpListenerResponse.End () [0x00000] in <filename unknown>:0 at Microsoft.Owin.Host.HttpListener.RequestProcessing.OwinHttpListenerContext.End () [0x00000] in <filename unknown>:0 at Microsoft.Owin.Host.HttpListener.RequestProcessing.OwinHttpListenerContext.End (System.Exception ex) [0x00000] in <filename unknown>:0 at Microsoft.Owin.Host.HttpListener.OwinHttpListener+<ProcessRequestAsync>d__5.MoveNext () [0x00000] in <filename unknown>:0 --- End of stack trace from previous location where exception was thrown --- at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw () [0x00000] in <filename unknown>:0 at System.Runtime.CompilerServices.TaskAwaiter.GetResult () [0x00000] in <filename unknown>:0 at Microsoft.Owin.Host.HttpListener.OwinHttpListener+<ProcessRequestsAsync>d__0.MoveNext () [0x00000] in <filename unknown>:0 

Think about what exceptions you catch:

  public override async Task Invoke(IOwinContext context) { try { await Next.Invoke(context); } catch (IOException ex) { if (ex.HResult == -2146233087) // The socket has been shut down { NLog.LogManager.GetLogger("OwinExceptionHandler").Trace(ex); } else { throw; } } } 

Answer 4

I think I have the same problem. I described this:

• https://katanaproject.codeplex.com/workitem/438
• https://bugzilla.xamarin.com/show_bug.cgi?id=33254#c4

I found a solution for my situation. I set the HttpListener.IgnoreWriteExceptions property to 'true' in the OWIN configuration method and then registered middlewares. For example:

 internal class Startup { public void Configuration(IAppBuilder app) { object httpListener; if (app.Properties.TryGetValue(typeof(HttpListener).FullName, out httpListener) && httpListener is HttpListener) { // HttpListener should not return exceptions that occur // when sending the response to the client ((HttpListener)httpListener).IgnoreWriteExceptions = true; } app.Use<TestOwinMiddleware>(); } } 

Hope this can help you.