Is .NET a non-blocking, single-threaded asynchronous web server (e.g. Node.js)?

Question

I looked at this question looking for a way to create a single-threaded asynchronous non-event web server in .NET.

This answer looked promising at first, claiming that the body of the code works in a single thread.

However, I tested this in C #:

using System; using System.IO; using System.Threading; class Program { static void Main() { Console.WriteLine(Thread.CurrentThread.ManagedThreadId); var sc = new SynchronizationContext(); SynchronizationContext.SetSynchronizationContext(sc); { var path = Environment.ExpandEnvironmentVariables( @"%SystemRoot%\Notepad.exe"); var fs = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.ReadWrite, 1024 * 4, true); var bytes = new byte[1024]; fs.BeginRead(bytes, 0, bytes.Length, ar => { sc.Post(dummy => { var res = fs.EndRead(ar); // Are we in the same thread? Console.WriteLine(Thread.CurrentThread.ManagedThreadId); }, null); }, null); } Thread.Sleep(100); } } 

And the result:

one
5

Thus, it seems that, contrary to the answer, the thread initiating the read and the thread ending the read do not match.

So now my question is: how do you achieve a single-threaded, non-blocking asynchronous web server in .NET?

Answer 1

The whole SetSynchronizationContext is a red herring, it is just a sorting mechanism, work is still going on in the I / O thread pool.

What you are asking for is the queue method and the harvest. Asynchronous procedure calls for all your I / O operations from the main thread. Many higher-level structures wrap this functionality, the most famous of which is libevent .

There is a great description of the various options here: What is the difference between epoll, poll, threadpool? .

.NET already takes care of scaling for you by having a special “IO thread pool” that handles IO access when you call BeginXYZ methods. This I / O thread pool must have at least 1 thread per processor per box. see ThreadPool.SetMaxThreads .

If a single-threaded application is a critical requirement (for some crazy reason), you could, of course, combine all this using DllImport (see example here )

However, this would be a very difficult and risky task :

Why don't we support APC as a termination mechanism? APCs are really not a good general purpose completion mechanism for user code. The reinsertion control introduced by the BTR is virtually impossible; anytime when you lock a lock, for example, some arbitrary I / O termination may take your thread. He may try to acquire his own locks, which can lead to problems with ordering locks and, thus, to deadlock. Preventing this requires careful design and the ability to make sure that another user's code will never work during your expected wait, and vice versa. This greatly limits the usefulness of APC.

So, we repeat. If you need a controlled process with a single thread that does all its work using APC ports and termination, you will have to pass its code. Construction would be risky and difficult.

If you just need a high-performance network, you can continue to use BeginXYZ and the family, and be sure that it will work well, since it uses APC. You pay a negligible markup between threads and a specific .NET implementation.

From: http://msdn.microsoft.com/en-us/magazine/cc300760.aspx

The next step in server scaling is to use asynchronous I / O. Asynchronous I / O makes it easy to create and manage threads. This results in significantly simpler code, as well as a more efficient I / O model. Asynchronous I / O uses callbacks to process incoming data and connections, which means there are no lists to configure and scan, and there is no need to create new workflows to work with pending I / O.

An interesting side fact is that single threaded is not the fastest way to make asynchronous sockets on Windows using completion ports: http://doc.sch130.nsc.ru/www.sysinternals.com/ntw2k/info/comport. shtml

The goal of the server is to switch to the context as little as possible, as its threads avoid unnecessary blocking, while maximizing parallelism with multiple threads. The ideal thing is that there will be a thread actively serving the client request on each processor, and that these threads should not be blocked if there are additional requests waiting for the request to complete. However, for correct operation, there must be a way for the application to activate another thread when one processing of a client request is blocked during I / O (for example, when it reads from a file as part of processing).

Answer 2

What you need is a “message loop” that performs the next task in the queue and executes it. In addition, each task should be encoded so that it does as much work as possible without blocking, and then gives out additional tasks to collect a task that takes time later. There is nothing magical about this: never use a blocking call and never create additional threads.

For example, during HTTP GET processing, the server can read as much data as is currently available on the socket. If there is not enough data to process the request to process the request, then in the future you can set a new task to read from the socket. In the case of FileStream, you want to set ReadTimeout in the instance to a low value and be prepared to read fewer bytes than the whole file.

C # 5 actually makes this pattern more trivial. Many people believe that the asynchronous function implies multithreading, but this is not so . Using async, you can essentially get the task queue that I mentioned earlier without explaining it.

Answer 3

Yes it's called Manos de mono

Seriously, the whole idea of ​​manos is an event-driven, single-threaded asynchronous web server.

High performance and scalability. A model created after tornadoweb, a technology that supports the power of friends, Manos supports thousands of simultaneous connections, ideal for applications that create permanent connections to the server.

The project is apparently low in maintenance and probably will not be ready for production, but it provides good research as evidence that this is possible.

Answer 4

In this article we will talk about which IO I / O ports and how they can be accessed through C # (i.e. you need to call PInvoke in Win32 API calls from the Kernel32.dll file).

Note. libuv the cross-platform I / O platform behind node.js uses IOCP for Windows and libev on unix operating systems.

http://www.theukwebdesigncompany.com/articles/iocp-thread-pooling.php

Answer 5

I'm curious no one mentioned kayak , this is basically C # answer on Pythons twisted , JavaScripts node.js or Rubys eventmachine

Answer 6

I was doing my simple implementation of such an architecture, and I put it on github . I do it rather as a subject of study. But it was a lot of fun, and I think I’m more promotable.

It is very alpha, so it can change, but the code looks something like this:

  //Start the event loop. EventLoop.Start(() => { //Create a Hello World server on port 1337. Server.Create((req, res) => { res.Write("<h1>Hello World</h1>"); }).Listen("http://*:1337"); }); 

More information about him can be found here .

Answer 7

I developed a server based on HttpListener and an event loop supporting MVC, WebApi and routing. For what I saw, the performance is much better than the standard IIS + MVC, for MVCMusicStore I moved from 100 requests per second and 100% of the CPU to 350 with a 30% processor. If someone tries to try, I fight for reviews! In fact, there is a template for creating sites based on this structure.

Please note that I DO NOT USE ASYNC / AWAIT until absolutely necessary. The only tasks I use for them are those related to I / O, such as writing to a socket or reading files.

PS any suggestion or correction is welcome!

• Documentation
• MvcMusicStore sample port on Node.Cs
• Packages in Nuget

Answer 8

you can use this infrastructure SignalR and Blog about it

Answer 9

Some kind of operating system support is important here. For example, Mono uses epoll for Linux with asynchronous I / O, so it should scale very well (still a thread pool). If you're looking for performance and scalability, be sure to give it a try.

On the other hand, an example of a C # web server (with native libs) based on the idea you mentioned might be Manos de Mono. Recently, the project is inactive; however, idea and code are usually available. Read this (especially in the "Closer Look at Manos" section).

Edit:

If you just want the callback to start in the main thread, you can slightly abuse existing synchronization contexts, such as WPF Manager. Your code translated to this approach:

 using System; using System.IO; using System.Threading; using System.Windows; namespace Node { class Program { public static void Main() { var app = new Application(); app.Startup += ServerStart; app.Run(); } private static void ServerStart(object sender, StartupEventArgs e) { var dispatcher = ((Application) sender).Dispatcher; Console.WriteLine(Thread.CurrentThread.ManagedThreadId); var path = Environment.ExpandEnvironmentVariables( @"%SystemRoot%\Notepad.exe"); var fs = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.ReadWrite, 1024 * 4, true); var bytes = new byte[1024]; fs.BeginRead(bytes, 0, bytes.Length, ar => { dispatcher.BeginInvoke(new Action(() => { var res = fs.EndRead(ar); // Are we in the same thread? Console.WriteLine(Thread.CurrentThread.ManagedThreadId); })); }, null); } } } 

prints what you want. In addition, you can set priorities with the dispatcher. But you must admit, this is ugly, hacks, and I don’t know why I will do this for any other reason than to answer your demo request;)

Answer 10

First about the SynchronizationContext. This is exactly the same as Sam wrote. The base class will not give you single-threaded functions. You probably got this idea from WindowsFormsSynchronizationContext, which provides functionality for executing code in a user interface thread.

You can read more here.

I wrote a code snippet that works with ThreadPool parameters. (Again, what Sam already pointed out).

This code logs 3 asynchronous actions that must be performed in a free thread. They work in parallel until one of them changes ThreadPool parameters. Then each action is performed in the same thread.

This only proves that you can force a .net application to use a single thread. The real implementation of a web server that will receive and process calls on only one thread is completely different :).

Here is the code:

 using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.Threading; using System.IO; namespace SingleThreadTest { class Program { class TestState { internal string ID { get; set; } internal int Count { get; set; } internal int ChangeCount { get; set; } } static ManualResetEvent s_event = new ManualResetEvent(false); static void Main(string[] args) { Console.WriteLine(Thread.CurrentThread.ManagedThreadId); int nWorkerThreads; int nCompletionPortThreads; ThreadPool.GetMaxThreads(out nWorkerThreads, out nCompletionPortThreads); Console.WriteLine(String.Format("Max Workers: {0} Ports: {1}",nWorkerThreads,nCompletionPortThreads)); ThreadPool.GetMinThreads(out nWorkerThreads, out nCompletionPortThreads); Console.WriteLine(String.Format("Min Workers: {0} Ports: {1}",nWorkerThreads,nCompletionPortThreads)); ThreadPool.QueueUserWorkItem(new WaitCallback(LetsRunLikeCrazy), new TestState() { ID = "A ", Count = 10, ChangeCount = 0 }); ThreadPool.QueueUserWorkItem(new WaitCallback(LetsRunLikeCrazy), new TestState() { ID = " B ", Count = 10, ChangeCount = 5 }); ThreadPool.QueueUserWorkItem(new WaitCallback(LetsRunLikeCrazy), new TestState() { ID = " C", Count = 10, ChangeCount = 0 }); s_event.WaitOne(); Console.WriteLine("Press enter..."); Console.In.ReadLine(); } static void LetsRunLikeCrazy(object o) { if (s_event.WaitOne(0)) { return; } TestState oState = o as TestState; if (oState != null) { // Are we in the same thread? Console.WriteLine(String.Format("Hello. Start id: {0} in thread: {1}",oState.ID, Thread.CurrentThread.ManagedThreadId)); Thread.Sleep(1000); oState.Count -= 1; if (oState.ChangeCount == oState.Count) { int nWorkerThreads = 1; int nCompletionPortThreads = 1; ThreadPool.SetMinThreads(nWorkerThreads, nCompletionPortThreads); ThreadPool.SetMaxThreads(nWorkerThreads, nCompletionPortThreads); ThreadPool.GetMaxThreads(out nWorkerThreads, out nCompletionPortThreads); Console.WriteLine(String.Format("New Max Workers: {0} Ports: {1}", nWorkerThreads, nCompletionPortThreads)); ThreadPool.GetMinThreads(out nWorkerThreads, out nCompletionPortThreads); Console.WriteLine(String.Format("New Min Workers: {0} Ports: {1}", nWorkerThreads, nCompletionPortThreads)); } if (oState.Count > 0) { Console.WriteLine(String.Format("Hello. End id: {0} in thread: {1}", oState.ID, Thread.CurrentThread.ManagedThreadId)); ThreadPool.QueueUserWorkItem(new WaitCallback(LetsRunLikeCrazy), oState); } else { Console.WriteLine(String.Format("Hello. End id: {0} in thread: {1}", oState.ID, Thread.CurrentThread.ManagedThreadId)); s_event.Set(); } } else { Console.WriteLine("Error !!!"); s_event.Set(); } } } } 

Answer 11

LibuvSharp is a wrapper for libuv, which is used in the node.js project for async IO. BUt it contains only low-level TCP / UDP / Pipe / Timer functions. And it will remain such that writing a web server on top of it is a completely different story. It does not even support dns resolution, since it is just a protocol on top of udp.

Answer 12

I believe this is possible, here is an open source example written in VB.NET and C #:

https://github.com/perrybutler/dotnetsockets/

It uses an event-based asynchronous pattern (EAP), an IAsyncResult pattern, and a thread pool (IOCP). It will serialize / organize the messages (messages can be any native objects, such as an instance of a class), into binary packets, transmit packets via TCP and then deserialize / decouple the packets on the receiving side so that you can work with your native object, This part is somewhat reminds Protobuf or RPC.

It was originally designed as "netcode" for real-time multiplayer games, but it can serve many purposes. Unfortunately, I never used it. Maybe someone else will.

The source code contains a lot of comments, so it should be easy to track. Enjoy it!

Answer 13

Here is another implementation of the event loop web server called SingleSand . It runs all the custom logic inside a single-threaded event loop, but the web server is hosted on asp.net. Answering the question, as a rule, it is impossible to launch a single-threaded single-user application due to the multithreaded .NET technology. There are some actions that are performed in separate threads, and the developer cannot change his behavior.