MPI vs Microsoft WCF vs Microsoft TPL

Question

I have a scientific program written in F # that I want to parallelize and run on the same server with several processors (64), as well as in the future and in the cloud (Windows Azure?). The program will have a simple 1-1 connection between nodes (without translation, etc.).

If I used WCF, would it be as fast as MPI? What is MPI that is not in WCF? There is Pure MPI.NET written for WCF, which puzzles me even more. I do not know whether to use WCF or MPI.NET or Pure Mpi in WCF.

PS: I think TPL left the game for 64 processors and more, right?

Answer 1

It is difficult to give a specific answer, because it all depends on the specific aspects of your application, its current architecture (I suppose you already have the application), etc.

• As you point out MPI and WCF, I assume that the application is written as several components that communicate with each other. The best way to structure such an application is to use F # agents . As far as I understand, you want to run the application on one server first. If you write it using agents, agents can simply communicate directly with each other (so you don't need MPI or WCF).

• TPL should work well on one server (with a large number of processors), but it will not scale to distributed settings - you cannot run Task on another machine. However, you can use it inside individual components (for example, agents) to be distributed.

Regarding MPI vs WCF - I have not enough experience to answer this question. However, if you are using an aggregated architecture, it should be easy for you to try various options. You can also check fracture and related projects whose goal is to implement high-performance sockets for F # (and possibly distributed agents in the future).

Answer 2

If you do this on the same server, you can simply execute one process and execute the code in parallel. That way, you could share memory more easily and faster than with messages like MPI and WCF. Although the communication overhead may not be the same, depending on your problem + solution.

In addition, the changes in your code will be much smaller, F # can usually be turned into parallel code without much effort. Switching to MPI / WCF will require rewriting large portions.

Googling for F # + parallel provides a lot of useful information that you should read first, like this for a good start: http://blogs.msdn.com/b/dsyme/archive/2010/01/09/async-and-parallel -design-patterns-in-f-parallelizing-cpu-and-io-computations.aspx

So, on 1 server, I will use F # parallel functions, which are intended for simple distribution.

Later, when you want to go to the cloud, this will turn it into a cleint-server. This is another problem, and then paralysis. I will relate to them and solve them separately.

In MPI vs WCF. WCF is designed as an RPC technology, i.e. You call remote procedures and get answers. If you want to use it for parallel programming with separate processes, you will need to create boilerplate code for it. (Keep track of subscribers, etc.)

MPI was designed to work with such an architecture and is easier to deal with. (the first process gets number 0 and is the leader, the others - subordinates, which will be numbered, etc.)

No matter how I think that MPI will be very good to the cloud, as it is related to http, protocols, security, etc. Not sure how well MPI works for this kind of thing, WCF will do it very well.

The fact that there is MPI.NET for WCF is that MPI is a specific style of paralysis code that many people are familiar with. This way you can use programming concepts and use them on the .NET platform using WCF for communication.

Something else you might want to peek at if you need to exchange a lot of data over the wire, these are protocol buffers (see protobuf-net for example). It can be easily combined with WCF for communication and very poorly serializes structured data so that you can efficiently send by wire.

Geert Yang

Answer 3

WCF and MPI are different concepts. WCF looks like a man A asks man B to do something when the MPI looks like a man A creates clones of himself (all clones have the same abilities / logic), and then these clones work on certain parts of the problem that need to be solved, and once they combine their results.

Thus, choosing between the one that suits your particular application depends on the problem your application is trying to solve. It can even be a combination of both WCF and MPI. If your client application asks WCF to perform some task, and WCF creates “problem solver” clones using MPI, and when the clone runs to solve the problem (in parallel), they return the aggregated result back to WCF, and then the result is sent to the client application.

Answer 4

You may also want to take the "mbrace" product, which provides a cloud monad (http://blogs.msdn.com/b/dsyme/archive/2011/08/23/m-brace-f- in-cloud.aspx). However, it is still at a fairly early stage. I am not an expert, but it can happen that you can run a solution based on mbrace, as well as a private cloud on your 64-processor setup. When you outgrow this, the transition to Azure will be smooth.