How to do distributed computing with F # (.Net)

Question

So, I recently had a little fun in F # with the Async library and was very surprised especially by Async.Parallel, which basically executes Async tasks and combines them under one Async task.

However, now I am curious how to distribute the task of computing between several computers, say, for example, two laptops on my desk.

How can I serialize these Async tasks and send them to another computer, which then performs the task and sends the result?

Or maybe I need to serialize the data itself and send it to another computer on which I have some code running that performs the calculations and sends the result back?

Or maybe there is another simplified way to do this?

What is the general approach to distributed computing in F # using .Net? (recommended design patterns, tools, libraries, etc.)

My ultimate goal is to break down the big task of computing into smaller parts and run them on multiple machines. Preferably in a simplified non-corporate, overly complex way.

Answer 1

There is a project called MBrace that does exactly what you describe :-).

This allows you to write cloud computing using a cloud block:

 let first = cloud { return 15 } let second = cloud { return 27 } 

You can compose them using let! as with asynchronous workflows, and you can also create them from asynchronous workflows using Cloud.ofAsync . Cloud computing can be distributed across the network using Cloud.Parallel :

 cloud { let! results = [ first; second ] |> Cloud.Parallel return List.sum results } 

There are currently MBrace bindings for performing local computations (for testing) and within the Azure cluster, but work is ongoing on Amazon support.

For more information, contact mbrace.io, and Matthias Brandewinder is pleased to talk about how to handle big data with MBrace.

Answer 2

fsharp.org has a page dedicated to Cloud Data, Compute and Messaging with F # , it provides modern resources for the cause.

As Tomas Petrichek said, MBrace seems to be an idiomatic way of distributing distributed computing in F #. Unfortunately, it focuses on cloud computing (Azure and Amazon) and provides very little information about on-premises clusters with multiple machines. I found thread that deals with the subject matter and seems to provide a solution, but the official tutorial (and maybe some built-in functions) would be nice.

Microsoft Prajna was developed in F # and offers an alternative to Sparks. The documentation explains how to create local clusters for multiple machines (only a Windows machine seems). It may be the simplest solution, but it seems dead.

Another option might be akka.net, which has an F # API .

Answer 3

I definitely recommend you akka.net. I am currently implementing distributed integration solutions using it and I can say that it is awesome. Microsoft Research's Orleans project is also pretty good, although it's not an idiomatic f # approach