Template for .NET parallelism outside of one computer

Question

I suspect that I will soon exhaust the possibilities of increasing the flow rate on several cores on the same computer.

What does this .NET desktop programmer have to do in order to learn how to move an acceptable problem in parallel on several computers? My preference is to minimize all life cycle programming efforts, so it would be preferable if there were minimal changes between on-premises deployment and outdoor deployment.

As for the programmer’s man-hours, is Linux, LAMP, or some other stack method better than C # .NET for Windows for such an application?

Edit: More information from my own comments below. The intensive computational part of the problem can be made arbitrarily large so the overhead for distribution / recombination is not to worry, because the overhead is a small percentage of the time you need to wait for the result. This is one development team. Just a suggestion, and I don’t know whether it’s any good or not: what about WCF and XML as a means of spreading the problem in a completely closed Azure-ignorant way and trust in it will (someday) work on Azure without any changes and without the benefits of Azure conscious. This is simply not a proven idea, and I hope that someone has a better one, even if it is not a Windows Solution.

Other editing: Digipede has a proposal for better performance and a document on the difference between a cluster and a grid.

http://www.digipede.net/downloads/Digipede_CCS_Whitepaper.pdf

Since my problem is more like a grid than a cluster, and I want to do it cheaply, I just try the WCF approach.

Answer 1

Creating a compute farm mechanism using WCF will be a simple IMO. Since you already use C # on Windows, this is a natural progression compared to a switch language or technology stack.

An early step in this process will be to develop a mechanism by which calculators can advertise their accessibility to the master machine. Either the owner would have a priori knowledge of the workers, or (better) they need a consistent mechanism for "localizing" the server, for example. in a well-known domain. Assuming the wizard, say, www.all-your-cycles-belong-to-us.org, all you need is for the WCF service to handle incoming computation time sentences. If the mechanism of your delegation can be adjusted according to the number of workers, all the better.

Identifying your services, data contracts, and failures between Teacher and workers may take some experimentation to achieve the best balance of programming, elegance, computational throughput, and flexibility / future validation.

From experience, the kind of problems of this (and other) approaches:

• The worker is quiet.

  Due to network problems, being “busy” for long periods of time or the actual downtime, it is difficult to say until the connection with the master is restored. In my daily work, we have thousands of cars that “call home” periodically and for hours on end, without calling home, it is considered “down”. Do you have to disconnect another employee to do the same job or wait a while to complete the original? Only you know your algorithm, but a combination of both approaches can help.

• Abuse of workers.

  If your computing problem is really complex, you can combine the CPU on all workers. Would that be acceptable? If you rent processor cycles, then yes. If you pump spare cycles on idle machines (a la SETI), then no.

• Results fail.

  Can your results be collected in the correct order by the master if different workers end at different times?

• Checking code versions.

  If you fix the code, how did you send it to all employees to make sure they have the correct version? There are many options for solving this problem, but it should be considered sooner rather than later.

• Peerless workers.

  Having a high-performance multiprocessor worker participating in your computing farm, as well as low-grade solo processors with processors, will bring strange behavior if you do not know that the workers were of different specifications. Adapting your WCF interfaces allows an Employee to hint at how much workload he can take on, maybe worth your attention.

Answer 2

The main thing to consider when moving from multi-threaded to distributed computing is the increase in overhead for running tasks on remote machines compared to spooling another thread on the current computer. The granularity of the work elements must be large enough to justify a much slower connection between nodes. Messaging between threads on the same computer is many orders of magnitude faster than messaging between different computers over a network.

Sharing resources is harder on machines. Sharing objects in memory is simple in multiple threads in the same process, but requires some engineering to achieve similar work on different machines. Locks basically do not exist on different machines. Look at using the Message Queuing / Server service to coordinate work between multiple machines, return results to an aggregator, etc.

You mention "offside". If you are considering external computing resources, be sure to look for them for cloud computing or flexible computing service providers. Oddly enough, they are not used in one breath, like parallel programming, as often as you think. Cloud computing offers you the ability to scale parallelism to hundreds or thousands of compute nodes, for which you only pay when you use them. When your calculations are completed or a live source for analyzing your data goes home at the end of the day, you can turn off your cloud nodes and stop billing hours until you start them again.

Amazon, Google and Microsoft are three major cloud service providers (among others), and each of them has very different characteristics, strengths and weaknesses. I am working on Azure materials at Microsoft. Laser embedded message queues are pretty handy for running workflows / workflows on a scale.

Do you use LAMP or .NET, since your platform is really less about performance issues and more about the tools and skill sets that you have in your development team. The deliberate choice of a target platform that is inconsistent with your skill set of the development team is a great way to add a lot of time and re-qualify expenses in your project schedule.

C # /. NET works very well for coding parallel systems compared to C ++ or scripts in other environments. Explore the language features, debugging tools, and ready-made libraries and services available to you when evaluating which platform is best for your skill set and desired system design.

Answer 3

I would recommend reading in CCR and DSS technology from Microsoft. This is a really good parallelization implementation by sending bits of work to the “ports”. These ports are read by workers (threads), which really use the available kernels as an added effect.

DSS is an extra layer that makes it easy to use the same concept for multiple machines.

a good introduction can be read here: parallel affairs

a very good third-party xcoappspace library is available as an alternative implementation of ccr-based intercomputer communication. I think this is even simpler than dss. A good article to read after you finish the CCR article; ^) xcoappspace

many of these concepts have been popularized by the Erlang language.

Answer 4

Honestly, I would say that there is no difference between the stacks. The task that you will have is to break the work and restore the output of each machine. Microsoft has an HIV research project that does exactly what you want using .NET technology to “split up and conquer” a big computational problem.