Deploying an MPI Application on Windows Azure

Question

There is an existing MPI application written in C (for linux) that I would like to run on Windows Azure. Is it possible?

• If possible, how to start deploying the application?

• Do I need to convert it to Microsoft MPI?

• Is there any kind of Azure service I need to buy for this?

• Do I need to write a managed shell for it to work?

Any suggestions / submissions / links would be very helpful.

PS I'm new to Azure.

Answer 1

This way, you don’t have to do anything too special to run MPI on Azure. Note that the Azure nodes (unlike talking instances of the Amazon cluster) are not actually configured for this well; you don’t need to have a high-speed network, and the nodes will not necessarily be networked next to each other. So tightly coupled code probably doesn't work as well, while many people have successfully used Amazon or other services for this kind of work.

In any case, the "Microsoft" MPI that comes with the HPC server is simply retrained and perhaps slightly modified by MPICH2 ; so this is the standard MPI for the swamp, and this should not cause any problems. Any MPI program that adheres to the MPI standard should be in order.

As for launching one MPI job, this is no more complicated than starting an MPI job for the first time on several (say) desktops; You must ensure that your host file is configured correctly and that MPI is configured to talk to the appropriate IP block. On Technet, they have an example of running the linpack test on Azure hosts (including a link to what you need to make sure your Azure Workers are running an HPC server ). They skip a couple of steps because linpack is a pre-built binary that you can download, but compiling with MPI is pretty simple.

As for deploying the application so that you can run it regularly (or others run it), I will give other people the best way to do it.

Answer 2

Microsoft will soon launch Linux VM support , which will be hosted on Azure . That way, you could create a Linux virtual machine, install and configure the MPI application, and paste it into Azure.

However, you should first think about what you want to achieve. If your goal is to take advantage of some Azure features (e.g. having multiple instances and scaling, for example), you need to increase the application with additional code. Thanks to the open source developer community and the rapidly growing Microsoft open-source support, you can create additional features, for example:

• Node.js runs on Linux or Windows
• Perl , Python , Ruby and / or PHP all run on Linux or Windows
• .NET code running in the Windows role or in Mono in your Linux virtual machine.

If you like to use the configuration of the Azure service the most, dynamically adding / removing worker roles, using azure storage, etc. then you are in luck - all this can be controlled using HTML / JSON REST interfaces that mean everything that can say HTTP, can (with the correct credentials) configure, manage, control your services and save / retrieve data to / from storage.

NTN.

Answer 3

I know this thread is pretty old! But in case someone else ends here:

Azure, meanwhile, is expanding with dedicated high-performance instances that support high-performance inter-instance connections.

So, if you look at the A8 and A9 instances , in addition to the 10Gbps network interface, they have an additional network interface optimized for MPI applications.

32 Gb / s RDMA capability: Provides low latency, high throughput application connectivity between instances in the same service or availability cloud. Reserved for MPI traffic only.

There are also several documentation for customization.