Geek Answers Handbook

Help me copy data through Amazon EC2 and run the script

I'm new to cloud computing, but I understand this concept and I am well versed in the following instructions. I would like to do some simulations according to my data, and each step takes several minutes. Given the hierarchy in my data, it takes several hours for each set. I would like to speed this up by running it on the Amazon EC2 cloud.

After reading this , I know how to start AMI, connect to it through the shell and run R at the command prompt.

I would like to get help in copying data (.rdata files) and a script and just send it to the R command line. Then, as soon as all the results are written to new .rdata files, I would like to copy them back to the local machine.

How should I do it?

+5
source share
2 answers

I don't know much about R, but I do similar things with other languages. What I suggest is likely to give you some ideas.

  • Install the FTP server on the local computer.
  • Create a "startup-script" that you run with your instance.
  • Let the script run download the R files from your local computer, initialize R and perform the calculations, and then download the new files to your computer.

Running script:

#!/bin/bash
set -e -x
apt-get update && apt-get install curl + "any packages you need"
wget ftp://yourlocalmachine:21/r_files > /mnt/data_old.R
R CMD BATCH data_old.R -> /mnt/data_new.R
/usr/bin/curl -T /mnt/data_new.r -u user:pass ftp://yourlocalmachine:21/new_r_files

script

ec2-run-instances --key KEYPAIR --user-data-file my_start_up_script ami-xxxxxx
+3

id amazon S3
,
, AMI
S3, , , / S3
, ( ), S3/

+1

More articles:


All Articles