AWS EC2 - Syncing source code files with S3 - is this the right approach?

Question

On an application server in which multiple source files change frequently, is the following approach recommended?

Use the cron job with S3tools to synchronize source files with a private S3 bucket (for example, every 15 minutes).

When starting the server - use user script data to synchronize with a bucket of sources to get the latest sources.

Advantages: 1. No need to attach EBS to the application server to save multiple files 2. A similar setting for all application servers. 3. Sources are automatically backed up. 4. As a by-product, code is automatically distributed to several application servers.

Disadvantages: keeping source code on S3 different?

What do you think of this methodology? Is it right to use EC2 with frequent changes to the source code (several times a day), please recommend the best approach for running EC2 instances where sources change frequently.

Answer 1

I think you'd better use the right source code repository like Subversion or Git, rather than store the source files in S3. This way you can have a central location of the source files, avoiding the consistency issues of updates mentioned in kdgregory.

You can put the source repository on one of your own servers outside of EC2 or place it on an EC2 instance (make sure that the repository files are on the EBS volume in the latter case).

Answer 2

If you are going to run a large number of EC2 instances, then there will be less effort to force them to synchronize themselves from a central location (i.e. synchronize with a private bucket, synchronize server applications from this bucket).

HOWEVER, find out that updates to the S3 bucket are atomic only at the object level and, more importantly, are not guaranteed to be immediately consistent (although I remember seeing a recent note that the us-west endpoint offers read-write-consistency )

This means that your server applications can upload a set of new files that are internally inconsistent - some of them will be old and some will be new. If this is a problem for you, you should implement a scheme that is downloaded directly to the application server and ensures consistency of changes (perhaps by downloading to a temporary directory, which is then renamed).