How to save apache logs in EC2 when using AWS auto scaling

Question

We created a website using AWS EC2 and autoscaling in a typical LAMP (ubuntu) stack.

Scaling, etc. works well, however, since the instances are "temporary", our apache logs are not saved (since we do not save volumes or instances) after the load jumps.

Is there a “best way / most reliable” way to keep our apache logs for these instances?

One idea was to copy the log files to S3 during shutdown by writing a bash script to execute using the /etc/rc0.d function (by running the script at shutdown).

Answer 1

"Best practice" will be to pool all your logs on the server, allowing you to archive and search for them. You can back up old magazines in S3 and eventually Glacier.

To do all this work, you need to configure apache to write hourly or small logs and write a cronjob to synchronize them with some central places or upload them to S3.

Give up on http://logstash.net/ for an open source aggregation and search solution that you can run on your own instance, and http://loggly.com/ for a non-free, fully hosted solution.

EDIT: My first thought was "not to do this when shutting down." You need to synchronize your log files as often as possible, and as a result (for example, if you make small logs) you get aggregation / search and search for logs "in real time".