Loading last file into S3 bucket using AWS CLI?

Question

I have an S3 bucket containing database backups. I am creating a script that would like to download the latest backup (and, ultimately, restore it somewhere else), but I'm not sure how to proceed only to capture the most recent file from the bucket.

Is it possible to copy only the last file from the s3 recycle bin to the local directory using AWS CLI tools?

Answer 1

This is an approach you can take.

You can list all objects in the bucket using aws s3 ls $BUCKET --recursive :

 $ aws s3 ls $BUCKET --recursive 2015-05-05 15:36:17 4 an_object.txt 2015-06-08 14:14:44 16322599 some/other/object 2015-04-29 12:09:29 32768 yet-another-object.sh 

They are sorted alphabetically by key, but this first column is the last modified time. A quick sort will reorder them by date:

 $ aws s3 ls $BUCKET --recursive | sort 2015-04-29 12:09:29 32768 yet-another-object.sh 2015-05-05 15:36:17 4 an_object.txt 2015-06-08 14:14:44 16322599 some/other/object 

tail -n 1 selects the last row, and awk '{print $4}' fetches the fourth column (object name).

 $ aws s3 ls $BUCKET --recursive | sort | tail -n 1 | awk '{print $4}' some/other/object 

Last but not least, drop this on aws s3 cp to load the object:

 $ KEY=`aws s3 ls $BUCKET --recursive | sort | tail -n 1 | awk '{print $4}'` $ aws s3 cp s3://$BUCKET/$KEY ./latest-object 

Answer 2

And here's a bash script to create based on @ error2007s answer. This script requires your aws profile and bucket name to be variables, and uploads the last object to the ~ / Downloads folder:

 #!/bin/sh PROFILE=your_profile BUCKET=your_bucket OBJECT="$(aws s3 ls --profile $PROFILE $BUCKET --recursive | sort | tail -n 1 | awk '{print $4}')" aws s3 cp s3://$BUCKET/$OBJECT ~/Downloads/$OBJECT --profile $PROFILE 

Answer 3

Above Solutions is Bash if you want to do the same in Powershell to load on Windows using the following script:

 $s3location = 's3://bucket/' $filename=C:\Progra~1\Amazon\AWSCLI\aws.exe s3 ls s3://bucket/PROD_FULL/ -- recursive | sort |select -last 3 $Dpath='I:\Data_S3' foreach($files in $filename) { #$files.ToString() $testpath1 = Split-Path $path -leaf $testpath1 $path=$s3location+$files C:\Progra~1\Amazon\AWSCLI\aws.exe s3 cp $path $Dpath echo(" ***Files Downloaded ***") }