AWS CLI Windows Batch for daily EBS snapshot

Question

TL;TR

Is aws describe-snapshots return list of snapshot ordered from the newest from the oldest?

In details:

I want make daily EBS snapshot with AWS CLI for Windows.

My script performs this operations:

Create the snapshot for specific volume;
Tag the snapshot;
Storing only a max quantity of snapshot with same tag and delete the others.

The problem is the point 3. When i get the list of snapshot, i need to order the list from the oldest to the newest and delete the oldest snapshots.

The solution i found is simple, I take the list of snapshot with the same tag:

aws ec2 describe-snapshots --region %AWS_REGION% --output=text --filters "Name=volume-id,Values=%AWS_VOLUME_ID%" "Name=tag:CreatedBy,Values=%TAGVALUE%"  --query Snapshots[].SnapshotId>%OUTSNAPSHOTS%

now, assuming the list is ordered from the newest to the oldest snapshot, loop on the list in the reverse order (from oldest to newest) with sort /r %OUTSNAPSHOTS%, skip some with skip=%AWS_MAX_BACKUPS%, and delete the snapshot:

for /f "tokens=2 skip=%AWS_MAX_BACKUPS%" %%s in ('sort /r %OUTSNAPSHOTS%') do (
    aws ec2 delete-snapshot --region %AWS_REGION% --snapshot-id %%s
)

Is my assumption correct? Is describe-snapshots return list of snapshot, ordered from the newest from the oldest?

Full code:

@echo OFF

:: Configuration Begin
SET AWS_BACKUP_NAME=Foo
SET AWS_MAX_BACKUPS=5
SET AWS_ACCESS_KEY_ID=XXXXXXXXXXXXXXXXXXXXXXXXXXXXX
SET AWS_SECRET_ACCESS_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXX
SET AWS_REGION=eu-west-1
SET AWS_VOLUME_ID=id-XXXXXXXXXXXXXX
:: Configuration End

:: Current path
SET CURRPATH=%cd%\

:: Get the current date/time according to os
set X=
for /f "skip=1 delims=" %%x in ('wmic os get localdatetime') do if not defined X set X=%%x

:: Set date parts
set DATE_YEAR=%X:~0,4%
set DATE_MONTH=%X:~4,2%
set DATE_DAY=%X:~6,2%
set DATE_HOUR=%X:~8,2%
set DATE_MINUTE=%X:~10,2%
set DATE_SECOND=%X:~12,2%
set DATE_FRACTIONS=%X:~15,6%
set DATE_OFFSET=%X:~21,4%

:: Set the snapshot description with AWS_BACKUP_NAME and current date/time
set SNAPSHOT_DESCRIPTION=%AWS_BACKUP_NAME%-Backup-%DATE_YEAR%-%DATE_MONTH%-%DATE_DAY%_%DATE_HOUR%-%DATE_MINUTE%-%DATE_SECOND%-%DATE_FRACTIONS%

:: Set a file for the aws-cli output
set OUTCREATESNAPSHOT="%CURRPATH%%SNAPSHOT_DESCRIPTION%.txt"

:: 1. Create the snapshot
aws ec2 create-snapshot --region %AWS_REGION% --output=text --description %SNAPSHOT_DESCRIPTION% --volume-id %AWS_VOLUME_ID% --query SnapshotId>%OUTCREATESNAPSHOT%

:: Get the snapshot id from output file
set /p SNAPSHOTID=<%OUTCREATESNAPSHOT%

:: Set the snapshot with a tag
set TAGVALUE=AutomatedBackup%AWS_BACKUP_NAME%

:: 2. Add the tag on the snapshot
aws ec2 create-tags --region %AWS_REGION% --resource %SNAPSHOTID% --tags Key=CreatedBy,Value=%TAGVALUE%

:: Set a file for the aws-cli output
set OUTSNAPSHOTS="%CURRPATH%%AWS_VOLUME_ID%_SNAPSHOTS.txt"

:: Get the snapshots with the current tag
aws ec2 describe-snapshots --region %AWS_REGION% --output=text --filters "Name=volume-id,Values=%AWS_VOLUME_ID%" "Name=tag:CreatedBy,Values=%TAGVALUE%"  --query Snapshots[].SnapshotId>%OUTSNAPSHOTS%

:: 3. Loop on the reverses list but skip the first AWS_MAX_BACKUPS and delete the others
for /f "tokens=2 skip=%AWS_MAX_BACKUPS%" %%s in ('sort /r %OUTSNAPSHOTS%') do (
    aws ec2 delete-snapshot --region %AWS_REGION% --snapshot-id %%s
)

The describe-snapshots command is not documented by AWS as always returning snapshots ordered by newest to oldest so I would be hesitant to rely on the assumption that it always will. — jzonthemtn

Frederic Henri Frederic Henri · Accepted Answer · 2016-10-13T20:24:21

You have no guarantee that the snapshot will be sorted by date.

You could achieve that by adding jq lib in your command

aws ec2 describe-snapshots --region %AWS_REGION% \
      --filters "Name=volume-id,Values=%AWS_VOLUME_ID%" "Name=tag:CreatedBy,Values=%TAGVALUE%" \
| jq -r '.Snapshots | sort_by(.StartTime)[] | .SnapshotId' > %OUTSNAPSHOTS%

I removed the query parameter and the output as text as I'll just push the whole JSon as jq parameter. jq will filter and sort by date, will take the SnapshotId value and push it as text for the input for the rest of your script.

The list will be ordered to have newest snapshots at the end of the list, and you can be sure it will always be this way.

AWS CLI Windows Batch for daily EBS snapshot

1 Answers