Take Scheduled EBS Snapshots using Cloud Watch and Cloud Formation

3
votes

The task seems to be simple: I want to take scheduled EBS snapshots of my EBS volumes on a daily basis. According to the documentation, CloudWatch seems to be the right place to do that: http://docs.aws.amazon.com/AmazonCloudWatch/latest/events/TakeScheduledSnapshot.html

Now I want to create such a scheduled rule when launching a new stack with CloudFormation. For this, there is a new resource type AWS::Events::Rule: http://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-events-rule.html

But now comes the tricky part: How can I use this resource type to create a built-in target that creates my EBS snapshot, like described in the above scenario?

I'm pretty sure there is way to do it, but I can't help myself right now. My resource template looks like this at the moment:

"DailyEbsSnapshotRule": {
      "Type": "AWS::Events::Rule",
      "Properties": {
        "Description": "creates a daily snapshot of EBS volume (8 a.m.)",
        "ScheduleExpression": "cron(0 8 * * ? *)",
        "State": "ENABLED",
        "Targets": [{
          "Arn": { "Fn::Join": [ "", "arn:aws:ec2:", { "Ref": "AWS::Region" }, ":", { "Ref": "AWS::AccountId" }, ":volume/", { "Ref": "EbsVolume" } ] },
          "Id": "SomeId1"
        }]
      }
    }

Any ideas?

4
amazon-s3amazon-ec2amazon-cloudformationamazon-cloudwatch
i would have a script (in python using botocore) run by cron in the instance that has the volume attached so it can pause the database (or whatever is using the volume), unmount the volume, do sync, wait several seconds, start the snapshot, verify the snapshot state is either pending or available (done), remount the volume to the original mount point in its original state (options), and resume the database (or whatever is using the volume). – Skaperen
Thanks for your feedback. Writing my own scripts is exactly what I'm trying to avoid. But maybe I will end up doing so :/ – tbmsu

4 Answers

6
votes

I found the solution to this over on a question about how to do this in terraform. The solution in plain CloudFormation JSON seems to be.

"EBSVolume": {
    "Type": "AWS::EC2::Volume",
    "Properties": {
        "Size": 1
    }
},
"EBSSnapshotRole": {
    "Type": "AWS::IAM::Role",
    "Properties": {
        "AssumeRolePolicyDocument": {
            "Version" : "2012-10-17",
            "Statement": [ {
                "Effect": "Allow",
                "Principal": {
                    "Service": ["events.amazonaws.com", "ec2.amazonaws.com"]
                },
                "Action": ["sts:AssumeRole"]
            }]
        },
        "Path": "/",
        "Policies": [{
            "PolicyName": "root",
            "PolicyDocument": {
                "Version" : "2012-10-17",
                "Statement": [ {
                    "Effect": "Allow",
                    "Action": [
                        "ec2:CreateSnapshot"
                    ],
                   "Resource": "*"
                } ]
            }
        }]
    }
},
"EBSSnapshotRule": {
    "Type": "AWS::Events::Rule",
    "Properties": {
        "Description": "creates a daily snapshot of EBS volume (1 a.m.)",
        "ScheduleExpression": "cron(0 1 * * ? *)",
        "State": "ENABLED",
        "Name": {"Ref": "AWS::StackName"},
        "RoleArn": {"Fn::GetAtt" : ["EBSSnapshotRole", "Arn"]},
        "Targets": [{
            "Arn": {
                "Fn::Join": [
                    "",
                    [
                        "arn:aws:automation:",
                        {"Ref": "AWS::Region"},
                        ":",
                        {"Ref": "AWS::AccountId"},
                        ":action/",
                        "EBSCreateSnapshot/EBSCreateSnapshot_",
                        {"Ref": "AWS::StackName"}
                    ]
                ]
            },
            "Input": {
                "Fn::Join": [
                    "",
                    [
                        "\"arn:aws:ec2:",
                        {"Ref": "AWS::Region"},
                        ":",
                        {"Ref": "AWS::AccountId"},
                        ":volume/",
                        {"Ref": "EBSVolume"},
                        "\""
                    ]
                ]
            },
           "Id": "EBSVolume"
        }]
    }
}
1
votes

Unfortunately, it is not (yet) possible to set up scheduled EBS snapshots via CloudWatch Events within a CloudFormation stack.

It is a bit hidden in the docs: http://docs.aws.amazon.com/AmazonCloudWatchEvents/latest/APIReference/API_PutTargets.html

Note that creating rules with built-in targets is supported only in the AWS Management Console.

And "EBSCreateSnapshot" is one of these so-called "built-in targets".

0
votes

Amazon seem to have removed their "built-in" targets and now its become possible to create Cloudwatch rules to schedule EBS snapshots.

First you must create a rule, which will be used to attach targets to. Replace XXXXXXXXXXXXX with your aws account-id

aws events put-rule \
--name create-disk-snapshot-for-ec2-instance \
--schedule-expression 'rate(1 day)' \
--description "Create EBS snapshot" \
--role-arn arn:aws:iam::XXXXXXXXXXXXX:role/AWS_Events_Actions_Execution

Then you simply add your targets (up to 10 targets allowed per rule).

 aws events put-targets \
 --rule create-disk-snapshot-for-ec2-instance \
 --targets "[{ \
     \"Arn\": \"arn:aws:automation:eu-central-1:XXXXXXXXXXXXX:action/EBSCreateSnapshot/EBSCreateSnapshot_mgmt-disk-snapshots\", \
     \"Id\": \"xxxx-yyyyy-zzzzz-rrrrr-tttttt\", \
     \"Input\": \"\\\"arn:aws:ec2:eu-central-1:XXXXXXXXXXXXX:volume/<VolumeId>\\\"\" \}]"
0
votes

There's a better way to automate EBS snapshots these days, using DLM (Data Lifecycle Manager). It's also available through Cloudformation. See these for details: