Merge AWS EBS snapshots

Question

I am exploring AWS EBS snapshot policy to minimize the data loss while any failure occurs to the server. I am thinking of an hourly snapshot policy with 7 days of retention. It will serve the purpose of minimizing the data loss but it will flood the AWS snapshot console which may lead to mistakes in future. To prevent this I am exploring a way so the hourly backups can be merged together daily.

Scenario

Hourly snapshot policy with 7 days retention means 24 snapshots daily till the end of the week = 168 snapshots for a server and 1 merged snapshot will be created at the end of the week.

What I am exploring

Hourly snapshot policy with 7 days retention and 1-day merging means it will create the snapshots hourly till the end of the day and then merge them to 1 single snapshot so I will have one snapshot for the day rather than 24.

I explored the AWS documentation but that doesn't help. Any help would be really appreciable.

Answer 1

If you delete any of the snapshots in between you will find that AWS will automatically perform this merge functionality to ensure there is no missing data in between snapshots.

Deleting a snapshot might not reduce your organization's data storage costs. Other snapshots might reference that snapshot's data, and referenced data is always preserved. If you delete a snapshot containing data being used by a later snapshot, costs associated with the referenced data are allocated to the later snapshot.

If you delete any snapshots (including the first) the data will be merged with the next snapshot that was taken.

Therefore you can relax and adjust the policies as required, without the risk of data loss.

More details are available in the how incremental snapshots work documentation.

Answer 2

I like to think of an Amazon EBS Snapshot as consisting of two items:

• Individual backups of each 'block' on the disk
• An 'index' of all the blocks on the disk and where their backup is stored

When an EBS Snapshot is created, a back-up is made of any blocks that are not already backed-up. An index is also made that lists all the blocks in that "backup".

For example, let's say that an EBS Volume has Snapshot #1 and then one block is modified on the disk. If another Snapshot (#2) is created, only one block will be backed-up, but the Snapshot index will point to all the blocks in the backup.

If the Snapshot #1 is then deleted, all the blocks will be retained for Snapshot #2 automatically. Thus, there is no need to "merge" snapshots -- this is all done automatically.

Bottom line: You can delete any snapshots you want. The blocks required to restore all remaining Snapshots will be retained.