How often do you back up your most important data in the cloud? There are many good reasons for having a recent backup: the threat of malware, mistaken deletions, data center failures, the list is ever growing.
If you’re like me, you’re always looking for more ways to protect and secure the data that’s most important to you.
Offering another way to protect and secure data using AWS was one of the main reasons behind EBS Snapper, an open source Python tool we recently made available to help automate, schedule and clean up snapshots.
EC2 Storage Types
When choosing block storage for your Elastic Compute Cloud (EC2) instances on AWS, you’re given two options: instance store or Elastic Block Storage (EBS). Instance store is best used for temporary, ephemeral storage that disappears when an EC2 instance is stopped or terminated. For any critical data, Rackspace recommends using EBS volumes. EBS volumes provide automatic in-zone replication, persistence when instances are stopped, incremental snapshots for backups, and the ability to encrypt data at rest.
You can easily back up the data on your EBS volumes by using the snapshot feature. Snapshots are incremental in nature, they only store changes from the previous snapshot, they allow new snapshots to be created quickly and remain small to save on cost. One feature that’s currently missing is the ability to schedule snapshots of EBS volumes, which is what prompted us to create EBS Snapper
EBS Snapper features include:
- Configurable scheduling based on crontab expressions, such as “0 15 10 ? * MON-FRI” or a simple frequency like “daily”
- Option to retain snapshots if fewer than a certain number remain
- Target specific instances by any EC2 filter, such as instance tags, names, states, or even specific autoscale groups
- Monitor failures using CloudWatch
- Copy tags from EC2 instances and EBS volumes to created snapshots
- Automatic deployment using CloudFormation and a built-in CLI
- Support for AWS Lambda or run manually via CLI
Our approach in designing this new tool was based on a Serverless philosophy, allowing us to focus on building value for customers without worrying about managing cloud servers. We chose the Python programming language because of our strong in-house expertise and great AWS API support with boto3.
CloudFormation allows us to deploy and manage the code and infrastructure easily and in a standard way. We selected DynamoDB for storing backup configurations with high availability and durability. And for our own convenience, we created a command-line utility to deploy the tool to Lambda or perform ad-hoc operations.