Data migration is now an integral part of data management, with more than 91 percent of today’s businesses undertaking migration projects. But that doesn’t mean it’s a low-risk proposition: 85 percent of those businesses experience significant problems during the migration process.
Fortunately, you can greatly reduce your risk by following established best practices and choosing the migration path best suited to your data profile. At Rackspace, we recommend taking these four steps to ensure a seamless data-centric migration.
Before you begin your migration, collect information that can shed light on your unique challenges and requirements. Try to answer these questions:
- What is your estimated database size?
- Are you read-heavy or write-heavy?
- What are your point points?
- What does your current downtime look like?
- Are you set up to observe downtime if you have it?
- During your migration, how much downtime is acceptable?
- How much risk is acceptable?
If you’re working with a provider, ask them to run commands on your databases that will help optimize your target database configuration.
Tap into expertise
Seek out experts who have experience with the kind of migration you’re planning. Data migrations have many objectives: leveraging new technology, boosting performance, increasing reliability, decreasing costs, etc. Whatever your goals, find specialists who have been there before and lean on their expertise early in the process.
For example, say you’re using MySQL and you want to do full-text searches and analytics on large amounts of data. You think MongoDB is the best choice. An expert would quickly point out that Elasticsearch is a better fit given your requirements. You obviously want to know this as early as possible to avoid missteps and reversals.
Or perhaps you’re locked into a proprietary platform that limits your ability to extend a replica set. That will make it hard to use cluster extensions during your migration. You want to work with an expert who can help you evaluate possible solutions.
Create a migration plan
A strong database migration plan has six elements. Carefully thinking through each one will give you the best chance for a smooth, zero-downtime migration.
- Data preparation: Ensure that data is compatible with the target database version — or, if you’re using an extract, transform and load (ETL) process, ensure that data is transformed correctly to fit the new technology.
- Migration: Map the source and target details and set up the environments on both sides to enable replication.
- Testing: Make sure that data is replicated/loaded into the new environment and applications interact with it as expected.
- Recovery plan: If you experience issues during or after the migration, revert to the source so you can make adjustments on the target side.
- Switchover plan: Establish actions required to transition the application to the new platform.
- Verification: Confirm that any web pages, applications or APIs that access the database are responding correctly.
As you complete your plan, you should be able to identify areas of particular risk. If you decide to hire a network security team, for example, you might realize that a team member could accidentally reverse some of your changes. You would then contact the team pre-migration, remain alert to changes, and be ready to roll back at the first hint of trouble.
Choose the right migration path
Data-centric migrations typically take one of three paths, and choosing the right one is critical to a successful migration. Let’s take a look at the benefits and drawbacks of each.
Backup and restore
This is the simplest and most reliable migration method, but it requires constant supervision and substantial downtime — up to several hours if you have a large data set. The key to success is understanding data flow, because you’ll need to stop application writes in order to transfer data. Once you’ve confirmed that the transfer is complete and accurate, you can connect your application to your new instance and pick up where you left off. During the migration, you’ll need to keep an eye out for errors and correct them immediately. Afterward, you’ll need to verify counts and ensure data consistency between the target and source sides.
Cluster extensions and replication
This method requires the least downtime and little to no supervision, but it’s quite complex. It involves setting up replication from one replica set to additional nodes on the destination side, replicating the data over and then switching to the destination side. You shouldn’t experience more than a few minutes of downtime, since your application can keep running with one very small hiccup, but you will have to complete these tasks:
- Open up networking end-to-end — without introducing security risks.
- Apply binary logs to the slave while ensuring it remains in sync (in MySQL).
- Ensure the key file is matched on either side (it allows different nodes to authenticate to each other).
Once you’re done, however, you can let the database internals do the work for you — no monitoring required. That advantage and minimal downtime make this the most popular migration path for businesses.
The custom code method involves variable downtime, variable complexity and limited visibility. It can be approached in different ways. You could use it to enable your application to handle dual rights from your current location into a new location, for example, or to take a snapshot of your data, transfer it, and then have the application log settle deltas for you. Custom code methods can deliver rapid migration and extremely low downtime, but they will be complex, and complexity can complicate visibility. (You could use monitoring platforms to instrument your migration code, but that’s overkill for most migrations.)
The managed migration advantage
Whenever you’re looking to migrate data, it’s smart to have a team of experts at your side. Rackspace has successfully managed cloud data migrations for thousands of businesses, and we can help with every stage of your journey, from information-gathering and planning to selecting the appropriate migration method. And we’re certified experts in all the leading clouds, including AWS, Azure, Google Cloud Platform, VMware and OpenStack.
Learn more about Rackspace public cloud migration services.