Data Migration 101

At a certain stage of its development, almost every company has to go through a data migration process. Despite all the available information, it’s not uncommon to experience hurdles along the way.  Often the cause of these issues is underestimating the project’s complexity, a lack of resources and poor planning. No matter how much time and effort you invest into a new system, it’s vital that you allocate sufficient time and resources for data migration. 

If you’re not yet convinced that having a data migration plan is essential, consider the following consequences:

• Adoption problems – no one wants to use a new system or functionality if it’s set up incorrectly
• Customer Relationship issues – if bad data results in wrong information being sent/exposed to customers, their orders not being completed or issues not being resolved in a timely manner, then the chance of losing them as customers will increase
• Analytics problems – if information is not recorded properly, you cannot account for it

Data Migration Steps

The complexity of a data migration project depends on the source system(s) configuration and state, data quality, existing data processes versus new data processes, configuration of a destination system (Salesforce) and the way it is used at the time of the migration. For example, migrating data from a number of excel spreadsheets is different than migration from an internal legacy system. Migrating data into a system that is currently used and already contains data is more complex than when your Salesforce org is brand new. And when you have a number of systems, some of which will be retired while others integrated, you add another layer of complexity. The logic you use for this type of a project depends on many parameters. But there are some important points you need to pay attention to:

• Identify Stakeholders
• Understand Data
• Prepare Data and a Destination Org
• Complete and Verify a Test Migration
• Migrate and Validate Data

Learn more about these important steps in my blog post: Data Migration Made Easy.

Common Data Migration Issues

Every type of project has common problems that businesses experience over and over again. No legacy system is perfect. If it were perfect, you wouldn’t be looking to replace it. No data managed by human beings is perfect either.

If you research common data migration issues, you will see that poor data quality is often at the top of the list. But it is not the only thing that can cause problems. Here are some other common problems:

• Insufficient tools and human resources
• Failure to translate data into a new structure and format
• Unexpected effect of existing data, validations and processes to newly migrated data
• Unexpected effect of newly migrated data to existing Salesforce data that is already used
• Unforeseen data issues and exceptions
• Access and permissions problems

Data Migration Tools

It is important to select the right tools for your project. These can be native Salesforce solutions, those designed by 3rd party companies or even by your internal developers. You will need to conduct proper testing and investigation before making a decision. There is no perfect solution that will work for every project.

In general, I would try to automate the process of data migration. Developers will be able to help you with this if you provide defined logic and criteria. This usually allows a good percentage of well-structured data to be migrated. There will also be exceptions, for example, data within a certain date frame or data that is not organized in a suitable manner will not necessarily migrate correctly. Dependent on your data quality, the source system and your Salesforce configuration, you may need to do a number of manual data fixes and amendments. This tends to be one of the more time consuming and tedious tasks.

Experience & Advice

No matter how many projects you work on, you will learn something new from each one of them.

My last data migration project involved moving data from legacy systems and integrating some of the internal systems with Salesforce. This meant that the original, source data was structured in different ways. Also, some of the customer data was stored in a variety of places, including the legacy systems, the internal systems and in Salesforce. To prepare the data for migration I needed help from developers, other Salesforce Admins, business stakeholders and the teams that actually work with the data and know it well. During a multi-staged testing process, we changed the exact steps of the migration a number of times. Extensive testing exposed some significant data issues. If we had skipped this step or decided to spend less time on it, the consequences could have been very serious because certain data needed to be exposed to the customers.

A number of teams worked on this task together and, with the help of the right tools and proper planning (together with some good teamwork!), we managed to successfully complete the project.

Having completed a complex data migration project recently, I wanted to share my experience with the community. Find my original blog post and presentation slides here.