Data Migration
What is Data Migration?
When I refer to Data Migration, I am talking about moving large amounts of data from one system into an entirely different system.
Who needs Data Migration?
Manually moving data into a system can be extremely tedious. It may take a minute to manually move a single record from one system to another. For systems with less than 60 records, this would be feasible. As you increase the number of records, you increase the likelihood of mistakes being made during the process. At some point, the number of records and the cost to move those records manually becomes impractical.
It then becomes more feasible to hire a developer to write a process specific to your data import/export needs.
What are some examples of Data Migration?
Common examples of Data Migration include:
- Moving content to a new website
- Importing product information
- Importing client information
- Synchronizing systems
What are the most common challenges with data migration?
To ensure you keep your costs to a minimum it is best to have the data organized, normalized, static and accessible.
Organized:
Let’s say you want to move your client records from one system to another. Having all of this information in one place is really the only practical way to do it. In contrast, if you have multiple copies of your client records list all of which need to be imported, and cross-referenced so that duplicates aren’t created, it makes the job quite a bit more difficult.
Normalized:
When you present your data, consistency goes a long way. For example, if you have product weights, the job becomes a lot more difficult if some weights are in grams, some are in kilograms, some are in ounces and some are in pounds. To make matters worse, you could store the information for a 1 lb product as follows:
- 1lb, 1 lb, 1lb., 1 lb., 1LB, 1 LB, 1LB., 1 LB.
- 1lbs, 1 lbs, 1lbs. 1 lbs. 1LBS, 1 LBS, 1LBS., 1 LBS.
- 0.453592 Kg,
- 453 g
- etc.
Writing code to handle every single possible case and do a conversion would be a lot more costly than if every product had the weight stored in a consistent manner. More to that point, the likelihood that a special case gets missed (I forgot about ounce conversion) is extremely high. This would ultimately lead to data corruption and the code would need to be modified to handle that special case and the products would need to be reimported.
Static:
Let’s say you want product inventory moved into your new system. Only, the inventory changes constantly This challenge will require far more than a single update and thus falls more into the realm of API Integration to keep data up to date through inventory change notifications.
Accessible:
The data can be easy to obtain, or you may have to jump through hoops to get at it. Sometimes legal contracts need to be signed or database purchases need to be made. Sometimes the underlying problem of what information to use needs to be solved before any data migration can begin. Any of these obstacles can hinder the process, delay execution and drive up costs.
