Data migration – the most underestimated moment in a system change

When an old system is replaced by a new one, attention almost always goes to the new one: the features, the interface, the possibilities. But the step that most often blows the timeline is a different one entirely – moving the existing data along with it. Data migration is treated as a technical detail at the end of the project, and it repeatedly turns out to be the hardest part of all. Here’s why it’s underestimated and how to plan it as the workstream of its own that it deserves to be.

Why migration blows timelines

The explanation isn’t the move itself, which is routine technical work, but the state of the data. Old data has almost always accumulated problems over the years: duplicate customer records, empty fields that should have been filled in, the same value in different formats in different places, and terms that mean one thing in one department and another in the next.

Moving clean, well-organized data is easy. Cleaning, interpreting, and making sense of the mess first is what drags things out – and that part doesn’t show up in the original plan, because nobody knew how bad it was until they started looking. The underestimate is therefore built in: you budget for the move but not for the cleanup, and the cleanup is the heavy part.

Data cleanup and duplicates are the business’s decision

The most common and costly mistake is treating data migration as a purely technical question. But the decisions that matter can’t be made by technology.

Are these two customer records the same person or two different ones? If a customer has two conflicting addresses, which one applies? Should an old category be merged with a new one? Only someone who knows the business can answer questions like these. Developers can execute – merge, transform, move – but they can’t decide what the data means.

So let the business side own the decisions on cleanup and duplicates, with technology as the tool. In practice, that means setting aside time from people who know the business, not just from developers. That time is easy to forget in planning and impossible to do without in execution.

Trial migrations with verification

No live data migration should happen without first running the process in a test environment and reviewing the result. A trial migration reveals what would otherwise only be discovered in production, when it’s most expensive to fix.

Verification matters at least as much as the run itself. The migration “completing” doesn’t mean the data came out right – records can have been quietly dropped or corrupted. So spot-check against the source data and have the business review the result before going live. Catching an error in a trial run is cheap. Catching it once customers are already using the new system can be a crisis.

Scope of history: not everything needs to move

A silent assumption is that all old data must come along. That makes the migration unnecessarily heavy and drags old mess into the new system.

Much history is rarely needed actively. Orders from ten years ago, closed cases, inactive customers – such things can often be archived separately instead of moved into the new system, where they’d just sit and weigh it down. Deliberately decide what scope is actually needed going forward: what do users need to access daily, what’s enough to be able to look up on demand, and what can be kept in an archive outside the system?

A narrower, cleaned data set moving in makes both the migration faster and the new system easier to live with.

A scenario: the project that planned for the data

A company switching customer systems did the opposite of the usual. They treated the data migration as its own workstream from the start: the business was tasked with going through and deciding on duplicates, they ran three trial migrations with verification, and they decided that only the last few years of active data would move in, while older data was archived.

The migration ended up being the project’s least dramatic part – precisely because it got the attention it’s usually denied. A comparable company that left the data until the last week instead pushed the launch back twice when the mess came to light. The difference wasn’t luck, but planning.

Switching systems and want to avoid the data being what trips up the timeline? We at Weapp are happy to help plan the migration as the workstream of its own it should be – get in touch with a description of what you’ll be moving.