Unlike a normal project where you are delivering a technical solution to solve a problem that has a life of years or even decades, a migration can be a one-off event. As the migration project forms there are a lot of interest in understanding what will happen “On the Day”. A migration is usually a critical event that will impact a large portion of the organisation including people involved with the old systems as well as the new system being migrated to.
One of the key questions that we needed to answer has to do with the expected duration of the migration. Although the actual amount of time that is needed for the migration is hard to define at this level, we can identify what dependencies drive when things can occur. The first key point is to understand when we could start the migration, when does the business want to take the current systems offline or in what state do the new systems need to be in to receive the data. Is there some type of End of Day activity that has to occur before we can start the migration. Once you have figured out when you can start the migration there is a standard set of technical steps involving Extracting Transforming and then Loading the data.
Then there is a step where we need to reconcile the data that has been loaded to confirm that everything occurred as it should have. This is a complex task to define early in the project because the business will find it hard to define what they need to reconcile. The first stab will be to reconcile basically every field or pretty close to. It really isn’t until you are able to perform your first trial migration that you can narrow down what needs to be reconciled and at what level. I think there is an aspect of trusting something that hasn’t been built. It isn’t until you do a migration and there is a report that says I migrated 100,000 orders and there are 100,000 orders in then new systems that you can delegate that responsibility to a piece of software.
Reconciliation will have a large bearing on how long the running of the migration on the day will take. The first trial migration you perform the reconciliation step will take a long time. There will be no pressure time wise as it will be a trial and there will be a natural tendency to be ultra cautious. Once you have completed this initial trial migration it could be a good check-point to review the requirements for reconciliation. I know a waterfall methodology wouldn’t cope with this approach but I think this is critical to ensuring the reconciliation is successful. It will also allow constructive conversations around the amount of information that needs to be visible versus presenting the information in summary to make the reconciliation more efficient. At the end of this review you should be able to come up with a straw-man schedule for the reconciliation and planned duration.
Laying out the migration activity you will have at a high level the following activities:
At some stage through this process you will have to identify what we referred to as “The Point of No Return”. This was a point in the migration past which you could not abort the migration and just re-enable the old system. The architecture of the migration to a large degree will influence when this point is in the migration and the goal will be to make it as late as possible. In conjunction with the Point of No Return there will need to be a business Go or No Go decision on the migration. The system constraints that will influence this will be unique for each migration, so will vary greatly. The key is to be aware that this point will exist and to identify and communicate it early.
What will happen “On the Day” is important but also is something that many people will influence so will evolve. I have outlined a high level process above, use it or create your own and progressively define the details of each of these steps as you go. The duration of the migration will be a key question as discussed in Migration is just ETL you need to identify points in the architecture where you can tune for performance.