Migration Strategies

Migration Strategies

 

Defining a Strategy

 

When faced with a data migration, a very early question posed is that of “How do we do it?” Too often, the initial response is the answer to not this but another question “How would we like to do it?”

 

There then starts a process where a preferred or going in approach is documented. It is then attempted to be validated, usually by a testing a succession of assumptions and finally rubber stamped by trial and error in an iterative fashion. The hope is that any major change factors are identified as early as possible.

As each assumption is confirmed (or thrown out and replaced with another) the approach gets refined until a final workable model materialises. In many cases, this is not produced until a final Dress Rehearsal - although by this point, the ability to absorb significant change is passed.

 

Rarely does the final migration shape match what was originally intended. The assumptions as laid down initially are often not backed up by serious analysis or experience but more on a misplaced optimism surrounding many aspects of the migration.

 

In fairness, the reason so many migrations are delayed or fail to meet business expectations is more to do with inexperience rather than any lack of managerial competency. Most IT/business people will rarely get involved in a data migrations in a career; those that do will either do the one (and may choose) not to get involved in another. Therefore, it is probably true to say that for most resource involved in a migration, it will be their first and probably last.

 

So what is a data migration strategy? The word strategy has militaristic connotations and is often bandied around during the earlier stages of a project.

In simple terms (and forget data migration) this could be how one gets from A to B or the achievement of a goal. Call it a route, roadmap, plan.  In IT migration terms it could be a document called Strategy, it could also be in a document called a High Level Solution Design, blueprint, approach, scope statement or Conceptual design.  It may even be a requirement or a set of requirements.

 

The point here is that strategy can pretty much be anything that someone wants it to be.  It should be pretty high level in detail but provide evidence that suggests the advocated approach is along the correct lines. It should be subject to change control but impacted only by major change.

The assumptions laid down to underpin a migration strategy should be clearly documented and validated – plus any change requests resulting should be measured against these assumptions to ensure they still hold.

 

Data migration is the culmination of many different activities and technical conversion jobs form only part of the solution, therefore whatever purports to be a strategy must include all activities, business and technical, either precursory to the live event, during or post-Live.

 

It must also fit in with other over-arching activities that may be going on, for example, how one chooses to integrated a system into a business’s IT operations will impact migration of data, therefore it only stands to reason that any strategy must include aspects of that integration, and vice-versa.

 

For the purposes of this document I am going to advocate that strategy is arrived at by defining a position on any activity that needs to occur to enable a data migration.  Identifying the required activities will firm scope and also provide input into lower level documentation. The definition and labelling of activities is also needed and must be clear within any strategy.

 

The output from any strategy document must serve a useful purpose in terms of how it used. Whilst the subject contained within will summarise and provide a shape, it is expected most aspects of it will require to be fleshed out later on in the program.

 

 

 

Documenting a Strategy

 

How a strategy is presented offers many possibilities.

The following documentation types vary from implementation to implementation, often contain similar topics and are normally all pitched at a combined business and technical readership.

By traditional Software Development Lifecycle yardsticks, these are produced very early on in any project.  The danger inherent in all of these is that insufficient detail is understood; this detail can heavily influence whether advocated approaches are feasible and are based around assumptions.

 

 

These assumptions require clear articulation and must be validated as early on as possible.

 

 

The two main initial positions on a migration should come jointly from IT and Business. Who actually produces them is largely irrelevant although it is important that they should compliment and not contradict each other.

These documents are: -

  • Business Requirement Report
  • Migration Strategy

 

Business Requirement Document

 

As a starting point, this document often provides the preferred approach.  By stating scope and the preferred outcome, inferences can be made as to how the design of the migration will be influenced.

In many senses it represents a collection of expectations as envisaged by the business about how they expect the migration to go.  In other ways, it provides clear scope as to what the migration is expected to achieve.

 

Traditionally, business requirement documents which purport to a solution rather than a clear statement of requirement are frowned upon.  Where migration projects are concerned, they should be embraced. As well as engaging the business early on, they allow scope for greater business participation as how the final migration event will be implemented.

 

As a minimum it should state clear positions on: -

 

  • Level of Automation. This will state a requirement to automate the migration of data - rather obvious but it should be stated.

 

  • Big Bang / Phased. If the expected outcome is that all business should be moved over in one go, this should be stated.

 

  • Data Sources.   Main source systems expected to provide the data should be identified.

 

  • State of Data.  Data that is migrated over to a target should be suitable for loading. Data Cleansing should not be done in transformation but tackled directly in the source system.

 

  • Reconciliation / Business Verification.  Provide the criteria against which the business is to sign-off. If there is any complex GL / financial requirements, this should be recorded here. If Tolerance Levels on failed records are known at this stage, they should be stated in absolute terms (rather than percentage).   

 

  • Entry Criterion into the Live Event.  This will dictate the level and extent of Dress Rehearsal activity.

 

  • Historical Data Requirement.  If there are clear requirements known, they should be stated. If not clear, a level of expectation should be set – most commonly that of all data but qualified with a requirement about what must be brought over. .

 

  • Target functionality. Detail what functionality the target system will offer as a result of the migration. It should also state whether any customer or business impacts will result because of the migration. This represents the main scope statement.

 

  • Weekend Implementation. The common expectation is that cut-over will occur over a weekend. This should be stated plus a position of whether a Month End or Mid-Month scenario is out or required.

 

  • Outage. Migrations often require systems to be down, in Read Only mode either in part or in an entirety over a cut-over period.  This should state the minimum level of outage the business expects or can reasonably handle. Boundary Start and End times should be included in this statement.

 

  • Deactivation and Decommissioning. This is primarily relevant to a Big Bang migration. There should be a statement describing what state the source system should be left in. This provides an opportunity to state a requirement that the source system is at least available in Read Only mode for reference purposes in the immediate period following a cut-over.

 

  • Back Out and Recovery. As migrations are high risk activities, statement detailing the expectation in the event of the migration failing following the commencement of migration activities.

 

 

The requirements as captured should be in form that they can be referenced in more detailed subsequent documentation. Whilst this perhaps sounds obvious, it is often left out and results in ambiguities later on.

 

Migration Strategy Document

 

In many respects, this is the IT response to the requirements. It crystallises the topics dealt with in the requirements and offers a position whether they are can be delivered, or part-delivered.

 

Quite adequately delivered in a summarised type presentation, the Strategy will provide the main function points and areas of technical activity associated with the migration.

It will highlight the main assumptions under pinning the strategy plus offering an initial view on risks and confidence levels associated with these assumptions.

 

Baseline - Requirements and Strategy

 

It is expected that Requirements documents and Strategy documents will be produced in conjunction with one another. As requirements are challenged or amended and as strategic options are understood in greater detail it is expected that they will both undergo a level of refinement.

 

However, at some point both must be Baselined. The key aspect about this is to have validated the big ticket assumptions as much as possible.  It is only by clearly understanding the risks associated with them it is that which provides the level of confidence that the strategy is the correct one.

This will go some way to ensure the number and size of iterations associated with the migration is kept to a minimum.    

 

The next level down of documentation takes the Business Requirements and Migration Strategy and offers a view on the way the migration event is likely to be shaped.

A common form of documentation to provide this view is that of: -

 

  • Conceptual Design Document. Normally produced by a Solution Architect at a very early stage. Whilst this is traditionally has a technical focus, it should be extended to cover all activities required in a migration, including precursory tasks (data cleansing) as well as those associated with the Live Event.
  • High Level Solution Design. Again, this normally associated with an Architect. It will detail the main technical components that will be required to support the migration. From this a more detailed Technical Specification document should be drawn up. As such, the main audience will be technical resource although it should not phase business resource.
  • Functional Overview. This document is primarily business focused and would normally be produced by a Senior Business Analyst. It will summarise the main technical aspects but not go into any real detail. The primary aim of this document is to demonstrate that the primary functional requirements are being met. It should endeavour to match data sources to target functionality and summarise aspects of the scope.

 

All three mediums tend to follow broadly similar templates albeit they may be pitched at various technical levels and go to varying degrees of detail.

 

The decision as to what level and detail of documentation is required is highly subjective. Most large blue chip organisations will have existing templates and procedures which must be followed. Within this, there should be scope to adopt and mould these templates in order to satisfactorily document the migration event.

 

As already mentioned, key assumptions and risks must not just be identified but instigate pieces of work which will validate or mitigate each of the assumptions and risk. There is a tendency for optimism to pay scant attention to this. The real danger is that when assumptions are proven to be false or risks become reality, it is relatively late on in the migration lifecycle.

 

 

Common Strategy Game Changers

 

No two migrations are exactly the same - as such, the opportunity for getting things wrong is high. Paradoxically, and rather ironically, the chances of ensuring a successful migration are to leverage as much from previous migrations.

 

The following represents a number of common threads where the initial strategic approach is flawed and a re-think is required. Detailed upfront analysis of these is vital if the number of iterations can be reduced.

 

  • Big Bang is desired where a Phased approach is most suitable.   This can be summarised by Unrealistic Scope and poor understanding of data.

Big Bang migrations are easier and are thus cheaper. They offer the possibility of going to the new target in a single implementation, do not require expensive parallel running of systems and do not require temporary integration work to be done.

As a rule, as the number of major source systems increase, so does the complexity. The fact that everything is now potentially on the critical path now means the whole migration cannot go ahead if there is a problem with one; the solution in such circumstances is to split the migration into phases. The key to understanding whether a migration should be split is often in the detail; at first pass all issues associated with the migration should be known and a path to solution understood.

The main challenge is that management often want irrefutable evidence that the initial approach cannot work; in many circumstances, this evidence will not normally be available until late on in a project. 

The solution therefore is to home in on and validate those assumptions on which a Big Bang approach is based. Like any other IT project, the earlier in the life cycle this is done, the more equipped everyone is to deal with any change.

A common cause of being unable to migrate is data associated with functionality that is not available on the target. For whatever reason, these functions have not been picked up in early Fit-Gap analysis but are deemed as required by the business.

It tends to represent a significant minority e.g. 5-10%, but which volume negates a manual solution.    

 

  • Testing / Dress Rehearsal Environments not suitable. Too often, the environment that is to be used to prove a migration is known before the criterion against which a migration is deemed to be successful is known. The result is that objective of the Dress Rehearsals cannot be met unless the environment is brought up to scratch. This will cause delays; the alternative is to reduce the scope of DR activity although the risk is to reduce the predictability of the migration in the Live Event. Detailed understanding of the risks is required. The solution here is to fully understand the scope and objectives of the Dress Rehearsals; this will feed in directly to any environment provisioning.  

 

  • Change Control procedures not enforced.  This often referred to as scope creep and can cause major problems if ignored.  In essence, it occurs where a succession of small changes occur, are not properly understood and conflict with other aspects of the migration. The ones which cause the real problems are those which invalidate assumptions and those which blow out risks.

 

  • Insufficient Fit-Gap analysis. As already stated, this can cause a Big Bang approach to be changed to a Phased approach and often involves a (significant) minority of the data. The Fit-Gap analysis that is done is often undertaken as part of the package assessment, is too high level in nature and fails to pick up some of the finer differences in Source / Target functionality. The scenario is often exacerbated by vendor optimism in the flexibility of their product coupled with customer ignorance of their own systems.  This is particularly true where the source is Legacy with limited available expertise and documentation.

 

  • Target System Load Timings. This area often causes the customer the most headaches. In many cases where some kind of interface is leveraged, the load performance is poor and compromises the ability of the migration to be done over a single weekend. The vendor is unwilling (or unable) to resort to a direct to table load or undertake parallel type processing.  The strategy may then have to change from a single weekend to a multiple weekend approach, either as a Phased migration or a stepped migration (which can potentially mean loading non-functional historical data off the critical path).

 

 

 

Many larger organisations with mature processes and development methodologies may have tick boxes to be completed. These may be suitable for leveraging and is recommended that they be used but only where they add value. 

 

 

 

 

Chapter Summary

 

To summarise, the main points identified in this section: -

 

  • Strategy and Requirements go together
  • Expect strategy to be subject to a level of iteration
  • Objective is to keep number of iterations to a minimum
  • Key to keeping iterations to a minimum is to have clarity on key Assumptions and Risks that underpin a Strategy
  • Assumptions and Risks should be identified as Strategy is formulated
    • Immediate focus is needed to validate each Assumption or mitigate each Risk
 
 

045 827 4802