What you will learn in this tip: As your company grows, there's a good chance you'll eventually outgrow your current storage environment. While the original move from direct-attached storage (DAS)
to an array might have been not too difficult, SAN storage data migration projects are much more labor-intensive and complicated. But with proper planning and our data migration checklist, you can ensure your project doesn't take more time and money than planned.
There's a common impression that data migrations are simply moving data from the old storage area network (SAN) storage array to the new. That impression is incorrect. Moving the data is just a very small part of data migration and generally the easiest part. And according to Bloor Research, 84% of SAN storage data migration projects are delivered over-time and/or over-budget. In other words, less than 16% of those projects are delivered on time and on budget.
A SAN storage data migration is a complicated series of manually intensive tasks. In fact, there are 43 discreet tasks; a mere six are semi-automated and 37 are labor-intensive (see "Data migration checklist" below). A task usually includes many steps or processes to complete. Take the task example of application, physical or virtual server data discovery and collection. All of the steps and processes must be repeated for each application, physical server and virtual server. But it is server remediation (making sure it will boot up, is up to date, has the correct drivers, micro codes, storage adapters, works with the new SAN storage, has the correct pathing, etc.) that takes huge swaths of time and effort, as it must be provided for each and every physical and virtual server. That’s a lot of steps for just one of the tasks.
SAN storage data migration checklist
|DATA MIGRATION CHECKLIST|
|1||Identify hosts for migration|
|2||Collect migration host data (host audit)|
|3||Collect array data (array audit)|
|4||Collate zoning and masking info|
|5||Correlate all data|
|6||Migration and destination remediation analysis|
|8||Document source and target array software versions and compatibility|
|9||Configure source and target array to facilitate migration|
|10||Create relationship between source and target array|
|11||Collect all hosts' storage configurations: installed OS, cluster status, DR partner|
|12||Collect each array's configuration|
|RAID, FA port configuration, local/remote replication status|
|13||Collect SAN fabric configuration: directors, switches, gateways|
|Oversubscription details per port—fan-in/out ratios|
|14||Plan storage layout at target arrays|
|15||Create migration configuration and pairing scripts for source and target array|
|16||Create FA port mapping and target array LUN assignment configuration scripts|
|17||Create target array LUN masking scripts|
|18||Create device alias scripts|
|19||Prepare host to target arrays SAN zoning|
|20||Group migration hosts in cooperation w/bus units|
|21||Update all operational backup and recovery scripts|
|22||Perform all hosts audit prior to migration|
|23||Create-run R1 to R2 relationship pairing scripts|
|24||Create-run target array mapping and masking scripts|
|25||Apply all hosts zoning configurations|
|26||Confirm hosts connections at target array|
|27||Commence data migration to new array|
|28||Snap, replicate or backup all old host configurations|
|29||Quiesce (stop) apps, then shut down hosts|
|30||Create-run scripts that split new Array 1 and Array 2 relationship once data has been synched|
|31||Bring hosts back up to check and validate data|
|33||Get bus units to sign off data migration complete|
|34||Check for any activity on old arrays|
|35||Create-run scripts to delete Array1 and Array2 relationship|
|36||Create-run scripts to unmap devices from FA’s on source array|
|37||Create-run scripts to dissolve meta configurations on source arrays|
|38||Create-run scripts to unmask source devices from migrated hosts|
|39||Reboot hosts, confirm new target devices visibility|
|40||Clean up zones and remove all unused zones|
|41||Final login table check on old arrays|
|42||Update device groups if used|
|43||Reports and updates documenting entire process|
Most data migration projects are managed manually usually Microsoft Excel spreadsheets. This approach requires multi-departmental processes, procedures and, most importantly, discipline. This is because the data migration spreadsheet must be continuously updated by multiple administrators.
Data migration technology timesavers
There are four technology workarounds to this manually intensive process and, while each has its trade-offs, all of them can significantly reduce data migration workloads and errors.
1. Write unique scripts for some of the
2. Utilize external SAN storage virtualization appliances or systems;
3. Leverage VMware vSphere Storage vMotion;
4. Implement data migration packaged scripts and services with products like SANpulse.
Scripts are the most common workaround. The good thing about scripting is they are customized for the specific environment, storage, servers, applications, vendors and infrastructure. The bad thing about scripting is that they are rarely documented, tested for quality assurance, patched, or updated as the environment changes; they have to be rewritten for every SAN storage data migration project.
There has been much hope and hype with external SAN storage virtualization appliances and systems to simplify SAN storage data migration. It does this by allowing storage systems behind the virtualization appliance/system to be migrated without touching any of the servers or infrastructure in front of the virtualization appliance/system. The trade-off is that it adds considerable upfront and ongoing costs that increase the total cost of ownership (TCO). There is additional complexity in managing both the appliances/systems and backend storage arrays. Plus, all of the appliances/systems have scalability limitations equal to or a bit more than the storage systems they front end. Some require even greater data migration complexity (both the virtualization appliance/system and the backend storage systems must be migrated) when they go through a technology refresh.
VMware’s Storage vMotion is becoming another popular workaround. It migrates the data from each of the physical SAN storage array LUNs assigned to that VMware server to LUNs on the new SAN storage array. The virtual LUNs assigned to the individual virtual machines (VMs) do not change, masking the actual physical change in the storage arrays. The downside is that this only works for the VMs running in a VMware infrastructure. It also doesn’t provide any SAN pathing remediation or VMware host remediation. Also, it doesn't move data from LUNs that are not assigned to that particular VMware host, and there is no data migration testing or validation.
Another workaround is the software, scripts and services packaged by SANpulse as SANlogics. SANlogics is designed to automate 27 SAN storage data migration processes and semi-automate another two. It can eliminate many human errors from the process; however, it only works on EMC Corp. and Hitachi Data Systems (HDS) storage systems today, and requires a data mover from EMC, HDS, or Symantec Corp.
SAN storage data migration will continue to be a daunting effort. This is why many small- to medium-sized businesses (SMBs) often turn to their SAN storage vendor data migration professional services. Yes, it’s quite expensive. But it gives you a way to hold your storage vendor accountable. Yet even this workaround has issues. It does not automate any of the processes, but instead, it just outsources the responsibility.
Whatever you choose, planning, discipline, focus, and attention to detail are the difference between success and failure when it comes to data migrations.
About the author: Marc Staimer is the founder, senior analyst, and CDS of Dragon Slayer Consulting in Beaverton, Ore. The consulting practice of 13 years has focused in the areas of strategic planning, product development and market development. With more than 31 years of marketing, sales and business experience in infrastructure, storage, server, software, and virtualization, he’s considered one of the industry’s leading experts. Marc can be reached at firstname.lastname@example.org.
This was first published in April 2011