In the modern data center the storage layer is one of the most critical layers you rely on. If you’re an ISP, ISV, CSP or an ESP you have been bestowed with the utmost responsibility of owning the storage layer from the operations and agreed upon service perspective. With the advent of modern virtualization technology and hypervisors moving forward as fast as it is, storage has become even more critical from the performance, operations and service agreement levels. Initial storage configuration can be complex when using large SAN storage, but migrating to new SAN hardware can be almost as complex.

One of the most critical operational risks that not everyone likes to talk about today is the migration of storage data from one device to another storage device. While the major hypervisor vendors (Microsoft, VMware for example) have constructs and application level services built into their products to migrate live VM data what happens when you need to decommission a storage device? Will server, or server OS vendors, be able to help you there? Moving small chunks of data can be done using OS tools, but this requires careful planning and operation.  But if it’s a large SAN device that is end of life or a unit failing to be replaced, the entire replacement of a storage system is an extremely critical operation.

We recently had an EMC VNX 8000 series unit supporting a 5,000 user configuration that we needed to swap out the entire EMC storage with updated storage.  Being a lab our reason for the trade out was different than end of life or failing hardware, but we still faced the issue of not wanting to lose any data because we were in the middle of a very large scale test, it would take months to rebuild the entire environment, and time was of the essence.  To accomplish this migration we leveraged our Services agreement and EMC services to do a data migration.  We were quite pleased with the final results. Overall we migrated 31 LUNs of critical data adding up to 51 terabytes of VDI storage resources from the LUN perspective. I prefer to mention LUNs perspective as when your operation is migrating LUNs, you migrate the complete LUN regardless of its percentage of consumed written data. That said the total data transfer time of the migration operation was about 16 hours, making total downtime 2 business work days in our solutions environment. The operation was committed using built-in migration/replication features. For this migration there was only block level data served via the iSCSI storage protocol and no file level storage involved. Details as these lead the storage team to use the MirrorView feature built into VNX for data mirroring and replication.

In preparation of the migration, the storage team set up and configured EMC MirrorView replication technology feature between the two arrays and utilized the Unisphere GUI to create a LUN mirror session for each migrated LUN.  Since the test team could afford some downtime, they were asked to shut down their environment one afternoon.  After all I/O’s were quiesced, the LUN mirror sessions were fired off at maximum speed (session throttle set to 10).  In the next morning, when we checked that all sessions completed successfully, we de-commissioned the IPs that were associated with the iSCSI targets on the original VNX, and assigned the same IPs to the new VNX.

Now came the moment of truth as the test team began to power on their environment post data replication operation.  As far as details around configuring the Microsoft iSCSI initiators, there was no need to change the target portal discovery because the iSCSI server IPs on the new VNX remained unchanged from the old one.  What needed to be done to complete the migration was to simply connect to the new iSCSI target names because each VNX has its own unique IQNs. Then Voilà! All the LUNs were brought back online just the way they were the day before prior to the migration.

As a postmortem of the migration process, the overall related downtime could have been further reduced had the storage team started incremental mirror sessions with a low priority or throttle rate setting in the background, then when the time came for migration only a partial data sync would have been required to push the deltas to the target storage system.  In the end the team was pleased with the smooth transition and continued to coast along with their Citrix Solutions testing programs. You can learn more about using and configuring EMC Mirrorview at this web location.

Seth Roth

Sr. Operations Manager

Citrix Solutions Lab