In this next test of vSphere Data Protection (VDP) interoperability, I wanted to see if a restored vCenter Server appliance would still be able to work with pre-configured vCloud Suite products such as vCenter Operations (vCops), vCloud Automation Center (vCAC), vSphere Orchestrator VCO and Network Virtualization (NSX). All of these products were running to some extent in my environment; vCAC had a simple blueprint for VM deployment, VCO had a simple workflow for renaming a VM and NSX included an Edge device providing a DHCP service. If all of this functionality was still in place post restore, then the backup and restore will have worked. Testing was done with vCenter Server appliance version 5.5U1 and VDP version 126.96.36.199.
Note that this test was working with the vCenter Server appliance, not the Windows version. Therefore the configuration is quite simple as everything is contained in a single VM and the idea would be to use an image based backup of the vCenter server appliance. Obviously vCenter can run in multiple different configurations when the Windows version is used, multi-site SSO for example. These were outside the scope of the testing on this occasion, but I may return to that in a future post.
My Data Center Configuration
The vCenter server appliance that is managing my production environment actually resides on a completely different infrastructure – following guidelines of not having your management tool sitting on the infrastructure it manages. Therefore a VDP instance is deployed on my management (core) infrastructure, connected to my management (core) vCenter, so that I can use it to back up my production vCenter server appliance. My configuration looks something like this:
For those of you who have been following my series on VDP interop, you’ll be familiar with how to back up and restore VM images at this stage. Backing up the vCenter server appliance is no different. I successfully backed up the production vCenter server appliance, and restored it to a different location in my management vCenter inventory. After the restore,I proceeded to power down the original production vCenter server appliance and powered up this newly restored vCenter server.
Observation: The first observation is that it took a very long time for the vCenter appliance to come online after the restore, somewhere in the region of 15-20 minutes. We assume that this is down to the various levels of recovery that need to take place since this backup and restore puts back a crash-consistent copy of the appliance.
Of course during this time, the web client issues various expected errors, such as unable to connect to vCenter Inventory Service, etc. However, the restored vCenter appliance did eventually power on successfully, and we could log into the vCenter appliance and mange inventory, etc, post install.
Once the restored vCenter Server appliance was up and running, I did the following tests to verify vCloud Suite functionality:
- Verify that vCAC could be accessed and deployed a blueprint/VM – success
- Verify that VCO server could be accessed and script/workflow ran – success
- Verify that NSX Manager/Controllers/Edge were online/normal – success
- Deploy a VM on NSX edge network and verify it gets an IP address – success
Therefore restoring the vCenter server appliance to a new location works just fine, and does not impact other suite products that are deployed alongside the vCenter server in the vCloud suite.
In-place restore of vCenter Server appliance
Since there is a dependency on VDP to use vCenter, how do you overcome the “chicken-and-egg” situation whereby a VM (in this case the vCenter appliance) that is being restored in-place by VDP must be powered down before it can be restored?
In our example, we have an “outer-layer” management vCenter server managing the infrastructure on which our production vCenter appliance is deployed. This makes it possible to do an in-place restore since VDP is associated to the management vCenter server and not our production vCenter server. So we can still work with VDP and do a restore when our management vCenter is powered down.
We proceeded to do an in-place restore test, which once again necessitated the powering down of the vCenter server appliance. However, when the restore was complete and the restored production vCenter server appliance was online (which once again took a considerable length of time ~20 minutes), all vCloud suite products (vCops, vCAC, VCO & NSX) were fully functional as highlighted above.
Restoring vCenter when VDP is connected to that vCenter
So its good to know that a vCenter server can be successfully restored. But what about smaller infrastructures that do not have a “outer-layer” management stack and your VDP wishes to restore the vCenter server to which it was connected? Would it still be possible to restore it?
Theoretically, you would think that this should be possible. You could restore the backed up disks to the same vCenter appliance, but you would choose to restore the virtual disks (VMDKs) to different SCSI IDs. Once the disks were restored, you would have to edit the vCenter server configuration, and remove the original SCSI disks, then change the SCSI Ids of the restore disks to match.Unfortunately, even with this method of restore, VDP requires that the VM be powered down. So, although good in theory, not good in practice given the current requirements of VDP.
For this reason, if you wish to back up and restore your vCenter Server appliance, I would make the best practice recommendation that the vCenter appliance that is managing your production infrastructure does not reside on the infrastructure that it is managing. This allows for in-place seamless restore of the vCenter server appliance.
There is still room for improvement when using VDP as a backup product for the vCloud suite. Having said that, we have had remarkable success in using it for backing up and restore the vCloud suite products. We could successfully backup and restore vCenter Operations, vCloud Automation Center, vCenter Orchestration and NSX, our network virtualization product. Notably, as we have just seen, we were also able to back up and restore the vCenter server appliance, and with all these products configured, everything continued to work as before after the restore of vCenter (to a new location) was completed.
This was a test done with the vCenter server appliance, which was successful. We have not yet looked at the Windows version of vCenter server, which is a different kettle of fish entirely.
Question for readers?
How do you backup your vCloud Suite components? What tools/backup products do you use? How do you verify consistency and prove that your backups are valid. I’d really like to hear from you, so please leave a comment if you can.