As part of a quick reference proof-of-concept/evaluation guide that I have been working on, it has become very clear that one of the areas that causes the most confusion is what happens when a storage device is either manually removed from a host participating in the Virtual SAN cluster or the device suffers a failure. These are not the same thing from a Virtual SAN perspective.
To explain the different behaviour, it is important to understand that Virtual SAN has 2 types of failure states for components: ABSENT and DEGRADED.
There was a very interesting discussion on our internal forums here at VMware over the past week. One of our guys had built out a VSAN cluster, and everything looked good. However on attempting to deploy a virtual machine on the VSAN datastore, he kept hitting an error which reported that it “cannot complete file creation operation”. As I said, everything looked healthy. The cluster formed correctly, there were no network partitions and the network status was normal. So what could be the problem?
Over the past month or so, I’ve been looking at disaster recovery of some of the vCloud Suite components. My experiences of using vSphere Replication and Site Recovery Manager to protect and recover vCenter Operations Manager in the event of a disaster can be found here and here. Now it was time to look at vCenter Orchestrator (vCO) to see if that could be protected and recovered.
In this configuration, I deployed vCO in HA mode, meaning that there were two vCenter Orchestrator servers, one running and one in standby mode. The database for vCO was an external SQL Server database, running in its own VM. So there were three VMs to protect in this setup.
I have been doing a bunch of stuff around disaster recovery (DR) recently, and my storage of choice at both the production site and the recovery site has been VSAN, VMware Virtual SAN. I have already done a number of tests already with products like vCenter Server, vCenter Operations Manager and NSX, our network virtualization product. Next up was VCO, our vCenter Orchestrator product. I set up vSphere Replication for my vCO servers (I deployed them in a HA configuration) and their associated SQL DB VM on Friday, but when I got in Monday morning, I could not log onto my vCenter. The problem was that my vCenter was running on VSAN (a bit of a chicken and egg type situation), so how do I troubleshoot this situation without my vCenter. And what was the actual problem? Was it a VSAN issue? This is what had to be done to resolve it.
I’ve been working on some Disaster Recovery (DR) scenarios recently with my good pal Paudie. Last month we looked at how we might be able to protect vCenter Operation Manager, by using a vApp construct and also using IP customization. After VMworld, we turned our attention to NSX, and how we might be able to implement a DR solution for NSX. This is still a work in progress, but we did learn some very useful NSX troubleshooting commands that I thought would be worth sharing with you.
A short note to clarify something that has come up a number of times in recent weeks here at VMware. There have been a number of discussions about whether or not we support NFS over IPv6 on vSphere 5.x, and again, on whether or not we support the VAAI-NAS primitives in the same context.
VAAI is an API for offloading tasks to the storage array, but for offloading tasks to NAS arrays, storage vendors need to create their own plugins for the ESXi hosts to achieve this. You can learn more about VAAI-NAS by clicking here. So what about IPv6 support and NFS? And VAAI-NAS?
I guess the next big tech preview at this year’s VMworld was around Virtual Volumes. Yes, we’ve done this before, but this year there were so many vendors showing demos of their VVol implementation, and so many presentations/sessions on the topic that I believe folks are beginning to realize that we are very close indeed to finally having this feature ready. It’s hard to believe that this was first discussed at VMworld 2011, and I alluded to this when I presented a VVol session that I co-delivered with the folks from Nimble Storage at this year’s VMworld.
This topic is going to be huge, and it is going to change the way storage is designed for virtualization environments. And I think almost every storage partner I spoke to at the show is on-board. In fact, “The Register” asked the question last month if there was ANY storage vendor that wasn’t doing a VVol implementation?
I’m not going to get into any technical details of Virtual Volumes in this post – there will be lots of time for this later. Instead, I’m going to share how we believe VVols will now enable Software Defined Storage (SDS) for SAN & NAS arrays, similar to how we have achieved this already with VSAN.