Getting VM EVC Mode Requirements via PowerCLI
One of my customers was preparing to do some major ESXi host reconfiguration and so needed to shift VM workload from one cluster to another. They had a challenge in that their clusters were running with different EVC modes, and they wanted to move VMs from the newer cluster to the older cluster. "Impossible!" the strawman says, "it can't be done!"
Well, yes and no. That's absolutely correct that you can't vMotion a VM that powered up on an Ivy Bridge CPU back onto an ESXi host with a Sandy Bridge processor. The reason for this is that the VM, during its power on operation, scans the CPU of its host for a list of CPU features that are available and begins potentially using those features, which means that it can't be moved to a processor that doesn't have those features. The VM, in effect, inherits its host's EVC Mode for the lifespan of this power cycle. Until the VM goes through a complete new power cycle (not a reboot from within the guest OS), it will maintain that EVC Mode. This means that any VMs that were originally powered on on older CPUs will maintain that older EVC Mode, even though they're running on an Ivy Bridge capable cluster!
You can actually see this through the vSphere client. If you select your cluster, then go to the Virtual Machines tab, you can add the "EVC Mode" column (it's about 3/4 of the way down the list), which will list the runtime EVC Mode for each VM. All that we had to do was select any VMs that were still running with an older mode (we literally had hundreds) and move them into the older cluster.
Well, it wasn't quite that easy. The customer wanted a report of which VMs were candidates to move, so that they could be selective regarding which VMs were migrated to the older cluster. Easy enough, right? You can export a list as a CSV from the vSphere client, and you're good to go... usually.
I'm not sure why, but the CSV export blew up. It had a bunch of lines that were just text; looking back, it was probably just due to carriage returns in the "notes" field of some VMs. Regardless, it inspired me to pull that data via PowerCLI rather than using the retired vSphere client.
So, I did some quick googling and found that there's lots of information out there about how to pull the EVC Mode of a cluster (Get-Cluster 'MyCluster' | select Name,EVCMode). There's lots of information about how to pull the EVC Mode of an ESXi host (Get-VMHost 'MyHost' | select Name,MaxEVCMode). Not so much information about how to get the current EVC Mode requirement for a particular VM or set of VMs. I figured that it must be in there somewhere, but I had no idea where. So, I cheated.
I used get-vm to grab a running VM, then piped that to a file via Export-CLIXML -depth 8. I chose to use Export-CLIXML because it serializes all of the properties and subproperties of the object, up to the specified depth. I chose 8 because I know that VMware loves to embed references to objects within objects within references to the original object and I didn't want my file to get too crazy. As it was, I ended up with a 3 MB XML file for this single VM.
Once I had that XML file, I searched it for EVC. The first couple of instances weren't very promising (one was buried under the VMHost and the second didn't actually specify the mode), but the third one (around line 3400) looked good. I started compacting the XML nodes to trace the path back towards the root, and eventually I found that the MinRequiredEVCModeKey parameter was under $_.ExtensionData.Runtime.
So, I put together the following command and generated a nice clean list of migration candidates:
get-cluster 'myCluster' | get-vm | ? {$_.powerstate -eq "poweredon"} | select name,@{Name="EVCMode";Expression={$_.ExtensionData.Runtime.MinRequiredEVCModeKey}} | ? {$_.EVCMode -ne "intel-ivybridge"}
For anyone who's curious, that command starts by getting the MyCluster cluster, then gets a list of all VMs within that cluster. It then filters that list to only look at powered on VMs and grabs 2 attributes from each VM: the Name and its EVCMode. Once that list is generated, it filters that list for all objects that aren't using the intel-ivybridge EVCMode.
Well, yes and no. That's absolutely correct that you can't vMotion a VM that powered up on an Ivy Bridge CPU back onto an ESXi host with a Sandy Bridge processor. The reason for this is that the VM, during its power on operation, scans the CPU of its host for a list of CPU features that are available and begins potentially using those features, which means that it can't be moved to a processor that doesn't have those features. The VM, in effect, inherits its host's EVC Mode for the lifespan of this power cycle. Until the VM goes through a complete new power cycle (not a reboot from within the guest OS), it will maintain that EVC Mode. This means that any VMs that were originally powered on on older CPUs will maintain that older EVC Mode, even though they're running on an Ivy Bridge capable cluster!
You can actually see this through the vSphere client. If you select your cluster, then go to the Virtual Machines tab, you can add the "EVC Mode" column (it's about 3/4 of the way down the list), which will list the runtime EVC Mode for each VM. All that we had to do was select any VMs that were still running with an older mode (we literally had hundreds) and move them into the older cluster.
Well, it wasn't quite that easy. The customer wanted a report of which VMs were candidates to move, so that they could be selective regarding which VMs were migrated to the older cluster. Easy enough, right? You can export a list as a CSV from the vSphere client, and you're good to go... usually.
I'm not sure why, but the CSV export blew up. It had a bunch of lines that were just text; looking back, it was probably just due to carriage returns in the "notes" field of some VMs. Regardless, it inspired me to pull that data via PowerCLI rather than using the retired vSphere client.
So, I did some quick googling and found that there's lots of information out there about how to pull the EVC Mode of a cluster (Get-Cluster 'MyCluster' | select Name,EVCMode). There's lots of information about how to pull the EVC Mode of an ESXi host (Get-VMHost 'MyHost' | select Name,MaxEVCMode). Not so much information about how to get the current EVC Mode requirement for a particular VM or set of VMs. I figured that it must be in there somewhere, but I had no idea where. So, I cheated.
I used get-vm to grab a running VM, then piped that to a file via Export-CLIXML -depth 8. I chose to use Export-CLIXML because it serializes all of the properties and subproperties of the object, up to the specified depth. I chose 8 because I know that VMware loves to embed references to objects within objects within references to the original object and I didn't want my file to get too crazy. As it was, I ended up with a 3 MB XML file for this single VM.
Once I had that XML file, I searched it for EVC. The first couple of instances weren't very promising (one was buried under the VMHost and the second didn't actually specify the mode), but the third one (around line 3400) looked good. I started compacting the XML nodes to trace the path back towards the root, and eventually I found that the MinRequiredEVCModeKey parameter was under $_.ExtensionData.Runtime.
So, I put together the following command and generated a nice clean list of migration candidates:
get-cluster 'myCluster' | get-vm | ? {$_.powerstate -eq "poweredon"} | select name,@{Name="EVCMode";Expression={$_.ExtensionData.Runtime.MinRequiredEVCModeKey}} | ? {$_.EVCMode -ne "intel-ivybridge"}
For anyone who's curious, that command starts by getting the MyCluster cluster, then gets a list of all VMs within that cluster. It then filters that list to only look at powered on VMs and grabs 2 attributes from each VM: the Name and its EVCMode. Once that list is generated, it filters that list for all objects that aren't using the intel-ivybridge EVCMode.
Hi Jason. If your command should works if we dont have enabled EVC on cluster ? im using your command to see this paramater but is empty... im using SDDC configuration on vSphere 6.0.
ReplyDeleteI haven't tested it on a cluster without EVC mode, sorry. I'd assume that it will still show something, as I think that a running VM should have an "EVC Mode" regardless for vMotion compatibility reasons. I think that I was working on a vSphere 6.0 environment when I put this together, but you can get it by doing (get-vm "some VM").ExtensionData.Runtime.MinRequiredEVCModeKey and see if it returns anything. If not, remove the last level of detail and try again, like this: (get-vm "some VM").ExtensionData.Runtime
DeleteHopefully that'll reveal the name of that parameter (if it's changed) or at least confirm that it's blank when EVC Mode is disabled (and might reveal some other useful parameter that exists when EVC Mode is disabled).