10 Gb Multi-NIC vMotion Testing
With the upgrade of our network core from 1 Gb to 10 Gb and with the installation of new Dell M620 blade servers I wanted to run some quick and dirty tests to see what performance looked like when performing vMotion on the new virtual infrastructure. Some of the tests were run during or right after a Windows reboot to see how long it took when in all memory is touched.
On the old hardware:
Hosts: HP Proliant DL380 G6, Intel Xeon x5660 @ 2.8 GHz , 256 GB RAM, multi-nic 1 Gb vMotion configured. 1 NIC uses the Intel 82850 chipset, 1 NIC uses Broadcom NetXtreme II BCM5709.
Switches: Cisco 6513 (host ports plugged into WS-6700-DFC3C 1 Gb ports. MTU set to 1500)
vSphere version: 5.0.0 Build 914586
Here are the results of a couple vMotion tests on 1 Gb links configured for multi-nic vMotion:
Windows 2003 Std 32 bit, 1 vCPU, 2 GB RAM, relatively idle: 25 seconds
Windows 2003 Std 32 bit, 1 vCPU, 4 GB RAM, relatively idle: 40 seconds
Windows 2008R2, 1 vCPU, 4 GB RAM, relatively idle: 26 seconds
Windows 2012, 2 vCPU, 8 GB RAM, relatively idle: 48 seconds
On the new hardware:
Hosts: Dell M620 blades, Intel Xeon E5-2600 @ 2 GHz, 256 GB RAM, multi-nic 10 Gb vMotion configured. 1 NIC uses the Intel 82599EB chipset, 1 NIC uses Broadcom NetXtreme II BCM57810)
Switches: DellForce10 MXL 10/40 (internal-facing 10 Gb switchports, MTU set to 9000). Note that the MXL switches are connected via MLAG to a pair of Cisco Nexus 7009 F2 modules with 10 Gb ports, which is relevant for the inter-chassis vMotion tests).
vSphere version: 5.0.0 Build 914586
Test results on 10 Gb links configured for multi-nic vMotion:
Between hosts within the same Dell M1000e chassis:
Windows 2003 Std 64 bit, 1 vCPU, 2GB RAM, idle: 7 seconds
Windows 2008 R2, 1 vCPU, 2 GB RAM, idle: 10 seconds
Windows 2012, 1 vCPU, 2 GB RAM, idle: 8 seconds
Windows 2012, 1 vCPU, 4 GB RAM, after reboot: 8 seconds
Windows 2012, 1 vCPU, 8 GB RAM, during OS reboot: 8 seconds
Windows 2012, 2 vCPU, 8 GB RAM, after reboot: 8 seconds
Windows 2012, 4 vCPU, 8 GB RAM, after reboot: 12 seconds
Between hosts across two different Dell M1000e chassis:
Windows 2003 Std 64 bit, 1 vCPU, 2GB RAM, idle: 5 seconds
Windows 2008 R2, 1vCPU, 2 GB RAM, idle: 9 seconds
Windows 2012, 2 vCPU, 8 GB RAM, after reboot: 8 seconds
As I mentioned at the beginning these were quick and dirty tests with few controls so make of the results whatever you will. There is no denying however that the improvement on the new hardware is huge. It will be interesting to see if the difference is the same once all the blades are loaded with production workloads.
May 17th, 2013 - 10:44
Thank you Keith. The backend storage is a Compellent array running a mix of FC, SAS, and SATA drives.
May 17th, 2013 - 08:17
Great write up! What is your backend storage for your virtual infrastructure?