As I described in my article about speeding up vMotion, Multi-NIC vMotion provides load balancing the vMotion network traffic over multiple network adapters. It means that one vMotion session is balanced between all available vmknic. In this post I will show how to configure this feature with vSphere Standard Switch (vSS) or vSphere Distributed Switch (vDS).
Requirements and prerequisites are as follows:
- at least two dedicated network adapters for vMotion per host (If you do not have dedicated links for vMotion, consider using Network I/O Control. vMotion can saturate a link)
- prepared IP and mask settings for each VMkernel
- ensure that all VMkernel ports configured are in the same IP subnet/VLAN on all hosts
To configure Multi-NIC vMotion in vSphere 5.x on a Standard vSwitch please follow steps:
- Log into the vSphere Client and select the host from the inventory panel.
- Click the Configuration tab and select Networking.
- Click Add Networking and choose VMkernel as the Connection Type.
- Click Next.
- Add two or more NICs to a new standard switch.
- Name the VMkernel portgroup (for example,
vMotion-01
), and assign a VLAN ID as required. Click Use this port group for vMotion, then click Next. - Configure the IP address and subnet mask, then click Next.
- Click the Properties tab of the vSwitch.
- Click Add to create a second VMkernel portgroup and follow steps 3-7 again.
- Select the vMotion-01 portgroup, and click Edit.
- Click the NIC Teaming tab.
- Under Failover Order, select Override switch failover order.
- Configure the first adapter (for example, vmnic0 in my case) as active and move the first adapter (fin my case, vmnic2) to standby. Click OK.
- Select the vMotion-02 portgroup, and click Edit.
- Click the NIC Teaming tab.
- Under Failover Order, select Override switch failover order.
- Configure the first adapter (for example, vmnic2 in my case) as active and move the first adapter (fin my case, vmnic0) to standby. Click OK.
- The configuration should look similar as follow.
- If you have more NICs, please follow steps 3-13 for every VMkernel. Also on the Properties tab of the vSwitch, select each vMotion portgroup in turn and confirm that the active and standby adapters are the reverse of each other.
To configure Multi-NIC vMotion in vSphere 5.x on a Distributed vSwitch please follow steps:
- Log into the vSphere Client and click the Networking inventory.
- Right-click the Distributed vswitch, then click New Port Group.
- Name the dvPortgroup (for example,
vMotion-01
). Click VLAN and assign a VLAN ID as required, then click Next and Finish. - Create the second portgroup following again steps 1-3.
- Select vMotion01 portgroup, click Edit Settings.
- Click the Teaming and Failover tab.
- Configure dvUplink1 as
Active Uplink
and move dvUplink2 toStandby Uplink
. Click OK. - Select vMotion02 portgroup, click Edit Settings. Click the Teaming and Failover tab.
- Configure dvUplink2 as
Active Uplink
and move dvUplink1 toStandby Uplink
. - Go the Hosts and Clusters Inventory tab, select a host's Networking, and click vSphere Distributed Switch.
- Click Manage Virtual Adapters and click Add to add new virtual adapter.
- Choose VMkernel as the Virtual Adapter Type.
- Select the
vMotion-01
portgroup, click Use this port group for vMotion, then click Next. - Configure the IP address and subnet mask, then click Next and Finish.
- Add another virtual adapter, then select the
vMotion-02
portgroup (follow again steps 12-14) - On the Distributed vSwitch, select each dvportgroup on VMKernel Port
vmk1
andvmk2
in turn, and confirm that the active and standby uplinks are the reverse of each other. - The configuration of Multi-NIC vMotion should like as follow:
Hi Mariusz
Just a question about load balancing policy: I always thought that on distributed switch, the balancing policy should be set to "physical nic load" that, in fact, exists only on distributed switch and that is considered a better option. Instead I saw that you leaved "route based on origination virtual port". Why? Is just for vmotion or do you always choose this kind of balancing even on distributed port group?
Hi Marco,
It depends on network topology (and availability of NICs) but in general - it's just for Multi-NIC VMotion - LBT can not balance as there is only one active NIC in each port group. If Enterprise Plus is available, I use LBT for VMs (sometimes Storage) traffic.
Mariusz
thanks a lot for your kind and quick reply.
Indeed with only one active NIC per port group, it really does make sense. But...leaving LBT on Multi-NIC VMotion could cause problems?
Hi Marco,
I do not think that it could cause problems. However, I haven't tested it and also I haven't seen such VMware recommendation. Btw. When VMotion and VMs share the same NICs, it's important to remember to use LBT for VMs traffic.
Ok Mariusz thanks again.
I founded other posts like this but none of them explained this detail. I think that other people will appreciate.
Moreover I founded very interesting articles on this blog and I'll follow it! Thanks for your great work.
Bye
Marco
You're welcome!
True .. I agree with Marco. Some of your posts were explained in a very Detailed manner. Thank you.
Hi Mariusz.
For Mutli
Quick question about Multi-NIC VMotion between two Standard Switches.
would you recommend to configure a port channel on Cisco Switch or we don't need it in this scenario ?
Thanks
Sid
Hi Sid,
a port channel is not required/recommended and Multi-NIC vMotion is within the same vSS or vDS.
Thanks Mariusz. I was just thinking about port-channel as i am migrating huge loads between two Clusters.
I am now going to configure Multi-NIC vMotion without port-channel.
Thanks again for the reply.