How to configure Multi-NIC vMotion on vSS or vDS?

By | January 21, 2015

As I described in my article about speeding up vMotion, Multi-NIC vMotion provides load balancing the vMotion network traffic over multiple network adapters. It means that one vMotion session is balanced between all available vmknic. In this post I will show how to configure this feature with vSphere Standard Switch (vSS) or vSphere Distributed Switch (vDS).

Requirements and prerequisites are as follows:

  • at least two dedicated network adapters for vMotion per host (If you do not have dedicated links for vMotion, consider using Network I/O Control. vMotion can saturate a link)
  • prepared IP and mask settings for each VMkernel
  • ensure that all VMkernel ports configured are in the same IP subnet/VLAN on all hosts

To configure Multi-NIC vMotion in vSphere 5.x on a Standard vSwitch please follow steps:

  1. Log into the vSphere Client and select the host from the inventory panel.
  2. Click the Configuration tab and select Networking.
  3. Click Add Networking and choose VMkernel as the Connection Type.Set up Multi-NIC vMotion on vSS via vSphere Client 1
  4. Click Next.
  5. Add two or more NICs to a new standard switch.Set up Multi-NIC vMotion on vSS via vSphere Client 2
  6. Name the VMkernel portgroup (for example, vMotion-01), and assign a VLAN ID as required. Click Use this port group for vMotion, then click Next.Set up Multi-NIC vMotion on vSS via vSphere Client 3
  7. Configure the IP address and subnet mask, then click Next.Set up Multi-NIC vMotion on vSS via vSphere Client 4
  8. Click the Properties tab of the vSwitch.Set up Multi-NIC vMotion on vSS via vSphere Client 5
  9. Click Add to create a second VMkernel portgroup and follow steps 3-7 again.
  10. Select the vMotion-01 portgroup, and click Edit.
  11. Click the NIC Teaming tab.
  12. Under Failover Order, select Override switch failover order.
  13. Configure the first adapter (for example, vmnic0 in my case) as active and move the first adapter (fin my case, vmnic2) to standby. Click OK.Set up Multi-NIC vMotion on vSS via vSphere Client 6
  14. Select the vMotion-02 portgroup, and click Edit.
  15. Click the NIC Teaming tab.
  16. Under Failover Order, select Override switch failover order.
  17. Configure the first adapter (for example, vmnic2 in my case) as active and move the first adapter (fin my case, vmnic0) to standby. Click OK.Set up Multi-NIC vMotion on vSS via vSphere Client 7
  18. The configuration should look similar as follow.Set up Multi-NIC vMotion on vSS via vSphere Client 8
  19. If you have more NICs, please follow steps 3-13 for every VMkernel. Also on the Properties tab of the vSwitch, select each vMotion portgroup in turn and confirm that the active and standby adapters are the reverse of each other.

 

To configure Multi-NIC vMotion in vSphere 5.x on a Distributed vSwitch please follow steps:

  1. Log into the vSphere Client and click the Networking inventory.
  2. Right-click the Distributed vswitch, then click New Port Group.Set up Multi-NIC vMotion on vDS via vSphere Client 1
  3. Name the dvPortgroup (for example, vMotion-01). Click VLAN and assign a VLAN ID as required, then click Next and Finish.Set up Multi-NIC vMotion on vDS via vSphere Client 2
  4. Create the second portgroup following again steps 1-3.
  5. Select vMotion01 portgroup, click Edit Settings.Set up Multi-NIC vMotion on vDS via vSphere Client 3
  6. Click the Teaming and Failover tab.
  7. Configure dvUplink1 as Active Uplink and move dvUplink2 to Standby Uplink. Click OK.Set up Multi-NIC vMotion on vDS via vSphere Client 4
  8. Select vMotion02 portgroup, click Edit Settings. Click the Teaming and Failover tab.
  9. Configure dvUplink2 as Active Uplink and move dvUplink1 to Standby Uplink.Set up Multi-NIC vMotion on vDS via vSphere Client 5
  10. Go the Hosts and Clusters Inventory tab, select a host's Networking, and click vSphere Distributed Switch.
  11. Click Manage Virtual Adapters and click Add to add new virtual adapter.Set up Multi-NIC vMotion on vDS via vSphere Client 6
  12. Choose VMkernel as the Virtual Adapter Type.Set up Multi-NIC vMotion on vDS via vSphere Client 7
  13. Select the vMotion-01 portgroup, click Use this port group for vMotion, then click Next.Set up Multi-NIC vMotion on vDS via vSphere Client 8
  14. Configure the IP address and subnet mask, then click Next and Finish.Set up Multi-NIC vMotion on vDS via vSphere Client 9
  15. Set up Multi-NIC vMotion on vDS via vSphere Client 10
  1. Add another virtual adapter, then select the vMotion-02 portgroup (follow again steps 12-14)
  2. On the Distributed vSwitch, select each dvportgroup on VMKernel Port vmk1 and vmk2 in turn, and confirm that the active and standby uplinks are the reverse of each other.
  3. The configuration of Multi-NIC vMotion should like as follow:Set up Multi-NIC vMotion on vDS via vSphere Client 11

10 thoughts on “How to configure Multi-NIC vMotion on vSS or vDS?

  1. Marco

    Hi Mariusz

    Just a question about load balancing policy: I always thought that on distributed switch, the balancing policy should be set to "physical nic load" that, in fact, exists only on distributed switch and that is considered a better option. Instead I saw that you leaved "route based on origination virtual port". Why? Is just for vmotion or do you always choose this kind of balancing even on distributed port group?

    Reply
    1. Mariusz Post author

      Hi Marco,

      It depends on network topology (and availability of NICs) but in general - it's just for Multi-NIC VMotion - LBT can not balance as there is only one active NIC in each port group. If Enterprise Plus is available, I use LBT for VMs (sometimes Storage) traffic.

      Reply
  2. Marco

    Mariusz

    thanks a lot for your kind and quick reply.
    Indeed with only one active NIC per port group, it really does make sense. But...leaving LBT on Multi-NIC VMotion could cause problems?

    Reply
    1. Mariusz Post author

      Hi Marco,

      I do not think that it could cause problems. However, I haven't tested it and also I haven't seen such VMware recommendation. Btw. When VMotion and VMs share the same NICs, it's important to remember to use LBT for VMs traffic.

      Reply
  3. Marco

    Ok Mariusz thanks again.
    I founded other posts like this but none of them explained this detail. I think that other people will appreciate.
    Moreover I founded very interesting articles on this blog and I'll follow it! Thanks for your great work.

    Bye
    Marco

    Reply
  4. Gopi Kiran

    True .. I agree with Marco. Some of your posts were explained in a very Detailed manner. Thank you.

    Reply
  5. Sid

    Hi Mariusz.

    For Mutli
    Quick question about Multi-NIC VMotion between two Standard Switches.
    would you recommend to configure a port channel on Cisco Switch or we don't need it in this scenario ?

    Thanks
    Sid

    Reply
    1. Mariusz Post author

      Hi Sid,
      a port channel is not required/recommended and Multi-NIC vMotion is within the same vSS or vDS.

      Reply
      1. Sid

        Thanks Mariusz. I was just thinking about port-channel as i am migrating huge loads between two Clusters.
        I am now going to configure Multi-NIC vMotion without port-channel.
        Thanks again for the reply.

        Reply

Leave a Reply

Your email address will not be published. Required fields are marked *