Today my customer contacted me that they had problems with connectivity between two VMs connected to the same port group based on VXLAN. Quick tests done during call confirmed that there was a problem with VXLAN because they moved VMs to normal vDS port group and virtual machines were reachable. During a quick Webex and some troubleshooting steps, I saw that VMs were running on a cluster that was not prepared for NSX and no modules were installed.
As you probably know NSX modules are installed per cluster (on all ESXi hosts) and you do not need to prepare all clusters (e.g. you have not bought NSX licenses for all clusters). When you prepare ESXi hosts for VXLAN you have to specify a vDS where VTEP and future port groups will be created. If this vDS is shared across clusters and there are some not prepared for NSX you can connect VMs to the port group even ESXi hosts running VM is not prepared for it. When you do it, your VMs will be not reachable.
There are two main solutions in this case:
1) License and prepare all clusters for NSX 😉
2) Create a new vDS for clusters prepared for NSX so VMs running on another one are not able to be connected to VXLAN port group (Logical Switch)
and the last: take care where you power on your VMs 😉
