Something that sprung into my head recently was “how efficient can we be with virtualised servers’ traffic?” which then sent me down a rabbit hole I certainly wasn’t anticipating!
In a traditional physical network we can use 802.1q VLAN tagging to isolate one set of traffic from another into distinct virtual networks. As each VLAN will have its own subnet that does not overlap with other VLANs in the organisation, any communication between them requires use of a router or layer 3 switch to make decisions on how to forward packets based on their IP address and allows access control to be implement to restrict the flow accordingly.
This is great and well established for physical networks, but in the virtualised world the process becomes quite inefficient. For example, if we’ve got a single host with two Virtual Machines existing on the same VLAN attached to the same vSwitch then any communication between them will not leave the host or touch the physical NIC on the host – this enables a massive boost in transfer speeds between the two VMs.
On the other hand, if our two VMs are on two different VLANs then we need to introduce a router or layer 3 switch into the setup – this is where things start to get messy.
We’ve got a couple of options for this:
Option 1 – Setup a Virtual Machine on the host with routing functionality (Microsoft Server, DD-WRT, pfSense, Sophos UTM are some examples) with vNIC in both VLANs and set this up as the default gateway for VMs in the corresponding VLANs.
Option 2 – Use VMware NSX for vSphere which is similar to option 1, but is massively optimised for a VMware environment
VMware’s NSX product fits into existing vSphere environments and is deployed as an OVA template. Following installation you can then connect the appliance to the vCenter Server to display the full VMware infrastructure inventory. Unfortunately I don’t have access to a NSX appliance or the relevant supporting infrastructure so I can’t give a technical demo of this, however I have done a bit of reading on the underlying concepts which I’ll outline here and hopefully soon I’ll demonstrate this in a lab environment.
NSX provides the hosts with a “Distributed Logical Router” (DLR) which allows East-West (between VMs) communication for different VLANs to be routed in the VMware Kernel, meaning it never touches the physical network. If interaction with the physical network is needed to reach other hosts then the IP traffic is encapsulated inside a VXLAN overlay on the traditional physical network. We can even use a different router type, the “Edge Services Router” (ESR), instead of or in addition to the DLR to grant us abilities to perform firewalling, load balancing, VPNs and other features. You can run multiple instances of both the DLR and ESR VMs for situations like multiple tenancies on the same host. It’s even possible to have the setup work across multiple hosts!
The NSX option is clearly the more attractive when it comes to simplicity, performance and functionality. It allows virtualisation teams to quickly and easily create networks in software using existing IP networks without having to involve networking teams, speeding up delivery of complex deployments.
Hopefully at some point in the near future I’ll get chance to demo this for real and see just how straight forward it is compared to other technologies and solutions.
Here are some links I’ve used in my research of this topic which you might find useful: