We are looking to upgrade our data center to a spine-leaf architecture using Cisco Nexus 9000 series switches. I am specifically interested in how VXLAN EVPN simplifies Layer 2 connectivity over a Layer 3 underlay. Is it really as scalable as the whitepapers claim for multi-tenant setups?
3 answers
VXLAN EVPN on the Nexus 9k is the gold standard for modern data centers. It solves the spanning-tree limitations by using BGP as the control plane to learn MAC addresses. This means you can have a Layer 2 domain that spans across different physical racks without the risk of loops. For multi-tenancy, it’s incredibly scalable because it uses VNIs to keep traffic isolated. The overhead is minimal compared to the flexibility you gain in workload mobility, especially if you are running a heavy VMware environment where vMotion needs to happen across different subnets.
Does moving to a VXLAN EVPN fabric require a complete hardware overhaul of my older Nexus 5000 and 7000 series, or can they participate as VTEPs in this new spine-leaf architecture?
VXLAN is definitely the way to go. It removes the 4096 VLAN limit which is a huge deal if you are providing hosting services or managing a massive private cloud infrastructure.
Matthew is right. The jump from 4k VLANs to 16 million VNIs is the biggest reason why service providers have jumped on the Nexus 9k bandwagon so quickly for their deployments.
Daniel, unfortunately, the older Nexus 5k/7k models often lack the ASIC support for hardware-based VXLAN encapsulation. While some newer 7k line cards can do it, you’ll get the best performance and features by using Nexus 9300s as Leaves and 9500s as Spines. It's usually better to use the older gear for simple 10G/40G connectivity and let the 9ks handle the VXLAN encapsulation.