We are designing a high-frequency trading platform where every microsecond of network latency counts. While the standard bridge driver is too slow, we are caught between choosing the Macvlan driver or the newer IPVlan L2/L3 drivers. Since both allow containers to have their own IP addresses on the physical network, which one provides better throughput and lower CPU overhead in a high-density environment with thousands of containers?
3 answers
If you need containers to communicate with the host directly, remember that Macvlan prevents this by default due to kernel security isolation.
For high-density environments, IPVlan is generally superior to Macvlan. Macvlan requires each container to have a unique MAC address, which can overwhelm the CAM tables of many physical switches if you scale to thousands of containers. IPVlan, however, shares the host's MAC address across all containers while giving each its own IP. This significantly reduces the overhead on the physical network infrastructure. In terms of raw performance, IPVlan L3 mode can be slightly faster because it eliminates the need for bridge-mode processing entirely, though it requires more complex routing configuration at the network layer.
Does your current network infrastructure support promiscuous mode, which is often a requirement for Macvlan to function correctly on many cloud and virtual platforms?
Christopher, that's a huge point. We found out the hard way that our enterprise switches had security policies blocking multiple MACs on a single port. Susan, your explanation of IPVlan sharing the host MAC is exactly why we are leaning that way now. It bypasses the switch security issues while still giving us the "near-metal" performance we need. We're going to benchmark the L3 mode this week to see if the routing complexity is worth the extra latency reduction for our trade execution engine.
Amanda is correct. To enable host-to-container communication with Macvlan, you have to create a separate child interface on the host, which adds more management overhead.