No matter what the size of the project, you’ll always need to ask yourself: How many network cards do I need? Unfortunately it’s not always immediately apparent and will vary depending on requirements for redundancy, throughput, security, scalability and network connectivity. However, I’ve put together a few general guidelines to help you on your way.
Let’s start with the facts: in its most simple form XenServer 5.6 requires a single network card per host and at a maximum supports 16 network cards and 8 bonds
So why not just use one card? The key problem with this approach is there’s no redundancy in the event of a single card, switch or cable failure. Although, not a big problem for most traditional desktops, this could affect a substantial number of users in a VDI environment. Fortunately, XenServer allows two network cards to be configured in a bond, allowing traffic to be automatically routed to the second card should the first one fail.
Many servers today are supplied with network cards offering two or more ports. It’s important that any bonds created consist of two separate diversely routed network cards so that a single failure doesn’t bring down the bond. Although cards in the bond don’t need to be identical, I’d at least recommend they share the same speed for consistency.
Now, what about throughput? The host’s networking resources are shared amongst the virtual desktops it supports and users will suffer from poor performance if there’s insufficient bandwidth available. As such, consider routing virtual machine traffic over an SLB bond so that it’s automatically load balanced across two NICs. Virtual machine traffic is load balanced by MAC address and rebalanced every ten seconds. Failover support is provided for all other traffic types, including management and IP-based storage traffic. The load balancing algorithm associates traffic from each virtual interface to one of two NICs in the bond. It’s important to understand that it doesn’t allow a single virtual interface to utilize both NICs in the bond simultaneously.
What other ways can you increase throughput? The overall throughput of the XenServer host can be increased by creating separate physical networks for each of the following traffic types:
- Storage – Typically the heaviest user of available throughput. It is often advisable to isolate IP-based storage onto a separate physical network to help alleviate network congestion. Implement multipathing where supported as bonding does not provide additional throughput for IP-based storage traffic.
- Provisioning – PVS is a network intensive application that could negatively affect the performance of other network traffic. For example, PVS typically streams 166MB of data across the network during the boot process of a single Windows 7 x86 desktop. Depending on the impact that PVS will have on the underlying network, it may be necessary to implement a separate physical network dedicated to provisioning traffic.
- Virtual Machine – Virtual machines (desktops and servers) typically communicate with a lot of backend infrastructure including domain controllers, web servers, database servers, file servers etc. The user experience will be affected if this traffic is squeezed.
- Management – The isolation of management traffic onto a separate physical network can be a useful strategy in environments with a high-number of live migrations.
- Backups – Non-provisioned virtual machines may need to be backed up. Most backups occur out of hours when the network isn’t heavily loaded. However, some deployments support users around the clock and may require a physical backup network to prevent congestion.
Throughput may also be increased by using fast networks, for example 1Gbps or greater. However, this doesn’t address the problem of different traffic types competing for available bandwidth. Third-party solutions exist which allow maximum bandwidth limits to be set for each VLAN. Alternatively, XenServer Quality of Service (QoS) functionality could be used to set a maximum transfer rate on the virtual network interface of any virtual machines using an unfair share of the network, but this can be difficult to predict. Extensive QoS settings are available in the Distributed vSwitch but that’s a whole separate blog post…
How about security? Well, separate physical networks may be required here too, particularly for storage and management traffic. Although VLANs could be used, some organizations implement separate physical networks due to a combination of security and throughput requirements. In addition, physical networks may also be required to connect specific desktop groups to ‘secure’ networks.
What do you do if you are concerned about resiliency and throughput but don’t have ten network cards available due to cost or hardware limitations? Every case is different, but personally I’d recommend the following as a general guide – let me know what you think:
2 x Network Cards:
Bond 1 – management, storage, virtual machine, provisioning, backup
4 x Network Cards:
Bond 1 – management, provisioning and virtual machine
Bond 2 – storage and backup
6 x Network Cards:
Bond 1 – management and virtual machine
Bond 2 – provisioning
Bond 3 – storage and backup
8 x Network Cards:
Bond 1 – management
Bond 2 – virtual machine
Bond 3 – storage and backup
Bond 4 – provisioning
Although there’s no magic answer, hopefully these guidelines can help you work out the number of network cards you’ll need on your next XenServer design.