VMware vSphere and Provisioning Traffic

---

Hello ITPro.TV! I'm looking for some guidance on configuration options for the "Provisioning Traffic" of vSphere 6 and above. I am very familiar with vMotion traffic and use the "Multi-NIC" setup for vMotion often in my single cluster deployments. I'd like to use the same approach to configure Provisioning Traffic, but I can't find any VMware documentation stating that vSphere actually supports "Multi-NIC" provisioning traffic.

Furthermore, I'm not sure what good Provisioning Traffic would do me in a single cluster scenario where all hosts in the cluster have access to all datastores. It seems that a "cold" migration of a VM from one datastore to another would all be IO traffic over iSCSI or NFS. The only time a cold migratoin of a VM would utilize the provisioning vmkernel seems to be if you were using the Provisioning TCP/IP stack and performing a cold migration or clone with routing to a different cluster and a separate set of storage.

Maybe I am off base, can anyone confirm when the "provisioning" traffic would be utilized in a single cluster where all hosts have direct access to all shared storage volumes?

Regards,
Adam Tyler

Adam,

The provisioning TCP/IP stack is used to isolate traffic for cold migration, VM clones, and snapshots, and to assign a dedicated default gateway, routing table, and DNS configuration for this traffic.

In your environment, there would still be a benefit to using the Provisioning Stack, but the benefit would really come down to being able to address traffic management/differentiation in a more formal way, allowing you to have more granular control. This may or may not be a desired outcome for you.

For instance, you may want to be able to VLAN this traffic, or implement Jumbo Frame support along the traffic pathway to optimize this traffic flow or secure it differently than other traffic. You may want to pass this traffic through an IDS/IPS solution, or some other in-line monitoring capability and want to be able to identify this traffic specifically based on whatever reasons you are addressing.

There are many scenarios where dedicated VMKs can offer benefits to you, as well as the additional benefit of an optimized traffic management stack, as noted below:

1. Route the traffic for migration of powered on or powered off virtual machines by using a default gateway that is different from the gateway assigned to the default stack on the host.

2. By using a separate default gateway, you can use DHCP for IP address assignment to the VMkernel adapters for migration in a flexible way.

3. Assign a separate set of buffers and sockets.

4. Avoid routing table conflicts that might otherwise appear when many features are using a common TCP/IP stack.

5. Isolate traffic to improve security.

To address the other part of your question, which was the use of a multi-nic capability, the short answer is YES, it is supported.

Remember that the way we leverage multiple Physical Adapters in the host is by having multiple uplinks/dvUplinks on our switches (standard or distributed) and then binding multiple physical adapters (vmnics) to the switch via uplinks. We then use a combination of a load balancing policy, failback, network failure notification, notify switches and teaming to sculpt the exact behavior we want. By using multiple vmnics bound to a dedicated standard switch that uses the Provisioning Stack, you can get the best results. If you do this, the results are that after you create a VMkernel adapter on the provisioning TCP/IP stack, you can use only this stack for cold migration, cloning, and snapshots on this host. The VMkernel adapters on the default TCP/IP stack are disabled for the provisioning service. If a live migration uses the default TCP/IP stack while you configure VMkernel adapters with the provisioning TCP/IP stack, the data transfer completes successfully. However, the involved VMkernel adapters on the default TCP/IP stack are disabled for future cold migration, cross-host cloning, and snapshot sessions.

I hope that helps to get it all sorted out for you. If you have any additional questions, just let me know.

Good Luck !!

Cheers,

Adam

Thanks for your reply Adam G. I was working with one of our vendors recently that is a bit of an authority on the VMware vSphere product line. They mentioned that "provisioning" traffic does NOT support multi-NIC load balancing the same way as vMotion can. That isn't to say you couldn't have more than one physical NIC tied to the vmk port, but multi-nic for "vMotion" is a different animal.

This is an approach where you create a vmk port and intentionally only bind one physical interface to it. Then create a second vmk port bound to a different physical interface. Both vmk ports created have unique IP addresses on the same vMotion VLAN and both vmk ports have vMotion support enabled. With this config, vMotion traffic is automatically balanced 50% across each vmk port enabled for vMotion.

With this config, you achieve load balancing without the requirement of LACP switch config or hashing algorithms. It is two completely different flows that saturate (unless you throttle) all available vMotion enabled vmk ports.

Regards,
Adam Tyler