The virtual machine must use RDMA - Hyper-V

---

Could someone help me clear up this practice exam question regarding RDMA being used from the guest VM?

You have a serer named Server1 that runs Windows Server 2016. Server1 is a Hyper-V host. You have two network adapter cards on Server1 that are Remote Direct Memory Access (RDMA) - capable. you need to aggregate the bandwidth of the network adapter cards for a virtual machine on Server1. The solution must ensure that the virtual machine can use the RDMA capabilities of the network adapter cards.

Which command shoudl you run First? (To answer, select teh appropriate options in teh answer area.)

Initially I got this question confused with the following command which enables NIC teaming at the guest.
Set-VMNetworkAdapter -VMName <VMname> -AllowTeaming On

However, the question is focused on passing RDMA capabilities to the guest VM. All of the research I am doing has highlighted that in Server 2012 R2 it was not possible to use RDMA technology on both the host and guest using the same physical NIC. You had to have dedicated NICs for the guest and the host.

Server 2016 has enabled the ability to share RDMA with the guest and host. However I am super unsure of the details. Also this change simply outlines a new capability of sharing this RDMA technology on the same physical NIC. the articles I have found don't really talk about how the feature is enabled for a guest. Just that in both 2012 R2 and 2016 it was capable of doing so.

So, in going back to the question, I did find the "New-VMSwitch" command (which wasn't an option in the practice question) that had the "-EnableEmbeddedTeaming" switch.. However the description of this switch doesn't really show that it is used to enable RDMA to the guest VM.

It states "Specifies whether this cmdlet enables teaming for the virtual switch."

So now we are back to teaming.... Can someone that understands this technology please explain? Is it required to deploy a Hyper-V switch (SDN) with embedded teaming in order to give RDMA features to the guest, or was this only to allow both the guest and the host access to this feature? What would the next step be if you were to configure this?

Regards,
Adam Tyler

Adam,

One of my MOST FAVORITE topics, and at times a tad confusing... First things first, a quick and simple answer, and then a link to help you get it all sorted out if you want to build it and test it out.

Short answer is that Switch Embedded Teaming (SET) is used to enable the use of RDMA from the VMs to the Host via the Hyper-V switch. New feature capability for Server 2016, which is why we are able to move away from the need for separate physical NICs as in prior versions.

Check out the following for a GREAT step by step breakdown of how to walk through building out a sample environment if you wan to, or great just to see the steps and understand what is happening when and why:

https://www.tech-coffee.net/how-to-deploy-a-converged-network-with-windows-server-2016/

Keep in mind the following:

With a SET configuration, the physical NICs in the host are effectively uplinks for the virtual switch.

A SET configuration also provides the ability for RDMA to be converged. This means that we can:

1. Deploy two RDMA enabled NICs (rNICs) on the host and enable DCB on them, which is recommended
2. Converge the two rNICs using a SET switch
3. Connect virtual machines to the SET switch
4. Create virtual NICs in the management O/S that are connected to the SET switch and assign QoS rules to them
5. Enable RDMA on some of the management O/S virtual NICs

Some of those virtual NICs might be for SMB 3.0 traffic that can now leverage SMB Direct and take advantage of storage, Live Migration, and redirected I/O communication that has a lower latency and impact on the host CPU and virtual machine services.

For a good general overview of RDMA and SET, in Server 2016, take a look at the following:

https://docs.microsoft.com/en-us/windows-server/virtualization/hyper-v-virtual-switch/rdma-and-switch-embedded-teaming

Let me know if you have any questions once you take a look.

Cheers,

Adam

Thanks Adam! I have a pretty nice home lab and a growing list of things I want to stand up as I go through this material. My lab is built on VMware ESXi 6.5 and some 1 Gb 3500 series cisco switches. I imagine that I can use nested Windows svr 16 VMs to try this out..

Just to clarify one point, You said that Switch embedded Teaming (SET) is used to allow RDMA to be utilized by both the host and the guests. The question asks for aggregation of the network links as well as passing through RDMA features to the guest, so that means we are deploying SET. Based on the article, that means we are running the following command:

new-VMSwitch -name vswitchname -NetAdapterName net1,net2 -EnableEmbeddedTeaming $True

Is that about right? If so, this seems to focus on convergence of all the network traffic over common links rather than anything to do with SDN. Am I correct in assuming this particular setting is not related to SDN in Svr 2016?

It seems like a lot of talk for a very basic feature. I come from the VMware world in virtualization.. I've been doing this with VMware vswitches and port groups for years. However when it comes to iSCSI storage connections in VMware I generally would deploy a separate vswitch and dedicated VMK ports allocated to different physical NICs so that multipathing functioned correctly. How would you approach this with Svr 16 and SET?

Regards,
Adam Tyler