ESXi DirectPath I/O on 970A-UD3

[cortexbomb]

I've seen multiple success stories using this motherboard for a whitebox ESXi 5.0 install and getting PCI passthrough (aka DirectPath I/O) working, but I'm having no luck in my own attempts. For example:

[H]ard|Forum - View Single Post - Your home ESX server lab hardware specs?
vm-help.com - View topic - GA 970A-UD3 works with Chilly's Realtek driver! :) Thnx!


In my case, ESXi installs fine but it reports that my hardware does not support DirectPath I/O despite having enabled Virtualization and IOMMU in the BIOS. I've also updated to the latest non-beta BIOS available (F4) but that didn't have any effect. One difference between the above examples and others that I've found is that they all use a Phenom II whereas I have an Athlon II CPU. I don't believe this should be the issue as both series of processors support AMD-V.

I haven't yet attempted to install ESXi 4.1 to see if there's any difference there because the onboard NIC does not work directly out of the box in that version of ESXi. I'd rather not have to mess with third party drivers to get that functioning, but if it meant PCI passthrough would function I certainly will.

Is there something simple that I'm forgetting/missing that the above users haven't mentioned? Help!

[Lsdmeasap]

Re: ESXi DirectPath I/O on 970A-UD3

You may need to use the latest beta BIOS, especially if it's mentioned in those threads you linked, that's often where this is corrected.

As for your CPU and compatibility with this feature, you may need to ask Gigabyte directly to see if there is any known issues with that CPU

http://www.gigabyte.com/support-downloads/technical-support.aspx

[cortexbomb]

Re: ESXi DirectPath I/O on 970A-UD3

Thanks for your reply. At least one of the threads I linked uses the same BIOS version as I do (F4), though I did briefly try the beta BIOS with no luck. I have a ticket open with Gigabyte support regarding the issue but I thought I'd ask around in some forums as their turnaround time can be a bit slow at times.

[Lsdmeasap]

Re: ESXi DirectPath I/O on 970A-UD3

Anytime, I always try to help when I can! I checked the links you gave and didn't see any mention of Beta, or BIOS (But I didn't look at the images for BIOS version so I guess I missed that).

Did you start by loading optimized defaults in the BIOS, then save/apply/reboot back to the BIOS before making any changes (Don't forget SATA Settings)? If not, that may help, along with clearing the CMOS before doing that. Personally I'd stick with the latest beta as it would have the latest fixes and improvements, go back and try that with the above mentioned optimized dance.

If you can't get it working, it might have something to do with the CPU, or the software version? Hopefully tech support will get back to you soon, sorry they are often slower that we'd like!

[Acebmxer]

Re: ESXi DirectPath I/O on 970A-UD3

I've seen multiple success stories using this motherboard for a whitebox ESXi 5.0 install and getting PCI passthrough (aka DirectPath I/O) working, but I'm having no luck in my own attempts. For example:

[H]ard|Forum - View Single Post - Your home ESX server lab hardware specs?
vm-help.com - View topic - GA 970A-UD3 works with Chilly's Realtek driver! :) Thnx!


In my case, ESXi installs fine but it reports that my hardware does not support DirectPath I/O despite having enabled Virtualization and IOMMU in the BIOS. I've also updated to the latest non-beta BIOS available (F4) but that didn't have any effect. One difference between the above examples and others that I've found is that they all use a Phenom II whereas I have an Athlon II CPU. I don't believe this should be the issue as both series of processors support AMD-V.

I haven't yet attempted to install ESXi 4.1 to see if there's any difference there because the onboard NIC does not work directly out of the box in that version of ESXi. I'd rather not have to mess with third party drivers to get that functioning, but if it meant PCI passthrough would function I certainly will.

Is there something simple that I'm forgetting/missing that the above users haven't mentioned? Help!

This may be a little different on AMD systems but his is how it works on Intel systems. Direct i/o path is only enabled when VT-D support is enabled. I was able to contact gigabyte and had VT-D support added to X58A-UD5 FF1 bios but then was removed in the next bios as gigabyte says VT-D is not supported by the motherboard. Yes it worked when they added it.

The other thing is the hardware needs to be supported by esxi/vsphere. Intel ich10 controller is supported by ESXi/Vsphere but only in AHCI mod not in raid mode. ESXi is the "Operating system" I have read the drivers can be installed into esxi to add support but have not ventured into it.

[cortexbomb]

Re: ESXi DirectPath I/O on 970A-UD3

My understanding is that AMD's version of VT-d is AMD-V on the cpu side and IOMMU on the motherboard chipset. The Athlon II (as well as Phenom IIs and some Semprons) is supposed to support AMD-V and the GA-970A-UD3 BIOS does have an IOMMU option which I've enabled. I haven't even gotten to a point where I can test if my hardware is actually supported for pass through. ESXi/vSphere is saying that DirectPath I/O is just outright not supported on my system, though as best as I can tell it should be.

Though I've done a fair bit of research and believe that the devices I want to pass through will be able to, at the moment I'm stuck on ESXi being under the impression that I don't have AMD-V/IOMMU available or enabled even though Virtualization and IOMMU are enabled in the BIOS. I was hoping I was simply missing some other BIOS option that I had to toggle but no such luck so far. Hopefully when Gigabyte gets back to me I'll find out if there's something about the BIOS that doesn't play nicely with my Athlon II that apparently does get along with others' Phenom IIs. Maybe it'll be as simple as a custom BIOS or similar. I can only hope.

[Acebmxer]

Re: ESXi DirectPath I/O on 970A-UD3

I just got done installing ESXi5.0 on a dell desktop and does not have vt-d or directpath i/o support. Vsphere client says Directpath i/o not supported. Installed Win2k8r2 in a vm and had no issues. Does the ESXi "HOST" not have network access or is it your guest vm's not having network access?

[cortexbomb]

Re: ESXi DirectPath I/O on 970A-UD3

Oh, I have network access. I just want to be able to pass some of my PCI devices directly into a VM, which requires DirectPath I/O. Based on my research others have had success with this motherboard and DirectPath I/O. The difference is that they've all used Phenom II processors whereas I have an Athlon II. Based on AMD's specifications, both processors have the AMD-V support needed but I suppose the issue could be that the BIOS supports DirectPath I/O (IOMMU) with Phenom chips and not Athlon for some reason.

[Acebmxer]

Re: ESXi DirectPath I/O on 970A-UD3

Okay yeah your best bet would to to contact gigabyte directly and see if they can give you a beta bios. It just may come down to the cpu also since you say others who had success were on another cpu.

One thing you can try is when sitting at the main screen of the bios press ctrl+F1 the screen should flash. Look to see if you see any added features, possibly in intergraded features or anywhere else in the bios. When VT-D support was added I had to access it this way.

[Lsdmeasap]

Re: ESXi DirectPath I/O on 970A-UD3

You could also directly contact the users that posted the info you linked that have the same boards as you do.

Then you could see if they had to do anything special, if they have any tips for you, or if they think your CPU might be the reason this isn't working for you.

[Christopher Piper]

Re: ESXi DirectPath I/O on 970A-UD3

Oh, I have network access. I just want to be able to pass some of my PCI devices directly into a VM, which requires DirectPath I/O. Based on my research others have had success with this motherboard and DirectPath I/O. The difference is that they've all used Phenom II processors whereas I have an Athlon II. Based on AMD's specifications, both processors have the AMD-V support needed but I suppose the issue could be that the BIOS supports DirectPath I/O (IOMMU) with Phenom chips and not Athlon for some reason.

Did you ever get this resolved? I am trying to set up an ESXi server on this same board, and, like you, have read about success stories, and am having absolutely no luck.

[Vinster]

Re: ESXi DirectPath I/O on 970A-UD3

What hardware are you running/

[Christopher Piper]

Re: ESXi DirectPath I/O on 970A-UD3

What hardware are you running/

I'm running a GA-970A-UD3 with a Phenom II X4 965. 16GB of slowish non-ECC DDR3.

[Vinster]

Re: ESXi DirectPath I/O on 970A-UD3

and a silly question, you do have virtualization enabled in the bios right? That is a requirement isn't it.

I'm up to snuff on VM's and getting them working but I don't fully understand what ESXi is really for... everything I find is full of fluff and not at all straight forward.

but he rig my sig I'm running a VM Windows Server 2011 with an older board and CPU... same amount of RAM... it works fine... don't know if it's related.

Vin

[BenJ]

Re: ESXi DirectPath I/O on 970A-UD3

I'm running a GA-970A-UD3 with a Phenom II X4 965. 16GB of slowish non-ECC DDR3.

Not sure what Virtulization technology you use.

This is my configuration and how I got it to work.

Mother Board GA-970A-UD3
Processor - AMD FX 8350
1 Video card
1 nic card integrated on the motherboard
2 pci express nic cards that are VLan capable
(This is my HOST machine that runs two VMs)

Host - VMWare ESXi 5.1 (VM Ware Hypervisor free edition )
Guest 1 - Windows Server 2012 Essentials - Direct I/O to 1 of the available PCI NIC card
Guest 2 - Windows 8 - Direct I/O to 1 of the available PCI NIC card

I have vSphere Client installed on another Windows 8 standalone desktop to manage my Host and VMs.

Initially when I installed ESXi on my Host machine i forgot to enable IOMMU controller in BIOS / Pheripherls section. So I enabled it and rebooted the machine, thinking all will be resolved and DirectPathI/O will be available, but it still said it was not supported by the hardware.

So this is what I did and worked for me.
Reboot the machine
Hit Del to enter bios
Load optimized defaults(on the right hand corner of the main bios screen it should show the function key for this)
Confirm that SVM(by default this is enabled) under Advance CPU features is enabled.
Enabled IOMMU controller

Re installed a fresh copy of ESXi 5.1 - this i believe loaded the necessary drivers for DirectPath I/O.
Now in the VSphere Client when I connect to my above mentiond Host (VSXi 5.1) the summary shows as DirectPath I/O supported.

On VSphere Client -
Now to set your NICs to be direct I/O go to Configuration/ Advanced Settings - Add passthrouh device.
(Make sure that you dont add your NIC that is used by your HOST machine to connect to the network as passthrough. If you do so you wont be able to access the HOST over network and it will take a lot of effort to set it back)
Once you added the NICs for passthrough(DirectPath i/o) restart your host.
(for me the host didnt successfully restart, it stuck in the process with a fatel error warning, so I had to power down and restart. but it worked anyway)

Now to add the DirectPath I/O to your guest VM
Create a VM then - right click and Edit Settings - Click Add - Select PCI Device(only available when the VM is powered down) - it should list your available NICs.

Now start your guest VM and it should have all the network adapters added. It will include the virtual network you may have added in the initial set up as well.

Hope this helps to resolve your situation.

Thanks
Ben