PCI-e bandwidth/capacity limitations

A forum for discussing FAH-related hardware choices and info on actual products (not speculation).

Moderator: Site Moderators

Forum rules
Please read the forum rules before posting.
bruce
Posts: 20824
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: PCI-e bandwidth/capacity limitations

Post by bruce »

When your system is first initialized, the CPU and the GPU establish communications using whatever lanes are available. I don't think that number is negotiable after it has been established. If FAH belives the connection supports x16 and some lanes top working, I'm not sure what happens -- unless you reboot. If the riser supports, say 1x or 4x or 8x, it will establish that connection and it should continue to work.
gordonbb
Posts: 511
Joined: Mon May 21, 2018 4:12 pm
Hardware configuration: Ubuntu 22.04.2 LTS; NVidia 525.60.11; 2 x 4070ti; 4070; 4060ti; 3x 3080; 3070ti; 3070
Location: Great White North

Re: PCI-e bandwidth/capacity limitations

Post by gordonbb »

foldy wrote:I tried that Post-It Note thing but system did not recognize my GPU anymore when taped as x4 or below, maybe I did it wrong.
Most newer BIOSes on higher end motherboards have an option to force the PCIe version, assumably on the CPU, from Auto to 3.0, 2.0 or 1.0 so forcing 2.0 as the Maximum link speed and taping the back pins for only 8 lanes access should feed a card with the same bandwidth as PCIe 3.0 x4 and, likewise, forcing the link speed to PCIe 1.0 would cause 8 lanes to work at the same bandwidth as PCIe 3.0 x2.
Image
bruce
Posts: 20824
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: PCI-e splitter?

Post by bruce »

hiigaran wrote:To my understanding, the only time PCI-e bandwidth is used is during the initial stage of each WU, when they are assigned to the GPU.
False (at least for FAH).

OpenCL Kernels are created by the CPU and transferred to the GPU together with with blocks of data for specific portions of the processing and data may be returned to Main RAM -- at least so the checkpoint can be written to disk and for validity verification. Otherwise a pause or a crash would always restart the WU from the initial state that was originally downloaded. Also if certain errors were encountered, the processing might very well run for days before discovering a corrupt WU that couldn't be returned to the server.

It's very likely that multiple kernels are also used, but I'm not sure.

It's also true that the PCIE bus is not used continuously, but NV has decided that their WIndows OpenCL driver will use 100% of a CPU core. (Linux uses a different driver.)

That's a good question for the NV support site or the OpenCL support site. Maybe they'll say that wasting the second half of a CPU increases the frame-rate in games by xx% or something like that.
Post Reply