GPU on PCIE 1x possible?

[Quad2000]

Hello together,

I have a Question regarding several GPU´s on one Mainboard as in a Mining Rig (Bitcoin, Ethereum, and so on).

I have the following Mainboard:
GA-Z170XP-SLI
1 x PCI Express x16 slot, running at x16 (PCIEX16)
1 x PCI Express x16 slot, running at x8 (PCIEX8)
1 x PCI Express x16 slot, running at x4 (PCIEX4)
2 x PCI Express x1 slots
2 x PCI slots

At the Moment there are 1 x RX480 and 1x R9 390 in use.
Question: can I add more GPU´s to that System? Does have the PCIE Speed (x8, x4, x1) have a big Impact in the Performance or can I add 5 PCIE GPU´s?
Thank you very much for your help!

[MeeLee]

Yes, pcie 3.0 x4 speeds or higher are recommended for core 22.
You could install Linux, and run any AMD or Nvidia GPU with an equivalent performance of a GTX 1650-1660 into an x1 slot.
If you run a GPU with equivalent of RTX 2060 in an x1 slot, even in Linux, you'll experience a performance penalty of well over 15%. Some others might chime in with the exact performance losses.

[HaloJones]

Under Linux you might get some points but on Windows it's a minimum of 4x really.

Your top PCIE will downclock to 8x if you plug another card in the second slot which will run at 8x.
The third PCIE slot will run at only 1x if the second slot is occupied.

So with five cards you would have 2x8 and 1x1 and maybe two others at 1x. So three of your five would be running at a very slow transfer rate. Try it and let us know.

What other cards are you planning on getting? IME, it's better to have no more than two cards per machine.

That said, with really expensive motherboards - X99, X299, X399 - it's possible to use weird cards connected to the NVME slots to use those PCIE lanes for running gpus. There is a threadin the Hardware section on here. Although it wouldn't work with a Z170 I think.

[Quad2000]

Thank you for your answer.
In this post you can see a mining Rig and he use 6 x PCIE 1x for every card. For Mining Code 1x is OK. This is the reason why I ask this Question here
https://www.ebay.de/itm/2x-Mining-Rig-i ... OSwxBNeqCX

[HaloJones]

Mining does little data transference I believe whereas FAH does a fair bit. I've not seen any impact with PCIE x4 under Linux but I've seen discussions on here of people who have tried and failed with 1x. The threads should still be here I suspect

EDIT: here

[Quad2000]

Thank you @ all! This Forum is greate!
I only use at max. 3 Cards. Then I should not have Performance Problems. Thanks a lot!

[Nuitari]

I run quite a few systems that are doing F@H.
2 of them are multi RX570 rigs that were previously used for mining.

Rig1:
5x RX570 (all on 1x risers)
1x RX560 (direct 16x)
CPU is an AMD A8-9600, which also shows up as a Carrizo GPU which can fold.

Rig2:
1x RX570 direct 16x
2x RX570 through risers
1x RX560 through a PCI 4x to 4 1x splitter
1x RX560OC through the same splitter.

My home desktop:
Ryzen 7 3700X 8 core
1x NVIDIA 1660 Super
1x NVIDIA 1050ti

2nd computer:
1x Phenom 9950 (quad core)
1x Nvidia 1060 3GB

(and a few others that can only do CPU)

A few days ago, starts lined up and I got a ton of WU for 14549 which allowed me to compare various effects on the GPU and these are my notes:
- TPF is about 20s slower on the cards with the 1x riser
- Stopping the CPU folding on the 2 big multi gpu rig boosted the speed of the GPUs, so there is some resource contention
- Plan on 2gb per GPU for the core 22 usage.
- NVIDIA seem to require much more CPU
- Bus usage with the NVIDIA cards in my desktop turns between 35% and 50% depending on the project. This is pcie 3.0 16x for both of them.

I'm still trying to get amdgpu-utils to gather telemetry and more precise information.

[MeeLee]

A lot of budget motherboards, having a x16/x8/x4 and 3x x1 slots, actually don't handle over 4 GPUs.
Even if you somehow could run the first GPU at x8, the second at x4, the third at x4, you'd have 1 PCIE lane left.
Adding 1 more GPU to a pcie lane, will show there aren't enough resources available, and you'll have to remove 1 GPU.

[Nuitari]

The rig with 6 GPU has a ASRock AB350 Pro4, the one with the splitter an Asus SABERTOOTH 990FX, and yes adding a 6 gpu on that one fails weirdly with some random pcie device missing at poweron.

[Quad2000]

OK very good informations. This is exactly what I want to know. Thank you!!!
Is there a tool or something else to make a benchmark related to f@h? Depending on the work package the folding Performance have a big deviation. From my Point of view I can not compare the Performance of my the 2 GPU´s. I want to check this because of overclocking.
Is the performance differenz caused by the different Projects or only by the WU´s?

[bruce]

FAHBench is probably what you're looking for.

Yes, there are variations depending on the project, but individula WUs within the same project are nearer in PPD than between projects.

Regarding overclocking, FAH tends to push portions of the hardware more than several of the traditional benchmarks. (officially, OC is not supported) Use caution.

[Nuitari]

There are variations between projects and between the WUs of a project itself. Its hard to compare but you can get a decent idea.

Avoid OC, and also avoid undervolting. These on the RX570s are asking for trouble.

[Quad2000]

Thank you for the Tipp. I have testet the following GPU´s with the following FAHBench setup:
Compute: OpenCL
Precission: single
WU: dhfr-implicit
Accuracy Check: Enabled
NaN Check: 0 - Disabled
Run length: 1 min

GPU: Score/Scaled:
R9 390: 230.513/18.9173
RX480: 272.638/22.3743
RX580: 294.886/24.2001
Vega 64: 477.156/39.1583
1070: 514.609/42.2319

In this benchmark it Looks like the 1070 is faster then the Vega 64. In real life the Vega 64 is approximately 20 to 30% faster (more PPD). Can somebody explain this? For me the results doesn´t show the real life in Our Team.

[Nuitari]

On some motherboards, especially the ones with PCIe 3.0, it is possible that the riser will be unstable. With AMD, I have to limit the cards to the slowest PCIe bus rate that is configurable for it. Otherwise I get tons of weird issues with WUs hanging.

[Nuitari]

Another data point, I had a cluster of WUs on project 16435 that got spread out on a few RX570.

RX 570 w/ Phenom on x16 bus
project:16435 run:3431 clone:1 gen:4 93035
project:16435 run:1920 clone:1 gen:6 95981
project:16435 run:700 clone:2 gen:5 93178

RX 570 w/ Phenom on x1 bus
project:16435 run:1963 clone:0 gen:5 87544.6 (30 minutes network delay because of CS server issue) (1st card)
project:16435 run:796 clone:4 gen:4 86972.6 (30 minutes network delay because of CS server issue) (2nd card)

RX 570 w/ A8-9600, all on 1x riser cards
card 1:
project:16435 run:2926 clone:1 gen:3 92120
card 2:
project:16435 run:3073 clone:0 gen:2 92921
project:16435 run:2002 clone:3 gen:6 88660
card 3:
project:16435 run:314 clone:0 gen:12 89975

So at worst there is a 3% loss in PPD with the riser. However it does allow for a much more compact setup to run multiple GPUs.
The network delay does make it harder to see how the CPU influences things. The A8 runs 7 GPUs, vs the Phenom with 4, so just that can have a big impact