Folding on Two GPUs Causes Low Performance

[compdewd]

A quick Google search did not result in what I needed to find so here's a new topic.

I have been running two GTX 460s in non-SLI since I obtained them and I noticed that the TPF and PPD were considerably low (7000-9000 PPD) compared to the entries found here. I thought that the automatic downclocking of the cards was the problem (which I resolved in this thread), but fixing that problem did not fix the bigger problem of poor performance. I was tempted to post on this forum asking for help, but I wanted to try something first.

For some reason, I thought that having the two cards folding at the same time was causing the poor performance. Sure enough, that's what it was. I removed one of the GTX 460s and, coincidentally, the TPF was split (exactly, I believe) in half. The single GTX 460 is running happily at 17000 PPD.

Now why does running two identical GTX 460s cause each 460 to lose half of its performance? Is this a driver problem? I switched to a previous driver trying to figure this poor performance out and that didn't fix it. I know people fold with multiple identical cards together, so I assume that it's not a software bug, but rather a misconfiguration on my end. Now the question is: what is wrong with my setup?

I am running SMP:6 and trying to run two GTX 460s in non-SLI, and I also have a third card that doesn't fold running two monitors. One 460 has one monitor plugged in and the other 460 does not. Previous driver version was 314.xx and currently is 306.23. The 460s' GPU clocks are 781 MHz core, 900 MHz memory, and 1560 MHz shader. The "System" section of the log is below.

Code: Select all

03:22:20:************************* Folding@home Client *************************
03:22:20:      Website: http://folding.stanford.edu/
03:22:20:    Copyright: (c) 2009-2013 Stanford University
03:22:20:       Author: Joseph Coffland <joseph@cauldrondevelopment.com>
03:22:20:         Args: --open-web-control
03:22:20:       Config: C:/Users/Patrick/AppData/Roaming/FAHClient/config.xml
03:22:20:******************************** Build ********************************
03:22:20:      Version: 7.3.6
03:22:20:         Date: Feb 18 2013
03:22:20:         Time: 15:25:17
03:22:20:      SVN Rev: 3923
03:22:20:       Branch: fah/trunk/client
03:22:20:     Compiler: Intel(R) C++ MSVC 1500 mode 1200
03:22:20:      Options: /TP /nologo /EHa /Qdiag-disable:4297,4103,1786,279 /Ox -arch:SSE
03:22:20:               /QaxSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 /Qopenmp /Qrestrict /MT /Qmkl
03:22:20:     Platform: win32 XP
03:22:20:         Bits: 32
03:22:20:         Mode: Release
03:22:20:******************************* System ********************************
03:22:20:          CPU: AMD FX(tm)-8120 Eight-Core Processor
03:22:20:       CPU ID: AuthenticAMD Family 21 Model 1 Stepping 2
03:22:20:         CPUs: 8
03:22:20:       Memory: 3.98GiB
03:22:20:  Free Memory: 2.94GiB
03:22:20:      Threads: WINDOWS_THREADS
03:22:20:  Has Battery: false
03:22:20:   On Battery: false
03:22:20:   UTC offset: -4
03:22:20:          PID: 4856
03:22:20:          CWD: C:/Users/Patrick/AppData/Roaming/FAHClient
03:22:20:           OS: Windows 8 Pro
03:22:20:      OS Arch: AMD64
03:22:20:         GPUs: 2
03:22:20:        GPU 0: NVIDIA:2 GF104 [GeForce GTX 460]
03:22:20:        GPU 1: UNSUPPORTED: G84 [GeForce 8400 GS]
03:22:20:         CUDA: 2.1
03:22:20:  CUDA Driver: 5000
03:22:20:Win32 Service: false
03:22:20:***********************************************************************

(The second 460 would take the place of "GPU 1")

Any help would be greatly appreciated

[bruce]

If you're certain that FAH saturates the shaders in both GPUs, then performance of two cards should be double what one GPU can do. If you're certain that FAH saturates the PCIe bus, constantly waiting for data transfers, then total performace of two GPUs should be the same as one (data transferring to each card at half the total rate).

Conventional wisdom assumes the former and says PCIe speed doesn't matter but what if that's wrong? I don't know how to prove either case. I only know that performance is always limited by SOMETHING being saturated.

[compdewd]

Hey Bruce, thanks for responding. I know that when there is a second card in my motherboard's PCIe slot that the speed of that card is always x8. I do not know of the first slot also runs at x8 when there is a second card. I do know that by itself, the first card will run at x16. I read recently an old topic about the PCIe speed not being a concern and that x2 is only what is necessary to run FAH, but I assume that things have changed in the past few years about that.

If I put the second card back in and GPU-Z tells me that the first card is then running at x8, I think we could conclude that it is necessary to have greater than x8 speed to run FAH (and I would need to then upgrade ASAP ). If GPU-Z tells me that the first card runs at x16 speed, then I suppose we're back to square one with this problem.

I'll put the second card back in and give my results. I also have a GTX 650 Ti I could throw in and see what happens then.

[compdewd]

Alright, well GPU-Z says the first two slots with two identical GTX 460s are running at x8 speed each. I guess the mystery is solved, though the truth is not favorable :/

[P5-133XL]

Check the % GPU usage on both cards while folding. They should both be a 99%.

I have run 2x GTX 460's on a computer for years. There should not be a slowdown adding the second card regardless of the PCI-e bus only using 8 lanes.

[compdewd]

One of the cards reports 99% and the other reports 0% while folding, however both are performing the same, earning an estimated 8500-9000 PPD. I can confirm that the TPF is split exactly in half when running just one of the GTX 460s rather than both.

[k1wi]

So it sounds like you're folding two clients on one of your GPUs and none on the other. You need to check your configuration and get the second client aligned to the second card

[Amaruk]

One of the cards reports 99% and the other reports 0% while folding, however both are performing the same, earning an estimated 8500-9000 PPD. I can confirm that the TPF is split exactly in half when running just one of the GTX 460s rather than both.

I'm with k1wi on this one - both slots are running on one card, the one at 99%.

...trying to run two GTX 460s in non-SLI, and I also have a third card that doesn't fold...

While V7 shows your GeForce 8400 GS as unsupported, it is still a CUDA-capable (1.1) card.

Have you tried removing the 8400 to see if both the 460s fold?

[compdewd]

I think you guys were right! I had manually changed the opencl-index and cuda-index both to 0 as mentioned in this thread and it started folding rather than keep throwing GPU_MEMTEST_ERRORS. I figured everything was fine, except that two slots were then configured to run on one card!

I just changed the opencl-index and cuda-index both to 1 since the card is slot 1 and it returned to throwing GPU_MEMTEST_ERRORS. This is because when the indices are 1, the slot points to the 8400 GS. I changed them both to 2 and now both GTX 460s are folding happily along

Thank, guys!

[bruce]

Finding which gpu-indices point to which device works best if you live in a kingdom that supports magic. Obviously pointing at an unsupported device is a bad idea and pointing two slots at the same device is also a bad idea. Since the client, itself, also has troubles with the default settings about half the time when there are mixed GPU types, all I can say is good luck. Microsoft does not enumerate devices the same way as the client.

At least the half-speed mystery is solved ... and the theory that PCIe speed is not all that important is still a pretty good theory.