Folding on Two GPUs Causes Low Performance

Moderators: Site Moderators, FAHC Science Team

Post Reply
compdewd
Posts: 165
Joined: Sat Jun 09, 2012 6:56 am
Hardware configuration: [1] Debian 8 64-bit: EVGA NVIDIA GTX 650 Ti, MSI NVIDIA GTX 460, AMD FX-8120
[2] Windows 7 64-bit: MSI NVIDIA GTX 460, AMD Phenom II X4
Location: Cincinnati, Ohio, USA
Contact:

Folding on Two GPUs Causes Low Performance

Post by compdewd »

A quick Google search did not result in what I needed to find so here's a new topic.

I have been running two GTX 460s in non-SLI since I obtained them and I noticed that the TPF and PPD were considerably low (7000-9000 PPD) compared to the entries found here. I thought that the automatic downclocking of the cards was the problem (which I resolved in this thread), but fixing that problem did not fix the bigger problem of poor performance. I was tempted to post on this forum asking for help, but I wanted to try something first.

For some reason, I thought that having the two cards folding at the same time was causing the poor performance. Sure enough, that's what it was. I removed one of the GTX 460s and, coincidentally, the TPF was split (exactly, I believe) in half. The single GTX 460 is running happily at 17000 PPD.

Now why does running two identical GTX 460s cause each 460 to lose half of its performance? Is this a driver problem? I switched to a previous driver trying to figure this poor performance out and that didn't fix it. I know people fold with multiple identical cards together, so I assume that it's not a software bug, but rather a misconfiguration on my end. Now the question is: what is wrong with my setup?

I am running SMP:6 and trying to run two GTX 460s in non-SLI, and I also have a third card that doesn't fold running two monitors. One 460 has one monitor plugged in and the other 460 does not. Previous driver version was 314.xx and currently is 306.23. The 460s' GPU clocks are 781 MHz core, 900 MHz memory, and 1560 MHz shader. The "System" section of the log is below.

Code: Select all

03:22:20:************************* Folding@home Client *************************
03:22:20:      Website: http://folding.stanford.edu/
03:22:20:    Copyright: (c) 2009-2013 Stanford University
03:22:20:       Author: Joseph Coffland <joseph@cauldrondevelopment.com>
03:22:20:         Args: --open-web-control
03:22:20:       Config: C:/Users/Patrick/AppData/Roaming/FAHClient/config.xml
03:22:20:******************************** Build ********************************
03:22:20:      Version: 7.3.6
03:22:20:         Date: Feb 18 2013
03:22:20:         Time: 15:25:17
03:22:20:      SVN Rev: 3923
03:22:20:       Branch: fah/trunk/client
03:22:20:     Compiler: Intel(R) C++ MSVC 1500 mode 1200
03:22:20:      Options: /TP /nologo /EHa /Qdiag-disable:4297,4103,1786,279 /Ox -arch:SSE
03:22:20:               /QaxSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 /Qopenmp /Qrestrict /MT /Qmkl
03:22:20:     Platform: win32 XP
03:22:20:         Bits: 32
03:22:20:         Mode: Release
03:22:20:******************************* System ********************************
03:22:20:          CPU: AMD FX(tm)-8120 Eight-Core Processor
03:22:20:       CPU ID: AuthenticAMD Family 21 Model 1 Stepping 2
03:22:20:         CPUs: 8
03:22:20:       Memory: 3.98GiB
03:22:20:  Free Memory: 2.94GiB
03:22:20:      Threads: WINDOWS_THREADS
03:22:20:  Has Battery: false
03:22:20:   On Battery: false
03:22:20:   UTC offset: -4
03:22:20:          PID: 4856
03:22:20:          CWD: C:/Users/Patrick/AppData/Roaming/FAHClient
03:22:20:           OS: Windows 8 Pro
03:22:20:      OS Arch: AMD64
03:22:20:         GPUs: 2
03:22:20:        GPU 0: NVIDIA:2 GF104 [GeForce GTX 460]
03:22:20:        GPU 1: UNSUPPORTED: G84 [GeForce 8400 GS]
03:22:20:         CUDA: 2.1
03:22:20:  CUDA Driver: 5000
03:22:20:Win32 Service: false
03:22:20:***********************************************************************
(The second 460 would take the place of "GPU 1")

Any help would be greatly appreciated :)
bruce
Posts: 20824
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Folding on Two GPUs Causes Low Performance

Post by bruce »

If you're certain that FAH saturates the shaders in both GPUs, then performance of two cards should be double what one GPU can do. If you're certain that FAH saturates the PCIe bus, constantly waiting for data transfers, then total performace of two GPUs should be the same as one (data transferring to each card at half the total rate).

Conventional wisdom assumes the former and says PCIe speed doesn't matter but what if that's wrong? I don't know how to prove either case. I only know that performance is always limited by SOMETHING being saturated.
compdewd
Posts: 165
Joined: Sat Jun 09, 2012 6:56 am
Hardware configuration: [1] Debian 8 64-bit: EVGA NVIDIA GTX 650 Ti, MSI NVIDIA GTX 460, AMD FX-8120
[2] Windows 7 64-bit: MSI NVIDIA GTX 460, AMD Phenom II X4
Location: Cincinnati, Ohio, USA
Contact:

Re: Folding on Two GPUs Causes Low Performance

Post by compdewd »

Hey Bruce, thanks for responding. I know that when there is a second card in my motherboard's PCIe slot that the speed of that card is always x8. I do not know of the first slot also runs at x8 when there is a second card. I do know that by itself, the first card will run at x16. I read recently an old topic about the PCIe speed not being a concern and that x2 is only what is necessary to run FAH, but I assume that things have changed in the past few years about that.

If I put the second card back in and GPU-Z tells me that the first card is then running at x8, I think we could conclude that it is necessary to have greater than x8 speed to run FAH (and I would need to then upgrade ASAP ;)). If GPU-Z tells me that the first card runs at x16 speed, then I suppose we're back to square one with this problem.

I'll put the second card back in and give my results. I also have a GTX 650 Ti I could throw in and see what happens then.
compdewd
Posts: 165
Joined: Sat Jun 09, 2012 6:56 am
Hardware configuration: [1] Debian 8 64-bit: EVGA NVIDIA GTX 650 Ti, MSI NVIDIA GTX 460, AMD FX-8120
[2] Windows 7 64-bit: MSI NVIDIA GTX 460, AMD Phenom II X4
Location: Cincinnati, Ohio, USA
Contact:

Re: Folding on Two GPUs Causes Low Performance

Post by compdewd »

Alright, well GPU-Z says the first two slots with two identical GTX 460s are running at x8 speed each. I guess the mystery is solved, though the truth is not favorable :/
P5-133XL
Posts: 2948
Joined: Sun Dec 02, 2007 4:36 am
Hardware configuration: Machine #1:

Intel Q9450; 2x2GB=8GB Ram; Gigabyte GA-X48-DS4 Motherboard; PC Power and Cooling Q750 PS; 2x GTX 460; Windows Server 2008 X64 (SP1).

Machine #2:

Intel Q6600; 2x2GB=4GB Ram; Gigabyte GA-X48-DS4 Motherboard; PC Power and Cooling Q750 PS; 2x GTX 460 video card; Windows 7 X64.

Machine 3:

Dell Dimension 8400, 3.2GHz P4 4x512GB Ram, Video card GTX 460, Windows 7 X32

I am currently folding just on the 5x GTX 460's for aprox. 70K PPD
Location: Salem. OR USA

Re: Folding on Two GPUs Causes Low Performance

Post by P5-133XL »

Check the % GPU usage on both cards while folding. They should both be a 99%.

I have run 2x GTX 460's on a computer for years. There should not be a slowdown adding the second card regardless of the PCI-e bus only using 8 lanes.
Image
compdewd
Posts: 165
Joined: Sat Jun 09, 2012 6:56 am
Hardware configuration: [1] Debian 8 64-bit: EVGA NVIDIA GTX 650 Ti, MSI NVIDIA GTX 460, AMD FX-8120
[2] Windows 7 64-bit: MSI NVIDIA GTX 460, AMD Phenom II X4
Location: Cincinnati, Ohio, USA
Contact:

Re: Folding on Two GPUs Causes Low Performance

Post by compdewd »

One of the cards reports 99% and the other reports 0% while folding, however both are performing the same, earning an estimated 8500-9000 PPD. I can confirm that the TPF is split exactly in half when running just one of the GTX 460s rather than both.
k1wi
Posts: 909
Joined: Tue Sep 22, 2009 10:48 pm

Re: Folding on Two GPUs Causes Low Performance

Post by k1wi »

So it sounds like you're folding two clients on one of your GPUs and none on the other. You need to check your configuration and get the second client aligned to the second card :)
Amaruk
Posts: 254
Joined: Fri Jun 20, 2008 3:57 am
Location: Watching from the Woods

Re: Folding on Two GPUs Causes Low Performance

Post by Amaruk »

compdewd wrote:One of the cards reports 99% and the other reports 0% while folding, however both are performing the same, earning an estimated 8500-9000 PPD. I can confirm that the TPF is split exactly in half when running just one of the GTX 460s rather than both.
I'm with k1wi on this one - both slots are running on one card, the one at 99%.
compdewd wrote:...trying to run two GTX 460s in non-SLI, and I also have a third card that doesn't fold...
While V7 shows your GeForce 8400 GS as unsupported, it is still a CUDA-capable (1.1) card.

Have you tried removing the 8400 to see if both the 460s fold?
Image
compdewd
Posts: 165
Joined: Sat Jun 09, 2012 6:56 am
Hardware configuration: [1] Debian 8 64-bit: EVGA NVIDIA GTX 650 Ti, MSI NVIDIA GTX 460, AMD FX-8120
[2] Windows 7 64-bit: MSI NVIDIA GTX 460, AMD Phenom II X4
Location: Cincinnati, Ohio, USA
Contact:

Re: Folding on Two GPUs Causes Low Performance

Post by compdewd »

I think you guys were right! I had manually changed the opencl-index and cuda-index both to 0 as mentioned in this thread and it started folding rather than keep throwing GPU_MEMTEST_ERRORS. I figured everything was fine, except that two slots were then configured to run on one card!

I just changed the opencl-index and cuda-index both to 1 since the card is slot 1 and it returned to throwing GPU_MEMTEST_ERRORS. This is because when the indices are 1, the slot points to the 8400 GS. I changed them both to 2 and now both GTX 460s are folding happily along :D

Thank, guys! :D
bruce
Posts: 20824
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Folding on Two GPUs Causes Low Performance

Post by bruce »

Finding which gpu-indices point to which device works best if you live in a kingdom that supports magic. Obviously pointing at an unsupported device is a bad idea and pointing two slots at the same device is also a bad idea. Since the client, itself, also has troubles with the default settings about half the time when there are mixed GPU types, all I can say is good luck. Microsoft does not enumerate devices the same way as the client.

At least the half-speed mystery is solved ... and the theory that PCIe speed is not all that important is still a pretty good theory.
Post Reply