Aurum wrote:.... If the GPU Folding Projects - Performance database is to be believed I should've seen about 2.84 million PPD...
The important thing you're missing in that sentence (and your understanding of how/why that db exists) is "if I fold the same PRCGs that were used and my cards are clocked the same as those used".
The entire point of that db was to look at PRCG xxx and compare your folding of PRCG xxx to someone else's folding of PRCG xxx and see if you're having issues.
The entire sample size in that db for a GTX 1080 is still less than 2 days worth of folding.
Aurum wrote:
The weak link in the chain is the Xeon E5-2603 CPU. I picked it becuase it was the cheapest 40-lane CPU at $229. I tried folding on 2 cores and not CPU folding with no difference. I was watching the CPU Usage on Windows Task Manager and it was consistently about 86% with no core pegged at 100%. If I CPU folded on 3 cores it did max out at 100%.
For my understanding you need 1 thread per GPU. As you use 4 GPUs you need a CPU which supports 5 threads to get full support.
So your Xeon might be the weak link in the chain but for other reasons you thought/state.
Try to use it with 3 cards only and see if you come closer to your reference values.
PS. I wouldn't believe so much in this document. It bases on some errors in reasoning. But for a quick look it's good enough.
Aurum wrote:.... If the GPU Folding Projects - Performance database is to be believed I should've seen about 2.84 million PPD...
The important thing you're missing in that sentence (and your understanding of how/why that db exists) is "if I fold the same PRCGs that were used and my cards are clocked the same as those used".
The entire point of that db was to look at PRCG xxx and compare your folding of PRCG xxx to someone else's folding of PRCG xxx and see if you're having issues.
The entire sample size in that db for a GTX 1080 is still less than 2 days worth of folding.
I'm pretty sure you are confusing two different databases. Click the link and see.
Sample Size for 1080 is currently 18 WUs. That should be around 2 or 3 days of folding...
There is an error in the calculation of the average value as well therefore values might differ from yours
Another way to think of it is add in the PassMark CPU benchmark. The E5-2603 is too weak for four 1070s. The question is where to draw the dotted line for minimum acceptable performance between the E5-2603 and the E5-1650 (comparable to the i7-5930K), all with 40 PCIe 3.0 lanes. Any recommendations
Edit: Ed mentions using an E5-1620 with 3 Titan X Pascals on an X99 board that should run x16x16x8. Then he went to i7-5930 and i7-6850. viewtopic.php?f=38&t=28847&start=105#p292541
Last edited by Aurum on Tue Mar 28, 2017 8:01 pm, edited 2 times in total.
I don't think that you can conlcude folding recommendations from PassMark CPU benchmark.
I don't think that a E5-2603 is too weak for four 1070. As long as someone proves me wrong
Aurum wrote:I believed the claims when I built my last rig that as long as the CPU has one core per GPU you'll get full performance. My experience seems to prove otherwise.
Intel Xeon E5-2603 v4 ($229, AVX 2.0, 6 cores & 40 PCIe 3.0 lanes) with CoolMaster EVO 212 https://ark.intel.com/products/92993/In ... -2603%20v4
Gigabyte GA-X99-SLI ($124, has four 16x slots that run x8x8x16x8 3.0 straight off the CPU) http://uk.gigabyte.com/Motherboard/GA-X99-SLI-rev-10#ov
Quadkit DDR4 2666 G.SKILL Ripjaws 4
250 GB M.2 SSD
Corsair AX1200 PSU
Four Gigagbyte cards: GV-N1070G1 (x8), GV-N1070G1 (x8),GV-N1080G1 (x16), and GV-N1070G1 (x8)
Open tray, no case and NO risers.
I started folding for a few days expecting 2.4 million PPD and never saw higher than 2.0 million PPD. If the GPU Folding Projects - Performance database is to be believed I should've seen about 2.84 million PPD. https://docs.google.com/spreadsheets/d/ ... utput=html
I then swapped out the 1080 for a GV-N1070G1 and saw the same level of PPD performance.
I then installed the Gigabyte Xtreme Gaming Engine and tried each of its 3 OC modes (Eco, Gaming & OC Mode) to all four cards. I saw no increase in PPD with OCing.
The weak link in the chain is the Xeon E5-2603 CPU. I picked it becuase it was the cheapest 40-lane CPU at $229. I tried folding on 2 cores and not CPU folding with no difference. I was watching the CPU Usage on Windows Task Manager and it was consistently about 86% with no core pegged at 100%. If I CPU folded on 3 cores it did max out at 100%.
(Note: My dual-GPU rigs are much more forgiving of the CPU.)
So I searched for FCLGA2011-3 CPUs with 40 lanes and had to look one by one as Intel does not search for that feature.
i7-6850K $610.00
i7-5930K $621.00
i7-6900K $1,050.00
i7-5960X $1,073.00
i7-6950X $1,650.00
Anyone know if there's a cheaper Xeon with 40 PCIe lanes
If you want to go Xeon you have two choices, get something like a E5-2630v4 or get an older v3 chip, you aren't going to lose much and you may save a bit. Also bear in mind that your current xeon will sell for decent money so you will be able to offset that against the price of the new cpu
The E5-2603 has 4 cores, and does not have HT. So if a core per GPU is reserved, that leaves no additional cores for CPU folding. There will be some OS overhead competing on those 4 cores with servicing the GPU's.
iMac 2.8 i7 12 GB smp8, Mac Pro 2.8 quad 12 GB smp6
MacBook Pro 2.9 i7 8 GB smp3
Aurum wrote:.... If the GPU Folding Projects - Performance database is to be believed I should've seen about 2.84 million PPD...
The important thing you're missing in that sentence (and your understanding of how/why that db exists) is "if I fold the same PRCGs that were used and my cards are clocked the same as those used".
The entire point of that db was to look at PRCG xxx and compare your folding of PRCG xxx to someone else's folding of PRCG xxx and see if you're having issues.
The entire sample size in that db for a GTX 1080 is still less than 2 days worth of folding.
I'm pretty sure you are confusing two different databases. Click the link and see.
Yeah, no....
TheBlademaster01 wrote:This database was created to compare frame times with other users in order to see whether your PPD is where it should be or if there is something wrong with the system (instability, incorrect client configuration, throttling etc.)
As for my sample size comment, the entire sample size of the db for the GTX 1080s is 18 WUs (my personal experience is an average of 11 WUs per day with a GTX 1080).
Joe_H wrote:The E5-2603 has 4 cores, and does not have HT. So if a core per GPU is reserved, that leaves no additional cores for CPU folding. There will be some OS overhead competing on those 4 cores with servicing the GPU's.
ComputerGenie wrote:Yeah, no....
As for my sample size comment, the entire sample size of the db for the GTX 1080s is 18 WUs (my personal experience is an average of 11 WUs per day with a GTX 1080).
Thanks, I now see I was misreading the db by only looking at the first screen it displayed. Sorting by GPU gives a much more clear story.
Gigabyte GA-X99-SLI ($124, has four 16x slots that run x8x8x16x8 3.0 straight off the CPU)
Run GPU-Z, and hover over "Bus Interface" field. It will show you what the lane usage (number of lanes and version) at which GPUs are is running during operation. Does "8X8X16X8" reflect what each slot's capability is when all four slots are populated? (If so, that's an impressive motherboard!)
I find it strange that your 6-core CPU would be a bottleneck. It would seem to me it has more than enough resources to feed all four GPUs, with two cores and speed to spare.
We have a a pretty rich thread here concerning multi-GPU and adequate lane assignments. Sorry, I couldn't find it at the moment.
Gigabyte GA-X99-SLI ($124, has four 16x slots that run x8x8x16x8 3.0 straight off the CPU)
Run GPU-Z, and hover over "Bus Interface" field. It will show you what the lane usage (number of lanes and version) at which GPUs are is running during operation. Does "8X8X16X8" reflect what each slot's capability is when all four slots are populated? (If so, that's an impressive motherboard!)
I find it strange that your 6-core CPU would be a bottleneck. It would seem to me it has more than enough resources to feed all four GPUs, with two cores and speed to spare.
We have a a pretty rich thread here concerning multi-GPU and adequate lane assignments. Sorry, I couldn't find it at the moment.
GPU-Z 1.18.0 reports while running:
GPU0: PCIe x16 3.0 @ x8 3.0
GPU1: PCIe x16 3.0 @ x8 3.0
GPU2: PCIe x16 3.0 @ x16 3.0
GPU3: PCIe x16 3.0 @ x8 3.0
That's 8+8+16+8=40 lanes as mb manual says.
The downside of this MB is that they placed the M.2 SSD socket under GPU1 and it chronically overheats. I've got a 120 mm fan sitting on top of the cards blowing down but it's not enough. I have a wall of three 120 mm fans across the back of the cards as well. I'm going to try adding a fan on the side.
What does the bus usage look like in gpu-z sensors tab? Also, what boost frequency are the cards running at (and the perfcap reasons)? Is it any different to when only 2 cards are running?
I've stuck with dual GPU setups for ease of cooling and cheapness of motherboard / CPU, but even lowly 2C/2T pentium and celerons haven't dented a pair of 1080s to the extent you're seeing. I've even downclocked to save a few extra watts.
Also, is booting to linux an option to see if things are improved there?
Aurum wrote:...
The downside of this MB is that they placed the M.2 SSD socket under GPU1 and it chronically overheats. I've got a 120 mm fan sitting on top of the cards blowing down but it's not enough. I have a wall of three 120 mm fans across the back of the cards as well. I'm going to try adding a fan on the side.
https://www.amazon.com/gp/product/B01E8YS122 << I have an "open air" rig, but I use 1 of those (plugged into MB and blowing "front to back") to cool 2 1080s in a room that sometimes hits above 90°F.
