Dedicated rig High end GPU Low end CPU?

[rwh202]

Today I got myself a powermeter and the results are 200W at bootup, 500W when folding. Wish I had the meter before I upgraded the drivers.
It´s nice to see that the new drivers keeps the load at 100%, no fluctuation as I had before. Increase of PPD seems to be 15%+, really nice.

I was getting keen to try a Fury until I saw those numbers - Are they just for just the folding PC? And this is a single Fury-X? If so, then wow, they're thirsty!

[JonasTheMovie]

At the time the WU had a calculated ppd at 480k, right now it´s around 460W for 380k. I might get another power meter to verify.
That the card draws 300W at full load is expected with a TDP of 275W.

[rwh202]

At the time the WU had a calculated ppd at 480k, right now it´s around 460W for 380k. I might get another power meter to verify.
That the card draws 300W at full load is expected with a TDP of 275W.

Thinking about it, I'd be inclined to suspect the power meter - it may be showing apparent power not real power and be influenced by the power factor of the PSU. 200 W base load for the system is too high and 500 W total is a lot of heat to expel from a small enclosure - the vents would feel like a jet exhaust.
For comparison, I've got an old i7 rig with dual GPUs and a bronze PSU that idles at 80 W and still only draws 420 W for 750k

[bruce]

The power meter should have a choice between displaying Watts or Volt-amperes.

[JonasTheMovie]

The powermeter I have has no settings to compensate for reactive power, it always show W, as in real power. But it reads out a power factor of 90, as in 0.90, which concurs with the voltage and amperage. Still seems high with 200W at boot. I kind of regret buying this PSU, its a pos, already changed the fan to keep it quiet, not only the fan but the controller circuit made noise.

[Nathan_P]

I did recommend buying a decent platinum rated PSU but even so, a system pulling 200w at idle doesn't sound right. My daily rig with a pair of x5670's, 2 SSD's, 2 HDD's, 2 optical drives and a GTX 670 only pulls 140w at idle and my 980ti/970 rig running on a Z9PE D8 idles at 100w.

Bear in mind that TDP is the card running flat out and they always err on the side of caution, my 970 has a TDP of 160w yet under a full folding load its only ever pulled a max of 140w according to my power meter, however a fury x can pull over 400w at full folding load so your 500w number is not far off.

[JonasTheMovie]

Well, it idles at 120W with everything fully loaded and folding paused. 200W was at bootup as I wrote, right before core kicked in and shot it up to 500W. Maybe I should of made that more clear. So the Fury seems to draw 300-400W depending on workload. It´s nice to see my average going up to 375kppd, but it should come up a bit more in a couple of days, the rig had some downtime. With electricity costs here in northern Sweden being quite low (you can heat your house on electricity with freezing winter outside) the efficiency wasn´t my concern, but the build quality of a platinum PSU would of been nice.

[FldngForGrandparents]

Couple of things to add that I have not seen discussed here. First my credentials as I am not just a self proclaimed guru. I am a systems engineer for a large mobile company. I am a RHCE/RHCSA and Windows admin with many years under my belt as the primary decider for hardware decisions of 10's of thousands of servers. So I tend to test and research plenty before I make decisions. Even on my home gear.

I am running dedicated folding machines. I am currently running GTX 970/980 reference and Asus Strix GTX 980Ti OC's with AMD 8350 and Phenom II X6 1100T CPU's all under Ubuntu 14.04 and only doing GPU folding. The OS does play a big part in your PPD. I saw 30-40k per card performance gain over Windows 7/10.

Another thing I see people on here saying is get the cheapest CPU when using GPU's. That is not always the case. As Nvidia GPU's require a lot CPU core for each card. My testing has shown that I gain an additional 20k or so of PPD per card on my 8350 compared to the 1100T CPU. I haven't had the resources to fully test if this is due to the CPU core speed increase (500Mhz on boost) or the bus/cache. I'll try with a fx-9590 next to see how much extra I get.

My next testing will be to see what the difference is between PCIE 1x and 16x is and if this will have any impact with running with Nvidia GPU's. I do not have any AMD GPU's as they require much more power and I have not seen compelling evidence that the cost/PPD is any better.

One last note about power. One of my machines I am testing/tuning with is a AMD Phenum II 1100T with a GTX 980 reference and the Asus 980Ti OC. Both cards have a heavy overclock. This machine also has a SSD and two 3TB drives/CD-ROM. I am running with a 700w PS without any issues. nvidia-smi shows 140w draw on GTX 980 and 240w on GTX 980Ti.

I have been buying used GTX cards from craigslist for smoking deals. Definitely a good place to pick them up and save some cash.

I will be posting a full how-to for building a dedicated Ubuntu folding server in the next week. As the docs I see are usually pieced together and don't include things like how to over-clock, additional monitoring tools, etc.

Brad

[PS3EdOlkkola]

I do not have any AMD GPU's as they require much more power and I have not seen compelling evidence that the cost/PPD is any better.

Regarding PCIe bus width and speed using AMD GPUs (Fury X) on AMD processors (FX-8350) and Intel processors (4771 and 4790K) the average PPD results are remarkably close:

All systems: Win7 x64, 16GB+ DDR3 (CAS 9), no paging file, SSDs, no SMP folding enabled, Fury X at stock GPU clock (1.050 GHz), 1,000+ watt platinum power supplies.

Project 9172:

FX-8350 on PCIe 2.0 8x: Average PPD = 337,768
i7-4771 on PCIe 3.0 8x: Average PPD = 337,768
i7-4790K on PCIe 3.0 8x: Average PPD = 342,604

It surprised me that the i7-4771 was exactly the same as the FX-8350 on an interface that is half as fast as the Intel CPU. At least for AMD GPUs (on this wu). the speed and width of the PCIe interface is not a huge factor in PPD performance. I'm currently (re)building a i7-3930K equipped with two Fury X's on PCIe 3.0 x16 interfaces. If there is an improvement in PPD given the wider PCIe lanes, I'll post the results.

[JonasTheMovie]

PS3EdOlkkola, have you updated to Vulcan beta yet?

[mmonnin]

What is the PCI-E bandwidth usage % on your FX and i7-4771? My Win7 3770k 3.0 16x is currently using 17-19% with a GTX-970. Although my 980Ti in Ubuntu is sitting at just 1% PCI bandwidth utilization. If there was limited PCI-E bandwidth then Linux may be better for someone.

[FldngForGrandparents]

What is the PCI-E bandwidth usage % on your FX and i7-4771? My Win7 3770k 3.0 16x is currently using 17-19% with a GTX-970. Although my 980Ti in Ubuntu is sitting at just 1% PCI bandwidth utilization. If there was limited PCI-E bandwidth then Linux may be better for someone.

Interesting enough I just brought up another machine with a AMD 8150. So running a AMD 8150, 8350, and AMD Phenum II 1100T. Some cards are in 16x and some in 4x. I can see that with folding this won't make a difference. I could even use 1x as the most throughput I have seen is 120MB/s of TX and 182MB/s of RX. These are just bursts as they mostly idle at 15MB/s TX and 60MB/s RX. A 1x PCIe maxes out at 250MB/s.

I think I am going to merge all six of my GPU's under the single 8150 on my GA-990FXA-UD3 R5 motherboard. And use these adapters. http://amzn.com/B00HCBOHKO Two 750W PS should do the trick.

[7im]

There are known performance bottlenecks at 1x despite your current estimates. I personally saw a 15% PPD decrease running a GTX 760 in a 16x slot wired at 1x. Good luck running that many.

[FldngForGrandparents]

There are known performance bottlenecks at 1x despite your current estimates. I personally saw a 15% PPD decrease running a GTX 760 in a 16x slot wired at 1x. Good luck running that many.

Well I just ordered a couple of these: http://amzn.com/B00IMYODGS I will see what the performance difference is in a couple days.

[PS3EdOlkkola]

@JonasTheMovie - I don't use beta drivers. I have too many rigs to maintain so I keep everything on the same driver version. Once a new driver has been proven to be stable and their is a performance improvement and it's officially WHQL released, then I'll upgrade. I've been bitten too many times in the past on upgrading drivers too soon, so I take a more conservative approach.

@ mmonnin - from what I recall, front-side bus utilization was very low using AMD/ATI GPUs, approximately 5% to 10% loading.