Plotting Overall Folding System Efficiency

A forum for discussing FAH-related hardware choices and info on actual products (not speculation).

Moderator: Site Moderators

Forum rules
Please read the forum rules before posting.
gordonbb
Posts: 511
Joined: Mon May 21, 2018 4:12 pm
Hardware configuration: Ubuntu 22.04.2 LTS; NVidia 525.60.11; 2 x 4070ti; 4070; 4060ti; 3x 3080; 3070ti; 3070
Location: Great White North

Plotting Overall Folding System Efficiency

Post by gordonbb »

I recently acquired a couple of Eaton dual conversion UPSes and Network Cards. I've managed to hook them up to a couple of folding Rigs and I have achieved Stats Nirvana.
GTX 1070Ti and RTX2070
Image
2 x GTX 1060 6GB
Image
I query the PPD from the FAHClient on the system and take that value and divide it by the UPS output wattage via SNMP from it's Network Interface and then divide by my average cost of $0.13Cdn per kW.
So the two higher end cards are much more efficient and I'm averaging about $1/day per system in Electrical usage.
Image
ProDigit
Posts: 234
Joined: Sun Dec 09, 2018 10:23 pm

Re: Plotting Overall Folding System Efficiency

Post by ProDigit »

Something appears to be off on the graph.

At $0.13/kWh
and assuming your systems are well set up,
the 1070 + 2070 will run at 1M PPD/card, and 450Watts (200Watts per card, ~50Watts for the system),
the 1060 + 1060 will run at 350k PPD/card, @ 250Watts (100Watts per card, ~50Watts for the system),

Your first system should consume about $1.4 / day,
Your second system about $0.78/day.

Your first system should run at around 1.4M PPD/$
Your second system should run at around 900k PPD/$

Minor variances in system are possible, but on average that's closer to a 1.5-1 ratio than the reported 2-1 ratio.
Most of the point differences are due to the bonus points on faster returns on the 1070/2070 system.
If bonus points were not counted, or finished WUs of same length were counted instead of PPDs, the ratio should be much closer to 1.2-1
gordonbb
Posts: 511
Joined: Mon May 21, 2018 4:12 pm
Hardware configuration: Ubuntu 22.04.2 LTS; NVidia 525.60.11; 2 x 4070ti; 4070; 4060ti; 3x 3080; 3070ti; 3070
Location: Great White North

Re: Plotting Overall Folding System Efficiency

Post by gordonbb »

No, these are measured values. The PPD agree with the numbers reported by HFM.net and Stanford and the Power agrees with measurements from a Kill-a-Watt.

The 2 1060s average 951kPPD over a 14 day period and draw 118W each (120W Limit set) so the calculated system overhead is 61.5W.

The 1070Ti averages 910kPPD and the 1070 1.23MPPD over a 14 day period and draw 138 and 168W respectively (140 and 170W power limits) so the calculated system overhead is 44.5W.

I believe the vast difference in system overhead on the Rig with the 1060s is due to the capacitors needing to be replaced on the inverter board on the UPSes which are several years old. I have to replace the fan in. The other UPS so I’ve ordered 2 fans and two sets of electrolytic capacitors and will change them both out at the same time as they are both of the same vintage.
Image
ProDigit
Posts: 234
Joined: Sun Dec 09, 2018 10:23 pm

Re: Plotting Overall Folding System Efficiency

Post by ProDigit »

odd. Most sites report the 1070 ti to run at 180Watts under load, and the 2070 at 220Watts.

My 1060 runs at about 350k PPD in a 16x slot, so the readout here is quite high at ~455k PPD per card.
rwh202
Posts: 410
Joined: Mon Nov 15, 2010 8:51 pm
Hardware configuration: 8x GTX 1080
3x GTX 1080 Ti
3x GTX 1060
Various other bits and pieces
Location: South Coast, UK

Re: Plotting Overall Folding System Efficiency

Post by rwh202 »

ProDigit wrote:odd. Most sites report the 1070 ti to run at 180Watts under load, and the 2070 at 220Watts.
Folding doesn't get the high end Nvidia GPUs near their TDP since it's not exercising everything on the card. Plus, most choose to set a lower power limit to operate more efficiently anyway.

I've got 1080 T's that don't get above 180W, despite 250W power limits and plenty of cooling.
Holdolin
Posts: 15
Joined: Thu Oct 10, 2013 3:38 am

Re: Plotting Overall Folding System Efficiency

Post by Holdolin »

While I have no idea what each of my 2070's are pulling power wise, they're not using 220W. All 3 of them are in one rig and total power draw according to my UPS is sitting around 650W. Not bad for 3.3M PPD.
Image
Mstenholm
Posts: 84
Joined: Fri Oct 22, 2010 10:17 pm
Hardware configuration: 4 x GTX 970. Win 7.

Re: Plotting Overall Folding System Efficiency

Post by Mstenholm »

I like it when somebody does make a little investigation into GPU power consumption during folding. I have been a bit lazy lately but what I found with a GTX 970 stock under Linux is that it uses 145 W (difference between idle and load) and that HWInfo report very close to real Kill-a-Watt meter under Windows for slightly OCed 1070, 1070ti and 2070. These values are 10 % lower then TPU average gaming.
I so envy your low kWh price.
Last edited by Mstenholm on Thu Jan 31, 2019 12:49 am, edited 1 time in total.
kbeeveer46
Posts: 14
Joined: Thu Jan 24, 2019 1:14 am

Re: Plotting Overall Folding System Efficiency

Post by kbeeveer46 »

Holdolin wrote:While I have no idea what each of my 2070's are pulling power wise, they're not using 220W. All 3 of them are in one rig and total power draw according to my UPS is sitting around 650W. Not bad for 3.3M PPD.
If you download the GPU-Z app and click on the Sensors tab it will show you how much power the card is pulling. I'm folding right now and my 2070 is pulling 162 W.
gordonbb
Posts: 511
Joined: Mon May 21, 2018 4:12 pm
Hardware configuration: Ubuntu 22.04.2 LTS; NVidia 525.60.11; 2 x 4070ti; 4070; 4060ti; 3x 3080; 3070ti; 3070
Location: Great White North

Re: Plotting Overall Folding System Efficiency

Post by gordonbb »

Sites may report a typical power draw but I purposely power limit the cards.

What I found during FAHBench testing, which does not take into account the Quick Return Bonus, is there is a Knee at the top of the power vs. Performance curve. So I run my cards power limited to 20W or so below the maximum theoretical limit and observe only a slight degradation in performance.

As for the 1060s performance these are installed in a Fractal Define R4 case with 6 140mm PWM fans and custom Fan Curves to balance noise and airflow so they manage to run around 1950MHz with the GPUs in the mid 60s for the lower card and low 70s for the upper card. The GPU fans are set to 50% as that is what seems to balance temperatures best between the two cards. The default fan curves work OK but tend to force the upper card to run in the high 70s.
Image
gordonbb
Posts: 511
Joined: Mon May 21, 2018 4:12 pm
Hardware configuration: Ubuntu 22.04.2 LTS; NVidia 525.60.11; 2 x 4070ti; 4070; 4060ti; 3x 3080; 3070ti; 3070
Location: Great White North

Re: Plotting Overall Folding System Efficiency

Post by gordonbb »

ProDigit wrote:... My 1060 runs at about 350k PPD in a 16x slot, so the readout here is quite high at ~455k PPD per card.
Check your temperatures and GPU clocks. I suspect GPU Boost is downclocking your card. Or Roy you have a 3GB version with less shaders?
Image
rwh202
Posts: 410
Joined: Mon Nov 15, 2010 8:51 pm
Hardware configuration: 8x GTX 1080
3x GTX 1080 Ti
3x GTX 1060
Various other bits and pieces
Location: South Coast, UK

Re: Plotting Overall Folding System Efficiency

Post by rwh202 »

gordonbb wrote:
ProDigit wrote:... My 1060 runs at about 350k PPD in a 16x slot, so the readout here is quite high at ~455k PPD per card.
Check your temperatures and GPU clocks. I suspect GPU Boost is downclocking your card. Or Roy you have a 3GB version with less shaders?
Also linux vs windows.
I've a 6GB 1060 under linux doing 450k PPD, and a 3GB 1060 in windows doing 340k PPD. Don't know how much is the loss of shaders and how much is OS, but I suspect it's a bit of both.
Nathan_P
Posts: 1164
Joined: Wed Apr 01, 2009 9:22 pm
Hardware configuration: Asus Z8NA D6C, 2 x5670@3.2 Ghz, , 12gb Ram, GTX 980ti, AX650 PSU, win 10 (daily use)

Asus Z87 WS, Xeon E3-1230L v3, 8gb ram, KFA GTX 1080, EVGA 750ti , AX760 PSU, Mint 18.2 OS

Not currently folding
Asus Z9PE- D8 WS, 2 E5-2665@2.3 Ghz, 16Gb 1.35v Ram, Ubuntu (Fold only)
Asus Z9PA, 2 Ivy 12 core, 16gb Ram, H folding appliance (fold only)
Location: Jersey, Channel islands

Re: Plotting Overall Folding System Efficiency

Post by Nathan_P »

I have a GTX 1070 and 1080 on a dual CPU Asus Z9PE with dual 12c/24t xeons and a 1070Ti and 1050Ti on a Z87 WS with an E3-1230L v3 currently pulling 742w from then wall for 2.316m PPD. Both systems run on Mint 18.2, clocks and voltages are whatever the cards are set to act the factory. I have seen power draw go as high as 840w but its all project dependent.
Image
ProDigit
Posts: 234
Joined: Sun Dec 09, 2018 10:23 pm

Re: Plotting Overall Folding System Efficiency

Post by ProDigit »

kbeeveer46 wrote:
Holdolin wrote:While I have no idea what each of my 2070's are pulling power wise, they're not using 220W. All 3 of them are in one rig and total power draw according to my UPS is sitting around 650W. Not bad for 3.3M PPD.
If you download the GPU-Z app and click on the Sensors tab it will show you how much power the card is pulling. I'm folding right now and my 2070 is pulling 162 W.
This is only true when they're running from a PCIE 16x slot.
If they're running from a PCIE1x/4x slot, the power reading is incorrect, as it gets additional power from a riser.
In my case, both 1060s are using between 100-110W, deducted from a killawat meter; yet GPUZ reports 107W for one plugged in teh PCIE16x slot, and 63W for the one plugged in the PCIE1x slot.
rwh202
Posts: 410
Joined: Mon Nov 15, 2010 8:51 pm
Hardware configuration: 8x GTX 1080
3x GTX 1080 Ti
3x GTX 1060
Various other bits and pieces
Location: South Coast, UK

Re: Plotting Overall Folding System Efficiency

Post by rwh202 »

ProDigit wrote:This is only true when they're running from a PCIE 16x slot.
If they're running from a PCIE1x/4x slot, the power reading is incorrect, as it gets additional power from a riser.
In my case, both 1060s are using between 100-110W, deducted from a killawat meter; yet GPUZ reports 107W for one plugged in teh PCIE16x slot, and 63W for the one plugged in the PCIE1x slot.
I'd be surprised if risers affected anything (other than slowing the cards down and reducing power consumption)
The power sensors are on the card and don't care whether the power comes from the slot, the riser or 6/8-pin power connectors. However, I'm not sure if they are pre- or post-VRM and whether the losses there are included.
The difference between killawat and GPUZ is likely down to PSU efficiency.
107 + 63 = 170 W total for GPU. Assuming 85% PSU efficiency, then you have 200 W from the wall as shown by the killawat.
ProDigit
Posts: 234
Joined: Sun Dec 09, 2018 10:23 pm

Re: Plotting Overall Folding System Efficiency

Post by ProDigit »

rwh202 wrote:
ProDigit wrote:This is only true when they're running from a PCIE 16x slot.
If they're running from a PCIE1x/4x slot, the power reading is incorrect, as it gets additional power from a riser.
In my case, both 1060s are using between 100-110W, deducted from a killawat meter; yet GPUZ reports 107W for one plugged in teh PCIE16x slot, and 63W for the one plugged in the PCIE1x slot.
I'd be surprised if risers affected anything (other than slowing the cards down and reducing power consumption)
The power sensors are on the card and don't care whether the power comes from the slot, the riser or 6/8-pin power connectors. However, I'm not sure if they are pre- or post-VRM and whether the losses there are included.
The difference between killawat and GPUZ is likely down to PSU efficiency.
107 + 63 = 170 W total for GPU. Assuming 85% PSU efficiency, then you have 200 W from the wall as shown by the killawat.
You are totally right. I forgot that the 1060 running on the 1x port is only operating at half the PPD, which might be the reason why the lower power draw.
Post Reply