Why are projects 9415 and 9414 such low PPD?

[Luscious]

I have noticed over the last four-eight weeks that my overall PPD has been declining, while my WU count has been increasing. Examining this closer I notice a heavy workload of projects 9414 and 9415 on my system. Other GPU projects are out there that give much higher PPD, but for some reason I seem to be getting work units from these two projects almost exclusively.

My cause preference is set to any and hasn't been changed.

My cards (GTX980Ti) can push 600K PPD on a good day, yet these two projects limit me to 390-400K only.

Curious if Pascal cards are showing the same low PPD on these projects.

[bollix47]

While there are some projects with lower atom counts that don't do as well as other projects on the high end GPUs these two projects seem to be fine with my 980ti, both getting around 640K PPD. With the lower atom count there will be more frequent traffic between the GPU and the CPU.

A few things to look at:

1. Temperatures of both the GPU & the CPU ... i.e. is there throttling happening?
2. Have you allowed a full CPU core for each GPU?
3. Power availability ... is there enough?
4. How much system RAM is available for the folding process?

You could post the System Info and Config sections of your log.txt file and we may be able to spot something that requires adjustment.

If you're unsure how to post that info please see this thread.

[Luscious]

A few things to look at:

1. Temperatures of both the GPU & the CPU ... i.e. is there throttling happening?
2. Have you allowed a full CPU core for each GPU?
3. Power availability ... is there enough?
4. How much system RAM is available for the folding process?

1. CPU hovers around 56C while the GPU's are between 40-42C. It's a well-cooled system with no throttling.
2. Pretty sure it's turned on. How do I check?
3. 1600W PSU pulling 1050W from the wall when under load.
4. 32GB installed with each x21 core using about 117MB according to task manager

Here is the log snippet with system info and config:

Code: Select all

*********************** Log Started 2017-11-02T03:51:20Z ***********************
03:51:20:************************* Folding@home Client *************************
03:51:20:      Website: http://folding.stanford.edu/
03:51:20:    Copyright: (c) 2009-2014 Stanford University
03:51:20:       Author: Joseph Coffland <joseph@cauldrondevelopment.com>
03:51:20:         Args: 
03:51:20:       Config: C:/ProgramData/FAHClient/config.xml
03:51:20:******************************** Build ********************************
03:51:20:      Version: 7.4.4
03:51:20:         Date: Mar 4 2014
03:51:20:         Time: 20:26:54
03:51:20:      SVN Rev: 4130
03:51:20:       Branch: fah/trunk/client
03:51:20:     Compiler: Intel(R) C++ MSVC 1500 mode 1200
03:51:20:      Options: /TP /nologo /EHa /Qdiag-disable:4297,4103,1786,279 /Ox -arch:SSE
03:51:20:               /QaxSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 /Qopenmp /Qrestrict /MT /Qmkl
03:51:20:     Platform: win32 XP
03:51:20:         Bits: 32
03:51:20:         Mode: Release
03:51:20:******************************* System ********************************
03:51:20:          CPU: Intel(R) Core(TM) i7-5960X CPU @ 3.00GHz
03:51:20:       CPU ID: GenuineIntel Family 6 Model 63 Stepping 2
03:51:20:         CPUs: 16
03:51:20:       Memory: 23.87GiB
03:51:20:  Free Memory: 18.03GiB
03:51:20:      Threads: WINDOWS_THREADS
03:51:20:   OS Version: 6.1
03:51:20:  Has Battery: true
03:51:20:   On Battery: false
03:51:20:   UTC Offset: -7
03:51:20:          PID: 3956
03:51:20:          CWD: C:/ProgramData/FAHClient
03:51:20:           OS: Windows 7 Professional
03:51:20:      OS Arch: AMD64
03:51:20:         GPUs: 4
03:51:20:        GPU 0: NVIDIA:7 GM200 [GeForce GTX 980 Ti] 5632
03:51:20:        GPU 1: NVIDIA:7 GM200 [GeForce GTX 980 Ti] 5632
03:51:20:        GPU 2: NVIDIA:7 GM200 [GeForce GTX 980 Ti] 5632
03:51:20:        GPU 3: NVIDIA:7 GM200 [GeForce GTX 980 Ti] 5632
03:51:20:         CUDA: 5.2
03:51:20:  CUDA Driver: 8000
03:51:20:Win32 Service: false
03:51:20:***********************************************************************
03:51:20:<config>
03:51:20:  <!-- Network -->
03:51:20:  <proxy v=':8080'/>
03:51:20:
03:51:20:  <!-- Slot Control -->
03:51:20:  <power v='full'/>
03:51:20:
03:51:20:  <!-- User Information -->
03:51:20:  <passkey v='********************************'/>
03:51:20:  <team v='155369'/>
03:51:20:  <user v='Luscious'/>
03:51:20:
03:51:20:  <!-- Folding Slots -->
03:51:20:  <slot id='0' type='CPU'>
03:51:20:    <client-type v='advanced'/>
03:51:20:    <cpus v='8'/>
03:51:20:  </slot>
03:51:20:  <slot id='1' type='GPU'>
03:51:20:    <client-type v='advanced'/>
03:51:20:    <max-packet-size v='big'/>
03:51:20:  </slot>
03:51:20:  <slot id='2' type='GPU'>
03:51:20:    <client-type v='advanced'/>
03:51:20:    <max-packet-size v='big'/>
03:51:20:  </slot>
03:51:20:  <slot id='3' type='GPU'>
03:51:20:    <client-type v='advanced'/>
03:51:20:    <max-packet-size v='big'/>
03:51:20:  </slot>
03:51:20:  <slot id='4' type='GPU'>
03:51:20:    <client-type v='advanced'/>
03:51:20:    <max-packet-size v='big'/>
03:51:20:  </slot>
03:51:20:</config>

Something does sound quite amiss if your 980TI is doing 640K on these two projects. Which GeForce driver are you using?

[foldy]

You could try to disable the CPU slot to see if GPU PPD goes up for these projects.

[bollix47]

4. 32GB installed with each x21 core using about 117MB according to task manager

03:51:20: Memory: 23.87GiB
03:51:20: Free Memory: 18.03GiB

Something doesn't look right there. I have both linux and Win systems and the difference between actual memory and what is shown in my log is very small compared to 32 vs 23. Have you run a memory test or is your actual memory 24GB?

In my case I'm using linux drivers 370.28 and I know linux does get a bit more PPD than Windows but the difference should not be as large as what you're seeing.

As foldy suggests try pausing the CPU slot next time you get one of these projects just to see if there's a difference.

2. Pretty sure it's turned on. How do I check?

According to your log you have 16 threads and you're only using 8 for the CPU slot leaving 8 for the 4 GPUs. That should be plenty.

Another area to look into is your motherboard. Are your pci-e slots all running at x16 when 4 of them are being used? What is their bandwidth?

[Kjetil]

My ppd is 900k(p13147) and 1.5k on (p13200). Evga 1080Ti sc2 at 2050.

[Nathan_P]

There is nothing wrong with the PPD on 9414 and 9415 on pascal, on a 1080 I am getting over 900k PPD and nearly 700k ppd on a 1070. My 980ti used to manage anything from 385k to 590k PPD but i'm not sure what the project spread was.

[Luscious]

Something doesn't look right there. I have both linux and Win systems and the difference between actual memory and what is shown in my log is very small compared to 32 vs 23. Have you run a memory test or is your actual memory 24GB?

That missing 9.76GB is most likely my RAMDISK allocation. All memory sticks show up and CPU-Z shows 32GB installed.

I've only recently started running the CPU slot as it is usually paused anyway. The points difference is around +70K PPD with the CPU slot running and It's not impacting the GPU from what I can tell. MB slots are configured at x16/x8/x8/x8.

I'm on 361.43 - is it possible these projects are driver optimized? I'm scratching my head at what else to think.

[toTOW]

Variability in PPD is expected ... and there is more variability on Windows because of the design of NV OpenCL that is more CPU and PCIe bandwidth dependant in Windows than it is in Linux.

I'm getting 380k PPD on my 980 (Windows 10), which is in the expected PPD range for this GPU (350-450k). Target is 420k. CPU feeding the GPU is a, i7 860 @ 3.5 GHz.

My 1070 is getting 600k PPD on these projects (Windows 10 too), which is in the expected range for this GPU (550-650k). Target is 620k. CPU feeding the GPU is a, i7 920 @ 3.5 GHz.

I don't know how the compare to such old drivers, but I noticed that 387.92 are faster than previous version of drivers that were continuously degrading performances update after update ... My systems are now all hitting their lowest average PPDs according to HFM.

[Luscious]

Back with an update here and it's evident projects 11432 and 11713 are putting my cards back at their previous performance level.

No system changes made whatsoever.

Now I am curious if someone in the know can chime in to explain what the differences are between projects 9415/9414 versus 11432/11713, and why those differences are impacting scoring by 25-33% for me.

[foldy]

I see the same lower PPD for 94xx work units, 11xxx work units get more PPD.

Linux gets better PPD than Windows, I don't know if that is an option for you?

[Luscious]

Linux gets better PPD than Windows, I don't know if that is an option for you?

Unfortunately no, not while the cards are in the same box as my main rig, and dual booting CentOS is out of the question too because of how my SSD is sized and partitioned. Not sure if VM would be an option here.

Since I don't want to set up separate rigs or deal with water cooling a rack, I would pretty much have to look at a case that could fit two motherboards. Maybe in the future

[Kuno]

VM is not an option as the only VM that gives linux full access is hyper-visor in Server 2016, or if you have GRID compatible cards. If you had linux installed as a base, it wouldn't be a problem but going the other way, is literally impossible. Windows as a host OS and linux as a guest with full hardware access is impossible, with consumer grade hardware/software.

[toTOW]

Now I am curious if someone in the know can chime in to explain what the differences are between projects 9415/9414 versus 11432/11713, and why those differences are impacting scoring by 25-33% for me.

First big difference is in the atom counts. Less atoms usually means poor scaling on higher end GPUs (I'm not afraid to call this a "CPU limited" situation).

I think there is another variable, but no one could ever show me clearly which one : simulation parameters are probably different (forcefield used, integrator, ...).

[bruce]

Less atoms usually means poor scaling on higher end GPUs ...

Actually, for those who run mid-range GPUs, these projects scale quite well. Thus it's reasonable to ask why projects OTHER THAN 9414 and 9415 scale higher than they should. It does make sense for FAH to hold the line against points inflation (thereby degrading the value of points earned earlier) so reducing the points on higher scaling projects does make sense. Is that what you want?