CPU requirement for GPU folding (NV)

[Frontiers]

I think that multi-GPU systems PPD degradation is because of copies of fahcore21.exe walking between CPU cores.
In rig with 4 GPUs you have 4 fahcore21.exe and all 4 doing a chinese firedrill all the way.
One of possible solutions to increase PPD in multi GPU rigs with PCIe 3.0 motherboards is Process Lasso utility: just nail 4 fahcore21.exe affinity to concrete physical CPU cores in tab of active processes in Process Lasso.
Do not nail affinity of any system or drivers processes - it could lead to very laggy Windows GUI, sometimes even near impossible to boot in and making changes.
For CPU GRO-A7 folding - one could nail it's fahcores to different physical CPU cores from those used by GPU's fahcore21.exe.
For multi-GPU systems on PCIe 2.0 boards only possible solution for PPD inproving is folding at Linux (Mint/Ubuntu or whatsoever), because with PCIe 2.0 Linux gives much higher GPU utilization with 372.20 drivers than Windows with any drivers.
But I can't give any advices how to tune multi-GPU Linux rig - because I still fold with only 1 GPU.

[Aurum]

Hi Frontiers, Nice idea, I'll give it a try. I wonder if when a WU finishes will PL lose its lock on that CPU core???

[Frontiers]

it allows GTX1070 @ 2.1GHz with p9431 folding load up to 82% while up to 85% without any heavy CPU occupation

You need to try Linux Mint with that how-to from tguskill (great thanks to him) from overclockers.ru board.
I saw same low GPU utilization under Windows with PCIe 2.0 and about 96-97% GPU utilization under Linux Mint 18.1 with same PCIe 2.0 motherboard.
This higher GPU utilization gives 200k more PPD under Linux Mint with same GTX 1070, clocked 1950 MHz in Linux instead of 2050 MHz in Windows 10.
Beacuse of so much higher GPU utilization - there is no need for such high overclocking under Linux as it was under Windows, as rule of thumb - overclock it 100 MHz lower than it was under Windows, with unlocking Power Loimit with "sudo nvidia-smi -i0 -plimit XX0" from console, XX0 - design power limit of your board.

I wonder if when a WU finishes will PL lose its lock on that CPU core???

If you change process affinity with Windows Task Manager - folding process will lose affinity after fahcore restarted with new WU.
If you change process affinity with Process Lasso - you will have an option (second or third from the top at right click menu) to make constant affinity - it will not change after fahcore restarted with new WU.

[Aurum]

Ok I installed on a 4 GPU headless rig. E5-4640v3 with 12c24t. At first I tried only using 4 CPU threads for the GPUs but they were constantly pegged at 100%.
Then I set the last 8 CPUs to FahCore_21.exe and the first 16 CPUs to FahCore_a4.exe. Manually set FAHControl to use 16 CPUs. This gives total CPU usage ~87% per Win7 Task Manager. The 4 GPUs don't need all 8 threads so that leaves headroom for other stuff to run without having to leave an unassigned CPU thread. WUs were already running so it's too early to tell but the estimated PPD has increased 2%. In an hour everyone will have a new WU and the test can begin.

[Duce H_K_]

Till Nov, 27 I expect second GPU, the Palit SJS 1080Ti which will be available for me. So I meant single GPU configuration. No heat related issues for last few months.
12c24t means HT=on of course 'cause CPU tasks are not doing really great without it. CPU model is 2658 as a seller wrote but I cannot read it directly from hardware reports (CPU-z).
My former hardware was AMD A8 5600k CPU @ Asus A-55M MoBo. Didn't get much luck with Mint installation inside VMWare from tguskill's manual. Despite my hard efforts GPU couldn't be seen by virtual machine. Cannot fully refuse Windows. Planning to try once again with current newer H/W, ETA of 2 months or so. I want to thank everybody for the recommendation.
To assign threads to 21 & a* FahCores more or less properly I use WinAFC with sub parameter assign=2 or assign=3

Code: Select all

*\FahCore_21.exe := CPU0+CPU1+CPU2 [priority=High, ssign=2]
*\FahCore_a*.exe := CPU1+CPU2+CPU3+CPU4+CPU5+CPU6+CPU7+CPU8+CPU9+CPU10+CPU11+CPU12+CPU13+CPU14+CPU15+CPU16+CPU17+CPU18+CPU19+CPU20+CPU21+CPU22+CPU23 [priority=BelowNormal]

But AFAIK this sample is not as precisely as it is needed.
EDIT CPU mode 4c8t @ 2.6GHz provided 780kPPD with p9414, p9415 and 730kPPD with p9430 project. That are the maximums.

[bruce]

Assigning 1-23 to _a* means that only cpu0 is "free" for _21. That also suggests that NOTHING ever interrupts it's use of cpu0, which is probably not true. Giving _21 three choices may seem like a good idea, but that also suggests that you actually expect there to be times when it might use cpu1 or cpu2. Any such sharing of resources will degrade your performance rather than increase it.

I don't see where you've posted FAH's config so I have to assume that you've configured _a* into multiple slots so that prime factors are avoided -- such as CPU:16 + CPU:6. There, too, you'll reduce performance if you try to use VM features to enable CPUs to be shared resources between more than one WU.

[foldy]

I made an experiment with 6 core/12 thread CPU and gtx 1080ti on Windows 10.
1) I could assign 10 threads to CPU work and leave 1 thread for GPU and 1 thread for OS without sacrificing GPU speed.
2) When using 11 threads for CPU work and 1 thread for GPU then GPU speed goes down.
3) When using 11 threads for CPU work and 1 thread for GPU and set FahCore_21 priority to high then GPU speed is only slightly reduced.

I guess for many core/thread CPU the best option to get best GPU performance is to set the FahCore_21 to high priority. Then it doesn't matter if all other threads are used at normal priority or below by CPU folding or something else. Maybe this could become an option in FahClient to set GPU slots FahCore_21 CPU priority to high? (There is currently such option for CPU core to increase priority from "low" to "lower than normal" but that does not help in this scenario)

Currently FahClient makes CPU usage thread count -1 for each GPU. Maybe that should be true only if core count = thread count and else make another -1 for OS?

[bruce]

Currently FahClient makes CPU usage thread count -1 for each GPU. Maybe that should be true only if core count = thread count and else make another -1 for OS?

The default setting is established ONLY at the time FAH is installed. FAH's code is not designed to override your choices if you choose to make poor choices. V7.4.4 will let you set it to 11 if you wish to, but that's a bad choice, not because it considers leaving another -1 for the OS, but because it's a prime number which will produce a high failure rate. V7.4.15+ will reduce it further if necessary to obtain a setting that is more likely to be assigned work and less likely to fail.

[foldy]

OK I see the prime number problem and the FahClient V7.4.15+ beta was adapted to not use prime numbers.

But for a 8 core/16 thread CPU and one GPU the default after FAH installed will be 16-1 = 15 FAH CPU cores usage. Not a prime but no thread for OS left and no high priority for GPU core's CPU part.

So my example is to illustrate maybe the FahClient default installation needs another adaptation to optimize GPU usage in general?

[Aurum]

Maybe this could become an option in FahClient to set GPU slots FahCore_21 CPU priority to high? (There is currently such option for CPU core to increase priority from "low" to "lower than normal" but that does not help in this scenario)

That was one of the suggestions we'd made for F@H many moons ago. Maybe some day.

[FldngForGrandparents]

One other thing to note. If it is a dedicated machine with multi GPU it is best to turn off Hyperthreading. Otherwise fahcore can jump to using the same physical core on both threads. Hyperthreading is only good for apps that do a lot of IO and are threaded.

[bruce]

It's your choice whether to turn off HT.

Based on rather old measurements (probably with a core2duo) using both halves of a single CPU will accomplish about 15% more than using just one half. You're correct that Apps that do a lot if I/O benefit most from HT but FAH's performance is based ~85% on the shared floating point (/SSE/AVX/etc.) registers, not 100%.

[Aurum]

...a dedicated machine with multi GPU it is best to turn off Hyperthreading. Otherwise fahcore can jump to using the same physical core on both threads. Hyperthreading is only good for apps that do a lot of IO and are threaded.

I'm trying this using Project Lasso. I let 4 GPUs have the last 8 threads (20-27) & set FahCore_21 priority to High. The Xeon has 14c28t so I let a4 etc have threads 0-19. I configured CPU to use 20 threads in FAHControl. CPUs 0-19 are always pegged so I may have to drop to 18 CPUs. Initially PPD dropped when making changes in mid-WUs.

[Aurum]

Built a new rig with an E5-2699v4 with 22c44t but F@H can only recognize 32 threads max. If I try to set it higher I get this error message:

Code: Select all

On client "Rig-06" 192.168.1.206:36330: Invalid value for option 'cpus'.  Option help:
  --cpus <integer=-1>
      How many CPUs a slot should use. <= 0 will use all the CPUs detected in the system.
Caused by: 38 is greater than maximum value 32

Hmm, Task Manager shows all 44 threads working on something. I'll try disabling hyperthreading in the BIOS and see what happens.

[foldy]

It can make a difference if you give last 8 threads or first to the GPUs because the OS by default also gives the first or last threads to any tasks.