FAH underperforming on new system

[arisu]

I thought there was an effort in GROMACS to improve the load balancer so that it could distribute load effectively even among a mix of E- and P-cores.

The issue is not with Gromacs but with OS scheduler (mostly Windows one) that will assign all threads to E core because they run at low priority ...

And yes, allowing FAH to handle all combinations automatically will be a pain.

Yeah it's not a GROMACS issue, although in theory it could be solved within GROMACS (but not easily). Having the client assign threads to P cores would be better and like I said I do that on my systems, but I don't know Windows development so it's currently Linux-only.

[foxpy]

When I pin all threads but one to a single core, the remaining thread drops to around 70%. I believe that is because it exceeds the limit of the load balancer. I'll check the source code to make sure, but I don't think it is using spinlocks.

Maybe it doesn't use spinlocks indeed, but still something weird happens under the hood. 70% usage is quite high in this scenario.

[arisu]

I can only imagine that the 70% is due to it performing load balancing, but not well enough to achieve 100%.

[tom.w.2277]

Thank-you Sirs @calxalot and @toTOW for pointing me in the right direction!

You need to not use e-cores.
Set cpus no higher than p-core threads.
Use Process Lasso to pin the FahCore_* to p-cores.

Make sure your system clock is correct.
Do not have spaces in the machine name.

Set the number of folding threads to match with you P core threads and use Process Lasso to pin the FahCore_* to P cores automatically. And next time you build a computer, think about AMD CPUs.

Regarding calxalot's first 3 suggestions
Not being familiar with Process Lasso, I initially just tried reducing the number of CPUs FAH was using. I didn't recall the number of P-cores in my i5-13500, so I started by going into Settings, and adjusting the number of CPUs that FAH uses from the default of 19 down to 8. This resulted in an almost immediate increase in PPD to around that of the i3-12100, which has 4 P-cores and supports 8 P-core Threads. When I later looked up the specs on my i5-13500 and found that it has 6 P-cores and supports 12 P-core Threads, I tried increasing the number of CPUs that FAH uses from 8 first to 10, and then 12. Oddly, in both cases I initially saw an increase in CPU utilization up to around 80 - 90%, but then it quickly dropped back down to around 50 - 60%, and PPD actually decreased relative to running at 8 CPUs.

When I had time to look into Process Lasso, it appears to be an extremely powerful tool capable of a lot more than I need. It was easy to pin the FAHCore process to P-cores.

For those not familiar with Process Lasso, here is how I did it
(See screenshots at https://drive.google.com/file/d/1O3-cWF ... sp=sharing):

• 1. Download Process Lasso at https://bitsum.com/download-process-lasso/
  2. Run the downloaded file to Install Process Lasso
  3. Once Process Lasso is running, just scroll down the list of processes, FAH was near the bottom of the list for me
  4. Right click on the FAHCore process, then select CPU Affinity, then select Always, then click on Select CPU Affinity
  5. When the CPU Affinity screen comes up, click the P-cores button on the bottom to select all of the P-cores(Performance Threads) for your processor.

Since making these changes, PPD on my new i5-13500 system are now around double that of my i3-12100 system and about 8-10 times what it was when I first started running FAH on this system. I also added an old RX570 graphics card(Ellesmere - that was no longer being used for folding in the i5-6600K system for some reason) to the new system and it is now producing almost twice the PPD as the i5-13500. Combined, the new system with RX570 has about quadrupled my PPD from around 230,000 to 800,000-950,000

Some questions for those with experience with Process Lasso and FAH:

• 1. The free version has some features that expire after 30 days. Is the ability to pin processes to specific CPUs one of those features that will expire? I ask because they are currently running a 40% off special that I would like to take advantage of if I will eventually have to purchase the PRO version.
  2. Would it make sense to pin FAHClient.exe and conhost.exe to E-cores since they are higher priority than the FahCore processes, to save more P-core capacity for the FAH Core processes?
  3. Would it similarly make sense to assign FahCore processes associated with folding on GPUs to E-cores?

Some suggestions for FAH developers:

• 1. Until you able to address this issue with E-cores, please run a test during installation, and if the processor has E-cores, display a popup that notifies users how to greatly improve FAH performance and perhaps refer them to this thread
  2. How about licensing just the portion of Process Lasso that allows you to pin FAH to P-cores with a stripped-down interface that makes it really easy to configure

[arisu]

Some suggestions for FAH developers:

• 1. Until you able to address this issue with E-cores, please run a test during installation, and if the processor has E-cores, display a popup that notifies users how to greatly improve FAH performance and perhaps refer them to this thread
  2. How about licensing just the portion of Process Lasso that allows you to pin FAH to P-cores with a stripped-down interface that makes it really easy to configure

There's an open ticket for disabling folding on E-cores: https://github.com/FoldingAtHome/fah-cl ... issues/349

[tom.w.2277]

I think I found the answer to my question regarding pinning processes to specific CPUs with the free version of Process Laso at https://bitsum.com/howfree/:

Unfortunately, I haven't been able to figure out how to format a table here, so according to the referenced website, CPU Affinities IS a feature included in Process Lasso Free. So no current need to purchase the PRO version. We shall see if bitsum changes that in the future.

Feature Process Lasso Free Process Lasso Pro
ProBalance YES YES
Performance Mode YES YES
Efficiency Mode YES YES
CPU Affinities YES YES
CPU Priorities YES YES
I/O Priorities YES YES
IdleSaver YES YES
Timed Keep PC Awake YES YES
Disable Hyper-Threading (SMT) YES YES
SmartTrim YES YES
Config Profile Switcher YES YES
Reserved CPU Sets YES YES
Dynamic Thread Boost YES YES
CPU Sets TRIAL ONLY* YES
Memory Priorities TRIAL ONLY* YES
Foreground Boosting TRIAL ONLY YES
CPU Limiter TRIAL ONLY YES

See https://bitsum.com/howfree/ for more