Page 5 of 5
Re: fahclient 7.6.x ignore <power value="*"/>
Posted: Wed May 20, 2020 1:49 pm
by PeterGarlic
PeterGarlic wrote:And this is the summary:
Installation/removal and other issues
### WARNING: components not removed on uninstall
### ERROR-01: Starting fahclient ... FAIL
### ERROR-02: systemctl unit created but not working
### ERROR-03: after installation FAHclient start with default config & error
### ERROR-04 - after 1st installation process start attempting downloads
### ISSUE-01: client must be configure by hand on systems without gui
### ISSUE-02: missed text configuration
### ERROR-03 - execution of /usr/bin/FAHclient
### ISSUE-04 - unreferenced error
### ISSUE-05 - undocumented features
### ISSUE-06 - no man pages
### INFO-07 - manual execution of corewrapper
### INFO-00 - process function(s)s not documented
### ISSUE-00 - non standard logging
### ISSUE-007 error on start/restart
Power control check
### NOTE-01 - pre-power control test
### ERROR-10 - missed medium power tag on control.xmlc
### ERROR-11 - issues on FAHControl power control & config.xlm
### ISSUE-10 - set <core-priority> from FAHControl has no effect
### ISSUE-11: pause from command line not implemented
### ERROR-13: CPU -1 have ho effect on FAHControl
### ERROR-14: all values on slot id are the same
### ISSUE-12 - When ID is assigned all CPU come back under FAHControl power control
### ISSUE-13 - <core-priority v='low'/> has no effect
We didn´t tested in deep and create this list of possible issues just for fun or to demonstrate anything.
For us is a standard software analysis and we hope that will be forwarded to development team and will be useful for the next client release.
Re: fahclient 7.6.x ignore <power value="*"/>
Posted: Wed May 20, 2020 4:00 pm
by bruce
PeterGarlic wrote:NRT_AntiKytherA wrote:F@H is set at a low scheduling priority by default so any other running task takes precedence, your customers should never run out of resources when their loads on your servers increase.
May you explain me a little bit in deep this concept?
Thanks in advance.
Consider a single 4-cpu VM, who might be trying to run something concurrently OTHER than FAH?
Looking at that single machine, OS makes the decisions for the entire machine. The owner might want to do some browsing so he starts up that task which has nothing to do with FAH. That foreground task gets priority over whatever CPU processing is being scheduled for FAH. FAH can use 100% of the 4 threads except when his browser wants 10%. when it does, it gets that 10% off the top and FAH gets the remaining 90%. FAH doesn't slow down his use of his browser.
Re: fahclient 7.6.x ignore <power value="*"/>
Posted: Mon May 25, 2020 11:18 am
by PeterGarlic
Consider a single 4-cpu VM, who might be trying to run something concurrently OTHER than FAH?
Looking at that single machine, OS makes the decisions for the entire machine. The owner might want to do some browsing so he starts up that task which has nothing to do with FAH. That foreground task gets priority over whatever CPU processing is being scheduled for FAH. FAH can use 100% of the 4 threads except when his browser wants 10%. when it does, it gets that 10% off the top and FAH gets the remaining 90%. FAH doesn't slow down his use of his browser.
VMs are dedicated to FAH and just few other process run there (monitoring agent, etc.) then the priority is all applyied to faclient process.
From a datacenter point of view a more dynamic control on priority and would make easier to automate cpu/gpu resource allocation (for example on 10% steps with a command line access)
This will be a benefit for the project because the cpu load changes during the day and over the weeks: For example a simple monitoring and resource control automation can use more resources when available and reduce the load when the "company load" groove.
We have a workaround that is based on pause/unpause the fahclient to reduce the load but this is not the best approach that I can imagine.
Another approach (currently in test) is a dynamic cpu slot reconfiguration on /etc/fahclient/config.xml but this is also just another workaround.
Last point on power control on our private cloud is related to monitoring: when one VM cpu usage goes over 90% alarms are triggered. This mean that we have to deploy VMs with one or two unused cores to avoid false positive alarms and this is not the best for resource utilization.
Some more flexibility on power control (via cli or api control) will be really nice for a cloud usage and automation.
Re: fahclient 7.6.x ignore <power value="*"/>
Posted: Mon May 25, 2020 4:29 pm
by PeterGarlic
bruce wrote:FAH doesn't specify which CPU it will use, only how much ... it's up to the scheduler in the OS to pick the available CPU. From the perspective of the program, 100% of 1 CPU is the same as 50% of two CPUs. or 33% of three CPUs.
right, but seems to be different on a kvm based cluster...
Few hours ago I moved our test environment up to 5 x 8 vCPU VMs with this configuration:
Code: Select all
<config>
<!-- Folding Slot Configuration -->
<gpu v='false'/>
<!-- HTTP Server -->
<allow v='127.0.0.1 x.x.x.x'/>
<!-- Network -->
<proxy v='x.x.x.x:yyyy'/>
<proxy-enable v='true'/>
<!-- Remote Command Server -->
<password v='*********'/>
<!-- Slot Control -->
<power v='full'/>
<!-- User Information -->
<passkey v='****************************'/>
<user v='******'/>
<!-- Folding Slots -->
<slot id='0' type='CPU'>
<cpus v='7'/>
</slot>
</config>
As expected from testing power control is unusable from FAHControl but leaving out one of the 8 vCPU of the VM limit the global VM CPU usage around 75%
This is also funny because was expected a different value but it works
Internally the VM seems to use the always the same 6 of 8 cores at 99%.
Re: fahclient 7.6.x ignore <power value="*"/>
Posted: Tue May 26, 2020 9:01 am
by PantherX
PeterGarlic wrote:...the VM seems to use the always the same 6 of 8 cores at 99%.
That because 7 is a prime number which causes issues when folding as the WU can't divide itself properly (see domain decomposition issues in the Forum) so it gets rounded down automatically to 6.
BTW, if you have a handful of VMs that are dedicated for F@H can you:
1) Disable alerting on them. It would be simple and can help you out by making it easier.
2) Reduce the VM priority to the lowest one so that any other VM has priority over it for CPU cycles thus, will not be impacted. I am aware that VMWare has that ability so is there something similar in the product that you use?