fahclient 7.6.x ignore <power value="*"/>

[PeterGarlic]

Hi all,
I saw many discussion about cpu control on fahclient 7.6.x but the argument is still not clear for me.

I work for a service provider and we are planning to automate deploy some hundreds Centos7/Fedora32 4 core VM instances on our cloud infrastructure (KVM based) but seems that the tag <power> on /etc/fahclient/config.xml is ignored from the client and this can be a probem for us.

I opened also one issue on github with all the details of the problem: "fahclient 7.6.x ignore <power value="*"/> - fahcontrol overwrite item on config.xml" but they closed the item and redirected me to this forum.

Someone can explain to me if:
- is possible to assign a power value (light/medium/high) to fahclient?
- is possible to avoid that the client take the full control of one Cpu and distribute the load to all available cpu?

For the moment we found a workaround for that (we define the number of CPU slots on config.xml to use 1 to 4 vCpu of the VM instance ( then we can set resource usage to 25%, 50%, 75%,100% ...) but is not the best and the job are always linked to a vCpu used at 100%

Any suggestion is welcome

Thanks in advance
Peter

[PantherX]

Welcome to the F@H Forum PeterGarlic,

This is what the power values mean (assuming no supported GPU is present)
Light -> Uses 50% of your CPUs
Medium -> Uses 1 CPU less than all your CPUs
Full -> Uses all your CPUs

The client only works on CPUs, thus if you have a 12 CPU system, you can assign it 3 CPUs for 25% CPU usage. The priority of folding is a low once thus, almost all other tasks are are not impacted by folding running in the background.

[PeterGarlic]

Hi PanterX,
the concept of power control is what are we looking for, but seems not working on our lab servers.
We made many different test (me on virtualbox (with Centos7, Fedora32, Ubuntu18.4LTS), my colleague on KVM & OpenStack cluster (with fedora32) but we got the same result:

Manually inserted on /etc/fahclient/config.xml or managed from FAHcontrol seems to have no effects.

What we have still to test is specifiy this value by command line but I don't know how.

[PantherX]

Do you mind posting the config.xml file without the passkey? That way we can see what's happening.

EDIT, the log file would be great too. Guidance is provided here: viewtopic.php?f=24&t=26036

[ajm]

As PantherX wrote on Github, you also can edit the number of CPUs allocated in /etc/fahclient/config.xml (sudo nano...)
I just tried it in a VBox 6.1.6 with an Ubuntu 18.04 guest (on Win 10 pro). Worked fine.

I think your problem might be that you have confused /etc/fahclient (correct) with /etc/fahcontrol (wrong).

[PeterGarlic]

I started reading the guidance... will take some time!

In the meanwhile I post a sample config.xml running now on a test vm (ubuntu18.04LTS, 4 CPU, 4Gb Ram):

Code: Select all

<config>
	
  <!-- Client Control -->
  <fold-anon v='false'/>

  <!-- Folding Slot Configuration -->
  <cause v='COVID_19'/>
	
  <!-- User Information -->
  <passkey v='********************************'/>
  <user v='************'/>

  <!-- Remote Command Server -->
  <password v='*******'/>

  <!-- Slot power Control -->
  <power v='light'/>

  <!-- Folding Slots -->
  <gpu v='false'/>
  <slot id='0' type='CPU'/>

</config>

I set basic options: usage of 1 of 4 CPU and power light. After I start the client top show me this (pressing 1 to see all cpu)

Code: Select all

top - 12:08:40 up  1:26,  1 user,  load average: 0,94, 0,46, 0,23
Tasks: 261 total,   2 running, 200 sleeping,   0 stopped,  10 zombie
%Cpu0  :  0,3 us,  0,0 sy,  0,0 ni, 99,7 id,  0,0 wa,  0,0 hi,  0,0 si,  0,0 st
%Cpu1  :  0,0 us,  0,0 sy,100,0 ni,  0,0 id,  0,0 wa,  0,0 hi,  0,0 si,  0,0 st
%Cpu2  :  0,3 us,  0,3 sy,  0,0 ni, 99,3 id,  0,0 wa,  0,0 hi,  0,0 si,  0,0 st
%Cpu3  :  0,0 us,  0,0 sy,  0,0 ni,100,0 id,  0,0 wa,  0,0 hi,  0,0 si,  0,0 st
KiB Mem :  4030252 total,  2115832 free,  1064676 used,   849744 buff/cache
KiB Swap:   999420 total,   999420 free,        0 used.  2728880 avail Mem

One cpu at 100% ignoring the <power v='light'/> directive.

I found also that the command /usr/bin/FAHClient --help give you a lot of ways to explore the fahclient status and options.

But if I use /usr/bin/FAHClient --print (Print configuration and exit) the output doesn´t reflect my current running configuration: maybe is the default config?

Code: Select all

/usr/bin/FAHClient --print
                         allow = 127.0.0.1
             capture-directory = capture
              capture-on-error = false
               capture-packets = false
              capture-requests = false
             capture-responses = false
               capture-sockets = false
                         cause = ANY
                         cause = ANY
              certificate-file = <undefined>
                    checkpoint = 15
                         child = false
                client-subtype = LINUX
                client-threads = 6
                   client-type = normal
               command-address = 0.0.0.0
                         allow = 127.0.0.1
         command-allow-no-pass = 127.0.0.1
                          deny = 0/0
          command-deny-no-pass = 0/0
                command-enable = true
                  command-port = 36330
                 config-rotate = true
             config-rotate-dir = configs
             config-rotate-max = 16
            connection-timeout = 60
                 core-priority = idle
                   cpu-species = X86_PENTIUM_II
                      cpu-type = AMD64
                     cpu-usage = 100
                          cpus = -1
                      crl-file = <undefined>
                    cuda-index = <undefined>
                    cycle-rate = 4
                        cycles = -1
                        daemon = false
                 debug-sockets = false
                          deny = 0/0
     disable-sleep-when-active = true
                   disable-viz = false
           dump-after-deadline = true
           exception-locations = true
                exit-when-done = false
               extra-core-args = <undefined>
                     fold-anon = false
                          fork = false
                           gpu = true
                     gpu-index = <undefined>
                     gpu-usage = 100
                   gui-enabled = true
                http-addresses = 0:7396
               https-addresses = 
                          idle = false
                  idle-seconds = 300
                           log = log.txt
                     log-color = true
                      log-crlf = false
                      log-date = false
         log-date-periodically = 21600
                    log-domain = false
             log-domain-levels = <undefined>
                    log-header = true
                     log-level = true
            log-no-info-header = true
                  log-redirect = false
                    log-rotate = true
                log-rotate-dir = logs
                log-rotate-max = 16
               log-short-level = false
            log-simple-domains = true
                 log-thread-id = false
             log-thread-prefix = true
                      log-time = true
                 log-to-screen = true
                  log-truncate = false
                    machine-id = 0
              max-connect-time = 900
               max-connections = 800
               max-packet-size = normal
                     max-queue = 16
            max-request-length = 52428800
             max-shutdown-wait = 60
               max-slot-errors = 10
               max-unit-errors = 5
                     max-units = 0
                        memory = <undefined>
              min-connect-time = 300
          next-unit-percentage = 99
                      priority = <undefined>
                   no-assembly = false
              open-web-control = false
                  opencl-index = <undefined>
                    os-species = UNKNOWN
                       os-type = LINUX
                       passkey = 
                      password = <undefined>
              pause-on-battery = true
                pause-on-start = false
                        paused = false
                           pid = false
                      pid-file = FAHClient.pid
                         power = light
                      priority = <undefined>
              private-key-file = <undefined>
                   project-key = 0
                         proxy = 
                  proxy-enable = false
                    proxy-pass = 
                    proxy-user = 
                       respawn = false
                        run-as = <undefined>
                       service = false
                session-cookie = sid
              session-lifetime = 86400
               session-timeout = 3600
                     set-group = <undefined>
                           smp = true
                  stack-traces = false
       stall-detection-enabled = false
                 stall-percent = 5
                 stall-timeout = 1800
                          team = 0
                          user = Anonymous
                     verbosity = 3
                     web-allow = 127.0.0.1
                      web-deny = 0/0
                    web-enable = true

[ajm]

PeterGarlic: on Github, you wrote:

edit you /etc/fahcontrol/config.xml and add the tag power control

The config that FAH will use is in /etc/fahclient/config.xml

[PeterGarlic]

As PantherX wrote on Github, you also can edit the number of CPUs allocated in /etc/fahclient/config.xml (sudo nano...)
I just tried it in a VBox 6.1.6 with an Ubuntu 18.04 guest (on Win 10 pro). Worked fine.

I think your problem might be that you have confused /etc/fahclient (correct) with /etc/fahcontrol (wrong).

Wops: a typo. (/etc/fahcontrol )
Please excuse me!!

@aja I´m running the same configuration at the moment and is clear that /etc/fahclient/config.xml is main fahclient configand FAHcontrol is just a gtk4 GUI for fahclient.

My fahclient is working fine but not the power control:
did you try to allocate more than one cpu and check the usage with top (pressing 1 to see the individual cpu usage), htop or "vmstat 1 100" to see how is distributed the cpu load?

[ajm]

No, I just checked in FAHControl whether the value I entered in etc/fahclient/config.xml (12 out of a total of 25 for that guest) was picked up when I restarted the service. And I'll have to wait for over an hour before trying that again.

But if you have edited /etc/fahcontrol/config.xml in all of your tests, you can be sure that this is the cause of your problem: in a normal installation, there is NO /etc/fahcontrol directory. No part of FAH will ever read anything in there.

[PeterGarlic]

@ajm: I repeat that /etc/fahcontrol was just a typo on my post.
As you wrote that folder doesn´t exists and I never used them.

[ajm]

With 24 CPUs allocated to FAH out of 25 for the guest (normal situation):

Code: Select all

top - 13:36:37 up 1 day, 20:36,  1 user,  load average: 24.25, 24.49, 24.35
Tasks: 418 total,   2 running, 280 sleeping,   0 stopped,   6 zombie
%Cpu0  :  0.0 us, 21.2 sy, 78.8 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu1  :  0.0 us, 22.8 sy, 76.9 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.3 si,  0.0 st
%Cpu2  :  0.0 us, 20.9 sy, 78.8 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.3 si,  0.0 st
%Cpu3  :  0.0 us, 21.2 sy, 78.8 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu4  :  0.0 us, 21.5 sy, 78.5 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu5  :  1.3 us, 33.0 sy, 64.0 ni,  1.7 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu6  :  0.0 us, 37.9 sy, 62.1 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu7  :  0.7 us, 38.0 sy, 61.4 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu8  :  0.0 us, 19.5 sy, 80.5 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu9  :  0.0 us, 22.8 sy, 77.2 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu10 : 13.8 us,  7.8 sy, 13.4 ni, 65.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu11 :  0.0 us, 20.6 sy, 79.4 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu12 :  0.0 us, 21.5 sy, 78.5 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu13 :  0.0 us, 36.8 sy, 63.2 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu14 :  0.0 us, 23.1 sy, 76.9 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu15 :  0.0 us, 39.9 sy, 60.1 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu16 :  0.0 us, 36.0 sy, 64.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu17 :  1.7 us, 20.1 sy, 64.5 ni, 13.7 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu18 :  0.0 us, 21.5 sy, 78.5 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu19 :  0.0 us, 21.2 sy, 78.8 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu20 :  0.0 us, 21.3 sy, 78.7 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu21 :  0.0 us, 20.6 sy, 79.4 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu22 :  0.0 us, 22.8 sy, 77.2 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu23 :  0.0 us, 23.5 sy, 76.5 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu24 :  4.0 us, 19.5 sy, 74.8 ni,  1.3 id,  0.3 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem : 18846172 total, 15599468 free,  1472208 used,  1774496 buff/cache
KiB Swap:  2097148 total,  2097148 free,        0 used. 17004148 avail Mem 

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND     
 6965 fahclie+  39  19 1959804 214868  13588 R  2390  1.1   1131:17 FahCore_a7  

Paused the WU, stopped the service, edit config.xml thus (just the number of CPUs):

Code: Select all

  GNU nano 2.9.3              /etc/fahclient/config.xml               Modified  

  <!-- Slot Control -->
  <power v='full'/>

  <!-- User Information -->
  <passkey v='XXX'/>
  <team v='182116'/>
  <user v='ajm'/>

  <!-- Folding Slots -->
  <slot id='0' type='CPU'>
    <cpus v='12'/>
    <paused v='true'/>
  </slot>
</config>

Started the client. Top:

Code: Select all

top - 13:42:51 up 1 day, 20:42,  1 user,  load average: 3.37, 13.44, 19.99
Tasks: 420 total,   2 running, 280 sleeping,   0 stopped,   6 zombie
%Cpu0  :  0.0 us, 11.3 sy, 88.7 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu1  :  0.3 us, 10.6 sy, 87.4 ni,  1.7 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu2  :  1.7 us,  0.0 sy,  0.0 ni, 98.3 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu3  :  0.7 us,  0.3 sy,  1.3 ni, 97.7 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu4  :  0.0 us, 12.6 sy, 87.4 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu5  :  0.0 us, 14.3 sy, 85.7 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu6  :  0.3 us,  0.0 sy,  0.0 ni, 99.7 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu7  :  0.0 us, 10.6 sy, 89.4 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu8  :  0.0 us,  0.0 sy,  0.0 ni,100.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu9  :  0.0 us, 12.6 sy, 87.4 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu10 :  0.3 us,  0.0 sy,  0.0 ni, 99.7 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu11 :  0.0 us, 11.6 sy, 86.8 ni,  1.7 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu12 :  0.3 us,  0.0 sy,  0.0 ni, 99.7 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu13 :  0.3 us,  0.0 sy,  0.0 ni, 99.7 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu14 :  0.3 us,  0.0 sy,  0.0 ni, 99.7 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu15 :  0.3 us,  0.0 sy,  0.0 ni, 99.7 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu16 :  2.0 us,  0.7 sy,  1.3 ni, 96.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu17 :  0.3 us,  0.3 sy,  0.0 ni, 99.3 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu18 :  0.0 us, 12.3 sy, 87.7 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu19 :  0.3 us,  0.3 sy,  0.0 ni, 99.3 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND     

[PeterGarlic]

mmm... seems that you have a better CPU load distribution than me.
I will try with run one instance with more vCpu to seee if something change.

Just for my personal curiosity:
this is the configuration you wrote me before (Win10, Virtualbox 6.1.6, Ubuntu 18.04)
What type of cpu are you using?

[ajm]

Yes, it is the configuration I was talking about.
X299 with an i9 7940X (14C/28T) running windows 10 Pro and VirtualBox 6.1.6 with an Ubuntu 18.04 guest and FAH (7.5) running within the guest.

[PantherX]

...

Code: Select all

<config>
	...
  <power v='light'/>
...
  <slot id='0' type='CPU'/>

</config>

I set basic options: usage of 1 of 4 CPU and power light. After I start the client top show me this (pressing 1 to see all cpu)

Code: Select all

top - 12:08:40 up  1:26,  1 user,  load average: 0,94, 0,46, 0,23
Tasks: 261 total,   2 running, 200 sleeping,   0 stopped,  10 zombie
%Cpu0  :  0,3 us,  0,0 sy,  0,0 ni, 99,7 id,  0,0 wa,  0,0 hi,  0,0 si,  0,0 st
%Cpu1  :  0,0 us,  0,0 sy,100,0 ni,  0,0 id,  0,0 wa,  0,0 hi,  0,0 si,  0,0 st
%Cpu2  :  0,3 us,  0,3 sy,  0,0 ni, 99,3 id,  0,0 wa,  0,0 hi,  0,0 si,  0,0 st
%Cpu3  :  0,0 us,  0,0 sy,  0,0 ni,100,0 id,  0,0 wa,  0,0 hi,  0,0 si,  0,0 st
KiB Mem :  4030252 total,  2115832 free,  1064676 used,   849744 buff/cache
KiB Swap:   999420 total,   999420 free,        0 used.  2728880 avail Mem

One cpu at 100% ignoring the <power v='light'/> directive...

The expected behavior of having light on 4 CPUs is 2 CPUs being used. In your example, only 1 CPU is being used and there's an edge case. If the WU was downloaded before the CPU slot was configured, it will only run on a single CPU thread regardless of the power value and the number of CPUs you have. Once that WU has completed, it will download the correct WU depending on what settings you have used. Checking the log will provide additional details to explain this situation.

Generally speaking, you can decrease the number of CPUs to an assigned WU but you can't increase it more than the value the WU had when it was first assigned.

[bruce]

FAH doesn't specify which CPU it will use, only how much ... it's up to the scheduler in the OS to pick the available CPU. From the perspective of the program, 100% of 1 CPU is the same as 50% of two CPUs. or 33% of three CPUs.