WU Shortages

[Foliant]

My Phenom II X6 1100T Spaceheater is also 'cold' on CPU Workunits.

[kwthom]

More...

21:08:41:WARNING:WU02:FS00:Failed to get assignment from '65.254.110.245:80': No WUs available for this configuration
21:08:41:WU02:FS00:Connecting to 18.218.241.186:80
21:08:42:WARNING:WU02:FS00:Failed to get assignment from '18.218.241.186:80': No WUs available for this configuration
21:08:42:WU02:FS00:Connecting to 65.254.110.245:80

GPU is still keeping the place warm, though...

[PantherX]

Restarting your system causes the client to restart which resets the retry counter. Thus, it can re-try quite a few times within the first 5 minutes giving the illusion that restarting the system works. You can get the same result if you were to pause and then unpause the Slots. However, the idea to have an exponential back-off (capped to 1 hour) is to prevent a friendly DDoS on the Servers.

[bruce]

the idea to have an exponential back-off (capped to 1 hour) is to prevent a friendly DDoS on the Servers.

So when you're impatient, you are intentionally contributing to DDoS on a server which might be frantically trying to recover from some down-time. Yes, you might be able to jump in front of others who are patiently waiting for their next turn at getting an assignment. Anything for a few extra points when there's a competition/challenge going on.

[ComputerUser]

Is there another CPU WU shortage at the moment? For the last 36 hours my cpu:4 folding slots take 1-6 hrs between WUs to get the next assignment. It seems that I'm mostly getting P17231 from WS 206.223.170.146, which has 0 public jobs most of the time according to the server stats. I wonder why I'm not getting any A8 WU from 129.32.209.203 or 143.89.243.111, they have thousands available and P16520 was even announced today.


This is one of my machines sitting there for over an hour doing nothing:

Code: Select all

09:52:33:WU00:FS00:Connecting to 65.254.110.245:8080
09:52:34:WU00:FS00:Assigned to work server 206.223.170.146
09:52:34:WU00:FS00:Requesting new work unit for slot 00: READY cpu:4 from 206.223.170.146
09:52:34:WU00:FS00:Connecting to 206.223.170.146:8080
09:52:34:WU00:FS00:Downloading 31.14MiB
09:52:40:WU00:FS00:Download 14.65%
09:52:46:WU00:FS00:Download 28.10%
09:52:52:WU00:FS00:Download 38.14%
09:52:58:WU00:FS00:Download 60.42%
09:53:04:WU00:FS00:Download 90.12%
09:53:05:WU00:FS00:Download complete
09:53:05:WU00:FS00:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:17231 run:5645 clone:2 gen:17 core:0xa7 unit:0x00000002000000110000434f0000160d
09:53:06:WU00:FS00:Starting
09:53:06:WU00:FS00:Running FahCore: "C:\Program Files\FAHClient/FAHCoreWrapper.exe" C:\Users\user\AppData\Roaming\FAHClient\cores/cores.foldingathome.org/win/64bit-avx-256/a7-0.0.19/Core_a7.fah/FahCore_a7.exe -dir 00 -suffix 01 -version 704 -lifeline 804 -checkpoint 15 -np 4
09:53:06:WU00:FS00:Started FahCore on PID 7668
09:53:06:WU00:FS00:Core PID:1152
09:53:06:WU00:FS00:FahCore 0xa7 started
09:53:06:WU00:FS00:0xa7:*********************** Log Started 2021-03-15T09:53:06Z ***********************
09:53:06:WU00:FS00:0xa7:************************** Gromacs Folding@home Core ***************************
09:53:06:WU00:FS00:0xa7:       Type: 0xa7
09:53:06:WU00:FS00:0xa7:       Core: Gromacs
09:53:06:WU00:FS00:0xa7:       Args: -dir 00 -suffix 01 -version 704 -lifeline 7668 -checkpoint 15 -np 4
09:53:06:WU00:FS00:0xa7:************************************ CBang *************************************
09:53:06:WU00:FS00:0xa7:       Date: Nov 27 2019
09:53:06:WU00:FS00:0xa7:       Time: 03:40:09
09:53:06:WU00:FS00:0xa7:   Revision: d25803215b59272441049dfa05a0a9bf7a6e3c48
09:53:06:WU00:FS00:0xa7:     Branch: master
09:53:06:WU00:FS00:0xa7:   Compiler: Visual C++ 2008
09:53:06:WU00:FS00:0xa7:    Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
09:53:06:WU00:FS00:0xa7:   Platform: win32 10
09:53:06:WU00:FS00:0xa7:       Bits: 64
09:53:06:WU00:FS00:0xa7:       Mode: Release
09:53:06:WU00:FS00:0xa7:************************************ System ************************************
09:53:06:WU00:FS00:0xa7:        CPU: Intel(R) Core(TM) i5-4590 CPU @ 3.30GHz
09:53:06:WU00:FS00:0xa7:     CPU ID: GenuineIntel Family 6 Model 60 Stepping 3
09:53:06:WU00:FS00:0xa7:       CPUs: 4
09:53:06:WU00:FS00:0xa7:     Memory: 7.91GiB
09:53:06:WU00:FS00:0xa7:Free Memory: 5.56GiB
09:53:06:WU00:FS00:0xa7:    Threads: WINDOWS_THREADS
09:53:06:WU00:FS00:0xa7: OS Version: 6.2
09:53:06:WU00:FS00:0xa7:Has Battery: false
09:53:06:WU00:FS00:0xa7: On Battery: false
09:53:06:WU00:FS00:0xa7: UTC Offset: 1
09:53:06:WU00:FS00:0xa7:        PID: 1152
09:53:06:WU00:FS00:0xa7:        CWD: C:\Users\user\AppData\Roaming\FAHClient\work
09:53:06:WU00:FS00:0xa7:******************************** Build - libFAH ********************************
09:53:06:WU00:FS00:0xa7:    Version: 0.0.19
09:53:06:WU00:FS00:0xa7:     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
09:53:06:WU00:FS00:0xa7:  Copyright: 2019 foldingathome.org
09:53:06:WU00:FS00:0xa7:   Homepage: https://foldingathome.org/
09:53:06:WU00:FS00:0xa7:       Date: Nov 25 2019
09:53:06:WU00:FS00:0xa7:       Time: 17:12:41
09:53:06:WU00:FS00:0xa7:   Revision: d5b5c747532224f986b7cd02c968ed9a20c16d6e
09:53:06:WU00:FS00:0xa7:     Branch: master
09:53:06:WU00:FS00:0xa7:   Compiler: Visual C++ 2008
09:53:06:WU00:FS00:0xa7:    Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
09:53:06:WU00:FS00:0xa7:   Platform: win32 10
09:53:06:WU00:FS00:0xa7:       Bits: 64
09:53:06:WU00:FS00:0xa7:       Mode: Release
09:53:06:WU00:FS00:0xa7:************************************ Build *************************************
09:53:06:WU00:FS00:0xa7:       SIMD: avx_256
09:53:06:WU00:FS00:0xa7:********************************************************************************
09:53:06:WU00:FS00:0xa7:Project: 17231 (Run 5645, Clone 2, Gen 17)
09:53:06:WU00:FS00:0xa7:Unit: 0x00000000000000000000000000000000
09:53:06:WU00:FS00:0xa7:Reading tar file core.xml
09:53:06:WU00:FS00:0xa7:Reading tar file frame17.tpr
09:53:06:WU00:FS00:0xa7:Digital signatures verified
09:53:06:WU00:FS00:0xa7:Calling: mdrun -s frame17.tpr -o frame17.trr -x frame17.xtc -cpt 15 -nt 4
09:53:07:WU00:FS00:0xa7:Steps: first=1062500 total=62500
09:53:11:WU00:FS00:0xa7:Completed 1 out of 62500 steps (0%)
09:58:04:WU00:FS00:0xa7:Completed 625 out of 62500 steps (1%)
[...]
18:00:46:WU00:FS00:0xa7:Completed 61875 out of 62500 steps (99%)
18:05:37:WU00:FS00:0xa7:Completed 62500 out of 62500 steps (100%)
18:05:38:WU01:FS00:Connecting to 65.254.110.245:8080
18:05:39:WU01:FS00:Assigned to work server 206.223.170.146
18:05:39:WU01:FS00:Requesting new work unit for slot 00: RUNNING cpu:4 from 206.223.170.146
18:05:39:WU01:FS00:Connecting to 206.223.170.146:8080
18:05:58:WU00:FS00:0xa7:Saving result file ..\logfile_01.txt
18:05:58:WU00:FS00:0xa7:Saving result file frame17.trr
18:05:58:WU00:FS00:0xa7:Saving result file frame17.xtc
18:05:58:WU00:FS00:0xa7:Saving result file md.log
18:05:58:WU00:FS00:0xa7:Saving result file science.log
18:05:58:WU00:FS00:0xa7:Folding@home Core Shutdown: FINISHED_UNIT
18:05:58:WU00:FS00:FahCore returned: FINISHED_UNIT (100 = 0x64)
18:05:58:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:17231 run:5645 clone:2 gen:17 core:0xa7 unit:0x00000002000000110000434f0000160d
18:05:58:WU00:FS00:Uploading 18.71MiB to 206.223.170.146
18:05:58:WU00:FS00:Connecting to 206.223.170.146:8080
18:06:00:WARNING:WU01:FS00:WorkServer connection failed on port 8080 trying 80
18:06:00:WU01:FS00:Connecting to 206.223.170.146:80
18:06:10:ERROR:WU01:FS00:Exception: Server did not assign work unit
18:06:10:WU01:FS00:Connecting to 65.254.110.245:8080
18:06:11:WARNING:WU01:FS00:Failed to get assignment from '65.254.110.245:8080': No WUs available for this configuration
18:06:11:WU01:FS00:Connecting to 18.218.241.186:80
18:06:11:WARNING:WU01:FS00:Failed to get assignment from '18.218.241.186:80': No WUs available for this configuration
18:06:11:ERROR:WU01:FS00:Exception: Could not get an assignment
18:06:15:WU00:FS00:Upload 0.67%
18:06:21:WU00:FS00:Upload 6.01%
18:06:27:WU00:FS00:Upload 17.37%
18:06:33:WU00:FS00:Upload 29.07%
18:06:39:WU00:FS00:Upload 40.43%
18:06:45:WU00:FS00:Upload 52.12%
18:06:51:WU00:FS00:Upload 63.48%
18:06:57:WU00:FS00:Upload 74.84%
18:07:03:WU00:FS00:Upload 86.53%
18:07:09:WU00:FS00:Upload 97.89%
18:07:10:WU00:FS00:Upload complete
18:07:10:WU00:FS00:Server responded WORK_ACK (400)
18:07:10:WU00:FS00:Final credit estimate, 12940.00 points
18:07:10:WU00:FS00:Cleaning up
18:07:10:WU01:FS00:Connecting to 65.254.110.245:8080
18:07:11:WU01:FS00:Assigned to work server 66.170.111.50
18:07:11:WU01:FS00:Requesting new work unit for slot 00: READY cpu:4 from 66.170.111.50
18:07:11:WU01:FS00:Connecting to 66.170.111.50:8080
18:07:12:ERROR:WU01:FS00:Exception: Server did not assign work unit
18:08:47:WU01:FS00:Connecting to 65.254.110.245:8080
18:08:48:WARNING:WU01:FS00:Failed to get assignment from '65.254.110.245:8080': No WUs available for this configuration
18:08:48:WU01:FS00:Connecting to 18.218.241.186:80
18:08:49:WARNING:WU01:FS00:Failed to get assignment from '18.218.241.186:80': No WUs available for this configuration
18:08:49:ERROR:WU01:FS00:Exception: Could not get an assignment
18:11:25:WU01:FS00:Connecting to 65.254.110.245:8080
18:11:25:WARNING:WU01:FS00:Failed to get assignment from '65.254.110.245:8080': No WUs available for this configuration
18:11:25:WU01:FS00:Connecting to 18.218.241.186:80
18:11:26:WARNING:WU01:FS00:Failed to get assignment from '18.218.241.186:80': No WUs available for this configuration
18:11:26:ERROR:WU01:FS00:Exception: Could not get an assignment
18:15:39:WU01:FS00:Connecting to 65.254.110.245:8080
18:15:40:WARNING:WU01:FS00:Failed to get assignment from '65.254.110.245:8080': No WUs available for this configuration
18:15:40:WU01:FS00:Connecting to 18.218.241.186:80
18:15:40:WARNING:WU01:FS00:Failed to get assignment from '18.218.241.186:80': No WUs available for this configuration
18:15:40:ERROR:WU01:FS00:Exception: Could not get an assignment
18:22:30:WU01:FS00:Connecting to 65.254.110.245:8080
18:22:31:WU01:FS00:Assigned to work server 206.223.170.146
18:22:31:WU01:FS00:Requesting new work unit for slot 00: READY cpu:4 from 206.223.170.146
18:22:31:WU01:FS00:Connecting to 206.223.170.146:8080
18:22:44:ERROR:WU01:FS00:Exception: Server did not assign work unit
18:33:36:WU01:FS00:Connecting to 65.254.110.245:8080
18:33:36:WARNING:WU01:FS00:Failed to get assignment from '65.254.110.245:8080': No WUs available for this configuration
18:33:36:WU01:FS00:Connecting to 18.218.241.186:80
18:33:37:WARNING:WU01:FS00:Failed to get assignment from '18.218.241.186:80': No WUs available for this configuration
18:33:37:ERROR:WU01:FS00:Exception: Could not get an assignment
******************************* Date: 2021-03-15 *******************************
18:51:32:WU01:FS00:Connecting to 65.254.110.245:8080
18:51:33:WARNING:WU01:FS00:Failed to get assignment from '65.254.110.245:8080': No WUs available for this configuration
18:51:33:WU01:FS00:Connecting to 18.218.241.186:80
18:51:34:WU01:FS00:Assigned to work server 128.252.203.9
18:51:34:WU01:FS00:Requesting new work unit for slot 00: READY cpu:4 from 128.252.203.9
18:51:34:WU01:FS00:Connecting to 128.252.203.9:8080
18:51:34:ERROR:WU01:FS00:Exception: Server did not assign work unit
19:20:35:WU01:FS00:Connecting to 65.254.110.245:8080
19:20:35:WARNING:WU01:FS00:Failed to get assignment from '65.254.110.245:8080': No WUs available for this configuration
19:20:35:WU01:FS00:Connecting to 18.218.241.186:80
19:20:36:WARNING:WU01:FS00:Failed to get assignment from '18.218.241.186:80': No WUs available for this configuration
19:20:36:ERROR:WU01:FS00:Exception: Could not get an assignment

Cheers,
Computeruser

[gunnarre]

129.32.209.203 or 143.89.243.111 have 2600 public jobs available at the moment but they might not fit the 4 or 8 core thread requirement? My 2-core CPUs get jobs almost immediately, but my 8 core CPU gets assigned to 128.252.203.9 which has only 5 public jobs available.

Edit: I got an 8-core job from 129.32.209.203 now, so it seems the main problem here is being assigned to a work server which has almost no public jobs left (128.252.203.9). Once those jobs get assigned, and the WU reports to the assignment server that it has no CPU jobs to give out, then the issue should resolve itself, I guess?

[wdanwatts]

Recent attempt at CPU:

01:31:03:WU01:FS00:Connecting to 65.254.110.245:8080
01:31:03:WARNING:WU01:FS00:Failed to get assignment from '65.254.110.245:8080': No WUs available for this configuration
01:31:03:WU01:FS00:Connecting to 18.218.241.186:80
01:31:03:WU01:FS00:Assigned to work server 206.223.170.146
01:31:03:WU01:FS00:Requesting new work unit for slot 00: READY cpu:3 from 206.223.170.146
01:31:03:WU01:FS00:Connecting to 206.223.170.146:8080
01:31:04:ERROR:WU01:FS00:Exception: Server did not assign work unit
01:31:12:WU02:FS01:0x22:Completed 510000 out of 750000 steps (68%)
01:31:15:WU02:FS01:0x22:Checkpoint completed at step 510000
01:32:40:WU01:FS00:Connecting to 65.254.110.245:8080
01:32:40:WU01:FS00:Assigned to work server 128.252.203.9
01:32:41:WU01:FS00:Requesting new work unit for slot 00: READY cpu:3 from 128.252.203.9
01:32:41:WU01:FS00:Connecting to 128.252.203.9:8080
01:32:41:ERROR:WU01:FS00:Exception: Server did not assign work unit
01:35:17:WU01:FS00:Connecting to 65.254.110.245:8080
01:35:18:WARNING:WU01:FS00:Failed to get assignment from '65.254.110.245:8080': No WUs available for this configuration
01:35:18:WU01:FS00:Connecting to 18.218.241.186:80
01:35:18:WU01:FS00:Assigned to work server 128.252.203.9
01:35:18:WU01:FS00:Requesting new work unit for slot 00: READY cpu:3 from 128.252.203.9
01:35:18:WU01:FS00:Connecting to 128.252.203.9:8080
01:35:18:ERROR:WU01:FS00:Exception: Server did not assign work unit

[[Ars] For Caitlin]

All my CPU slots (4, 8, 12 cpu) keep getting assigned to the 128.252.203.9 server, which has no work units.

As soon as I posted that, I connected to 129.32.209.204 and got a WU. I guess my complaint helped.

[strombergFs]

no CPU WUs out there ?

20:14:06:WU00:FS00:Connecting to assign1.foldingathome.org:80
20:14:07:WARNING:WU00:FS00:Failed to get assignment from 'assign1.foldingathome.org:80': No WUs available for this configuration
20:14:07:WU00:FS00:Connecting to assign2.foldingathome.org:80
20:14:08:WARNING:WU00:FS00:Failed to get assignment from 'assign2.foldingathome.org:80': No WUs available for this configuration
20:14:08:WU00:FS00:Connecting to assign3.foldingathome.org:80
20:14:08:WARNING:WU00:FS00:Failed to get assignment from 'assign3.foldingathome.org:80': No WUs available for this configuration
20:14:08:WU00:FS00:Connecting to assign4.foldingathome.org:80
20:14:09:WARNING:WU00:FS00:Failed to get assignment from 'assign4.foldingathome.org:80': No WUs available for this configuration
20:14:09:ERROR:WU00:FS00:Exception: Could not get an assignment
20:18:20:WU00:FS00:Connecting to assign1.foldingathome.org:80
20:18:21:WARNING:WU00:FS00:Failed to get assignment from 'assign1.foldingathome.org:80': No WUs available for this configuration
20:18:21:WU00:FS00:Connecting to assign2.foldingathome.org:80
20:18:22:WARNING:WU00:FS00:Failed to get assignment from 'assign2.foldingathome.org:80': No WUs available for this configuration
20:18:22:WU00:FS00:Connecting to assign3.foldingathome.org:80
20:18:23:WARNING:WU00:FS00:Failed to get assignment from 'assign3.foldingathome.org:80': No WUs available for this configuration
20:18:23:WU00:FS00:Connecting to assign4.foldingathome.org:80
20:18:23:WARNING:WU00:FS00:Failed to get assignment from 'assign4.foldingathome.org:80': No WUs available for this configuration
20:18:23:ERROR:WU00:FS00:Exception: Could not get an assignment


20:08:18: CPU: Intel(R) Core(TM) i7-8650U CPU @ 1.90GHz
20:08:18: CPU ID: GenuineIntel Family 6 Model 142 Stepping 10
20:08:18: CPUs: 8
20:08:18: Memory: 7.65GiB

[PaulTV]

The server stats show that there are WUs available, and even that jobs are being assigned, but I didn't get any new CPU nor GPU jobs either since about that time. It'd be strange if both of them run out at the same time, so something else may need attention.

[PaulTV]

The server stats show that there are WUs available, and even that jobs are being assigned, but I didn't get any new CPU nor GPU jobs either since about that time. It'd be strange if both of them run out at the same time, so something else may need attention.

One of my clients got a new WU about an hour ago somthing got unstuck

[mgetz]

At the time I post this Serverstats is only showing 1 assignable A8 unit and zero A7 on a non-zero weighted server... so yeah I'd say that there is a CPU WU shortage at the moment or something is very wonky. Either with serverstats or the assign servers.

Edit: and immediately after I posted this it looks like Vav20 got a dump of about 2.8k CPU A8 units

[gunnarre]

I made a new thread about being assigned to work servers without jobs on them. I also pinged the Discord about it. viewtopic.php?f=18&t=36964

[DeeGee]

Yeah, both my 5950x and 3950x (both set to one 28 thread CPU slot + GPU slot) have been idling the CPU for couple of days now. There has been same kind of WU shortages for high thread counts before, and then I just splitted up CPU to 2x 14 thread slots. I'll do that again to see if there are WU's for lower thread counts.

[edit] Even dropping thread count of slot doesn't seem to help.

[mgetz]

It looks like there are new A8 projects rolling in... but the assignment servers aren't actually giving those servers for work servers. it seems like 206.223.170.146 has extremely high weight for some reason. There are a ton of jobs sitting on 129.32.209.203... but they aren't getting assigned. Right now Serverstats is saying that only 96/hr A8 Units are getting assigned and 0/hr A7.