Unable to download new WU

rhavern · Post by **rhavern** » Mon Dec 07, 2015 7:07 am

Woke up to a cold folding machine, I had a similar issue as reported previously about dumping 92xx WU, but mine went further by not being able to download a new WU. I restarted the client but still no joy. I switched to client-type beta and was able to get a WU:

Code: Select all

*********************** Log Started 2015-12-07T06:56:43Z ***********************
06:56:43:************************* Folding@home Client *************************
06:56:43:      Website: http://folding.stanford.edu/
06:56:43:    Copyright: (c) 2009-2014 Stanford University
06:56:43:       Author: Joseph Coffland <joseph@cauldrondevelopment.com>
06:56:43:         Args: --open-web-control
06:56:43:       Config: C:/Users/Rick/AppData/Roaming/FAHClient/config.xml
06:56:43:******************************** Build ********************************
06:56:43:      Version: 7.4.4
06:56:43:         Date: Mar 4 2014
06:56:43:         Time: 20:26:54
06:56:43:      SVN Rev: 4130
06:56:43:       Branch: fah/trunk/client
06:56:43:     Compiler: Intel(R) C++ MSVC 1500 mode 1200
06:56:43:      Options: /TP /nologo /EHa /Qdiag-disable:4297,4103,1786,279 /Ox -arch:SSE
06:56:43:               /QaxSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 /Qopenmp /Qrestrict /MT /Qmkl
06:56:43:     Platform: win32 XP
06:56:43:         Bits: 32
06:56:43:         Mode: Release
06:56:43:******************************* System ********************************
06:56:43:          CPU: Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz
06:56:43:       CPU ID: GenuineIntel Family 6 Model 15 Stepping 11
06:56:43:         CPUs: 4
06:56:43:       Memory: 6.00GiB
06:56:43:  Free Memory: 4.72GiB
06:56:43:      Threads: WINDOWS_THREADS
06:56:43:   OS Version: 6.1
06:56:43:  Has Battery: false
06:56:43:   On Battery: false
06:56:43:   UTC Offset: 0
06:56:43:          PID: 1596
06:56:43:          CWD: C:/Users/Rick/AppData/Roaming/FAHClient
06:56:43:           OS: Windows 7 Professional
06:56:43:      OS Arch: AMD64
06:56:43:         GPUs: 2
06:56:43:        GPU 0: ATI:5 Tahiti XT [Radeon R9 200/HD 7900/8970]
06:56:43:        GPU 1: ATI:5 Tahiti XT [Radeon R9 200/HD 7900/8970]
06:56:43:         CUDA: Not detected
06:56:43:Win32 Service: false
06:56:43:***********************************************************************
06:56:43:<config>
06:56:43:  <!-- HTTP Server -->
06:56:43:  <allow v='127.0.0.1,192.168.1.0/24'/>
06:56:43:
06:56:43:  <!-- Network -->
06:56:43:  <proxy v=':8080'/>
06:56:43:
06:56:43:  <!-- Remote Command Server -->
06:56:43:  <command-allow-no-pass v='127.0.0.1,192.168.1.0/24'/>
06:56:43:
06:56:43:  <!-- Slot Control -->
06:56:43:  <power v='full'/>
06:56:43:
06:56:43:  <!-- User Information -->
06:56:43:  <passkey v='********************************'/>
06:56:43:  <team v='33'/>
06:56:43:  <user v='rhavern'/>
06:56:43:
06:56:43:  <!-- Folding Slots -->
06:56:43:  <slot id='1' type='GPU'/>
06:56:43:  <slot id='2' type='GPU'/>
06:56:43:</config>
06:56:43:Trying to access database...
06:56:43:Successfully acquired database lock
06:56:43:Enabled folding slot 01: READY gpu:0:Tahiti XT [Radeon R9 200/HD 7900/8970]
06:56:43:Enabled folding slot 02: READY gpu:1:Tahiti XT [Radeon R9 200/HD 7900/8970]
06:56:43:WU00:FS01:Connecting to 171.67.108.45:80
06:56:43:WU01:FS02:Connecting to 171.67.108.45:80
06:56:44:WARNING:WU01:FS02:Failed to get assignment from '171.67.108.45:80': Empty work server assignment
06:56:44:WU01:FS02:Connecting to 171.67.108.204:80
06:56:45:WARNING:WU00:FS01:Failed to get assignment from '171.67.108.45:80': Empty work server assignment
06:56:45:WU00:FS01:Connecting to 171.67.108.204:80
06:56:45:WARNING:WU01:FS02:Failed to get assignment from '171.67.108.204:80': Empty work server assignment
06:56:45:ERROR:WU01:FS02:Exception: Could not get an assignment
06:56:45:WARNING:WU00:FS01:Failed to get assignment from '171.67.108.204:80': Empty work server assignment
06:56:45:ERROR:WU00:FS01:Exception: Could not get an assignment
06:56:45:WU01:FS02:Connecting to 171.67.108.45:80
06:56:45:WU00:FS01:Connecting to 171.67.108.45:80
06:56:46:WARNING:WU00:FS01:Failed to get assignment from '171.67.108.45:80': Empty work server assignment
06:56:46:WU00:FS01:Connecting to 171.67.108.204:80
06:56:46:WARNING:WU01:FS02:Failed to get assignment from '171.67.108.45:80': Empty work server assignment
06:56:46:WU01:FS02:Connecting to 171.67.108.204:80
06:56:47:WARNING:WU01:FS02:Failed to get assignment from '171.67.108.204:80': Empty work server assignment
06:56:47:ERROR:WU01:FS02:Exception: Could not get an assignment
06:56:47:WARNING:WU00:FS01:Failed to get assignment from '171.67.108.204:80': Empty work server assignment
06:56:47:ERROR:WU00:FS01:Exception: Could not get an assignment
06:56:47:11:127.0.0.1:New Web connection
06:57:46:WU00:FS01:Connecting to 171.67.108.45:80
06:57:46:WU01:FS02:Connecting to 171.67.108.45:80
06:57:46:WARNING:WU00:FS01:Failed to get assignment from '171.67.108.45:80': Empty work server assignment
06:57:46:WU00:FS01:Connecting to 171.67.108.204:80
06:57:47:WARNING:WU01:FS02:Failed to get assignment from '171.67.108.45:80': Empty work server assignment
06:57:47:WU01:FS02:Connecting to 171.67.108.204:80
06:57:47:WARNING:WU00:FS01:Failed to get assignment from '171.67.108.204:80': Empty work server assignment
06:57:47:ERROR:WU00:FS01:Exception: Could not get an assignment
06:57:47:WARNING:WU01:FS02:Failed to get assignment from '171.67.108.204:80': Empty work server assignment
06:57:47:ERROR:WU01:FS02:Exception: Could not get an assignment
06:59:23:WU00:FS01:Connecting to 171.67.108.45:80
06:59:23:WU01:FS02:Connecting to 171.67.108.45:80
06:59:24:WARNING:WU00:FS01:Failed to get assignment from '171.67.108.45:80': Empty work server assignment
06:59:24:WU00:FS01:Connecting to 171.67.108.204:80
06:59:24:WARNING:WU01:FS02:Failed to get assignment from '171.67.108.45:80': Empty work server assignment
06:59:24:WU01:FS02:Connecting to 171.67.108.204:80
06:59:24:WARNING:WU00:FS01:Failed to get assignment from '171.67.108.204:80': Empty work server assignment
06:59:24:ERROR:WU00:FS01:Exception: Could not get an assignment
06:59:25:WARNING:WU01:FS02:Failed to get assignment from '171.67.108.204:80': Empty work server assignment
06:59:25:ERROR:WU01:FS02:Exception: Could not get an assignment
07:01:13:Saving configuration to config.xml
07:01:13:<config>
07:01:13:  <!-- Folding Slot Configuration -->
07:01:13:  <client-type v='beta'/>
07:01:13:
07:01:13:  <!-- HTTP Server -->
07:01:13:  <allow v='127.0.0.1,192.168.1.0/24'/>
07:01:13:
07:01:13:  <!-- Network -->
07:01:13:  <proxy v=':8080'/>
07:01:13:
07:01:13:  <!-- Remote Command Server -->
07:01:13:  <command-allow-no-pass v='127.0.0.1,192.168.1.0/24'/>
07:01:13:
07:01:13:  <!-- Slot Control -->
07:01:13:  <power v='full'/>
07:01:13:
07:01:13:  <!-- User Information -->
07:01:13:  <passkey v='********************************'/>
07:01:13:  <team v='33'/>
07:01:13:  <user v='rhavern'/>
07:01:13:
07:01:13:  <!-- Folding Slots -->
07:01:13:  <slot id='1' type='GPU'/>
07:01:13:  <slot id='2' type='GPU'/>
07:01:13:</config>
07:01:48:Saving configuration to config.xml
07:01:48:<config>
07:01:48:  <!-- Folding Slot Configuration -->
07:01:48:  <client-type v='beta'/>
07:01:48:
07:01:48:  <!-- HTTP Server -->
07:01:48:  <allow v='127.0.0.1,192.168.1.0/24'/>
07:01:48:
07:01:48:  <!-- Network -->
07:01:48:  <proxy v=':8080'/>
07:01:48:
07:01:48:  <!-- Remote Command Server -->
07:01:48:  <command-allow-no-pass v='127.0.0.1,192.168.1.0/24'/>
07:01:48:
07:01:48:  <!-- Slot Control -->
07:01:48:  <power v='full'/>
07:01:48:
07:01:48:  <!-- User Information -->
07:01:48:  <passkey v='********************************'/>
07:01:48:  <team v='33'/>
07:01:48:  <user v='rhavern'/>
07:01:48:
07:01:48:  <!-- Folding Slots -->
07:01:48:  <slot id='1' type='GPU'/>
07:01:48:  <slot id='2' type='GPU'/>
07:01:48:</config>
07:02:00:WU00:FS01:Connecting to 171.67.108.45:80
07:02:00:WU01:FS02:Connecting to 171.67.108.45:80
07:02:01:WU00:FS01:Assigned to work server 140.163.4.245
07:02:01:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:Tahiti XT [Radeon R9 200/HD 7900/8970] from 140.163.4.245
07:02:01:WU00:FS01:Connecting to 140.163.4.245:8080
07:02:01:WU01:FS02:Assigned to work server 171.67.108.159
07:02:01:WU01:FS02:Requesting new work unit for slot 02: READY gpu:1:Tahiti XT [Radeon R9 200/HD 7900/8970] from 171.67.108.159
07:02:01:WU01:FS02:Connecting to 171.67.108.159:8080
07:02:01:WU00:FS01:Downloading 5.13MiB
07:02:02:WU01:FS02:Downloading 19.19MiB
07:02:05:WU00:FS01:Download complete
07:02:05:WU00:FS01:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:10495 run:39 clone:1 gen:99 core:0x21 unit:0x0000006f8ca304f555c39d6729438a87
07:02:05:WU00:FS01:Downloading core from http://web.stanford.edu/~pande/Win32/AMD64/ATI/R600/beta/Core_21.fah
07:02:05:WU00:FS01:Connecting to web.stanford.edu:80
07:02:08:WU01:FS02:Download 20.84%
07:02:09:WU00:FS01:FahCore 21: Downloading 3.35MiB
07:02:14:WU01:FS02:Download 54.06%
07:02:15:WU00:FS01:FahCore 21: 42.96%
07:02:18:WU01:FS02:Download complete
07:02:18:WU01:FS02:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:9152 run:16 clone:3 gen:0 core:0x18 unit:0x00000001ab436c9f56623c607990541e
07:02:18:WU01:FS02:Downloading core from http://web.stanford.edu/~pande/Win32/AMD64/ATI/R600/beta/Core_18.fah
07:02:18:WU01:FS02:Connecting to web.stanford.edu:80
07:02:19:WU01:FS02:FahCore 18: Downloading 2.54MiB
07:02:21:WU00:FS01:FahCore 21: 84.05%
07:02:22:WU00:FS01:FahCore 21: Download complete
07:02:22:WU00:FS01:Valid core signature
07:02:22:WU00:FS01:Unpacked 11.44MiB to cores/web.stanford.edu/~pande/Win32/AMD64/ATI/R600/beta/Core_21.fah/FahCore_21.exe
07:02:22:WU00:FS01:Starting
07:02:22:WU00:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Rick/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/ATI/R600/beta/Core_21.fah/FahCore_21.exe -dir 00 -suffix 01 -version 704 -lifeline 1596 -checkpoint 15 -gpu 0 -gpu-vendor ati
07:02:22:WU00:FS01:Started FahCore on PID 1868
07:02:23:WU00:FS01:Core PID:3552
07:02:23:WU00:FS01:FahCore 0x21 started
07:02:24:WU00:FS01:0x21:*********************** Log Started 2015-12-07T07:02:23Z ***********************
07:02:24:WU00:FS01:0x21:Project: 10495 (Run 39, Clone 1, Gen 99)
07:02:24:WU00:FS01:0x21:Unit: 0x0000006f8ca304f555c39d6729438a87
07:02:24:WU00:FS01:0x21:CPU: 0x00000000000000000000000000000000
07:02:24:WU00:FS01:0x21:Machine: 1
07:02:24:WU00:FS01:0x21:Reading tar file core.xml
07:02:24:WU00:FS01:0x21:Reading tar file system.xml
07:02:24:WU00:FS01:0x21:Reading tar file integrator.xml
07:02:24:WU00:FS01:0x21:Reading tar file state.xml
07:02:25:WU01:FS02:FahCore 18: 81.30%
07:02:25:WU00:FS01:0x21:Digital signatures verified
07:02:25:WU00:FS01:0x21:Folding@home GPU Core21 Folding@home Core
07:02:25:WU00:FS01:0x21:Version 0.0.14
07:02:26:WU01:FS02:FahCore 18: Download complete
07:02:26:WU01:FS02:Valid core signature
07:02:26:WU01:FS02:Unpacked 8.47MiB to cores/web.stanford.edu/~pande/Win32/AMD64/ATI/R600/beta/Core_18.fah/FahCore_18.exe
07:02:27:WU01:FS02:Starting
07:02:27:WU01:FS02:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Rick/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/ATI/R600/beta/Core_18.fah/FahCore_18.exe -dir 01 -suffix 01 -version 704 -lifeline 1596 -checkpoint 15 -gpu 1 -gpu-vendor ati
07:02:27:WU01:FS02:Started FahCore on PID 3520
07:02:28:WU01:FS02:Core PID:5068
07:02:28:WU01:FS02:FahCore 0x18 started
07:02:28:WU01:FS02:0x18:*********************** Log Started 2015-12-07T07:02:28Z ***********************
07:02:28:WU01:FS02:0x18:Project: 9152 (Run 16, Clone 3, Gen 0)
07:02:28:WU01:FS02:0x18:Unit: 0x00000001ab436c9f56623c607990541e
07:02:28:WU01:FS02:0x18:CPU: 0x00000000000000000000000000000000
07:02:28:WU01:FS02:0x18:Machine: 2
07:02:28:WU01:FS02:0x18:Reading tar file core.xml
07:02:28:WU01:FS02:0x18:Reading tar file integrator.xml
07:02:28:WU01:FS02:0x18:Reading tar file state.xml
07:02:28:WU01:FS02:0x18:Reading tar file system.xml
07:02:28:WU01:FS02:0x18:Digital signatures verified
07:02:28:WU01:FS02:0x18:Folding@home GPU core18
07:02:28:WU01:FS02:0x18:Version 0.0.4
07:03:03:WU00:FS01:0x21:Completed 0 out of 5000000 steps (0%)
07:03:03:WU00:FS01:0x21:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
07:03:24:WU01:FS02:0x18:Completed 0 out of 2500000 steps (0%)
07:03:24:WU01:FS02:0x18:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900

davidcoton · Post by **davidcoton** » Mon Dec 07, 2015 9:06 am

9201 has been taken offline until the server space can be increased. There seems to be a bigger problem somewhere, I have had problems getting assignments with some settings (which I won't detail here). Either there is a widespread shortage of WUs for some machine types or there are problems with the assignment logic.

No I will not report this on the dreaded reddit.

kwerboom · Post by **kwerboom** » Mon Dec 07, 2015 9:36 am

I'm having the same problem.

Post by **toTOW** » Mon Dec 07, 2015 10:54 am

That's weird ... you should be getting work even with one project disabled ...

PS3EdOlkkola · Post by **PS3EdOlkkola** » Mon Dec 07, 2015 12:41 pm

Having the same issue as the OP. Just one slot out of 34, but it has been unable to download for about 12 hours. Same IP addresses, 171.67.108.45:80 and 171.67.108.204:80 Pausing, exiting and restarting FAH had no effect, same issue.

Post by **toTOW** » Mon Dec 07, 2015 12:55 pm

Does setting max-packet-size = big help ?

rhavern · Post by **rhavern** » Mon Dec 07, 2015 1:27 pm

I set client-type to advanced, and this was also able to pull a new WU. At totow's request, I've set max-packet-size to big and removed the client-type. Next WU drops in about an hour, we'll see what happens then.

Post by **Joe_H** » Mon Dec 07, 2015 2:44 pm

PS3EdOlkkola wrote:Having the same issue as the OP. Just one slot out of 34, but it has been unable to download for about 12 hours. Same IP addresses, 171.67.108.45:80 and 171.67.108.204:80 Pausing, exiting and restarting FAH had no effect, same issue.

Those addresses are the two AS's and do not have WU's to hand out. If they can not match you with a WS it is not going to do any good reporting those addresses. Useful information is which IP addresses of the WS's the machines were getting assigned to in the recent past and the settings that got work. Then those servers can be checked.

PS3EdOlkkola · Post by **PS3EdOlkkola** » Mon Dec 07, 2015 3:08 pm

Joe, I was confirming OP's post that it wasn't just his issue, but something more systemic. The IP addresses referenced were to show it was the same issue and not a different one. Check server 171.67.108.52, the last time the folding slot in question was assigned to the work server at that IP address with a successful wu download.

Post by **Joe_H** » Mon Dec 07, 2015 3:35 pm

Ed, that is just one work server and should not be keeping all assignments from happening. That server was taken offline intentionally due to lack of space - viewtopic.php?f=18&t=28358&p=281482#p281479. So that slot should be eligible for assigning to a different WS depending on what GPU is associated with it and the other client settings.

rhavern · Post by **rhavern** » Mon Dec 07, 2015 3:38 pm

No joy with no client-type flag and max-packet-size set to big:

Code: Select all

15:18:39:WU00:FS01:Upload 97.40%
15:18:42:WU00:FS01:Upload complete
15:18:42:WU00:FS01:Server responded WORK_ACK (400)
15:18:42:WU00:FS01:Cleaning up
15:21:35:WU01:FS01:Connecting to 171.67.108.45:80
15:21:35:WARNING:WU01:FS01:Failed to get assignment from '171.67.108.45:80': Empty work server assignment
15:21:35:WU01:FS01:Connecting to 171.67.108.204:80
15:21:36:WARNING:WU01:FS01:Failed to get assignment from '171.67.108.204:80': Empty work server assignment
15:21:36:ERROR:WU01:FS01:Exception: Could not get an assignment
15:26:42:WU02:FS02:0x21:Completed 275000 out of 2500000 steps (11%)
15:28:26:WU01:FS01:Connecting to 171.67.108.45:80
15:28:27:WARNING:WU01:FS01:Failed to get assignment from '171.67.108.45:80': Empty work server assignment
15:28:27:WU01:FS01:Connecting to 171.67.108.204:80
15:28:28:WARNING:WU01:FS01:Failed to get assignment from '171.67.108.204:80': Empty work server assignment
15:28:28:ERROR:WU01:FS01:Exception: Could not get an assignment

ArVee · Post by **ArVee** » Mon Dec 07, 2015 3:46 pm

PS3EdOlkkola wrote:Having the same issue as the OP. Just one slot out of 34, but it has been unable to download for about 12 hours. Same IP addresses, 171.67.108.45:80 and 171.67.108.204:80 Pausing, exiting and restarting FAH had no effect, same issue.

Same as this (same addy's) under no flags and then still nothing trying -advanced for about 8 hours. Then with no further change (still -advanced flag) started getting 9712's from 171.64.65.98 . Just got another, same deal.

Post by **JohnChodera** » Mon Dec 07, 2015 4:53 pm

I think we may be running low on full FAH jobs since all core 21 projects were rolled back to ADV or earlier. My lab is advancing a few projects to FAH to help make sure sufficient jobs are available.

Will check in with Stanford a bit later today to make sure there are now AS/WS issues.

Post by **bruce** » Mon Dec 07, 2015 5:57 pm

PS3EdOlkkola wrote:Joe, I was confirming OP's post that it wasn't just his issue, but something more systemic. The IP addresses referenced were to show it was the same issue and not a different one. Check server 171.67.108.52, the last time the folding slot in question was assigned to the work server at that IP address with a successful wu download.

Those addresses are still useless information. They do not imply it's the same problem. The "systemic" problem is the answer to the question: Do any of FAH's Work Servers have WUs that can be distribute to me? If YES, point me to that WS. If NO, tell me it can't find any WUs for me anywhere.

Your client contacts an Assignment Server (currently one of those two addresses) and the AS looks at the list of active Work Servers. It eliminates entries from that list that don't have WUs that will run on your system with your settings. If the list is now empty, there are no WUs anywhere for you and it gives that message. If it finds one or more WS, you're redirected to it for it to give you an assignment.

The addresses of the Assignment Servers don't change whether or not it finds a WS that satisfies the needs of your client.

* Maybe there were 6 WS's that had WUs for you and one goes down or one runs out of WUs. You'll never know it because you'll be redirected to one of the other 5.
* Maybe there is 1 WS that has WUs for you and it goes down or runs out of WUs. You'll get that error.
* * * * In both cases, the addresses of the AS that figured that out won't change.

PS3EdOlkkola · Post by **PS3EdOlkkola** » Mon Dec 07, 2015 6:25 pm

If I have a problem getting wu's and another person has the same issue getting wu's referencing the same IP addresses, the likelihood of that being coincidental and two different issues is far less than it being the same issue, therefore more systemic by implication given others have chimed in with the same problem. Regardless of the definition, a problem was reported. Further investigation crawling through older logs on that folding slot provided additional information intended to narrow down the problem. That didn't help. So, anyone with a suggestion on how to improve problem identification on my part -- or for anyone else posting on this issue -- please define the problem identification methodology. Otherwise, I'm out of ideas on what to do next.

Folding Forum

Unable to download new WU

Unable to download new WU

Re: Unable to download new WU

Re: Unable to download new WU

Re: Unable to download new WU

Re: Unable to download new WU

Re: Unable to download new WU

Re: Unable to download new WU

Re: Unable to download new WU

Re: Unable to download new WU

Re: Unable to download new WU

Re: Unable to download new WU

Re: Unable to download new WU

Re: Unable to download new WU

Re: Unable to download new WU

Re: Unable to download new WU