Page 1 of 2
Client failing to connect to assigned work server
Posted: Sun Mar 15, 2020 3:51 pm
by Dvallin
Code: Select all
*********************** Log Started 2020-03-15T15:37:01Z ***********************
15:37:01:WU01:FS01:Connecting to 65.254.110.245:8080
15:37:02:WU01:FS01:Assigned to work server 140.163.4.231
15:37:02:WU01:FS01:Requesting new work unit for slot 01: READY gpu:0:GP107 [GeForce GTX 1050 Ti] 2138 from 140.163.4.231
15:37:02:WU01:FS01:Connecting to 140.163.4.231:8080
15:37:23:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
15:37:23:WU01:FS01:Connecting to 140.163.4.231:80
15:37:44:ERROR:WU01:FS01:Exception: Failed to connect to 140.163.4.231:80: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
15:37:44:WU01:FS01:Connecting to 65.254.110.245:8080
15:37:45:WU01:FS01:Assigned to work server 140.163.4.231
15:37:45:WU01:FS01:Requesting new work unit for slot 01: READY gpu:0:GP107 [GeForce GTX 1050 Ti] 2138 from 140.163.4.231
15:37:45:WU01:FS01:Connecting to 140.163.4.231:8080
15:38:06:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
15:38:06:WU01:FS01:Connecting to 140.163.4.231:80
15:38:27:ERROR:WU01:FS01:Exception: Failed to connect to 140.163.4.231:80: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
15:38:45:WU01:FS01:Connecting to 65.254.110.245:8080
15:38:45:WU01:FS01:Assigned to work server 140.163.4.231
15:38:45:WU01:FS01:Requesting new work unit for slot 01: READY gpu:0:GP107 [GeForce GTX 1050 Ti] 2138 from 140.163.4.231
15:38:45:WU01:FS01:Connecting to 140.163.4.231:8080
15:39:06:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
15:39:06:WU01:FS01:Connecting to 140.163.4.231:80
15:39:27:ERROR:WU01:FS01:Exception: Failed to connect to 140.163.4.231:80: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
15:40:22:WU01:FS01:Connecting to 65.254.110.245:8080
15:40:22:WU01:FS01:Assigned to work server 140.163.4.231
15:40:22:WU01:FS01:Requesting new work unit for slot 01: READY gpu:0:GP107 [GeForce GTX 1050 Ti] 2138 from 140.163.4.231
15:40:22:WU01:FS01:Connecting to 140.163.4.231:8080
15:40:43:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
15:40:43:WU01:FS01:Connecting to 140.163.4.231:80
15:41:05:ERROR:WU01:FS01:Exception: Failed to connect to 140.163.4.231:80: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
15:42:59:WU01:FS01:Connecting to 65.254.110.245:8080
15:42:59:WU01:FS01:Assigned to work server 140.163.4.241
15:42:59:WU01:FS01:Requesting new work unit for slot 01: READY gpu:0:GP107 [GeForce GTX 1050 Ti] 2138 from 140.163.4.241
15:42:59:WU01:FS01:Connecting to 140.163.4.241:8080
15:43:21:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
15:43:21:WU01:FS01:Connecting to 140.163.4.241:80
15:43:42:ERROR:WU01:FS01:Exception: Failed to connect to 140.163.4.241:80: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
I'm not sure what's going on. The CPU slot seems to work fine, but the GPU slot isn't. Earlier neither were working, then the GPU slot worked, and I restarted to install new graphics drivers and now the GPU slot can't seem to talk to the server. I don't think the graphics drivers are the problem.
Re: Client failing to connect to assigned work server
Posted: Sun Mar 15, 2020 6:09 pm
by Joe_H
Re: Client failing to connect to assigned work server
Posted: Sun Mar 15, 2020 7:19 pm
by markfw
I have had various issues the last 2 days. There are so many, I will just put this one here, as after rebooting twice, it finally got a work unit, but now its back to error.
BTW, I have 12 machines that have been on-line and up for YEARS, and at last for the last several months they have been fine until the last several days. I am number 20 in the stats, so I am quite experienced.
19:14:21:WU01:FS00:Connecting to 140.163.4.241:8080
19:14:24:WU00:FS00:Upload 95.02%
19:14:24:WU00:FS00:Upload complete
19:14:24:WU00:FS00:Server responded WORK_ACK (400)
19:14:24:WU00:FS00:Final credit estimate, 79534.00 points
19:14:24:WU00:FS00:Cleaning up
19:14:42:WARNING:WU01:FS00:WorkServer connection failed on port 8080 trying 80
19:14:42:WU01:FS00:Connecting to 140.163.4.241:80
19:15:03:ERROR:WU01:FS00:Exception: Failed to connect to 140.163.4.241:80: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
19:15:58:WU01:FS00:Connecting to 65.254.110.245:80
19:15:58:WU01:FS00:Assigned to work server 140.163.4.231
19:15:58:WU01:FS00:Requesting new work unit for slot 00: READY gpu:0:GP102 [GeForce GTX 1080 Ti] 11380 from 140.163.4.231
19:15:58:WU01:FS00:Connecting to 140.163.4.231:8080
19:16:20:WARNING:WU01:FS00:WorkServer connection failed on port 8080 trying 80
19:16:20:WU01:FS00:Connecting to 140.163.4.231:80
Here is another box:
Code: Select all
18:49:06:WU00:FS01:Server responded WORK_ACK (400)
18:49:06:WU00:FS01:Final credit estimate, 160772.00 points
18:49:06:WU00:FS01:Cleaning up
18:50:38:ERROR:WU01:FS01:Exception: Failed to connect to 140.163.4.231:80: Connection timed out
18:50:38:WU01:FS01:Connecting to 65.254.110.245:8080
18:50:38:WU01:FS01:Assigned to work server 155.247.164.213
18:50:38:WU01:FS01:Requesting new work unit for slot 01: READY gpu:0:TU102 [GeForce RTX 2080 Ti Rev. A] M 13448 from 155.247.164.213
18:50:38:WU01:FS01:Connecting to 155.247.164.213:8080
18:50:39:ERROR:WU01:FS01:Exception: 10001: Server responded: HTTP_SERVICE_UNAVAILABLE
18:52:15:WU01:FS01:Connecting to 65.254.110.245:8080
18:52:15:WU01:FS01:Assigned to work server 140.163.4.231
18:52:15:WU01:FS01:Requesting new work unit for slot 01: READY gpu:0:TU102 [GeForce RTX 2080 Ti Rev. A] M 13448 from 140.163.4.231
18:52:15:WU01:FS01:Connecting to 140.163.4.231:8080
18:54:25:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
18:54:25:WU01:FS01:Connecting to 140.163.4.231:80
18:56:36:ERROR:WU01:FS01:Exception: Failed to connect to 140.163.4.231:80: Connection timed out
18:56:36:WU01:FS01:Connecting to 65.254.110.245:8080
18:56:37:WU01:FS01:Assigned to work server 140.163.4.241
18:56:37:WU01:FS01:Requesting new work unit for slot 01: READY gpu:0:TU102 [GeForce RTX 2080 Ti Rev. A] M 13448 from 140.163.4.241
18:56:37:WU01:FS01:Connecting to 140.163.4.241:8080
18:58:47:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
18:58:47:WU01:FS01:Connecting to 140.163.4.241:80
18:59:17:ERROR:WU01:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
19:00:51:WU01:FS01:Connecting to 65.254.110.245:8080
19:00:51:WU01:FS01:Assigned to work server 140.163.4.231
19:00:51:WU01:FS01:Requesting new work unit for slot 01: READY gpu:0:TU102 [GeForce RTX 2080 Ti Rev. A] M 13448 from 140.163.4.231
19:00:51:WU01:FS01:Connecting to 140.163.4.231:8080
19:03:01:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
19:03:01:WU01:FS01:Connecting to 140.163.4.231:80
19:05:12:ERROR:WU01:FS01:Exception: Failed to connect to 140.163.4.231:80: Connection timed out
19:07:42:WU01:FS01:Connecting to 65.254.110.245:8080
19:07:42:WU01:FS01:Assigned to work server 140.163.4.241
19:07:42:WU01:FS01:Requesting new work unit for slot 01: READY gpu:0:TU102 [GeForce RTX 2080 Ti Rev. A] M 13448 from 140.163.4.241
19:07:42:WU01:FS01:Connecting to 140.163.4.241:8080
19:08:06:ERROR:WU01:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
19:18:47:WU01:FS01:Connecting to 65.254.110.245:8080
19:18:48:WU01:FS01:Assigned to work server 140.163.4.241
19:18:48:WU01:FS01:Requesting new work unit for slot 01: READY gpu:0:TU102 [GeForce RTX 2080 Ti Rev. A] M 13448 from 140.163.4.241
19:18:48:WU01:FS01:Connecting to 140.163.4.241:8080
19:20:58:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
19:20:58:WU01:FS01:Connecting to 140.163.4.241:80
Re: Client failing to connect to assigned work server
Posted: Sun Mar 15, 2020 7:23 pm
by wdanwatts
- 19:13:20:WU01:FS00:Connecting to 65.254.110.245:8080
19:13:21:WU01:FS00:Assigned to work server 140.163.4.241
19:13:21:WU01:FS00:Requesting new work unit for slot 00: READY gpu:0:TU116 [GeForce GTX 1660] from 140.163.4.241
19:13:21:WU01:FS00:Connecting to 140.163.4.241:8080
19:15:31:WARNING:WU01:FS00:WorkServer connection failed on port 8080 trying 80
19:15:31:WU01:FS00:Connecting to 140.163.4.241:80
19:17:42:ERROR:WU01:FS00:Exception: Failed to connect to 140.163.4.241:80: Connection timed out
19:17:42:WU01:FS00:Connecting to 65.254.110.245:8080
19:17:43:WU01:FS00:Assigned to work server 140.163.4.231
19:17:43:WU01:FS00:Requesting new work unit for slot 00: READY gpu:0:TU116 [GeForce GTX 1660] from 140.163.4.231
19:17:43:WU01:FS00:Connecting to 140.163.4.231:8080
19:19:53:WARNING:WU01:FS00:WorkServer connection failed on port 8080 trying 80
19:19:53:WU01:FS00:Connecting to 140.163.4.231:80
19:22:04:ERROR:WU01:FS00:Exception: Failed to connect to 140.163.4.231:80: Connection timed out
etc. for the last 90+ minutes.
And then it got better:
Code: Select all
19:28:56:WU01:FS00:Connecting to 65.254.110.245:8080
19:28:56:WU01:FS00:Assigned to work server 140.163.4.241
19:28:56:WU01:FS00:Requesting new work unit for slot 00: READY gpu:0:TU116 [GeForce GTX 1660] from 140.163.4.241
19:28:56:WU01:FS00:Connecting to 140.163.4.241:8080
19:30:25:WU01:FS00:Downloading 50.83MiB
19:30:32:WU01:FS00:Download 3.69%
19:30:38:WU01:FS00:Download 5.90%
19:30:45:WU01:FS00:Download 8.24%
19:30:51:WU01:FS00:Download 10.82%
19:30:57:WU01:FS00:Download 13.03%
19:31:03:WU01:FS00:Download 15.98%
19:31:09:WU01:FS00:Download 21.15%
19:31:15:WU01:FS00:Download 32.83%
19:31:21:WU01:FS00:Download 37.62%
19:31:27:WU01:FS00:Download 41.93%
19:31:33:WU01:FS00:Download 48.56%
19:31:39:WU01:FS00:Download 51.88%
19:31:45:WU01:FS00:Download 55.94%
19:31:51:WU01:FS00:Download 61.35%
19:31:58:WU01:FS00:Download 66.15%
19:32:04:WU01:FS00:Download 72.66%
19:32:10:WU01:FS00:Download 78.81%
19:32:16:WU01:FS00:Download 83.85%
19:32:22:WU01:FS00:Download 90.24%
19:32:28:WU01:FS00:Download 96.88%
19:32:30:WU01:FS00:Download complete
19:32:30:WU01:FS00:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:11744 run:0 clone:10031 gen:0 core:0x22 unit:0x000000008ca304f15e6d7ac3b74f0549
19:32:30:WU01:FS00:Starting
19:32:30:WU01:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/v7/lin/64bit/Core_22.fah/FahCore_22 -dir 01 -suffix 01 -version 705 -lifeline 1235 -checkpoint 30 -gpu-vendor nvidia -opencl-device 0 -cuda-device 0 -gpu 0
19:32:30:WU01:FS00:Started FahCore on PID 16571
19:32:30:WU01:FS00:Core PID:16575
19:32:30:WU01:FS00:FahCore 0x22 started
19:32:30:WU01:FS00:0x22:*********************** Log Started 2020-03-15T19:32:30Z ***********************
19:32:30:WU01:FS00:0x22:*************************** Core22 Folding@home Core ***************************
19:32:30:WU01:FS00:0x22: Type: 0x22
19:32:30:WU01:FS00:0x22: Core: Core22
19:32:30:WU01:FS00:0x22: Website: https://foldingathome.org/
19:32:30:WU01:FS00:0x22: Copyright: (c) 2009-2018 foldingathome.org
19:32:30:WU01:FS00:0x22: Author: John Chodera <john.chodera@choderalab.org> and Rafal Wiewiora
19:32:30:WU01:FS00:0x22: <rafal.wiewiora@choderalab.org>
19:32:30:WU01:FS00:0x22: Args: -dir 01 -suffix 01 -version 705 -lifeline 16571 -checkpoint 30
19:32:30:WU01:FS00:0x22: -gpu-vendor nvidia -opencl-device 0 -cuda-device 0 -gpu 0
19:32:30:WU01:FS00:0x22: Config: <none>
19:32:30:WU01:FS00:0x22:************************************ Build *************************************
19:32:30:WU01:FS00:0x22: Version: 0.0.2
19:32:30:WU01:FS00:0x22: Date: Dec 6 2019
19:32:30:WU01:FS00:0x22: Time: 21:20:17
19:32:30:WU01:FS00:0x22: Repository: Git
19:32:30:WU01:FS00:0x22: Revision: f87d92b58abdf7e6bf2e173cfbc4dc3e837c7042
19:32:30:WU01:FS00:0x22: Branch: core22
19:32:30:WU01:FS00:0x22: Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
19:32:30:WU01:FS00:0x22: Options: -std=gnu++98 -O3 -funroll-loops
19:32:30:WU01:FS00:0x22: Platform: linux2 4.9.87-linuxkit-aufs
19:32:30:WU01:FS00:0x22: Bits: 64
19:32:30:WU01:FS00:0x22: Mode: Release
19:32:30:WU01:FS00:0x22:************************************ System ************************************
19:32:30:WU01:FS00:0x22: CPU: AMD Phenom(tm) II X2 545 Processor
19:32:30:WU01:FS00:0x22: CPU ID: AuthenticAMD Family 16 Model 4 Stepping 2
19:32:30:WU01:FS00:0x22: CPUs: 2
19:32:30:WU01:FS00:0x22: Memory: 3.83GiB
19:32:30:WU01:FS00:0x22:Free Memory: 491.37MiB
19:32:30:WU01:FS00:0x22: Threads: POSIX_THREADS
19:32:30:WU01:FS00:0x22: OS Version: 5.4
19:32:30:WU01:FS00:0x22:Has Battery: false
19:32:30:WU01:FS00:0x22: On Battery: false
19:32:30:WU01:FS00:0x22: UTC Offset: -5
19:32:30:WU01:FS00:0x22: PID: 16575
19:32:30:WU01:FS00:0x22: CWD: /var/lib/fahclient/work
19:32:30:WU01:FS00:0x22: OS: Linux 5.4.20-200.fc31.x86_64 x86_64
19:32:30:WU01:FS00:0x22: OS Arch: AMD64
19:32:30:WU01:FS00:0x22:********************************************************************************
19:32:30:WU01:FS00:0x22:Project: 11744 (Run 0, Clone 10031, Gen 0)
19:32:30:WU01:FS00:0x22:Unit: 0x000000008ca304f15e6d7ac3b74f0549
19:32:30:WU01:FS00:0x22:Reading tar file core.xml
19:32:30:WU01:FS00:0x22:Reading tar file integrator.xml
19:32:30:WU01:FS00:0x22:Reading tar file state.xml
19:32:30:WU01:FS00:0x22:Reading tar file system.xml
19:32:30:WU01:FS00:0x22:Digital signatures verified
19:32:30:WU01:FS00:0x22:Folding@home GPU Core22 Folding@home Core
19:32:30:WU01:FS00:0x22:Version 0.0.2
19:33:15:WU01:FS00:0x22:Completed 0 out of 1000000 steps (0%)
19:33:15:WU01:FS00:0x22:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
19:34:59:WU01:FS00:0x22:Completed 10000 out of 1000000 steps (1%)
Re: Client failing to connect to assigned work server
Posted: Mon Mar 16, 2020 4:14 pm
by markfw
8 of 12 still trying to get work.
Re: Client failing to connect to assigned work server
Posted: Mon Mar 16, 2020 4:22 pm
by JimboPalmer
It is just now 9 AM in CA where the Standford campus is. Soon researchers should have their coffee and start reading their emails.
I am hoping they can then generate enough WUs so we have work (yes, both My GPUs and one of my CPUs are looking for work)
I am hoping this is the last hour I need to make excuses!
Re: Client failing to connect to assigned work server
Posted: Mon Mar 16, 2020 4:32 pm
by Joe_H
Though researchers at Temple, Washington University St Louis and MSKCC are already up and working away.
There are just a few still working at Stanford since Dr. Pande left for a job in the private sector. Former grad students and post-docs are carrying on the F@h project elsewhere as the Folding@home Consortium. Dr. Bowman at WUSTL is the current director.
Re: Client failing to connect to assigned work server
Posted: Mon Mar 16, 2020 8:33 pm
by markfw
I have 10 of 12 GPU's idle. My PPD has gone from 18 million down to 3. This is ridiculous. This has been going on since they announced the coronavirus units.
While its good they did that, its also counter-productive to not have any work for us to do !
Re: Client failing to connect to assigned work server
Posted: Mon Mar 16, 2020 8:37 pm
by Nathan_P
They know that but they were not expecting such a huge response and it takes days to spin up additional servers, even if they have the hardware on hand. Its getting better - very slowly
Re: Client failing to connect to assigned work server
Posted: Mon Mar 16, 2020 9:14 pm
by markfw
Its getting worse for me. I was at 8 working, then 6, then 4, now 2 !
I even volunteered my 7551 EPYC server to help them (PM to a site admin)
Re: Client failing to connect to assigned work server
Posted: Mon Mar 16, 2020 10:44 pm
by heikosch
Still no work assignment!
Re: Client failing to connect to assigned work server
Posted: Mon Mar 16, 2020 10:50 pm
by petnek
All day my two PC's can't get assignment for CPU job and even GPU had a problem. According to server status page, work units should be there. Where is the problem?
- 22:45:48:WU02:FS00:Connecting to 65.254.110.245:8080
22:45:48:WARNING:WU02:FS00:Failed to get assignment from '65.254.110.245:8080': No WUs available for this configuration
22:45:48:WU02:FS00:Connecting to 18.218.241.186:80
22:45:48:WARNING:WU02:FS00:Failed to get assignment from '18.218.241.186:80': No WUs available for this configuration
22:45:48:ERROR:WU02:FS00:Exception: Could not get an assignment
This is from Ryzen 5 2600, 32GB RAM. No WUs available for this configuration? This is happening with or without -adv flag. Underscored server is not even on the server stats page
Re: Client failing to connect to assigned work server
Posted: Mon Mar 16, 2020 11:03 pm
by Joe_H
18.218.241.186 is one of the Assignment Servers located in the cloud, hosted on AWS. At times it has shown up in the Server Status page, not sure why it is not at the moment.
Otherwise they are still working on expanding resources to be able to provide more assignments to both GPU and CPU folding. But the demand is higher than the current servers can keep up with, so it may take time for your request to go in at a time something is available.
Re: Client failing to connect to assigned work server
Posted: Tue Mar 17, 2020 12:06 am
by Sandman192
I think the is a love-hate thing.
On one hand, our GPUs are folding so fast that Folding at Home can't keep up and were getting are work done QUICK.
On the other hand, we want our GPUs to keep running with something to do.
No 1080Ti GPU work for me today.
'18.218.241.186:80': No WUs available for this configuration
23:51:54:ERROR:WU01:FS01:Exception: Could not get an assignment
23:58:45:WARNING:WU01:FS01:Failed to get assignment from '65.254.110.245:8080': No WUs available for this configuration
Re: Client failing to connect to assigned work server
Posted: Tue Mar 17, 2020 12:17 am
by bruce
Understood.
Thos who administer the servers on each campus are tearing their hair out trying to combat what is (essentially) a DoS attack caused by many, many people who really want to help fight the Corona Virus. Gearing up from our normal level of adacemic activity to where we are now has happened EXTREMELY rapidly.