08:49:55:WU02:FS01:Sending unit results: id:02 state:SEND error:NO_ERROR project:13877 run:0 clone:1712 gen:28 core:0x22 unit:0x0000002634e06d4a5e80cfe9a7206a5f
08:49:55:WU02:FS01:Uploading 48.06MiB to 52.224.109.74
08:49:55:WU02:FS01:Connecting to 52.224.109.74:8080
08:50:16:WARNING:WU02:FS01:WorkServer connection failed on port 8080 trying 80
08:50:16:WU02:FS01:Connecting to 52.224.109.74:80
08:50:37:WARNING:WU02:FS01:Exception: Failed to send results to work server: Failed to connect to 52.224.109.74:80: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
It appears that due to this server being down, the work performed cannot be sent. This is because the client refuses to send it to any other server and just keeps retrying the same down server.
It would be nice to see an update to the result return process, whereby there are one or more alternate servers that will accept a job after 2 failures to the primary server. This way completed work does not end up being abandoned due to timeouts.
I even managed a partial upload before this same server seems to have died mid-transmission.
******************************* Date: 2020-04-14 *******************************
23:40:17:WU01:FS01:0x22:Completed 1000000 out of 1000000 steps (100%)
23:40:37:WU01:FS01:0x22:Saving result file ../logfile_01.txt
23:40:37:WU01:FS01:0x22:Saving result file checkpointState.xml
23:40:38:WU01:FS01:0x22:Saving result file checkpt.crc
23:40:38:WU01:FS01:0x22:Saving result file positions.xtc
23:40:38:WU01:FS01:0x22:Saving result file science.log
23:40:38:WU01:FS01:0x22:Folding@home Core Shutdown: FINISHED_UNIT
23:40:39:WU01:FS01:FahCore returned: FINISHED_UNIT (100 = 0x64)
23:40:39:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:13878 run:0 clone:1035 gen:29 core:0x22 unit:0x0000002534e06d4a5e80cfe5a2e60368
23:40:39:WU01:FS01:Uploading 48.08MiB to 52.224.109.74
23:40:39:WU01:FS01:Connecting to 52.224.109.74:8080
23:42:49:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
23:42:49:WU01:FS01:Connecting to 52.224.109.74:80
23:45:00:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 52.224.109.74:80: Connection timed out
23:45:00:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:13878 run:0 clone:1035 gen:29 core:0x22 unit:0x0000002534e06d4a5e80cfe5a2e60368
23:45:01:WU01:FS01:Uploading 48.08MiB to 52.224.109.74
23:45:01:WU01:FS01:Connecting to 52.224.109.74:8080
23:47:11:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
23:47:11:WU01:FS01:Connecting to 52.224.109.74:80
23:49:22:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 52.224.109.74:80: Connection timed out
23:49:23:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:13878 run:0 clone:1035 gen:29 core:0x22 unit:0x0000002534e06d4a5e80cfe5a2e60368
23:49:23:WU01:FS01:Uploading 48.08MiB to 52.224.109.74
23:49:23:WU01:FS01:Connecting to 52.224.109.74:8080
23:51:33:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
23:51:33:WU01:FS01:Connecting to 52.224.109.74:80
23:53:44:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 52.224.109.74:80: Connection timed out
23:53:45:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:13878 run:0 clone:1035 gen:29 core:0x22 unit:0x0000002534e06d4a5e80cfe5a2e60368
23:53:45:WU01:FS01:Uploading 48.08MiB to 52.224.109.74
23:53:45:WU01:FS01:Connecting to 52.224.109.74:8080
23:55:56:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
23:55:56:WU01:FS01:Connecting to 52.224.109.74:80
23:58:07:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 52.224.109.74:80: Connection timed out
23:58:07:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:13878 run:0 clone:1035 gen:29 core:0x22 unit:0x0000002534e06d4a5e80cfe5a2e60368
23:58:07:WU01:FS01:Uploading 48.08MiB to 52.224.109.74
23:58:07:WU01:FS01:Connecting to 52.224.109.74:8080
00:00:18:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
00:00:18:WU01:FS01:Connecting to 52.224.109.74:80
00:02:29:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 52.224.109.74:80: Connection timed out
00:02:29:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:13878 run:0 clone:1035 gen:29 core:0x22 unit:0x0000002534e06d4a5e80cfe5a2e60368
00:02:29:WU01:FS01:Uploading 48.08MiB to 52.224.109.74
00:02:29:WU01:FS01:Connecting to 52.224.109.74:8080
00:04:40:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
00:04:40:WU01:FS01:Connecting to 52.224.109.74:80
00:06:51:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 52.224.109.74:80: Connection timed out
00:09:20:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:13878 run:0 clone:1035 gen:29 core:0x22 unit:0x0000002534e06d4a5e80cfe5a2e60368
00:09:20:WU01:FS01:Uploading 48.08MiB to 52.224.109.74
00:09:20:WU01:FS01:Connecting to 52.224.109.74:8080
00:11:31:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
00:11:31:WU01:FS01:Connecting to 52.224.109.74:80
00:13:43:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 52.224.109.74:80: Connection timed out
00:20:26:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:13878 run:0 clone:1035 gen:29 core:0x22 unit:0x0000002534e06d4a5e80cfe5a2e60368
00:20:26:WU01:FS01:Uploading 48.08MiB to 52.224.109.74
00:20:26:WU01:FS01:Connecting to 52.224.109.74:8080
00:22:37:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
00:22:37:WU01:FS01:Connecting to 52.224.109.74:80
00:24:48:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 52.224.109.74:80: Connection timed out
00:38:23:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:13878 run:0 clone:1035 gen:29 core:0x22 unit:0x0000002534e06d4a5e80cfe5a2e60368
00:38:23:WU01:FS01:Uploading 48.08MiB to 52.224.109.74
00:38:23:WU01:FS01:Connecting to 52.224.109.74:8080
00:40:32:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
00:40:32:WU01:FS01:Connecting to 52.224.109.74:80
00:42:43:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 52.224.109.74:80: Connection timed out
01:07:25:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:13878 run:0 clone:1035 gen:29 core:0x22 unit:0x0000002534e06d4a5e80cfe5a2e60368
01:07:25:WU01:FS01:Uploading 48.08MiB to 52.224.109.74
01:07:25:WU01:FS01:Connecting to 52.224.109.74:8080
01:15:16:WU01:FS01:Upload 1.43%
01:15:22:WU01:FS01:Upload 1.82%
01:15:44:WU01:FS01:Upload 2.60%
01:16:03:WU01:FS01:Upload 3.12%
01:16:40:WU01:FS01:Upload 3.51%
01:16:50:WU01:FS01:Upload 4.03%
01:17:12:WU01:FS01:Upload 4.42%
01:17:29:WU01:FS01:Upload 7.02%
01:17:35:WU01:FS01:Upload 7.54%
01:17:49:WU01:FS01:Upload 7.93%
01:17:56:WU01:FS01:Upload 8.32%
01:18:05:WU01:FS01:Upload 10.14%
01:18:18:WU01:FS01:Upload 10.53%
01:21:23:WU01:FS01:Upload 10.92%
01:21:23:WARNING:WU01:FS01:Exception: Failed to send results to work server: Transfer failed
01:31:12:WU02:FS01:Connecting to 18.218.241.186:80
01:54:24:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:13878 run:0 clone:1035 gen:29 core:0x22 unit:0x0000002534e06d4a5e80cfe5a2e60368
01:54:24:WU01:FS01:Uploading 48.08MiB to 52.224.109.74
01:54:24:WU01:FS01:Connecting to 52.224.109.74:8080
01:56:33:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
01:56:33:WU01:FS01:Connecting to 52.224.109.74:80
01:58:44:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 52.224.109.74:80: Connection timed out
03:10:25:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:13878 run:0 clone:1035 gen:29 core:0x22 unit:0x0000002534e06d4a5e80cfe5a2e60368
03:10:25:WU01:FS01:Uploading 48.08MiB to 52.224.109.74
03:10:25:WU01:FS01:Connecting to 52.224.109.74:8080
03:12:34:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
03:12:34:WU01:FS01:Connecting to 52.224.109.74:80
03:14:45:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 52.224.109.74:80: Connection timed out
05:13:24:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:13878 run:0 clone:1035 gen:29 core:0x22 unit:0x0000002534e06d4a5e80cfe5a2e60368
05:13:24:WU01:FS01:Uploading 48.08MiB to 52.224.109.74
05:13:24:WU01:FS01:Connecting to 52.224.109.74:8080
05:15:35:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
05:15:35:WU01:FS01:Connecting to 52.224.109.74:80
05:17:46:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 52.224.109.74:80: Connection timed out
******************************* Date: 2020-04-15 *******************************
08:32:25:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:13878 run:0 clone:1035 gen:29 core:0x22 unit:0x0000002534e06d4a5e80cfe5a2e60368
08:32:25:WU01:FS01:Uploading 48.08MiB to 52.224.109.74
08:32:25:WU01:FS01:Connecting to 52.224.109.74:8080
08:34:35:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
08:34:35:WU01:FS01:Connecting to 52.224.109.74:80
08:36:46:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 52.224.109.74:80: Connection timed out
10:12:43:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:13878 run:0 clone:1035 gen:29 core:0x22 unit:0x0000002534e06d4a5e80cfe5a2e60368
10:12:43:WU01:FS01:Uploading 48.08MiB to 52.224.109.74
10:12:43:WU01:FS01:Connecting to 52.224.109.74:8080
10:14:54:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
10:14:54:WU01:FS01:Connecting to 52.224.109.74:80
10:17:05:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 52.224.109.74:80: Connection timed out
10:17:05:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:13878 run:0 clone:1035 gen:29 core:0x22 unit:0x0000002534e06d4a5e80cfe5a2e60368
10:17:05:WU01:FS01:Uploading 48.08MiB to 52.224.109.74
10:17:05:WU01:FS01:Connecting to 52.224.109.74:8080
10:19:16:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
10:19:16:WU01:FS01:Connecting to 52.224.109.74:80
Some WS have CS(s) ... Others for various technical reasons don't ... The team are working on improving this but in the mean time the best that we can do is report issues and be patient whilst the team try to resolve ... even with a CS the WU has to be returned to the WS before the Science can continue (and points accrue iirc).