uyaem wrote:The client will always attempt to upload to the Work Server (WS) where it got the WU.
If that one is busy, or unavailable, the collection server (which is just an extended queue as I understand it) will take care of it (if available).
Not all WS have a CS though.
I must disagree with this statement. My own logs prove otherwise. It appears that the designated server for uploading the results will be tried until the work either expires or is finally accepted.
******************************* Date: 2020-04-14 *******************************
23:40:17:WU01:FS01:0x22:Completed 1000000 out of 1000000 steps (100%)
23:40:37:WU01:FS01:0x22:Saving result file ../logfile_01.txt
23:40:37:WU01:FS01:0x22:Saving result file checkpointState.xml
23:40:38:WU01:FS01:0x22:Saving result file checkpt.crc
23:40:38:WU01:FS01:0x22:Saving result file positions.xtc
23:40:38:WU01:FS01:0x22:Saving result file science.log
23:40:38:WU01:FS01:0x22:Folding@home Core Shutdown: FINISHED_UNIT
23:40:39:WU01:FS01:FahCore returned: FINISHED_UNIT (100 = 0x64)
23:40:39:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:13878 run:0 clone:1035 gen:29 core:0x22 unit:0x0000002534e06d4a5e80cfe5a2e60368
23:40:39:WU01:FS01:Uploading 48.08MiB to 52.224.109.74
23:40:39:WU01:FS01:Connecting to 52.224.109.74:8080
23:42:49:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
23:42:49:WU01:FS01:Connecting to 52.224.109.74:80
23:45:00:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 52.224.109.74:80: Connection timed out
23:45:00:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:13878 run:0 clone:1035 gen:29 core:0x22 unit:0x0000002534e06d4a5e80cfe5a2e60368
23:45:01:WU01:FS01:Uploading 48.08MiB to 52.224.109.74
23:45:01:WU01:FS01:Connecting to 52.224.109.74:8080
23:47:11:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
23:47:11:WU01:FS01:Connecting to 52.224.109.74:80
23:49:22:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 52.224.109.74:80: Connection timed out
23:49:23:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:13878 run:0 clone:1035 gen:29 core:0x22 unit:0x0000002534e06d4a5e80cfe5a2e60368
23:49:23:WU01:FS01:Uploading 48.08MiB to 52.224.109.74
23:49:23:WU01:FS01:Connecting to 52.224.109.74:8080
23:51:33:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
23:51:33:WU01:FS01:Connecting to 52.224.109.74:80
23:53:44:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 52.224.109.74:80: Connection timed out
23:53:45:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:13878 run:0 clone:1035 gen:29 core:0x22 unit:0x0000002534e06d4a5e80cfe5a2e60368
23:53:45:WU01:FS01:Uploading 48.08MiB to 52.224.109.74
23:53:45:WU01:FS01:Connecting to 52.224.109.74:8080
23:55:56:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
23:55:56:WU01:FS01:Connecting to 52.224.109.74:80
23:58:07:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 52.224.109.74:80: Connection timed out
23:58:07:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:13878 run:0 clone:1035 gen:29 core:0x22 unit:0x0000002534e06d4a5e80cfe5a2e60368
23:58:07:WU01:FS01:Uploading 48.08MiB to 52.224.109.74
23:58:07:WU01:FS01:Connecting to 52.224.109.74:8080
00:00:18:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
00:00:18:WU01:FS01:Connecting to 52.224.109.74:80
00:02:29:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 52.224.109.74:80: Connection timed out
00:02:29:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:13878 run:0 clone:1035 gen:29 core:0x22 unit:0x0000002534e06d4a5e80cfe5a2e60368
00:02:29:WU01:FS01:Uploading 48.08MiB to 52.224.109.74
00:02:29:WU01:FS01:Connecting to 52.224.109.74:8080
00:04:40:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
00:04:40:WU01:FS01:Connecting to 52.224.109.74:80
00:06:51:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 52.224.109.74:80: Connection timed out
00:09:20:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:13878 run:0 clone:1035 gen:29 core:0x22 unit:0x0000002534e06d4a5e80cfe5a2e60368
00:09:20:WU01:FS01:Uploading 48.08MiB to 52.224.109.74
00:09:20:WU01:FS01:Connecting to 52.224.109.74:8080
00:11:31:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
00:11:31:WU01:FS01:Connecting to 52.224.109.74:80
00:13:43:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 52.224.109.74:80: Connection timed out
00:20:26:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:13878 run:0 clone:1035 gen:29 core:0x22 unit:0x0000002534e06d4a5e80cfe5a2e60368
00:20:26:WU01:FS01:Uploading 48.08MiB to 52.224.109.74
00:20:26:WU01:FS01:Connecting to 52.224.109.74:8080
00:22:37:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
00:22:37:WU01:FS01:Connecting to 52.224.109.74:80
00:24:48:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 52.224.109.74:80: Connection timed out
00:38:23:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:13878 run:0 clone:1035 gen:29 core:0x22 unit:0x0000002534e06d4a5e80cfe5a2e60368
00:38:23:WU01:FS01:Uploading 48.08MiB to 52.224.109.74
00:38:23:WU01:FS01:Connecting to 52.224.109.74:8080
00:40:32:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
00:40:32:WU01:FS01:Connecting to 52.224.109.74:80
00:42:43:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 52.224.109.74:80: Connection timed out
01:07:25:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:13878 run:0 clone:1035 gen:29 core:0x22 unit:0x0000002534e06d4a5e80cfe5a2e60368
01:07:25:WU01:FS01:Uploading 48.08MiB to 52.224.109.74
01:07:25:WU01:FS01:Connecting to 52.224.109.74:8080
01:15:16:WU01:FS01:Upload 1.43%
01:15:22:WU01:FS01:Upload 1.82%
01:15:44:WU01:FS01:Upload 2.60%
01:16:03:WU01:FS01:Upload 3.12%
01:16:40:WU01:FS01:Upload 3.51%
01:16:50:WU01:FS01:Upload 4.03%
01:17:12:WU01:FS01:Upload 4.42%
01:17:29:WU01:FS01:Upload 7.02%
01:17:35:WU01:FS01:Upload 7.54%
01:17:49:WU01:FS01:Upload 7.93%
01:17:56:WU01:FS01:Upload 8.32%
01:18:05:WU01:FS01:Upload 10.14%
01:18:18:WU01:FS01:Upload 10.53%
01:21:23:WU01:FS01:Upload 10.92%
01:21:23:WARNING:WU01:FS01:Exception: Failed to send results to work server: Transfer failed
01:31:12:WU02:FS01:Connecting to 18.218.241.186:80
01:54:24:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:13878 run:0 clone:1035 gen:29 core:0x22 unit:0x0000002534e06d4a5e80cfe5a2e60368
01:54:24:WU01:FS01:Uploading 48.08MiB to 52.224.109.74
01:54:24:WU01:FS01:Connecting to 52.224.109.74:8080
01:56:33:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
01:56:33:WU01:FS01:Connecting to 52.224.109.74:80
01:58:44:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 52.224.109.74:80: Connection timed out
03:10:25:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:13878 run:0 clone:1035 gen:29 core:0x22 unit:0x0000002534e06d4a5e80cfe5a2e60368
03:10:25:WU01:FS01:Uploading 48.08MiB to 52.224.109.74
03:10:25:WU01:FS01:Connecting to 52.224.109.74:8080
03:12:34:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
03:12:34:WU01:FS01:Connecting to 52.224.109.74:80
03:14:45:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 52.224.109.74:80: Connection timed out
05:13:24:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:13878 run:0 clone:1035 gen:29 core:0x22 unit:0x0000002534e06d4a5e80cfe5a2e60368
05:13:24:WU01:FS01:Uploading 48.08MiB to 52.224.109.74
05:13:24:WU01:FS01:Connecting to 52.224.109.74:8080
05:15:35:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
05:15:35:WU01:FS01:Connecting to 52.224.109.74:80
05:17:46:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 52.224.109.74:80: Connection timed out
******************************* Date: 2020-04-15 *******************************
08:32:25:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:13878 run:0 clone:1035 gen:29 core:0x22 unit:0x0000002534e06d4a5e80cfe5a2e60368
08:32:25:WU01:FS01:Uploading 48.08MiB to 52.224.109.74
08:32:25:WU01:FS01:Connecting to 52.224.109.74:8080
08:34:35:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
08:34:35:WU01:FS01:Connecting to 52.224.109.74:80
08:36:46:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 52.224.109.74:80: Connection timed out
10:12:43:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:13878 run:0 clone:1035 gen:29 core:0x22 unit:0x0000002534e06d4a5e80cfe5a2e60368
10:12:43:WU01:FS01:Uploading 48.08MiB to 52.224.109.74
10:12:43:WU01:FS01:Connecting to 52.224.109.74:8080
10:14:54:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
10:14:54:WU01:FS01:Connecting to 52.224.109.74:80
10:17:05:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 52.224.109.74:80: Connection timed out
10:17:05:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:13878 run:0 clone:1035 gen:29 core:0x22 unit:0x0000002534e06d4a5e80cfe5a2e60368
10:17:05:WU01:FS01:Uploading 48.08MiB to 52.224.109.74
10:17:05:WU01:FS01:Connecting to 52.224.109.74:8080
10:19:16:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
10:19:16:WU01:FS01:Connecting to 52.224.109.74:80
This WS has no CS ... https://apps.foldingathome.org/serverstats ... Your log shows it repeatedly trying to get the WU back to it (the WS) ... uyaem's post is correct - the second sentence can only apply if there is a CS as his third sentence indicates - the server stats page indicates there is no CS.
tbonse wrote:I must disagree with this statement. My own logs prove otherwise. It appears that the designated server for uploading the results will be tried until the work either expires or is finally accepted.
As Neil-B posted, your logs do not "prove otherwise", in fact they prove the exact point you are disagreeing with. This particular WS and project does not currently have a designated CS, it would show on your client as 0.0.0.0 for the WU in FAHControl.
iMac 2.8 i7 12 GB smp8, Mac Pro 2.8 quad 12 GB smp6
MacBook Pro 2.9 i7 8 GB smp3
Neil-B wrote:This WS has no CS ... https://apps.foldingathome.org/serverstats ... Your log shows it repeatedly trying to get the WU back to it (the WS) ... uyaem's post is correct - the second sentence can only apply if there is a CS as his third sentence indicates - the server stats page indicates there is no CS.
At the time of the attempted submission, I believe that 52.224.109.74 was actually listed as having a CS, though I refreshed after the server was brought back into service and the submission finally went through. Unfortunately I don't have the part of the logs that would've shown the beginning of that job since it rolled off the end of the log buffer for the windows client. Though the submission that was having issues was from one of my headless linux servers to 52.224.109.74. And at no point prior to the down server being brought back online did the client make any attempt to try another server for the upload. This was despite repeated failures and lack of response from the FAH server in question, spanning more than 12 hours.
What part of my prior statement differs from this assessment?
Did the WU have a CS defined? If it did, it would have tried that.
A WS may have one or more CS showing up on the Server Status page, but none of those might apply for the project the WU is from. If, and only if, a CS was defined for the project at the time your WU was downloaded, then it would have, a) shown up on FAHControl, and b) been tried when the upload to the WS failed.
All your log shows is continuous attempts to the WS and none to a CS, your log does not prove otherwise.
iMac 2.8 i7 12 GB smp8, Mac Pro 2.8 quad 12 GB smp6
MacBook Pro 2.9 i7 8 GB smp3
tbonse wrote:...Unfortunately I don't have the part of the logs that would've shown the beginning of that job since it rolled off the end of the log buffer for the windows client. Though the submission that was having issues was from one of my headless linux servers to 52.224.109.74. And at no point prior to the down server being brought back online did the client make any attempt to try another server for the upload. This was despite repeated failures and lack of response from the FAH server in question, spanning more than 12 hours...
Please note that by default, the client maintains a history of the last 16 log files. A new log file is generated upon the restart of FAHClient.
Last edited by Joe_H on Thu Apr 16, 2020 6:12 am, edited 1 time in total.
Reason:fixed quote attribution
ETA:
Now ↞ Very Soon ↔ Soon ↔ Soon-ish ↔ Not Soon ↠ End Of Time
If the server appears to be on but your upload is going extremely slowly and timing out, I suggest trying to configure a proxy that is closer to the server.
Packets get dropped when certain hops are over capacity, and this is especially the case with international links.
15:26:41:WU00:FS01:Sending unit results: id:00 state:SEND error:FAULTY project:14416 run:0 clone:1659 gen:8 core:0x22 unit:0x0000000a0d5262775e84c8a778dce240
15:26:41:WU00:FS01:Uploading 213.16MiB to 13.82.98.119
15:26:41:WU00:FS01:Connecting to 13.82.98.119:8080
15:27:24:WU00:FS01:Upload 0.09%
15:27:25:WARNING:WU00:FS01:Exception: Failed to send results to work server: Transfer failed
15:27:25:WU00:FS01:Trying to send results to collection server
15:27:25:WU00:FS01:Uploading 213.16MiB to 52.224.109.74
15:27:25:WU00:FS01:Connecting to 52.224.109.74:8080
15:27:26:ERROR:WU00:FS01:Exception: Transfer failed
15:27:26:WU00:FS01:Sending unit results: id:00 state:SEND error:FAULTY project:14416 run:0 clone:1659 gen:8 core:0x22 unit:0x0000000a0d5262775e84c8a778dce240
15:27:26:WU00:FS01:Uploading 213.16MiB to 13.82.98.119
15:27:26:WU00:FS01:Connecting to 13.82.98.119:8080
15:27:47:WARNING:WU00:FS01:WorkServer connection failed on port 8080 trying 80
15:27:47:WU00:FS01:Connecting to 13.82.98.119:80
15:28:08:WARNING:WU00:FS01:Exception: Failed to send results to work server: Failed to connect to 13.82.98.119:80: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
15:28:08:WU00:FS01:Trying to send results to collection server
15:28:08:WU00:FS01:Uploading 213.16MiB to 52.224.109.74
15:28:08:WU00:FS01:Connecting to 52.224.109.74:8080
15:28:08:ERROR:WU00:FS01:Exception: Transfer failed
15:29:03:WU00:FS01:Sending unit results: id:00 state:SEND error:FAULTY project:14416 run:0 clone:1659 gen:8 core:0x22 unit:0x0000000a0d5262775e84c8a778dce240
15:29:03:WU00:FS01:Uploading 213.16MiB to 13.82.98.119
15:29:03:WU00:FS01:Connecting to 13.82.98.119:8080
15:29:24:WARNING:WU00:FS01:WorkServer connection failed on port 8080 trying 80
15:29:24:WU00:FS01:Connecting to 13.82.98.119:80
15:29:45:WARNING:WU00:FS01:Exception: Failed to send results to work server: Failed to connect to 13.82.98.119:80: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
15:29:45:WU00:FS01:Trying to send results to collection server
15:29:45:WU00:FS01:Uploading 213.16MiB to 52.224.109.74
15:29:45:WU00:FS01:Connecting to 52.224.109.74:8080
15:29:45:ERROR:WU00:FS01:Exception: Transfer failed
15:31:40:WU00:FS01:Sending unit results: id:00 state:SEND error:FAULTY project:14416 run:0 clone:1659 gen:8 core:0x22 unit:0x0000000a0d5262775e84c8a778dce240
15:31:40:WU00:FS01:Uploading 213.16MiB to 13.82.98.119
15:31:40:WU00:FS01:Connecting to 13.82.98.119:8080
15:32:15:WU00:FS01:Upload 0.09%
15:32:15:WARNING:WU00:FS01:Exception: Failed to send results to work server: Transfer failed
15:32:15:WU00:FS01:Trying to send results to collection server
15:32:15:WU00:FS01:Uploading 213.16MiB to 52.224.109.74
15:32:15:WU00:FS01:Connecting to 52.224.109.74:8080
15:32:15:ERROR:WU00:FS01:Exception: Transfer failed
15:35:55:WU00:FS01:Sending unit results: id:00 state:SEND error:FAULTY project:14416 run:0 clone:1659 gen:8 core:0x22 unit:0x0000000a0d5262775e84c8a778dce240
15:35:55:WU00:FS01:Uploading 213.16MiB to 13.82.98.119
15:35:55:WU00:FS01:Connecting to 13.82.98.119:8080
15:36:16:WARNING:WU00:FS01:WorkServer connection failed on port 8080 trying 80
15:36:16:WU00:FS01:Connecting to 13.82.98.119:80
15:36:17:WU00:FS01:Upload 0.03%
15:36:57:WU00:FS01:Upload 0.09%
15:36:57:WARNING:WU00:FS01:Exception: Failed to send results to work server: Transfer failed
15:36:57:WU00:FS01:Trying to send results to collection server
15:36:57:WU00:FS01:Uploading 213.16MiB to 52.224.109.74
15:36:57:WU00:FS01:Connecting to 52.224.109.74:8080
15:36:58:ERROR:WU00:FS01:Exception: Transfer failed
15:42:46:WU00:FS01:Sending unit results: id:00 state:SEND error:FAULTY project:14416 run:0 clone:1659 gen:8 core:0x22 unit:0x0000000a0d5262775e84c8a778dce240
15:42:46:WU00:FS01:Uploading 213.16MiB to 13.82.98.119
15:42:46:WU00:FS01:Connecting to 13.82.98.119:8080
15:42:53:WU00:FS01:Upload 0.03%
15:43:00:WU00:FS01:Upload 0.06%
15:43:34:WU00:FS01:Upload 0.09%
15:43:34:WARNING:WU00:FS01:Exception: Failed to send results to work server: Transfer failed
15:43:34:WU00:FS01:Trying to send results to collection server
15:43:34:WU00:FS01:Uploading 213.16MiB to 52.224.109.74
15:43:34:WU00:FS01:Connecting to 52.224.109.74:8080
15:43:35:ERROR:WU00:FS01:Exception: Transfer failed
@LB23, what username do you use while folding? Were you able to submit your results? If not, can you post your entire log, including the part where it downloaded the WU?
I use LB23 as that user name to. My results expired and I don't have those logs anymore I have had to restart a few times for updates on my rig. I do have another failing though.
21:53:19:WU04:FS01:Sending unit results: id:04 state:SEND error:FAULTY project:14417 run:0 clone:499 gen:4 core:0x22 unit:0x0000000a0d5262775e84c8a61fce69a5
21:53:19:WU04:FS01:Uploading 258.62MiB to 13.82.98.119
21:53:19:WU04:FS01:Connecting to 13.82.98.119:8080
21:53:29:WU04:FS01:Upload 0.07%
21:53:29:WARNING:WU04:FS01:Exception: Failed to send results to work server: Transfer failed
21:53:29:WU04:FS01:Trying to send results to collection server
21:53:29:WU04:FS01:Uploading 258.62MiB to 52.224.109.74
21:53:29:WU04:FS01:Connecting to 52.224.109.74:8080
21:53:31:ERROR:WU04:FS01:Exception: Transfer failed
21:53:31:WU04:FS01:Sending unit results: id:04 state:SEND error:FAULTY project:14417 run:0 clone:499 gen:4 core:0x22 unit:0x0000000a0d5262775e84c8a61fce69a5
21:53:31:WU04:FS01:Uploading 258.62MiB to 13.82.98.119
21:53:31:WU04:FS01:Connecting to 13.82.98.119:8080
21:53:40:WU04:FS01:Upload 0.07%
21:53:40:WARNING:WU04:FS01:Exception: Failed to send results to work server: Transfer failed
21:53:40:WU04:FS01:Trying to send results to collection server
21:53:40:WU04:FS01:Uploading 258.62MiB to 52.224.109.74
21:53:40:WU04:FS01:Connecting to 52.224.109.74:8080
21:53:40:ERROR:WU04:FS01:Exception: Transfer failed
21:55:08:WU04:FS01:Sending unit results: id:04 state:SEND error:FAULTY project:14417 run:0 clone:499 gen:4 core:0x22 unit:0x0000000a0d5262775e84c8a61fce69a5
21:55:08:WU04:FS01:Uploading 258.62MiB to 13.82.98.119
21:55:08:WU04:FS01:Connecting to 13.82.98.119:8080
21:55:22:WU04:FS01:Upload 0.07%
21:55:22:WARNING:WU04:FS01:Exception: Failed to send results to work server: Transfer failed
21:55:22:WU04:FS01:Trying to send results to collection server
21:55:22:WU04:FS01:Uploading 258.62MiB to 52.224.109.74
21:55:22:WU04:FS01:Connecting to 52.224.109.74:8080
21:55:22:ERROR:WU04:FS01:Exception: Transfer failed
*********************** Log Started 2020-04-21T21:53:18Z ***********************
21:53:19:WU04:FS01:Sending unit results: id:04 state:SEND error:FAULTY project:14417 run:0 clone:499 gen:4 core:0x22 unit:0x0000000a0d5262775e84c8a61fce69a5
21:53:19:WU04:FS01:Uploading 258.62MiB to 13.82.98.119
21:53:19:WU04:FS01:Connecting to 13.82.98.119:8080
21:53:29:WU04:FS01:Upload 0.07%
21:53:29:WARNING:WU04:FS01:Exception: Failed to send results to work server: Transfer failed
21:53:29:WU04:FS01:Trying to send results to collection server
21:53:29:WU04:FS01:Uploading 258.62MiB to 52.224.109.74
21:53:29:WU04:FS01:Connecting to 52.224.109.74:8080
21:53:31:ERROR:WU04:FS01:Exception: Transfer failed
21:53:31:WU04:FS01:Sending unit results: id:04 state:SEND error:FAULTY project:14417 run:0 clone:499 gen:4 core:0x22 unit:0x0000000a0d5262775e84c8a61fce69a5
21:53:31:WU04:FS01:Uploading 258.62MiB to 13.82.98.119
21:53:31:WU04:FS01:Connecting to 13.82.98.119:8080
21:53:40:WU04:FS01:Upload 0.07%
21:53:40:WARNING:WU04:FS01:Exception: Failed to send results to work server: Transfer failed
21:53:40:WU04:FS01:Trying to send results to collection server
21:53:40:WU04:FS01:Uploading 258.62MiB to 52.224.109.74
21:53:40:WU04:FS01:Connecting to 52.224.109.74:8080
21:53:40:ERROR:WU04:FS01:Exception: Transfer failed
21:55:08:WU04:FS01:Sending unit results: id:04 state:SEND error:FAULTY project:14417 run:0 clone:499 gen:4 core:0x22 unit:0x0000000a0d5262775e84c8a61fce69a5
21:55:08:WU04:FS01:Uploading 258.62MiB to 13.82.98.119
21:55:08:WU04:FS01:Connecting to 13.82.98.119:8080
21:55:22:WU04:FS01:Upload 0.07%
21:55:22:WARNING:WU04:FS01:Exception: Failed to send results to work server: Transfer failed
21:55:22:WU04:FS01:Trying to send results to collection server
21:55:22:WU04:FS01:Uploading 258.62MiB to 52.224.109.74
21:55:22:WU04:FS01:Connecting to 52.224.109.74:8080
21:55:22:ERROR:WU04:FS01:Exception: Transfer failed
21:57:45:WU04:FS01:Sending unit results: id:04 state:SEND error:FAULTY project:14417 run:0 clone:499 gen:4 core:0x22 unit:0x0000000a0d5262775e84c8a61fce69a5
21:57:45:WU04:FS01:Uploading 258.62MiB to 13.82.98.119
21:57:45:WU04:FS01:Connecting to 13.82.98.119:8080
21:57:52:WU04:FS01:Upload 0.07%
21:57:52:WARNING:WU04:FS01:Exception: Failed to send results to work server: Transfer failed
21:57:52:WU04:FS01:Trying to send results to collection server
21:57:52:WU04:FS01:Uploading 258.62MiB to 52.224.109.74
21:57:52:WU04:FS01:Connecting to 52.224.109.74:8080
21:57:52:ERROR:WU04:FS01:Exception: Transfer failed
Since the history is muddled, I'm not going to try to address the CS issue. At the present time, the serverstat app shows that 52.224.109.74 is in Accept mode rather than Assign mode, which means that a server is beng drained and some kind of reboot or specific changes are planned once the WUs that are "in flight" arrive or expire. I can only surmise what might be planned. No new assignments are being made and I don't know how many WUs/hr are currently being uploaded.
Further discussion is meaningless without new information. If your uploads are successful, good. If not, I'm sorry. There's nothing else that can be done at this point. I closing this thread.