Folding Forum

Posted: **Tue Nov 26, 2013 5:44 pm**

woke up today to wu's not uploading.

*********************** Log Started 2013-11-26T17:34:44Z ***********************
17:34:44:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:7810 run:0 clone:1644 gen:1 core:0x17 unit:0x000000020a3b1e8651d3585fd46c7d05
17:34:44:WU00:FS00:Uploading 5.76MiB to 171.64.65.98
17:34:44:WU00:FS00:Connecting to 171.64.65.98:8080
17:34:44:WARNING:WU00:FS00:WorkServer connection failed on port 8080 trying 80
17:34:44:WU00:FS00:Connecting to 171.64.65.98:80
17:34:45:WARNING:WU00:FS00:Exception: Failed to send results to work server: Failed to connect to 171.64.65.98:80: A socket operation was attempted to an unreachable network.
17:34:45:WU00:FS00:Trying to send results to collection server
17:34:45:WU00:FS00:Uploading 5.76MiB to 171.65.103.160
17:34:45:WU00:FS00:Connecting to 171.65.103.160:8080
17:34:45:WARNING:WU00:FS00:WorkServer connection failed on port 8080 trying 80
17:34:45:WU00:FS00:Connecting to 171.65.103.160:80
17:34:45:ERROR:WU00:FS00:Exception: Failed to connect to 171.65.103.160:80: A socket operation was attempted to an unreachable network.
17:34:46:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:7810 run:0 clone:1644 gen:1 core:0x17 unit:0x000000020a3b1e8651d3585fd46c7d05
17:34:46:WU00:FS00:Uploading 5.76MiB to 171.64.65.98
17:34:46:WU00:FS00:Connecting to 171.64.65.98:8080
17:35:07:WARNING:WU00:FS00:WorkServer connection failed on port 8080 trying 80
17:35:07:WU00:FS00:Connecting to 171.64.65.98:80
17:35:28:WARNING:WU00:FS00:Exception: Failed to send results to work server: Failed to connect to 171.64.65.98:80: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
17:35:28:WU00:FS00:Trying to send results to collection server
17:35:28:WU00:FS00:Uploading 5.76MiB to 171.65.103.160
17:35:28:WU00:FS00:Connecting to 171.65.103.160:8080
17:35:34:WU00:FS00:Upload 6.51%
17:35:40:WU00:FS00:Upload 11.93%
17:35:47:WU00:FS00:Upload 17.35%
17:35:53:WU00:FS00:Upload 24.94%
17:35:59:WU00:FS00:Upload 29.28%
17:36:05:WU00:FS00:Upload 36.87%
17:36:11:WU00:FS00:Upload 45.54%
17:36:17:WU00:FS00:Upload 53.14%
17:36:23:WU00:FS00:Upload 62.90%
17:36:29:WU00:FS00:Upload 71.57%
17:36:35:WU00:FS00:Upload 79.16%
17:36:41:WU00:FS00:Upload 87.84%
17:36:47:WU00:FS00:Upload 97.60%
17:36:49:WU00:FS00:Upload complete
17:36:49:WU00:FS00:Server responded PLEASE_WAIT (464)
17:36:49:WARNING:WU00:FS00:Failed to send results, will try again later
17:36:49:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:7810 run:0 clone:1644 gen:1 core:0x17 unit:0x000000020a3b1e8651d3585fd46c7d05
17:36:49:WU00:FS00:Uploading 5.76MiB to 171.64.65.98
17:36:49:WU00:FS00:Connecting to 171.64.65.98:8080
17:37:10:WARNING:WU00:FS00:WorkServer connection failed on port 8080 trying 80
17:37:10:WU00:FS00:Connecting to 171.64.65.98:80
17:37:31:WARNING:WU00:FS00:Exception: Failed to send results to work server: Failed to connect to 171.64.65.98:80: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
17:37:31:WU00:FS00:Trying to send results to collection server
17:37:31:WU00:FS00:Uploading 5.76MiB to 171.65.103.160
17:37:31:WU00:FS00:Connecting to 171.65.103.160:8080
17:37:37:WU00:FS00:Upload 7.59%
17:37:43:WU00:FS00:Upload 15.18%
17:37:49:WU00:FS00:Upload 24.94%
17:37:55:WU00:FS00:Upload 32.53%
17:38:01:WU00:FS00:Upload 41.21%
17:38:07:WU00:FS00:Upload 50.97%
17:38:13:WU00:FS00:Upload 58.56%
17:38:19:WU00:FS00:Upload 67.23%
17:38:25:WU00:FS00:Upload 75.91%
17:38:31:WU00:FS00:Upload 84.58%
17:38:37:WU00:FS00:Upload 94.34%
17:38:41:WU00:FS00:Upload complete
17:38:41:WU00:FS00:Server responded PLEASE_WAIT (464)
17:38:41:WARNING:WU00:FS00:Failed to send results, will try again later
17:38:42:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:7810 run:0 clone:1644 gen:1 core:0x17 unit:0x000000020a3b1e8651d3585fd46c7d05
17:38:42:WU00:FS00:Uploading 5.76MiB to 171.64.65.98
17:38:42:WU00:FS00:Connecting to 171.64.65.98:8080
17:39:03:WARNING:WU00:FS00:WorkServer connection failed on port 8080 trying 80
17:39:03:WU00:FS00:Connecting to 171.64.65.98:80
17:39:24:WARNING:WU00:FS00:Exception: Failed to send results to work server: Failed to connect to 171.64.65.98:80: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
17:39:24:WU00:FS00:Trying to send results to collection server
17:39:24:WU00:FS00:Uploading 5.76MiB to 171.65.103.160
17:39:24:WU00:FS00:Connecting to 171.65.103.160:8080
17:39:30:WU00:FS00:Upload 8.68%
17:39:36:WU00:FS00:Upload 18.43%
17:39:42:WU00:FS00:Upload 24.94%
17:39:49:WU00:FS00:Upload 29.28%
17:39:56:WU00:FS00:Upload 33.62%
17:40:03:WU00:FS00:Upload 37.95%
17:40:10:WU00:FS00:Upload 42.29%
17:40:17:WU00:FS00:Upload 46.63%
17:40:23:WU00:FS00:Upload 49.88%
17:40:31:WU00:FS00:Upload 54.22%
17:40:37:WU00:FS00:Upload 57.47%
17:40:43:WU00:FS00:Upload 60.73%
17:40:49:WU00:FS00:Upload 63.98%
17:40:55:WU00:FS00:Upload 67.23%
17:41:02:WU00:FS00:Upload 70.49%
17:41:09:WU00:FS00:Upload 74.82%
17:41:15:WU00:FS00:Upload 78.08%
17:41:21:WU00:FS00:Upload 81.33%
17:41:27:WU00:FS00:Upload 87.84%
17:41:33:WU00:FS00:Upload 94.34%
17:41:39:WU00:FS00:Upload 100.00%
17:41:39:WU00:FS00:Upload complete
17:41:39:WU00:FS00:Server responded PLEASE_WAIT (464)
17:41:39:WARNING:WU00:FS00:Failed to send results, will try again later
17:41:39:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:7810 run:0 clone:1644 gen:1 core:0x17 unit:0x000000020a3b1e8651d3585fd46c7d05
17:41:39:WU00:FS00:Uploading 5.76MiB to 171.64.65.98
17:41:39:WU00:FS00:Connecting to 171.64.65.98:8080
17:42:00:WARNING:WU00:FS00:WorkServer connection failed on port 8080 trying 80
17:42:00:WU00:FS00:Connecting to 171.64.65.98:80
17:42:22:WARNING:WU00:FS00:Exception: Failed to send results to work server: Failed to connect to 171.64.65.98:80: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
17:42:22:WU00:FS00:Trying to send results to collection server
17:42:22:WU00:FS00:Uploading 5.76MiB to 171.65.103.160
17:42:22:WU00:FS00:Connecting to 171.65.103.160:8080
17:42:28:WU00:FS00:Upload 5.42%
17:42:34:WU00:FS00:Upload 11.93%
17:42:40:WU00:FS00:Upload 19.52%
17:42:46:WU00:FS00:Upload 23.86%

and

Code: Select all

*********************** Log Started 2013-11-26T17:34:44Z ***********************
17:34:44:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:7808 run:4 clone:165 gen:281 core:0xa4 unit:0x000001cb0a3b1e874e30f855ea1d6e3c
17:34:44:WU01:FS01:Uploading 4.05MiB to 171.64.65.99
17:34:44:WU01:FS01:Connecting to 171.64.65.99:8080
17:34:44:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
17:34:44:WU01:FS01:Connecting to 171.64.65.99:80
17:34:45:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 171.64.65.99:80: A socket operation was attempted to an unreachable network.
17:34:45:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:7808 run:4 clone:165 gen:281 core:0xa4 unit:0x000001cb0a3b1e874e30f855ea1d6e3c
17:34:45:WU01:FS01:Uploading 4.05MiB to 171.64.65.99
17:34:45:WU01:FS01:Connecting to 171.64.65.99:8080
17:34:45:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
17:34:45:WU01:FS01:Connecting to 171.64.65.99:80
17:34:46:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 171.64.65.99:80: A socket operation was attempted to an unreachable network.
17:35:46:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:7808 run:4 clone:165 gen:281 core:0xa4 unit:0x000001cb0a3b1e874e30f855ea1d6e3c
17:35:46:WU01:FS01:Uploading 4.05MiB to 171.64.65.99
17:35:46:WU01:FS01:Connecting to 171.64.65.99:8080
17:36:07:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
17:36:07:WU01:FS01:Connecting to 171.64.65.99:80
17:36:28:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 171.64.65.99:80: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
17:37:23:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:7808 run:4 clone:165 gen:281 core:0xa4 unit:0x000001cb0a3b1e874e30f855ea1d6e3c
17:37:23:WU01:FS01:Uploading 4.05MiB to 171.64.65.99
17:37:23:WU01:FS01:Connecting to 171.64.65.99:8080
17:37:44:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
17:37:44:WU01:FS01:Connecting to 171.64.65.99:80
17:38:05:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 171.64.65.99:80: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
17:40:00:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:7808 run:4 clone:165 gen:281 core:0xa4 unit:0x000001cb0a3b1e874e30f855ea1d6e3c
17:40:00:WU01:FS01:Uploading 4.05MiB to 171.64.65.99
17:40:00:WU01:FS01:Connecting to 171.64.65.99:8080
17:40:21:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
17:40:21:WU01:FS01:Connecting to 171.64.65.99:80
17:40:42:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 171.64.65.99:80: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.

as a note: both of these show as down on the serverstat page

Posted: **Tue Nov 26, 2013 6:47 pm**

Thanks for your report.

The servers were down for a while but appear to be fully operational now.

Posted: **Tue Nov 26, 2013 7:56 pm**

When servers go down, and then back up it can sometimes take a while to clear up the backlog and operate normally again.

Posted: **Thu Nov 28, 2013 3:47 pm**

I'm still getting 171.64.65.99:80 as down, and it's on a day I doubt it will be fixed very soon.

Posted: **Thu Nov 28, 2013 3:55 pm**

The server stats page is showing quite a few of the 171.64.65.xxx servers as down, and my log suggests they've been down for over 14 hours

Don't server managers get notifications from the monitoring programs?

Posted: **Thu Nov 28, 2013 5:37 pm**

Wondering how to delete or send this completed work unit.
Work Unit for project 7809 has attempted to send results 13 times.
Something is wrong...

Work Server: 171.64.65.99
Collection Server: 0.0.0.0

15:06:00:WU01:FS00:Sending unit results: id:01 state:SEND error:NO_ERROR project:7809 run:8 clone:386 gen:167 core:0xa4 unit:0x000000e60a3b1e874e311792afedfbd7
15:06:00:WU01:FS00:Uploading 4.13MiB to 171.64.65.99
15:06:00:WU01:FS00:Connecting to 171.64.65.99:8080
15:07:16:WARNING:WU01:FS00:WorkServer connection failed on port 8080 trying 80
15:07:16:WU01:FS00:Connecting to 171.64.65.99:80
15:07:20:WU00:FS00:0xa4:Completed 207500 out of 250000 steps (83%)
15:08:33:WARNING:WU01:FS00:Exception: Failed to send results to work server: Failed to connect to 171.64.65.99:80: Operation timed out

Posted: **Thu Nov 28, 2013 6:40 pm**

kiwiplant wrote:Wondering how to delete or send this completed work unit.
Work Unit for project 7809 has attempted to send results 13 times.
Something is wrong...

Work Server: 171.64.65.99
Collection Server: 0.0.0.0

Just leave the completed WU on your system. Once the server is back up the WU will be accepted. As you can see from the other reports, this server is currently down. I do not have any current information on when the server will be back up, but since it is a holiday there probably will be a delay in response. Unfortunately this project does not have a designated backup Collection Server, so it needs to go back to the WS.

Posted: **Thu Nov 28, 2013 6:40 pm**

kiwiplant wrote:Wondering how to delete or send this completed work unit.

Just ignore it- the client will keep trying at intervals until the server comes back on line and the completed WU can be returned.

I've got 2 WUs waiting for the damn thing to wake up again

Posted: **Thu Nov 28, 2013 6:49 pm**

billford wrote:The server stats page is showing quite a few of the 171.64.65.xxx servers as down, and my log suggests they've been down for over 14 hours

Don't server managers get notifications from the monitoring programs?

There are only 5 servers in the 171.64.65.xxx range listed as down, and three of those look to be VM's on a single server. Most of the rest in that range that have active projects are up and accepting connections. As for notifications, I would assume they do. But if the issue is hardware related or something else the manager can not correct, on a major holiday there will be delays in response.

Posted: **Thu Nov 28, 2013 6:57 pm**

Joe_H wrote:… on a major holiday there will be delays in response.

And when else do these things usually happen?

Fair enough, thanks for the reply. My home LAN has been sending me towards a nervous breakdown today, the last thing I needed was to find WUs that couldn't be sent

Posted: **Thu Nov 28, 2013 7:41 pm**

I have sent a message to the server manager.

Posted: **Thu Nov 28, 2013 7:55 pm**

Unfortunately these servers went down at the end of the day yesterday, and due to the holiday we can't get them back up. These may be down for the duration of the weekend, but will be promptly started on Monday at the latest.

Thanks for your patience,
Christian

Posted: **Thu Nov 28, 2013 8:00 pm**

schwancr wrote:These may be down for the duration of the weekend

Ouch… ah well, these things happen.

Thanks for the info.

Posted: **Thu Nov 28, 2013 10:38 pm**

I don't who did it or what they did, but it's now accepting again

Code: Select all

22:31:48:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:7808 run:5 clone:202 gen:38 core:0xa4 unit:0x000000410a3b1e874e30fa9fae0e81f6
22:31:48:WU00:FS00:Uploading 4.05MiB to 171.64.65.99
22:31:48:WU00:FS00:Connecting to 171.64.65.99:8080
22:31:54:WU00:FS00:Upload 72.55%
22:31:56:WU00:FS00:Upload complete
22:31:56:WU00:FS00:Server responded WORK_ACK (400)
22:31:56:WU00:FS00:Final credit estimate, 5485.00 points

One down, one to go (in about an hour and a half), many thanks to somebody!

Posted: **Fri Nov 29, 2013 4:47 am**

Unfortunately (edit: after running ~8 hours) Servers VSPG13 are Down again:

171.64.65.98 - vspg13a
171.64.65.99 - vspg13b
171.64.65.100 - vspg13c

Folding Forum

171.64.65.98/99

171.64.65.98/99

Re: 171.64.65.98/99

Re: 171.64.65.98/99

Re: 171.64.65.98/99

Re: 171.64.65.98/99

WU not sending

Re: WU not sending

Re: WU not sending

Re: 171.64.65.98/99

Re: 171.64.65.98/99

Re: 171.64.65.98/99

Re: 171.64.65.98/99

Re: 171.64.65.98/99

Re: 171.64.65.98/99

Re: 171.64.65.98/99