Page 1 of 2

Unable to upload to server 69.94.66.6

Posted: Wed Dec 16, 2020 1:49 am
by comixgoddess
Here is a section of my log --

01:30:42:WU01:FS00:Sending unit results: id:01 state:SEND error:NO_ERROR project:14809 run:1070 clone:2 gen:81 core:0xa7 unit:0x0000005c455e420600000000042e0002
01:30:42:WU01:FS00:Uploading 6.79MiB to 69.94.66.6
01:30:42:WU01:FS00:Connecting to 69.94.66.6:8080
01:30:48:WU01:FS00:Upload 21.16%
01:30:54:WU01:FS00:Upload 45.99%
01:31:00:WU01:FS00:Upload 71.75%
01:31:06:WU01:FS00:Upload 96.59%
01:31:33:WU01:FS00:Upload complete
01:31:33:WU01:FS00:Server responded PLEASE_WAIT (464)
01:31:33:WARNING:WU01:FS00:Failed to send results, will try again later
01:33:19:WU01:FS00:Sending unit results: id:01 state:SEND error:NO_ERROR project:14809 run:1070 clone:2 gen:81 core:0xa7 unit:0x0000005c455e420600000000042e0002
01:33:19:WU01:FS00:Uploading 6.79MiB to 69.94.66.6
01:33:19:WU01:FS00:Connecting to 69.94.66.6:8080
01:33:52:WARNING:WU01:FS00:WorkServer connection failed on port 8080 trying 80
01:33:52:WU01:FS00:Connecting to 69.94.66.6:80
01:34:25:WARNING:WU01:FS00:Exception: Failed to send results to work server: Failed to connect to 69.94.66.6:80: Operation timed out
01:37:33:WU01:FS00:Sending unit results: id:01 state:SEND error:NO_ERROR project:14809 run:1070 clone:2 gen:81 core:0xa7 unit:0x0000005c455e420600000000042e0002
01:37:33:WU01:FS00:Uploading 6.79MiB to 69.94.66.6
01:37:33:WU01:FS00:Connecting to 69.94.66.6:8080
01:38:06:WARNING:WU01:FS00:WorkServer connection failed on port 8080 trying 80
01:38:06:WU01:FS00:Connecting to 69.94.66.6:80
01:38:23:WU01:FS00:Upload 0.92%
01:39:39:WU01:FS00:Upload 3.68%
01:39:45:WU01:FS00:Upload 23.00%
01:39:51:WU01:FS00:Upload 47.83%
01:39:57:WU01:FS00:Upload 72.67%
01:40:03:WU01:FS00:Upload 99.35%
01:40:33:WU01:FS00:Upload complete
01:40:34:WU01:FS00:Server responded PLEASE_WAIT (464)
01:40:34:WARNING:WU01:FS00:Failed to send results, will try again later

It looks like server 69.94.66.6 is out of disk space and has no collection server.

Re: Unable to upload to server 69.94.66.6

Posted: Wed Dec 16, 2020 2:22 am
by Welderserg
Having the same issue here.

Re: Unable to upload to server 69.94.66.6

Posted: Wed Dec 16, 2020 3:21 am
by psaam0001
Yup, I'm virtually waiting in line behind you comixgoddess....

Paul

Re: Unable to upload to server 69.94.66.6

Posted: Wed Dec 16, 2020 5:38 am
by Joe_H
From the server status page it is out of disk space. The appropriate persons had been informed when I checked.

Re: Unable to upload to server 69.94.66.6

Posted: Wed Dec 16, 2020 10:44 am
by zzsept
Hi,
Any news about this disk space issue?
I have now a dozen of servers stuck because they can't send results to Work Server 69.94.66.6
Is it better to delete pending results and restart all my servers or to wait?

Regards.

Re: Unable to upload to server 69.94.66.6

Posted: Wed Dec 16, 2020 12:50 pm
by MoelTryfan
Same here.

Re: Unable to upload to server 69.94.66.6

Posted: Wed Dec 16, 2020 3:15 pm
by kwthom
...another one here...

Re: Unable to upload to server 69.94.66.6

Posted: Wed Dec 16, 2020 3:17 pm
by Joe_H
zzsept wrote:Hi,
Any news about this disk space issue?
I have now a dozen of servers stuck because they can't send results to Work Server 69.94.66.6
Is it better to delete pending results and restart all my servers or to wait?

Regards.
Just leave the WUs alone in the queue waiting to upload, once there is space on the server the client will upload them. The upload retry should back off to trying once an hour.

WUs waiting to upload will still allow new WUs to be downloaded. There may be a delay for a new CPU WU, currently there are more requests than the current supply that is available for assignment. As WUs are returned the next generation is being generated and becomes available for assignment.

Re: Unable to upload to server 69.94.66.6

Posted: Wed Dec 16, 2020 7:59 pm
by Anyone
I have the same problem

Re: Unable to upload to server 69.94.66.6

Posted: Wed Dec 16, 2020 8:03 pm
by GraceCourt
Joe_H wrote: Just leave the WUs alone in the queue waiting to upload, once there is space on the server the client will upload them. The upload retry should back off to trying once an hour.
OK, will do, but the results of at least one "stuck" completed WU here will be discarded as "expired" if the situation isn't resolved soon (so will earn 0 points) :( ... any ETA for clearing some disk space?

Thanks.

Re: Unable to upload to server 69.94.66.6

Posted: Wed Dec 16, 2020 8:48 pm
by psaam0001
:::: Looking in my credit union account to see if I can spare a few dollars, for getting F@H an extra set of 18 Terrabyte HD's. ::::

Credit balance check results: Sorry, you will have to wait a few months to buy more computer parts. And by the way, your local electric co-op would like a payment by Jan. 4th, 2021. :|

Paul

Re: Unable to upload to server 69.94.66.6

Posted: Thu Dec 17, 2020 1:15 am
by sejtam
I have the same issue. It's not so much about the points, but that the work could be wasted if it will be
ignored once the expiration date comes around w/o having been up upload

Doesn't F@H have a way to redirect results to another collection server?

Re: Unable to upload to server 69.94.66.6

Posted: Thu Dec 17, 2020 1:18 am
by sejtam
Add to that the fact that somehow the uploads all show Complete and *then* get rejected

00:10:19:WU01:FS01:Uploading 6.67MiB to 69.94.66.6
00:10:19:WU01:FS01:Connecting to 69.94.66.6:8080
00:10:25:WU01:FS01:Upload 26.24%
00:10:31:WU01:FS01:Upload 56.23%
00:10:37:WU01:FS01:Upload 87.15%
00:10:42:WU00:FS01:0xa7:Completed 58750 out of 125000 steps (47%)
00:10:53:WU01:FS01:Upload complete
00:10:53:WU01:FS01:Server responded PLEASE_WAIT (464)
00:10:53:WARNING:WU01:FS01:Failed to send results, will try again later

Should they not reject before sending? Over time sending each request many times just to be rejected from the buffer area is wasteful?

Re: Unable to upload to server 69.94.66.6

Posted: Thu Dec 17, 2020 2:48 pm
by comixgoddess
My work unit just uploaded successfully :D

14:16:41:WU01:FS00:Sending unit results: id:01 state:SEND error:NO_ERROR project:14809 run:1070 clone:2 gen:81 core:0xa7 unit:0x0000005c455e420600000000042e0002
14:16:41:WU01:FS00:Uploading 6.79MiB to 69.94.66.6
14:16:41:WU01:FS00:Connecting to 69.94.66.6:8080
14:17:56:WARNING:WU01:FS00:WorkServer connection failed on port 8080 trying 80
14:17:56:WU01:FS00:Connecting to 69.94.66.6:80
14:19:04:WU01:FS00:Upload 0.92%
14:19:36:WU01:FS00:Upload 3.68%
14:19:42:WU01:FS00:Upload 28.52%
14:19:48:WU01:FS00:Upload 55.19%
14:19:54:WU01:FS00:Upload 81.87%
14:20:24:WU01:FS00:Upload complete
14:20:24:WU01:FS00:Server responded WORK_ACK (400)
14:20:24:WU01:FS00:Final credit estimate, 1097.00 points
14:20:24:WU01:FS00:Cleaning up

Hopefully everyone's backlog should be clearing up.

Re: Unable to upload to server 69.94.66.6

Posted: Fri Dec 18, 2020 1:33 am
by GraceCourt
Too late... a good number of my work units expired before this was fixed, so they were automatically deleted and all of the results were lost. Hopefully this extended outage will be reviewed in order to maximise the learning outcomes and to avoid future losses... :-(