171.64.65.106 503 Error FYI

[Plazzman]

Client can't connect to retrieve new work unit.
11:10:12] - Preparing to get new work unit...
[11:10:12] + Attempting to get work packet
[11:10:12] - Connecting to assignment server
[11:10:12] - Successful: assigned to (171.64.65.106).
[11:10:12] + News From Folding@Home: GPU folding beta
[11:10:12] Loaded queue successfully.
[11:10:13] - Couldn't send HTTP request to server
[11:10:13] (Got status 503)
[11:10:13] + Could not connect to Work Server
[11:10:13] - Attempt #1 to get work failed, and no other work to do.
Waiting before retry.
[11:10:20] + Attempting to get work packet
[11:10:20] - Connecting to assignment server
[11:10:20] - Successful: assigned to (171.64.65.106).
[11:10:20] + News From Folding@Home: GPU folding beta
[11:10:20] Loaded queue successfully.
[11:10:21] - Couldn't send HTTP request to server
[11:10:21] (Got status 503)
[11:10:21] + Could not connect to Work Server
[11:10:21] - Attempt #2 to get work failed, and no other work to do.
Waiting before retry.
[11:10:42] + Attempting to get work packet
[11:10:42] - Connecting to assignment server
[11:10:42] - Successful: assigned to (171.64.65.106).
[11:10:42] + News From Folding@Home: GPU folding beta
[11:10:43] Loaded queue successfully.
[11:10:43] - Couldn't send HTTP request to server
[11:10:43] (Got status 503)
[11:10:43] + Could not connect to Work Server
[11:10:43] - Attempt #3 to get work failed, and no other work to do.
Waiting before retry.
[11:11:12] + Attempting to get work packet
[11:11:12] - Connecting to assignment server
[11:11:13] - Successful: assigned to (171.64.65.106).
[11:11:13] + News From Folding@Home: GPU folding beta
[11:11:13] Loaded queue successfully.
[11:11:13] - Couldn't send HTTP request to server
[11:11:13] (Got status 503)
[11:11:13] + Could not connect to Work Server
[11:11:13] - Attempt #4 to get work failed, and no other work to do.
Waiting before retry.
[11:11:58] + Attempting to get work packet
[11:11:58] - Connecting to assignment server
[11:11:59] - Successful: assigned to (171.64.65.106).
[11:11:59] + News From Folding@Home: GPU folding beta
[11:11:59] Loaded queue successfully.
[11:11:59] - Couldn't send HTTP request to server
[11:11:59] (Got status 503)
[11:11:59] + Could not connect to Work Server
[11:11:59] - Attempt #5 to get work failed, and no other work to do.
Waiting before retry.
[11:13:32] + Attempting to get work packet
[11:13:32] - Connecting to assignment server
[11:13:32] - Successful: assigned to (171.64.65.106).
[11:13:32] + News From Folding@Home: GPU folding beta
[11:13:32] Loaded queue successfully.
[11:13:33] - Couldn't send HTTP request to server
[11:13:33] (Got status 503)
[11:13:33] + Could not connect to Work Server
[11:13:33] - Attempt #6 to get work failed, and no other work to do.
Waiting before retry.

Thanks, Plazzman

[caintry_boy]

I'm having the same problem with the same server.......

[MichaelB]

same here it needs a nudge

[dempaSD]

Same here! >10 gpu clients stalled and waiting...

[bollix47]

Yes, same problem here. Tried adding -advmethods but that didn't help. Not sure why it's not switching to a different server.

Server says it's accepting but both cpu and net loads look kinda busy.

Three clients waiting for work.

[kiore]

Same problem, 9 attempts and 9 "Got status 503"s,checked it on the server status and seemed to be good to go (to my untrained eyes). Then was switched to 171.64.122.70 and got a 5904 so switching is working just taking some time.
kiore.

[MtM]

Yup same here

Can't connect, not being reassigned to diffrent server. Changed advmethods as well just as bollix but to the same avail

Strange thing is also I noticed the gpu clients don't write failed uploads to queue.dat, counter is zero there for some reason the work server is down to ( http://171.67.108.25:8080/ or 80 ).

[vvoelz]

Thanks for the reports -- we're working on it. --Vince

[toTOW]

I've been able to return two WUs to it, but it's impossible to get new work (503 error too).

[MtM]

Everything fixed now ( one client just uploaded, other is bussy uploading, both have new work )

Thanks for the quick responce Vince!

[vvoelz]

An update on the situation:

Everything should be working smoothly for now. The problem was that a background analysis job was taking up some swap memory, compounded by the fact that 171.64.65.106 was getting hit pretty hard.

Unfortunately, the background analysis job is critical, so we need to restart this, albeit in a less intrusive form. In the meantime, I have changed the assignment weights so that the other NVIDIA work servers will be picking up the slack. If you notice this problem again, please let me know (post in this forum). We are keeping our eyes peeled on our side to avoid such problems.

Thanks,
Vince

[bruce]

Thanks, Vince. We do appreciate your diligence.

[kiore]

Status 503's on this one again, then diverted to 171.67.108.25 and 503ed on that one too, in the queue.
Not complaining just thought someone should know.
kiore.

[vvoelz]

kiore -- Are you still seeing this problem? The load was high on this machine last night, but should be fine now. (The background analysis job on this machine is done now).

Vince

[kiore]

I did see this again with 2 servers 503ing me on return W.Us in a row, put in queue and returned with the next W.U.
Yesterday it was not being able to get work, this was a return problem, have done a few more since with no problems.
kiore.