Failing to get WU today work server refuses [171.67.108.157]

[silverpulser]

I am getting an error as follows:

17:24:50:WU00:FS01:Connecting to 171.67.108.157:80
17:24:51:ERROR:WU00:FS01:Exception: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it

[FldngForGrandparents]

Same issue here.

[silverpulser]

Just checked and a large WU (over 50000 points) has started so was only a temporary problem for me.

[darksab0r]

Same issue here:

Code: Select all

17:55:13:WU01:FS00:Uploading 7.81MiB to 171.67.108.157
17:55:13:WU01:FS00:Connecting to 171.67.108.157:8080
17:55:13:WARNING:WU01:FS00:WorkServer connection failed on port 8080 trying 80
17:55:13:WU01:FS00:Connecting to 171.67.108.157:80
17:55:13:WARNING:WU01:FS00:Exception: Failed to send results to work server: Failed to connect to 171.67.108.157:80: Connection refused
17:55:15:WU03:FS00:Connecting to 171.67.108.45:80
17:55:16:WARNING:WU03:FS00:Failed to get assignment from '171.67.108.45:80': No WUs available for this configuration
17:55:16:WU03:FS00:Connecting to 171.64.65.35:80
17:55:17:WARNING:WU03:FS00:Failed to get assignment from '171.64.65.35:80': No WUs available for this configuration
17:55:17:ERROR:WU03:FS00:Exception: Could not get an assignment

[Joe_H]

171.67.108.157 is shown to have gone into Standby mode on the Server Status page. Someone else has already notified PG about the outage. 171.67.108.157 mostly supplies GPU WU's, there are a few other sources but a relatively limited number of WU's available on them compared to this WS

[Aurum]

My problem is I cannot UL my completed WUs:

Code: Select all

17:10:45:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:9415 run:1727 clone:4 gen:26 core:0x21 unit:0x0000001cab436c9d585e06d8c4ebdc8c
17:10:45:WU01:FS01:Uploading 7.79MiB to 171.67.108.157
17:10:45:WU01:FS01:Connecting to 171.67.108.157:8080
17:10:46:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
17:10:46:WU01:FS01:Connecting to 171.67.108.157:80
17:10:48:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.
17:14:59:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:9415 run:1727 clone:4 gen:26 core:0x21 unit:0x0000001cab436c9d585e06d8c4ebdc8c
17:14:59:WU01:FS01:Uploading 7.79MiB to 171.67.108.157
17:14:59:WU01:FS01:Connecting to 171.67.108.157:8080
17:15:01:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
17:15:01:WU01:FS01:Connecting to 171.67.108.157:80
17:15:02:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.
17:21:51:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:9415 run:1727 clone:4 gen:26 core:0x21 unit:0x0000001cab436c9d585e06d8c4ebdc8c
17:21:51:WU01:FS01:Uploading 7.79MiB to 171.67.108.157
17:21:51:WU01:FS01:Connecting to 171.67.108.157:8080
17:21:52:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
17:21:52:WU01:FS01:Connecting to 171.67.108.157:80
17:21:53:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.
17:32:56:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:9415 run:1727 clone:4 gen:26 core:0x21 unit:0x0000001cab436c9d585e06d8c4ebdc8c
17:32:56:WU01:FS01:Uploading 7.79MiB to 171.67.108.157
17:32:56:WU01:FS01:Connecting to 171.67.108.157:8080
17:32:57:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
17:32:57:WU01:FS01:Connecting to 171.67.108.157:80
17:32:59:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.
17:50:53:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:9415 run:1727 clone:4 gen:26 core:0x21 unit:0x0000001cab436c9d585e06d8c4ebdc8c
17:50:53:WU01:FS01:Uploading 7.79MiB to 171.67.108.157
17:50:53:WU01:FS01:Connecting to 171.67.108.157:8080
17:50:54:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
17:50:54:WU01:FS01:Connecting to 171.67.108.157:80
17:50:55:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.
******************************* Date: 2018-04-24 *******************************
18:19:55:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:9415 run:1727 clone:4 gen:26 core:0x21 unit:0x0000001cab436c9d585e06d8c4ebdc8c
18:19:55:WU01:FS01:Uploading 7.79MiB to 171.67.108.157
18:19:55:WU01:FS01:Connecting to 171.67.108.157:8080
18:19:56:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
18:19:56:WU01:FS01:Connecting to 171.67.108.157:80
18:19:58:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.
19:06:54:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:9415 run:1727 clone:4 gen:26 core:0x21 unit:0x0000001cab436c9d585e06d8c4ebdc8c
19:06:54:WU01:FS01:Uploading 7.79MiB to 171.67.108.157
19:06:54:WU01:FS01:Connecting to 171.67.108.157:8080
19:06:55:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
19:06:55:WU01:FS01:Connecting to 171.67.108.157:80
19:06:57:WARNING:WU01:FS01:Exception: Failed to send results to work server: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.

[Aurum]

I have 29 completed WUs waiting to be ULed. Will the dead server delay wipe out my quick return bonus?

[Kuno]

I have 29 completed WUs waiting to be ULed. Will the dead server delay wipe out my quick return bonus?

Yes it will, sadly.

[stealthswifttlk]

Just checked and a large WU (over 50000 points) has started so was only a temporary problem for me.

Not if it isn't accepting uploads for completed WUs

Code: Select all

20:41:05:WU01:FS00:FahCore returned: FINISHED_UNIT (100 = 0x64)
20:41:05:WU01:FS00:Sending unit results: id:01 state:SEND error:NO_ERROR project:9415 run:2458 clone:4 gen:27 core:0x21 unit:0x0000001bab436c9d585e06e0cb56ff14
20:41:05:WU01:FS00:Uploading 7.77MiB to 171.67.108.157
20:41:05:WU01:FS00:Connecting to 171.67.108.157:8080
20:41:07:WARNING:WU01:FS00:WorkServer connection failed on port 8080 trying 80
20:41:07:WU01:FS00:Connecting to 171.67.108.157:80
20:41:08:WARNING:WU01:FS00:Exception: Failed to send results to work server: Failed to connect to 171.67.108.157:80: No connection could be made because the target machine actively refused it.

And repeats... Slightly irritating just watching the work queue with send ETA unknown. Good luck sending that 50000 point unit up when it's finished.

[toTOW]

As a temporary workaround, the Chodera lab added 10k WUs on plfah1-1 (mostly p1171x). You should at least be able to get work to feed your GPUs !

[Aurum]

I'm not having any problem getting WUs, just giving them back.

[toTOW]

There's nothing you can do about it until the work server comes back ... Unfortunately, projects from 171.67.108.157 have no collection server set, so this feature won't help either

[stealthswifttlk]

I actually bailed and gave up on F@H back in 2009 because under the old client (V6) I had about a week's worth of WUs that wouldn't upload at one point, and it was starting to over-write slots. I basically left feeling 'what's the point?' back then because I figured I'd done the work but it was irrelevant if WUs were going over time frames because they wouldn't upload! It felt like it was a waste of time and resources (electric/wear and tear on my machine). I only came back to F@H just before Christmas with relatively few problems this time around. I know the stats server goes down periodically but that's seen as a lower priority as it's not affecting the science and that's fine - they usually sort that out eventually anyway. I also see it as less of an issue if they don't have WU's to assign to me, for instance first thing Monday morning in my timezone as the USA is between 5 and 8 hours behind me - folders might have just exhausted the available workload over the weekend and they haven't had chance to re-populate the assignment servers. But it's more galling to me as a contributor to not be able to pass back work that has been done than not be able to be assigned something. With the extra wattage that's pulled running a GPU slot, I'm again putting myself in a mindset to just not bother running a GPU slot until I know that the GPU WUs will actually upload as it's a possible waste of resources to do stuff that could potentially be reassigned to others through no fault of my own. The up-time and reliability of the F@H network has a definite baring on my confidence in this project, and subsequently my ability to recruit potential new participants to the cause.

As I haven't been using the V7 client that long, does anyone know how many queue slots are available for each slot (CPU/GPU) I have? From memory, it used to be something like 10 under the old client and then started dropping the oldest and over-writing that queue slot.

[SteveWillis]

I had a half dozen WUs waiting to upload and after I restarted the client within a minute they all had done so.
On my Linux box I execute the following. A reboot should do the same in windows.

systemctl stop FAHClient
systemctl restart FAHClient

actually I have a script that
systemctl stop FAHClient
sleep 2
pkill -e -9 FahCore
pkill -e -9 FAHClient
systemctl restart FAHClient


but the extra steps are to restart hung folding slots

[bruce]

I talked to cxh (the owner of that server) this morning and he said he was away and he wouldn't be able to fix it immediately. According to my information, the server went down about 10:00 PDT and came back online accepting WUs at about 18:00 PDT. From the comments above, I'd assume that it then faced quite a backlog of WUs attempting to upload. Your posts support my observations.