155.247.166.220 downloads stalled

Moderators: Site Moderators, FAHC Science Team

vvoelz
Pande Group Member
Posts: 552
Joined: Sun Dec 02, 2007 8:07 pm
Location: Temple University, Philadelphia PA

Re: 155.247.166.220 downloads stalled

Post by vvoelz »

Thanks Sparkly -- I'll pass that on to our IT folks
scott@bjorn3d
Posts: 80
Joined: Tue Dec 19, 2017 12:19 pm

Re: 155.247.166.220 downloads stalled

Post by scott@bjorn3d »

Good luck with it but for now I can not have 3 2080 TI's sitting idle half the time. So blocked in my firewall.
HaloJones
Posts: 906
Joined: Thu Jul 24, 2008 10:16 am

Re: 155.247.166.220 downloads stalled

Post by HaloJones »

I've temporarily unblocked since I'm WFH in front of one of my computers so can sort if there is a problem. So far, have not been directed there.
single 1070

Image
HaloJones
Posts: 906
Joined: Thu Jul 24, 2008 10:16 am

Re: 155.247.166.220 downloads stalled

Post by HaloJones »

Had a stall on a Linux box, ran the following:

sudo netstat -tunp | grep .220

that will give you a response that includes the PID and hopefully FAHClient as the process.

kill the PID.

That worked for me as the client then re-started that connection and got a unit elsewhere.
single 1070

Image
parkut
Posts: 363
Joined: Tue Feb 12, 2008 7:33 am
Hardware configuration: Running exclusively Linux headless blades. All are dedicated crunching machines.
Location: SE Michigan, USA

Re: 155.247.166.220 downloads stalled

Post by parkut »

stalled download, around 5 hours ago

21:22:29:WU01:FS02:Connecting to assign1.foldingathome.org:80
21:22:30:WU01:FS02:Assigned to work server 155.247.166.220
21:22:30:WU01:FS02:Requesting new work unit for slot 02: RUNNING gpu:0:GM206 [GeForce GTX 960] 2308 from 155.247.166.220
21:22:30:WU01:FS02:Connecting to 155.247.166.220:8080
21:22:30:WU01:FS02:Downloading 3.56MiB
21:22:38:WU01:FS02:Download 5.26%
21:23:53:WU01:FS02:Download 7.02%
kc2lrc
Posts: 21
Joined: Sun Jan 04, 2009 11:10 pm

Re: 155.247.166.220 downloads stalled

Post by kc2lrc »

Can confirm, things are still very sluggy on this server as of writing. For the past few days I've had lots of stalled downloads, and just reset 3 tonight between my 11 GPU slots.

My thoughts on this matter are that the work server seems as though it's out of bandwidth - a 5MB download from it took about 2 minutes. It's as if the server is being DDoS'd by the clients. Perhaps the assignment server could be reconfigured to direct fewer clients at this server? That could help with the network consternation here.

In addition, this situation is exercising the bug in FAHClient in which stalled downloads never reset - I've manually killed about 50 of them in the past week just myself - and I imagine this is resulting in substantial idled resources across the Folding@home project in its entirety since it requires manual user intervention before the GPU comes back online. So I think reducing the assignments frequency to this server, if reasonable, would benefit the entire program.

Cheers -
Sam

PS - those of you running into this can reset the connection to un-jam the stalled download. On Windows, use the Sysinternals tcpview utility. On Linux, the 'ss' command (as root) does the trick, as such: 'ss -K dst 155.247.166.220 dport = 8080'
Image
Ichbin3
Posts: 96
Joined: Thu May 28, 2020 8:06 am
Hardware configuration: MSI H81M, G3240, RTX 2080Ti_Rev-A@220W, Ubuntu 18.04
Location: Germany

Re: 155.247.166.220 downloads stalled

Post by Ichbin3 »

kc2lrc wrote:'ss -K dst 155.247.166.220 dport = 8080'
The option -K is not working in my ss.
Image
MSI H81M, G3240, RTX 2080Ti_Rev-A@220W, Ubuntu 18.04
kc2lrc
Posts: 21
Joined: Sun Jan 04, 2009 11:10 pm

Re: 155.247.166.220 downloads stalled

Post by kc2lrc »

It's possible you need to be logged on as root, or you have a different distribution that does the command a bit differently. This is from Debian 10.
Image
HendricksSA
Posts: 336
Joined: Fri Jun 26, 2009 4:34 am

Re: 155.247.166.220 downloads stalled

Post by HendricksSA »

I sure hope this problem will be resolved soon. I've lost half my folding time due to stalled downloads from this server ... 36 hours in the last 3 days. For me the quickest solution is to disable the network adaptor and then re-enable it. That does require the root password.
bruce
Posts: 20824
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: 155.247.166.220 downloads stalled

Post by bruce »

155.247.166.220 has been the target of cyber attacks on and off for the past several days. (I have to wonder why somebody has decided to do that.)
psaam0001
Posts: 377
Joined: Mon May 18, 2020 2:02 am
Location: Ruckersville, Virginia, USA

Re: 155.247.166.220 downloads stalled

Post by psaam0001 »

Let's see.... Classic denial of service via ping flooding? Malware on clients computers from sources other than the F@H client distribution servers? Infrastructure level intrusion to the router's IOS? Just my guesses.

Good luck getting to the bottom of this....

Paul
bruce
Posts: 20824
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: 155.247.166.220 downloads stalled

Post by bruce »

That server is replying to pings but the server code isn't running. I understand it's shut down to off-load some accumulated data and make room for your uploads.

155.247.166.220 / 503 HTTP_SERVICE_UNAVAILABLE
psaam0001
Posts: 377
Joined: Mon May 18, 2020 2:02 am
Location: Ruckersville, Virginia, USA

Re: 155.247.166.220 downloads stalled

Post by psaam0001 »

Thanks for the update Bruce... I can only try to deal with my clients that are running on what was part of the latest & greatest technology 10-12 years ago (and one recently built Win10 system).

Paul
bruce
Posts: 20824
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: 155.247.166.220 downloads stalled

Post by bruce »

Have you tried it today? It might be back up.
HendricksSA
Posts: 336
Joined: Fri Jun 26, 2009 4:34 am

Re: 155.247.166.220 downloads stalled

Post by HendricksSA »

Just lost 8 hours after a hung download. I was assigned to .220 at 1900z so I am not a good test. I will try firing up the computer tomorrow if this problem is really fixed.
Post Reply