I'm starting to experience a different, but related problem of WU taking longer to upload (original posts from: viewtopic.php?f=18&t=31876&p=308997#p308997) than it takes a 2080Ti / 1080Ti to complete them.
It could just be my cable internet connection, but currently having 37+ concurrent 50MB-100MB uploads is choking it to death (fast or slow servers: once the connection is clogged, then none of them get uploaded for hours or days).
My uploads are 4+ WU deep on some machines, and getting worse today.
Example:
<<Image: Had 5 WUs waiting to send for a single folding slot>>
Are there any plans for the larger WU sizes and not enough return data channel capacity to handle it (like longer WU run times)?
Any suggestions, other than turning off half (~20) of the GPUs, or getting a fiber connection (not really available for me yet)?
Or, like a setting to limit only uploads to 1 upload per FAH Client?
Without running a speed test, I know I can typically upload 100-300 KB/s with my connection. But, all it takes is a few slow FAH uploads and it starts backing up when the WUs take ~30 minutes to fold.
Thanks
GPU WU Uploads Backing Up P14220 ~ P14250
Moderators: Site Moderators, FAHC Science Team
GPU WU Uploads Backing Up P14220 ~ P14250
Last edited by dfgirl12 on Sat Nov 09, 2019 4:16 am, edited 1 time in total.
Re: GPU WU Uploads Backing Up P14220 ~ P14250
I 'finished' all the current WUs so they were idle, and disconnected half of the PCs from the network. It took 8-12 hours for half of the PCs to clear out the completed WUs. I'm adding in the others, one every 1-2 hours.
This problem has only happened with these larger WUs and short folding times.
I'm going to try FAH setting of: max-queue=2 (which didn't help, or limit the upload queue). Hopefully, something like that will keep the upload queue from spiraling out of control. Any other ideas?
This problem has only happened with these larger WUs and short folding times.
I'm going to try FAH setting of: max-queue=2 (which didn't help, or limit the upload queue). Hopefully, something like that will keep the upload queue from spiraling out of control. Any other ideas?
Last edited by dfgirl12 on Sat Sep 28, 2019 7:22 am, edited 1 time in total.
Re: GPU WU Uploads Backing Up P14220 ~ P14250
there are several topics discussing the same problem. There have been network problems at temple.edu. The network admins are aware of the problem. See the post here.
The two Work Servers vav3.ocis.temple.edu and vav4.ocis.temple.edu apparently have been throttled (perhaps by a shared campus router) and they require a certain minimum bandwidth to support FAH's communications needs..
The two Work Servers vav3.ocis.temple.edu and vav4.ocis.temple.edu apparently have been throttled (perhaps by a shared campus router) and they require a certain minimum bandwidth to support FAH's communications needs..
Posting FAH's log:
How to provide enough info to get helpful support.
How to provide enough info to get helpful support.
Re: GPU WU Uploads Backing Up P14220 ~ P14250
It looks like with those servers back online again, the congestion is being alleviated with more WU variety. Thanks!