Page 1 of 2

New bottleneck demands WU optimization

Posted: Mon Mar 23, 2020 1:12 pm
by Gulliver.Workstation
It seems the Folding@Home project has reached a tipping point.
I like many others have answered LTT's call and contributed my workstation to the project for the time being.
It seems the situation has shifted from infinite work fighting for computational resources, to infinite computational resources idling by and waiting for WUs
Now that the bottleneck has moved to the server side, i see a need to optimize the workload and map the network of donors based on their strengths.
My workstation has dual xeon processors totalling 32 cores with 128GB or RAM and a W7100 graphics card
clearly there are more powerful gaming PCs out there in terms of GPU horsepower, but much fewer machines with significantly more CPU power available on the network.
However, my workstation is currently folding a GPU WU and the CPUs are just sitting there waiting. I can't help thinking there are better suited computers out there to run that GPU WU much faster, while some poor i5 laptop is out there struggling with a CPU WU and my xeons just chillin...
I don't mean to put down anyone contributing to the project, every little bit counts and every contribution is welcome, but sitting here watching 32 cores idling is a real shame..
The F@H network should map all donor machines and start optimizing WU allocation based on best performance.

Anyway.. just my thoughts. Amazing project. Really felt good to see that my machine is contributing to COVID-19 research but really any disease research is worthwhile.
Cheers and keep up the great work

Re: New bottleneck demands WU optimization

Posted: Mon Mar 23, 2020 4:10 pm
by davidcoton
The team is aware of the problem and is trying to increase server capacity to meet the demand (there is plenty of work, it just can't be handed out and collected fast enough).
Prioritising the bigger machines doesn't actually help much, it just transfers who is disappointed to the low-end owners.
Since low-end users can only do the smaller work units, large work units are automatically kept for high-end clients.

Re: New bottleneck demands WU optimization

Posted: Mon Mar 23, 2020 5:00 pm
by Gulliver.Workstation
roger that.
again - not trying to discourage lower-end clients at all. the Apollo space missions could only dream of the processing power of consumer chips these days. just wanted to make sure jobs are handed out with some consideration and seems from your reply that this is indeed the case.
I hope the server capacity issue will be resolved soon enough and my machine will be better utilized ;)

Re: New bottleneck demands WU optimization

Posted: Wed Mar 25, 2020 12:32 pm
by Neil-B
davidcoton wrote:The team is aware of the problem and is trying to increase server capacity to meet the demand (there is plenty of work, it just can't be handed out and collected fast enough).
Prioritising the bigger machines doesn't actually help much, it just transfers who is disappointed to the low-end owners.
Since low-end users can only do the smaller work units, large work units are automatically kept for high-end clients.
I realise that "low-end" owners (hell most are damn sight higher spec than my kit from GPU perspective) may well be in many cases the solid core of folders over the years … I am surprised a bit at "doesn't actually help much" - a 24+ core slot can clear 14 day WUs in less than 4 hrs and 3 day (even sometimes 7 day) WUs in around 30 minutes (faster if greater number cores of course) … This has to speed up the progression of a CPU project through the various generations simply because WUs are returned quicker? … It won't help with assignment/work server loads, and it won't keep as many people happy (that they are participating/folding) but it may well get the science completed quicker?

Really hard and emotive topic I know … and I am sure that "prioritising/scheduling" to capabilities of system is non trivial … I am also aware that at this time the team must be so overloaded just trying to get back to the surface that debate of such may well be the last thing they need … but felt I wanted to "gently challenge" (in a non targeted, personal way) the "doesn't actually help much" comment … Please in no way take this as a personal attack on the poster of the comment - It isn't - I am just interested in whether fast turn round of WUs actually matters much.

Re: New bottleneck demands WU optimization

Posted: Wed Mar 25, 2020 1:00 pm
by anandhanju
Hi Neil-B, welcome to the forum. Thanks for helping out.

The fast turnaround of the WUs matters a lot. In FAH, WUs are generated based on the result of the last result. So, a WU, with Project=P; Run=R; Clone=C; Gen=G is only created and released for assignment when PRC(G-1) is returned and found to be valid.

If enough slow folders hold up the results from being returned, this could result in a shortage of WUs. The Project Type Stats shows there are only around 85k CPU WUs at this time, which at an assignment rate of 25k/ hour gets depleted quickly if results don't arrive at the same rate. I believe the researchers have released or will be releasing a whole bunch of new COVID projects for the CPU (core GRO_A7) that should increase the pool of available WUs.

Re: New bottleneck demands WU optimization

Posted: Wed Mar 25, 2020 1:12 pm
by puuteknikko
Is CPU work something specialized that can be done only with the CPU folding, or is it something that both GPU and CPU can do? I've been wondering if I should put my CPU to work as well, currenty I'm encoding videos 24/7 but can put that project on hold for some time.

Re: New bottleneck demands WU optimization

Posted: Wed Mar 25, 2020 1:13 pm
by Neil-B
Awesome … and thanks for the clarification - much appreciated - I'll leave my server configured to a couple of high cpu slots to help quick throughput as and when WUs are available

Re: New bottleneck demands WU optimization

Posted: Wed Mar 25, 2020 1:17 pm
by Jonazz
puuteknikko wrote:Is CPU work something specialized that can be done only with the CPU folding, or is it something that both GPU and CPU can do? I've been wondering if I should put my CPU to work as well, currenty I'm encoding videos 24/7 but can put that project on hold for some time.
Yes, some work can only be done by a CPU.

Re: New bottleneck demands WU optimization

Posted: Wed Mar 25, 2020 1:25 pm
by puuteknikko
JonazzDJ wrote:
puuteknikko wrote:Is CPU work something specialized that can be done only with the CPU folding, or is it something that both GPU and CPU can do? I've been wondering if I should put my CPU to work as well, currenty I'm encoding videos 24/7 but can put that project on hold for some time.
Yes, some work can only be done by a CPU.
Cheers, I'll put my CPU into F@H use then :D

Re: New bottleneck demands WU optimization

Posted: Wed Mar 25, 2020 1:36 pm
by peterjammo
Hi folks,

I've set up 2 elderly core2duo linux laptops, and have a third waiting in the wings. first one is chugging away on a 7ish day WU and is 51% through after 15 hours. Second, just configured, has been looking for a wu for the last 1/2 hour or so but "no wu's available for this configuration".

After reading this thread, I'm wondering whether my "help" is actually counter productive and whether I should just uninstall. Any advice?

I had intended to try to recruit friends and family, but I'm now suspicious that the minimum system requirements are out of date, and that the average home pc user is not really helpful. Again, any comments?

If I'm right that I should just take a bow and exit stage left, does anyone know of a similar project that can use low spec stuff like mine

Thanks to all for the wonderful project, whether I can help or not.

cheers
Peter

Re: New bottleneck demands WU optimization

Posted: Wed Mar 25, 2020 3:07 pm
by davidcoton
@peterjammo
Every contribution is welcome. As long as you meet the deadlines, that's all good.
AIUI there is plenty of work, the team are working hard to increase server capacity which will make them available.

Re: New bottleneck demands WU optimization

Posted: Wed Mar 25, 2020 3:25 pm
by Neil-B
@peterjammo

Please don't let questions like these (including mine) put you off … as @davidcoton said - as long as it is within the deadline (and the figures you quote easily best that) then every bit counts.

The current (rather peculiar/anomalous) situation where a significant outside factor has pretty much overnight increased the potential folding pool by some 20x (iirc) means that until the available WUs and infrastructure adapt and catch up there are opportunities for some hopefully healthy discussion on subjects not normally a focus (such as prioritisation of WUs to different machines) … as soon as a new normal is established this type of debate will be parked again.

F@H (from my observation) has grown over the years by allowing everyone to do their part (and long may it be so) … new project will spin up granting new CPU WUs and everyone will be busy before we know it :) … Please don't let questions/discussions by people like myself who are interested in things like optimisation put you off as I am sure that is in no way what anyone intended.

Re: New bottleneck demands WU optimization

Posted: Wed Mar 25, 2020 3:46 pm
by peterjammo
That's encouraging. As an avid collector of "spare" laptops, I've possibly got about 4 which I could get running again that wouldn't be much worse than the one I have folding at the moment. With all them up 24/7 I can probably cut down my coal bill too......

Once I see wu appearing regularly again I'll get on with my recruitment drive too.

Good to be doing something useful. I hope that others, like me, will consider continuing to support the project post Covid-19.

Thanks

Re: New bottleneck demands WU optimization

Posted: Wed Mar 25, 2020 10:50 pm
by Joe_H
peterjammo wrote:I had intended to try to recruit friends and family, but I'm now suspicious that the minimum system requirements are out of date, and that the average home pc user is not really helpful. Again, any comments?
Feel free to post here on the forum if you have a question about whether some older hardware/software can fold. Someone should be able to give an answer.

Have not looked at what is listed as minimum requirements in a while, probably needs some updating. Basically for CPU folding hardware that supports 64-bits is sufficient, Core 2 Duo or newer. Linux and OS X are just 64-bit, the 32-bit Linux support has ended. For Windows I will have to check, there were some projects recently that were still being done on both 32 and 64-bit Windows versions as the processing could be handled within the 32-bit memory limits.

GPU folding - Minimum of a Fermi based nVidia card, Kepler based is better, some of the low end cards in the older series are not fast enough. For AMD the minimum is the Radeon HD 5800 & HD 6900 series cards. Later series need to be based on GCN or later, the Terascale based cards lack Double Precision. Many of the laptop versions of these cards also lack DP.

Re: New bottleneck demands WU optimization

Posted: Wed Mar 25, 2020 11:59 pm
by bruce
JonazzDJ wrote:Yes, some work can only be done by a CPU.
Every project P can be done ONLY by a GPU or ONLY by a CPU. If the available assignments happen to be one or the other and you clinet wants to do the other, you're not likely to get an assignment. The scientists are busy insuring that plenty of both are available but it takes constant attention.

If I'm a scientist (I'm not) running a particular research project that is best suited for one or the other, then that's the type of project I'll configure.