New Servers Idle?

[PantherX]

...how much computational work was actually completed by how many clients (the "hours clients spent computing" divided by "wall time" mentioned by Joseph Coffland is a good start)...

There's a new number of compute hours logged per hour graph here for you to view: https://apps.foldingathome.org/credit-log

When there's a new Work Server that is being on-boarded, there's a lot of stuff that needs to happen. It is a mixture of human input, waiting for various tasks to finish and then the verification before it goes into production. A blank Work Server will need to be populated with WUs so that it can then assign them to the clients. This step is rather intensive and can take a while depending on how many WUs are gong to be generated.

Moreover, F@H Team works with other teams to decide which proteins to focus on. For sake of argument, let's have a massive team of researches the bottleneck would not be F@H, instead, it would be the teams involved in X-ray crystallography. That unfortunately can't be solved by distributed computing since it requires experimentation and very specialized expensive equipment (not to mention time, money, scientists). It is great to see so many teams collaborating and working with each other to fight the common enemy!

My take is that while significant changes have happened over the last few weeks, the momentum is bound to continue for some more weeks so am looking forward to them

[UofM.MartinK]

There's a new number of compute hours logged per hour graph here for you to view: https://apps.foldingathome.org/credit-log

When there's a new

That is a great addition to the existing stats! How did you hear about it?

The "Credits" in the top is WUs, I assume?

Next cool stat would be how many assignments did the AS actually hand out per hour

[JimF]

"Conventional" supercomputers for the most part are (unlike a decade or so ago) a very closely connected high bandwidth low latency distributed compute platform (without much distributedness) … They are actually much better for some types of science and calculations than the distributed platforms and as such for the most part are used for purposes that require their performance characteristics - "It is not all about the FLOPs" is fair to say, or even "It is not how much power you have got but how you use it" … They are also fiercely expensive to purchase and support … FAH is probably even more expensive but its model minimises costs to the scientists and effectively crowdsources the costs outwards, by having people donate parts of their electricity bills, pc hardware/software budget, network charges - total all this up and it will be even more scary that the total compute power !! - oh yes and support, guess that is "paid for" by the myriad volunteers who try each to their own ability to help the next folder who encounters problems they have already seen/dealt with - no call centre costs, or salaries for First Line and Deep Technical support, or 24hr call out charges

The latency is a key point that I don't think most crunchers appreciate. They purposely created the MSM in order to allow for the high-latency (delay) in distributed computing. That reduces the efficiency as compared to a typical supercomputer, which would not allow for more than a few microseconds of delay, not minutes or hours (or days with BOINC).

However, "efficiency" is a tricky proposition. It depends on what you are comparing it to. If you can treat some diseases with Folding (or other DC projects), it is a lot cheaper than a pandemic.

Crunch on.

[PantherX]

...That is a great addition to the existing stats! How did you hear about it?...

I actually saw it early last week before the Fire Side Dev chat and then heard Joseph talk about it. There was another Stats dashboard that he was working on but had to disable it since the surge of new donors meant that it wasn't keeping up (as he was still experimenting and it wasn't ready for production). If you know of people who would be able to represent Stats in a better and informative way or have database experience, they would be welcomed to help out: https://www.reddit.com/r/pcmasterrace/c ... s/fkyg9ns/

...The "Credits" in the top is WUs, I assume?...

Yep, how may WUs were processed by the Stats system.

[warrelium]

Seems like we're running out of GPU WUs. They've been trickling in for me for the past several days and it looks like even the ec2 spot donations from AWS are having a problem getting work since 4/8: https://folding.extremeoverclocking.com ... =&u=928671

[Endgame124]

Seems like we're running out of GPU WUs. They've been trickling in for me for the past several days and it looks like even the ec2 spot donations from AWS are having a problem getting work since 4/8: https://folding.extremeoverclocking.com ... =&u=928671

Is aws having problems with work, or did they pull the plug?

[UofM.MartinK]

https://apps.foldingathome.org/credit-log

...The "Credits" in the top is WUs, I assume?...

Yep, how may WUs were processed by the Stats system.

Do you (or anybody) have a screenshot (or the data) of this plot from two or more days ago?

Suddenly, it looks a LOT more even than yesterday, and I forget to save the dataset from earlier...

I am working on correlating these "credit-log" WU stats with
* WS WU reports from https://apps.foldingathome.org/serverstats (separated by GPU & CPU),
* client logs for statistic of how often clients were successful or failed in retrieving WUs.

I currently use my own client logs from the last 21 days (6 CPU-Only clients), so it is not representative,
but should help putting some numbers into perspective.

[Neil-B]

A couple of days back the log had a big spike on it that related to one of the stats catch ups - this has now aged off left and the graph is much more regular - could that be it?

[UofM.MartinK]

There was definitely a spike, yes, and some more extreme ups/downs. Is it certain that this was all only stats catch up?

The database knows when each WU was returned - and it is important to calculate the speed bonus points. So even if there was a backlog in processing the data, in theory the graph could reflect the actual work returned in each interval. So which of the two things happened:
* all stats finally processed correctly and we are now more real-time (then a two-week or month view would be great - since it would also be much more even and representative)
* this plot does not account for the actual return date of the WUs, it is merely a gauge for what the stats system is crunching

I know this credit-log is kinda "work-in-progress", so there is no "good" or "bad", "right" or "wrong", but can we quickly figure out/get feedback on what this (very useful!) plot is based on?

[Neil-B]

Think the graph shows when the credit was made to the stats pool not then the credit was earned - but may well be wrong.

I'll know in a bit cause I have 80+ WUs returned to one of the severs that has yet to post anything to stats - and I won't be alone on that ... when they hit there should be a smallish but noticeable spike if my guess is right - if they work off credit earned there won't be a spike

[bruce]

Think the graph shows when the credit was made to the stats pool not then the credit was earned - but may well be wrong.

Right.

[UofM.MartinK]

Think the graph shows when the credit was made to the stats pool not then the credit was earned - but may well be wrong.

Right.

Right that he is wrong, or right that "the graph shows when the credit was made to the stats pool" ?

[Neil-B]

Normally I'd bet on me being wrong … but I think bruce might have been indicating that for once I'd got it right … or maybe I've got that wrong as well

[UofM.MartinK]

I now also think he meant you got it right. I just reviewed some of the historical data (unfortunately the one older than 8 days is lost to me), and once an hourly value was "published" it seems to never change. This is supporting the hypothesis of "the graph shows when the credit was made to the stats pool not then the credit was earned"

[PantherX]

The credits are allocated to the completed WU when it is successfully uploaded to the WS or CS. The Stats system simply goes out to each of the WS/CS and collects the credits to process it. Thus, if a WS/CS is offline, the processing of the credits is delayed but not lost.

Unfortunately, I don't have a screenshot of what the graph was few days/week ago.