Bigadv reform

[bruce]

I like the score idea. I'm not sure how complicated it would be to implement, but it seems like a better solution than straight-up core count (though, as has been mentioned, almost anything is). It also appears to address requests for things like keeping different bigadv levels, assuming that that's something PG would want to do.

We have said repeatedly that thread-count is a poor tool to manage WU selection, but it's the only thing that the Assignment Server knows that even approximates performance. A complete redesign of performance measurement would be required and that's not one of the options being discussed in this topic. It wouldn't be a bad idea, but let's be realistic. A dependable client for OS-X and fixes to V7 and improvements to several FahCores and several Servers and etc all have higher priority.

Science can use a lot more resources than the sum of all the Donors can provide. but the sharp distinctions between Uni/SMP/Big/GPU/PS3/(etc?) is too granular. Thus at a particular point in time, the sum of all the projects in a single one of those categories might be under- or over- served by the number of donors trying to process those WUs. All projects need their fair-share of donors, and it's not easy to migrate Projects from one category to another if that's what needs to be done to properly tune this virtual super-computer.

. . . its up to you to decide if you can reliably meet the requirements or not. Its status will most likely change over time at the PG's discretion, they've said that. There's a lot of things that can be improved with many aspects of F@h, some have a greater benefit-cost ratio than others.

The Pande Group will continue to adjust things to improve the overall scientific throughput. Such changes can be disruptive to donors, however, so changes of that type are made RARELY, and with months and months of notice. In fact, the reason the 6903 WUs are inconsistent with the others is that they are the left overs from before a change that was announced long, long ago.

Moore's Law caused the number of 8-thread machines to grow much more rapidly than the need for machines to process the most difficult 10% of the Projects so the deadlines and minimum hardware requirements were changed to represent the revised top 10% of the Hardware.

And, yes, this has all been discussed over and over and over again ... which is why the other topic was locked, and why the current thread is about to be locked. None of the suggestions made in either topic are things the Pande Group hasn't heard before or thought of, themselves.

[PinHead]

Hopefully, when an AVX enabled core from GROMACs rolls out; all of these discussions will be mute. As AMD and Intel have both gone the route of more cores at lower GHz with an AVX pipe that might make up the difference with floating point precision. Only time will tell, I only say "might" because we haven't seen AVX yet. Hang in there!

[7im]

What does the addition of AVX change in the current BA program on the current BA hardware of choice? How does it moot, and what?

[PinHead]

Should make lower GHz processors perform 2 to 4 times the work in a single cpu clock cycle given a 32 bit client with a 256 bit register.

[Zagen30]

Science can use a lot more resources than the sum of all the Donors can provide. but the sharp distinctions between Uni/SMP/Big/GPU/PS3/(etc?) is too granular. Thus at a particular point in time, the sum of all the projects in a single one of those categories might be under- or over- served by the number of donors trying to process those WUs. All projects need their fair-share of donors, and it's not easy to migrate Projects from one category to another if that's what needs to be done to properly tune this virtual super-computer.

Wait, how are you thinking the different hardware types would factor into this? I was thinking it would primarily apply within SMP folding only, maybe making its way to GPU folding if applicable, but not overlapping between the two. I certainly wasn't advocating trying to move projects between hardware types because I know that's an intensive process.

I wasn't thinking ironclad groups that would only get certain projects and none from any other class, but more like being blocked from ones that are out of your league. Say, for instance, bigadv-32 (theoretical) is assigned to grade 10, bigadv-16 to grade 8, bigadv-6 to grade 6, and a few different classes of "regular" SMP to grades 2-5. A machine that got a score of 10 would be able to get any grade of work, but the assignment servers would be heavily weighted to give them bigadv-32 since there'd be so few machines capable of doing it. Like it is currently with the -bigadv flag, they wouldn't be guaranteed to get grade 10 work, and would dip down if there was a shortage, like when everyone was running the original bigadv work on their i7s. If you've got a machine that only has a score of 4, you'd be eligible to pull in any work with a grade of 4 or lower, but obviously all of what we now term bigadv work would be off-limits, as would the grade 5 "regular" SMP work. If there was a shortage of grade 4 work or a surplus of grade 3 work, maybe the assignment servers tweak the weighting to give more 3 work to the grade 4 machines.

I figured that every so often there'd be a review to determine whether the grades should be changed; in the above example, if they're getting set to introduce bigadv-64, they make bigadv-32 now have a grade of 9 and -64 work becomes grade 10. This would require that everyone's client be updated, though, which could be problematic considering how many donors don't upgrade the client very often. Either that, or pushing out automatic updates, which I'm not sure PG necessarily wants to do.

That doesn't even factor in the point system, of course; would there still be a bonus for bigadv? Would it only be for, say grades 7 and up? I can't imagine a different point scheme for each grade since that would likely really annoy the low-end users.

You may be right that that much granularity is unnecessary, though. And the complexity of the server code would certainly be a factor.

[bruce]

You're assuming that the only granularity that matters is between BA and SMP. My point is that the same granularity problem exists elsewhere in FAH and potentially has the same sort of issues. As long as there are hard lines between categories of projects, there will be the potential of problems getting everybody's projects completed. It's not like the Pande Group can increase/decrease Project priority settings and migrate the processing resources to where they're needed most this month.

Adjusting the minimum number of cores is NOT a dependable way to manage BA so no matter how you answering the poll, it doesn't solve the real problem for BA and it doesn't address the bigger problem that exists for all of the other categories of Projects. The only thing it addresses is a percentage of the people who stumble into the BA setting without a full knowledge of how important the deadlines are.

[PinHead]

AMD seems to have structured their core to rely on AVX with the shared AVX per 2 core pipe. Intel is giving 2 256bit AVX pipes ( haven't found if it is because it takes more clock ticks to setup the instruction, something they have a history of ). But AMD 32xx, 42xx, 62xx series and upcoming PileDriver opterons have the potential to benifit. These are not just server class cpu's. Intel Sandy Bridge and Ivy Bridge based processors also have the ability to benefit. Again, not just server class.

So I guess the worst thing about the 16 true core recommendation is that there isn't ( Until the 10 core E5 with 10 HT ) an Intel solution for the 16 real core minimum guideline ( that I have been able to locate for purchase ). I'm not an overclocker and I realize they can do amazing things and that their abilities play a factor here. But a dual processor 8 core 8 HT system runs a bit like a true 12 core system. I say all of that to say that when AVX runs on an Intel, the HT might become more like 75% of a true core than the 50% or so that they are known for (O.C. or not).