Unbalanced Scoring

[Guru]

That's too bad. Width and precision, as you put it, is exactly what it will take to fix this issue. There are multiple ways to perform math calculations which will result in the same correct answer. The software simply needs to be changed so that processors will be able to make the calculations quickly. As for right now, it seems that there has been a great deal of concentration on making the software work well with graphics cards. This is completely anti-productive for multiple reasons, and that's without even bringing the point system into the equation. That's primarily considering results, which is the goal, correct?

[P5-133XL]

I'm not a big fan of people whining because others do more and get more. If they were not doing more, then I suppose there is a fairness point but that is not the case here.

For a sports type analogy. There are professional players and there are millions of children and everyone else inbetween. Should we cap the professional players rewards because they are too good and no one else can compete with them? Should we elevate everyone else and spread the reward system to all the amaturs. To me, arbitarily manipulating the reward system is rediculous because it will distort the true values. People should be compensated according to their value and the same goes with folding -- Whining be dammed.

The fact is that no one has diminished the rewards for the CPU folder. They have the same scientific vlaue and are receiving the same reward. Along comes a new kid called the SMP client and he is better skilled and produces more scientific value and receives more points -- People whine. Now an even better folder has appeared the Nvidia GPU folder and it can fold far better; produces far more scientific value; and gets far more points -- People whine again. No one has done anything anti-CPU. The old rewards haven't changed and their scientific value hasn't changed.

What can Stanford do to satisfy everyone? Should they cap the reward system for GPU's? Should they reward the CPU extra? The truth is that people optimize their folding choices by optimizing for points. Scientific value is hard for an external person to judge. So if Stanford arbitrarily increased the CPU reward system emphasizing them over GPU's then people would choose to fold more with CPU's and not use GPU's. The net result would be that the scientific value that GPU folding gives would suffer.

Please allow Stanford to decide the scientific value and assign points appropiately and the folding effort will be optimized. Anything else distorts the optimization and causes the scientific value to suffer. What is the important goal here -- the science, or the donators ego?

[crapiecorn]

I have the following processors folding:
-AMD Athlon 64 4000+ (1 x 2.4 GHz) San Diego
-Opteron 180 (2 x 2.4 GHz) Denmark
-2 x Intel Xeon 5310 (4 x 1.6 GHz) Clovertown

They add up to about 1000 points/day (11 processors).

So why not run the smp client ? Looking at your ppd i'm guessing you are running the normal client ?

[Guru]

It's funny that everyone keeps talking about how no one has decreased the points that the CPU folder earns. Personally, I believe that the software has been designed to take advantage of GPUs, which results in CPUs earning less points. That's basically the same thing.

It's even funnier when very competitive people talk about how the science part is important to them. If you weren't competitive and folding specifically for the points and the rank, you wouldn't be on this forum or displaying your points in your signature. XD Let's face it. As far as you're concerned, the science behind it is just a bonus... Let's see them modify the software so that CPUs go 100 times faster than GPUs and I'm sure that "whining" will be what you are doing...

[7im]

I don't see how focusing on releasing a new client that has a potential to produce as much a science as all the other clients combined is anti-productive.

Also, a + b is simple math, however, simulating complex protein strings is not. The Gromacs software is already highly optimized for that process, and optimized for several different types of hardware. Your comment about changing the calculations to "just make it go faster" doesn't make sense to me. Make what go faster?

Maybe you could explain your post in more detail...?

[Guru]

Is performance/competition more important or is sheer numbers?

Better, Is performance/competition more important or is Science?

Performance / competition is ESSENTIAL to improving Science at a faster rate than if it were not present. With this being the case, tell me whether or not it's better to have more scientific knowledge or less. Science is great, but having performance / competition driving the rate at which science is learned truly is better, IMO.

[Guru]

I don't see how focusing on releasing a new client that has a potential to produce as much a science as all the other clients combined is anti-productive.

Also, a + b is simple math, however, simulating complex protein strings is not. The Gromacs software is already highly optimized for that process, and optimized for several different types of hardware. Your comment about changing the calculations to "just make it go faster" doesn't make sense to me. Make what go faster?

Maybe you could explain your post in more detail...?

Just as you said, (and I bolded it), the software was optimized for several types of hardware. It was optimized for GPUs.... CPUs are faster and more plentiful. Making the software go faster on a multi-processor machine is the key to "fixing" the point system as well as the amount of science-related work that is completed. As I stated earlier, there is always more than one way to perform a calculation, and there are multiple methods of implementing the way this is achieved. The software needs to be optimized so that processors are utilized to their full potential, as they clearly are not currently...

[uncle_fungus]

It was optimized for GPUs.... CPUs are faster and more plentiful.

Actually this is completely wrong. GROMACS has been running, optimised, on CPUs for years. The main processing code is in hand coded assembly for optimum efficiency on a wide range of CPU architectures. GROMACS on GPUs is a fairly new development which utilises the streaming architecture of modern GPUs allowing the same calculations to proceed MUCH faster.

[Xilikon]

Most of the debates about points vs science always involve people who feel shafted by the latest trends and doesn't want to pay a dime to follow them.

Trying to turn the table to your advantage doesn't work. You said that the performance / competition drive the science further and that's true. That also mean that if you want to get the reward for being competitive, you will have to invest into what produce the most.

[FordGT90Concept]

I will admit that having high scoring clients does have the appearance of devaluing the older clients, though not true because no points values have been reduced.

This is happening without a doubt. My 8 x 1.6 GHz instances of 5.04 client only adds up to about 800 points/day while 4 x 2.4 GHz on SMP adds up to around 1500 points/day. That again brings me back to the great equalizer: time. My 8 cores consumers more power than your 4 cores and that shows in terms of execution time spent. Yours is more efficient but mine is spending more time in contributing. Both should be valued--not in terms of work accomplished because the clients weigh heavily on that but the important thing here is the people and the effort they put into it. It takes me hours to get all 8 clients downloaded and running, it costs an enormous amount of power to run that EATX motherboard, 6 HDDs, and 8 x 1 GB ECC sticks of memory. What am I really contributing to the project? I don't care about the work accomplished because that is solely determined by the efficiency of the clients. I care about the time I am letting you use, free of charge, on my hardware. You should value that time I contributed more so than the actual hardware. As with anything free, you get what is given to you. You should be rewarding my donated processor time, not my donated hardware.

But none of the alternatives are any better. If your remove the link between points and scientific production to make the older and newer clients score more evenly, then the value of points becomes zero, arbitrary, pointless (pardon the pun). And then how does one compete at all when there is no measure of performance? That's not an improvement, IMO.

The measure of performance is in efficiency. Since clearly the GPU clients are the only ones that are efficient, I really think this aspect has become moot; however, in order to satisfy those that do fold competitively, all you got to do is run some numbers on the processor time, processor count, and WUs...

It has been said many times that the points are, for all intensive purposes, pointless. I wholely agree because it has been clearly demonstrated that there is little science to the point system. It is completely and utterly broken. The people that live for points can still have them but the main factor in determine contributions should not be the broken point system, it should be processor time.

The issues of points vs. performance has been debated for as many years as the project has been running, and no one has yet to suggest a change that would be a win-win for all parties involved. I wish I could point you at the multipage rants from the old Folding Forum that went on for 10s of pages, and still didn't come to any workable improvements.

I am growing more and more convinced that the software is not consistent enough for it to even be a subject of debate. F@H is not a means of benchmarking systems. SuperPI is. If that is one of F@H's objectives then wow, a lot more effort needs to be put into standardizing the core of all clients. I know this isn't going to happen so again, I stress the importance of donated processor time. It is something readily available, clearly shows contributions, and from it, can calculate hardware contributions (in terms of power using the power formula).

You can't add points to the CPU clients without devaluing the work us CPU folders done over the last 6 years, or without devaluing the much higher amounts of science the faster clients are producing. And if you reduced the points given to the GPU clients to make the CPU clients more competative, then all the GPU users complain. You can't steal points from one type of client without offending the other types. And because scientific production is why we are all here, I think it best to keep the points tied to that.

Average the amount of time it took per WU over the past 6 years, multiple it time the number of WUs, and make that the base processor time. All past work is effectively accounted for. The clients should be completely irrelevant in terms of donations and only relevant to competition which is completely separate.

P.S. I'm not trying to disuade anyone from trying to suggest improvements, but please consider that the topic may have been beaten to death many times over already. Sorry.

Then beat my suggestion to death and we'll try to come up with a comprise that makes everyone as happy as they can reasonably be. Like I said, I would be happy just by displaying the total processor time contributed. I really don't care about the points system because, for all intensive purposes, it is broken beyond repair. In fact, I doubt there is a way to fix it. Hardware and clients just vary too much. That's why it is critical to break it down to a lower, fundamental level. The one thing that is constant across all users, all hardware, and all clients.

I do believe that the power formula (power = WU / time) would promote competition among high end contributors far better than the current points system. In order to get the highest number, they would have to have exclusively powerful computers in order to get a high score. If the number of active processors is thrown into that formula, it could offset the weaker ones the user is also donating as to not damage their score.


.........
Let me break it down. There is three important variables:
-Work Units completed (directly impacted by hardware and clients)
-Total thread time (tangible donations)
-Folding threads active (user effort)

The score is some how derived from those three. An infinite number of scores could be derived from those values. One possible formula is:
Score = (WU / Time) * Active Threads

The important thing, I think, is to make donated time more important than actual scores. Scores should be competition centric and constantly evolving. That is, if someone quits donating, it shouldn't take long to pass them them up for at least the short term. But in terms of contributions, it should be based on thread time and maybe to a lesser extent, the number of active threads. As contributors, those are the only two variables we can control. They are tangible, absolute, and don't change in significance over time.

Active threads and thread time accounts for physical contributions and WUs completed drives those that fold competitively to do better.

[Xilikon]

Keep in mind that a WU is not constant in size. There are small ones, average and big ones. Each WU family will take a different amount of time to crunch. This make your argument flawed.

What if someone is unlucky to always get extra chewy ones like 1487 while someone else get very small ones exclusively ?

[Guru]

GROMACS on GPUs is a fairly new development which utilises the streaming architecture of modern GPUs allowing the same calculations to proceed MUCH faster.

Thank you. You just proved that I am entirely correct...


I have thought of a way to make the scoring system fair. Basing points on WUs finished is no longer adequate.

They should display the number of cores / processors you have running, the number of WUs you've went through, and the total time that each core / processor has been active =)

If they did all 3, there could be more fair comparisons made, especially if they listed whether or not it was a GPU or a CPU...

Listing exactly what the user has done would remove people from appearing to be contributing a lot when in reality they only contribute 3 hours a day with their GPU...

There are currently several assumptions that can be made by looking at a person's score that used to be accurate, but no longer are. By listing specific details about the user's contribution, you no longer have to make assumptions.

It really is just that easy. It's sad that no one else has thought of this before now... XD

[FordGT90Concept]

Keep in mind that a WU is not constant in size. There are small ones, average and big ones. Each WU family will take a different amount of time to crunch. This make your argument flawed.

What if someone is unlucky to always get extra chewy ones like 1487 while someone else get very small ones exclusively ?

Then base WU score on number of steps or time of execution (which is already being done). Again, any scoring method directly involving the WUs is extremely client and hardware dependant. Use WUs to drive competition, not contributions.

[uncle_fungus]

GROMACS on GPUs is a fairly new development which utilises the streaming architecture of modern GPUs allowing the same calculations to proceed MUCH faster.

Thank you. You just proved that I am entirely correct...

How so? It's physically impossible to get CPUs to process WUs at the same rate as a GPU due to the calculations involved. CPUs are multipurpose units, GPUs aren't, and their strengths are something that FAH can put to good use.

[John Naylor]

Listing exactly what the user has done would remove people from appearing to be contributing a lot when in reality they only contribute 3 hours a day with their GPU...

If you think of contributions as merely an amount of time contributed, then yes of course that seems like a small contribution compared to a CPU running 24/7. However the assumption that GPUs are not 20+ times more powerful than CPUs is wrong.

A Q6600 runs at a theoretical maximum of 42 gigaflops. Per core that is 10.5 gigaflops. nVidia's 8800GT has a theoretical maximum of 504 gigaflops (a card which is not even from the last series, but the one before that). I don't know about you but I make that a 48-fold increase in power. ATI's HD4870 theoretically tops out at just over 1 teraflop. Using a nice round 1000 gigaflops that is a 95-fold increase in power. Providing this sort of additional power to the project should be rewarded... even if the project currently only uses half of the theoretical maximum, one client adding the equivalent of 24 or 48 Q6600 cores (6 or 12 entire Q6600 processors) should be rewarded by the project, IMO. That increase assumes that a Q6600 can operate at its theoretical maximum running Folding@home, a feat that is impossible with most any program. The gains are even bigger if you factor that in, and bigger still when compared to older processors.