Suggested Change to the PPD System

[k1wi]

I haven't given up on developing this - have just taken a small step back to let the air cool and had a chat with a few folders via PM.

One thing I would note is that to anyone who says computers don't experience declines in PPD under the current system - that clearly isn't true. I just have to look at my i7 920's PPD over the past 30 months to see that PPD has dropped on that machine. Core changes/optimisations & increases in the size of protiens and of course, drops from bigadv to regular SMP can also have a detrimental effect on older hardware which also results in lower PPD. Because people 'expect' that the ppd of their computer should remain constant any of these changes leads to quite a lot of disconcent amongst folders whenever something happens that causes any sort of change.

I would suggest that building an expectation into the system that over time the PPD on a particular system will decline roughly at the rate of technological would be a positive thing for folders - especially considering they would understand that the PPD of the latest and greatest computers will not, in return, grow exponentially. It makes it clear that PPD is relative and decreases over time are to be expected, unless you continue to fold at the same relative rate as you buy new computers.

At the same time, the revised system protects your previous 'investment', encouraging you to continue to fold knowing that all your hard work in the past won't be exceeded in a week by a user with a nice new computer. I don't think anything turns small-scale folders off more than that. There is still the same relative point incentive for powerful folders to fold, it's just in order to 'shoot' up the rankings they have to continue to fold over a lengthy period.

[vbironchef]

I haven't given up on developing this - have just taken a small step back to let the air cool and had a chat with a few folders via PM.

One thing I would note is that to anyone who says computers don't experience declines in PPD under the current system - that clearly isn't true. I just have to look at my i7 920's PPD over the past 30 months to see that PPD has dropped on that machine. Core changes/optimisations & increases in the size of protiens and of course, drops from bigadv to regular SMP can also have a detrimental effect on older hardware which also results in lower PPD. Because people 'expect' that the ppd of their computer should remain constant any of these changes leads to quite a lot of disconcent amongst folders whenever something happens that causes any sort of change.

I would suggest that building an expectation into the system that over time the PPD on a particular system will decline roughly at the rate of technological would be a positive thing for folders - especially considering they would understand that the PPD of the latest and greatest computers will not, in return, grow exponentially. It makes it clear that PPD is relative and decreases over time are to be expected, unless you continue to fold at the same relative rate as you buy new computers.

At the same time, the revised system protects your previous 'investment', encouraging you to continue to fold knowing that all your hard work in the past won't be exceeded in a week by a user with a nice new computer. I don't think anything turns small-scale folders off more than that. There is still the same relative point incentive for powerful folders to fold, it's just in order to 'shoot' up the rankings they have to continue to fold over a lengthy period.

+1 Exactly

[ChasR]

K1wi,
I have to disagree that one can expect the ppd of a system to go down over time. In fact I believe the opposite is true. It's happened since I became involved, ppd has crept upward. The original SMP benchmark was 1760 ppd for a near Q6600 machine (actually a 2P westmere @ 2.33 GHz). Core a1 WUs made slightly higher than that on a Q6600 at 2.4 GHz. With the introduction of core a2, machines that were making 2000 ppd started making 4000 ppd (equal pay for equal work was the reasoning, otherwise the increased efficiency of core a2 would have been benchmarked out). THe benchmark machine changed to an i5 with the introduction of a3 WUs and the points were "normalized" so that a q6600 at 3.2 GHz would make 7125 ppd, again significantly more than on a2 WUs, In reality, while some WUs do produce the benchmarked value on a Q6600, a large number of them make far more (10-12K on p790x to name a few). So on the Q6600 @ 3.2 GHz, I've seen the ppd increase from a max around 3200 ppd to over 12,000 ppd in 5 years. Rather than recite the history across all hardware, I'll just say that I haven't observed a declining trend in ppd on any hardware I have.

[artoar_11]

In November 2009 I bought a C2Q 9400 (@ 3.5GHz).

Core_A1 (Win) x1WU (SMP) ~ 2000+ PPD; x2WUs (SMP) ~ 3000+ PPD.
With Linux - Core_A2 (SMP) ~ 5600 PPD;
February 2010 - Core_A3 (SMP2) Linux ~ 9000 PPD.
February 2012 - Core_A3 (SMP2) Linux ~ 10-11 000 PPD.

Hardware is the same - 2+ years. Who wishes, to comment

PS: february 2010 was introduced QRB system.

[7im]

Maybe Grampa_01 would like to comment on this points inflation as compared to his recent rant about low PPD on P8101s.

[vbironchef]

Just to defend K1wi: ChasR, I looked at your PPD. Very interesting to say the least. You and _s3v3n_ - User Summary - EXTREME Overclocking Folding @ Home Stats team must be using HPCS, but that isn't the topic or sort of it is. My point is that I was also using a QX6700 f@H 24/7 and I had to stop because the time frame to finish a smp wu just was not worth the electricity for the points given. Please don't ban me for trying to give constructive criticism.

[ChasR]

If you doubt my numbers, put some out there that prove me wrong.

If you are going to accuse someone of theft of service from HPCS, you better know what you're talking about. I've not gotten a single point from HPCS and your "must be using HPCS" is dead flat wrong and unappreciated. What I'm making now is on machines that I don't pay the power bill on, mostly from folks I recruited into the Project. I, like you, turned the others off for economic reasons, hence the 300,000 ppd decline in production.

[Grandpa_01]

Maybe Grampa_01 would like to comment on this points inflation as compared to his recent rant about low PPD on P8101s.

And just maybe 7im will build a rig that will fold the 8101 and have a qualified statement about them and quit posting usless digs at people. And I make more off of a smp than I do a 8101 when you figure in the dificulity. After all the smp's will fold with no problems at the same settings as the rest of the bigadv WU's where as the 8101's will not. So yes they are very much undervalued in more was than 1

Lets all stand still and let progress pass us by

[orion]

PS: february 2010 was introduced QRB system.

More like July 2009 viewtopic.php?t=10697#p105038

[k1wi]

I welcome Grandpa contributing to the discussion - not because he 'rants', but as a contributor running high powered computers. Is the reason he gets new hardware due to the exponential increase in points, and would he still invest in hardware if proportional scale stayed the same but PPD decreased over time at the pace of technological improvement?

As to my own numbers, my i7 920 has been folding for around 3 years now. In the past I earned 800,000 - 1 million points a month, now I'm down to 600,000 despite the addition of a stock Sandy Bridge i5. Yes most of that is due to the drop from Bigadv to regular SMP and an increase due to QRB (which could be seen as a step change), but your own results suggest that PPD is hardly constant (in your case the trend is upward) - yet there seems to be an underlying belief that they are absolutely constant.

To make a point on HPCS - I don't want to drag this into the debate too deeply as it will most likely end up being a flash in the pan (unless people choose to stump up cash once it leaves beta) - and I certainly don't want to throw around accusations of theft of service. I have my own opinion on that but others such as bruce have stated similar sentiments so I have left it to them to state.

I do, however, think that it does serve an interesting case study. Under the current point inflation process, running a (or a number of) HPCS can quite quickly surpass a user who has spent 4 years folding on a relatively average quad core computer, or, for that matter, even a relatively powerful computer. It gives those users a rapid ascent up the rankings, surpassing all the week-in week-out folders (who are often the ones that hang out around here). In the process all those relatively small but still very important folders (because they must be very numerous) see this rapid ascent and think "well that 100,000 ppd+ folder completely blows my contribution out of the water - in a month they'll have completely surpassed my 4 years of folding, I might as well switch it off and be done with it". I contend that if the proportion was the same, but the absolute PPD was smaller(and constant on average over time) , there would still be the incentive for smaller folders to contribute, but there wouldn't be the disenchantment.

Edit: by QRB I mean it made my initial 800,000-1,000,000 ppd possible, not that it led to the decrease my PPD.

[Jesse_V]

Quite the lengthy but intriguing discussion. K1wi's propose idea would be a significant change, but it makes sense to me and I see the advantage that it would have. I'm going to say that I agree that k1wi's system where the average changed would solve these points problems. Considering the length of this thread and the argumentative 9-page thread originally titled "Really" created after the 8=>16 core change for bigadv, I think that there's going to be some disruption when the change is made, even if there was some way to hold the inflation at its current level or something. I would also speculate that this change wouldn't be a trivial thing for the PG to implement. But over time, I think it will solve a lot of issues and make up for any problems it causes at the beginning.

[mdk777]

OK,

I spent a few quarters studying accounting before the the mind-numbing adherence to Arbitrary Government rules induced me to move on.
One of the concepts that I did find interesting was how to determine return on investment.

KEY RULE:

Forget SUNK COSTS

It doesn't matter if you have 50 cents, or 5 billion invested in a project. The past will not change.
All you can do is look forward and make your decisions based on what your costs are today, how they will change, and how the cash flows will change depending on if you invest more.

If you can't make a return on your investment...you don't throw good money after bad.

SUNK COSTS are history, they perhaps can be used as cautionary tales, or bar room braggadocio, but they don't enter into any rational business decision.

This entire thread has been about trying to make the past equalize with not only the present, but also the future:

see this rapid ascent and think "well that 100,000 ppd+ folder completely blows my contribution out of the water - in a month they'll have completely surpassed my 4 years of folding, I might as well switch it off and be done with it". I contend that if the proportion was the same, but the absolute PPD was smaller(and constant on average over time) , there would still be the incentive for smaller folders to contribute, but there wouldn't be the disenchantment.

This is absurd. You can't make your past points hold value in the future.
Say there is a game changing move from Silcon to Graphine and FLOPS increase by 1000x in 3 years. Would you really want those computers to generate the same "average points" as your quad? Why? What would be the point?

What will you use as your metric for an average machine? One core, four core, six core, eight core?
What will you use as your metric for "doubling in power over time" ? Single precision FLOPS? Double precision FLOPS? Memory? Total system or just CPU performance? Limited by memory and IO bandwidth or just theoretical CPU performance?

The list goes on and on and on. Hence, you are making hundreds if not thousands of arbitrary decisions about valuing compute capability....rather than just letting the compute capability demonstrate itself and let the points follow.

I wish you luck in administrating any such system.

[Jesse_V]

@mdk777, couldn't this metric be PPD? Because that takes all of those factors into account. The PG could then change how many points are awarded depending on how F@h's overall average PPD changes from one interval to the next. Unless k1wi has a better idea...

[k1wi]

This entire thread has been about trying to make the past equalize with not only the present, but also the future.
This is absurd.

What will you use as your metric for an average machine? One core, four core, six core, eight core?
What will you use as your metric for "doubling in power over time" ? Single precision FLOPS? Double precision FLOPS? Memory? Total system or just CPU performance? Limited by memory and IO bandwidth or just theoretical CPU performance?

The list goes on and on and on. Hence, you are making hundreds if not thousands of arbitrary decisions about valuing compute capability....rather than just letting the compute capability demonstrate itself and let the points follow.

I wish you luck in administrating any such system.

I highly doubt it's going to be hundreds if not thousands of arbitrary decisions about valuing compute capability. If it was decided that a quarterly adjustment of the Y value was ideal (and in my mind, having done a lot of thinking I think it is a good balance between infrequent but still small adjustments) then it would be a decision made 4 times a year.

People keep asking me how I'd determine it, which is fair enough, but bear in mind I don't have access to all the statistics on FAH that PG has - therefore I suggest they are the best placed to design a system. I certainly don't think it needs to be as intricate as taking into account SP-FLOPS or DP-Flops or Memory. Why would it? What is important, as Jesse_V alludes to, is FAH performance. The most simple method would be to use PPD and increases in PPD (while accommodating additional users into the system so the same amount of PPD is not spread over additional folders, as this wouldn't be logical) as a starting point. But they could also look at the spread of PPD across clients and changes to that spread in order to ensure that things remain equatible.

If most countries in the world can determine a CPI then I don't see why FAH cannot determine the relative rate of computational improvement.

Furthermore, I would suggest that the accuracy of the 'technological improvement' does not need to be particularly accurate as it could still be very effective (and an improvement over the current system) even with only a general approximation as these general approximations would still be affecting only very small absolute changes. After all, what is a 4% decrease on 1,000 ppd over 3 months rather than a 4.5% decrease? Not a huge amount. I would suggest that normal fluctuations in point generation would be greater. Certainly looking at my point production that would be the case. Perhaps if it was slightly higher one quarter then the next might be a little lower and over time it'd even out but we'd still be in the same absolute ball park range of PPD. That said, I do think that PG should attempt to be as accurate as possible.

[ChasR]

The relative performance of past, present and future machines is immutable. All you can do is change the scale by which you measure it. If you eliminate the exponential growth in future machines, you will simply turn the "hockey stick" 180 degrees and introduce points deflation equal to the inflation you seek to cure in the past and present machines. THat gains you nothing. It inflates the relative value of your past contributions at the expense of your future contributions. Instead of the "carrot" of increased ppd with an hardware upgrade the project would use the "stick" of decreased ppd for failure to upgrade. I for one believe the "carrot" approach is more popular. Eliminate the other sources of points inflation and let technology take ppd where it will.