Bigadv points change

Moderators: Site Moderators, FAHC Science Team

VijayPande
Pande Group Member
Posts: 2058
Joined: Fri Nov 30, 2007 6:25 am
Location: Stanford

Re: point system is getting ridiculous...

Post by VijayPande »

It's clear that any change to the points system is controversial, but this one has been raised as a very serious problem by many donors. Over the last few weeks, we've been consulting with several donors on this change and based on their feedback, they have strongly suggested that we make a change soon and do so quickly, like "removing a bandaid" rather than drag out the pain. I'm sorry to have to make any changes at all, since any change is disruptive, but we (and many donors) felt very strongly that this change was very important.
Prof. Vijay Pande, PhD
Departments of Chemistry, Structural Biology, and Computer Science
Chair, Biophysics
Director, Folding@home Distributed Computing Project
Stanford University
VijayPande
Pande Group Member
Posts: 2058
Joined: Fri Nov 30, 2007 6:25 am
Location: Stanford

Re: Bigadv points change

Post by VijayPande »

This one was a tough call. After much donor concern on this issue, we did our own research into the matter which showed that bigadv was out of balance. However, we know that any sort of points change will be unpopular. This puts us in a very tough position. If we do nothing, we're ignoring the majority of donors (i.e. "Pande Group doesn't care.") and going against our own data, in favor of the high end FAH donors (and some have called FAH "elitist" because of this). If we make the change that's right based on our own numbers (i.e. the change we've just made), we get the bigadv donors naturally upset, which obviously is very bad too. No way to win here.

In these tough situations, the best thing to do from my point of view as Director of FAH is for us to make the call that's best for the project as a whole, take the hits from those who will be unhappy, and move on and try to do the best we can from here. I can't guarantee everything we do is right, but I can guarantee that we take donor input seriously, take the time to investigate major issues to donors, and aren't afraid to make unpopular decisions to do what's right.

PS One question was why not give more notice. Here, our numbers were showing such high PPD (in some cases 500,000), that this was making a huge disparity for donors points. I did not want to give this another week to get worse. Also, in cases where there are points changes, we do not generally give notice. Please note that we will continue to give notice if a project type will be discontinued.
Prof. Vijay Pande, PhD
Departments of Chemistry, Structural Biology, and Computer Science
Chair, Biophysics
Director, Folding@home Distributed Computing Project
Stanford University
Jester
Posts: 102
Joined: Sun Mar 30, 2008 1:03 pm

Re: Bigadv points change

Post by Jester »

I too am sorry it came to this, but the 500,000 point work units you described were at the root of the vast majority of recent complaints,
instead of looking at these in isolation or in closed beta and attending to any issues they could cause they were released too early into
open beta, and as large a work unit as they are it seems there's no release time if at all for any Win based version as that client is still 32bit,
so we have the current devaluation of win based work units that so many have returned for so long having a new Linux based big point work
unit as the catalyst, and no opportunity to cut any losses by running the very same type of work unit,
I know personally of at least 20 sytems that have been upgraded with Hexcore Cpu's in the last few months, not to mention my 6 month old
SR-2 system, all with the sole justification for that sort of expenditure being our beloved project, if that passion for the science leads
to going the extra mile in hardware and power costs over and above the norm of what others are willing or able to do, and that just brands
any who do as "FAH elite", or "Tall Poppies" that have to be torn down, the project as a whole has a far more serious issue than a recent points
disparity,
It will always be the few who can, who will be the vanguard of any new project that has steep hardware requirements, and or needs something done
more quickly, so don't be too disheartened the next time the trumpet is sounded and fewer answer the call,
I've poured heart and soul into the project for close to ten years now, encouraged many good friends to either join or increase their participation,
and now have to go and make my own apology for something that was both unforeseen and none of my doing.
FlipBack
Posts: 29
Joined: Sat Aug 29, 2009 11:44 pm

Re: Bigadv points change

Post by FlipBack »

VijayPande wrote:This one was a tough call. After much donor concern on this issue, we did our own research into the matter which showed that bigadv was out of balance. However, we know that any sort of points change will be unpopular. This puts us in a very tough position. If we do nothing, we're ignoring the majority of donors (i.e. "Pande Group doesn't care.") and going against our own data, in favor of the high end FAH donors (and some have called FAH "elitist" because of this). If we make the change that's right based on our own numbers (i.e. the change we've just made), we get the bigadv donors naturally upset, which obviously is very bad too. No way to win here.
I read this as meaning that bigadv WUs were previously getting more points than they deserved based upon how much 'science' they were doing. If this is a correct interpretation, then I fully support the change (especially when we have always known that -bigadv and the QRB have been experimental from the start and that we were warned they are subject to change).
Jester
Posts: 102
Joined: Sun Mar 30, 2008 1:03 pm

Re: Bigadv points change

Post by Jester »

FlipBack wrote:
VijayPande wrote:This one was a tough call. After much donor concern on this issue, we did our own research into the matter which showed that bigadv was out of balance. However, we know that any sort of points change will be unpopular. This puts us in a very tough position. If we do nothing, we're ignoring the majority of donors (i.e. "Pande Group doesn't care.") and going against our own data, in favor of the high end FAH donors (and some have called FAH "elitist" because of this). If we make the change that's right based on our own numbers (i.e. the change we've just made), we get the bigadv donors naturally upset, which obviously is very bad too. No way to win here.
I read this as meaning that bigadv WUs were previously getting more points than they deserved based upon how much 'science' they were doing. If this is a correct interpretation, then I fully support the change (especially when we have always known that -bigadv and the QRB have been experimental from the start and that we were warned they are subject to change).
Is that "we" an indication that you too are a Bigadv folder ?
FlipBack
Posts: 29
Joined: Sat Aug 29, 2009 11:44 pm

Re: Bigadv points change

Post by FlipBack »

Jester wrote:
FlipBack wrote:
VijayPande wrote:This one was a tough call. After much donor concern on this issue, we did our own research into the matter which showed that bigadv was out of balance. However, we know that any sort of points change will be unpopular. This puts us in a very tough position. If we do nothing, we're ignoring the majority of donors (i.e. "Pande Group doesn't care.") and going against our own data, in favor of the high end FAH donors (and some have called FAH "elitist" because of this). If we make the change that's right based on our own numbers (i.e. the change we've just made), we get the bigadv donors naturally upset, which obviously is very bad too. No way to win here.
I read this as meaning that bigadv WUs were previously getting more points than they deserved based upon how much 'science' they were doing. If this is a correct interpretation, then I fully support the change (especially when we have always known that -bigadv and the QRB have been experimental from the start and that we were warned they are subject to change).
Is that "we" an indication that you too are a Bigadv folder ?
No, I intended it to mean we as folders in general. I only have 1 quad core and 1 GPU folding, not big guns by any means. My apologies for making it appear otherwise.

However, I don't want change just because Joe Shmo is making more points than me. I have no problem with people making 500,000 PPD compared to my 8,000 PPD as long as it accurately represents the 'science' being done. I believe (and I think just about everyone would agree) that the points system should represent the 'science' being accomplished as accurately as possible. If that is the reasoning behind this change, then I would support it regardless of whether or not it will give me less points.
Jester
Posts: 102
Joined: Sun Mar 30, 2008 1:03 pm

Re: Bigadv points change

Post by Jester »

No problem, :ewink:
Big Gun's ? if that's in ppd then it doesn't really matter to me, if you are doing as much as you are able that makes you a "Big Gun" in my book.....
I agree with the science for science parity between different projects, but when a new project like Bigadv is offered, with both steeper hardware requirements and fast return times,
"those who can" take up the challenge, and can spend lots of extra dollars in doing so, despite the usual "beta project warnings", and then suddenly the hardware and quick return times are seen as somehow
less important than they were at release, which was long before the release of the infamous 500k ppd Wu's, so you might see how "those few" might be a little upset at the current changes,
If it were done because the quick return of Bigadv Wu's are now of less importance to the science I'm sure most would live with that, but to do it because of a points disparity that was there from the start
due to the bonus for hardware and return times what suddenly changed ? only the recent release of some Linux only Bigger adv Wu's,
This is all soley my opinion of course, but I hope it gives a little more insight into "those who can", who despite having seen similar events in the past continue to take up the challenge that the science offers.
Jester
Posts: 102
Joined: Sun Mar 30, 2008 1:03 pm

Re: Bigadv points change

Post by Jester »

Sadly it appears that "head down, bum up 24/7 Folding" doesn't seem as important as having a regular presence here on the support forums,
I have been as guilty as anyone on that count by "having faith" that sanity will always prevail and just "getting on with it",
but I fear the only way to counter the influence of the "forum trolls and whiners" is for all dedicated Folders from every project to make a more active contribution
on these support forums and make their voices heard as loud as is needed.
FlipBack
Posts: 29
Joined: Sat Aug 29, 2009 11:44 pm

Re: Bigadv points change

Post by FlipBack »

Jester wrote:No problem, :ewink:
Big Gun's ? if that's in ppd then it doesn't really matter to me, if you are doing as much as you are able that makes you a "Big Gun" in my book.....
I agree with the science for science parity between different projects, but when a new project like Bigadv is offered, with both steeper hardware requirements and fast return times,
"those who can" take up the challenge, and can spend lots of extra dollars in doing so, despite the usual "beta project warnings", and then suddenly the hardware and quick return times are seen as somehow
less important than they were at release, which was long before the release of the infamous 500k ppd Wu's, so you might see how "those few" might be a little upset at the current changes,
If it were done because the quick return of Bigadv Wu's are now of less importance to the science I'm sure most would live with that, but to do it because of a points disparity that was there from the start
due to the bonus for hardware and return times what suddenly changed ? only the recent release of some Linux only Bigger adv Wu's,
This is all soley my opinion of course, but I hope it gives a little more insight into "those who can", who despite having seen similar events in the past continue to take up the challenge that the science offers.
I understand that it is disappointing for those affected. It is certainly unfortunate that it wasn't corrected sooner, however, I still think that if quick returns were in fact being overvalued, it is a necessary change. And again, I think a key point is that it has been subject to change since the beginning.

At the end of the day, if it is a step forward to making the points more accurately reflect the science, it is a good one. Hopefully PG will learn from this and will spend more time making sure the points for new projects accurately reflect their value before and immediately after releasing them.
mdk777
Posts: 480
Joined: Fri Dec 21, 2007 4:12 am

Re: Bigadv points change

Post by mdk777 »

we did our own research into the matter which showed that bigadv was out of balance. However, we know that any sort of points change will be unpopular. This puts us in a very tough position. If we do nothing, we're ignoring the majority of donors (i.e. "Pande Group doesn't care.") and going against our own data, in favor of the high end FAH donors (and some have called FAH "elitist" because of this). If we make the change that's right based on our own numbers (i.e. the change we've just made), we get the bigadv donors naturally upset, which obviously is very bad too. No way to win here.
Well here is the irony; for two years I and several other vocal donors expressed our concerns that allowing 4 core machines to run these units didn't make sense.

For two years we were told that it did, that we just didn't understand.

How after two years have the numbers changed ?

Two years is way too long of a time to claim that a trial program is just being adjusted.

One month, maybe six is a period of time when you review and adjust trial programs, not after two years.

Hence the understandable and predictable consternation of donors who have made investments.

It is a sad mess. but that it could have been easily prevented makes it worse.
Transparency and Accountability, the necessary foundation of any great endeavor!
Jester
Posts: 102
Joined: Sun Mar 30, 2008 1:03 pm

Re: Bigadv points change

Post by Jester »

FlipBack wrote:
Jester wrote:No problem, :ewink:
Big Gun's ? if that's in ppd then it doesn't really matter to me, if you are doing as much as you are able that makes you a "Big Gun" in my book.....
I agree with the science for science parity between different projects, but when a new project like Bigadv is offered, with both steeper hardware requirements and fast return times,
"those who can" take up the challenge, and can spend lots of extra dollars in doing so, despite the usual "beta project warnings", and then suddenly the hardware and quick return times are seen as somehow
less important than they were at release, which was long before the release of the infamous 500k ppd Wu's, so you might see how "those few" might be a little upset at the current changes,
If it were done because the quick return of Bigadv Wu's are now of less importance to the science I'm sure most would live with that, but to do it because of a points disparity that was there from the start
due to the bonus for hardware and return times what suddenly changed ? only the recent release of some Linux only Bigger adv Wu's,
This is all soley my opinion of course, but I hope it gives a little more insight into "those who can", who despite having seen similar events in the past continue to take up the challenge that the science offers.
I understand that it is disappointing for those affected. It is certainly unfortunate that it wasn't corrected sooner, however, I still think that if quick returns were in fact being overvalued, it is a necessary change. And again, I think a key point is that it has been subject to change since the beginning.

At the end of the day, if it is a step forward to making the points more accurately reflect the science, it is a good one. Hopefully PG will learn from this and will spend more time making sure the points for new projects accurately reflect their value before and immediately after releasing them.
I couldn't agree more, if that had been done with the new "500k ppd Wu's" I'm sure this and other similar threads would simply not exist at all.
Napoleon
Posts: 887
Joined: Wed May 26, 2010 2:31 pm
Hardware configuration: Atom330 (overclocked):
Windows 7 Ultimate 64bit
Intel Atom330 dualcore (4 HyperThreads)
NVidia GT430, core_15 work
2x2GB Kingston KVR1333D3N9K2/4G 1333MHz memory kit
Asus AT3IONT-I Deluxe motherboard
Location: Finland

Re: point system is getting ridiculous...

Post by Napoleon »

I'm late in this game, it seems. However, I would like to put forth a simple pseudocode proposal. Feel free to debug - I'm not much of a programmer. My previous post in the 18th page should provide some graphic clues as to how P & T values adjust points per WU. My point is to dodge the divide-by-zero problem in the WU_points formula...

Code: Select all

// original formula without max(), assumption is that return happens within preferred deadline
WU_points( base_points, k, deadline_length, elapsed_time )  {
  return( base_points * sqrt(k*deadline_length/elapsed_time) );
}

// P and T - Parameter and Threshold - are fractions (percentages) of deadline_length
// Let P = 0.01 and T = 0.05 (1% and 5%) for example
P = 0.01; T = 0.05;

// Time shift adjustment
points_offset = 
  WU_points( base_points, k, deadline_length, T*deadline_length ) -
  WU_points( base_points, k, deadline_length, T*deadline_length + P*deadline_length );

// "If" statement presents current situation
final_points( base_points, k, deadline_length, elapsed_time, P, T )  {
  if( elapsed_time > T*deadline_length ) {
    return WU_points( base_points, k, deadline_length, elapsed_time );
  }
// else
return points_offset + WU_points(base_points, k, deadline_length, elapsed_time + P*deadline_length);
}
EDIT: Updated the spreadsheets to match the code above better.
Last edited by Napoleon on Sun Jul 03, 2011 6:39 am, edited 2 times in total.
Win7 64bit, FAH v7, OC'd
2C/4T Atom330 3x667MHz - GT430 2x832.5MHz - ION iGPU 3x466.7MHz
NaCl - Core_15 - display
Jester
Posts: 102
Joined: Sun Mar 30, 2008 1:03 pm

Re: point system is getting ridiculous...

Post by Jester »

All too late sadly,
I'd guess a lot of those "alternative formula" posts were not looked into in great detail anyway,
Once a tough choice is made I can't see any avenue for modifications that wouldn't cause more anger on one or both sides,
It'll most likely be "Ostrich mode" now and allow the wounds to heal quickly, which they will...
I can't say the same for my memories though, they'll still be there when the next "call for something extra" goes out. :(
GreyWhiskers
Posts: 660
Joined: Mon Oct 25, 2010 5:57 am
Hardware configuration: a) Main unit
Sandybridge in HAF922 w/200 mm side fan
--i7 2600K@4.2 GHz
--ASUS P8P67 DeluxeB3
--4GB ADATA 1600 RAM
--750W Corsair PS
--2Seagate Hyb 750&500 GB--WD Caviar Black 1TB
--EVGA 660GTX-Ti FTW - Signature 2 GPU@ 1241 Boost
--MSI GTX560Ti @900MHz
--Win7Home64; FAH V7.3.2; 327.23 drivers

b) 2004 HP a475c desktop, 1 core Pent 4 HT@3.2 GHz; Mem 2GB;HDD 160 GB;Zotac GT430PCI@900 MHz
WinXP SP3-32 FAH v7.3.6 301.42 drivers - GPU slot only

c) 2005 Toshiba M45-S551 laptop w/2 GB mem, 160GB HDD;Pent M 740 CPU @ 1.73 GHz
WinXP SP3-32 FAH v7.3.6 [Receiving Core A4 work units]
d) 2011 lappy-15.6"-1920x1080;i7-2860QM,2.5;IC Diamond Thermal Compound;GTX 560M 1,536MB u/c@700;16GB-1333MHz RAM;HDD:500GBHyb w/ 4GB SSD;Win7HomePrem64;320.18 drivers FAH 7.4.2ß
Location: Saratoga, California USA

Re: Bigadv points change

Post by GreyWhiskers »

I've not been on FAH that long - about 1 year now, with about 3 months with my Sandy Bridge i7 2600k that allows me to fold bigadv. A couple of observations.

a. Ironically, after running my i7 2600k CPU at the clock rate set by the system builders at Digital Storm (they had slightly overclocked the 3.4 GHz to 3.9 GHz), I finally decided on Thursday to go into the ASUS UEFI bios for the mobo and up the overclock. I was at 4.6 GHz, then brought it a hair down to 4.5 GHz to bring the ac power consumption, Vcore & temps to what I wanted in the summer -- just at the time this announcement on bigadv came up. This overclock brings me to almost parity running the bigadv - I had a little twinge when my first WU under the overclock was the last I would see under the old points system; I just loaded my first p6900 under the new points structure, and it's folding away at 30,553 ppd per HFM.net estimates. I finish one of the WUs several hours sooner than before (getting a slightly higher QRB to improve the science return), and since I completed faster, I can fold more WUs per week, upping the ppd.

b. Given all that, my own personal feeling is that the points structure is relative - I haven't gotten emotionally attached to the the absolute numbers. Going forward from here, being able to complete a P6901 in 50 hours will still give me more points than someone who takes 72 hours to complete the WU, and fewer points than someone who can complete it in 24 hours. The bigger the iron brought to the party, the higher the points awarded, still. The "hierarchy" of the numbers is still the same - but the absolute numbers are different. :| :|

EDIT: and oh, by the way, I'm still running my two old systems for a few points. An old Toshiba Pent M laptop that gets about 120 ppd very reliably, and an old HP a475c Costco special with Pentium 4/HT single core at 3.2 GHz, that gets about 1,100 ppd with an AGP bus ATI HD 4670 GPU under core 11, and about 100 ppd for a single uniprocessor slot. They are still producing a continuous stream of science, they seem to be interesting in their quirks through running the FAH V7 client, and overall keeping up my personal interest.
Last edited by GreyWhiskers on Sun Jul 03, 2011 4:25 am, edited 1 time in total.
FlipBack
Posts: 29
Joined: Sat Aug 29, 2009 11:44 pm

Re: Bigadv points change

Post by FlipBack »

GreyWhiskers wrote: b. Given all that, my own personal feeling is that the points structure is relative - I haven't gotten emotionally attached to the the absolute numbers. Going forward from here, being able to complete a P6901 in 50 hours will still give me more points that someone who takes 72 hours to complete the WU, and fewer points than someone who can complete it in 24 hours. The bigger the iron brought to the party, the higher the points awarded, still. The "hierarchy" of the numbers is still the same - but the absolute numbers are different. :| :|
Agree 100%. I was going to bring up this point earlier but I didn't get it to make sense. I couldn't have said it better.
Post Reply