Bigadv points change

Moderators: Site Moderators, FAHC Science Team

kiore
Posts: 921
Joined: Fri Jan 16, 2009 5:45 pm
Location: USA

Re: point system is getting ridiculous...

Post by kiore »

I am happy to see the dragsters rewarded (a la Zims analogy) if this is really valuable, those systems cost a lot to build and run, but am concerned that the reward is possibly disproportionate to the science. I do think we need to value contributions appropriately, this rewards performers and gives clues to people purchasing hardware as to where to put the cash.
The points system has been a success especially the QRB for directing donators IMO but risks alienating small producers. I get asked is it 'worth' me running my dual core 2 GHz laptop for the project and as a team leader I say yes..But these people see other team members getting massive points on another system (a 2600k for example) and it appears their contributions are not needed.. what we risk is a distortion of the distributing computing ethic that a horde of small systems can out perform a Kray.. Does the exponential 'payment' really = exponential science done? the scale seems quite extreme, I am fully prepared to explain this to people that some systems get more points because they do more and that speed is helpful, but surely there is a bottleneck where less than a day or less than 12 hours starts to make less difference?
I am not complaining, but I would really like to know and am concerned that smaller producers are feeling redundant. If I am to stick to the line that points = science value would like this as easy as possible to explain.
Image
i7 7800x RTX 3070 OS= win10. AMD 3700x RTX 2080ti OS= win10 .

Team page: https://www.rationalskepticism.org/viewtopic.php?t=616
MtM
Posts: 1579
Joined: Fri Jun 27, 2008 2:20 pm
Hardware configuration: Q6600 - 8gb - p5q deluxe - gtx275 - hd4350 ( not folding ) win7 x64 - smp:4 - gpu slot
E6600 - 4gb - p5wdh deluxe - 9600gt - 9600gso - win7 x64 - smp:2 - 2 gpu slots
E2160 - 2gb - ?? - onboard gpu - win7 x32 - 2 uniprocessor slots
T5450 - 4gb - ?? - 8600M GT 512 ( DDR2 ) - win7 x64 - smp:2 - gpu slot
Location: The Netherlands
Contact:

Re: point system is getting ridiculous...

Post by MtM »

Well one thing to keep in mind is that there are active classic client work unit's being generated, and I'm going to assume Stanford would not do that if it wouldn't benefit from them. So while we might get a small amount of points for running them, the fact that we're 'asked' to run them should already give the donor some sense of being valued :)

Also, point them to the project descriptions, as each projectrange targets a different issue it should also indicate each project has individual worth for science.

Explain how much of an improvement smp is over classic, how much faster it can compute complex proteins. But don't forget to point out that not all research involves those more complex proteins and that each project will be of value to science.

I am repeating the same 'thought' with slightly different wording and explanations, it's the thought that needs conveying so pick any you think is easiest to use?
bruce
Posts: 20824
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: point system is getting ridiculous...

Post by bruce »

By watching the server status, GreyWhiskers points to an issue that I've been wondering about, too.
viewtopic.php?f=16&t=18797&start=90#p188513

The only solution that I see to the QRB issues is to create a variable system of bonuses. That will be unpopular to those who try to track the value of every WU but it's already true with the present QRB so I think folks will be able to accept it, once somebody figures out what needs to be done.

The problem, as I see it, is that we're trying to find one set of equations that define two conflicting types of scientific values. It's impossible unless we understand the problem better and recognize that FAH needs to be optimized differently in those two very clear cases.

1) When there are lots of WUs on a server compared to the number of donors, most of the delay is in the "back end" because every time a WU is completed, a new WU is generated but it sits on the server for a while not being assigned. In those cases, there really is very little scientific value in returning the WUs promptly.

2) When there are very few WUs on a server, as soon as a WU is completed, a new WU is generated and it's immediately assigned to someone. The server never has any WUs waiting to be assigned and all of the delay is in the client. Those are the projects that really need a fast client.

Adjusting the points based just on quick returns has created an incentive for donors to compete for projects with scarce WUs and to ignore projects which have lots of WUs. That providing an incentive for bad behavior. It rapidly produces long trajectories but does really bad things for donor attitudes.

When a server is out of WUs, the bonus needs to be REDUCED until donors stop fighting for those scarce WUs. In fact, that's directly opposite to encouraging the rapid return of scarce trajectories but it also will stop the bad feelings generated due to total PPD getting too high.

One of the factors in the QRB formula must be the Number of WUs Available vs. Number of WUs Demanded (i.e.- including some method of estimating how many donors have machines that COULD be running this project but are unable to get work).

Proper tuning of the system should somehow avoid both extremes dynamically and encourage the system to run smoothly near the optimum condition. Until somebody understands that, there is no possibility of establishing a single set of equations which makes FAH run smoothy.
MtM
Posts: 1579
Joined: Fri Jun 27, 2008 2:20 pm
Hardware configuration: Q6600 - 8gb - p5q deluxe - gtx275 - hd4350 ( not folding ) win7 x64 - smp:4 - gpu slot
E6600 - 4gb - p5wdh deluxe - 9600gt - 9600gso - win7 x64 - smp:2 - 2 gpu slots
E2160 - 2gb - ?? - onboard gpu - win7 x32 - 2 uniprocessor slots
T5450 - 4gb - ?? - 8600M GT 512 ( DDR2 ) - win7 x64 - smp:2 - gpu slot
Location: The Netherlands
Contact:

Re: point system is getting ridiculous...

Post by MtM »

I agree with scientific value being tied to where the bottleneck is, but adjusting scientific value on the fly based on a moving bottleneck in the sense you're advocating seems like
a> really allot of work, and aren't you the one who always said that the work done to something must be weigh up against the gains?
b> the gains would be small imo because you're creating fluctuating ppd for the same projects on the same hardware, when there is no way to choose projects. You choose a client type, not a project.
Leonardo
Posts: 260
Joined: Tue Dec 04, 2007 5:09 am
Hardware configuration: GPU slots on home-built, purpose-built PCs.
Location: Eagle River, Alaska

Re: point system is getting ridiculous...

Post by Leonardo »

Who cares about the points? It's all about the science and health breakthroughs. If you're folding solely for the points, enjoy having a big e-penis.
Philosophically, I doubt any of would disagree with you. Unfortunately, reality is that human nature brings in other elements, such as the desire for competition. Wishful thinking will not expunge that. There will always be a certain percentage of users participating purely from a sense of altruism, but I think the F@H project requires more participants than that.
Image
MtM
Posts: 1579
Joined: Fri Jun 27, 2008 2:20 pm
Hardware configuration: Q6600 - 8gb - p5q deluxe - gtx275 - hd4350 ( not folding ) win7 x64 - smp:4 - gpu slot
E6600 - 4gb - p5wdh deluxe - 9600gt - 9600gso - win7 x64 - smp:2 - 2 gpu slots
E2160 - 2gb - ?? - onboard gpu - win7 x32 - 2 uniprocessor slots
T5450 - 4gb - ?? - 8600M GT 512 ( DDR2 ) - win7 x64 - smp:2 - gpu slot
Location: The Netherlands
Contact:

Re: point system is getting ridiculous...

Post by MtM »

Leonardo wrote:
Who cares about the points? It's all about the science and health breakthroughs. If you're folding solely for the points, enjoy having a big e-penis.
Philosophically, I doubt any of would disagree with you. Unfortunately, reality is that human nature brings in other elements, such as the desire for competition. Wishful thinking will not expunge that. There will always be a certain percentage of users participating purely from a sense of altruism, but I think the F@H project requires more participants than that.
I'm afraid there is a well known indication supporting your point, even while I don't like the thought :(

If altruism was the main drive, we wouldn't have needed the QRB to ensure people didn't run dual smp's ect. That's an argument which can not be denied.

But, I think I can say with 100% accuracy that AtlasFolder never ran clients optimized for points, since it's well known as to why he is involved in this project. On the other hand I know others who ran dual smp and preached it their team's, I'm glad to see some of those people have left the f@h project behind now but many still remain, and i'm sure those people would abuse a point system if it meant less science but more points if possible in the future :(
mdk777
Posts: 480
Joined: Fri Dec 21, 2007 4:12 am

Re: point system is getting ridiculous...

Post by mdk777 »

I think almost everyone agrees that some sort of point system has to exist.
The positives far outweigh the negatives.
1. A method of direction to point people to where their contribution is the most effective.
2. A method to quantify the success.

Bill Gates has pledged to give his entire fortune to philanthropic endeavors.
However, he is not doing it blindly.
His full time job (along with his wife), is directed toward judging the effectiveness of any endeavor, and quantifying the return on the expenditure.
That is just good stewardship of resources.

Questioning the good intention, or lack of good intention of donors is pointless. :wink:
1. They are donors :!:
2. They are human.

The question under discussion is(As defined by VJ earlier in the thread):
What is the most appropriate equation to describe the benefit to PG of faster return of WU? :mrgreen:
Transparency and Accountability, the necessary foundation of any great endeavor!
JimF
Posts: 651
Joined: Thu Jan 21, 2010 2:03 pm

Re: point system is getting ridiculous...

Post by JimF »

Another value of the point system to users is to inform them when they should retire or upgrade their systems. The newer systems are more energy-efficient, and at some point it makes sense to let someone else do the crunching. The points criteria should be clearly stated and hopefully easy to understand, but beyond that it should keep PG happy, not me. I can then decide how to best use my resources. No economic system can work efficiently without the right price structure providing accurate feedback.
Amaruk
Posts: 254
Joined: Fri Jun 20, 2008 3:57 am
Location: Watching from the Woods

Re: point system is getting ridiculous...

Post by Amaruk »

ChasR wrote:multiplier = 1+IF(TO<(PDL-1),(PDL-1-TO)*K,0)
How do you calculate the value of K?
ChasR wrote:PDL = preferred Deadline (in this formula you have to beat the preferred deadline by one day to earn any bonus. Personally I think you should have to beat it by more than that to earn a bonus. For Uniprocessor WUs, you would subtract far more than 1.)
How would this be determined for various deadlines? That is, if (PDL-y-TO), then what determines the relationship between PDL and y?
Image
VijayPande
Pande Group Member
Posts: 2058
Joined: Fri Nov 30, 2007 6:25 am
Location: Stanford

Re: point system is getting ridiculous...

Post by VijayPande »

7im wrote:Same type of thinking here... it's very difficult to put specific numbers on this discussion because we don't know the scientific value of time.
We set up the QRB system with a reasonable plan for how the science connects to time. To first order, that's a good estimate of the value of science vs time.

However, what I've suggested above is that you suggest what you think would be accepted from a donor point of view and we can compare plans and see what sort of compromise we can make. It looks like nobody wants to put something specific down, perhaps due to issues 7im raises above, but without any concrete plans beyond what we have now, we're stuck.

So, please just suggest *something* and we can discuss from there. Here's one suggestion to get the ball rolling: take our current system and suggest changes in the k-factor, PPD, benchmark machine, timeout formulae, etc.
Prof. Vijay Pande, PhD
Departments of Chemistry, Structural Biology, and Computer Science
Chair, Biophysics
Director, Folding@home Distributed Computing Project
Stanford University
MtM
Posts: 1579
Joined: Fri Jun 27, 2008 2:20 pm
Hardware configuration: Q6600 - 8gb - p5q deluxe - gtx275 - hd4350 ( not folding ) win7 x64 - smp:4 - gpu slot
E6600 - 4gb - p5wdh deluxe - 9600gt - 9600gso - win7 x64 - smp:2 - 2 gpu slots
E2160 - 2gb - ?? - onboard gpu - win7 x32 - 2 uniprocessor slots
T5450 - 4gb - ?? - 8600M GT 512 ( DDR2 ) - win7 x64 - smp:2 - gpu slot
Location: The Netherlands
Contact:

Re: point system is getting ridiculous...

Post by MtM »

Isn't it a good idea to first start a poll and ask people if they object to the current system if it according to you is in line with the scientific value. If it was chosen for that reason, I think allot of people won't object. If you're really in it for the points, you can invest abit more and science benefits proportionally from your point increase. That's not something I have a problem with personally, it's fair it it's proportional to the science and I have to assume the original formula is what comes closest to reflecting that or is wouldn't have been chosen?
Grandpa_01
Posts: 1122
Joined: Wed Mar 04, 2009 7:36 am
Hardware configuration: 3 - Supermicro H8QGi-F AMD MC 6174=144 cores 2.5Ghz, 96GB G.Skill DDR3 1333Mhz Ubuntu 10.10
2 - Asus P6X58D-E i7 980X 4.4Ghz 6GB DDR3 2000 A-Data 64GB SSD Ubuntu 10.10
1 - Asus Rampage Gene III 17 970 4.3Ghz DDR3 2000 2-500GB Segate 7200.11 0-Raid Ubuntu 10.10
1 - Asus G73JH Laptop i7 740QM 1.86Ghz ATI 5870M

Re: point system is getting ridiculous...

Post by Grandpa_01 »

Vijay I agree with MyM here put a poll up with an explanation. The one thing that needs to be addressed with the current point system is if the points are out of line too much compared to other WU's off the same class they should they be adjusted to produce a closer PPD 6700, 6701, 2684 etc. These types of point variances were causing some problems with some people dumping the lesser valued WU's and some people were having more than there fair share of them. On the other end of the spectrum is the 6903 with its high value and 12 core requirement I am guessing that these are pretty important to you. But there is such a long deadline on them that people with 2600k and Phenom II X6 can run them and meat the deadline and make more points ruining them than the can regular bigadv so how many people are going to start running them with the machines that they were not really designed for and vastly slowing down the project. (They are already being run on them) On something like that maybe you could shorten the deadline and change the k factor to reflect what you want. The reason I said they might be a little high in value to begin with is because I figured there would be a public out cry in the beginning because of the rather large spread in points. Do not forget to figure in the Human factor quite a few people want to be able to compete even if they do not have the equipment or means to compete.
Image
2 - SM H8QGi-F AMD 6xxx=112 cores @ 3.2 & 3.9Ghz
5 - SM X9QRI-f+ Intel 4650 = 320 cores @ 3.15Ghz
2 - I7 980X 4.4Ghz 2-GTX680
1 - 2700k 4.4Ghz GTX680
Total = 464 cores folding
7im
Posts: 10179
Joined: Thu Nov 29, 2007 4:30 pm
Hardware configuration: Intel i7-4770K @ 4.5 GHz, 16 GB DDR3-2133 Corsair Vengence (black/red), EVGA GTX 760 @ 1200 MHz, on an Asus Maximus VI Hero MB (black/red), in a blacked out Antec P280 Tower, with a Xigmatek Night Hawk (black) HSF, Seasonic 760w Platinum (black case, sleeves, wires), 4 SilenX 120mm Case fans with silicon fan gaskets and silicon mounts (all black), a 512GB Samsung SSD (black), and a 2TB Black Western Digital HD (silver/black).
Location: Arizona
Contact:

Re: point system is getting ridiculous...

Post by 7im »

With a topic title like this, and almost 9 pages of comment, I think poll is a little redonkulous at this point. :roll:

An no offense to anyone, but polls, especially forum polls, are unrepresentative of the whole, and never cover all the possible responses to as to affect the outcome, so I think they're lame.

If you want to poll, make it a simple yes/no question and answer, otherwise we'll need another 8 pages to debate the results of the poll.
How to provide enough information to get helpful support
Tell me and I forget. Teach me and I remember. Involve me and I learn.
Grandpa_01
Posts: 1122
Joined: Wed Mar 04, 2009 7:36 am
Hardware configuration: 3 - Supermicro H8QGi-F AMD MC 6174=144 cores 2.5Ghz, 96GB G.Skill DDR3 1333Mhz Ubuntu 10.10
2 - Asus P6X58D-E i7 980X 4.4Ghz 6GB DDR3 2000 A-Data 64GB SSD Ubuntu 10.10
1 - Asus Rampage Gene III 17 970 4.3Ghz DDR3 2000 2-500GB Segate 7200.11 0-Raid Ubuntu 10.10
1 - Asus G73JH Laptop i7 740QM 1.86Ghz ATI 5870M

Re: point system is getting ridiculous...

Post by Grandpa_01 »

Then what do you suggest like you said 8 pages and not a single point system like was asked for. Simple pole keep the current point system yes / no Really the only place the point system is being discussed is here. I have not really seen it on any of the other forums I visit.
Image
2 - SM H8QGi-F AMD 6xxx=112 cores @ 3.2 & 3.9Ghz
5 - SM X9QRI-f+ Intel 4650 = 320 cores @ 3.15Ghz
2 - I7 980X 4.4Ghz 2-GTX680
1 - 2700k 4.4Ghz GTX680
Total = 464 cores folding
7im
Posts: 10179
Joined: Thu Nov 29, 2007 4:30 pm
Hardware configuration: Intel i7-4770K @ 4.5 GHz, 16 GB DDR3-2133 Corsair Vengence (black/red), EVGA GTX 760 @ 1200 MHz, on an Asus Maximus VI Hero MB (black/red), in a blacked out Antec P280 Tower, with a Xigmatek Night Hawk (black) HSF, Seasonic 760w Platinum (black case, sleeves, wires), 4 SilenX 120mm Case fans with silicon fan gaskets and silicon mounts (all black), a 512GB Samsung SSD (black), and a 2TB Black Western Digital HD (silver/black).
Location: Arizona
Contact:

Re: point system is getting ridiculous...

Post by 7im »

I suggest someone quant like step up and put some numbers out there on a slightly less steep curve. I'm too busy and brain drained to break open and relearn my high school adv math class chapters on function graphing and curve fitting. Heck, there's probably an app for that now. ;)

That's what Vijay suggested, put some numbers out there, not collect poll answers. :twisted:

And if the topic stagnates a while for the lack of a quant, I'll come back in a bit and break out the math books and my HP-28c. :lol:
How to provide enough information to get helpful support
Tell me and I forget. Teach me and I remember. Involve me and I learn.
Post Reply