Prove it

[Napoleon]

Inspired by this topic: viewtopic.php?f=16&t=18797

Here comes my challenge:

1. Must have -bigadv capable setup. 4, (6), 8, 16, 32 cores et cetera. Real cores? Surreal cores? - Doesn't really matter
2. Install the latest beta v7 client
3. Set up x amount of uniprocessor slots to match your former bigadv core count
4. Report any and all perceived differences - if any. PPD versus FLOPS versus ScientificContribution(TM) and so forth

Let's discuss FAH scientific value vs points once again, within aforementioned context (bullets 1-4). I'm holding my breath while 24/7 bigadv folders ponder/laugh at my challenge. I'd expect this to be a fairly quiet thread, as far as bigadv user experience goes.

[orion]

A question about #3. Do you want the smp X number or the actual core/thread count of the cpu?

I ask because to get the x6 to run bigadv in v7 one must use the smp 8 flag. By doing this it slows the TPF down my several minutes, enough to take it over three days to finish making it not worth the effort IMHO.

So if v7 would even allow to extra (not there) slots to be added and used it would slow the uniprocessor down even more.

[mdk777]

If anyone where to take up your "challenge", they would just prove the existence of the QRB.

I don't think its very existence was ever in any doubt.

[Amaruk]

If anyone where to take up your "challenge", they would just prove the existence of the QRB.

I don't think so. I believe Napoleon is looking for factual, statistical data on how QRB affects the relationship between clients.

I'll bite. But first a few thoughts.

Here comes my challenge:

1. Must have -bigadv capable setup. 4, (6), 8, 16, 32 cores et cetera. Real cores? Surreal cores? - Doesn't really matter
2. Install the latest beta v7 client
3. Set up x amount of uniprocessor slots to match your former bigadv core count
4. Report any and all perceived differences - if any. PPD versus FLOPS versus ScientificContribution(TM) and so forth

1) Check. And check. and both with surreal cores...

2) Why v7? The cores do the work, not the client. I used 6.34+HFM for testing because v7 doesn't calculate the bonus. Monitoring software for real-time tracking of PPD (FahMon, HFM, ect) doesn't work with v7 either.

3) Several thoughts about this.

3a) Hyperthreaded processors (read: virtually all Intel-based -bigadv folders) cannot do so if they follow the recommended practice of 1 classic client per physical core. SOURCE

For other machines, one can use multiple processors by running multiple clients (one for each real CPU core).

3b) From my experience running 48 classic clients on a 4X 6172 rig would be a waste of resources since all you really need is the average PPD per client X 48 for total PPD. I've run 1, 4, and 8 clients and the variation between them has been minimal. Not to mention one of the 'big dogs' would likely be more amenable to using -smp 40 (w/affinity locked if needed) while running 8 classic clients to get an average TPF for statistical comparison.

3c) The classic WUs need to qualify for QRB.

3d) SMP results would be good too.

4) Not to sound like a broken record but, Flops != Science.

My sole interest (and likely many others) is in PPD, since that is the only measure of scientific value Stanford provides.

Which brings me to comparisons between clients. With the recent release of classic projects that qualify for QRB, I've been able to test this.

So, without further ado...


First machine is a 930 @ 3.8 GHz (4C8T):


UNI 10720 A4 00:27:34 - 4,414.52 PPD (1103.63 X4)

UNI 10721 A4 00:17:45 - 4,637.28 PPD (1159.32 X4)

~~~

SMP 6052 A3 00:03:10 - 16,556.58 PPD

SMP 6067 A3 00:03:17 - 15,682.01 PPD

SMP 7137 A3 00:03:17 - 19,313.38 PPD

~~~

bigadv 2685 A5 00:35:50 - 28,711.51 PPD

bigadv 6900 A5 00:35:55 - 29,611.65 PPD

bigadv 6901 A5 00:36:36 - 27,814.12 PPD


Averaging out the PPD by client type:

Uni @4,525.90 PPD vs SMP @17,183.99 PPD = 3.7968:1

SMP @17,183.99 PPD vs bigadv @ 28,712.43 PPD = 1.6709:1



Second machine is dual 5620 @3.8 GHz (8C16T)

UNI clients same as 930, so 1,131.48 PPD/client X 8 = 9051.84 PPD

~~~

SMP 6052 A3 00:01:36 - 46,099.28 PPD

SMP 6067 A3 00:01:37 - 45,388.24 PPD

SMP 7136 A3 00:01:36 - 56,774.18 PPD

~~~

bigadv 2685 A5 00:18:30 - 77,397.93 PPD

bigadv 6900 A5 00:17:48 - 82,008.13 PPD

bigadv 6901 A5 00:17:49 - 81,893.09 PPD


Averaging out the PPD by client type:

Uni @9,051.84 PPD vs SMP @ 49,420.57 PPD = 5.4597:1

SMP @ 49,420.57PPD vs bigadv @ 80,433.05 PPD = 1.6275:1

[Napoleon]

@Amaruk: thank you for taking the bait and providing some solid figures
@mdk777: yes, I believe QRB exists. If you read between the lines, the real question is whether or not it should exist in its current form

<RAMBLE>
I freely admit I was rather unspecific what I meant with my "challenge", and was really surprised to see even 1 person to really try it out, just for the fun of it. This led me to thinking, bonus points should show some personality... since I'm a fan of lists and binary logic, here I go again:

1. Some Mad Scientists (Pande Group membership isn't mandatory but it helps a little, I think) come up with a protein. Whatever a "protein" actually means
2. Said Mad Scientists propose a Project to simulate it first
3. Said Mad Scientists split up a Project into Runs, Clones & Generations. Just for kicks, not because they already have all the computing power they want, solely at their disposal

Eventually - with much coffee involved in the process - a Project gets released to the wilderness after it is clinically tested internally. By actually creating said protein and then digesting it, and then by some well behaved test subjects. The Mad Scientists proceed to waiting. Again, life-dangering consumption of coffee is involved in the process.

Surpise, surprise! Something unusual comes up. Following binary logic, there are two possible outcomes, assuming a perfect world (I am perfect as such, the world is not)

1. WaWaWoom, this is actually interesting. Let's publish a paper on it to appease those constantly nagging people who constantly complain about things like "funding", "resources", "alarming coffee consumption" and so on. And let's not forget people who overuse the word "constantly"
2. Boring. This won't help Mad Scientist Coffee Addiction Syndrome at all, let's just kill it

Now then, what does that mean to people like me (I consume green tea)? Points are fun, everybody likes points. Some even need a daily dose. Behold, a nasty word gets inserted into the play: "Addiction". Actually, there is another word, "Recognition". Combine the two, and you get "Addicted to Recognition", and that gets really nasty sometimes. Or funny, depending on your POV.

So what does all that have to do with the current FAH points system? Considering the "WaWaWoom" list, how about a bonus system that rewards for both? Not daily, but "when it's done"?

1. If outcome is good, a paper gets published. In those papers, it is usually considered good form to mention some people who helped you along, even if you did just about everything yourself
2. If outcome is bad, a paper doesn't get published. It's a dead end. But something a (not so) Mad Scientist can recognise as unworthy - let's not try this again

</RAMBLE>

In both cases, some reward should be given to those who contributed steadily. I don't have a proposal for new and improved QRB formula, I'm just thinking that the PG member or associate responsible for a Project should do a manual recredit of bonus points after a Project gets finished one way or another. Until then, everyone gets only base points. Does that sound reasonable / doable? Points matching scientific value, like?

[MtM]

No it doesn't sound resonable...

One reason: define a project 'ending'. Do you consider it 'ending' when a misfold is seen? Or just when the conclusion would be that this trajectory and adjecent one's aren't leading to anything of interest? Should you only credit people who run the wu's which get an interesting result ( hey let's do it like bitcoin?! ). After all you could argue that they need to get credited for the usefull scientific results more then those who just gave back 'useless' data.

Not fair, not reasonable, not possible without so much work it's not worth the effort to even consider it.

Points already match science, that's why we have the QRB formula handed down to us from PG, I for one trust their ability to come up with a formula which matches their perceived value of work units and I can't stop wondering why others seem to have troubles with that perception.

[mdk777]

Until then, everyone gets only base points. Does that sound reasonable / doable? Points matching scientific value, like?

OK, you know this was a complete hash of mixed comparisons right? Different size WU, different points awarded based on different time frames.

In order to do this right, you would need to start at the large WU and go in reverse.

How many -SMP computers will it take to get the results of one bigadv.?
How many uni computers are required to generate the same results as one -smp?

These are not linear relationships. The extra cache, reduced latencies and memory access capabilities of a 4-core CPU do not translate equally to 4 uni-core CPU.

An interesting and very real "challenge" would be to see how you could solve the bigadv 2865 in under 18 minutes using only single processor machines.
Do you want to hazard a guess? 8, 16, 32, 64 ?

My guess is that it impossible; that your communication latencies would make it impossible, even if you had access to an infinite number of single
CPU machines.

[Grandpa_01]

Points already match science, that's why we have the QRB formula handed down to us from PG, I for one trust their ability to come up with a formula which matches their perceived value of work units and I can't stop wondering why others seem to have troubles with that perception.

I do not think others have trouble with trusting PG's determination of value. They just have trouble with person x receiving more credit than person y the (H) factor. I frankly do not care where my value is as long as it reflects what I am doing. I fold for personal reasons not the points, but the (H) factor in me wants to be rewarded for the $XXXXXX I have put out for the dedicated folding equipment and the $$$$ It cost me monthly / yearly to fold. They can lower the gap between PPD it does not matter, may be it would take care of the (H) factor. But then again there are probably other bigadv folders that believe they are not rewarded enough. I think there is never going to be a satisfactory resolution to the point problem

[Slash_2CPU]

So you think receiving results in 28 days is just as good as receiving results on 40 hours?

It seems a lot of these projects are based on information from previous runs. They are generational. To run 100 generations on a typical -bigadv would take 5-6 months. To run 100 generations on uniprocessor clients would take 7-8 Years. So half a year versus half a career to get the same job done.

As for GPU, they can do computations just as fast as -bigadv systems, but they can only do certain types of models. Given their limitations, PandeGroup understandably does not want people building GPU farms when they could build a couple-few -bigadv systems that are also very fast but with no limits on how they can apply that speed. The entire folding project running on GPU's only would be a very limited project. They need -bigadv to do some of their work.

You are trying to say that a hammer is better than a soldering iron while ignoring the task at hand. It is a facetious argument in a literal sense of the word. In fact, the whole idea of people getting bent over points is very facetious. What are you saving them up for? You can't spend them on anything and they are not transferable. They may as well be monopoly money.

[codysluder]

In fact, the whole idea of people getting bent over points is very facetious. What are you saving them up for? You can't spend them on anything and they are not transferable. They may as well be monopoly money.

By that same argument maybe you'd recommend removing the points system entirely since in your opinion they're not worth anything. Then people wouldn't get bent over points because there wouldn't be any and the forum would be a lot more hospitable. Either they're important enough to fix or they're not important enough to fix. Choose one and state your recommendation.

[Grandpa_01]

In fact, the whole idea of people getting bent over points is very facetious. What are you saving them up for? You can't spend them on anything and they are not transferable. They may as well be monopoly money.

(H) factor = epeen = Man thumps himsel on the chest and say's, I own Boardwalk and Parkplace you own Baltic.

[MtM]

In fact, the whole idea of people getting bent over points is very facetious. What are you saving them up for? You can't spend them on anything and they are not transferable. They may as well be monopoly money.

(H) factor = epeen = Man thumps himsel on the chest and say's, I own Boardwalk and Parkplace you own Baltic.

Yet you laugh, but this is a serious manner :p

You should come up with a way to keep it fun with both the QRB and the (H) factor! I don't get the (H) Factor that well, I do know the fun Grandpa_01 is displaying is what keeps folders going. It's not the points on their own, it's the comparison with others by any means and the companionship which exists between all folders those in the same team in particular.

So, as Vijay asked us: what would keep folding fun? I know I'm taking his words very liberal, I hope he forgives me for it, but let's just assume he would. What would keep this fun? Forget starring at just the points. Forget in this thread for the next 10 posts at least that there is a points discussion going on, which btw is always going on somewhere so what's new here?

For me, fun would be able to compare my donations with people with comparable hardware, that way I can gauge my contribution to science relevant to what I could be doing, and what others with my configuration are doing. That would keep me folding for a long time, if I would get 1p or 1000p per work unit as it's not dependent on their size but their amount.

What is your fun part of folding??

[ChasR]

For me, fun would be returning to the days when a machine 2x as fast as mine got 2x the ppd, rather than 2.8x the ppd.

[Grandpa_01]

In fact, the whole idea of people getting bent over points is very facetious. What are you saving them up for? You can't spend them on anything and they are not transferable. They may as well be monopoly money.

(H) factor = epeen = Man thumps himsel on the chest and say's, I own Boardwalk and Parkplace you own Baltic.

Yet you laugh, but this is a serious manner :p

You should come up with a way to keep it fun with both the QRB and the (H) factor! I don't get the (H) Factor that well, I do know the fun Grandpa_01 is displaying is what keeps folders going. It's not the points on their own, it's the comparison with others by any means and the companionship which exists between all folders those in the same team in particular.

So, as Vijay asked us: what would keep folding fun? I know I'm taking his words very liberal, I hope he forgives me for it, but let's just assume he would. What would keep this fun? Forget starring at just the points. Forget in this thread for the next 10 posts at least that there is a points discussion going on, which btw is always going on somewhere so what's new here?

For me, fun would be able to compare my donations with people with comparable hardware, that way I can gauge my contribution to science relevant to what I could be doing, and what others with my configuration are doing. That would keep me folding for a long time, if I would get 1p or 1000p per work unit as it's not dependent on their size but their amount.

What is your fun part of folding??

Actually I do not fold for the fun of it. I fold for my family and or anybody the folding might help. I would fold for 0 PPD but the point system does add to the enjoyment I get out of folding. The true joy I get comes from the hope that some day there will be a cure for some of these diseases that are being worked on and that I might be a small part of that.

The poking fun at the H factor is just that as long as there are humans involved anywhere there might be the slightest competition the H factor is going to come into play, that is what these discussions about points are all about, the H factor. What is motivating people to say one group of people are getting too many points or another group is not getting enough points.? (Competition)

Now how are you going to satisfy everybody ?, you are not going to. ! 7im is not wrong and neither is MtM or mdk777 or anybody else who posted there point of view about points.

Truthfully the only people who are right here are the people at Stanford who set up the system to begin with and know how much value is added for quick returns. And the system appears to be working. Look at how many people are running bigadv WU's and how many people are buying equipment for just that purpose. I do not know how much that has sped up the science, but if should happen to be 1 day sooner that a cure to a terminal disease or a deadly flu is discovered just how many life's will that save. How many PPD is that worth.

May be everybody should think about that for a little while and factor that in.

To answer you question MtM The hope for a cure gives me more joy and happiness and motivation than anything else. Thank you for asking.

[bruce]

OK. Here's another comment on the "fun" factor.

I'm not particularly competitive, so I don't spend a lot of time comparing my output (or my team's output) with others (or other teams) but I do read what is said and have an appreciation for those who are motivated by competitive factors. I don't have the $$ to buy a 48-core server, nor do I have an employer who will let me use his. that pretty much constrains me to the Little League (plus a few Geezer League machines).

Periodically, the discussions prod me into upgrading my hardware. If I pick up an obsolete (geezer league) machine somewhere and add it to my farm, I feel good about donating more electricity. If I upgrade to a higher output GPU, I feel good about my increased PPD. If I decide to build/buy a Big League machine, I feel good about my increased PPD. In essence, I'm competing with myself, like an athlete going for his personal best, even if he's way back in the pack.

For me, higher PPD is a good thing, but I know enough math that I don't particularly care what formula is used. Any points system is an approximate measurement of value, just like when the Doctor reads out a bunch of numbers from various tests and concludes that "You {are/are_not} in good health." Both are rather crude measurement of how long you have to live.

Any formula is fine with me if it approximates science and if it improves the "fun factor"

I'm reading this topic just to look for potential improvements that would increase the "fun factor" or would encourage all of you to fold more (and, in like manner, for me to be encouraged to upgrade my hardware so I fold more).

Like Grandpa_01, I'm here to advance science and to hasten the development of a cure for a disease my family has encountered or perhaps one I might have in the future (and to watch you guys compete with each other while I'm busy competing with myself).