Comparative scientific productivity between clients

[7up1n3]

I first participated in the Folding@Home project back in 2002, and have watched with interest as new client options continue to be made available (most recently the GPU2 client) and existing clients optimized (eg v4, v5, v6 of the standard Window's client). Flash forward to today, and we now have five major client platforms:

• Windows
• Linux
• MacOS
• PS3
• GPU

With subsets (graphical, text, screensaver) within each category as well the ability to choose SMP clients for multi-core / multi-proc platforms.

With all the different options available, with the options in different developmental stages (eg Windows SMP remains in rather rough beta stage after several years), I think it would be helpful to know how the clients measure up against each other in regard to their *scientific value*. While I'm assuming there isn't a lot of difference between the different OS platforms in the regard, it would make sense to me that there would be some variance between:

• CPU specific standard vs SMP client productivity
• CPU vs GPU vs PS3 client productivity
• ATi vs Nvidia productivity

Why helpful? Because, while points and competition is pretty fun, the scientific value of our participation is (or can be with the right information) a key factor to consider when one is in a purchasing mode. And it may very well be that points aren't a very good indicator of the actual scientific value contributed across the different platforms. Armed with this information, users would be able to consider more than just the perceived point productivity when looking at adding new hardware to their 'Fold', perhaps focusing instead on the upgrade's scientific value.

[Ivoshiee]

Currently running and completed projects per client type is one benchmark as well.

[gwildperson]

Good questions. I'd like to see the answers, too.

It's too soon to know the answer to at least one of your questions: ATi vs Nvidia productivity.

The ATi client has been running long enough to have produced some genuine scientific results, but the Nvidia client is still running 100% test assignments so although they're useful to debug the software, they have produced zero new scientific results. It's reasonable to assume that the scientific value of the results will be equivalent for hardware that runs the same speed since the science code in FahCore_11 is identical, but that's still an assumption.

That brings up another question. How should the answers manage hardware differences? You didn't ask about the scientific differences between Intel vs. AMD CPUs. That's reasonable because they run identical software so the only differences are in the features of the specific hardware you're comparing. Presumably the same answer will apply to ATI vs Nvidia so it's not a fair question.

BTW, there are two distinctly different MacOS clients. Maybe you forgot about the old uniprocessor client which is very different than the new smp client for the Intel hardware.

[7up1n3]

Currently running and completed projects per client type is one benchmark as well.

Wouldn't that be Indicative rather than definitive, in regard to actual scientific value?

That brings up another question. How should the answers manage hardware differences? You didn't ask about the scientific differences between Intel vs. AMD CPUs. That's reasonable because they run identical software so the only differences are in the features of the specific hardware you're comparing. Presumably the same answer will apply to ATI vs Nvidia so it's not a fair question.

If there is a scientific difference between different CPU architectures (other than # of cores probably), the answers would certainly relevant to the question as well. I haven't Folded with a current AMD quad-core processor, so do not know how they compare clock for clock from a productivity basis.

BTW, there are two distinctly different MacOS clients. Maybe you forgot about the old uniprocessor client which is very different than the new smp client for the Intel hardware.

Yeah, I gave a pretty rough lineup of the existing clients, so there certainly are some gaps. I did, however, differentiate between the SMP & uniprocessor clients, though my perspective is based on Windows/Linux clients, so its likely I'm not aware of all the differentiating factors within the MacOS client arsenal.

[codysluder]

One way of looking at it is flops. I presume many of you have seen this:
Subject: Highwater Mark

The Pande Group has repeatedly stated that flops are not an ideal measure of scientific results, so we have to use the information presented there more intelligently.

Perhaps the Pande Group can give us a science factor for each client. For example, they've said that the PS3 can't do everything that the uniprocessor client can do so let's guess that 2.5 PS3 FLOPS produces about the same amount of science as 1 FLOPS on Intel/AMD. (The 2.5 number is a sheer guess with no basis, but perhaps Dr.Pande can give a realistic number.) Wouldn't that make the data on the chart meaningful?

[7im]

You won't get the answers you are looking for, not in the last 5 years since I started folding, and not now. It's like asking a parent to pick their favorite child.

While one client might be more productive, it might be less relevant. Or a less productive client might be generating results for an important paper that is just about to be published. And like work units and PPD, the science changes from week to week, so today's answer would be different next week.

The only clear choice is points based. Points are mostly science based, but do factor in other criteria. Go where the points take you.

[VijayPande]

I think 7im put it well. We try to balance the points based on both performance and scientific usefulness. We understand that donors will optimize for points, so our goal is to align points with the scientific usefulness (as well as we can, without making dramatic changes to the system too frequently).

[Maverick21]

For some 'time I've been searching an answer to this question.

If I have understood correctly, points are an indicator of how important is in that moment that kind of work, is right?
So in this moment one can deduce that the project has a great need of GPU since they get a lot of points compared to CPU, but there is no doubt that a 5000 PPD GPU doesn't give a 10x "scientific work" compared to a 500 PPD CPU.

Regarding the PPD difference between the normal cpu client and the SMP can we say the same thing? They are different about the importance for the project but not about actual scientific work done?

[bruce]

I think you're confusing points with points per day.

If project A does important work, we can assign it a value of 1 point.
If project B does the same type of work but it includes twice as much analysis, we can award it 2 points (You've done the same amount of important work if you complete two projects similar to project A.

If project C is much more important than project A but takes the same amount of time as project A, it should earn 2 points.

Now suppose Machine A runs 24x7 and it can finish 100 work assignments from project A or 50 work assignments from project B That machine earns 100 PPD. If it happens to be assigned project C that same machine might earn as much as 200 PPD, but with a mix of A, B, and C, it will earn more than 100 and less than 200 PPD.

Now suppose the GPU can complete 2000 work assignments from project A in a single day. That machine earns 2000 PPD. Since the GPU is unable to do work of type C, it will earn precisely 2000 PPD.

As a practical matter, the Pande Group might package a few hundred blocks of work of type A or B or C into a single package called a Work Unit.

Of course if the first machine can run project C, it can be earning any amount between

Does it begin to make more sense now?

[chumbucket843]

well now its 2009. how is the nvidia gpu2 client doing? is bigadv doing well? ETA for GPU3?

one of the most gratifying things to see as a folding enthusiast is results. even a post that says its working is good to hear.

[Flathead74]

well now its 2009. how is the nvidia gpu2 client doing? is bigadv doing well? ETA for GPU3?

one of the most gratifying things to see as a folding enthusiast is results. even a post that says its working is good to hear.

Prof. Vijay Pande's blog:
http://folding.typepad.com/news/

[7im]

It's working.

[toTOW]

BigAdv projects are doing fine.

[chumbucket843]

It's working.

good to hear.