Project 16921 extremely low ppd

[Neil-B]

The faqs are notoriously out of date so this may have changed a bit .. not saying they have just they may have done .. benchmarking is fraught with complexity and there appears to be a review of this along with how WUs are assigned going on .. this doesn't mean things are chsnging/will change - but they may - and in true FaH fashion we may only get to here how/what when if actually happens .. past benchmarking has to the best of my knowledge also allowed for various adjustments if points variations have been spotted at beta or fof certain types of WUs such as some of the covid - but I may well be wrong on this

[jcabana]

4) FAH is guilty of points inflation and it changes your expectations. I have no specific knowledge about your GPU, but maybe its average production should be lower than your expectations I'm sure you only want the highest numbers you've ever seen, even if they happened to be higher than they should be.

I do not necessarily want the highest points amount, I want the card to be use at it's maximum potential, so results can be sent back ASAP and science can advance at full potential. All I did is share some obervations I made in the past months. Observations that can answer or hint to an answer(maybe?), to previous questions in this thread . My card is a non CUDA card (OpenCL). It was infer that the problem was CUDA related.

I am also well aware of the point inflation you are referring to, and the number I have gave was based on a normal average. On an inflationed period, I was making 25% more per day.
I have done my homework, there data spreadsheets circulating about average production per card (even specific SKU) per day - some of them with data from end of 2019. There is also a chrome extension that compîle stats available here: https://folding.lar.systems/

Just wanted to clarify things.

[PantherX]

Humm, it seems that in this case, the word "benchmark" has dual meanings:
1) Benchmark - Points allocation
2) Benchmark - WU allocation

The FAQs are correct were the baseline is done on a standard set of hardware. However, minor tweaks might be needed which is where the Beta team comes in.

What bruce mentioned is regarding actual performance of GPUs across a variety of Projects that would allow the optimum WU to be assigned to your GPU. This is still in development with no ETA.

Hopefully, this provides some clarity

[bruce]

I was under the impression based on the FAQ that all projects were run on a standard benchmarking setup and points were determined from that. Is the point allocation method changing?

That has not been discussed, but it wouldn't surprise me if that's part of the final outcome.

The standard benchmarking setup made a lot of sense as long as most WUs were in a range where GPU performance was somewhat linear but every time a new generation of GPUs comes out, the GPU nonlinearities get worse.

[mgetz]

The standard benchmarking setup made a lot of sense as long as most WUs were in a range where GPU performance was somewhat linear but every time a new generation of GPUs comes out, the GPU nonlinearities get worse.

Fair enough, however one of the advantages it had/has was that it allowed for some comparison to see if certain WUs were outliers as I'm alleging here. While my 2080 and 2070Super would in theory be a bit more efficient they aren't going to beat it that handily and should scale similarly in relation to baseline even if the scale from the 10 series card I recall was allegedly in that setup isn't linear.

Hence my concern here, It seems odd that the 16921 WUs are so off of baseline in comparison to everything else. As mentioned elsewhere most WUs regardless of cause range from 2mPPD to 3mPPD. I'm assuming anything over 2.5mPPD is covid inflation (this isn't necessarily true. 16918 (48, 11, 139) is currently clocking in at 2.6mPPD on the 2080, and 14487 (0, 1008, 119) is clocking in at 2.89mPPD on the 2070Super). But the 16921 WU are down in the 1mPPD -1.3mPPD range which by any statistical analysis is definitely an outlier, hence this thread. Normally I expect variance and wouldn't have posted this thread if it was honestly above 1.5mPPD. The only reason I made this thread was because it was literally 1/2 to 1/3rd the PPD of other WU.

To me that speaks to a WU that should either: Not be a candidate for GPU, was benchmarked wrong, or has an unanticipated issue in the WUs themselves similar to what happened with 13424. My guess is that the former two options are likely the case based on the responses in this thread.

[ajm]

It is not as straightforward: in average, the 16921 WU's are very weak for the 3090: https://folding.lar.systems/folding_dat ... e_rtx_3090
But they are only in the lower part for the 2080ti: https://folding.lar.systems/folding_dat ... _a_m_13448
They are in the middle of the pack for a 1070: https://folding.lar.systems/folding_dat ... _1070_6463
And they are rather good for a 1050ti: https://folding.lar.systems/folding_dat ... 50_ti_2138

[bruce]

To me that speaks to a WU that should either: Not be a candidate for GPU, was benchmarked wrong, or has an unanticipated issue in the WUs themselves similar to what happened with 13424. My guess is that the former two options are likely the case based on the responses in this thread.

It is not as straightforward: ... they are rather good for a 1050ti: https://folding.lar.systems/folding_dat ... 50_ti_2138

So it's reasonable to conclude it was benchmarked on a GPU comparable to a GTX1050Ti. The benchmarking project (when it's ready for production) should identify this WU characteristic and assign it to 10-series GPUs or lower (species 5) rather than to your RTX2080 (species 8) GPU.

[bruce]

GPU WUs for Project 169xx are not uniform:

. Proj. Atoms
16907 17,000 OPENMM_21 sizhang
16918 29,000 OPENMM_22 tuj66686
16920 3,893 OPENMM_22 sizhang
16921 3,893 OPENMM_22 sizhang

[ajm]

And the PPD can vary significantly even during the processing of the WU, as shown here in v1.1.2 of LAR's Dark Extension (will be available in a couple days).

[bruce]

You're presenting data based on information that's not really available. When FAHClient runs any WU, it makes two types of reports. The client logs a time as the WU passes each 1% ... sometimes called the TPF. It also stores a progress report to disk whenever a checkpoint occurs. Progress between either type of report is not likely to proceed at a uniform rate. For example, it may stop processing long enough to write the checkpoint to disk during any interval but not necessarily during the next such interval.

From those intervals, the rate of progress is estimated. The data you're presenting suggests that you believe every such interval is 100% consistent leading to a 100% constant PPD which is, at best, estimated. Look more carefully at the durations of the intervals that are being collected and on which the estimates are based. The only estimated PPD that really matters is between the download time and the upload time.

[ajm]

Do I, or do LAR_Systems? Or do the people who simply "report" fabulous PPD with their shiny setup?
This extension just shows most of the infos that the client provides, with a context (averages) that allows for a more measured and truthful view of the performance. I'd say.

But thank you for this comment! I'll pass it to LAR and see what they mean.

[ajm]

I submitted the exchange to the developer of the Dark Extension. His reaction:

(...) Regarding the comments the folding forum:

it's an uphill battle sometimes to get across to people that are used to doing something a certain way, or have expert level insight compared to the average user, and you're suggesting a different more accessible way.

The interface tries to make it clear showing dynamic change, but may not be clear if the person critiquing it does not use it and only sees a static image, not seeing it's very upfront about being variable data and that the performance averages are constantly changing on the DB site as well to help convey the concerns they are voicing.

Total PPD is estimated (labeled as such) and constantly changing no claims otherwise.

Average PPD is labeled as being work unit and GPU combination specific and the number of samples used to create that average based on the PPD swings as reported by different users clients, sampled at random to get peaks and valleys as reported from the F@H service. It's based on how the F@H reports PPD in real-time, as that's what average users are looking at.

This is a tool intended to provide an indication to folders that don't understand PPD is not constant and does wildly vary by project WU during individual folds and overall from start to finish of processing.

The addition of charting is to demonstrate to the new / average folder that PPD is not consistent at all and that the wave form seen is the folds generally synced with the TPF (time per fold) / checkpoints that the default client does not report transparently as an educational tool for users that might not dig in to or understand logs. The types of users confused when they can't get their PPD up to the value someone told them their hardware is capable of and flood forums with the same posts over how to get there.

Charts when viewed near the end of completion of the work unit show these users how PPD / WU Points were reported at the start, throughout processing and near the end, which is to demonstrate that it always starts high, levels out in to a fluctuating pattern, and PPD / points over the duration of the WU trend down the majority of the time as is seen in logs, but is presented in a way for an average user to view at a glance, in context to their GPU and the project/WU.

Ultimately if you're a F@H power user, you might mine all your own log data, have all this figured out, like calculating performance in context to your personal experience and specific hardware etc. you probably find the stock web client useless, use advanced client with log parsing and have no use for this extension or the related GPU database.

For people without that expertise or the time, this provides baseline information at a glance as an education tool and general indicator, charts changes in PPD performance as reported by the folding service in real-time to teach about impacts to performance tweaking GPU settings or using your system in different ways (watching videos, playing games etc.) and of tracks and provides aggregate averages of different hardware / project information to further demonstrate the differences.

The data in the extension is also just skin deep, and for those interested in looking deeper at GPU PPD average trends etc. the database site shows the average changes over time etc. as a further education tool as F@H cores / projects change.

End of the day, no one is forcing anyone to use it, it may not be for everyone and it's up to the user if it's of value to them.