Folding Forum

I followed the suggestion and checked my log before and after a reboot.
So since one of the WUs lost a couple percent I confirm that there is no checkpoint on reboot. I also at least partly confirm that on my system checkpoints are made very often, consistent with what I saw on the file timestamps.

And by the way, All my folding is by GPU

Checkpoint files for WU's from which projects? The time you set in the client is currently only used by the CPU folding cores - A4 and A7. The value is passed to all folding cores, but the GPU ones do not make use of it.

Checkpoints for the GPU projects are set by the person running the project and are based on a number of steps done, or percentage of completion. Typically that has been every 2-5% of processing to be done on a GPU WU. That will be more often time wise for a fast GPU and less often on a slower one.

Joe_H
GPU ones do not make use of it

Thanks for clarifying that.

Aurum wrote: This morning I checked FAHControl and just happened to see a GPU about to hit Progress 99% on the Status/Work Queue list. I've had next-unit-percentage=99.4 set globally on the Expert tab for days and watched to see if it worked. But the GPU did not poll the AS until Progress crossed 99.8%. But in the Log tab work progress is reported as integers:

Code: Select all

08:10:51:WU02:FS01:0x21:Completed 50000 out of 5000000 steps (1%)
08:42:23:WU02:FS01:0x21:Completed 100000 out of 5000000 steps (2%)
09:14:17:WU02:FS01:0x21:Completed 150000 out of 5000000 steps (3%)
09:45:52:WU02:FS01:0x21:Completed 200000 out of 5000000 steps (4%)

I bet the "next-unit-percentage" option uses a rounded up or off version of the "value" one assigns. If "value" can only be integers it'd be nice to know, e.g. reject a decimal entry when adding the option or a note in the Help file. It might be better to use ETA to trigger DLing the next WU.

The help text does say integers:
"next-unit-percentage <integer=99>
Pre-download the next work unit when the current one is this far along.
Values less than 90 are not allowed."
viewtopic.php?f=61&t=26036

If one enters 89.9 then a popup explains it's being rejected. If one enters 93.3 it goes in but seems to interpret it as 94%. I suggest a popup saying intergers only.

Aurum wrote:The 99.9% bug: When a GPU is infected by the 99.9% bug it never seems to recover. The only way I know to cure it and get the GPU back to folding is to Quit F@H and restart it which wastes a lot of work when the other GPUs in the folding rig get set back to their checkpoints. I bet you're already working on a fix for 7.4.16 but since I see it daily just thought I'd mention it.

Are any of these options useful in restarting a GPU afflicted with the 99.9% bug

stall-detection-enabled <boolean=false>
Attempt to detect stalled work units and restart them.

stall-percent <integer=5>
Minimum estimated percent work unit completion since last frame before a WU
can be considered stalled, if zero the percentage is ignored.

stall-timeout <integer=1800>
Minimum time, in seconds, since last frame before a WU can be considered
stalled.

Aurum wrote:Are any of these options useful in restarting a GPU afflicted with the 99.9% bug

stall-detection-enabled <boolean=false>
Attempt to detect stalled work units and restart them.

stall-percent <integer=5>
Minimum estimated percent work unit completion since last frame before a WU
can be considered stalled, if zero the percentage is ignored.

stall-timeout <integer=1800>
Minimum time, in seconds, since last frame before a WU can be considered
stalled.

No. The "99.9% bug" is not related to the core being stalled, it is related to a driver and/or core crash. Until the driver and core processes are restarted for that specific GPU, the client is not able to restart. The percentage going to 99.9% is caused by the client not detecting that the GPU core has crashed.

Work has been done trying to detect and handle this condition within the folding core, but the last reports I saw on this was that it was only partially successful.

As a side note, the code to detect stalled processing was defaulted to off when testing during development of the client showed it to have a number of problems. It remains in the client as the developers hope to improve it and make it useful at some point in the future. As I understand it, that was not a priority item for the current public beta release - 7.4.15.