new fahcore 22 version 0.0.18 fail

[TheUndeadOne]

pardon my bad english but im out of idea ive been trying for hours and now i have given up, came here signedup and ask for help
i got everything needed but it dosent work... everything was fine like 2 day ago on older core22 version
its now stuck in a deadline-missing speed on crapCL i ruinned my passphrase and lost the QRB trying to get the core to fold (kept asking for new WUs instead of resuming, had too many had to dump)


im out of idea and kind of sad that i didnt got to enjoy my already retired folding machine for more than 2 days (i folded alot in the pass ( 15K WUs ))
i was hoping that was because of that cuda 11.2 thing but nope i got it... am i missing some arguments with that "new"(?) " --gpu-architecture (-arch) "
i guess im gonna be cpu only on that good old 3930k

no idea how you do that windowed stuff for the logs thats my first post ever even if ive read this forum for a decade+ but well here it is...
in the hope of a new core with beter compability or a way to stay with the older version or a fix or something


(welp looks like i cant post w/o figuring out that windowcode thingy hope it works)

Code: Select all

*********************** Log Started 2021-11-20T09:37:54Z ***********************
09:37:54:******************************* libFAH ********************************
09:37:54:           Date: Oct 20 2020
09:37:54:           Time: 13:36:55
09:37:54:       Revision: 5ca109d295a6245e2a2f590b3d0085ad5e567aeb
09:37:54:         Branch: master
09:37:54:       Compiler: Visual C++ 2015
09:37:54:        Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Zc:throwingNew /MT
09:37:54:       Platform: win32 10
09:37:54:           Bits: 32
09:37:54:           Mode: Release
09:37:54:****************************** FAHClient ******************************
09:37:54:        Version: 7.6.21
09:37:54:         Author: Joseph Coffland <joseph@cauldrondevelopment.com>
09:37:54:      Copyright: 2020 foldingathome.org
09:37:54:       Homepage: https://foldingathome.org/
09:37:54:           Date: Oct 20 2020
09:37:54:           Time: 13:41:04
09:37:54:       Revision: 6efbf0e138e22d3963e6a291f78dcb9c6422a278
09:37:54:         Branch: master
09:37:54:       Compiler: Visual C++ 2015
09:37:54:        Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Zc:throwingNew /MT
09:37:54:       Platform: win32 10
09:37:54:           Bits: 32
09:37:54:           Mode: Release
09:37:54:         Config: C:\ProgramData\FAHClient\config.xml
09:37:54:******************************** CBang ********************************
09:37:54:           Date: Oct 20 2020
09:37:54:           Time: 11:36:18
09:37:54:       Revision: 7e4ce85225d7eaeb775e87c31740181ca603de60
09:37:54:         Branch: master
09:37:54:       Compiler: Visual C++ 2015
09:37:54:        Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Zc:throwingNew /MT
09:37:54:       Platform: win32 10
09:37:54:           Bits: 32
09:37:54:           Mode: Release
09:37:54:******************************* System ********************************
09:37:54:            CPU: Intel(R) Core(TM) i7-3930K CPU @ 3.20GHz
09:37:54:         CPU ID: GenuineIntel Family 6 Model 45 Stepping 7
09:37:54:           CPUs: 12
09:37:54:         Memory: 15.94GiB
09:37:54:    Free Memory: 12.42GiB
09:37:54:        Threads: WINDOWS_THREADS
09:37:54:     OS Version: 6.2
09:37:54:    Has Battery: false
09:37:54:     On Battery: false
09:37:54:     UTC Offset: -5
09:37:54:            PID: 13584
09:37:54:            CWD: C:\ProgramData\FAHClient
09:37:54:  Win32 Service: false
09:37:54:             OS: Windows 10 Enterprise
09:37:54:        OS Arch: AMD64
09:37:54:           GPUs: 2
09:37:54:          GPU 0: Bus:1 Slot:0 Func:0 NVIDIA:7 GP104 [GeForce GTX 1080] 8873
09:37:54:          GPU 1: Bus:4 Slot:0 Func:0 NVIDIA:4 GK104 [GeForce GTX 680] 3250
09:37:54:  CUDA Device 0: Platform:0 Device:0 Bus:1 Slot:0 Compute:6.1 Driver:11.2
09:37:54:  CUDA Device 1: Platform:0 Device:1 Bus:4 Slot:0 Compute:3.0 Driver:11.2
09:37:54:OpenCL Device 0: Platform:0 Device:0 Bus:1 Slot:0 Compute:1.2 Driver:461.92
09:37:54:OpenCL Device 1: Platform:0 Device:1 Bus:4 Slot:0 Compute:1.2 Driver:461.92
09:37:54:***********************************************************************
09:37:54:<config>
09:37:54:  <!-- Folding Core -->
09:37:54:  <checkpoint v='5'/>
09:37:54:
09:37:54:  <!-- Folding Slot Configuration -->
09:37:54:  <cause v='COVID_19'/>
09:37:54:
09:37:54:  <!-- Network -->
09:37:54:  <proxy v=':8080'/>
09:37:54:
09:37:54:  <!-- Slot Control -->
09:37:54:  <power v='full'/>
09:37:54:
09:37:54:  <!-- User Information -->
09:37:54:  <passkey v='*****'/>
09:37:54:  <team v='224497'/>
09:37:54:  <user v='TheUndead_ALL_17cCocUE9EGG95cDe6YatVMNMV8sgx91AL'/>
09:37:54:
09:37:54:  <!-- Folding Slots -->
09:37:54:  <slot id='0' type='CPU'>
09:37:54:    <cpus v='10'/>
09:37:54:    <paused v='true'/>
09:37:54:  </slot>
09:37:54:  <slot id='2' type='GPU'>
09:37:54:    <pci-bus v='4'/>
09:37:54:    <pci-slot v='0'/>
09:37:54:  </slot>
09:37:54:  <slot id='1' type='GPU'>
09:37:54:    <paused v='true'/>
09:37:54:    <pci-bus v='1'/>
09:37:54:    <pci-slot v='0'/>
09:37:54:  </slot>
09:37:54:</config>
09:37:54:Trying to access database...
09:37:54:Successfully acquired database lock
09:37:54:FS00:Initialized folding slot 00: cpu:10
09:37:54:FS02:Initialized folding slot 02: gpu:4:0 GK104 [GeForce GTX 680] 3250
09:37:54:FS01:Initialized folding slot 01: gpu:1:0 GP104 [GeForce GTX 1080] 8873
09:37:54:WU00:FS02:Downloading core from http://cores.foldingathome.org/win/64bit/22-0.0.18/Core_22.fah
09:37:54:WU00:FS02:Connecting to cores.foldingathome.org:80
09:37:56:WU00:FS02:FahCore 22: Downloading 156.53MiB
09:38:02:WU00:FS02:FahCore 22: 47.16%
09:38:08:WU00:FS02:FahCore 22: 92.87%
09:38:08:WU00:FS02:FahCore 22: Download complete
09:38:08:WU00:FS02:Valid core signature
09:38:09:WU00:FS02:Unpacked 5.58MiB to cores/cores.foldingathome.org/win/64bit/22-0.0.18/Core_22.fah/FahCore_22.exe
09:38:09:WU00:FS02:Unpacked 24.45KiB to cores/cores.foldingathome.org/win/64bit/22-0.0.18/Core_22.fah/api-ms-win-crt-runtime-l1-1-0.dll
09:38:10:WU00:FS02:Unpacked 179.58MiB to cores/cores.foldingathome.org/win/64bit/22-0.0.18/Core_22.fah/cufft64_10.dll
09:38:10:WU00:FS02:Unpacked 3.25MiB to cores/cores.foldingathome.org/win/64bit/22-0.0.18/Core_22.fah/libcrypto-1_1-x64.dll
09:38:10:WU00:FS02:Unpacked 667.00KiB to cores/cores.foldingathome.org/win/64bit/22-0.0.18/Core_22.fah/libssl-1_1-x64.dll
09:38:10:WU00:FS02:Unpacked 552.38KiB to cores/cores.foldingathome.org/win/64bit/22-0.0.18/Core_22.fah/msvcp140.dll
09:38:10:WU00:FS02:Unpacked 23.38KiB to cores/cores.foldingathome.org/win/64bit/22-0.0.18/Core_22.fah/msvcp140_1.dll
09:38:10:WU00:FS02:Unpacked 181.38KiB to cores/cores.foldingathome.org/win/64bit/22-0.0.18/Core_22.fah/msvcp140_2.dll
09:38:10:WU00:FS02:Unpacked 54.88KiB to cores/cores.foldingathome.org/win/64bit/22-0.0.18/Core_22.fah/msvcp140_atomic_wait.dll
09:38:10:WU00:FS02:Unpacked 19.88KiB to cores/cores.foldingathome.org/win/64bit/22-0.0.18/Core_22.fah/msvcp140_codecvt_ids.dll
09:38:10:WU00:FS02:Unpacked 5.29MiB to cores/cores.foldingathome.org/win/64bit/22-0.0.18/Core_22.fah/nvrtc-builtins64_112.dll
09:38:11:WU00:FS02:Unpacked 30.51MiB to cores/cores.foldingathome.org/win/64bit/22-0.0.18/Core_22.fah/nvrtc64_112_0.dll
09:38:11:WU00:FS02:Unpacked 2.75MiB to cores/cores.foldingathome.org/win/64bit/22-0.0.18/Core_22.fah/OpenMM.dll
09:38:11:WU00:FS02:Unpacked 302.00KiB to cores/cores.foldingathome.org/win/64bit/22-0.0.18/Core_22.fah/OpenMMAmoeba.dll
09:38:11:WU00:FS02:Unpacked 1.04MiB to cores/cores.foldingathome.org/win/64bit/22-0.0.18/Core_22.fah/OpenMMAmoebaCUDA.dll
09:38:11:WU00:FS02:Unpacked 950.00KiB to cores/cores.foldingathome.org/win/64bit/22-0.0.18/Core_22.fah/OpenMMAmoebaOpenCL.dll
09:38:11:WU00:FS02:Unpacked 458.00KiB to cores/cores.foldingathome.org/win/64bit/22-0.0.18/Core_22.fah/OpenMMAmoebaReference.dll
09:38:11:WU00:FS02:Unpacked 507.50KiB to cores/cores.foldingathome.org/win/64bit/22-0.0.18/Core_22.fah/OpenMMCPU.dll
09:38:11:WU00:FS02:Unpacked 1.76MiB to cores/cores.foldingathome.org/win/64bit/22-0.0.18/Core_22.fah/OpenMMCUDA.dll
09:38:11:WU00:FS02:Unpacked 60.50KiB to cores/cores.foldingathome.org/win/64bit/22-0.0.18/Core_22.fah/OpenMMCudaCompiler.dll
09:38:11:WU00:FS02:Unpacked 98.00KiB to cores/cores.foldingathome.org/win/64bit/22-0.0.18/Core_22.fah/OpenMMDrude.dll
09:38:11:WU00:FS02:Unpacked 116.50KiB to cores/cores.foldingathome.org/win/64bit/22-0.0.18/Core_22.fah/OpenMMDrudeCUDA.dll
09:38:11:WU00:FS02:Unpacked 116.50KiB to cores/cores.foldingathome.org/win/64bit/22-0.0.18/Core_22.fah/OpenMMDrudeOpenCL.dll
09:38:11:WU00:FS02:Unpacked 86.50KiB to cores/cores.foldingathome.org/win/64bit/22-0.0.18/Core_22.fah/OpenMMDrudeReference.dll
09:38:11:WU00:FS02:Unpacked 1.78MiB to cores/cores.foldingathome.org/win/64bit/22-0.0.18/Core_22.fah/OpenMMOpenCL.dll
09:38:11:WU00:FS02:Unpacked 61.00KiB to cores/cores.foldingathome.org/win/64bit/22-0.0.18/Core_22.fah/OpenMMPME.dll
09:38:11:WU00:FS02:Unpacked 56.00KiB to cores/cores.foldingathome.org/win/64bit/22-0.0.18/Core_22.fah/OpenMMRPMD.dll
09:38:11:WU00:FS02:Unpacked 137.00KiB to cores/cores.foldingathome.org/win/64bit/22-0.0.18/Core_22.fah/OpenMMRPMDCUDA.dll
09:38:11:WU00:FS02:Unpacked 137.00KiB to cores/cores.foldingathome.org/win/64bit/22-0.0.18/Core_22.fah/OpenMMRPMDOpenCL.dll
09:38:11:WU00:FS02:Unpacked 75.00KiB to cores/cores.foldingathome.org/win/64bit/22-0.0.18/Core_22.fah/OpenMMRPMDReference.dll
09:38:11:WU00:FS02:Unpacked 94.88KiB to cores/cores.foldingathome.org/win/64bit/22-0.0.18/Core_22.fah/vcruntime140.dll
09:38:11:WU00:FS02:Unpacked 36.38KiB to cores/cores.foldingathome.org/win/64bit/22-0.0.18/Core_22.fah/vcruntime140_1.dll
09:38:11:WU00:FS02:Starting
09:38:11:WU00:FS02:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\ProgramData\FAHClient\cores/cores.foldingathome.org/win/64bit/22-0.0.18/Core_22.fah/FahCore_22.exe -dir 00 -suffix 01 -version 706 -lifeline 13584 -checkpoint 5 -opencl-platform 0 -opencl-device 1 -cuda-device 1 -gpu-vendor nvidia -gpu 1 -gpu-usage 100
09:38:11:WU00:FS02:Started FahCore on PID 1760
09:38:11:WU00:FS02:Core PID:10872
09:38:11:WU00:FS02:FahCore 0x22 started
09:38:12:WU00:FS02:0x22:*********************** Log Started 2021-11-20T09:38:11Z ***********************
09:38:12:WU00:FS02:0x22:*************************** Core22 Folding@home Core ***************************
09:38:12:WU00:FS02:0x22:       Core: Core22
09:38:12:WU00:FS02:0x22:       Type: 0x22
09:38:12:WU00:FS02:0x22:    Version: 0.0.18
09:38:12:WU00:FS02:0x22:     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
09:38:12:WU00:FS02:0x22:  Copyright: 2020 foldingathome.org
09:38:12:WU00:FS02:0x22:   Homepage: https://foldingathome.org/
09:38:12:WU00:FS02:0x22:       Date: Sep 28 2021
09:38:12:WU00:FS02:0x22:       Time: 05:55:05
09:38:12:WU00:FS02:0x22:   Revision: cfe3d7d990e8f456e371f8ce63b5fcc6daab2103
09:38:12:WU00:FS02:0x22:     Branch: HEAD
09:38:12:WU00:FS02:0x22:   Compiler: Visual C++
09:38:12:WU00:FS02:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
09:38:12:WU00:FS02:0x22:             -DOPENMM_VERSION="\"7.6.0\""
09:38:12:WU00:FS02:0x22:   Platform: win32 10
09:38:12:WU00:FS02:0x22:       Bits: 64
09:38:12:WU00:FS02:0x22:       Mode: Release
09:38:12:WU00:FS02:0x22:Maintainers: John Chodera <john.chodera@choderalab.org> and Peter Eastman
09:38:12:WU00:FS02:0x22:             <peastman@stanford.edu>
09:38:12:WU00:FS02:0x22:       Args: -dir 00 -suffix 01 -version 706 -lifeline 1760 -checkpoint 5
09:38:12:WU00:FS02:0x22:             -opencl-platform 0 -opencl-device 1 -cuda-device 1 -gpu-vendor
09:38:12:WU00:FS02:0x22:             nvidia -gpu 1 -gpu-usage 100
09:38:12:WU00:FS02:0x22:************************************ libFAH ************************************
09:38:12:WU00:FS02:0x22:       Date: Sep 28 2021
09:38:12:WU00:FS02:0x22:       Time: 05:53:43
09:38:12:WU00:FS02:0x22:   Revision: 44301ed97b996b63fe736bb8073f22209cb2b603
09:38:12:WU00:FS02:0x22:     Branch: HEAD
09:38:12:WU00:FS02:0x22:   Compiler: Visual C++
09:38:12:WU00:FS02:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
09:38:12:WU00:FS02:0x22:   Platform: win32 10
09:38:12:WU00:FS02:0x22:       Bits: 64
09:38:12:WU00:FS02:0x22:       Mode: Release
09:38:12:WU00:FS02:0x22:************************************ CBang *************************************
09:38:12:WU00:FS02:0x22:       Date: Sep 28 2021
09:38:12:WU00:FS02:0x22:       Time: 05:52:38
09:38:12:WU00:FS02:0x22:   Revision: 33fcfc2b3ed2195a423606a264718e31e6b3903f
09:38:12:WU00:FS02:0x22:     Branch: HEAD
09:38:12:WU00:FS02:0x22:   Compiler: Visual C++
09:38:12:WU00:FS02:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
09:38:12:WU00:FS02:0x22:   Platform: win32 10
09:38:12:WU00:FS02:0x22:       Bits: 64
09:38:12:WU00:FS02:0x22:       Mode: Release
09:38:12:WU00:FS02:0x22:************************************ System ************************************
09:38:12:WU00:FS02:0x22:        CPU: Intel(R) Core(TM) i7-3930K CPU @ 3.20GHz
09:38:12:WU00:FS02:0x22:     CPU ID: GenuineIntel Family 6 Model 45 Stepping 7
09:38:12:WU00:FS02:0x22:       CPUs: 12
09:38:12:WU00:FS02:0x22:     Memory: 15.94GiB
09:38:12:WU00:FS02:0x22:Free Memory: 12.19GiB
09:38:12:WU00:FS02:0x22:    Threads: WINDOWS_THREADS
09:38:12:WU00:FS02:0x22: OS Version: 6.2
09:38:12:WU00:FS02:0x22:Has Battery: false
09:38:12:WU00:FS02:0x22: On Battery: false
09:38:12:WU00:FS02:0x22: UTC Offset: -5
09:38:12:WU00:FS02:0x22:        PID: 10872
09:38:12:WU00:FS02:0x22:        CWD: C:\ProgramData\FAHClient\work
09:38:12:WU00:FS02:0x22:************************************ OpenMM ************************************
09:38:12:WU00:FS02:0x22:    Version: 7.6.0
09:38:12:WU00:FS02:0x22:********************************************************************************
09:38:12:WU00:FS02:0x22:Project: 18432 (Run 34, Clone 7, Gen 4)
09:38:12:WU00:FS02:0x22:Unit: 0x00000000000000000000000000000000
09:38:12:WU00:FS02:0x22:Digital signatures verified
09:38:12:WU00:FS02:0x22:Folding@home GPU Core22 Folding@home Core
09:38:12:WU00:FS02:0x22:Version 0.0.18
09:38:12:WU00:FS02:0x22:  Checkpoint write interval: 100000 steps (2%) [50 total]
09:38:12:WU00:FS02:0x22:  JSON viewer frame write interval: 50000 steps (1%) [100 total]
09:38:12:WU00:FS02:0x22:  XTC frame write interval: 250000 steps (5%) [20 total]
09:38:12:WU00:FS02:0x22:  Global context and integrator variables write interval: disabled
09:38:13:WU00:FS02:0x22:There are 4 platforms available.
09:38:13:WU00:FS02:0x22:Platform 0: Reference
09:38:13:WU00:FS02:0x22:Platform 1: CPU
09:38:13:WU00:FS02:0x22:Platform 2: OpenCL
09:38:13:WU00:FS02:0x22:  opencl-device 1 specified
09:38:13:WU00:FS02:0x22:Platform 3: CUDA
09:38:13:WU00:FS02:0x22:  cuda-device 1 specified
09:38:17:WU00:FS02:0x22:Attempting to create CUDA context:
09:38:17:WU00:FS02:0x22:  Configuring platform CUDA
09:38:17:WU00:FS02:0x22:Failed to create CUDA context:
09:38:17:WU00:FS02:0x22:Error compiling program: nvrtc: error: invalid value for --gpu-architecture (-arch)
09:38:17:WU00:FS02:0x22:Attempting to create OpenCL context:
09:38:17:WU00:FS02:0x22:  Configuring platform OpenCL
09:38:19:WU00:FS02:0x22:  Using OpenCL on platformId 0 and gpu 1
09:38:19:WU00:FS02:0x22:Completed 0 out of 5000000 steps (0%)
09:38:20:WU00:FS02:0x22:Checkpoint completed at step 0

[Neil-B]

The gtx680 might have issues as it is kepler and to quote an unofficial non fah statement "Not official statement You should consider Kepler as being phased out and abandoned soon on FAH, as nVidia stopped updating drivers for it. Not official statement" ... I believe this is fs2 which is the one reverting to opencl ... I would have expected you could get the gtx1080 to fold cuda ... may be a case of needing newer drivers - 496.76 may be available for you cards ... it might (long shot) get the keeler card working?

[PaulTV]

Project 16484 to 16488 is still on core 22-0.0.13. If you set the cause preference to cancer, you may still enjoy cuda folding on that card for a bit longer.

[TheUndeadOne]

the gtx 1080 is temp this was gonna be a fold to death machine on gtx680 but i guess not
also the client re-add all gpu download 3 unit everytime I boot dumping 2 when i re-remove the slot
still getting broken core unit, more dump
im getting further and further from QRB

do you know why you are removing kepler support ? those card are really good at compute for what it was, served me well

thx for the help i may put back the 1080 at work when the 4080 get out

[aetch]

You're not getting the usual CUDA error but I'd suggest updating your geforce driver anyway. The most recent one to support the GTX680 looks to be 472.12.

[Neil-B]

do you know why you are removing kepler support ?

The latest FaH core for gpus 0.0.18 utilises a more recent openmm than the 0.0.13 core which provides significant performance improvements and functionality enhancements for current gpus .. this openmm requires a more recent cuda version which some of the oldest gpus might not be able to run .. as to kepler support being dropped in current drivers that is nvidias decision.

It may be possible that the issues with your gpu not running cuda can be resolved by updating to the newest driver that supports your gpu or it might be an installation feature but even if you can get it working now you may need to bd prepared for it to become obsolete from a fah perspective relatively soon

This is just my understanding of the state of play .. it isn't an official statement.

[TheUndeadOne]

well i tryed a second run on the 1080 it does run cuda so kepler is out
but it run slow too... gpu at 80% then error out
it was fine too on the older version

im out of gpu folding for now this has gotten very troublesome compared to a couple years ago.. ill be back later rocking a 1080 + 4080 with probably a new core revision

here is the error in case it help something or someone later

Code: Select all

19:07:31:WU01:FS01:0x22:Completed 150000 out of 5000000 steps (3%)
19:08:49:WU01:FS01:0x22:Completed 200000 out of 5000000 steps (4%)
19:10:07:WU01:FS01:0x22:Completed 250000 out of 5000000 steps (5%)
19:10:07:WU01:FS01:0x22:An exception occurred at step 250000: Kinetic energy error of 12.0661, threshold of 10
19:10:07:WU01:FS01:0x22:Reference Kinetic Energy: 78180.9 | Given Kinetic Energy: 78193
19:10:07:WU01:FS01:0x22:ERROR:98: Attempting to restart from last good checkpoint by restarting core.
19:10:07:WU01:FS01:0x22:Folding@home Core Shutdown: CORE_RESTART
19:10:08:WARNING:WU01:FS01:FahCore returned: CORE_RESTART (98 = 0x62)
19:10:08:WU01:FS01:Starting

[Neil-B]

Could you post a bit more of ghe log on that one including the project and rcg part .. that actually looks as if it might be a bad wu and would be useful to know which one

[TheUndeadOne]

looks like FaH came to the same conclusion i restarted the same WU to make a log and this happened

Code: Select all

15:54:23:WU01:FS01:Started FahCore on PID 6876
15:54:23:WU01:FS01:Core PID:14928
15:54:23:WU01:FS01:FahCore 0x22 started
15:54:23:WU01:FS01:0x22:*********************** Log Started 2021-11-26T15:54:23Z ***********************
15:54:23:WU01:FS01:0x22:*************************** Core22 Folding@home Core ***************************
15:54:23:WU01:FS01:0x22:       Core: Core22
15:54:23:WU01:FS01:0x22:       Type: 0x22
15:54:23:WU01:FS01:0x22:    Version: 0.0.18
15:54:23:WU01:FS01:0x22:     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
15:54:23:WU01:FS01:0x22:  Copyright: 2020 foldingathome.org
15:54:23:WU01:FS01:0x22:   Homepage: https://foldingathome.org/
15:54:23:WU01:FS01:0x22:       Date: Sep 28 2021
15:54:23:WU01:FS01:0x22:       Time: 05:55:05
15:54:23:WU01:FS01:0x22:   Revision: cfe3d7d990e8f456e371f8ce63b5fcc6daab2103
15:54:23:WU01:FS01:0x22:     Branch: HEAD
15:54:23:WU01:FS01:0x22:   Compiler: Visual C++
15:54:23:WU01:FS01:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
15:54:23:WU01:FS01:0x22:             -DOPENMM_VERSION="\"7.6.0\""
15:54:23:WU01:FS01:0x22:   Platform: win32 10
15:54:23:WU01:FS01:0x22:       Bits: 64
15:54:23:WU01:FS01:0x22:       Mode: Release
15:54:23:WU01:FS01:0x22:Maintainers: John Chodera <john.chodera@choderalab.org> and Peter Eastman
15:54:23:WU01:FS01:0x22:             <peastman@stanford.edu>
15:54:23:WU01:FS01:0x22:       Args: -dir 01 -suffix 01 -version 706 -lifeline 6876 -checkpoint 3
15:54:23:WU01:FS01:0x22:             -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor
15:54:23:WU01:FS01:0x22:             nvidia -gpu 0 -gpu-usage 100
15:54:23:WU01:FS01:0x22:************************************ libFAH ************************************
15:54:23:WU01:FS01:0x22:       Date: Sep 28 2021
15:54:23:WU01:FS01:0x22:       Time: 05:53:43
15:54:23:WU01:FS01:0x22:   Revision: 44301ed97b996b63fe736bb8073f22209cb2b603
15:54:23:WU01:FS01:0x22:     Branch: HEAD
15:54:23:WU01:FS01:0x22:   Compiler: Visual C++
15:54:23:WU01:FS01:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
15:54:23:WU01:FS01:0x22:   Platform: win32 10
15:54:23:WU01:FS01:0x22:       Bits: 64
15:54:23:WU01:FS01:0x22:       Mode: Release
15:54:23:WU01:FS01:0x22:************************************ CBang *************************************
15:54:23:WU01:FS01:0x22:       Date: Sep 28 2021
15:54:23:WU01:FS01:0x22:       Time: 05:52:38
15:54:23:WU01:FS01:0x22:   Revision: 33fcfc2b3ed2195a423606a264718e31e6b3903f
15:54:23:WU01:FS01:0x22:     Branch: HEAD
15:54:23:WU01:FS01:0x22:   Compiler: Visual C++
15:54:23:WU01:FS01:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
15:54:23:WU01:FS01:0x22:   Platform: win32 10
15:54:23:WU01:FS01:0x22:       Bits: 64
15:54:23:WU01:FS01:0x22:       Mode: Release
15:54:23:WU01:FS01:0x22:************************************ System ************************************
15:54:23:WU01:FS01:0x22:        CPU: Intel(R) Core(TM) i7-3930K CPU @ 3.20GHz
15:54:23:WU01:FS01:0x22:     CPU ID: GenuineIntel Family 6 Model 45 Stepping 7
15:54:23:WU01:FS01:0x22:       CPUs: 12
15:54:23:WU01:FS01:0x22:     Memory: 15.94GiB
15:54:23:WU01:FS01:0x22:Free Memory: 8.94GiB
15:54:23:WU01:FS01:0x22:    Threads: WINDOWS_THREADS
15:54:23:WU01:FS01:0x22: OS Version: 6.2
15:54:23:WU01:FS01:0x22:Has Battery: false
15:54:23:WU01:FS01:0x22: On Battery: false
15:54:23:WU01:FS01:0x22: UTC Offset: -5
15:54:23:WU01:FS01:0x22:        PID: 14928
15:54:23:WU01:FS01:0x22:        CWD: C:\ProgramData\FAHClient\work
15:54:23:WU01:FS01:0x22:************************************ OpenMM ************************************
15:54:23:WU01:FS01:0x22:    Version: 7.6.0
15:54:23:WU01:FS01:0x22:********************************************************************************
15:54:23:WU01:FS01:0x22:Project: 17806 (Run 30, Clone 1, Gen 119)
15:54:23:WU01:FS01:0x22:Unit: 0x00000000000000000000000000000000
15:54:23:WU01:FS01:0x22:Digital signatures verified
15:54:23:WU01:FS01:0x22:Folding@home GPU Core22 Folding@home Core
15:54:23:WU01:FS01:0x22:Version 0.0.18
15:54:23:WU01:FS01:0x22:  Checkpoint write interval: 250000 steps (5%) [20 total]
15:54:23:WU01:FS01:0x22:  JSON viewer frame write interval: 50000 steps (1%) [100 total]
15:54:23:WU01:FS01:0x22:  XTC frame write interval: 25000 steps (0.5%) [200 total]
15:54:23:WU01:FS01:0x22:  Global context and integrator variables write interval: disabled
15:54:23:WU01:FS01:0x22:There are 4 platforms available.
15:54:23:WU01:FS01:0x22:Platform 0: Reference
15:54:23:WU01:FS01:0x22:Platform 1: CPU
15:54:23:WU01:FS01:0x22:Platform 2: OpenCL
15:54:23:WU01:FS01:0x22:  opencl-device 0 specified
15:54:23:WU01:FS01:0x22:Platform 3: CUDA
15:54:23:WU01:FS01:0x22:  cuda-device 0 specified
15:54:25:WU01:FS01:0x22:Attempting to create CUDA context:
15:54:25:WU01:FS01:0x22:  Configuring platform CUDA
15:54:27:WU01:FS01:0x22:  Using CUDA and gpu 0
15:54:27:WU01:FS01:0x22:Completed 500000 out of 5000000 steps (10%)
15:55:40:WU01:FS01:0x22:Completed 550000 out of 5000000 steps (11%)
15:56:55:WU01:FS01:0x22:Completed 600000 out of 5000000 steps (12%)
15:58:10:WU01:FS01:0x22:Completed 650000 out of 5000000 steps (13%)
15:59:24:WU01:FS01:0x22:Completed 700000 out of 5000000 steps (14%)
16:00:38:WU01:FS01:0x22:Completed 750000 out of 5000000 steps (15%)
16:00:39:WU01:FS01:0x22:An exception occurred at step 750000: Kinetic energy error of 13.367, threshold of 10
16:00:39:WU01:FS01:0x22:Reference Kinetic Energy: 78136.1 | Given Kinetic Energy: 78149.5
16:00:39:WU01:FS01:0x22:Max number of attempts to resume from last checkpoint (2) reached. Aborting.
16:00:39:WU01:FS01:0x22:ERROR:114: Max number of attempts to resume from last checkpoint reached.
16:00:39:WU01:FS01:0x22:Saving result file ..\logfile_01.txt
16:00:39:WU01:FS01:0x22:Saving result file positions.xtc
16:00:39:WU01:FS01:0x22:Saving result file science.log
16:00:39:WU01:FS01:0x22:Saving result file state.xml.bz2
16:00:39:WU01:FS01:0x22:Folding@home Core Shutdown: BAD_WORK_UNIT
16:00:39:WARNING:WU01:FS01:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
16:00:39:WU01:FS01:Sending unit results: id:01 state:SEND error:FAULTY project:17806 run:30 clone:1 gen:119 core:0x22 unit:0x00000001000000770000458e0000001e
16:00:39:WU01:FS01:Uploading 2.48MiB to 207.53.233.146

the 1080 is doing fine on a new WU so it was a false alarm, just a bad WU

[Joe_H]

There have been some reports from the initial testers of this version of Core_18 that it results in a bit higher utilization of the GPU when running as compared to prior versions. In some cases they had to slightly reduce overclocks for their cards to be stable. So it is also possible if your 1080 is overclocked that the WU just managed to put your card into instability calculating the data. A different WU might not push the card that far.

So if you start to see a pattern of errors over different WUs , consider reducing the GPU clock a bit. In the case of this WU it is too early to say if it is bad, only one return so far in the database, also a failure from a different user.

[TheUndeadOne]

well nevermind.. i never saw a bad work unit in 15k as far as i remember but multiple in a row ?

its stuck in this cycle

Code: Select all

16:52:11:WU03:FS01:0x22:An exception occurred at step 645069: Particle coordinate is nan
16:52:11:WU03:FS01:0x22:Max number of attempts to resume from last checkpoint (2) reached. Aborting.
16:52:11:WU03:FS01:0x22:ERROR:114: Max number of attempts to resume from last checkpoint reached.
16:52:11:WU03:FS01:0x22:Saving result file ..\logfile_01.txt
16:52:11:WU03:FS01:0x22:Saving result file science.log
16:52:11:WU03:FS01:0x22:Saving result file state.xml.bz2
16:52:11:WU03:FS01:0x22:Folding@home Core Shutdown: BAD_WORK_UNIT
16:52:12:WARNING:WU03:FS01:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
16:52:12:WU03:FS01:Sending unit results: id:03 state:SEND error:FAULTY project:17804 run:87 clone:219 gen:391 core:0x22 unit:0x000000db000001870000458c00000057
16:52:12:WU03:FS01:Uploading 5.51MiB to 207.53.233.146
16:52:12:WU03:FS01:Connecting to 207.53.233.146:8080
16:52:12:WU01:FS01:Connecting to assign1.foldingathome.org:80
16:52:12:WU01:FS01:Assigned to work server 207.53.233.146
16:52:12:WU01:FS01:Requesting new work unit for slot 01: gpu:1:0 GP104 [GeForce GTX 1080] 8873 from 207.53.233.146
16:52:12:WU01:FS01:Connecting to 207.53.233.146:8080
16:52:12:WU01:FS01:Downloading 6.10MiB
16:52:13:WU01:FS01:Download complete
16:52:13:WU01:FS01:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:17804 run:80 clone:65 gen:430 core:0x22 unit:0x00000041000001ae0000458c00000050
16:52:13:WU01:FS01:Starting
16:52:13:WU01:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\ProgramData\FAHClient\cores/cores.foldingathome.org/win/64bit/22-0.0.18/Core_22.fah/FahCore_22.exe -dir 01 -suffix 01 -version 706 -lifeline 4880 -checkpoint 3 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
16:52:13:WU01:FS01:Started FahCore on PID 15928
16:52:13:WU01:FS01:Core PID:17936
16:52:13:WU01:FS01:FahCore 0x22 started
16:52:14:WU01:FS01:0x22:*********************** Log Started 2021-11-26T16:52:13Z ***********************
16:52:14:WU01:FS01:0x22:*************************** Core22 Folding@home Core ***************************
16:52:14:WU01:FS01:0x22:       Core: Core22
16:52:14:WU01:FS01:0x22:       Type: 0x22
16:52:14:WU01:FS01:0x22:    Version: 0.0.18
16:52:14:WU01:FS01:0x22:     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
16:52:14:WU01:FS01:0x22:  Copyright: 2020 foldingathome.org
16:52:14:WU01:FS01:0x22:   Homepage: https://foldingathome.org/
16:52:14:WU01:FS01:0x22:       Date: Sep 28 2021
16:52:14:WU01:FS01:0x22:       Time: 05:55:05
16:52:14:WU01:FS01:0x22:   Revision: cfe3d7d990e8f456e371f8ce63b5fcc6daab2103
16:52:14:WU01:FS01:0x22:     Branch: HEAD
16:52:14:WU01:FS01:0x22:   Compiler: Visual C++
16:52:14:WU01:FS01:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
16:52:14:WU01:FS01:0x22:             -DOPENMM_VERSION="\"7.6.0\""
16:52:14:WU01:FS01:0x22:   Platform: win32 10
16:52:14:WU01:FS01:0x22:       Bits: 64
16:52:14:WU01:FS01:0x22:       Mode: Release
16:52:14:WU01:FS01:0x22:Maintainers: John Chodera <john.chodera@choderalab.org> and Peter Eastman
16:52:14:WU01:FS01:0x22:             <peastman@stanford.edu>
16:52:14:WU01:FS01:0x22:       Args: -dir 01 -suffix 01 -version 706 -lifeline 15928 -checkpoint 3
16:52:14:WU01:FS01:0x22:             -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor
16:52:14:WU01:FS01:0x22:             nvidia -gpu 0 -gpu-usage 100
16:52:14:WU01:FS01:0x22:************************************ libFAH ************************************
16:52:14:WU01:FS01:0x22:       Date: Sep 28 2021
16:52:14:WU01:FS01:0x22:       Time: 05:53:43
16:52:14:WU01:FS01:0x22:   Revision: 44301ed97b996b63fe736bb8073f22209cb2b603
16:52:14:WU01:FS01:0x22:     Branch: HEAD
16:52:14:WU01:FS01:0x22:   Compiler: Visual C++
16:52:14:WU01:FS01:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
16:52:14:WU01:FS01:0x22:   Platform: win32 10
16:52:14:WU01:FS01:0x22:       Bits: 64
16:52:14:WU01:FS01:0x22:       Mode: Release
16:52:14:WU01:FS01:0x22:************************************ CBang *************************************
16:52:14:WU01:FS01:0x22:       Date: Sep 28 2021
16:52:14:WU01:FS01:0x22:       Time: 05:52:38
16:52:14:WU01:FS01:0x22:   Revision: 33fcfc2b3ed2195a423606a264718e31e6b3903f
16:52:14:WU01:FS01:0x22:     Branch: HEAD
16:52:14:WU01:FS01:0x22:   Compiler: Visual C++
16:52:14:WU01:FS01:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
16:52:14:WU01:FS01:0x22:   Platform: win32 10
16:52:14:WU01:FS01:0x22:       Bits: 64
16:52:14:WU01:FS01:0x22:       Mode: Release
16:52:14:WU01:FS01:0x22:************************************ System ************************************
16:52:14:WU01:FS01:0x22:        CPU: Intel(R) Core(TM) i7-3930K CPU @ 3.20GHz
16:52:14:WU01:FS01:0x22:     CPU ID: GenuineIntel Family 6 Model 45 Stepping 7
16:52:14:WU01:FS01:0x22:       CPUs: 12
16:52:14:WU01:FS01:0x22:     Memory: 15.94GiB
16:52:14:WU01:FS01:0x22:Free Memory: 8.22GiB
16:52:14:WU01:FS01:0x22:    Threads: WINDOWS_THREADS
16:52:14:WU01:FS01:0x22: OS Version: 6.2
16:52:14:WU01:FS01:0x22:Has Battery: false
16:52:14:WU01:FS01:0x22: On Battery: false
16:52:14:WU01:FS01:0x22: UTC Offset: -5
16:52:14:WU01:FS01:0x22:        PID: 17936
16:52:14:WU01:FS01:0x22:        CWD: C:\ProgramData\FAHClient\work
16:52:14:WU01:FS01:0x22:************************************ OpenMM ************************************
16:52:14:WU01:FS01:0x22:    Version: 7.6.0
16:52:14:WU01:FS01:0x22:********************************************************************************
16:52:14:WU01:FS01:0x22:Project: 17804 (Run 80, Clone 65, Gen 430)
16:52:14:WU01:FS01:0x22:Unit: 0x00000000000000000000000000000000
16:52:14:WU01:FS01:0x22:Reading tar file core.xml
16:52:14:WU01:FS01:0x22:Reading tar file integrator.xml.bz2
16:52:14:WU01:FS01:0x22:Reading tar file state.xml.bz2
16:52:14:WU01:FS01:0x22:Reading tar file system.xml.bz2
16:52:14:WU01:FS01:0x22:Digital signatures verified
16:52:14:WU01:FS01:0x22:Folding@home GPU Core22 Folding@home Core
16:52:14:WU01:FS01:0x22:Version 0.0.18
16:52:14:WU01:FS01:0x22:  Checkpoint write interval: 125000 steps (5%) [20 total]
16:52:14:WU01:FS01:0x22:  JSON viewer frame write interval: 25000 steps (1%) [100 total]
16:52:14:WU01:FS01:0x22:  XTC frame write interval: 25000 steps (1%) [100 total]
16:52:14:WU01:FS01:0x22:  Global context and integrator variables write interval: disabled
16:52:14:WU01:FS01:0x22:There are 4 platforms available.
16:52:14:WU01:FS01:0x22:Platform 0: Reference
16:52:14:WU01:FS01:0x22:Platform 1: CPU
16:52:14:WU01:FS01:0x22:Platform 2: OpenCL
16:52:14:WU01:FS01:0x22:  opencl-device 0 specified
16:52:14:WU01:FS01:0x22:Platform 3: CUDA
16:52:14:WU01:FS01:0x22:  cuda-device 0 specified
16:52:18:WU03:FS01:Upload 99.87%
16:52:18:WU03:FS01:Upload complete
16:52:18:WU03:FS01:Server responded WORK_ACK (400)
16:52:18:WU03:FS01:Cleaning up
16:52:23:WU01:FS01:0x22:Attempting to create CUDA context:
16:52:23:WU01:FS01:0x22:  Configuring platform CUDA
16:52:35:WU01:FS01:0x22:  Using CUDA and gpu 0
16:52:35:WU01:FS01:0x22:Completed 0 out of 2500000 steps (0%)
16:52:36:WU01:FS01:0x22:Checkpoint completed at step 0
16:54:18:WU01:FS01:0x22:Completed 25000 out of 2500000 steps (1%)
16:55:59:WU01:FS01:0x22:Completed 50000 out of 2500000 steps (2%)
16:57:41:WU01:FS01:0x22:Completed 75000 out of 2500000 steps (3%)
16:59:19:WU01:FS01:0x22:Completed 100000 out of 2500000 steps (4%)
17:00:58:WU01:FS01:0x22:Completed 125000 out of 2500000 steps (5%)
17:00:59:WU01:FS01:0x22:Checkpoint completed at step 125000
17:02:37:WU01:FS01:0x22:Completed 150000 out of 2500000 steps (6%)
17:04:16:WU01:FS01:0x22:Completed 175000 out of 2500000 steps (7%)
17:05:54:WU01:FS01:0x22:Completed 200000 out of 2500000 steps (8%)
17:07:33:WU01:FS01:0x22:Completed 225000 out of 2500000 steps (9%)
17:09:12:WU01:FS01:0x22:Completed 250000 out of 2500000 steps (10%)
17:09:13:WU01:FS01:0x22:Checkpoint completed at step 250000
17:10:52:WU01:FS01:0x22:Completed 275000 out of 2500000 steps (11%)
17:11:38:WU01:FS01:0x22:An exception occurred at step 285888: Particle coordinate is nan
17:11:38:WU01:FS01:0x22:ERROR:98: Attempting to restart from last good checkpoint by restarting core.
17:11:38:WU01:FS01:0x22:Folding@home Core Shutdown: CORE_RESTART
17:11:39:WARNING:WU01:FS01:FahCore returned: CORE_RESTART (98 = 0x62)
17:11:39:WU01:FS01:Starting

it restart a couple time then fail rince and repeat

my card is even underclocked, but i recently updated the driver to get cuda 11.2 and this could be related im gonna try another version