1600 Super - Error loading CUDA module

If you're new to FAH and need help getting started or you have very basic questions, start here.

Moderators: Site Moderators, FAHC Science Team

Post Reply
bozz4science
Posts: 3
Joined: Wed Jan 20, 2021 8:39 pm

1600 Super - Error loading CUDA module

Post by bozz4science »

Dear all,

I am quite new here, and recently deployed a dual-GPU system here (1660 Super and 750Ti) that operate in dual x8 mode. QRB is activated and passkey is stored in the client. My 1660 Super now reaches ~640k PPD and my 750Ti ~230k PPD. It has now been running nonstop ~24hrs, so I waited some time to have those values average out over time. Taking a look in the web client view of the Folding in the dark Chrome extension, it reports my 1660 Super card to be ~40% below average. Thus, I had a look in the log file and it states a couple times that it --> Failed to create CUDA context: --> Error loading CUDA module: CUDA_ERROR_FILE_NOT_FOUND (301) --> Attempting to create OpenCL context. Also, I see this kind of message very often in my logs: --> Watchdog shutdown failed, hard shutdown triggered --> FahCore returned an unknown error code which probably indicates that it crashed --> FahCore returned: WU_STALLED (127 = 0x7f)

Really can't make a whole lot out of these error messages, so I'd appreciate any pointers to what might be causing this, or the PPD seems to imply that the card is working properly and gets science done. Thanks

Code: Select all

*********************** Log Started 2021-01-18T16:06:29Z ***********************
16:06:29:******************************* libFAH ********************************
16:06:29:           Date: Oct 20 2020
16:06:29:           Time: 13:36:55
16:06:29:       Revision: 5ca109d295a6245e2a2f590b3d0085ad5e567aeb
16:06:29:         Branch: master
16:06:29:       Compiler: Visual C++ 2015
16:06:29:        Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Zc:throwingNew /MT
16:06:29:       Platform: win32 10
16:06:29:           Bits: 32
16:06:29:           Mode: Release
16:06:29:****************************** FAHClient ******************************
16:06:29:        Version: 7.6.21
16:06:29:         Author: Joseph Coffland <joseph@cauldrondevelopment.com>
16:06:29:      Copyright: 2020 foldingathome.org
16:06:29:       Homepage: https://foldingathome.org/
16:06:29:           Date: Oct 20 2020
16:06:29:           Time: 13:41:04
16:06:29:       Revision: 6efbf0e138e22d3963e6a291f78dcb9c6422a278
16:06:29:         Branch: master
16:06:29:       Compiler: Visual C++ 2015
16:06:29:        Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Zc:throwingNew /MT
16:06:29:       Platform: win32 10
16:06:29:           Bits: 32
16:06:29:           Mode: Release
16:06:29:           Args: --open-web-control
16:06:29:         Config: C:\ProgramData\FAHClient\config.xml
16:06:29:******************************** CBang ********************************
16:06:29:           Date: Oct 20 2020
16:06:29:           Time: 11:36:18
16:06:29:       Revision: 7e4ce85225d7eaeb775e87c31740181ca603de60
16:06:29:         Branch: master
16:06:29:       Compiler: Visual C++ 2015
16:06:29:        Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Zc:throwingNew /MT
16:06:29:       Platform: win32 10
16:06:29:           Bits: 32
16:06:29:           Mode: Release
16:06:29:******************************* System ********************************
16:06:29:            CPU: AMD Ryzen 7 3700X 8-Core Processor
16:06:29:         CPU ID: AuthenticAMD Family 23 Model 113 Stepping 0
16:06:29:           CPUs: 16
16:06:29:         Memory: 15.92GiB
16:06:29:    Free Memory: 9.71GiB
16:06:29:        Threads: WINDOWS_THREADS
16:06:29:     OS Version: 6.2
16:06:29:    Has Battery: false
16:06:29:     On Battery: false
16:06:29:     UTC Offset: 1
16:06:29:            PID: 14108
16:06:29:            CWD: C:\ProgramData\FAHClient
16:06:29:  Win32 Service: false
16:06:29:             OS: Windows 10 Home
16:06:29:        OS Arch: AMD64
16:06:29:           GPUs: 2
16:06:29:          GPU 0: Bus:8 Slot:0 Func:0 NVIDIA:4 GM107 [GeForce GTX 750 Ti] 1389
16:06:29:          GPU 1: Bus:7 Slot:0 Func:0 NVIDIA:7 TU116 [GeForce GTX 1660 SUPER]
16:06:29:  CUDA Device 0: Platform:0 Device:0 Bus:7 Slot:0 Compute:7.5 Driver:11.2
16:06:29:  CUDA Device 1: Platform:0 Device:1 Bus:8 Slot:0 Compute:5.0 Driver:11.2
16:06:29:OpenCL Device 0: Platform:0 Device:0 Bus:7 Slot:0 Compute:1.2 Driver:461.9
16:06:29:OpenCL Device 1: Platform:0 Device:1 Bus:8 Slot:0 Compute:1.2 Driver:461.9
16:06:29:***********************************************************************
16:06:29:<config>
16:06:29:  <!-- Folding Slot Configuration -->
16:06:29:  <cause v='HIGH_PRIORITY'/>
16:06:29:
16:06:29:  <!-- Network -->
16:06:29:  <proxy v=':8080'/>
16:06:29:
16:06:29:  <!-- Slot Control -->
16:06:29:  <pause-on-battery v='false'/>
16:06:29:  <power v='full'/>
16:06:29:
16:06:29:  <!-- User Information -->
16:06:29:  <passkey v='*****'/>
16:06:29:  <team v='43829'/>
16:06:29:  <user v='bozz4science'/>
16:06:29:
16:06:29:  <!-- Folding Slots -->
16:06:29:  <slot id='0' type='CPU'/>
16:06:29:  <slot id='2' type='GPU'>
16:06:29:    <pci-bus v='7'/>
16:06:29:    <pci-slot v='0'/>
16:06:29:  </slot>
16:06:29:  <slot id='1' type='GPU'>
16:06:29:    <pci-bus v='8'/>
16:06:29:    <pci-slot v='0'/>
16:06:29:  </slot>
16:06:29:</config>
16:06:29:Trying to access database...
16:06:29:Successfully acquired database lock
16:06:29:FS00:Initialized folding slot 00: cpu:14
16:06:29:FS02:Initialized folding slot 02: gpu:7:0 TU116 [GeForce GTX 1660 SUPER]
16:06:29:FS01:Initialized folding slot 01: gpu:8:0 GM107 [GeForce GTX 750 Ti] 1389
16:06:29:WU01:FS01:Starting
16:06:29:WU01:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\ProgramData\FAHClient\cores/cores.foldingathome.org/win/64bit/22-0.0.13/Core_22.fah/FahCore_22.exe -dir 01 -suffix 01 -version 706 -lifeline 14108 -checkpoint 15 -opencl-platform 0 -opencl-device 1 -cuda-device 1 -gpu-vendor nvidia -gpu 1 -gpu-usage 100
16:06:30:WU01:FS01:Started FahCore on PID 10980
16:06:30:WU01:FS01:Core PID:12040
16:06:30:WU01:FS01:FahCore 0x22 started
16:06:30:WU00:FS00:Connecting to assign1.foldingathome.org:80
16:06:30:WU02:FS02:Connecting to assign1.foldingathome.org:80
16:06:31:WU01:FS01:0x22:*********************** Log Started 2021-01-18T16:06:30Z ***********************
16:06:31:WU01:FS01:0x22:*************************** Core22 Folding@home Core ***************************
16:06:31:WU01:FS01:0x22:       Core: Core22
16:06:31:WU01:FS01:0x22:       Type: 0x22
16:06:31:WU01:FS01:0x22:    Version: 0.0.13
16:06:31:WU01:FS01:0x22:     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
16:06:31:WU01:FS01:0x22:  Copyright: 2020 foldingathome.org
16:06:31:WU01:FS01:0x22:   Homepage: https://foldingathome.org/
16:06:31:WU01:FS01:0x22:       Date: Sep 19 2020
16:06:31:WU01:FS01:0x22:       Time: 02:35:58
16:06:31:WU01:FS01:0x22:   Revision: 571cf95de6de2c592c7c3ed48fcfb2e33e9ea7d3
16:06:31:WU01:FS01:0x22:     Branch: core22-0.0.13
16:06:31:WU01:FS01:0x22:   Compiler: Visual C++ 2015
16:06:31:WU01:FS01:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
16:06:31:WU01:FS01:0x22:             -DOPENMM_GIT_HASH="\"189320d0\""
16:06:31:WU01:FS01:0x22:   Platform: win32 10
16:06:31:WU01:FS01:0x22:       Bits: 64
16:06:31:WU01:FS01:0x22:       Mode: Release
16:06:31:WU01:FS01:0x22:Maintainers: John Chodera <john.chodera@choderalab.org> and Peter Eastman
16:06:31:WU01:FS01:0x22:             <peastman@stanford.edu>
16:06:31:WU01:FS01:0x22:       Args: -dir 01 -suffix 01 -version 706 -lifeline 10980 -checkpoint 15
16:06:31:WU01:FS01:0x22:             -opencl-platform 0 -opencl-device 1 -cuda-device 1 -gpu-vendor
16:06:31:WU01:FS01:0x22:             nvidia -gpu 1 -gpu-usage 100
16:06:31:WU01:FS01:0x22:************************************ libFAH ************************************
16:06:31:WU01:FS01:0x22:       Date: Sep 7 2020
16:06:31:WU01:FS01:0x22:       Time: 19:09:56
16:06:31:WU01:FS01:0x22:   Revision: 44301ed97b996b63fe736bb8073f22209cb2b603
16:06:31:WU01:FS01:0x22:     Branch: HEAD
16:06:31:WU01:FS01:0x22:   Compiler: Visual C++ 2015
16:06:31:WU01:FS01:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
16:06:31:WU01:FS01:0x22:   Platform: win32 10
16:06:31:WU01:FS01:0x22:       Bits: 64
16:06:31:WU01:FS01:0x22:       Mode: Release
16:06:31:WU01:FS01:0x22:************************************ CBang *************************************
16:06:31:WU01:FS01:0x22:       Date: Sep 7 2020
16:06:31:WU01:FS01:0x22:       Time: 19:08:30
16:06:31:WU01:FS01:0x22:   Revision: 33fcfc2b3ed2195a423606a264718e31e6b3903f
16:06:31:WU01:FS01:0x22:     Branch: HEAD
16:06:31:WU01:FS01:0x22:   Compiler: Visual C++ 2015
16:06:31:WU01:FS01:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
16:06:31:WU01:FS01:0x22:   Platform: win32 10
16:06:31:WU01:FS01:0x22:       Bits: 64
16:06:31:WU01:FS01:0x22:       Mode: Release
16:06:31:WU01:FS01:0x22:************************************ System ************************************
16:06:31:WU01:FS01:0x22:        CPU: AMD Ryzen 7 3700X 8-Core Processor
16:06:31:WU01:FS01:0x22:     CPU ID: AuthenticAMD Family 23 Model 113 Stepping 0
16:06:31:WU01:FS01:0x22:       CPUs: 16
16:06:31:WU01:FS01:0x22:     Memory: 15.92GiB
16:06:31:WU01:FS01:0x22:Free Memory: 9.62GiB
16:06:31:WU01:FS01:0x22:    Threads: WINDOWS_THREADS
16:06:31:WU01:FS01:0x22: OS Version: 6.2
16:06:31:WU01:FS01:0x22:Has Battery: false
16:06:31:WU01:FS01:0x22: On Battery: false
16:06:31:WU01:FS01:0x22: UTC Offset: 1
16:06:31:WU01:FS01:0x22:        PID: 12040
16:06:31:WU01:FS01:0x22:        CWD: C:\ProgramData\FAHClient\work
16:06:31:WU01:FS01:0x22:************************************ OpenMM ************************************
16:06:31:WU01:FS01:0x22:   Revision: 189320d0
16:06:31:WU01:FS01:0x22:********************************************************************************
16:06:31:WU01:FS01:0x22:Project: 17313 (Run 0, Clone 1942, Gen 139)
16:06:31:WU01:FS01:0x22:Unit: 0x00000000000000000000000000000000
16:06:31:WU01:FS01:0x22:Digital signatures verified
16:06:31:WU01:FS01:0x22:Folding@home GPU Core22 Folding@home Core
16:06:31:WU01:FS01:0x22:Version 0.0.13
16:06:31:WU01:FS01:0x22:  Checkpoint write interval: 25000 steps (2%) [50 total]
16:06:31:WU01:FS01:0x22:  JSON viewer frame write interval: 12500 steps (1%) [100 total]
16:06:31:WU01:FS01:0x22:  XTC frame write interval: 250000 steps (20%) [5 total]
16:06:31:WU01:FS01:0x22:  Global context and integrator variables write interval: disabled
16:06:31:WU01:FS01:0x22:There are 4 platforms available.
16:06:31:WU01:FS01:0x22:Platform 0: Reference
16:06:31:WU01:FS01:0x22:Platform 1: CPU
16:06:31:WU01:FS01:0x22:Platform 2: OpenCL
16:06:31:WU01:FS01:0x22:  opencl-device 1 specified
16:06:31:WU01:FS01:0x22:Platform 3: CUDA
16:06:31:WU01:FS01:0x22:  cuda-device 1 specified
16:06:33:WU02:FS02:Assigned to work server 140.163.4.200
16:06:33:WU02:FS02:Requesting new work unit for slot 02: gpu:7:0 TU116 [GeForce GTX 1660 SUPER] from 140.163.4.200
16:06:33:WU02:FS02:Connecting to 140.163.4.200:8080
16:06:33:WU00:FS00:Assigned to work server 128.252.203.9
16:06:33:WU00:FS00:Requesting new work unit for slot 00: cpu:14 from 128.252.203.9
16:06:33:WU00:FS00:Connecting to 128.252.203.9:8080
16:06:33:5:127.0.0.1:New Web session
16:06:33:WU00:FS00:Downloading 7.54MiB
16:06:33:WU02:FS02:Downloading 26.59MiB
16:06:35:WU01:FS01:0x22:Attempting to create CUDA context:
16:06:35:WU01:FS01:0x22:  Configuring platform CUDA
16:06:35:WU01:FS01:0x22:Failed to create CUDA context:
16:06:35:WU01:FS01:0x22:Error loading CUDA module: CUDA_ERROR_FILE_NOT_FOUND (301)
16:06:35:WU01:FS01:0x22:Attempting to create OpenCL context:
16:06:35:WU01:FS01:0x22:  Configuring platform OpenCL
16:06:36:WU01:FS01:0x22:  Using OpenCL on platformId 0 and gpu 1
16:06:36:WU01:FS01:0x22:Completed 850000 out of 1250000 steps (68%)
16:06:38:FS00:Paused
16:06:39:WU02:FS02:Download 28.21%
16:06:39:WU00:FS00:Download 59.65%
16:06:42:WU00:FS00:Download complete
16:06:42:WU00:FS00:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:17423 run:0 clone:4048 gen:133 core:0xa7 unit:0x00000fd0000000850000440f00000000
16:06:45:WU02:FS02:Download 59.24%
16:06:51:WU02:FS02:Download 92.38%
16:06:52:WU02:FS02:Download complete
16:06:52:WU02:FS02:Received Unit: id:02 state:DOWNLOAD error:NO_ERROR project:17320 run:0 clone:257 gen:85 core:0x22 unit:0x0000010100000055000043a800000000
16:06:52:WU02:FS02:Starting
16:06:52:WU02:FS02:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\ProgramData\FAHClient\cores/cores.foldingathome.org/win/64bit/22-0.0.13/Core_22.fah/FahCore_22.exe -dir 02 -suffix 01 -version 706 -lifeline 14108 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
16:06:52:WU02:FS02:Started FahCore on PID 10804
16:06:52:WU02:FS02:Core PID:13148
16:06:52:WU02:FS02:FahCore 0x22 started
16:06:52:WU02:FS02:0x22:*********************** Log Started 2021-01-18T16:06:52Z ***********************
16:06:53:WU02:FS02:0x22:*************************** Core22 Folding@home Core ***************************
16:06:53:WU02:FS02:0x22:       Core: Core22
16:06:53:WU02:FS02:0x22:       Type: 0x22
16:06:53:WU02:FS02:0x22:    Version: 0.0.13
16:06:53:WU02:FS02:0x22:     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
16:06:53:WU02:FS02:0x22:  Copyright: 2020 foldingathome.org
16:06:53:WU02:FS02:0x22:   Homepage: https://foldingathome.org/
16:06:53:WU02:FS02:0x22:       Date: Sep 19 2020
16:06:53:WU02:FS02:0x22:       Time: 02:35:58
16:06:53:WU02:FS02:0x22:   Revision: 571cf95de6de2c592c7c3ed48fcfb2e33e9ea7d3
16:06:53:WU02:FS02:0x22:     Branch: core22-0.0.13
16:06:53:WU02:FS02:0x22:   Compiler: Visual C++ 2015
16:06:53:WU02:FS02:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
16:06:53:WU02:FS02:0x22:             -DOPENMM_GIT_HASH="\"189320d0\""
16:06:53:WU02:FS02:0x22:   Platform: win32 10
16:06:53:WU02:FS02:0x22:       Bits: 64
16:06:53:WU02:FS02:0x22:       Mode: Release
16:06:53:WU02:FS02:0x22:Maintainers: John Chodera <john.chodera@choderalab.org> and Peter Eastman
16:06:53:WU02:FS02:0x22:             <peastman@stanford.edu>
16:06:53:WU02:FS02:0x22:       Args: -dir 02 -suffix 01 -version 706 -lifeline 10804 -checkpoint 15
16:06:53:WU02:FS02:0x22:             -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor
16:06:53:WU02:FS02:0x22:             nvidia -gpu 0 -gpu-usage 100
16:06:53:WU02:FS02:0x22:************************************ libFAH ************************************
16:06:53:WU02:FS02:0x22:       Date: Sep 7 2020
16:06:53:WU02:FS02:0x22:       Time: 19:09:56
16:06:53:WU02:FS02:0x22:   Revision: 44301ed97b996b63fe736bb8073f22209cb2b603
16:06:53:WU02:FS02:0x22:     Branch: HEAD
16:06:53:WU02:FS02:0x22:   Compiler: Visual C++ 2015
16:06:53:WU02:FS02:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
16:06:53:WU02:FS02:0x22:   Platform: win32 10
16:06:53:WU02:FS02:0x22:       Bits: 64
16:06:53:WU02:FS02:0x22:       Mode: Release
16:06:53:WU02:FS02:0x22:************************************ CBang *************************************
16:06:53:WU02:FS02:0x22:       Date: Sep 7 2020
16:06:53:WU02:FS02:0x22:       Time: 19:08:30
16:06:53:WU02:FS02:0x22:   Revision: 33fcfc2b3ed2195a423606a264718e31e6b3903f
16:06:53:WU02:FS02:0x22:     Branch: HEAD
16:06:53:WU02:FS02:0x22:   Compiler: Visual C++ 2015
16:06:53:WU02:FS02:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
16:06:53:WU02:FS02:0x22:   Platform: win32 10
16:06:53:WU02:FS02:0x22:       Bits: 64
16:06:53:WU02:FS02:0x22:       Mode: Release
16:06:53:WU02:FS02:0x22:************************************ System ************************************
16:06:53:WU02:FS02:0x22:        CPU: AMD Ryzen 7 3700X 8-Core Processor
16:06:53:WU02:FS02:0x22:     CPU ID: AuthenticAMD Family 23 Model 113 Stepping 0
16:06:53:WU02:FS02:0x22:       CPUs: 16
16:06:53:WU02:FS02:0x22:     Memory: 15.92GiB
16:06:53:WU02:FS02:0x22:Free Memory: 8.95GiB
16:06:53:WU02:FS02:0x22:    Threads: WINDOWS_THREADS
16:06:53:WU02:FS02:0x22: OS Version: 6.2
16:06:53:WU02:FS02:0x22:Has Battery: false
16:06:53:WU02:FS02:0x22: On Battery: false
16:06:53:WU02:FS02:0x22: UTC Offset: 1
16:06:53:WU02:FS02:0x22:        PID: 13148
16:06:53:WU02:FS02:0x22:        CWD: C:\ProgramData\FAHClient\work
16:06:53:WU02:FS02:0x22:************************************ OpenMM ************************************
16:06:53:WU02:FS02:0x22:   Revision: 189320d0
16:06:53:WU02:FS02:0x22:********************************************************************************
16:06:53:WU02:FS02:0x22:Project: 17320 (Run 0, Clone 257, Gen 85)
16:06:53:WU02:FS02:0x22:Unit: 0x00000000000000000000000000000000
16:06:53:WU02:FS02:0x22:Reading tar file core.xml
16:06:53:WU02:FS02:0x22:Reading tar file integrator.xml.bz2
16:06:53:WU02:FS02:0x22:Reading tar file state.xml.bz2
16:06:53:WU02:FS02:0x22:Reading tar file system.xml.bz2
16:06:53:WU02:FS02:0x22:Digital signatures verified
16:06:53:WU02:FS02:0x22:Folding@home GPU Core22 Folding@home Core
16:06:53:WU02:FS02:0x22:Version 0.0.13
16:06:53:WU02:FS02:0x22:  Checkpoint write interval: 25000 steps (2%) [50 total]
16:06:53:WU02:FS02:0x22:  JSON viewer frame write interval: 12500 steps (1%) [100 total]
16:06:53:WU02:FS02:0x22:  XTC frame write interval: 250000 steps (20%) [5 total]
16:06:53:WU02:FS02:0x22:  Global context and integrator variables write interval: disabled
16:06:53:WU02:FS02:0x22:There are 4 platforms available.
16:06:53:WU02:FS02:0x22:Platform 0: Reference
16:06:53:WU02:FS02:0x22:Platform 1: CPU
16:06:53:WU02:FS02:0x22:Platform 2: OpenCL
16:06:53:WU02:FS02:0x22:  opencl-device 0 specified
16:06:53:WU02:FS02:0x22:Platform 3: CUDA
16:06:53:WU02:FS02:0x22:  cuda-device 0 specified
16:07:19:WU02:FS02:0x22:Attempting to create CUDA context:
16:07:19:WU02:FS02:0x22:  Configuring platform CUDA
16:07:19:WU02:FS02:0x22:Failed to create CUDA context:
16:07:19:WU02:FS02:0x22:Error loading CUDA module: CUDA_ERROR_FILE_NOT_FOUND (301)
16:07:19:WU02:FS02:0x22:Attempting to create OpenCL context:
16:07:19:WU02:FS02:0x22:  Configuring platform OpenCL
16:07:30:Removing old file 'configs/config-20210108-224542.xml'
16:07:30:Saving configuration to config.xml
16:07:30:<config>
16:07:30:  <!-- Folding Slot Configuration -->
16:07:30:  <cause v='HIGH_PRIORITY'/>
16:07:30:
16:07:30:  <!-- Network -->
16:07:30:  <proxy v=':8080'/>
16:07:30:
16:07:30:  <!-- Slot Control -->
16:07:30:  <pause-on-battery v='false'/>
16:07:30:  <power v='full'/>
16:07:30:
16:07:30:  <!-- User Information -->
16:07:30:  <passkey v='*****'/>
16:07:30:  <team v='43829'/>
16:07:30:  <user v='bozz4science'/>
16:07:30:
16:07:30:  <!-- Folding Slots -->
16:07:30:  <slot id='0' type='CPU'>
16:07:30:    <paused v='true'/>
16:07:30:  </slot>
16:07:30:  <slot id='2' type='GPU'>
16:07:30:    <pci-bus v='7'/>
16:07:30:    <pci-slot v='0'/>
16:07:30:  </slot>
16:07:30:  <slot id='1' type='GPU'>
16:07:30:    <pci-bus v='8'/>
16:07:30:    <pci-slot v='0'/>
16:07:30:  </slot>
16:07:30:</config>
16:07:33:WU02:FS02:0x22:  Using OpenCL on platformId 0 and gpu 0
16:07:33:WU02:FS02:0x22:Completed 0 out of 1250000 steps (0%)
16:07:35:WU02:FS02:0x22:Checkpoint completed at step 0
16:08:53:WU01:FS01:0x22:Completed 862500 out of 1250000 steps (69%)
16:11:08:WU01:FS01:0x22:Completed 875000 out of 1250000 steps (70%)
16:11:08:WU01:FS01:0x22:Checkpoint completed at step 875000
16:13:07:WU02:FS02:0x22:Completed 12500 out of 1250000 steps (1%)
16:13:23:WU01:FS01:0x22:Completed 887500 out of 1250000 steps (71%)
16:15:39:WU01:FS01:0x22:Completed 900000 out of 1250000 steps (72%)
16:15:39:WU01:FS01:0x22:Checkpoint completed at step 900000
16:17:54:WU01:FS01:0x22:Completed 912500 out of 1250000 steps (73%)
16:18:32:WU02:FS02:0x22:Completed 25000 out of 1250000 steps (2%)
16:18:35:WU02:FS02:0x22:Checkpoint completed at step 25000
16:20:09:WU01:FS01:0x22:Completed 925000 out of 1250000 steps (74%)
16:20:10:WU01:FS01:0x22:Checkpoint completed at step 925000
16:22:25:WU01:FS01:0x22:Completed 937500 out of 1250000 steps (75%)
16:24:01:WU02:FS02:0x22:Completed 37500 out of 1250000 steps (3%)
16:24:40:WU01:FS01:0x22:Completed 950000 out of 1250000 steps (76%)
16:24:40:WU01:FS01:0x22:Checkpoint completed at step 950000
16:26:55:WU01:FS01:0x22:Completed 962500 out of 1250000 steps (77%)
16:29:11:WU01:FS01:0x22:Completed 975000 out of 1250000 steps (78%)
16:29:11:WU01:FS01:0x22:Checkpoint completed at step 975000
16:29:25:WU02:FS02:0x22:Completed 50000 out of 1250000 steps (4%)
16:29:28:WU02:FS02:0x22:Checkpoint completed at step 50000
16:31:27:WU01:FS01:0x22:Completed 987500 out of 1250000 steps (79%)
16:33:42:WU01:FS01:0x22:Completed 1000000 out of 1250000 steps (80%)
16:33:42:WU01:FS01:0x22:Checkpoint completed at step 1000000
16:34:50:WU02:FS02:0x22:Completed 62500 out of 1250000 steps (5%)
16:35:57:WU01:FS01:0x22:Completed 1012500 out of 1250000 steps (81%)
16:38:12:WU01:FS01:0x22:Completed 1025000 out of 1250000 steps (82%)
16:38:13:WU01:FS01:0x22:Checkpoint completed at step 1025000
16:40:13:WU02:FS02:0x22:Completed 75000 out of 1250000 steps (6%)
16:40:16:WU02:FS02:0x22:Checkpoint completed at step 75000
16:40:28:WU01:FS01:0x22:Completed 1037500 out of 1250000 steps (83%)
16:42:44:WU01:FS01:0x22:Completed 1050000 out of 1250000 steps (84%)
16:42:44:WU01:FS01:0x22:Checkpoint completed at step 1050000
16:44:59:WU01:FS01:0x22:Completed 1062500 out of 1250000 steps (85%)
16:45:41:WU02:FS02:0x22:Completed 87500 out of 1250000 steps (7%)
16:47:14:WU01:FS01:0x22:Completed 1075000 out of 1250000 steps (86%)
16:47:14:WU01:FS01:0x22:Checkpoint completed at step 1075000
16:49:30:WU01:FS01:0x22:Completed 1087500 out of 1250000 steps (87%)
16:51:06:WU02:FS02:0x22:Completed 100000 out of 1250000 steps (8%)
16:51:09:WU02:FS02:0x22:Checkpoint completed at step 100000
16:51:46:WU01:FS01:0x22:Completed 1100000 out of 1250000 steps (88%)
16:51:46:WU01:FS01:0x22:Checkpoint completed at step 1100000
16:54:02:WU01:FS01:0x22:Completed 1112500 out of 1250000 steps (89%)
16:56:17:WU01:FS01:0x22:Completed 1125000 out of 1250000 steps (90%)
16:56:17:WU01:FS01:0x22:Checkpoint completed at step 1125000
16:58:29:FS01:Finishing
16:58:32:WU01:FS01:0x22:Completed 1137500 out of 1250000 steps (91%)
17:00:47:WU01:FS01:0x22:Completed 1150000 out of 1250000 steps (92%)
17:00:48:WU01:FS01:0x22:Checkpoint completed at step 1150000
17:03:03:WU01:FS01:0x22:Completed 1162500 out of 1250000 steps (93%)
17:03:31:WU02:FS02:0x22:Watchdog triggered, requesting soft shutdown down
17:04:26:FS01:Paused
17:04:26:FS01:Shutting core down
17:04:26:WU01:FS01:0x22:WARNING:Console control signal 1 on PID 12040
17:04:26:WU01:FS01:0x22:Exiting, please wait. . .
17:04:26:WU01:FS01:0x22:Folding@home Core Shutdown: INTERRUPTED
17:04:26:WU01:FS01:FahCore returned: INTERRUPTED (102 = 0x66)
17:04:43:FS01:Unpaused
17:04:43:WU01:FS01:Starting
17:04:43:WU01:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\ProgramData\FAHClient\cores/cores.foldingathome.org/win/64bit/22-0.0.13/Core_22.fah/FahCore_22.exe -dir 01 -suffix 01 -version 706 -lifeline 14108 -checkpoint 15 -opencl-platform 0 -opencl-device 1 -cuda-device 1 -gpu-vendor nvidia -gpu 1 -gpu-usage 100
17:04:44:WU01:FS01:Started FahCore on PID 8756
17:04:44:WU01:FS01:Core PID:9716
17:04:44:WU01:FS01:FahCore 0x22 started
17:04:45:WU01:FS01:0x22:*********************** Log Started 2021-01-18T17:04:44Z ***********************
17:04:45:WU01:FS01:0x22:*************************** Core22 Folding@home Core ***************************
17:04:45:WU01:FS01:0x22:       Core: Core22
17:04:45:WU01:FS01:0x22:       Type: 0x22
17:04:45:WU01:FS01:0x22:    Version: 0.0.13
17:04:45:WU01:FS01:0x22:     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
17:04:45:WU01:FS01:0x22:  Copyright: 2020 foldingathome.org
17:04:45:WU01:FS01:0x22:   Homepage: https://foldingathome.org/
17:04:45:WU01:FS01:0x22:       Date: Sep 19 2020
17:04:45:WU01:FS01:0x22:       Time: 02:35:58
17:04:45:WU01:FS01:0x22:   Revision: 571cf95de6de2c592c7c3ed48fcfb2e33e9ea7d3
17:04:45:WU01:FS01:0x22:     Branch: core22-0.0.13
17:04:45:WU01:FS01:0x22:   Compiler: Visual C++ 2015
17:04:45:WU01:FS01:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
17:04:45:WU01:FS01:0x22:             -DOPENMM_GIT_HASH="\"189320d0\""
17:04:45:WU01:FS01:0x22:   Platform: win32 10
17:04:45:WU01:FS01:0x22:       Bits: 64
17:04:45:WU01:FS01:0x22:       Mode: Release
17:04:45:WU01:FS01:0x22:Maintainers: John Chodera <john.chodera@choderalab.org> and Peter Eastman
17:04:45:WU01:FS01:0x22:             <peastman@stanford.edu>
17:04:45:WU01:FS01:0x22:       Args: -dir 01 -suffix 01 -version 706 -lifeline 8756 -checkpoint 15
17:04:45:WU01:FS01:0x22:             -opencl-platform 0 -opencl-device 1 -cuda-device 1 -gpu-vendor
17:04:45:WU01:FS01:0x22:             nvidia -gpu 1 -gpu-usage 100
17:04:45:WU01:FS01:0x22:************************************ libFAH ************************************
17:04:45:WU01:FS01:0x22:       Date: Sep 7 2020
17:04:45:WU01:FS01:0x22:       Time: 19:09:56
17:04:45:WU01:FS01:0x22:   Revision: 44301ed97b996b63fe736bb8073f22209cb2b603
17:04:45:WU01:FS01:0x22:     Branch: HEAD
17:04:45:WU01:FS01:0x22:   Compiler: Visual C++ 2015
17:04:45:WU01:FS01:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
17:04:45:WU01:FS01:0x22:   Platform: win32 10
17:04:45:WU01:FS01:0x22:       Bits: 64
17:04:45:WU01:FS01:0x22:       Mode: Release
17:04:45:WU01:FS01:0x22:************************************ CBang *************************************
17:04:45:WU01:FS01:0x22:       Date: Sep 7 2020
17:04:45:WU01:FS01:0x22:       Time: 19:08:30
17:04:45:WU01:FS01:0x22:   Revision: 33fcfc2b3ed2195a423606a264718e31e6b3903f
17:04:45:WU01:FS01:0x22:     Branch: HEAD
17:04:45:WU01:FS01:0x22:   Compiler: Visual C++ 2015
17:04:45:WU01:FS01:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
17:04:45:WU01:FS01:0x22:   Platform: win32 10
17:04:45:WU01:FS01:0x22:       Bits: 64
17:04:45:WU01:FS01:0x22:       Mode: Release
17:04:45:WU01:FS01:0x22:************************************ System ************************************
17:04:45:WU01:FS01:0x22:        CPU: AMD Ryzen 7 3700X 8-Core Processor
17:04:45:WU01:FS01:0x22:     CPU ID: AuthenticAMD Family 23 Model 113 Stepping 0
17:04:45:WU01:FS01:0x22:       CPUs: 16
17:04:45:WU01:FS01:0x22:     Memory: 15.92GiB
17:04:45:WU01:FS01:0x22:Free Memory: 6.29GiB
17:04:45:WU01:FS01:0x22:    Threads: WINDOWS_THREADS
17:04:45:WU01:FS01:0x22: OS Version: 6.2
17:04:45:WU01:FS01:0x22:Has Battery: false
17:04:45:WU01:FS01:0x22: On Battery: false
17:04:45:WU01:FS01:0x22: UTC Offset: 1
17:04:45:WU01:FS01:0x22:        PID: 9716
17:04:45:WU01:FS01:0x22:        CWD: C:\ProgramData\FAHClient\work
17:04:45:WU01:FS01:0x22:************************************ OpenMM ************************************
17:04:45:WU01:FS01:0x22:   Revision: 189320d0
17:04:45:WU01:FS01:0x22:********************************************************************************
17:04:45:WU01:FS01:0x22:Project: 17313 (Run 0, Clone 1942, Gen 139)
17:04:45:WU01:FS01:0x22:Unit: 0x00000000000000000000000000000000
17:04:45:WU01:FS01:0x22:Digital signatures verified
17:04:45:WU01:FS01:0x22:Folding@home GPU Core22 Folding@home Core
17:04:45:WU01:FS01:0x22:Version 0.0.13
17:04:45:WU01:FS01:0x22:  Checkpoint write interval: 25000 steps (2%) [50 total]
17:04:45:WU01:FS01:0x22:  JSON viewer frame write interval: 12500 steps (1%) [100 total]
17:04:45:WU01:FS01:0x22:  XTC frame write interval: 250000 steps (20%) [5 total]
17:04:45:WU01:FS01:0x22:  Global context and integrator variables write interval: disabled
17:04:45:WU01:FS01:0x22:There are 4 platforms available.
17:04:45:WU01:FS01:0x22:Platform 0: Reference
17:04:45:WU01:FS01:0x22:Platform 1: CPU
17:04:45:WU01:FS01:0x22:Platform 2: OpenCL
17:04:45:WU01:FS01:0x22:  opencl-device 1 specified
17:04:45:WU01:FS01:0x22:Platform 3: CUDA
17:04:45:WU01:FS01:0x22:  cuda-device 1 specified
17:04:48:WU01:FS01:0x22:Attempting to create CUDA context:
17:04:48:WU01:FS01:0x22:  Configuring platform CUDA
17:04:48:WU01:FS01:0x22:Failed to create CUDA context:
17:04:48:WU01:FS01:0x22:Error loading CUDA module: CUDA_ERROR_FILE_NOT_FOUND (301)
17:04:48:WU01:FS01:0x22:Attempting to create OpenCL context:
17:04:48:WU01:FS01:0x22:  Configuring platform OpenCL
17:04:49:FS02:Paused
17:04:49:FS02:Shutting core down
17:04:49:WU02:FS02:0x22:WARNING:Console control signal 1 on PID 13148
17:04:49:WU02:FS02:0x22:Exiting, please wait. . .
17:04:50:WU01:FS01:0x22:  Using OpenCL on platformId 0 and gpu 1
17:04:50:WU01:FS01:0x22:Completed 1150000 out of 1250000 steps (92%)
17:05:27:Removing old file 'configs/config-20210109-134022.xml'
17:05:27:Saving configuration to config.xml
17:05:27:<config>
17:05:27:  <!-- Folding Slot Configuration -->
17:05:27:  <cause v='HIGH_PRIORITY'/>
17:05:27:
17:05:27:  <!-- Network -->
17:05:27:  <proxy v=':8080'/>
17:05:27:
17:05:27:  <!-- Slot Control -->
17:05:27:  <pause-on-battery v='false'/>
17:05:27:  <power v='full'/>
17:05:27:
17:05:27:  <!-- User Information -->
17:05:27:  <passkey v='*****'/>
17:05:27:  <team v='43829'/>
17:05:27:  <user v='bozz4science'/>
17:05:27:
17:05:27:  <!-- Folding Slots -->
17:05:27:  <slot id='0' type='CPU'>
17:05:27:    <paused v='true'/>
17:05:27:  </slot>
17:05:27:  <slot id='2' type='GPU'>
17:05:27:    <paused v='true'/>
17:05:27:    <pci-bus v='7'/>
17:05:27:    <pci-slot v='0'/>
17:05:27:  </slot>
17:05:27:  <slot id='1' type='GPU'>
17:05:27:    <pci-bus v='8'/>
17:05:27:    <pci-slot v='0'/>
17:05:27:  </slot>
17:05:27:</config>
17:05:50:WARNING:FS02:Killing WU02
17:05:50:WU02:FS02:FahCore returned: INTERRUPTED (102 = 0x66)
17:06:45:FS02:Unpaused
17:06:45:WU02:FS02:Starting
17:06:45:WU02:FS02:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\ProgramData\FAHClient\cores/cores.foldingathome.org/win/64bit/22-0.0.13/Core_22.fah/FahCore_22.exe -dir 02 -suffix 01 -version 706 -lifeline 14108 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
17:06:45:WU02:FS02:Started FahCore on PID 9640
17:06:45:WU02:FS02:Core PID:7712
17:06:45:WU02:FS02:FahCore 0x22 started
17:06:45:WU02:FS02:0x22:*********************** Log Started 2021-01-18T17:06:45Z ***********************
17:06:45:WU02:FS02:0x22:*************************** Core22 Folding@home Core ***************************
17:06:45:WU02:FS02:0x22:       Core: Core22
17:06:45:WU02:FS02:0x22:       Type: 0x22
17:06:45:WU02:FS02:0x22:    Version: 0.0.13
17:06:45:WU02:FS02:0x22:     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
17:06:45:WU02:FS02:0x22:  Copyright: 2020 foldingathome.org
17:06:45:WU02:FS02:0x22:   Homepage: https://foldingathome.org/
17:06:45:WU02:FS02:0x22:       Date: Sep 19 2020
17:06:45:WU02:FS02:0x22:       Time: 02:35:58
17:06:45:WU02:FS02:0x22:   Revision: 571cf95de6de2c592c7c3ed48fcfb2e33e9ea7d3
17:06:45:WU02:FS02:0x22:     Branch: core22-0.0.13
17:06:45:WU02:FS02:0x22:   Compiler: Visual C++ 2015
17:06:45:WU02:FS02:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
17:06:45:WU02:FS02:0x22:             -DOPENMM_GIT_HASH="\"189320d0\""
17:06:45:WU02:FS02:0x22:   Platform: win32 10
17:06:45:WU02:FS02:0x22:       Bits: 64
17:06:45:WU02:FS02:0x22:       Mode: Release
17:06:45:WU02:FS02:0x22:Maintainers: John Chodera <john.chodera@choderalab.org> and Peter Eastman
17:06:45:WU02:FS02:0x22:             <peastman@stanford.edu>
17:06:45:WU02:FS02:0x22:       Args: -dir 02 -suffix 01 -version 706 -lifeline 9640 -checkpoint 15
17:06:45:WU02:FS02:0x22:             -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor
17:06:45:WU02:FS02:0x22:             nvidia -gpu 0 -gpu-usage 100
17:06:45:WU02:FS02:0x22:************************************ libFAH ************************************
17:06:45:WU02:FS02:0x22:       Date: Sep 7 2020
17:06:45:WU02:FS02:0x22:       Time: 19:09:56
17:06:45:WU02:FS02:0x22:   Revision: 44301ed97b996b63fe736bb8073f22209cb2b603
17:06:45:WU02:FS02:0x22:     Branch: HEAD
17:06:45:WU02:FS02:0x22:   Compiler: Visual C++ 2015
17:06:45:WU02:FS02:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
17:06:45:WU02:FS02:0x22:   Platform: win32 10
17:06:45:WU02:FS02:0x22:       Bits: 64
17:06:45:WU02:FS02:0x22:       Mode: Release
17:06:45:WU02:FS02:0x22:************************************ CBang *************************************
17:06:45:WU02:FS02:0x22:       Date: Sep 7 2020
17:06:45:WU02:FS02:0x22:       Time: 19:08:30
17:06:45:WU02:FS02:0x22:   Revision: 33fcfc2b3ed2195a423606a264718e31e6b3903f
17:06:45:WU02:FS02:0x22:     Branch: HEAD
17:06:45:WU02:FS02:0x22:   Compiler: Visual C++ 2015
17:06:45:WU02:FS02:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
17:06:45:WU02:FS02:0x22:   Platform: win32 10
17:06:45:WU02:FS02:0x22:       Bits: 64
17:06:45:WU02:FS02:0x22:       Mode: Release
17:06:45:WU02:FS02:0x22:************************************ System ************************************
17:06:45:WU02:FS02:0x22:        CPU: AMD Ryzen 7 3700X 8-Core Processor
17:06:45:WU02:FS02:0x22:     CPU ID: AuthenticAMD Family 23 Model 113 Stepping 0
17:06:45:WU02:FS02:0x22:       CPUs: 16
17:06:45:WU02:FS02:0x22:     Memory: 15.92GiB
17:06:45:WU02:FS02:0x22:Free Memory: 6.92GiB
17:06:45:WU02:FS02:0x22:    Threads: WINDOWS_THREADS
17:06:45:WU02:FS02:0x22: OS Version: 6.2
17:06:45:WU02:FS02:0x22:Has Battery: false
17:06:45:WU02:FS02:0x22: On Battery: false
17:06:45:WU02:FS02:0x22: UTC Offset: 1
17:06:45:WU02:FS02:0x22:        PID: 7712
17:06:45:WU02:FS02:0x22:        CWD: C:\ProgramData\FAHClient\work
17:06:45:WU02:FS02:0x22:************************************ OpenMM ************************************
17:06:45:WU02:FS02:0x22:   Revision: 189320d0
17:06:45:WU02:FS02:0x22:********************************************************************************
17:06:45:WU02:FS02:0x22:Project: 17320 (Run 0, Clone 257, Gen 85)
17:06:45:WU02:FS02:0x22:Unit: 0x00000000000000000000000000000000
17:06:45:WU02:FS02:0x22:Digital signatures verified
17:06:45:WU02:FS02:0x22:Folding@home GPU Core22 Folding@home Core
17:06:45:WU02:FS02:0x22:Version 0.0.13
17:06:45:WU02:FS02:0x22:  Checkpoint write interval: 25000 steps (2%) [50 total]
17:06:45:WU02:FS02:0x22:  JSON viewer frame write interval: 12500 steps (1%) [100 total]
17:06:45:WU02:FS02:0x22:  XTC frame write interval: 250000 steps (20%) [5 total]
17:06:45:WU02:FS02:0x22:  Global context and integrator variables write interval: disabled
17:06:45:WU02:FS02:0x22:There are 4 platforms available.
17:06:45:WU02:FS02:0x22:Platform 0: Reference
17:06:45:WU02:FS02:0x22:Platform 1: CPU
17:06:45:WU02:FS02:0x22:Platform 2: OpenCL
17:06:45:WU02:FS02:0x22:  opencl-device 0 specified
17:06:45:WU02:FS02:0x22:Platform 3: CUDA
17:06:45:WU02:FS02:0x22:  cuda-device 0 specified
17:07:05:WU01:FS01:0x22:Completed 1162500 out of 1250000 steps (93%)
17:07:11:WU02:FS02:0x22:Attempting to create CUDA context:
17:07:11:WU02:FS02:0x22:  Configuring platform CUDA
17:07:11:WU02:FS02:0x22:Failed to create CUDA context:
17:07:11:WU02:FS02:0x22:Error loading CUDA module: CUDA_ERROR_FILE_NOT_FOUND (301)
17:07:11:WU02:FS02:0x22:Attempting to create OpenCL context:
17:07:11:WU02:FS02:0x22:  Configuring platform OpenCL
17:07:19:WU02:FS02:0x22:  Using OpenCL on platformId 0 and gpu 0
17:07:20:WU02:FS02:0x22:Completed 100000 out of 1250000 steps (8%)
17:07:29:Removing old file 'configs/config-20210109-134224.xml'
17:07:29:Saving configuration to config.xml
17:07:29:<config>
17:07:29:  <!-- Folding Slot Configuration -->
17:07:29:  <cause v='HIGH_PRIORITY'/>
17:07:29:
17:07:29:  <!-- Network -->
17:07:29:  <proxy v=':8080'/>
17:07:29:
17:07:29:  <!-- Slot Control -->
17:07:29:  <pause-on-battery v='false'/>
17:07:29:  <power v='full'/>
17:07:29:
17:07:29:  <!-- User Information -->
17:07:29:  <passkey v='*****'/>
17:07:29:  <team v='43829'/>
17:07:29:  <user v='bozz4science'/>
17:07:29:
17:07:29:  <!-- Folding Slots -->
17:07:29:  <slot id='0' type='CPU'>
17:07:29:    <paused v='true'/>
17:07:29:  </slot>
17:07:29:  <slot id='2' type='GPU'>
17:07:29:    <pci-bus v='7'/>
17:07:29:    <pci-slot v='0'/>
17:07:29:  </slot>
17:07:29:  <slot id='1' type='GPU'>
17:07:29:    <pci-bus v='8'/>
17:07:29:    <pci-slot v='0'/>
17:07:29:  </slot>
17:07:29:</config>
17:09:20:WU01:FS01:0x22:Completed 1175000 out of 1250000 steps (94%)
17:09:20:WU01:FS01:0x22:Checkpoint completed at step 1175000
17:11:35:WU01:FS01:0x22:Completed 1187500 out of 1250000 steps (95%)
17:12:46:WU02:FS02:0x22:Completed 112500 out of 1250000 steps (9%)
17:13:51:WU01:FS01:0x22:Completed 1200000 out of 1250000 steps (96%)
17:13:51:WU01:FS01:0x22:Checkpoint completed at step 1200000
17:16:06:WU01:FS01:0x22:Completed 1212500 out of 1250000 steps (97%)
17:17:07:FS02:Finishing
17:18:13:WU02:FS02:0x22:Completed 125000 out of 1250000 steps (10%)
17:18:16:WU02:FS02:0x22:Checkpoint completed at step 125000
17:18:21:WU01:FS01:0x22:Completed 1225000 out of 1250000 steps (98%)
17:18:22:WU01:FS01:0x22:Checkpoint completed at step 1225000
17:20:37:WU01:FS01:0x22:Completed 1237500 out of 1250000 steps (99%)
17:20:38:WU03:FS01:Connecting to assign1.foldingathome.org:80
17:20:39:WU03:FS01:Assigned to work server 128.174.73.74
17:20:39:WU03:FS01:Requesting new work unit for slot 01: gpu:8:0 GM107 [GeForce GTX 750 Ti] 1389 from 128.174.73.74
17:20:39:WU03:FS01:Connecting to 128.174.73.74:8080
17:20:46:WU03:FS01:Downloading 13.01MiB
17:20:52:WU03:FS01:Download 60.06%
17:20:55:WU03:FS01:Download complete
17:20:55:WU03:FS01:Received Unit: id:03 state:DOWNLOAD error:NO_ERROR project:14915 run:0 clone:1023 gen:0 core:0x22 unit:0x0000000080ae494a5fdd30ace37d0128
17:22:52:WU01:FS01:0x22:Completed 1250000 out of 1250000 steps (100%)
17:22:52:WU01:FS01:0x22:Average performance: 318.819 ns/day
17:22:52:WU01:FS01:0x22:Checkpoint completed at step 1250000
17:22:56:WU01:FS01:0x22:Saving result file ..\logfile_01.txt
17:22:56:WU01:FS01:0x22:Saving result file checkpointIntegrator.xml.bz2
17:22:56:WU01:FS01:0x22:Saving result file checkpointState.xml.bz2
17:22:56:WU01:FS01:0x22:Saving result file positions.xtc
17:22:56:WU01:FS01:0x22:Saving result file science.log
17:22:56:WU01:FS01:0x22:Folding@home Core Shutdown: FINISHED_UNIT
17:22:56:WU01:FS01:FahCore returned: FINISHED_UNIT (100 = 0x64)
17:22:56:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:17313 run:0 clone:1942 gen:139 core:0x22 unit:0x000007960000008b000043a100000000
17:22:56:WU01:FS01:Uploading 3.48MiB to 140.163.4.200
17:22:56:WU01:FS01:Connecting to 140.163.4.200:8080
17:22:56:WU03:FS01:Starting
17:22:56:WU03:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\ProgramData\FAHClient\cores/cores.foldingathome.org/win/64bit/22-0.0.13/Core_22.fah/FahCore_22.exe -dir 03 -suffix 01 -version 706 -lifeline 14108 -checkpoint 15 -opencl-platform 0 -opencl-device 1 -cuda-device 1 -gpu-vendor nvidia -gpu 1 -gpu-usage 100
17:22:56:WU03:FS01:Started FahCore on PID 12740
17:22:56:WU03:FS01:Core PID:11152
17:22:56:WU03:FS01:FahCore 0x22 started
17:22:57:WU03:FS01:0x22:*********************** Log Started 2021-01-18T17:22:56Z ***********************
17:22:57:WU03:FS01:0x22:*************************** Core22 Folding@home Core ***************************
17:22:57:WU03:FS01:0x22:       Core: Core22
17:22:57:WU03:FS01:0x22:       Type: 0x22
17:22:57:WU03:FS01:0x22:    Version: 0.0.13
17:22:57:WU03:FS01:0x22:     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
17:22:57:WU03:FS01:0x22:  Copyright: 2020 foldingathome.org
17:22:57:WU03:FS01:0x22:   Homepage: https://foldingathome.org/
17:22:57:WU03:FS01:0x22:       Date: Sep 19 2020
17:22:57:WU03:FS01:0x22:       Time: 02:35:58
17:22:57:WU03:FS01:0x22:   Revision: 571cf95de6de2c592c7c3ed48fcfb2e33e9ea7d3
17:22:57:WU03:FS01:0x22:     Branch: core22-0.0.13
17:22:57:WU03:FS01:0x22:   Compiler: Visual C++ 2015
17:22:57:WU03:FS01:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
17:22:57:WU03:FS01:0x22:             -DOPENMM_GIT_HASH="\"189320d0\""
17:22:57:WU03:FS01:0x22:   Platform: win32 10
17:22:57:WU03:FS01:0x22:       Bits: 64
17:22:57:WU03:FS01:0x22:       Mode: Release
17:22:57:WU03:FS01:0x22:Maintainers: John Chodera <john.chodera@choderalab.org> and Peter Eastman
17:22:57:WU03:FS01:0x22:             <peastman@stanford.edu>
17:22:57:WU03:FS01:0x22:       Args: -dir 03 -suffix 01 -version 706 -lifeline 12740 -checkpoint 15
17:22:57:WU03:FS01:0x22:             -opencl-platform 0 -opencl-device 1 -cuda-device 1 -gpu-vendor
17:22:57:WU03:FS01:0x22:             nvidia -gpu 1 -gpu-usage 100
17:22:57:WU03:FS01:0x22:************************************ libFAH ************************************
17:22:57:WU03:FS01:0x22:       Date: Sep 7 2020
17:22:57:WU03:FS01:0x22:       Time: 19:09:56
17:22:57:WU03:FS01:0x22:   Revision: 44301ed97b996b63fe736bb8073f22209cb2b603
17:22:57:WU03:FS01:0x22:     Branch: HEAD
17:22:57:WU03:FS01:0x22:   Compiler: Visual C++ 2015
17:22:57:WU03:FS01:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
17:22:57:WU03:FS01:0x22:   Platform: win32 10
17:22:57:WU03:FS01:0x22:       Bits: 64
17:22:57:WU03:FS01:0x22:       Mode: Release
17:22:57:WU03:FS01:0x22:************************************ CBang *************************************
17:22:57:WU03:FS01:0x22:       Date: Sep 7 2020
17:22:57:WU03:FS01:0x22:       Time: 19:08:30
17:22:57:WU03:FS01:0x22:   Revision: 33fcfc2b3ed2195a423606a264718e31e6b3903f
17:22:57:WU03:FS01:0x22:     Branch: HEAD
17:22:57:WU03:FS01:0x22:   Compiler: Visual C++ 2015
17:22:57:WU03:FS01:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
17:22:57:WU03:FS01:0x22:   Platform: win32 10
17:22:57:WU03:FS01:0x22:       Bits: 64
17:22:57:WU03:FS01:0x22:       Mode: Release
17:22:57:WU03:FS01:0x22:************************************ System ************************************
17:22:57:WU03:FS01:0x22:        CPU: AMD Ryzen 7 3700X 8-Core Processor
17:22:57:WU03:FS01:0x22:     CPU ID: AuthenticAMD Family 23 Model 113 Stepping 0
17:22:57:WU03:FS01:0x22:       CPUs: 16
17:22:57:WU03:FS01:0x22:     Memory: 15.92GiB
17:22:57:WU03:FS01:0x22:Free Memory: 6.78GiB
17:22:57:WU03:FS01:0x22:    Threads: WINDOWS_THREADS
17:22:57:WU03:FS01:0x22: OS Version: 6.2
17:22:57:WU03:FS01:0x22:Has Battery: false
17:22:57:WU03:FS01:0x22: On Battery: false
17:22:57:WU03:FS01:0x22: UTC Offset: 1
17:22:57:WU03:FS01:0x22:        PID: 11152
17:22:57:WU03:FS01:0x22:        CWD: C:\ProgramData\FAHClient\work
17:22:57:WU03:FS01:0x22:************************************ OpenMM ************************************
17:22:57:WU03:FS01:0x22:   Revision: 189320d0
17:22:57:WU03:FS01:0x22:********************************************************************************
17:22:57:WU03:FS01:0x22:Project: 14915 (Run 0, Clone 1023, Gen 0)
17:22:57:WU03:FS01:0x22:Unit: 0x0000000080ae494a5fdd30ace37d0128
17:22:57:WU03:FS01:0x22:Reading tar file core.xml
17:22:57:WU03:FS01:0x22:Reading tar file integrator.xml
17:22:57:WU03:FS01:0x22:Reading tar file state.xml
17:22:57:WU03:FS01:0x22:Reading tar file system.xml
17:22:59:WU03:FS01:0x22:Digital signatures verified
17:22:59:WU03:FS01:0x22:Folding@home GPU Core22 Folding@home Core
17:22:59:WU03:FS01:0x22:Version 0.0.13
17:22:59:WU03:FS01:0x22:  Checkpoint write interval: 30000 steps (3%) [33 total]
17:22:59:WU03:FS01:0x22:  JSON viewer frame write interval: 10000 steps (1%) [100 total]
17:22:59:WU03:FS01:0x22:  XTC frame write interval: 25000 steps (2.5%) [40 total]
17:22:59:WU03:FS01:0x22:  Global context and integrator variables write interval: disabled
17:22:59:WU03:FS01:0x22:There are 4 platforms available.
17:22:59:WU03:FS01:0x22:Platform 0: Reference
17:22:59:WU03:FS01:0x22:Platform 1: CPU
17:22:59:WU03:FS01:0x22:Platform 2: OpenCL
17:22:59:WU03:FS01:0x22:  opencl-device 1 specified
17:22:59:WU03:FS01:0x22:Platform 3: CUDA
17:22:59:WU03:FS01:0x22:  cuda-device 1 specified
17:23:02:WU01:FS01:Upload 95.05%
17:23:02:WU01:FS01:Upload complete
17:23:02:WU01:FS01:Server responded WORK_ACK (400)
17:23:02:WU01:FS01:Final credit estimate, 6231.00 points
17:23:02:WU01:FS01:Cleaning up
17:23:13:WU03:FS01:0x22:Attempting to create CUDA context:
17:23:13:WU03:FS01:0x22:  Configuring platform CUDA
17:23:13:WU03:FS01:0x22:Failed to create CUDA context:
17:23:13:WU03:FS01:0x22:Error loading CUDA module: CUDA_ERROR_FILE_NOT_FOUND (301)
17:23:13:WU03:FS01:0x22:Attempting to create OpenCL context:
17:23:13:WU03:FS01:0x22:  Configuring platform OpenCL
17:23:24:WU03:FS01:0x22:  Using OpenCL on platformId 0 and gpu 1
17:23:24:WU03:FS01:0x22:Completed 0 out of 1000000 steps (0%)
17:23:26:WU03:FS01:0x22:Checkpoint completed at step 0
17:23:45:WU02:FS02:0x22:Completed 137500 out of 1250000 steps (11%)
17:28:49:WU03:FS01:0x22:Completed 10000 out of 1000000 steps (1%)
17:29:11:WU02:FS02:0x22:Completed 150000 out of 1250000 steps (12%)
17:29:14:WU02:FS02:0x22:Checkpoint completed at step 150000
17:34:13:WU03:FS01:0x22:Completed 20000 out of 1000000 steps (2%)
17:39:36:WU03:FS01:0x22:Completed 30000 out of 1000000 steps (3%)
17:39:38:WU03:FS01:0x22:Checkpoint completed at step 30000
17:43:47:WU02:FS02:0x22:Watchdog triggered, requesting soft shutdown down
17:45:00:WU03:FS01:0x22:Completed 40000 out of 1000000 steps (4%)
17:50:22:WU03:FS01:0x22:Completed 50000 out of 1000000 steps (5%)
17:53:48:WU02:FS02:0x22:Watchdog shutdown failed, hard shutdown triggered
17:53:48:WARNING:WU02:FS02:FahCore returned an unknown error code which probably indicates that it crashed
17:53:48:WARNING:WU02:FS02:FahCore returned: WU_STALLED (127 = 0x7f)
17:53:48:WU02:FS02:Starting
Top
bozz4science
Posts: 3
Joined: Wed Jan 20, 2021 8:39 pm

Re: 1600 Super - Error loading CUDA module

Post by bozz4science »

By the way, I had the CUDA development toolkit and CUDA runtime installed with the latest NVIDIA drivers as I am programming with Keras and this was a requirement for it. I just paused the WU in question, then uninstalled both programs, restarted the WU that jumped back from ~50% done back to 0% that has me thinking that maybe it could have been this CUDA installation that screwed the CUDA module loading up in the F@H client... Maybe that helps.
Top
Joe_H
Site Admin
Posts: 7937
Joined: Tue Apr 21, 2009 4:41 pm
Hardware configuration: Mac Pro 2.8 quad 12 GB smp4
MacBook Pro 2.9 i7 8 GB smp2
Location: W. MA

Re: 1600 Super - Error loading CUDA module

Post by Joe_H »

Conflicts with the CUDA toolkit are a known problem. The folding core should be using the runtime provided with it, but for a currently unknown reason the runtime from the installed toolkit gets used instead. The are looking into fixing this once the cause is determined.

The runtime selected for the folding core is one that will support the range of GPUs being supported for GPU folding and use of CUDA. That starts with Kepler based cards. If the installed toolkit happens to have a compatible runtime, then CUDA folding works. But that leaves out most versions of the toolkit.

For now the options are to either not have the toolkit installed, or to fold using the OpenCL code at a performance reduction.
Image

iMac 2.8 i7 12 GB smp8, Mac Pro 2.8 quad 12 GB smp6
MacBook Pro 2.9 i7 8 GB smp3
Top
bozz4science
Posts: 3
Joined: Wed Jan 20, 2021 8:39 pm

Re: 1600 Super - Error loading CUDA module

Post by bozz4science »

Thanks Joe for getting back to me so quickly. Kind of suspected this to likely become an issue from the very beginning upon installation of the CUDA toolkit and runtime. I decided to uninstall both software programs (the CUDA development toolkit + CUDA runtime). Is there anything else I should do to get back to working with the CUDA code of the application?

And that's so far to the "CUDA module loading error". Any reason I should worry about the "Watchdog shutdown failed, hard shutdown triggered --> FahCore returned an unknown error code which probably indicates that it crashed" messages? Are both error messages possibly related?

I will definitely continue monitoring the log files for any repeated pop-up of said message and assume it to be fixed if they don't reappear.

Thanks again for your kind explanation and thorough answer!
Top
PantherX
Site Moderator
Posts: 6986
Joined: Wed Dec 23, 2009 9:33 am
Hardware configuration: V7.6.21 -> Multi-purpose 24/7
Windows 10 64-bit
CPU:2/3/4/6 -> Intel i7-6700K
GPU:1 -> Nvidia GTX 1080 Ti
§
Retired:
2x Nvidia GTX 1070
Nvidia GTX 675M
Nvidia GTX 660 Ti
Nvidia GTX 650 SC
Nvidia GTX 260 896 MB SOC
Nvidia 9600GT 1 GB OC
Nvidia 9500M GS
Nvidia 8800GTS 320 MB

Intel Core i7-860
Intel Core i7-3840QM
Intel i3-3240
Intel Core 2 Duo E8200
Intel Core 2 Duo E6550
Intel Core 2 Duo T8300
Intel Pentium E5500
Intel Pentium E5400
Location: Land Of The Long White Cloud
Contact:
Contact PantherX

Re: 1600 Super - Error loading CUDA module

Post by PantherX »

Welcome to the F@H Forum bozz4science,

Please note that the reason by the WU restarted from 0% instead of the previous checkpoint is that the checkpoint file created when using OpenCL is not compatible with CUDA. Vice-versa too. However, there are plans to change that to ensure that checkpoints are platform agnostic but that's on a to-do list with no ETA.

It is generally rare to come across this issue:
17:53:48:WARNING:WU02:FS02:FahCore returned: WU_STALLED (127 = 0x7f)

However, now that you have removed the CUDA toolkit and runtime, restart your client and see how it performs. If you encounter any issues, do post a fresh log file for us :)
ETA:
Now ↞ Very Soon ↔ Soon ↔ Soon-ish ↔ Not Soon ↠ End Of Time

Welcome To The F@H Support Forum Ӂ Troubleshooting Bad WUs Ӂ Troubleshooting Server Connectivity Issues
Top
jimmarsen
Posts: 1
Joined: Tue Mar 09, 2021 4:26 pm

Re: 1600 Super - Error loading CUDA module

Post by jimmarsen »

I was getting a similar error with the latest FAHclient 7.6.21 and NVIDIA driver for my GTX 1600s:

06:24:25:WU01:FS01:0x22:Failed to create CUDA context:
06:24:25:WU01:FS01:0x22:Error loading CUDA module: CUDA_ERROR_FILE_NOT_FOUND (301)

As suggested, I uninstalled NVIDIA CUDA Development and NVIDIA CUDA runtime using Control Panel Programs and Features and rebooted (I'm not doing CUDA development).

Now I'm not getting the error it looks like CUDA is being used by FAHclient:

18:26:37:WU01:FS00:0x22:Attempting to create CUDA context:
18:26:37:WU01:FS00:0x22: Configuring platform CUDA
18:26:55:WU01:FS00:0x22: Using CUDA and gpu 0

And now I'm seeing Estimated PPD values between 800000 and 960000.

Thanks for your help.
Top
JimboPalmer
Posts: 2522
Joined: Mon Feb 16, 2009 4:12 am
Location: Greenwood MS USA

Re: 1600 Super - Error loading CUDA module

Post by JimboPalmer »

16:06:29:OpenCL Device 1: Platform:0 Device:1 Bus:8 Slot:0 Compute:1.2 Driver:461.9

I may be imagining this but 461.9 does not seem to be any standard release, the latest is 461.72

(I am wrong, there was a "GeForce Game Ready Driver WHQL 461.09 January 7, 2021" now I assume it reports as 461.9 Sorry.)
Tsar of all the Rushers
I tried to remain childlike, all I achieved was childish.
A friend to those who want no friends
Top
Post Reply

Return to “New Donors start here”