repeating BAD_WORK_UNIT (114 = 0x72)
Posted: Wed Apr 22, 2020 10:52 pm
See viewtopic.php?f=108&t=34690&p=328875#p328875
If I'm in the wrong forum, please point me to the right one! Thanks.
BETA TESTER
7.6.9 and 7.6.10 Linux. Clean install Linux Mint 19.3 and FAH 7.6.x and nVidia 435. Second (lower) slot fails. Replaced video card with another RTX 2080 blower. Open-air case with fans; No cooling problems or high temps. No overclocking. Memory is adequate and I tested it. Extra CPU cores available.
Reset video card and blew out slot. Reverted to 430 drivers and rebooted. Same problem.
This rig runs all day with the RTX 2080Ti in upper slot 0. Add the second slot, and failures come immediately or within an hour. So, I've replaced the FAH version, video card, and nVidia driver version. I'm out of ideas...
Also, from another run ... see 19:01:44:WU01:FS00:0x22:ERROR:exception: Error compiling kernel:
shorttack
If I'm in the wrong forum, please point me to the right one! Thanks.
BETA TESTER
7.6.9 and 7.6.10 Linux. Clean install Linux Mint 19.3 and FAH 7.6.x and nVidia 435. Second (lower) slot fails. Replaced video card with another RTX 2080 blower. Open-air case with fans; No cooling problems or high temps. No overclocking. Memory is adequate and I tested it. Extra CPU cores available.
Reset video card and blew out slot. Reverted to 430 drivers and rebooted. Same problem.
This rig runs all day with the RTX 2080Ti in upper slot 0. Add the second slot, and failures come immediately or within an hour. So, I've replaced the FAH version, video card, and nVidia driver version. I'm out of ideas...
Code: Select all
*********************** Log Started 2020-04-22T19:00:58Z ***********************
19:00:58:****************************** FAHClient ******************************
19:00:58: Version: 7.6.10
19:00:58: Author: Joseph Coffland <joseph@cauldrondevelopment.com>
19:00:58: Copyright: 2020 foldingathome.org
19:00:58: Homepage: https://foldingathome.org/
19:00:58: Date: Apr 18 2020
19:00:58: Time: 03:31:35
19:00:58: Revision: 05835e6a32e37abf3c82dbb9d2050d906637a345
19:00:58: Branch: master
19:00:58: Compiler: GNU 8.3.0
19:00:58: Options: -std=c++11 -ffunction-sections -fdata-sections -O3
19:00:58: -funroll-loops -fno-pie
19:00:58: Platform: linux2 4.19.0-5-amd64
19:00:58: Bits: 64
19:00:58: Mode: Release
19:00:58: Args: --child /etc/fahclient/config.xml --run-as fahclient
19:00:58: --pid-file=/var/run/fahclient.pid --daemon
19:00:58: Config: /etc/fahclient/config.xml
19:00:58:******************************** CBang ********************************
19:00:58: Date: Apr 17 2020
19:00:58: Time: 18:10:13
19:00:58: Revision: 2fb0be7809c5e45287a122ca5fbc15b5ae859a3b
19:00:58: Branch: master
19:00:58: Compiler: GNU 8.3.0
19:00:58: Options: -std=c++11 -ffunction-sections -fdata-sections -O3
19:00:58: -funroll-loops -fno-pie -fPIC
19:00:58: Platform: linux2 4.19.0-5-amd64
19:00:58: Bits: 64
19:00:58: Mode: Release
19:00:58:******************************* System ********************************
19:00:58: CPU: Intel(R) Core(TM) i3-8350K CPU @ 4.00GHz
19:00:58: CPU ID: GenuineIntel Family 6 Model 158 Stepping 11
19:00:58: CPUs: 4
19:00:58: Memory: 3.78GiB
19:00:58: Free Memory: 2.95GiB
19:00:58: Threads: POSIX_THREADS
19:00:58: OS Version: 5.0
19:00:58: Has Battery: false
19:00:58: On Battery: false
19:00:58: UTC Offset: -4
19:00:58: PID: 1579
19:00:58: CWD: /var/lib/fahclient
19:00:58: OS: Linux 5.0.0-32-generic x86_64
19:00:58: OS Arch: AMD64
19:00:58: GPUs: 2
19:00:58: GPU 0: Bus:1 Slot:0 Func:0 NVIDIA:8 TU102 [GeForce RTX 2080 Ti] M
19:00:58: 13448
19:00:58: GPU 1: Bus:2 Slot:0 Func:0 NVIDIA:8 TU104 [GeForce RTX 2080]
19:00:58: CUDA Device 0: Platform:0 Device:0 Bus:1 Slot:0 Compute:7.5 Driver:10.1
19:00:58: CUDA Device 1: Platform:0 Device:1 Bus:2 Slot:0 Compute:7.5 Driver:10.1
19:00:58:OpenCL Device 0: Platform:0 Device:0 Bus:1 Slot:0 Compute:1.2 Driver:435.21
19:00:58:OpenCL Device 1: Platform:0 Device:1 Bus:2 Slot:0 Compute:1.2 Driver:435.21
19:00:58:******************************* libFAH ********************************
19:00:58: Date: Apr 15 2020
19:00:58: Time: 21:43:24
19:00:58: Revision: 216968bc7025029c841ed6e36e81a03a316890d3
19:00:58: Branch: master
19:00:58: Compiler: GNU 8.3.0
19:00:58: Options: -std=c++11 -ffunction-sections -fdata-sections -O3
19:00:58: -funroll-loops -fno-pie
19:00:58: Platform: linux2 4.19.0-5-amd64
19:00:58: Bits: 64
19:00:58: Mode: Release
19:00:58:***********************************************************************
19:00:58:<config>
19:00:58: <!-- Client Control -->
19:00:58: <fold-anon v='true'/>
19:00:58:
19:00:58: <!-- Folding Slot Configuration -->
19:00:58: <cause v='ALZHEIMERS'/>
19:00:58: <client-type v='beta'/>
19:00:58:
19:00:58: <!-- HTTP Server -->
19:00:58: <allow v='127.0.0.1 192.168.1.0/24'/>
19:00:58:
19:00:58: <!-- Network -->
19:00:58: <proxy v=':8080'/>
19:00:58:
19:00:58: <!-- Remote Command Server -->
19:00:58: <command-allow-no-pass v='127.0.0.1 192.168.1.0/24'/>
19:00:58: <password v='*****'/>
19:00:58:
19:00:58: <!-- Slot Control -->
19:00:58: <power v='full'/>
19:00:58:
19:00:58: <!-- User Information -->
19:00:58: <passkey v='*****'/>
19:00:58: <team v='224497'/>
19:00:58: <user v='Catalina588_ALL_1EMQiByPxuaffjHVyb4RDLXChMkwgWmYUn'/>
19:00:58:
19:00:58: <!-- Folding Slots -->
19:00:58: <slot id='1' type='GPU'/>
19:00:58: <slot id='0' type='GPU'/>
19:00:58:</config>
19:00:58:Trying to access database...
19:00:58:Successfully acquired database lock
19:00:58:Enabled folding slot 01: READY gpu:0:TU102 [GeForce RTX 2080 Ti] M 13448
19:00:58:Enabled folding slot 00: READY gpu:1:TU104 [GeForce RTX 2080]
19:00:58:WU00:FS01:Starting
19:00:58:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/v7/lin/64bit/beta/Core_22.fah/FahCore_22 -dir 00 -suffix 01 -version 706 -lifeline 1579 -checkpoint 15 -gpu-vendor nvidia -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu 0
19:00:58:WU00:FS01:Started FahCore on PID 1589
19:00:58:WU00:FS01:Core PID:1593
19:00:58:WU00:FS01:FahCore 0x22 started
19:00:59:WU01:FS00:Connecting to 65.254.110.245:80
19:00:59:WU00:FS01:0x22:*********************** Log Started 2020-04-22T19:00:58Z ***********************
19:00:59:WU00:FS01:0x22:*************************** Core22 Folding@home Core ***************************
19:00:59:WU00:FS01:0x22: Type: 0x22
19:00:59:WU00:FS01:0x22: Core: Core22
19:00:59:WU00:FS01:0x22: Website: https://foldingathome.org/
19:00:59:WU00:FS01:0x22: Copyright: (c) 2009-2018 foldingathome.org
19:00:59:WU00:FS01:0x22: Author: John Chodera <john.chodera@choderalab.org> and Rafal Wiewiora
19:00:59:WU00:FS01:0x22: <rafal.wiewiora@choderalab.org>
19:00:59:WU00:FS01:0x22: Args: -dir 00 -suffix 01 -version 706 -lifeline 1589 -checkpoint 15
19:00:59:WU00:FS01:0x22: -gpu-vendor nvidia -opencl-platform 0 -opencl-device 0 -cuda-device
19:00:59:WU00:FS01:0x22: 0 -gpu 0
19:00:59:WU00:FS01:0x22: Config: <none>
19:00:59:WU00:FS01:0x22:************************************ Build *************************************
19:00:59:WU00:FS01:0x22: Version: 0.0.2
19:00:59:WU00:FS01:0x22: Date: Dec 6 2019
19:00:59:WU00:FS01:0x22: Time: 21:20:17
19:00:59:WU00:FS01:0x22: Repository: Git
19:00:59:WU00:FS01:0x22: Revision: f87d92b58abdf7e6bf2e173cfbc4dc3e837c7042
19:00:59:WU00:FS01:0x22: Branch: core22
19:00:59:WU00:FS01:0x22: Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
19:00:59:WU00:FS01:0x22: Options: -std=gnu++98 -O3 -funroll-loops
19:00:59:WU00:FS01:0x22: Platform: linux2 4.9.87-linuxkit-aufs
19:00:59:WU00:FS01:0x22: Bits: 64
19:00:59:WU00:FS01:0x22: Mode: Release
19:00:59:WU00:FS01:0x22:************************************ System ************************************
19:00:59:WU00:FS01:0x22: CPU: Intel(R) Core(TM) i3-8350K CPU @ 4.00GHz
19:00:59:WU00:FS01:0x22: CPU ID: GenuineIntel Family 6 Model 158 Stepping 11
19:00:59:WU00:FS01:0x22: CPUs: 4
19:00:59:WU00:FS01:0x22: Memory: 3.78GiB
19:00:59:WU00:FS01:0x22:Free Memory: 2.93GiB
19:00:59:WU00:FS01:0x22: Threads: POSIX_THREADS
19:00:59:WU00:FS01:0x22: OS Version: 5.0
19:00:59:WU00:FS01:0x22:Has Battery: false
19:00:59:WU00:FS01:0x22: On Battery: false
19:00:59:WU00:FS01:0x22: UTC Offset: -4
19:00:59:WU00:FS01:0x22: PID: 1593
19:00:59:WU00:FS01:0x22: CWD: /var/lib/fahclient/work
19:00:59:WU00:FS01:0x22: OS: Linux 5.0.0-32-generic x86_64
19:00:59:WU00:FS01:0x22: OS Arch: AMD64
19:00:59:WU00:FS01:0x22:********************************************************************************
19:00:59:WU00:FS01:0x22:Project: 16434 (Run 83, Clone 0, Gen 2)
19:00:59:WU00:FS01:0x22:Unit: 0x0000000203854c135e9cbacd953d7621
19:00:59:WU00:FS01:0x22:Digital signatures verified
19:00:59:WU00:FS01:0x22:Folding@home GPU Core22 Folding@home Core
19:00:59:WU00:FS01:0x22:Version 0.0.2
19:00:59:WU00:FS01:0x22: Found a checkpoint file
19:01:00:WU01:FS00:Assigned to work server 3.133.76.19
19:01:00:WU01:FS00:Requesting new work unit for slot 00: READY gpu:1:TU104 [GeForce RTX 2080] from 3.133.76.19
19:01:00:WU01:FS00:Connecting to 3.133.76.19:8080
19:01:01:WU01:FS00:Downloading 67.19MiB
19:01:07:WU01:FS00:Download 13.77%
19:01:07:WU00:FS01:0x22:Completed 340000 out of 2500000 steps (13%)
19:01:07:WU00:FS01:0x22:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
19:01:13:WU01:FS00:Download 28.37%
19:01:19:WU01:FS00:Download 43.63%
19:01:25:WU01:FS00:Download 58.79%
19:01:31:WU01:FS00:Download 74.61%
19:01:37:WU01:FS00:Download 90.98%
19:01:39:WU00:FS01:0x22:Completed 350000 out of 2500000 steps (14%)
19:01:40:WU01:FS00:Download complete
19:01:40:WU01:FS00:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:16434 run:176 clone:0 gen:2 core:0x22 unit:0x0000000503854c135e9cbaccdad7526d
19:01:40:WU01:FS00:Starting
19:01:40:WU01:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/v7/lin/64bit/beta/Core_22.fah/FahCore_22 -dir 01 -suffix 01 -version 706 -lifeline 1579 -checkpoint 15 -gpu-vendor nvidia -opencl-platform 0 -opencl-device 1 -cuda-device 1 -gpu 1
19:01:40:WU01:FS00:Started FahCore on PID 1854
19:01:40:WU01:FS00:Core PID:1858
19:01:40:WU01:FS00:FahCore 0x22 started
19:01:41:WU01:FS00:0x22:*********************** Log Started 2020-04-22T19:01:40Z ***********************
19:01:41:WU01:FS00:0x22:*************************** Core22 Folding@home Core ***************************
19:01:41:WU01:FS00:0x22: Type: 0x22
19:01:41:WU01:FS00:0x22: Core: Core22
19:01:41:WU01:FS00:0x22: Website: https://foldingathome.org/
19:01:41:WU01:FS00:0x22: Copyright: (c) 2009-2018 foldingathome.org
19:01:41:WU01:FS00:0x22: Author: John Chodera <john.chodera@choderalab.org> and Rafal Wiewiora
19:01:41:WU01:FS00:0x22: <rafal.wiewiora@choderalab.org>
19:01:41:WU01:FS00:0x22: Args: -dir 01 -suffix 01 -version 706 -lifeline 1854 -checkpoint 15
19:01:41:WU01:FS00:0x22: -gpu-vendor nvidia -opencl-platform 0 -opencl-device 1 -cuda-device
19:01:41:WU01:FS00:0x22: 1 -gpu 1
19:01:41:WU01:FS00:0x22: Config: <none>
19:01:41:WU01:FS00:0x22:************************************ Build *************************************
19:01:41:WU01:FS00:0x22: Version: 0.0.2
19:01:41:WU01:FS00:0x22: Date: Dec 6 2019
19:01:41:WU01:FS00:0x22: Time: 21:20:17
19:01:41:WU01:FS00:0x22: Repository: Git
19:01:41:WU01:FS00:0x22: Revision: f87d92b58abdf7e6bf2e173cfbc4dc3e837c7042
19:01:41:WU01:FS00:0x22: Branch: core22
19:01:41:WU01:FS00:0x22: Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
19:01:41:WU01:FS00:0x22: Options: -std=gnu++98 -O3 -funroll-loops
19:01:41:WU01:FS00:0x22: Platform: linux2 4.9.87-linuxkit-aufs
19:01:41:WU01:FS00:0x22: Bits: 64
19:01:41:WU01:FS00:0x22: Mode: Release
19:01:41:WU01:FS00:0x22:************************************ System ************************************
19:01:41:WU01:FS00:0x22: CPU: Intel(R) Core(TM) i3-8350K CPU @ 4.00GHz
19:01:41:WU01:FS00:0x22: CPU ID: GenuineIntel Family 6 Model 158 Stepping 11
19:01:41:WU01:FS00:0x22: CPUs: 4
19:01:41:WU01:FS00:0x22: Memory: 3.78GiB
19:01:41:WU01:FS00:0x22:Free Memory: 1.55GiB
19:01:41:WU01:FS00:0x22: Threads: POSIX_THREADS
19:01:41:WU01:FS00:0x22: OS Version: 5.0
19:01:41:WU01:FS00:0x22:Has Battery: false
19:01:41:WU01:FS00:0x22: On Battery: false
19:01:41:WU01:FS00:0x22: UTC Offset: -4
19:01:41:WU01:FS00:0x22: PID: 1858
19:01:41:WU01:FS00:0x22: CWD: /var/lib/fahclient/work
19:01:41:WU01:FS00:0x22: OS: Linux 5.0.0-32-generic x86_64
19:01:41:WU01:FS00:0x22: OS Arch: AMD64
19:01:41:WU01:FS00:0x22:********************************************************************************
19:01:41:WU01:FS00:0x22:Project: 16434 (Run 176, Clone 0, Gen 2)
19:01:41:WU01:FS00:0x22:Unit: 0x0000000503854c135e9cbaccdad7526d
19:01:41:WU01:FS00:0x22:Reading tar file core.xml
19:01:41:WU01:FS00:0x22:Reading tar file integrator.xml
19:01:41:WU01:FS00:0x22:Reading tar file state.xml
19:01:41:WU01:FS00:0x22:Reading tar file system.xml
19:01:41:WU01:FS00:0x22:Digital signatures verified
19:01:41:WU01:FS00:0x22:Folding@home GPU Core22 Folding@home Core
19:01:41:WU01:FS00:0x22:Version 0.0.2
19:01:44:WU01:FS00:0x22:ERROR:exception: Error compiling kernel:
19:01:44:WU01:FS00:0x22:Saving result file ../logfile_01.txt
19:01:44:WU01:FS00:0x22:Saving result file science.log
19:01:44:WU01:FS00:0x22:Folding@home Core Shutdown: BAD_WORK_UNIT
19:01:45:WARNING:WU01:FS00:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
19:01:45:WU01:FS00:Sending unit results: id:01 state:SEND error:FAULTY project:16434 run:176 clone:0 gen:2 core:0x22 unit:0x0000000503854c135e9cbaccdad7526d
19:01:45:WU01:FS00:Uploading 8.00KiB to 3.133.76.19
19:01:45:WU01:FS00:Connecting to 3.133.76.19:8080
19:01:45:WU01:FS00:Upload complete
19:01:45:WU01:FS00:Server responded WORK_ACK (400)
19:01:45:WU01:FS00:Cleaning up
19:01:45:WU02:FS00:Connecting to 65.254.110.245:80
19:01:45:WU02:FS00:Assigned to work server 3.133.76.19
19:01:45:WU02:FS00:Requesting new work unit for slot 00: READY gpu:1:TU104 [GeForce RTX 2080] from 3.133.76.19
19:01:45:WU02:FS00:Connecting to 3.133.76.19:8080
19:01:46:WU02:FS00:Downloading 9.20MiB
19:01:50:WU02:FS00:Download complete
19:01:50:WU02:FS00:Received Unit: id:02 state:DOWNLOAD error:NO_ERROR project:16433 run:83 clone:0 gen:1 core:0x22 unit:0x0000000103854c135e9a4efef9f24e10
19:01:50:WU02:FS00:Starting
19:01:50:WU02:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/v7/lin/64bit/beta/Core_22.fah/FahCore_22 -dir 02 -suffix 01 -version 706 -lifeline 1579 -checkpoint 15 -gpu-vendor nvidia -opencl-platform 0 -opencl-device 1 -cuda-device 1 -gpu 1
19:01:50:WU02:FS00:Started FahCore on PID 1891
19:01:50:WU02:FS00:Core PID:1895
19:01:50:WU02:FS00:FahCore 0x22 started
19:01:50:WU02:FS00:0x22:*********************** Log Started 2020-04-22T19:01:50Z ***********************
19:01:50:WU02:FS00:0x22:*************************** Core22 Folding@home Core ***************************
19:01:50:WU02:FS00:0x22: Type: 0x22
19:01:50:WU02:FS00:0x22: Core: Core22
19:01:50:WU02:FS00:0x22: Website: https://foldingathome.org/
19:01:50:WU02:FS00:0x22: Copyright: (c) 2009-2018 foldingathome.org
19:01:50:WU02:FS00:0x22: Author: John Chodera <john.chodera@choderalab.org> and Rafal Wiewiora
19:01:50:WU02:FS00:0x22: <rafal.wiewiora@choderalab.org>
19:01:50:WU02:FS00:0x22: Args: -dir 02 -suffix 01 -version 706 -lifeline 1891 -checkpoint 15
19:01:50:WU02:FS00:0x22: -gpu-vendor nvidia -opencl-platform 0 -opencl-device 1 -cuda-device
19:01:50:WU02:FS00:0x22: 1 -gpu 1
19:01:50:WU02:FS00:0x22: Config: <none>
19:01:50:WU02:FS00:0x22:************************************ Build *************************************
19:01:50:WU02:FS00:0x22: Version: 0.0.2
19:01:50:WU02:FS00:0x22: Date: Dec 6 2019
19:01:50:WU02:FS00:0x22: Time: 21:20:17
19:01:50:WU02:FS00:0x22: Repository: Git
19:01:50:WU02:FS00:0x22: Revision: f87d92b58abdf7e6bf2e173cfbc4dc3e837c7042
19:01:50:WU02:FS00:0x22: Branch: core22
19:01:50:WU02:FS00:0x22: Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
19:01:50:WU02:FS00:0x22: Options: -std=gnu++98 -O3 -funroll-loops
19:01:50:WU02:FS00:0x22: Platform: linux2 4.9.87-linuxkit-aufs
19:01:50:WU02:FS00:0x22: Bits: 64
19:01:50:WU02:FS00:0x22: Mode: Release
19:01:50:WU02:FS00:0x22:************************************ System ************************************
19:01:50:WU02:FS00:0x22: CPU: Intel(R) Core(TM) i3-8350K CPU @ 4.00GHz
19:01:50:WU02:FS00:0x22: CPU ID: GenuineIntel Family 6 Model 158 Stepping 11
19:01:50:WU02:FS00:0x22: CPUs: 4
19:01:50:WU02:FS00:0x22: Memory: 3.78GiB
19:01:50:WU02:FS00:0x22:Free Memory: 1.64GiB
19:01:50:WU02:FS00:0x22: Threads: POSIX_THREADS
19:01:50:WU02:FS00:0x22: OS Version: 5.0
19:01:50:WU02:FS00:0x22:Has Battery: false
19:01:50:WU02:FS00:0x22: On Battery: false
19:01:50:WU02:FS00:0x22: UTC Offset: -4
19:01:50:WU02:FS00:0x22: PID: 1895
19:01:50:WU02:FS00:0x22: CWD: /var/lib/fahclient/work
19:01:50:WU02:FS00:0x22: OS: Linux 5.0.0-32-generic x86_64
19:01:50:WU02:FS00:0x22: OS Arch: AMD64
19:01:50:WU02:FS00:0x22:********************************************************************************
19:01:50:WU02:FS00:0x22:Project: 16433 (Run 83, Clone 0, Gen 1)
19:01:50:WU02:FS00:0x22:Unit: 0x0000000103854c135e9a4efef9f24e10
19:01:50:WU02:FS00:0x22:Reading tar file core.xml
19:01:50:WU02:FS00:0x22:Reading tar file integrator.xml
19:01:50:WU02:FS00:0x22:Reading tar file state.xml
19:01:50:WU02:FS00:0x22:Reading tar file system.xml
19:01:50:WU02:FS00:0x22:Digital signatures verified
19:01:50:WU02:FS00:0x22:Folding@home GPU Core22 Folding@home Core
19:01:50:WU02:FS00:0x22:Version 0.0.2
19:01:53:WU02:FS00:0x22:Completed 0 out of 8000000 steps (0%)
19:01:53:WU02:FS00:0x22:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
19:01:54:WU02:FS00:0x22:Bad State detected... attempting to resume from last good checkpoint. Is your system overclocked?
19:01:54:WU02:FS00:0x22:Following exception occured: Particle coordinate is nan
19:01:55:WU02:FS00:0x22:Bad State detected... attempting to resume from last good checkpoint. Is your system overclocked?
19:01:55:WU02:FS00:0x22:Following exception occured: Particle coordinate is nan
19:01:56:WU02:FS00:0x22:Bad State detected... attempting to resume from last good checkpoint. Is your system overclocked?
19:01:56:WU02:FS00:0x22:Following exception occured: Particle coordinate is nan
19:01:56:WU02:FS00:0x22:ERROR:114: Max Retries Reached
19:01:56:WU02:FS00:0x22:Saving result file ../logfile_01.txt
19:01:56:WU02:FS00:0x22:Saving result file badstate-0.xml
19:01:56:WU02:FS00:0x22:Saving result file badstate-1.xml
19:01:56:WU02:FS00:0x22:Saving result file badstate-2.xml
19:01:56:WU02:FS00:0x22:Saving result file checkpt.crc
19:01:56:WU02:FS00:0x22:Saving result file science.log
19:01:56:WU02:FS00:0x22:Folding@home Core Shutdown: BAD_WORK_UNIT
19:01:57:WARNING:WU02:FS00:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
19:01:57:WU02:FS00:Sending unit results: id:02 state:SEND error:FAULTY project:16433 run:83 clone:0 gen:1 core:0x22 unit:0x0000000103854c135e9a4efef9f24e10
19:01:57:WU02:FS00:Uploading 6.35MiB to 3.133.76.19
19:01:57:WU02:FS00:Connecting to 3.133.76.19:8080
19:01:57:WU01:FS00:Connecting to 65.254.110.245:80
19:01:57:WU01:FS00:Assigned to work server 128.252.203.2
19:01:57:WU01:FS00:Requesting new work unit for slot 00: READY gpu:1:TU104 [GeForce RTX 2080] from 128.252.203.2
19:01:57:WU01:FS00:Connecting to 128.252.203.2:8080
19:02:03:WU02:FS00:Upload 91.51%
19:02:04:WU02:FS00:Upload complete
19:02:04:WU02:FS00:Server responded WORK_ACK (400)
19:02:04:WU02:FS00:Cleaning up
19:02:32:WU01:FS00:Downloading 28.40MiB
19:02:38:WU01:FS00:Download 29.05%
19:02:44:WU01:FS00:Download 64.27%
19:02:50:WU01:FS00:Download 98.39%
19:02:50:WU01:FS00:Download complete
19:02:50:WU01:FS00:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:14562 run:0 clone:1113 gen:12 core:0x22 unit:0x0000001080fccb025e9615bc26e6c65a
19:02:50:WU01:FS00:Starting
19:02:50:WU01:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/v7/lin/64bit/beta/Core_22.fah/FahCore_22 -dir 01 -suffix 01 -version 706 -lifeline 1579 -checkpoint 15 -gpu-vendor nvidia -opencl-platform 0 -opencl-device 1 -cuda-device 1 -gpu 1
19:02:50:WU01:FS00:Started FahCore on PID 1927
19:02:50:WU01:FS00:Core PID:1931
19:02:50:WU01:FS00:FahCore 0x22 started
19:02:50:WU01:FS00:0x22:*********************** Log Started 2020-04-22T19:02:50Z ***********************
19:02:50:WU01:FS00:0x22:*************************** Core22 Folding@home Core ***************************
19:02:50:WU01:FS00:0x22: Type: 0x22
19:02:50:WU01:FS00:0x22: Core: Core22
19:02:50:WU01:FS00:0x22: Website: https://foldingathome.org/
19:02:50:WU01:FS00:0x22: Copyright: (c) 2009-2018 foldingathome.org
19:02:50:WU01:FS00:0x22: Author: John Chodera <john.chodera@choderalab.org> and Rafal Wiewiora
19:02:50:WU01:FS00:0x22: <rafal.wiewiora@choderalab.org>
19:02:50:WU01:FS00:0x22: Args: -dir 01 -suffix 01 -version 706 -lifeline 1927 -checkpoint 15
19:02:50:WU01:FS00:0x22: -gpu-vendor nvidia -opencl-platform 0 -opencl-device 1 -cuda-device
19:02:50:WU01:FS00:0x22: 1 -gpu 1
19:02:50:WU01:FS00:0x22: Config: <none>
19:02:50:WU01:FS00:0x22:************************************ Build *************************************
19:02:50:WU01:FS00:0x22: Version: 0.0.2
19:02:50:WU01:FS00:0x22: Date: Dec 6 2019
19:02:50:WU01:FS00:0x22: Time: 21:20:17
19:02:50:WU01:FS00:0x22: Repository: Git
19:02:50:WU01:FS00:0x22: Revision: f87d92b58abdf7e6bf2e173cfbc4dc3e837c7042
19:02:50:WU01:FS00:0x22: Branch: core22
19:02:50:WU01:FS00:0x22: Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
19:02:50:WU01:FS00:0x22: Options: -std=gnu++98 -O3 -funroll-loops
19:02:50:WU01:FS00:0x22: Platform: linux2 4.9.87-linuxkit-aufs
19:02:50:WU01:FS00:0x22: Bits: 64
19:02:50:WU01:FS00:0x22: Mode: Release
19:02:50:WU01:FS00:0x22:************************************ System ************************************
19:02:50:WU01:FS00:0x22: CPU: Intel(R) Core(TM) i3-8350K CPU @ 4.00GHz
19:02:50:WU01:FS00:0x22: CPU ID: GenuineIntel Family 6 Model 158 Stepping 11
19:02:50:WU01:FS00:0x22: CPUs: 4
19:02:50:WU01:FS00:0x22: Memory: 3.78GiB
19:02:50:WU01:FS00:0x22:Free Memory: 1.48GiB
19:02:50:WU01:FS00:0x22: Threads: POSIX_THREADS
19:02:50:WU01:FS00:0x22: OS Version: 5.0
19:02:50:WU01:FS00:0x22:Has Battery: false
19:02:50:WU01:FS00:0x22: On Battery: false
19:02:50:WU01:FS00:0x22: UTC Offset: -4
19:02:50:WU01:FS00:0x22: PID: 1931
19:02:50:WU01:FS00:0x22: CWD: /var/lib/fahclient/work
19:02:50:WU01:FS00:0x22: OS: Linux 5.0.0-32-generic x86_64
19:02:50:WU01:FS00:0x22: OS Arch: AMD64
19:02:50:WU01:FS00:0x22:********************************************************************************
19:02:50:WU01:FS00:0x22:Project: 14562 (Run 0, Clone 1113, Gen 12)
19:02:50:WU01:FS00:0x22:Unit: 0x0000001080fccb025e9615bc26e6c65a
19:02:50:WU01:FS00:0x22:Reading tar file core.xml
19:02:50:WU01:FS00:0x22:Reading tar file integrator.xml
19:02:50:WU01:FS00:0x22:Reading tar file state.xml
19:02:53:WU01:FS00:0x22:Reading tar file system.xml
19:02:54:WU01:FS00:0x22:Digital signatures verified
19:02:54:WU01:FS00:0x22:Folding@home GPU Core22 Folding@home Core
19:02:54:WU01:FS00:0x22:Version 0.0.2
19:03:00:WU00:FS01:0x22:Completed 375000 out of 2500000 steps (15%)
19:03:11:WU01:FS00:0x22:ERROR:exception: Error compiling kernel:
19:03:11:WU01:FS00:0x22:Saving result file ../logfile_01.txt
19:03:11:WU01:FS00:0x22:Saving result file science.log
19:03:11:WU01:FS00:0x22:Folding@home Core Shutdown: BAD_WORK_UNIT
19:03:11:WARNING:WU01:FS00:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
19:03:11:WU01:FS00:Sending unit results: id:01 state:SEND error:FAULTY project:14562 run:0 clone:1113 gen:12 core:0x22 unit:0x0000001080fccb025e9615bc26e6c65a
19:03:11:WU01:FS00:Uploading 2.61KiB to 128.252.203.2
19:03:11:WU01:FS00:Connecting to 128.252.203.2:8080
19:03:11:WU01:FS00:Upload complete
19:03:11:WU01:FS00:Server responded WORK_ACK (400)
19:03:11:WU01:FS00:Cleaning up
19:03:11:WU02:FS00:Connecting to 65.254.110.245:80
19:03:12:WU02:FS00:Assigned to work server 128.252.203.2
19:03:12:WU02:FS00:Requesting new work unit for slot 00: READY gpu:1:TU104 [GeForce RTX 2080] from 128.252.203.2
19:03:12:WU02:FS00:Connecting to 128.252.203.2:8080
ailed
Code: Select all
*********************** Log Started 2020-04-22T19:00:58Z ***********************
19:00:58:****************************** FAHClient ******************************
19:00:58: Version: 7.6.10
19:00:58: Author: Joseph Coffland <joseph@cauldrondevelopment.com>
19:00:58: Copyright: 2020 foldingathome.org
19:00:58: Homepage: https://foldingathome.org/
19:00:58: Date: Apr 18 2020
19:00:58: Time: 03:31:35
19:00:58: Revision: 05835e6a32e37abf3c82dbb9d2050d906637a345
19:00:58: Branch: master
19:00:58: Compiler: GNU 8.3.0
19:00:58: Options: -std=c++11 -ffunction-sections -fdata-sections -O3
19:00:58: -funroll-loops -fno-pie
19:00:58: Platform: linux2 4.19.0-5-amd64
19:00:58: Bits: 64
19:00:58: Mode: Release
19:00:58: Args: --child /etc/fahclient/config.xml --run-as fahclient
19:00:58: --pid-file=/var/run/fahclient.pid --daemon
19:00:58: Config: /etc/fahclient/config.xml
19:00:58:******************************** CBang ********************************
19:00:58: Date: Apr 17 2020
19:00:58: Time: 18:10:13
19:00:58: Revision: 2fb0be7809c5e45287a122ca5fbc15b5ae859a3b
19:00:58: Branch: master
19:00:58: Compiler: GNU 8.3.0
19:00:58: Options: -std=c++11 -ffunction-sections -fdata-sections -O3
19:00:58: -funroll-loops -fno-pie -fPIC
19:00:58: Platform: linux2 4.19.0-5-amd64
19:00:58: Bits: 64
19:00:58: Mode: Release
19:00:58:******************************* System ********************************
19:00:58: CPU: Intel(R) Core(TM) i3-8350K CPU @ 4.00GHz
19:00:58: CPU ID: GenuineIntel Family 6 Model 158 Stepping 11
19:00:58: CPUs: 4
19:00:58: Memory: 3.78GiB
19:00:58: Free Memory: 2.95GiB
19:00:58: Threads: POSIX_THREADS
19:00:58: OS Version: 5.0
19:00:58: Has Battery: false
19:00:58: On Battery: false
19:00:58: UTC Offset: -4
19:00:58: PID: 1579
19:00:58: CWD: /var/lib/fahclient
19:00:58: OS: Linux 5.0.0-32-generic x86_64
19:00:58: OS Arch: AMD64
19:00:58: GPUs: 2
19:00:58: GPU 0: Bus:1 Slot:0 Func:0 NVIDIA:8 TU102 [GeForce RTX 2080 Ti] M
19:00:58: 13448
19:00:58: GPU 1: Bus:2 Slot:0 Func:0 NVIDIA:8 TU104 [GeForce RTX 2080]
19:00:58: CUDA Device 0: Platform:0 Device:0 Bus:1 Slot:0 Compute:7.5 Driver:10.1
19:00:58: CUDA Device 1: Platform:0 Device:1 Bus:2 Slot:0 Compute:7.5 Driver:10.1
19:00:58:OpenCL Device 0: Platform:0 Device:0 Bus:1 Slot:0 Compute:1.2 Driver:435.21
19:00:58:OpenCL Device 1: Platform:0 Device:1 Bus:2 Slot:0 Compute:1.2 Driver:435.21
19:00:58:******************************* libFAH ********************************
19:00:58: Date: Apr 15 2020
19:00:58: Time: 21:43:24
19:00:58: Revision: 216968bc7025029c841ed6e36e81a03a316890d3
19:00:58: Branch: master
19:00:58: Compiler: GNU 8.3.0
19:00:58: Options: -std=c++11 -ffunction-sections -fdata-sections -O3
19:00:58: -funroll-loops -fno-pie
19:00:58: Platform: linux2 4.19.0-5-amd64
19:00:58: Bits: 64
19:00:58: Mode: Release
19:00:58:***********************************************************************
19:00:58:<config>
19:00:58: <!-- Client Control -->
19:00:58: <fold-anon v='true'/>
19:00:58:
19:00:58: <!-- Folding Slot Configuration -->
19:00:58: <cause v='ALZHEIMERS'/>
19:00:58: <client-type v='beta'/>
19:00:58:
19:00:58: <!-- HTTP Server -->
19:00:58: <allow v='127.0.0.1 192.168.1.0/24'/>
19:00:58:
19:00:58: <!-- Network -->
19:00:58: <proxy v=':8080'/>
19:00:58:
19:00:58: <!-- Remote Command Server -->
19:00:58: <command-allow-no-pass v='127.0.0.1 192.168.1.0/24'/>
19:00:58: <password v='*****'/>
19:00:58:
19:00:58: <!-- Slot Control -->
19:00:58: <power v='full'/>
19:00:58:
19:00:58: <!-- User Information -->
19:00:58: <passkey v='*****'/>
19:00:58: <team v='224497'/>
19:00:58: <user v='Catalina588_ALL_1EMQiByPxuaffjHVyb4RDLXChMkwgWmYUn'/>
19:00:58:
19:00:58: <!-- Folding Slots -->
19:00:58: <slot id='1' type='GPU'/>
19:00:58: <slot id='0' type='GPU'/>
19:00:58:</config>
19:00:58:Trying to access database...
19:00:58:Successfully acquired database lock
19:00:58:Enabled folding slot 01: READY gpu:0:TU102 [GeForce RTX 2080 Ti] M 13448
19:00:58:Enabled folding slot 00: READY gpu:1:TU104 [GeForce RTX 2080]
19:00:58:WU00:FS01:Starting
19:00:58:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/v7/lin/64bit/beta/Core_22.fah/FahCore_22 -dir 00 -suffix 01 -version 706 -lifeline 1579 -checkpoint 15 -gpu-vendor nvidia -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu 0
19:00:58:WU00:FS01:Started FahCore on PID 1589
19:00:58:WU00:FS01:Core PID:1593
19:00:58:WU00:FS01:FahCore 0x22 started
19:00:59:WU01:FS00:Connecting to 65.254.110.245:80
19:00:59:WU00:FS01:0x22:*********************** Log Started 2020-04-22T19:00:58Z ***********************
19:00:59:WU00:FS01:0x22:*************************** Core22 Folding@home Core ***************************
19:00:59:WU00:FS01:0x22: Type: 0x22
19:00:59:WU00:FS01:0x22: Core: Core22
19:00:59:WU00:FS01:0x22: Website: https://foldingathome.org/
19:00:59:WU00:FS01:0x22: Copyright: (c) 2009-2018 foldingathome.org
19:00:59:WU00:FS01:0x22: Author: John Chodera <john.chodera@choderalab.org> and Rafal Wiewiora
19:00:59:WU00:FS01:0x22: <rafal.wiewiora@choderalab.org>
19:00:59:WU00:FS01:0x22: Args: -dir 00 -suffix 01 -version 706 -lifeline 1589 -checkpoint 15
19:00:59:WU00:FS01:0x22: -gpu-vendor nvidia -opencl-platform 0 -opencl-device 0 -cuda-device
19:00:59:WU00:FS01:0x22: 0 -gpu 0
19:00:59:WU00:FS01:0x22: Config: <none>
19:00:59:WU00:FS01:0x22:************************************ Build *************************************
19:00:59:WU00:FS01:0x22: Version: 0.0.2
19:00:59:WU00:FS01:0x22: Date: Dec 6 2019
19:00:59:WU00:FS01:0x22: Time: 21:20:17
19:00:59:WU00:FS01:0x22: Repository: Git
19:00:59:WU00:FS01:0x22: Revision: f87d92b58abdf7e6bf2e173cfbc4dc3e837c7042
19:00:59:WU00:FS01:0x22: Branch: core22
19:00:59:WU00:FS01:0x22: Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
19:00:59:WU00:FS01:0x22: Options: -std=gnu++98 -O3 -funroll-loops
19:00:59:WU00:FS01:0x22: Platform: linux2 4.9.87-linuxkit-aufs
19:00:59:WU00:FS01:0x22: Bits: 64
19:00:59:WU00:FS01:0x22: Mode: Release
19:00:59:WU00:FS01:0x22:************************************ System ************************************
19:00:59:WU00:FS01:0x22: CPU: Intel(R) Core(TM) i3-8350K CPU @ 4.00GHz
19:00:59:WU00:FS01:0x22: CPU ID: GenuineIntel Family 6 Model 158 Stepping 11
19:00:59:WU00:FS01:0x22: CPUs: 4
19:00:59:WU00:FS01:0x22: Memory: 3.78GiB
19:00:59:WU00:FS01:0x22:Free Memory: 2.93GiB
19:00:59:WU00:FS01:0x22: Threads: POSIX_THREADS
19:00:59:WU00:FS01:0x22: OS Version: 5.0
19:00:59:WU00:FS01:0x22:Has Battery: false
19:00:59:WU00:FS01:0x22: On Battery: false
19:00:59:WU00:FS01:0x22: UTC Offset: -4
19:00:59:WU00:FS01:0x22: PID: 1593
19:00:59:WU00:FS01:0x22: CWD: /var/lib/fahclient/work
19:00:59:WU00:FS01:0x22: OS: Linux 5.0.0-32-generic x86_64
19:00:59:WU00:FS01:0x22: OS Arch: AMD64
19:00:59:WU00:FS01:0x22:********************************************************************************
19:00:59:WU00:FS01:0x22:Project: 16434 (Run 83, Clone 0, Gen 2)
19:00:59:WU00:FS01:0x22:Unit: 0x0000000203854c135e9cbacd953d7621
19:00:59:WU00:FS01:0x22:Digital signatures verified
19:00:59:WU00:FS01:0x22:Folding@home GPU Core22 Folding@home Core
19:00:59:WU00:FS01:0x22:Version 0.0.2
19:00:59:WU00:FS01:0x22: Found a checkpoint file
19:01:00:WU01:FS00:Assigned to work server 3.133.76.19
19:01:00:WU01:FS00:Requesting new work unit for slot 00: READY gpu:1:TU104 [GeForce RTX 2080] from 3.133.76.19
19:01:00:WU01:FS00:Connecting to 3.133.76.19:8080
19:01:01:WU01:FS00:Downloading 67.19MiB
19:01:07:WU01:FS00:Download 13.77%
19:01:07:WU00:FS01:0x22:Completed 340000 out of 2500000 steps (13%)
19:01:07:WU00:FS01:0x22:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
19:01:13:WU01:FS00:Download 28.37%
19:01:19:WU01:FS00:Download 43.63%
19:01:25:WU01:FS00:Download 58.79%
19:01:31:WU01:FS00:Download 74.61%
19:01:37:WU01:FS00:Download 90.98%
19:01:39:WU00:FS01:0x22:Completed 350000 out of 2500000 steps (14%)
19:01:40:WU01:FS00:Download complete
19:01:40:WU01:FS00:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:16434 run:176 clone:0 gen:2 core:0x22 unit:0x0000000503854c135e9cbaccdad7526d
19:01:40:WU01:FS00:Starting
19:01:40:WU01:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/v7/lin/64bit/beta/Core_22.fah/FahCore_22 -dir 01 -suffix 01 -version 706 -lifeline 1579 -checkpoint 15 -gpu-vendor nvidia -opencl-platform 0 -opencl-device 1 -cuda-device 1 -gpu 1
19:01:40:WU01:FS00:Started FahCore on PID 1854
19:01:40:WU01:FS00:Core PID:1858
19:01:40:WU01:FS00:FahCore 0x22 started
19:01:41:WU01:FS00:0x22:*********************** Log Started 2020-04-22T19:01:40Z ***********************
19:01:41:WU01:FS00:0x22:*************************** Core22 Folding@home Core ***************************
19:01:41:WU01:FS00:0x22: Type: 0x22
19:01:41:WU01:FS00:0x22: Core: Core22
19:01:41:WU01:FS00:0x22: Website: https://foldingathome.org/
19:01:41:WU01:FS00:0x22: Copyright: (c) 2009-2018 foldingathome.org
19:01:41:WU01:FS00:0x22: Author: John Chodera <john.chodera@choderalab.org> and Rafal Wiewiora
19:01:41:WU01:FS00:0x22: <rafal.wiewiora@choderalab.org>
19:01:41:WU01:FS00:0x22: Args: -dir 01 -suffix 01 -version 706 -lifeline 1854 -checkpoint 15
19:01:41:WU01:FS00:0x22: -gpu-vendor nvidia -opencl-platform 0 -opencl-device 1 -cuda-device
19:01:41:WU01:FS00:0x22: 1 -gpu 1
19:01:41:WU01:FS00:0x22: Config: <none>
19:01:41:WU01:FS00:0x22:************************************ Build *************************************
19:01:41:WU01:FS00:0x22: Version: 0.0.2
19:01:41:WU01:FS00:0x22: Date: Dec 6 2019
19:01:41:WU01:FS00:0x22: Time: 21:20:17
19:01:41:WU01:FS00:0x22: Repository: Git
19:01:41:WU01:FS00:0x22: Revision: f87d92b58abdf7e6bf2e173cfbc4dc3e837c7042
19:01:41:WU01:FS00:0x22: Branch: core22
19:01:41:WU01:FS00:0x22: Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
19:01:41:WU01:FS00:0x22: Options: -std=gnu++98 -O3 -funroll-loops
19:01:41:WU01:FS00:0x22: Platform: linux2 4.9.87-linuxkit-aufs
19:01:41:WU01:FS00:0x22: Bits: 64
19:01:41:WU01:FS00:0x22: Mode: Release
19:01:41:WU01:FS00:0x22:************************************ System ************************************
19:01:41:WU01:FS00:0x22: CPU: Intel(R) Core(TM) i3-8350K CPU @ 4.00GHz
19:01:41:WU01:FS00:0x22: CPU ID: GenuineIntel Family 6 Model 158 Stepping 11
19:01:41:WU01:FS00:0x22: CPUs: 4
19:01:41:WU01:FS00:0x22: Memory: 3.78GiB
19:01:41:WU01:FS00:0x22:Free Memory: 1.55GiB
19:01:41:WU01:FS00:0x22: Threads: POSIX_THREADS
19:01:41:WU01:FS00:0x22: OS Version: 5.0
19:01:41:WU01:FS00:0x22:Has Battery: false
19:01:41:WU01:FS00:0x22: On Battery: false
19:01:41:WU01:FS00:0x22: UTC Offset: -4
19:01:41:WU01:FS00:0x22: PID: 1858
19:01:41:WU01:FS00:0x22: CWD: /var/lib/fahclient/work
19:01:41:WU01:FS00:0x22: OS: Linux 5.0.0-32-generic x86_64
19:01:41:WU01:FS00:0x22: OS Arch: AMD64
19:01:41:WU01:FS00:0x22:********************************************************************************
19:01:41:WU01:FS00:0x22:Project: 16434 (Run 176, Clone 0, Gen 2)
19:01:41:WU01:FS00:0x22:Unit: 0x0000000503854c135e9cbaccdad7526d
19:01:41:WU01:FS00:0x22:Reading tar file core.xml
19:01:41:WU01:FS00:0x22:Reading tar file integrator.xml
19:01:41:WU01:FS00:0x22:Reading tar file state.xml
19:01:41:WU01:FS00:0x22:Reading tar file system.xml
19:01:41:WU01:FS00:0x22:Digital signatures verified
19:01:41:WU01:FS00:0x22:Folding@home GPU Core22 Folding@home Core
19:01:41:WU01:FS00:0x22:Version 0.0.2
19:01:44:WU01:FS00:0x22:ERROR:exception: Error compiling kernel:
19:01:44:WU01:FS00:0x22:Saving result file ../logfile_01.txt
19:01:44:WU01:FS00:0x22:Saving result file science.log
19:01:44:WU01:FS00:0x22:Folding@home Core Shutdown: BAD_WORK_UNIT
19:01:45:WARNING:WU01:FS00:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
19:01:45:WU01:FS00:Sending unit results: id:01 state:SEND error:FAULTY project:16434 run:176 clone:0 gen:2 core:0x22 unit:0x0000000503854c135e9cbaccdad7526d
19:01:45:WU01:FS00:Uploading 8.00KiB to 3.133.76.19
19:01:45:WU01:FS00:Connecting to 3.133.76.19:8080
19:01:45:WU01:FS00:Upload complete
19:01:45:WU01:FS00:Server responded WORK_ACK (400)
19:01:45:WU01:FS00:Cleaning up
19:01:45:WU02:FS00:Connecting to 65.254.110.245:80
19:01:45:WU02:FS00:Assigned to work server 3.133.76.19
19:01:45:WU02:FS00:Requesting new work unit for slot 00: READY gpu:1:TU104 [GeForce RTX 2080] from 3.133.76.19
19:01:45:WU02:FS00:Connecting to 3.133.76.19:8080
19:01:46:WU02:FS00:Downloading 9.20MiB
19:01:50:WU02:FS00:Download complete
19:01:50:WU02:FS00:Received Unit: id:02 state:DOWNLOAD error:NO_ERROR project:16433 run:83 clone:0 gen:1 core:0x22 unit:0x0000000103854c135e9a4efef9f24e10
19:01:50:WU02:FS00:Starting
19:01:50:WU02:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/v7/lin/64bit/beta/Core_22.fah/FahCore_22 -dir 02 -suffix 01 -version 706 -lifeline 1579 -checkpoint 15 -gpu-vendor nvidia -opencl-platform 0 -opencl-device 1 -cuda-device 1 -gpu 1
19:01:50:WU02:FS00:Started FahCore on PID 1891
19:01:50:WU02:FS00:Core PID:1895
19:01:50:WU02:FS00:FahCore 0x22 started
19:01:50:WU02:FS00:0x22:*********************** Log Started 2020-04-22T19:01:50Z ***********************
19:01:50:WU02:FS00:0x22:*************************** Core22 Folding@home Core ***************************
19:01:50:WU02:FS00:0x22: Type: 0x22
19:01:50:WU02:FS00:0x22: Core: Core22
19:01:50:WU02:FS00:0x22: Website: https://foldingathome.org/
19:01:50:WU02:FS00:0x22: Copyright: (c) 2009-2018 foldingathome.org
19:01:50:WU02:FS00:0x22: Author: John Chodera <john.chodera@choderalab.org> and Rafal Wiewiora
19:01:50:WU02:FS00:0x22: <rafal.wiewiora@choderalab.org>
19:01:50:WU02:FS00:0x22: Args: -dir 02 -suffix 01 -version 706 -lifeline 1891 -checkpoint 15
19:01:50:WU02:FS00:0x22: -gpu-vendor nvidia -opencl-platform 0 -opencl-device 1 -cuda-device
19:01:50:WU02:FS00:0x22: 1 -gpu 1
19:01:50:WU02:FS00:0x22: Config: <none>
19:01:50:WU02:FS00:0x22:************************************ Build *************************************
19:01:50:WU02:FS00:0x22: Version: 0.0.2
19:01:50:WU02:FS00:0x22: Date: Dec 6 2019
19:01:50:WU02:FS00:0x22: Time: 21:20:17
19:01:50:WU02:FS00:0x22: Repository: Git
19:01:50:WU02:FS00:0x22: Revision: f87d92b58abdf7e6bf2e173cfbc4dc3e837c7042
19:01:50:WU02:FS00:0x22: Branch: core22
19:01:50:WU02:FS00:0x22: Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
19:01:50:WU02:FS00:0x22: Options: -std=gnu++98 -O3 -funroll-loops
19:01:50:WU02:FS00:0x22: Platform: linux2 4.9.87-linuxkit-aufs
19:01:50:WU02:FS00:0x22: Bits: 64
19:01:50:WU02:FS00:0x22: Mode: Release
19:01:50:WU02:FS00:0x22:************************************ System ************************************
19:01:50:WU02:FS00:0x22: CPU: Intel(R) Core(TM) i3-8350K CPU @ 4.00GHz
19:01:50:WU02:FS00:0x22: CPU ID: GenuineIntel Family 6 Model 158 Stepping 11
19:01:50:WU02:FS00:0x22: CPUs: 4
19:01:50:WU02:FS00:0x22: Memory: 3.78GiB
19:01:50:WU02:FS00:0x22:Free Memory: 1.64GiB
19:01:50:WU02:FS00:0x22: Threads: POSIX_THREADS
19:01:50:WU02:FS00:0x22: OS Version: 5.0
19:01:50:WU02:FS00:0x22:Has Battery: false
19:01:50:WU02:FS00:0x22: On Battery: false
19:01:50:WU02:FS00:0x22: UTC Offset: -4
19:01:50:WU02:FS00:0x22: PID: 1895
19:01:50:WU02:FS00:0x22: CWD: /var/lib/fahclient/work
19:01:50:WU02:FS00:0x22: OS: Linux 5.0.0-32-generic x86_64
19:01:50:WU02:FS00:0x22: OS Arch: AMD64
19:01:50:WU02:FS00:0x22:********************************************************************************
19:01:50:WU02:FS00:0x22:Project: 16433 (Run 83, Clone 0, Gen 1)
19:01:50:WU02:FS00:0x22:Unit: 0x0000000103854c135e9a4efef9f24e10
19:01:50:WU02:FS00:0x22:Reading tar file core.xml
19:01:50:WU02:FS00:0x22:Reading tar file integrator.xml
19:01:50:WU02:FS00:0x22:Reading tar file state.xml
19:01:50:WU02:FS00:0x22:Reading tar file system.xml
19:01:50:WU02:FS00:0x22:Digital signatures verified
19:01:50:WU02:FS00:0x22:Folding@home GPU Core22 Folding@home Core
19:01:50:WU02:FS00:0x22:Version 0.0.2
19:01:53:WU02:FS00:0x22:Completed 0 out of 8000000 steps (0%)
19:01:53:WU02:FS00:0x22:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
19:01:54:WU02:FS00:0x22:Bad State detected... attempting to resume from last good checkpoint. Is your system overclocked?
19:01:54:WU02:FS00:0x22:Following exception occured: Particle coordinate is nan
19:01:55:WU02:FS00:0x22:Bad State detected... attempting to resume from last good checkpoint. Is your system overclocked?
19:01:55:WU02:FS00:0x22:Following exception occured: Particle coordinate is nan
19:01:56:WU02:FS00:0x22:Bad State detected... attempting to resume from last good checkpoint. Is your system overclocked?
19:01:56:WU02:FS00:0x22:Following exception occured: Particle coordinate is nan
19:01:56:WU02:FS00:0x22:ERROR:114: Max Retries Reached
19:01:56:WU02:FS00:0x22:Saving result file ../logfile_01.txt
19:01:56:WU02:FS00:0x22:Saving result file badstate-0.xml
19:01:56:WU02:FS00:0x22:Saving result file badstate-1.xml
19:01:56:WU02:FS00:0x22:Saving result file badstate-2.xml
19:01:56:WU02:FS00:0x22:Saving result file checkpt.crc
19:01:56:WU02:FS00:0x22:Saving result file science.log
19:01:56:WU02:FS00:0x22:Folding@home Core Shutdown: BAD_WORK_UNIT
19:01:57:WARNING:WU02:FS00:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
19:01:57:WU02:FS00:Sending unit results: id:02 state:SEND error:FAULTY project:16433 run:83 clone:0 gen:1 core:0x22 unit:0x0000000103854c135e9a4efef9f24e10
19:01:57:WU02:FS00:Uploading 6.35MiB to 3.133.76.19
19:01:57:WU02:FS00:Connecting to 3.133.76.19:8080
19:01:57:WU01:FS00:Connecting to 65.254.110.245:80
19:01:57:WU01:FS00:Assigned to work server 128.252.203.2
19:01:57:WU01:FS00:Requesting new work unit for slot 00: READY gpu:1:TU104 [GeForce RTX 2080] from 128.252.203.2
19:01:57:WU01:FS00:Connecting to 128.252.203.2:8080
19:02:03:WU02:FS00:Upload 91.51%
19:02:04:WU02:FS00:Upload complete
19:02:04:WU02:FS00:Server responded WORK_ACK (400)
19:02:04:WU02:FS00:Cleaning up
19:02:32:WU01:FS00:Downloading 28.40MiB
19:02:38:WU01:FS00:Download 29.05%
19:02:44:WU01:FS00:Download 64.27%
19:02:50:WU01:FS00:Download 98.39%
19:02:50:WU01:FS00:Download complete
19:02:50:WU01:FS00:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:14562 run:0 clone:1113 gen:12 core:0x22 unit:0x0000001080fccb025e9615bc26e6c65a
19:02:50:WU01:FS00:Starting
19:02:50:WU01:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/v7/lin/64bit/beta/Core_22.fah/FahCore_22 -dir 01 -suffix 01 -version 706 -lifeline 1579 -checkpoint 15 -gpu-vendor nvidia -opencl-platform 0 -opencl-device 1 -cuda-device 1 -gpu 1
19:02:50:WU01:FS00:Started FahCore on PID 1927
19:02:50:WU01:FS00:Core PID:1931
19:02:50:WU01:FS00:FahCore 0x22 started
19:02:50:WU01:FS00:0x22:*********************** Log Started 2020-04-22T19:02:50Z ***********************
19:02:50:WU01:FS00:0x22:*************************** Core22 Folding@home Core ***************************
19:02:50:WU01:FS00:0x22: Type: 0x22
19:02:50:WU01:FS00:0x22: Core: Core22
19:02:50:WU01:FS00:0x22: Website: https://foldingathome.org/
19:02:50:WU01:FS00:0x22: Copyright: (c) 2009-2018 foldingathome.org
19:02:50:WU01:FS00:0x22: Author: John Chodera <john.chodera@choderalab.org> and Rafal Wiewiora
19:02:50:WU01:FS00:0x22: <rafal.wiewiora@choderalab.org>
19:02:50:WU01:FS00:0x22: Args: -dir 01 -suffix 01 -version 706 -lifeline 1927 -checkpoint 15
19:02:50:WU01:FS00:0x22: -gpu-vendor nvidia -opencl-platform 0 -opencl-device 1 -cuda-device
19:02:50:WU01:FS00:0x22: 1 -gpu 1
19:02:50:WU01:FS00:0x22: Config: <none>
19:02:50:WU01:FS00:0x22:************************************ Build *************************************
19:02:50:WU01:FS00:0x22: Version: 0.0.2
19:02:50:WU01:FS00:0x22: Date: Dec 6 2019
19:02:50:WU01:FS00:0x22: Time: 21:20:17
19:02:50:WU01:FS00:0x22: Repository: Git
19:02:50:WU01:FS00:0x22: Revision: f87d92b58abdf7e6bf2e173cfbc4dc3e837c7042
19:02:50:WU01:FS00:0x22: Branch: core22
19:02:50:WU01:FS00:0x22: Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
19:02:50:WU01:FS00:0x22: Options: -std=gnu++98 -O3 -funroll-loops
19:02:50:WU01:FS00:0x22: Platform: linux2 4.9.87-linuxkit-aufs
19:02:50:WU01:FS00:0x22: Bits: 64
19:02:50:WU01:FS00:0x22: Mode: Release
19:02:50:WU01:FS00:0x22:************************************ System ************************************
19:02:50:WU01:FS00:0x22: CPU: Intel(R) Core(TM) i3-8350K CPU @ 4.00GHz
19:02:50:WU01:FS00:0x22: CPU ID: GenuineIntel Family 6 Model 158 Stepping 11
19:02:50:WU01:FS00:0x22: CPUs: 4
19:02:50:WU01:FS00:0x22: Memory: 3.78GiB
19:02:50:WU01:FS00:0x22:Free Memory: 1.48GiB
19:02:50:WU01:FS00:0x22: Threads: POSIX_THREADS
19:02:50:WU01:FS00:0x22: OS Version: 5.0
19:02:50:WU01:FS00:0x22:Has Battery: false
19:02:50:WU01:FS00:0x22: On Battery: false
19:02:50:WU01:FS00:0x22: UTC Offset: -4
19:02:50:WU01:FS00:0x22: PID: 1931
19:02:50:WU01:FS00:0x22: CWD: /var/lib/fahclient/work
19:02:50:WU01:FS00:0x22: OS: Linux 5.0.0-32-generic x86_64
19:02:50:WU01:FS00:0x22: OS Arch: AMD64
19:02:50:WU01:FS00:0x22:********************************************************************************
19:02:50:WU01:FS00:0x22:Project: 14562 (Run 0, Clone 1113, Gen 12)
19:02:50:WU01:FS00:0x22:Unit: 0x0000001080fccb025e9615bc26e6c65a
19:02:50:WU01:FS00:0x22:Reading tar file core.xml
19:02:50:WU01:FS00:0x22:Reading tar file integrator.xml
19:02:50:WU01:FS00:0x22:Reading tar file state.xml
19:02:53:WU01:FS00:0x22:Reading tar file system.xml
19:02:54:WU01:FS00:0x22:Digital signatures verified
19:02:54:WU01:FS00:0x22:Folding@home GPU Core22 Folding@home Core
19:02:54:WU01:FS00:0x22:Version 0.0.2
19:03:00:WU00:FS01:0x22:Completed 375000 out of 2500000 steps (15%)
19:03:11:WU01:FS00:0x22:ERROR:exception: Error compiling kernel:
19:03:11:WU01:FS00:0x22:Saving result file ../logfile_01.txt
19:03:11:WU01:FS00:0x22:Saving result file science.log
19:03:11:WU01:FS00:0x22:Folding@home Core Shutdown: BAD_WORK_UNIT
19:03:11:WARNING:WU01:FS00:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
19:03:11:WU01:FS00:Sending unit results: id:01 state:SEND error:FAULTY project:14562 run:0 clone:1113 gen:12 core:0x22 unit:0x0000001080fccb025e9615bc26e6c65a
19:03:11:WU01:FS00:Uploading 2.61KiB to 128.252.203.2
19:03:11:WU01:FS00:Connecting to 128.252.203.2:8080
19:03:11:WU01:FS00:Upload complete
19:03:11:WU01:FS00:Server responded WORK_ACK (400)
19:03:11:WU01:FS00:Cleaning up
19:03:11:WU02:FS00:Connecting to 65.254.110.245:80
19:03:12:WU02:FS00:Assigned to work server 128.252.203.2
19:03:12:WU02:FS00:Requesting new work unit for slot 00: READY gpu:1:TU104 [GeForce RTX 2080] from 128.252.203.2
19:03:12:WU02:FS00:Connecting to 128.252.203.2:8080
shorttack