Bad Work Unit for Project 12222

[SandyG]

Seeing this also for WU 12222, FS02 is a 3090 card also running on the Mint Linux (Ubuntu) machine, like the other reported 12214 Work unit earlier that I saw. More details there as well. Not sure if important as these look like new WU that I have not see before so something may be up with them. Or just not working on my 3090 and 4090 cards. HTH

04:24:09:WU04:FS02:Assigned to work server 158.130.118.23
04:24:09:WU04:FS02:Requesting new work unit for slot 02: gpu:101:0 GA102 [GeForce RTX 3090] from 158.130.118.23
04:24:09:WU04:FS02:Connecting to 158.130.118.23:8080
04:24:09:WU04:FS02:Downloading 7.31MiB
04:24:14:WU03:FS03:0x22:Completed 280000 out of 500000 steps (56%)
04:24:14:WU04:FS02:Download complete
04:24:14:WU04:FS02:Received Unit: id:04 state:DOWNLOAD error:NO_ERROR project:12222 run:0 clone:31 gen:6 core:0x22 unit:0x0000001f0000000600002fbe00000000
04:24:14:WU03:FS03:0x22:Checkpoint completed at step 280000
04:24:18:WU00:FS04:0x22:Completed 490000 out of 500000 steps (98%)
04:24:19:WU00:FS04:0x22:Checkpoint completed at step 490000
04:24:26:WU03:FS03:0x22:Completed 285000 out of 500000 steps (57%)
04:24:29:WU01:FS02:0x22:Completed 2500000 out of 2500000 steps (100%)
04:24:29:WU01:FS02:0x22:Average performance: 401.86 ns/day
04:24:29:WU01:FS02:0x22:Checkpoint completed at step 2500000
04:24:30:WU00:FS04:0x22:Completed 495000 out of 500000 steps (99%)
04:24:31:WU05:FS04:Connecting to assign1.foldingathome.org:80
04:24:31:WU01:FS02:0x22:Saving result file ../logfile_01.txt
04:24:31:WU01:FS02:0x22:Saving result file checkpointIntegrator.xml
04:24:31:WU01:FS02:0x22:Saving result file checkpointState.xml.bz2
04:24:31:WU01:FS02:0x22:Saving result file positions.xtc
04:24:31:WU01:FS02:0x22:Saving result file science.log
04:24:31:WU01:FS02:0x22:Saving result file xtcAtoms.csv.bz2
04:24:31:WU01:FS02:0x22:Folding@home Core Shutdown: FINISHED_UNIT
04:24:32:WU05:FS04:Assigned to work server 140.163.4.210
04:24:32:WU05:FS04:Requesting new work unit for slot 04: gpu:184:0 AD102 [GeForce RTX 4090] from 140.163.4.210
04:24:32:WU05:FS04:Connecting to 140.163.4.210:8080
04:24:32:WU01:FS02:FahCore returned: FINISHED_UNIT (100 = 0x64)
04:24:32:WU01:FS02:Sending unit results: id:01 state:SEND error:NO_ERROR project:12211 run:0 clone:41 gen:8 core:0x22 unit:0x000000290000000800002fb300000000
04:24:32:WU01:FS02:Uploading 6.75MiB to 158.130.118.23
04:24:32:WU01:FS02:Connecting to 158.130.118.23:8080
04:24:32:WU04:FS02:Starting
04:24:32:WU04:FS02:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/lin/64bit/22-0.0.20/Core_22.fah/FahCore_22 -dir 04 -suffix 01 -version 706 -lifeline 1607 -checkpoint 15 -opencl-platform 0 -opencl-device 3 -cuda-device 3 -gpu-vendor nvidia -gpu 3 -gpu-usage 100
04:24:32:WU04:FS02:Started FahCore on PID 168311
04:24:32:WU04:FS02:Core PID:168315
04:24:32:WU04:FS02:FahCore 0x22 started
04:24:32:WU05:FS04:Downloading 7.45MiB
04:24:32:WU04:FS02:0x22:*********************** Log Started 2023-07-29T04:24:32Z ***********************
04:24:32:WU04:FS02:0x22:*************************** Core22 Folding@home Core ***************************
04:24:32:WU04:FS02:0x22: Core: Core22
04:24:32:WU04:FS02:0x22: Type: 0x22
04:24:32:WU04:FS02:0x22: Version: 0.0.20
04:24:32:WU04:FS02:0x22: Author: Joseph Coffland <joseph@cauldrondevelopment.com>
04:24:32:WU04:FS02:0x22: Copyright: 2020 foldingathome.org
04:24:32:WU04:FS02:0x22: Homepage: https://foldingathome.org/
04:24:32:WU04:FS02:0x22: Date: Jan 20 2022
04:24:32:WU04:FS02:0x22: Time: 00:57:52
04:24:32:WU04:FS02:0x22: Revision: 3f211b8a4346514edbff34e3cb1c0e0ec951373c
04:24:32:WU04:FS02:0x22: Branch: HEAD
04:24:32:WU04:FS02:0x22: Compiler: GNU 9.4.0
04:24:32:WU04:FS02:0x22: Options: -faligned-new -std=c++11 -fsigned-char -ffunction-sections
04:24:32:WU04:FS02:0x22: -fdata-sections -O3 -funroll-loops -fno-pie
04:24:32:WU04:FS02:0x22: -DOPENMM_VERSION="\"7.7.0\""
04:24:32:WU04:FS02:0x22: Platform: linux 5.11.0-1025-azure
04:24:32:WU04:FS02:0x22: Bits: 64
04:24:32:WU04:FS02:0x22: Mode: Release
04:24:32:WU04:FS02:0x22:Maintainers: John Chodera <john.chodera@choderalab.org> and Peter Eastman
04:24:32:WU04:FS02:0x22: <peastman@stanford.edu>
04:24:32:WU04:FS02:0x22: Args: -dir 04 -suffix 01 -version 706 -lifeline 168311 -checkpoint 15
04:24:32:WU04:FS02:0x22: -opencl-platform 0 -opencl-device 3 -cuda-device 3 -gpu-vendor
04:24:32:WU04:FS02:0x22: nvidia -gpu 3 -gpu-usage 100
04:24:32:WU04:FS02:0x22:************************************ libFAH ************************************
04:24:32:WU04:FS02:0x22: Date: Jan 20 2022
04:24:32:WU04:FS02:0x22: Time: 00:57:22
04:24:32:WU04:FS02:0x22: Revision: 9f4ad694e75c2350d4bb6b8b5b769ba27e483a2f
04:24:32:WU04:FS02:0x22: Branch: HEAD
04:24:32:WU04:FS02:0x22: Compiler: GNU 9.4.0
04:24:32:WU04:FS02:0x22: Options: -faligned-new -std=c++11 -fsigned-char -ffunction-sections
04:24:32:WU04:FS02:0x22: -fdata-sections -O3 -funroll-loops -fno-pie
04:24:32:WU04:FS02:0x22: Platform: linux 5.11.0-1025-azure
04:24:32:WU04:FS02:0x22: Bits: 64
04:24:32:WU04:FS02:0x22: Mode: Release
04:24:32:WU04:FS02:0x22:************************************ CBang *************************************
04:24:32:WU04:FS02:0x22: Date: Jan 20 2022
04:24:32:WU04:FS02:0x22: Time: 00:57:00
04:24:32:WU04:FS02:0x22: Revision: ab023d155b446906d55b0f6c9a1eedeea04f7a1a
04:24:32:WU04:FS02:0x22: Branch: HEAD
04:24:32:WU04:FS02:0x22: Compiler: GNU 9.4.0
04:24:32:WU04:FS02:0x22: Options: -faligned-new -std=c++11 -fsigned-char -ffunction-sections
04:24:32:WU04:FS02:0x22: -fdata-sections -O3 -funroll-loops -fno-pie -fPIC
04:24:32:WU04:FS02:0x22: Platform: linux 5.11.0-1025-azure
04:24:32:WU04:FS02:0x22: Bits: 64
04:24:32:WU04:FS02:0x22: Mode: Release
04:24:32:WU04:FS02:0x22:************************************ System ************************************
04:24:32:WU04:FS02:0x22: CPU: Intel(R) Xeon(R) W-2155 CPU @ 3.30GHz
04:24:32:WU04:FS02:0x22: CPU ID: GenuineIntel Family 6 Model 85 Stepping 4
04:24:32:WU04:FS02:0x22: CPUs: 20
04:24:32:WU04:FS02:0x22: Memory: 125.50GiB
04:24:32:WU04:FS02:0x22:Free Memory: 115.55GiB
04:24:32:WU04:FS02:0x22: Threads: POSIX_THREADS
04:24:32:WU04:FS02:0x22: OS Version: 5.15
04:24:32:WU04:FS02:0x22:Has Battery: false
04:24:32:WU04:FS02:0x22: On Battery: false
04:24:32:WU04:FS02:0x22: UTC Offset: -7
04:24:32:WU04:FS02:0x22: PID: 168315
04:24:32:WU04:FS02:0x22: CWD: /var/lib/fahclient/work
04:24:32:WU04:FS02:0x22:************************************ OpenMM ************************************
04:24:32:WU04:FS02:0x22: Version: 7.7.0
04:24:32:WU04:FS02:0x22:********************************************************************************
04:24:32:WU04:FS02:0x22:Project: 12222 (Run 0, Clone 31, Gen 6)
04:24:32:WU04:FS02:0x22:Reading tar file core.xml
04:24:32:WU04:FS02:0x22:Reading tar file integrator.xml
04:24:32:WU04:FS02:0x22:Reading tar file state.xml.bz2
04:24:32:WU04:FS02:0x22:Reading tar file system.xml.bz2
04:24:32:WU04:FS02:0x22:Digital signatures verified
04:24:32:WU04:FS02:0x22:Folding@home GPU Core22 Folding@home Core
04:24:32:WU04:FS02:0x22:Version 0.0.20
04:24:32:WU04:FS02:0x22: Checkpoint write interval: 50000 steps (2%) [50 total]
04:24:32:WU04:FS02:0x22: JSON viewer frame write interval: 25000 steps (1%) [100 total]
04:24:32:WU04:FS02:0x22: XTC frame write interval: 25000 steps (1%) [100 total]
04:24:32:WU04:FS02:0x22: Global context and integrator variables write interval: disabled
04:24:32:WU04:FS02:0x22:There are 4 platforms available.
04:24:32:WU04:FS02:0x22:Platform 0: Reference
04:24:32:WU04:FS02:0x22:Platform 1: CPU
04:24:32:WU04:FS02:0x22:Platform 2: OpenCL
04:24:32:WU04:FS02:0x22: opencl-device 3 specified
04:24:32:WU04:FS02:0x22:Platform 3: CUDA
04:24:32:WU04:FS02:0x22: cuda-device 3 specified
04:24:35:WU04:FS02:0x22:Attempting to create CUDA context:
04:24:35:WU04:FS02:0x22: Configuring platform CUDA
04:24:38:WU05:FS04:Download 65.40%
04:24:38:WU01:FS02:Upload 62.05%
04:24:38:WU03:FS03:0x22:Completed 290000 out of 500000 steps (58%)
04:24:39:WU03:FS03:0x22:Checkpoint completed at step 290000
04:24:39:WU04:FS02:0x22:ERROR:Force RMSE error of 10.0771 with threshold of 10
04:24:39:WU04:FS02:0x22:Saving result file ../logfile_01.txt
04:24:39:WU04:FS02:0x22:Saving result file science.log
04:24:39:WU04:FS02:0x22:Saving result file state.xml.bz2
04:24:39:WU04:FS02:0x22:Folding@home Core Shutdown: BAD_WORK_UNIT
04:24:40:WARNING:WU04:FS02:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
04:24:40:WU04:FS02:Sending unit results: id:04 state:SEND error:FAULTY project:12222 run:0 clone:31 gen:6 core:0x22 unit:0x0000001f0000000600002fbe00000000
04:24:40:WU04:FS02:Uploading 6.09MiB to 158.130.118.23
04:24:40:WU04:FS02:Connecting to 158.130.118.23:8080
04:24:40:WU06:FS02:Connecting to assign1.foldingathome.org:80
04:24:42:WU06:FS02:Assigned to work server 129.32.209.202
04:24:42:WU06:FS02:Requesting new work unit for slot 02: gpu:101:0 GA102 [GeForce RTX 3090] from 129.32.209.202
04:24:42:WU06:FS02:Connecting to 129.32.209.202:8080
04:24:42:WU05:FS04:Download complete
04:24:42:WU05:FS04:Received Unit: id:05 state:DOWNLOAD error:NO_ERROR project:17636 run:6 clone:0 gen:292 core:0x22 unit:0x0000000000000124000044e400000006
04:24:42:WU00:FS04:0x22:Completed 500000 out of 500000 steps (100%)
04:24:42:WU00:FS04:0x22:Average performance: 75.1304 ns/day
04:24:43:WU00:FS04:0x22:Checkpoint completed at step 500000
04:24:43:WU06:FS02:Downloading 56.95MiB
04:24:44:WU01:FS02:Upload complete
04:24:45:WU01:FS02:Server responded WORK_ACK (400)
04:24:45:WU01:FS02:Final credit estimate, 200154.00 points
04:24:45:WU01:FS02:Cleaning up
04:24:45:WU02:FS01:0x22:Completed 1150000 out of 2500000 steps (46%)
04:24:45:WU02:FS01:0x22:Checkpoint completed at step 1150000
04:24:46:WU04:FS02:Upload 25.67%
04:24:47:WU00:FS04:0x22:Saving result file ../logfile_01.txt
04:24:47:WU00:FS04:0x22:Saving result file checkpointIntegrator.xml
04:24:47:WU00:FS04:0x22:Saving result file checkpointState.xml
04:24:49:WU06:FS02:Download 10.10%
04:24:51:WU03:FS03:0x22:Completed 295000 out of 500000 steps (59%)
04:24:52:WU04:FS02:Upload 98.59%
04:24:52:WU00:FS04:0x22:Saving result file positions.xtc
04:24:52:WU00:FS04:0x22:Saving result file science.log
04:24:52:WU00:FS04:0x22:Saving result file xtcAtoms.csv.bz2
04:24:52:WU00:FS04:0x22:Folding@home Core Shutdown: FINISHED_UNIT
04:24:53:WU00:FS04:FahCore returned: FINISHED_UNIT (100 = 0x64)
04:24:53:WU00:FS04:Sending unit results: id:00 state:SEND error:NO_ERROR project:12424 run:2 clone:71 gen:270 core:0x22 unit:0x000000470000010e0000308800000002
04:24:53:WU00:FS04:Uploading 38.24MiB to 129.32.209.200
04:24:53:WU00:FS04:Connecting to 129.32.209.200:8080
04:24:53:WU05:FS04:Starting
04:24:53:WU05:FS04:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/lin/64bit/22-0.0.20/Core_22.fah/FahCore_22 -dir 05 -suffix 01 -version 706 -lifeline 1607 -checkpoint 15 -opencl-platform 0 -opencl-device 1 -cuda-device 1 -gpu-vendor nvidia -gpu 1 -gpu-usage 100
04:24:53:WU05:FS04:Started FahCore on PID 168444
04:24:54:WU05:FS04:Core PID:168448
04:24:54:WU05:FS04:FahCore 0x22 started
04:24:54:WU05:FS04:0x22:*********************** Log Started 2023-07-29T04:24:54Z ***********************
04:24:54:WU05:FS04:0x22:*************************** Core22 Folding@home Core ***************************
04:24:54:WU05:FS04:0x22: Core: Core22
04:24:54:WU05:FS04:0x22: Type: 0x22
04:24:54:WU05:FS04:0x22: Version: 0.0.20
04:24:54:WU05:FS04:0x22: Author: Joseph Coffland <joseph@cauldrondevelopment.com>
04:24:54:WU05:FS04:0x22: Copyright: 2020 foldingathome.org
04:24:54:WU05:FS04:0x22: Homepage: https://foldingathome.org/
04:24:54:WU05:FS04:0x22: Date: Jan 20 2022
04:24:54:WU05:FS04:0x22: Time: 00:57:52
04:24:54:WU05:FS04:0x22: Revision: 3f211b8a4346514edbff34e3cb1c0e0ec951373c
04:24:54:WU05:FS04:0x22: Branch: HEAD
04:24:54:WU05:FS04:0x22: Compiler: GNU 9.4.0
04:24:54:WU05:FS04:0x22: Options: -faligned-new -std=c++11 -fsigned-char -ffunction-sections
04:24:54:WU05:FS04:0x22: -fdata-sections -O3 -funroll-loops -fno-pie
04:24:54:WU05:FS04:0x22: -DOPENMM_VERSION="\"7.7.0\""
04:24:54:WU05:FS04:0x22: Platform: linux 5.11.0-1025-azure
04:24:54:WU05:FS04:0x22: Bits: 64
04:24:54:WU05:FS04:0x22: Mode: Release
04:24:54:WU05:FS04:0x22:Maintainers: John Chodera <john.chodera@choderalab.org> and Peter Eastman
04:24:54:WU05:FS04:0x22: <peastman@stanford.edu>
04:24:54:WU05:FS04:0x22: Args: -dir 05 -suffix 01 -version 706 -lifeline 168444 -checkpoint 15
04:24:54:WU05:FS04:0x22: -opencl-platform 0 -opencl-device 1 -cuda-device 1 -gpu-vendor
04:24:54:WU05:FS04:0x22: nvidia -gpu 1 -gpu-usage 100
04:24:54:WU05:FS04:0x22:************************************ libFAH ************************************
04:24:54:WU05:FS04:0x22: Date: Jan 20 2022
04:24:54:WU05:FS04:0x22: Time: 00:57:22
04:24:54:WU05:FS04:0x22: Revision: 9f4ad694e75c2350d4bb6b8b5b769ba27e483a2f
04:24:54:WU05:FS04:0x22: Branch: HEAD
04:24:54:WU05:FS04:0x22: Compiler: GNU 9.4.0
04:24:54:WU05:FS04:0x22: Options: -faligned-new -std=c++11 -fsigned-char -ffunction-sections
04:24:54:WU05:FS04:0x22: -fdata-sections -O3 -funroll-loops -fno-pie
04:24:54:WU05:FS04:0x22: Platform: linux 5.11.0-1025-azure
04:24:54:WU05:FS04:0x22: Bits: 64
04:24:54:WU05:FS04:0x22: Mode: Release
04:24:54:WU05:FS04:0x22:************************************ CBang *************************************
04:24:54:WU05:FS04:0x22: Date: Jan 20 2022
04:24:54:WU05:FS04:0x22: Time: 00:57:00
04:24:54:WU05:FS04:0x22: Revision: ab023d155b446906d55b0f6c9a1eedeea04f7a1a
04:24:54:WU05:FS04:0x22: Branch: HEAD
04:24:54:WU05:FS04:0x22: Compiler: GNU 9.4.0
04:24:54:WU05:FS04:0x22: Options: -faligned-new -std=c++11 -fsigned-char -ffunction-sections
04:24:54:WU05:FS04:0x22: -fdata-sections -O3 -funroll-loops -fno-pie -fPIC
04:24:54:WU05:FS04:0x22: Platform: linux 5.11.0-1025-azure
04:24:54:WU05:FS04:0x22: Bits: 64
04:24:54:WU05:FS04:0x22: Mode: Release
04:24:54:WU05:FS04:0x22:************************************ System ************************************
04:24:54:WU05:FS04:0x22: CPU: Intel(R) Xeon(R) W-2155 CPU @ 3.30GHz
04:24:54:WU05:FS04:0x22: CPU ID: GenuineIntel Family 6 Model 85 Stepping 4
04:24:54:WU05:FS04:0x22: CPUs: 20
04:24:54:WU05:FS04:0x22: Memory: 125.50GiB
04:24:54:WU05:FS04:0x22:Free Memory: 118.10GiB
04:24:54:WU05:FS04:0x22: Threads: POSIX_THREADS
04:24:54:WU05:FS04:0x22: OS Version: 5.15
04:24:54:WU05:FS04:0x22:Has Battery: false
04:24:54:WU05:FS04:0x22: On Battery: false
04:24:54:WU05:FS04:0x22: UTC Offset: -7
04:24:54:WU05:FS04:0x22: PID: 168448
04:24:54:WU05:FS04:0x22: CWD: /var/lib/fahclient/work
04:24:54:WU05:FS04:0x22:************************************ OpenMM ************************************
04:24:54:WU05:FS04:0x22: Version: 7.7.0
04:24:54:WU05:FS04:0x22:********************************************************************************
04:24:54:WU05:FS04:0x22:Project: 17636 (Run 6, Clone 0, Gen 292)
04:24:54:WU05:FS04:0x22:Reading tar file core.xml
04:24:54:WU05:FS04:0x22:Reading tar file integrator.xml.bz2
04:24:54:WU05:FS04:0x22:Reading tar file state.xml.bz2
04:24:54:WU05:FS04:0x22:Reading tar file system.xml.bz2
04:24:54:WU05:FS04:0x22:Digital signatures verified
04:24:54:WU05:FS04:0x22:Folding@home GPU Core22 Folding@home Core
04:24:54:WU05:FS04:0x22:Version 0.0.20
04:24:54:WU05:FS04:0x22: Checkpoint write interval: 125000 steps (5%) [20 total]
04:24:54:WU05:FS04:0x22: JSON viewer frame write interval: 25000 steps (1%) [100 total]
04:24:54:WU05:FS04:0x22: XTC frame write interval: 250000 steps (10%) [10 total]
04:24:54:WU05:FS04:0x22: Global context and integrator variables write interval: disabled
04:24:54:WU05:FS04:0x22:There are 4 platforms available.
04:24:54:WU05:FS04:0x22:Platform 0: Reference
04:24:54:WU05:FS04:0x22:Platform 1: CPU
04:24:54:WU05:FS04:0x22:Platform 2: OpenCL
04:24:54:WU05:FS04:0x22: opencl-device 1 specified
04:24:54:WU05:FS04:0x22:Platform 3: CUDA
04:24:54:WU05:FS04:0x22: cuda-device 1 specified
04:24:54:WU04:FS02:Upload complete

[toTOW]

Reported to the researcher.

[SandyG]

Thanks, hope it's a simple fix for them!