17424 r0 c1536 g151 - ERROR:Force RMSE
Posted: Thu Feb 11, 2021 10:00 pm
I saw this today, not seen before (ERROR:Force RMSE error of 6.46327 with threshold of 5):
Something for me to wory about?
Thanks,
Ben N1NP
Code: Select all
*********************** Log Started 2021-01-21T16:14:08Z ***********************
16:14:08:******************************* libFAH ********************************
16:14:08: Date: Oct 20 2020
16:14:08: Time: 20:36:39
16:14:08: Revision: 5ca109d295a6245e2a2f590b3d0085ad5e567aeb
16:14:08: Branch: master
16:14:08: Compiler: GNU 8.3.0
16:14:08: Options: -faligned-new -std=c++11 -fsigned-char -ffunction-sections
16:14:08: -fdata-sections -O3 -funroll-loops -fno-pie
16:14:08: Platform: linux2 5.8.0-1-amd64
16:14:08: Bits: 64
16:14:08: Mode: Release
16:14:08:****************************** FAHClient ******************************
16:14:08: Version: 7.6.21
16:14:08: Author: Joseph Coffland <joseph@cauldrondevelopment.com>
16:14:08: Copyright: 2020 foldingathome.org
16:14:08: Homepage: https://foldingathome.org/
16:14:08: Date: Oct 20 2020
16:14:08: Time: 20:39:00
16:14:08: Revision: 6efbf0e138e22d3963e6a291f78dcb9c6422a278
16:14:08: Branch: master
16:14:08: Compiler: GNU 8.3.0
16:14:08: Options: -faligned-new -std=c++11 -fsigned-char -ffunction-sections
16:14:08: -fdata-sections -O3 -funroll-loops -fno-pie
16:14:08: Platform: linux2 5.8.0-1-amd64
16:14:08: Bits: 64
16:14:08: Mode: Release
16:14:08: Args: --child /etc/fahclient/config.xml --run-as fahclient
16:14:08: --pid-file=/var/run/fahclient.pid --daemon
16:14:08: Config: /etc/fahclient/config.xml
16:14:08:******************************** CBang ********************************
16:14:08: Date: Oct 20 2020
16:14:08: Time: 18:37:59
16:14:08: Revision: 7e4ce85225d7eaeb775e87c31740181ca603de60
16:14:08: Branch: master
16:14:08: Compiler: GNU 8.3.0
16:14:08: Options: -faligned-new -std=c++11 -fsigned-char -ffunction-sections
16:14:08: -fdata-sections -O3 -funroll-loops -fno-pie -fPIC
16:14:08: Platform: linux2 5.8.0-1-amd64
16:14:08: Bits: 64
16:14:08: Mode: Release
16:14:08:******************************* System ********************************
16:14:08: CPU: Intel(R) Xeon(R) CPU E5-2640 v2 @ 2.00GHz
16:14:08: CPU ID: GenuineIntel Family 6 Model 62 Stepping 4
16:14:08: CPUs: 32
16:14:08: Memory: 62.87GiB
16:14:08: Free Memory: 61.71GiB
16:14:08: Threads: POSIX_THREADS
16:14:08: OS Version: 5.4
16:14:08: Has Battery: false
16:14:08: On Battery: false
16:14:08: UTC Offset: -5
16:14:08: PID: 1594
16:14:08: CWD: /var/lib/fahclient
16:14:08: OS: Linux 5.4.0-62-generic x86_64
16:14:08: OS Arch: AMD64
16:14:08: GPUs: 2
16:14:08: GPU 0: Bus:4 Slot:0 Func:0 NVIDIA:3 GK104 [Quadro K4000]
16:14:08: GPU 1: Bus:33 Slot:0 Func:0 NVIDIA:3 GK104 [Quadro K4000]
16:14:08: CUDA Device 0: Platform:0 Device:0 Bus:4 Slot:0 Compute:3.0 Driver:11.0
16:14:08: CUDA Device 1: Platform:0 Device:1 Bus:33 Slot:0 Compute:3.0 Driver:11.0
16:14:08:OpenCL Device 0: Platform:0 Device:0 Bus:4 Slot:0 Compute:1.2 Driver:450.102
16:14:08:OpenCL Device 1: Platform:0 Device:1 Bus:33 Slot:0 Compute:1.2 Driver:450.102
16:14:08:***********************************************************************
16:14:08:<config>
16:14:08: <!-- Client Control -->
16:14:08: <fold-anon v='true'/>
16:14:08:
16:14:08: <!-- HTTP Server -->
16:14:08: <allow v='192.168.0.1/24'/>
16:14:08:
16:14:08: <!-- Network -->
16:14:08: <proxy v=':8080'/>
16:14:08:
16:14:08: <!-- Remote Command Server -->
16:14:08: <password v='*****'/>
16:14:08:
16:14:08: <!-- User Information -->
16:14:08: <passkey v='*****'/>
16:14:08: <team v='12912'/>
16:14:08: <user v='n1np'/>
16:14:08:
16:14:08: <!-- Folding Slots -->
16:14:08: <slot id='0' type='CPU'>
16:14:08: <cpus v='28'/>
16:14:08: <paused v='true'/>
16:14:08: </slot>
16:14:08: <slot id='1' type='GPU'>
16:14:08: <paused v='true'/>
16:14:08: <pci-bus v='4'/>
16:14:08: <pci-slot v='0'/>
16:14:08: </slot>
16:14:08: <slot id='2' type='GPU'>
16:14:08: <paused v='true'/>
16:14:08: <pci-bus v='33'/>
16:14:08: <pci-slot v='0'/>
16:14:08: </slot>
16:14:08:</config>
16:14:08:Trying to access database...
16:14:08:Successfully acquired database lock
16:14:08:FS00:Initialized folding slot 00: cpu:28
16:14:08:FS01:Initialized folding slot 01: gpu:4:0 GK104 [Quadro K4000]
16:14:08:FS02:Initialized folding slot 02: gpu:33:0 GK104 [Quadro K4000]
16:16:08:FS00:Unpaused
16:16:08:FS01:Unpaused
16:16:08:FS02:Unpaused
**** EDIT ****
20:45:35:WU00:FS01:0x22:Completed 1250000 out of 1250000 steps (100%)
20:45:35:WU00:FS01:0x22:Average performance: 256.38 ns/day
20:45:35:WU00:FS01:0x22:Checkpoint completed at step 1250000
20:45:42:WU00:FS01:0x22:Saving result file ../logfile_01.txt
20:45:42:WU00:FS01:0x22:Saving result file checkpointIntegrator.xml.bz2
20:45:42:WU00:FS01:0x22:Saving result file checkpointState.xml.bz2
20:45:42:WU00:FS01:0x22:Saving result file positions.xtc
20:45:42:WU00:FS01:0x22:Saving result file science.log
20:45:42:WU00:FS01:0x22:Folding@home Core Shutdown: FINISHED_UNIT
20:45:43:WU00:FS01:FahCore returned: FINISHED_UNIT (100 = 0x64)
20:45:43:WU00:FS01:Sending unit results: id:00 state:SEND error:NO_ERROR project:17313 run:0 clone:1900 gen:133 core:0x22 unit:0x0000076c00000085000043a100000000
20:45:43:WU00:FS01:Uploading 3.81MiB to 140.163.4.200
20:45:43:WU00:FS01:Connecting to 140.163.4.200:8080
20:45:49:WU00:FS01:Upload 80.47%
20:45:51:WU00:FS01:Upload complete
20:45:51:WU00:FS01:Server responded WORK_ACK (400)
20:45:51:WU00:FS01:Final credit estimate, 13518.00 points
20:45:51:WU00:FS01:Cleaning up
20:46:57:WU04:FS02:0x22:Completed 2500000 out of 5000000 steps (50%)
20:46:57:WU04:FS02:0x22:Checkpoint completed at step 2500000
20:48:03:WU02:FS01:Connecting to assign1.foldingathome.org:80
20:48:03:WU02:FS01:Assigned to work server 206.223.170.146
20:48:03:WU02:FS01:Requesting new work unit for slot 01: gpu:4:0 GK104 [Quadro K4000] from 206.223.170.146
20:48:03:WU02:FS01:Connecting to 206.223.170.146:8080
20:48:04:WU02:FS01:Downloading 13.44MiB
20:48:10:WU02:FS01:Download 82.32%
20:48:11:WU02:FS01:Download complete
20:48:11:WU02:FS01:Received Unit: id:02 state:DOWNLOAD error:NO_ERROR project:17424 run:0 clone:1536 gen:151 core:0x22 unit:0x00000600000000970000441000000000
20:48:11:WU02:FS01:Starting
20:48:11:WU02:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/lin/64bit/22-0.0.13/Core_22.fah/FahCore_22 -dir 02 -suffix 01 -version 706 -lifeline 1594 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
20:48:11:WU02:FS01:Started FahCore on PID 37006
20:48:11:WU02:FS01:Core PID:37010
20:48:11:WU02:FS01:FahCore 0x22 started
20:48:12:WU02:FS01:0x22:*********************** Log Started 2021-02-11T20:48:11Z ***********************
20:48:12:WU02:FS01:0x22:*************************** Core22 Folding@home Core ***************************
20:48:12:WU02:FS01:0x22: Core: Core22
20:48:12:WU02:FS01:0x22: Type: 0x22
20:48:12:WU02:FS01:0x22: Version: 0.0.13
20:48:12:WU02:FS01:0x22: Author: Joseph Coffland <joseph@cauldrondevelopment.com>
20:48:12:WU02:FS01:0x22: Copyright: 2020 foldingathome.org
20:48:12:WU02:FS01:0x22: Homepage: https://foldingathome.org/
20:48:12:WU02:FS01:0x22: Date: Sep 19 2020
20:48:12:WU02:FS01:0x22: Time: 01:10:35
20:48:12:WU02:FS01:0x22: Revision: 571cf95de6de2c592c7c3ed48fcfb2e33e9ea7d3
20:48:12:WU02:FS01:0x22: Branch: core22-0.0.13
20:48:12:WU02:FS01:0x22: Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
20:48:12:WU02:FS01:0x22: Options: -std=c++11 -fsigned-char -ffunction-sections -fdata-sections -O3
20:48:12:WU02:FS01:0x22: -funroll-loops -DOPENMM_GIT_HASH=\"\\\"189320d0\\\"\"
20:48:12:WU02:FS01:0x22: Platform: linux2 4.19.76-linuxkit
20:48:12:WU02:FS01:0x22: Bits: 64
20:48:12:WU02:FS01:0x22: Mode: Release
20:48:12:WU02:FS01:0x22:Maintainers: John Chodera <john.chodera@choderalab.org> and Peter Eastman
20:48:12:WU02:FS01:0x22: <peastman@stanford.edu>
20:48:12:WU02:FS01:0x22: Args: -dir 02 -suffix 01 -version 706 -lifeline 37006 -checkpoint 15
20:48:12:WU02:FS01:0x22: -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor
20:48:12:WU02:FS01:0x22: nvidia -gpu 0 -gpu-usage 100
20:48:12:WU02:FS01:0x22:************************************ libFAH ************************************
20:48:12:WU02:FS01:0x22: Date: Sep 15 2020
20:48:12:WU02:FS01:0x22: Time: 05:14:43
20:48:12:WU02:FS01:0x22: Revision: 44301ed97b996b63fe736bb8073f22209cb2b603
20:48:12:WU02:FS01:0x22: Branch: HEAD
20:48:12:WU02:FS01:0x22: Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
20:48:12:WU02:FS01:0x22: Options: -std=c++11 -fsigned-char -ffunction-sections -fdata-sections -O3
20:48:12:WU02:FS01:0x22: -funroll-loops
20:48:12:WU02:FS01:0x22: Platform: linux2 4.19.76-linuxkit
20:48:12:WU02:FS01:0x22: Bits: 64
20:48:12:WU02:FS01:0x22: Mode: Release
20:48:12:WU02:FS01:0x22:************************************ CBang *************************************
20:48:12:WU02:FS01:0x22: Date: Sep 15 2020
20:48:12:WU02:FS01:0x22: Time: 05:11:04
20:48:12:WU02:FS01:0x22: Revision: 33fcfc2b3ed2195a423606a264718e31e6b3903f
20:48:12:WU02:FS01:0x22: Branch: HEAD
20:48:12:WU02:FS01:0x22: Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
20:48:12:WU02:FS01:0x22: Options: -std=c++11 -fsigned-char -ffunction-sections -fdata-sections -O3
20:48:12:WU02:FS01:0x22: -funroll-loops -fPIC
20:48:12:WU02:FS01:0x22: Platform: linux2 4.19.76-linuxkit
20:48:12:WU02:FS01:0x22: Bits: 64
20:48:12:WU02:FS01:0x22: Mode: Release
20:48:12:WU02:FS01:0x22:************************************ System ************************************
20:48:12:WU02:FS01:0x22: CPU: Intel(R) Xeon(R) CPU E5-2640 v2 @ 2.00GHz
20:48:12:WU02:FS01:0x22: CPU ID: GenuineIntel Family 6 Model 62 Stepping 4
20:48:12:WU02:FS01:0x22: CPUs: 32
20:48:12:WU02:FS01:0x22: Memory: 62.87GiB
20:48:12:WU02:FS01:0x22:Free Memory: 704.75MiB
20:48:12:WU02:FS01:0x22: Threads: POSIX_THREADS
20:48:12:WU02:FS01:0x22: OS Version: 5.4
20:48:12:WU02:FS01:0x22:Has Battery: false
20:48:12:WU02:FS01:0x22: On Battery: false
20:48:12:WU02:FS01:0x22: UTC Offset: -5
20:48:12:WU02:FS01:0x22: PID: 37010
20:48:12:WU02:FS01:0x22: CWD: /var/lib/fahclient/work
20:48:12:WU02:FS01:0x22:************************************ OpenMM ************************************
20:48:12:WU02:FS01:0x22: Revision: 189320d0
20:48:12:WU02:FS01:0x22:********************************************************************************
20:48:12:WU02:FS01:0x22:Project: 17424 (Run 0, Clone 1536, Gen 151)
20:48:12:WU02:FS01:0x22:Unit: 0x00000000000000000000000000000000
20:48:12:WU02:FS01:0x22:Reading tar file core.xml
20:48:12:WU02:FS01:0x22:Reading tar file integrator.xml.bz2
20:48:12:WU02:FS01:0x22:Reading tar file state.xml.bz2
20:48:12:WU02:FS01:0x22:Reading tar file system.xml.bz2
20:48:12:WU02:FS01:0x22:Digital signatures verified
20:48:12:WU02:FS01:0x22:Folding@home GPU Core22 Folding@home Core
20:48:12:WU02:FS01:0x22:Version 0.0.13
20:48:12:WU02:FS01:0x22: Checkpoint write interval: 25000 steps (2%) [50 total]
20:48:12:WU02:FS01:0x22: JSON viewer frame write interval: 12500 steps (1%) [100 total]
20:48:12:WU02:FS01:0x22: XTC frame write interval: 10000 steps (0.8%) [125 total]
20:48:12:WU02:FS01:0x22: Global context and integrator variables write interval: disabled
20:48:12:WU02:FS01:0x22:There are 4 platforms available.
20:48:12:WU02:FS01:0x22:Platform 0: Reference
20:48:12:WU02:FS01:0x22:Platform 1: CPU
20:48:12:WU02:FS01:0x22:Platform 2: OpenCL
20:48:12:WU02:FS01:0x22: opencl-device 0 specified
20:48:12:WU02:FS01:0x22:Platform 3: CUDA
20:48:12:WU02:FS01:0x22: cuda-device 0 specified
20:48:31:WU02:FS01:0x22:Attempting to create CUDA context:
20:48:31:WU02:FS01:0x22: Configuring platform CUDA
*** PROBLEM HERE:
20:48:37:WU02:FS01:0x22:ERROR:Force RMSE error of 6.46327 with threshold of 5
20:48:37:WU02:FS01:0x22:Saving result file ../logfile_01.txt
20:48:37:WU02:FS01:0x22:Saving result file science.log
20:48:37:WU02:FS01:0x22:Saving result file state.xml.bz2
20:48:37:WU02:FS01:0x22:Folding@home Core Shutdown: BAD_WORK_UNIT
\x1b[93m20:48:37:WARNING:WU02:FS01:FahCore returned: BAD_WORK_UNIT (114 = 0x72)\x1b[0m
20:48:37:WU02:FS01:Sending unit results: id:02 state:SEND error:FAULTY project:17424 run:0 clone:1536 gen:151 core:0x22 unit:0x00000600000000970000441000000000
*** /PROBLEM
20:48:37:WU02:FS01:Uploading 11.73MiB to 206.223.170.146
20:48:37:WU02:FS01:Connecting to 206.223.170.146:8080
20:48:37:WU00:FS01:Connecting to assign1.foldingathome.org:80
\x1b[93m20:48:38:WARNING:WU00:FS01:Failed to get assignment from 'assign1.foldingathome.org:80': No WUs available for this configuration\x1b[0m
20:48:38:WU00:FS01:Connecting to assign2.foldingathome.org:80
\x1b[93m20:48:39:WARNING:WU00:FS01:Failed to get assignment from 'assign2.foldingathome.org:80': No WUs available for this configuration\x1b[0m
20:48:39:WU00:FS01:Connecting to assign3.foldingathome.org:80
\x1b[93m20:48:39:WARNING:WU00:FS01:Failed to get assignment from 'assign3.foldingathome.org:80': No WUs available for this configuration\x1b[0m
20:48:39:WU00:FS01:Connecting to assign4.foldingathome.org:80
\x1b[93m20:48:40:WARNING:WU00:FS01:Failed to get assignment from 'assign4.foldingathome.org:80': No WUs available for this configuration\x1b[0m
\x1b[91m20:48:40:ERROR:WU00:FS01:Exception: Could not get an assignment\x1b[0m
20:48:40:WU00:FS01:Connecting to assign1.foldingathome.org:80
\x1b[93m20:48:40:WARNING:WU00:FS01:Failed to get assignment from 'assign1.foldingathome.org:80': No WUs available for this configuration\x1b[0m
20:48:40:WU00:FS01:Connecting to assign2.foldingathome.org:80
\x1b[93m20:48:41:WARNING:WU00:FS01:Failed to get assignment from 'assign2.foldingathome.org:80': No WUs available for this configuration\x1b[0m
20:48:41:WU00:FS01:Connecting to assign3.foldingathome.org:80
\x1b[93m20:48:42:WARNING:WU00:FS01:Failed to get assignment from 'assign3.foldingathome.org:80': No WUs available for this configuration\x1b[0m
20:48:42:WU00:FS01:Connecting to assign4.foldingathome.org:80
\x1b[93m20:48:42:WARNING:WU00:FS01:Failed to get assignment from 'assign4.foldingathome.org:80': No WUs available for this configuration\x1b[0m
\x1b[91m20:48:42:ERROR:WU00:FS01:Exception: Could not get an assignment\x1b[0m
20:48:43:WU02:FS01:Upload 28.24%
20:48:49:WU02:FS01:Upload 58.62%
20:48:55:WU02:FS01:Upload 88.46%
20:48:58:WU02:FS01:Upload complete
20:48:58:WU02:FS01:Server responded WORK_ACK (400)
20:48:58:WU02:FS01:Cleaning up
20:49:38:WU01:FS00:0xa8:Completed 1900000 out of 5000000 steps (38%)
20:49:40:WU00:FS01:Connecting to assign1.foldingathome.org:80
\x1b[93m20:49:40:WARNING:WU00:FS01:Failed to get assignment from 'assign1.foldingathome.org:80': No WUs available for this configuration\x1b[0m
20:49:40:WU00:FS01:Connecting to assign2.foldingathome.org:80Thanks,
Ben N1NP