AMD Radeon HD 7870 Pitcairn cannot handle the force

It seems that a lot of GPU problems revolve around specific versions of drivers. Though AMD has their own support structure, you can often learn from information reported by others who fold.

Moderators: Site Moderators, FAHC Science Team

Post Reply
haveyaseen
Posts: 4
Joined: Mon Mar 16, 2020 10:37 am

AMD Radeon HD 7870 Pitcairn cannot handle the force

Post by haveyaseen »

After a long time I have finally managed to get the amdgpu driver installed on my Debian 10 machine with OpenCL support. Now I even installed kernel version linux-image-5.6.0-0.bpo.2-amd64 from buster-backports and successfully compiled rocm-dkms from the AMD repositories according to the instructions on https://rocmdocs.amd.com/en/latest/Inst ... tml#ubuntu (it won't compile for Linux 5.7 but that's a known issue as far as I'm aware).

Both ROCm and the Radeon driver 20.30 (https://wiki.debian.org/AMDGPUDriverOnStretchAndBuster2) do not seem to work properly for me. FahCore 22 keeps throwing various errors at below 1 percent progress such as
  • NaNs detected in forces,
  • Force RMSE error,
  • Discrepancy: Forces are blowing up!, and
  • 114: Max number of attempts to resume from last checkpoint reached. (Particle coordinate is nan)
The CPU core has forever been working just fine. And IIRC the GPU folds correctly on Windows with the latest Radeon driver.

Code: Select all

*********************** Log Started 2020-09-03T11:32:17Z ***********************
11:32:17:Trying to access database...
11:32:17:Successfully acquired database lock
11:32:17:Read GPUs.txt
11:32:17:Enabled folding slot 00: PAUSED cpu:5 (by user)
11:32:20:Enabled folding slot 01: PAUSED gpu:0:Pitcairn [Radeon HD 7800] (by user)
11:32:20:****************************** FAHClient ******************************
11:32:20:        Version: 7.6.13
11:32:20:         Author: Joseph Coffland <joseph@cauldrondevelopment.com>
11:32:20:      Copyright: 2020 foldingathome.org
11:32:20:       Homepage: https://foldingathome.org/
11:32:20:           Date: Apr 28 2020
11:32:20:           Time: 04:20:16
11:32:20:       Revision: 5a652817f46116b6e135503af97f18e094414e3b
11:32:20:         Branch: master
11:32:20:       Compiler: GNU 8.3.0
11:32:20:        Options: -std=c++11 -ffunction-sections -fdata-sections -O3
11:32:20:                 -funroll-loops -fno-pie
11:32:20:       Platform: linux2 4.19.0-5-amd64
11:32:20:           Bits: 64
11:32:20:           Mode: Release
11:32:20:         Config: /home/<user>/config.xml
11:32:20:******************************** CBang ********************************
11:32:20:           Date: Apr 25 2020
11:32:20:           Time: 00:07:53
11:32:20:       Revision: ea081a3b3b0f4a37c4d0440b4f1bc184197c7797
11:32:20:         Branch: master
11:32:20:       Compiler: GNU 8.3.0
11:32:20:        Options: -std=c++11 -ffunction-sections -fdata-sections -O3
11:32:20:                 -funroll-loops -fno-pie -fPIC
11:32:20:       Platform: linux2 4.19.0-5-amd64
11:32:20:           Bits: 64
11:32:20:           Mode: Release
11:32:20:******************************* System ********************************
11:32:20:            CPU: AMD Phenom(tm) II X6 1055T Processor
11:32:20:         CPU ID: AuthenticAMD Family 16 Model 10 Stepping 0
11:32:20:           CPUs: 6
11:32:20:         Memory: 7.78GiB
11:32:20:    Free Memory: 4.96GiB
11:32:20:        Threads: POSIX_THREADS
11:32:20:     OS Version: 5.6
11:32:20:    Has Battery: false
11:32:20:     On Battery: false
11:32:20:     UTC Offset: 2
11:32:20:            PID: 4307
11:32:20:            CWD: /home/<user>
11:32:20:             OS: Linux 5.6.0-0.bpo.2-amd64 x86_64
11:32:20:        OS Arch: AMD64
11:32:20:           GPUs: 1
11:32:20:          GPU 0: Bus:1 Slot:0 Func:0 AMD:5 Pitcairn [Radeon HD 7800]
11:32:20:           CUDA: Not detected: Failed to open dynamic library 'libcuda.so':
11:32:20:                 libcuda.so: cannot open shared object file: No such file or
11:32:20:                 directory
11:32:20:OpenCL Device 0: Platform:0 Device:0 Bus:1 Slot:0 Compute:1.2 Driver:3143.9
11:32:20:OpenCL Device 1: Platform:1 Device:0 Bus:NA Slot:NA Compute:1.1 Driver:18.3
11:32:20:******************************* libFAH ********************************
11:32:20:           Date: Apr 15 2020
11:32:20:           Time: 21:43:24
11:32:20:       Revision: 216968bc7025029c841ed6e36e81a03a316890d3
11:32:20:         Branch: master
11:32:20:       Compiler: GNU 8.3.0
11:32:20:        Options: -std=c++11 -ffunction-sections -fdata-sections -O3
11:32:20:                 -funroll-loops -fno-pie
11:32:20:       Platform: linux2 4.19.0-5-amd64
11:32:20:           Bits: 64
11:32:20:           Mode: Release
11:32:20:***********************************************************************
11:32:20:<config>
11:32:20:  <!-- Network -->
11:32:20:  <proxy v=':8080'/>
11:32:20:
11:32:20:  <!-- Slot Control -->
11:32:20:  <power v='full'/>
11:32:20:
11:32:20:  <!-- User Information -->
11:32:20:  <passkey v='*****'/>
11:32:20:  <team v='******'/>
11:32:20:  <user v='******'/>
11:32:20:
11:32:20:  <!-- Folding Slots -->
11:32:20:  <slot id='0' type='CPU'>
11:32:20:    <paused v='true'/>
11:32:20:  </slot>
11:32:20:  <slot id='1' type='GPU'>
11:32:20:    <paused v='true'/>
11:32:20:  </slot>
11:32:20:</config>
11:32:23:FS00:Unpaused
11:32:23:FS01:Unpaused
11:32:23:WU01:FS01:Starting
11:32:23:WU01:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /home/<user>/cores/cores.foldingathome.org/lin/64bit/22-0.0.11/Core_22.fah/FahCore_22 -dir 01 -suffix 01 -version 706 -lifeline 4307 -checkpoint 15 -gpu-vendor amd -opencl-platform 0 -opencl-device 0 -gpu 0
11:32:23:WU01:FS01:Started FahCore on PID 4403
11:32:23:WU01:FS01:Core PID:4407
11:32:23:WU01:FS01:FahCore 0x22 started
11:32:23:WU02:FS00:Starting
11:32:23:WU02:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /home/<user>/cores/cores.foldingathome.org/lin/64bit-sse2/a7-0.0.19/Core_a7.fah/FahCore_a7 -dir 02 -suffix 01 -version 706 -lifeline 4307 -checkpoint 15 -np 5
11:32:23:WU02:FS00:Started FahCore on PID 4408
11:32:23:WU02:FS00:Core PID:4412
11:32:23:WU02:FS00:FahCore 0xa7 started
11:32:24:WU01:FS01:0x22:*********************** Log Started 2020-09-03T11:32:23Z ***********************
11:32:24:WU01:FS01:0x22:*************************** Core22 Folding@home Core ***************************
11:32:24:WU01:FS01:0x22:       Core: Core22
11:32:24:WU01:FS01:0x22:       Type: 0x22
11:32:24:WU01:FS01:0x22:    Version: 0.0.11
11:32:24:WU01:FS01:0x22:     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
11:32:24:WU01:FS01:0x22:  Copyright: 2020 foldingathome.org
11:32:24:WU01:FS01:0x22:   Homepage: https://foldingathome.org/
11:32:24:WU01:FS01:0x22:       Date: Jun 27 2020
11:32:24:WU01:FS01:0x22:       Time: 22:50:00
11:32:24:WU01:FS01:0x22:   Revision: cfc2940c5dd1aa80f60daa6e28d4a2a417f74edb
11:32:24:WU01:FS01:0x22:     Branch: core22-0.0.11
11:32:24:WU01:FS01:0x22:   Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
11:32:24:WU01:FS01:0x22:    Options: -std=c++11 -fsigned-char -ffunction-sections -fdata-sections -O3
11:32:24:WU01:FS01:0x22:             -funroll-loops
11:32:24:WU01:FS01:0x22:   Platform: linux2 4.19.76-linuxkit
11:32:24:WU01:FS01:0x22:       Bits: 64
11:32:24:WU01:FS01:0x22:       Mode: Release
11:32:24:WU01:FS01:0x22:Maintainers: John Chodera <john.chodera@choderalab.org> and Peter Eastman
11:32:24:WU01:FS01:0x22:             <peastman@stanford.edu>
11:32:24:WU01:FS01:0x22:       Args: -dir 01 -suffix 01 -version 706 -lifeline 4403 -checkpoint 15
11:32:24:WU01:FS01:0x22:             -gpu-vendor amd -opencl-platform 0 -opencl-device 0 -gpu 0
11:32:24:WU01:FS01:0x22:************************************ libFAH ************************************
11:32:24:WU01:FS01:0x22:       Date: Jun 27 2020
11:32:24:WU01:FS01:0x22:       Time: 22:11:04
11:32:24:WU01:FS01:0x22:   Revision: 2b383f4f04f38511dff592885d7c0400e72bdf43
11:32:24:WU01:FS01:0x22:     Branch: HEAD
11:32:24:WU01:FS01:0x22:   Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
11:32:24:WU01:FS01:0x22:    Options: -std=c++11 -fsigned-char -ffunction-sections -fdata-sections -O3
11:32:24:WU01:FS01:0x22:             -funroll-loops
11:32:24:WU01:FS01:0x22:   Platform: linux2 4.19.76-linuxkit
11:32:24:WU01:FS01:0x22:       Bits: 64
11:32:24:WU01:FS01:0x22:       Mode: Release
11:32:24:WU01:FS01:0x22:************************************ CBang *************************************
11:32:24:WU01:FS01:0x22:       Date: Jun 27 2020
11:32:24:WU01:FS01:0x22:       Time: 22:10:11
11:32:24:WU01:FS01:0x22:   Revision: f8529962055b0e7bde23e429f5072ff758089dee
11:32:24:WU01:FS01:0x22:     Branch: HEAD
11:32:24:WU01:FS01:0x22:   Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
11:32:24:WU01:FS01:0x22:    Options: -std=c++11 -fsigned-char -ffunction-sections -fdata-sections -O3
11:32:24:WU01:FS01:0x22:             -funroll-loops -fPIC
11:32:24:WU01:FS01:0x22:   Platform: linux2 4.19.76-linuxkit
11:32:24:WU01:FS01:0x22:       Bits: 64
11:32:24:WU01:FS01:0x22:       Mode: Release
11:32:24:WU01:FS01:0x22:************************************ System ************************************
11:32:24:WU01:FS01:0x22:        CPU: AMD Phenom(tm) II X6 1055T Processor
11:32:24:WU01:FS01:0x22:     CPU ID: AuthenticAMD Family 16 Model 10 Stepping 0
11:32:24:WU01:FS01:0x22:       CPUs: 6
11:32:24:WU01:FS01:0x22:     Memory: 7.78GiB
11:32:24:WU01:FS01:0x22:Free Memory: 4.94GiB
11:32:24:WU01:FS01:0x22:    Threads: POSIX_THREADS
11:32:24:WU01:FS01:0x22: OS Version: 5.6
11:32:24:WU01:FS01:0x22:Has Battery: false
11:32:24:WU01:FS01:0x22: On Battery: false
11:32:24:WU01:FS01:0x22: UTC Offset: 2
11:32:24:WU01:FS01:0x22:        PID: 4407
11:32:24:WU01:FS01:0x22:        CWD: /home/<user>/work
11:32:24:WU01:FS01:0x22:********************************************************************************
11:32:24:WU01:FS01:0x22:Project: 13424 (Run 1439, Clone 24, Gen 3)
11:32:24:WU01:FS01:0x22:Unit: 0x0000000612bc7d9a5f4ef9711310f0bd
11:32:24:WU01:FS01:0x22:Digital signatures verified
11:32:24:WU01:FS01:0x22:Folding@home GPU Core22 Folding@home Core
11:32:24:WU01:FS01:0x22:Version 0.0.11
11:32:24:WU02:FS00:0xa7:*********************** Log Started 2020-09-03T11:32:23Z ***********************
11:32:24:WU02:FS00:0xa7:************************** Gromacs Folding@home Core ***************************
11:32:24:WU02:FS00:0xa7:       Type: 0xa7
11:32:24:WU02:FS00:0xa7:       Core: Gromacs
11:32:24:WU02:FS00:0xa7:       Args: -dir 02 -suffix 01 -version 706 -lifeline 4408 -checkpoint 15 -np 5
11:32:24:WU02:FS00:0xa7:************************************ CBang *************************************
11:32:24:WU02:FS00:0xa7:       Date: Nov 27 2019
11:32:24:WU02:FS00:0xa7:       Time: 11:26:54
11:32:24:WU02:FS00:0xa7:   Revision: d25803215b59272441049dfa05a0a9bf7a6e3c48
11:32:24:WU02:FS00:0xa7:     Branch: master
11:32:24:WU02:FS00:0xa7:   Compiler: GNU 8.3.0
11:32:24:WU02:FS00:0xa7:    Options: -std=c++11 -ffunction-sections -fdata-sections -O3 -funroll-loops
11:32:24:WU02:FS00:0xa7:             -fno-pie -fPIC
11:32:24:WU02:FS00:0xa7:   Platform: linux2 4.19.0-5-amd64
11:32:24:WU02:FS00:0xa7:       Bits: 64
11:32:24:WU02:FS00:0xa7:       Mode: Release
11:32:24:WU02:FS00:0xa7:************************************ System ************************************
11:32:24:WU02:FS00:0xa7:        CPU: AMD Phenom(tm) II X6 1055T Processor
11:32:24:WU02:FS00:0xa7:     CPU ID: AuthenticAMD Family 16 Model 10 Stepping 0
11:32:24:WU02:FS00:0xa7:       CPUs: 6
11:32:24:WU02:FS00:0xa7:     Memory: 7.78GiB
11:32:24:WU02:FS00:0xa7:Free Memory: 4.94GiB
11:32:24:WU02:FS00:0xa7:    Threads: POSIX_THREADS
11:32:24:WU02:FS00:0xa7: OS Version: 5.6
11:32:24:WU02:FS00:0xa7:Has Battery: false
11:32:24:WU02:FS00:0xa7: On Battery: false
11:32:24:WU02:FS00:0xa7: UTC Offset: 2
11:32:24:WU02:FS00:0xa7:        PID: 4412
11:32:24:WU02:FS00:0xa7:        CWD: /home/<user>/work
11:32:24:WU02:FS00:0xa7:******************************** Build - libFAH ********************************
11:32:24:WU02:FS00:0xa7:    Version: 0.0.19
11:32:24:WU02:FS00:0xa7:     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
11:32:24:WU02:FS00:0xa7:  Copyright: 2019 foldingathome.org
11:32:24:WU02:FS00:0xa7:   Homepage: https://foldingathome.org/
11:32:24:WU02:FS00:0xa7:       Date: Nov 26 2019
11:32:24:WU02:FS00:0xa7:       Time: 00:41:43
11:32:24:WU02:FS00:0xa7:   Revision: d5b5c747532224f986b7cd02c968ed9a20c16d6e
11:32:24:WU02:FS00:0xa7:     Branch: master
11:32:24:WU02:FS00:0xa7:   Compiler: GNU 8.3.0
11:32:24:WU02:FS00:0xa7:    Options: -std=c++11 -ffunction-sections -fdata-sections -O3 -funroll-loops
11:32:24:WU02:FS00:0xa7:             -fno-pie
11:32:24:WU02:FS00:0xa7:   Platform: linux2 4.19.0-5-amd64
11:32:24:WU02:FS00:0xa7:       Bits: 64
11:32:24:WU02:FS00:0xa7:       Mode: Release
11:32:24:WU02:FS00:0xa7:************************************ Build *************************************
11:32:24:WU02:FS00:0xa7:       SIMD: sse2
11:32:24:WU02:FS00:0xa7:********************************************************************************
11:32:24:WU02:FS00:0xa7:Project: 13830 (Run 92, Clone 5, Gen 115)
11:32:24:WU02:FS00:0xa7:Unit: 0x0000008e80fccb095e73a4fc0b49acc3
11:32:24:WU02:FS00:0xa7:Digital signatures verified
11:32:24:WU02:FS00:0xa7:Reducing thread count from 5 to 4 to avoid domain decomposition by a prime number > 3
11:32:24:WU02:FS00:0xa7:Calling: mdrun -s frame115.tpr -o frame115.trr -x frame115.xtc -cpi state.cpt -cpt 15 -nt 4
11:32:24:WU02:FS00:0xa7:Steps: first=14375000 total=125000
11:32:24:WU01:FS01:0x22:  Checkpoint write interval: 50000 steps (5%) [20 total]
11:32:24:WU01:FS01:0x22:  JSON viewer frame write interval: 10000 steps (1%) [100 total]
11:32:24:WU01:FS01:0x22:  XTC frame write interval: 250000 steps (25%) [4 total]
11:32:24:WU01:FS01:0x22:  Global context and integrator variables write interval: 25000 steps (2.5%) [40 total]
11:32:27:WU02:FS00:0xa7:Completed 5332 out of 125000 steps (4%)
11:33:15:WU01:FS01:0x22:ERROR:NaNs detected in forces. 4656 0
11:33:15:WU01:FS01:0x22:Saving result file ../logfile_01.txt
11:33:15:WU01:FS01:0x22:Saving result file science.log
11:33:15:WU01:FS01:0x22:Saving result file state.xml.bz2
11:33:17:WU01:FS01:0x22:Folding@home Core Shutdown: BAD_WORK_UNIT
11:33:17:WARNING:WU01:FS01:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
11:33:17:WU01:FS01:Sending unit results: id:01 state:SEND error:FAULTY project:13424 run:1439 clone:24 gen:3 core:0x22 unit:0x0000000612bc7d9a5f4ef9711310f0bd
11:33:17:WU01:FS01:Uploading 5.61MiB to 18.188.125.154
11:33:17:WU01:FS01:Connecting to 18.188.125.154:8080
11:33:18:Removing old file 'configs/config-20200827-220305.xml'
11:33:18:Saving configuration to config.xml
11:33:18:<config>
11:33:18:  <!-- Network -->
11:33:18:  <proxy v=':8080'/>
11:33:18:
11:33:18:  <!-- Slot Control -->
11:33:18:  <power v='full'/>
11:33:18:
11:33:18:  <!-- User Information -->
11:33:18:  <passkey v='*****'/>
11:33:18:  <team v='******'/>
11:33:18:  <user v='******'/>
11:33:18:
11:33:18:  <!-- Folding Slots -->
11:33:18:  <slot id='0' type='CPU'/>
11:33:18:  <slot id='1' type='GPU'/>
11:33:18:</config>
11:33:18:WU00:FS01:Connecting to assign1.foldingathome.org:80
11:33:18:WU00:FS01:Assigned to work server 18.188.125.154
11:33:18:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:Pitcairn [Radeon HD 7800] from 18.188.125.154
11:33:18:WU00:FS01:Connecting to 18.188.125.154:8080
11:33:20:WU00:FS01:Downloading 6.19MiB
11:33:23:WU01:FS01:Upload 45.64%
11:33:23:WU00:FS01:Download complete
11:33:23:WU00:FS01:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:13424 run:2059 clone:5 gen:4 core:0x22 unit:0x0000000612bc7d9a5f4ef9869b1552bb
11:33:23:WU00:FS01:Starting
11:33:23:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /home/<user>/cores/cores.foldingathome.org/lin/64bit/22-0.0.11/Core_22.fah/FahCore_22 -dir 00 -suffix 01 -version 706 -lifeline 4307 -checkpoint 15 -gpu-vendor amd -opencl-platform 0 -opencl-device 0 -gpu 0
11:33:23:WU00:FS01:Started FahCore on PID 6256
11:33:23:WU00:FS01:Core PID:6260
11:33:23:WU00:FS01:FahCore 0x22 started
11:33:23:WU00:FS01:0x22:*********************** Log Started 2020-09-03T11:33:23Z ***********************
11:33:23:WU00:FS01:0x22:*************************** Core22 Folding@home Core ***************************
11:33:23:WU00:FS01:0x22:       Core: Core22
11:33:23:WU00:FS01:0x22:       Type: 0x22
11:33:23:WU00:FS01:0x22:    Version: 0.0.11
11:33:23:WU00:FS01:0x22:     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
11:33:23:WU00:FS01:0x22:  Copyright: 2020 foldingathome.org
11:33:23:WU00:FS01:0x22:   Homepage: https://foldingathome.org/
11:33:23:WU00:FS01:0x22:       Date: Jun 27 2020
11:33:23:WU00:FS01:0x22:       Time: 22:50:00
11:33:23:WU00:FS01:0x22:   Revision: cfc2940c5dd1aa80f60daa6e28d4a2a417f74edb
11:33:23:WU00:FS01:0x22:     Branch: core22-0.0.11
11:33:23:WU00:FS01:0x22:   Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
11:33:23:WU00:FS01:0x22:    Options: -std=c++11 -fsigned-char -ffunction-sections -fdata-sections -O3
11:33:23:WU00:FS01:0x22:             -funroll-loops
11:33:23:WU00:FS01:0x22:   Platform: linux2 4.19.76-linuxkit
11:33:23:WU00:FS01:0x22:       Bits: 64
11:33:23:WU00:FS01:0x22:       Mode: Release
11:33:23:WU00:FS01:0x22:Maintainers: John Chodera <john.chodera@choderalab.org> and Peter Eastman
11:33:23:WU00:FS01:0x22:             <peastman@stanford.edu>
11:33:23:WU00:FS01:0x22:       Args: -dir 00 -suffix 01 -version 706 -lifeline 6256 -checkpoint 15
11:33:23:WU00:FS01:0x22:             -gpu-vendor amd -opencl-platform 0 -opencl-device 0 -gpu 0
11:33:23:WU00:FS01:0x22:************************************ libFAH ************************************
11:33:23:WU00:FS01:0x22:       Date: Jun 27 2020
11:33:23:WU00:FS01:0x22:       Time: 22:11:04
11:33:23:WU00:FS01:0x22:   Revision: 2b383f4f04f38511dff592885d7c0400e72bdf43
11:33:23:WU00:FS01:0x22:     Branch: HEAD
11:33:23:WU00:FS01:0x22:   Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
11:33:23:WU00:FS01:0x22:    Options: -std=c++11 -fsigned-char -ffunction-sections -fdata-sections -O3
11:33:23:WU00:FS01:0x22:             -funroll-loops
11:33:23:WU00:FS01:0x22:   Platform: linux2 4.19.76-linuxkit
11:33:23:WU00:FS01:0x22:       Bits: 64
11:33:23:WU00:FS01:0x22:       Mode: Release
11:33:23:WU00:FS01:0x22:************************************ CBang *************************************
11:33:23:WU00:FS01:0x22:       Date: Jun 27 2020
11:33:23:WU00:FS01:0x22:       Time: 22:10:11
11:33:23:WU00:FS01:0x22:   Revision: f8529962055b0e7bde23e429f5072ff758089dee
11:33:23:WU00:FS01:0x22:     Branch: HEAD
11:33:23:WU00:FS01:0x22:   Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
11:33:23:WU00:FS01:0x22:    Options: -std=c++11 -fsigned-char -ffunction-sections -fdata-sections -O3
11:33:23:WU00:FS01:0x22:             -funroll-loops -fPIC
11:33:23:WU00:FS01:0x22:   Platform: linux2 4.19.76-linuxkit
11:33:23:WU00:FS01:0x22:       Bits: 64
11:33:23:WU00:FS01:0x22:       Mode: Release
11:33:23:WU00:FS01:0x22:************************************ System ************************************
11:33:23:WU00:FS01:0x22:        CPU: AMD Phenom(tm) II X6 1055T Processor
11:33:23:WU00:FS01:0x22:     CPU ID: AuthenticAMD Family 16 Model 10 Stepping 0
11:33:23:WU00:FS01:0x22:       CPUs: 6
11:33:23:WU00:FS01:0x22:     Memory: 7.78GiB
11:33:23:WU00:FS01:0x22:Free Memory: 4.45GiB
11:33:23:WU00:FS01:0x22:    Threads: POSIX_THREADS
11:33:23:WU00:FS01:0x22: OS Version: 5.6
11:33:23:WU00:FS01:0x22:Has Battery: false
11:33:23:WU00:FS01:0x22: On Battery: false
11:33:23:WU00:FS01:0x22: UTC Offset: 2
11:33:23:WU00:FS01:0x22:        PID: 6260
11:33:23:WU00:FS01:0x22:        CWD: /home/<user>/work
11:33:23:WU00:FS01:0x22:********************************************************************************
11:33:23:WU00:FS01:0x22:Project: 13424 (Run 2059, Clone 5, Gen 4)
11:33:23:WU00:FS01:0x22:Unit: 0x0000000612bc7d9a5f4ef9869b1552bb
11:33:23:WU00:FS01:0x22:Reading tar file core.xml
11:33:23:WU00:FS01:0x22:Reading tar file integrator.xml
11:33:23:WU00:FS01:0x22:Reading tar file state.xml.bz2
11:33:24:WU00:FS01:0x22:Reading tar file system.xml.bz2
11:33:24:WU00:FS01:0x22:Digital signatures verified
11:33:24:WU00:FS01:0x22:Folding@home GPU Core22 Folding@home Core
11:33:24:WU00:FS01:0x22:Version 0.0.11
11:33:24:WU00:FS01:0x22:  Checkpoint write interval: 50000 steps (5%) [20 total]
11:33:24:WU00:FS01:0x22:  JSON viewer frame write interval: 10000 steps (1%) [100 total]
11:33:24:WU00:FS01:0x22:  XTC frame write interval: 250000 steps (25%) [4 total]
11:33:24:WU00:FS01:0x22:  Global context and integrator variables write interval: 25000 steps (2.5%) [40 total]
11:33:29:WU01:FS01:Upload 95.73%
11:33:31:WU01:FS01:Upload complete
11:33:31:WU01:FS01:Server responded WORK_ACK (400)
11:33:31:WU01:FS01:Cleaning up
11:34:13:WU00:FS01:0x22:ERROR:Discrepancy: Forces are blowing up! 4656 0
11:34:13:WU00:FS01:0x22:Saving result file ../logfile_01.txt
11:34:13:WU00:FS01:0x22:Saving result file science.log
11:34:13:WU00:FS01:0x22:Saving result file state.xml.bz2
11:34:15:WU00:FS01:0x22:Folding@home Core Shutdown: BAD_WORK_UNIT
11:34:15:WARNING:WU00:FS01:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
11:34:15:WU00:FS01:Sending unit results: id:00 state:SEND error:FAULTY project:13424 run:2059 clone:5 gen:4 core:0x22 unit:0x0000000612bc7d9a5f4ef9869b1552bb
11:34:15:WU00:FS01:Uploading 5.61MiB to 18.188.125.154
11:34:15:WU00:FS01:Connecting to 18.188.125.154:8080
11:34:16:WU01:FS01:Connecting to assign1.foldingathome.org:80
11:34:16:WU01:FS01:Assigned to work server 18.188.125.154
11:34:16:WU01:FS01:Requesting new work unit for slot 01: READY gpu:0:Pitcairn [Radeon HD 7800] from 18.188.125.154
11:34:16:WU01:FS01:Connecting to 18.188.125.154:8080
11:34:18:WU01:FS01:Downloading 6.19MiB
11:34:21:WU00:FS01:Upload 45.64%
11:34:21:WU01:FS01:Download complete
11:34:21:WU01:FS01:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:13424 run:2333 clone:8 gen:0 core:0x22 unit:0x0000000112bc7d9a5f4ef9899c986cb8
11:34:21:WU01:FS01:Starting
11:34:21:WU01:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /home/<user>/cores/cores.foldingathome.org/lin/64bit/22-0.0.11/Core_22.fah/FahCore_22 -dir 01 -suffix 01 -version 706 -lifeline 4307 -checkpoint 15 -gpu-vendor amd -opencl-platform 0 -opencl-device 0 -gpu 0
11:34:21:WU01:FS01:Started FahCore on PID 8226
11:34:21:WU01:FS01:Core PID:8230
11:34:21:WU01:FS01:FahCore 0x22 started
11:34:22:WU01:FS01:0x22:*********************** Log Started 2020-09-03T11:34:21Z ***********************
11:34:22:WU01:FS01:0x22:*************************** Core22 Folding@home Core ***************************
11:34:22:WU01:FS01:0x22:       Core: Core22
11:34:22:WU01:FS01:0x22:       Type: 0x22
11:34:22:WU01:FS01:0x22:    Version: 0.0.11
11:34:22:WU01:FS01:0x22:     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
11:34:22:WU01:FS01:0x22:  Copyright: 2020 foldingathome.org
11:34:22:WU01:FS01:0x22:   Homepage: https://foldingathome.org/
11:34:22:WU01:FS01:0x22:       Date: Jun 27 2020
11:34:22:WU01:FS01:0x22:       Time: 22:50:00
11:34:22:WU01:FS01:0x22:   Revision: cfc2940c5dd1aa80f60daa6e28d4a2a417f74edb
11:34:22:WU01:FS01:0x22:     Branch: core22-0.0.11
11:34:22:WU01:FS01:0x22:   Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
11:34:22:WU01:FS01:0x22:    Options: -std=c++11 -fsigned-char -ffunction-sections -fdata-sections -O3
11:34:22:WU01:FS01:0x22:             -funroll-loops
11:34:22:WU01:FS01:0x22:   Platform: linux2 4.19.76-linuxkit
11:34:22:WU01:FS01:0x22:       Bits: 64
11:34:22:WU01:FS01:0x22:       Mode: Release
11:34:22:WU01:FS01:0x22:Maintainers: John Chodera <john.chodera@choderalab.org> and Peter Eastman
11:34:22:WU01:FS01:0x22:             <peastman@stanford.edu>
11:34:22:WU01:FS01:0x22:       Args: -dir 01 -suffix 01 -version 706 -lifeline 8226 -checkpoint 15
11:34:22:WU01:FS01:0x22:             -gpu-vendor amd -opencl-platform 0 -opencl-device 0 -gpu 0
11:34:22:WU01:FS01:0x22:************************************ libFAH ************************************
11:34:22:WU01:FS01:0x22:       Date: Jun 27 2020
11:34:22:WU01:FS01:0x22:       Time: 22:11:04
11:34:22:WU01:FS01:0x22:   Revision: 2b383f4f04f38511dff592885d7c0400e72bdf43
11:34:22:WU01:FS01:0x22:     Branch: HEAD
11:34:22:WU01:FS01:0x22:   Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
11:34:22:WU01:FS01:0x22:    Options: -std=c++11 -fsigned-char -ffunction-sections -fdata-sections -O3
11:34:22:WU01:FS01:0x22:             -funroll-loops
11:34:22:WU01:FS01:0x22:   Platform: linux2 4.19.76-linuxkit
11:34:22:WU01:FS01:0x22:       Bits: 64
11:34:22:WU01:FS01:0x22:       Mode: Release
11:34:22:WU01:FS01:0x22:************************************ CBang *************************************
11:34:22:WU01:FS01:0x22:       Date: Jun 27 2020
11:34:22:WU01:FS01:0x22:       Time: 22:10:11
11:34:22:WU01:FS01:0x22:   Revision: f8529962055b0e7bde23e429f5072ff758089dee
11:34:22:WU01:FS01:0x22:     Branch: HEAD
11:34:22:WU01:FS01:0x22:   Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
11:34:22:WU01:FS01:0x22:    Options: -std=c++11 -fsigned-char -ffunction-sections -fdata-sections -O3
11:34:22:WU01:FS01:0x22:             -funroll-loops -fPIC
11:34:22:WU01:FS01:0x22:   Platform: linux2 4.19.76-linuxkit
11:34:22:WU01:FS01:0x22:       Bits: 64
11:34:22:WU01:FS01:0x22:       Mode: Release
11:34:22:WU01:FS01:0x22:************************************ System ************************************
11:34:22:WU01:FS01:0x22:        CPU: AMD Phenom(tm) II X6 1055T Processor
11:34:22:WU01:FS01:0x22:     CPU ID: AuthenticAMD Family 16 Model 10 Stepping 0
11:34:22:WU01:FS01:0x22:       CPUs: 6
11:34:22:WU01:FS01:0x22:     Memory: 7.78GiB
11:34:22:WU01:FS01:0x22:Free Memory: 4.55GiB
11:34:22:WU01:FS01:0x22:    Threads: POSIX_THREADS
11:34:22:WU01:FS01:0x22: OS Version: 5.6
11:34:22:WU01:FS01:0x22:Has Battery: false
11:34:22:WU01:FS01:0x22: On Battery: false
11:34:22:WU01:FS01:0x22: UTC Offset: 2
11:34:22:WU01:FS01:0x22:        PID: 8230
11:34:22:WU01:FS01:0x22:        CWD: /home/<user>/work
11:34:22:WU01:FS01:0x22:********************************************************************************
11:34:22:WU01:FS01:0x22:Project: 13424 (Run 2333, Clone 8, Gen 0)
11:34:22:WU01:FS01:0x22:Unit: 0x0000000112bc7d9a5f4ef9899c986cb8
11:34:22:WU01:FS01:0x22:Reading tar file core.xml
11:34:22:WU01:FS01:0x22:Reading tar file integrator.xml
11:34:22:WU01:FS01:0x22:Reading tar file state.xml.bz2
11:34:22:WU01:FS01:0x22:Reading tar file system.xml.bz2
11:34:22:WU01:FS01:0x22:Digital signatures verified
11:34:22:WU01:FS01:0x22:Folding@home GPU Core22 Folding@home Core
11:34:22:WU01:FS01:0x22:Version 0.0.11
11:34:23:WU01:FS01:0x22:  Checkpoint write interval: 50000 steps (5%) [20 total]
11:34:23:WU01:FS01:0x22:  JSON viewer frame write interval: 10000 steps (1%) [100 total]
11:34:23:WU01:FS01:0x22:  XTC frame write interval: 250000 steps (25%) [4 total]
11:34:23:WU01:FS01:0x22:  Global context and integrator variables write interval: 25000 steps (2.5%) [40 total]
11:34:27:WU00:FS01:Upload 91.29%
11:34:29:WU00:FS01:Upload complete
11:34:29:WU00:FS01:Server responded WORK_ACK (400)
11:34:29:WU00:FS01:Cleaning up
11:35:34:WU01:FS01:0x22:ERROR:Force RMSE error of 5.11078 with threshold of 5
11:35:34:WU01:FS01:0x22:Saving result file ../logfile_01.txt
11:35:34:WU01:FS01:0x22:Saving result file science.log
11:35:34:WU01:FS01:0x22:Saving result file state.xml.bz2
11:35:35:WU01:FS01:0x22:Folding@home Core Shutdown: BAD_WORK_UNIT
11:35:36:WARNING:WU01:FS01:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
11:35:36:WU01:FS01:Sending unit results: id:01 state:SEND error:FAULTY project:13424 run:2333 clone:8 gen:0 core:0x22 unit:0x0000000112bc7d9a5f4ef9899c986cb8
11:35:36:WU01:FS01:Uploading 5.60MiB to 18.188.125.154
11:35:36:WU01:FS01:Connecting to 18.188.125.154:8080
11:35:36:WU00:FS01:Connecting to assign1.foldingathome.org:80
11:35:37:WU00:FS01:Assigned to work server 18.188.125.154
11:35:37:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:Pitcairn [Radeon HD 7800] from 18.188.125.154
11:35:37:WU00:FS01:Connecting to 18.188.125.154:8080
11:35:38:WU00:FS01:Downloading 6.20MiB
11:35:41:WU00:FS01:Download complete
11:35:41:WU00:FS01:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:13424 run:2247 clone:25 gen:1 core:0x22 unit:0x0000000112bc7d9a5f4ef988abf63ab6
11:35:41:WU00:FS01:Starting
11:35:41:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /home/<user>/cores/cores.foldingathome.org/lin/64bit/22-0.0.11/Core_22.fah/FahCore_22 -dir 00 -suffix 01 -version 706 -lifeline 4307 -checkpoint 15 -gpu-vendor amd -opencl-platform 0 -opencl-device 0 -gpu 0
11:35:41:WU00:FS01:Started FahCore on PID 10876
11:35:41:WU00:FS01:Core PID:10880
11:35:41:WU00:FS01:FahCore 0x22 started
11:35:42:WU01:FS01:Upload 50.19%
11:35:42:WU00:FS01:0x22:*********************** Log Started 2020-09-03T11:35:41Z ***********************
11:35:42:WU00:FS01:0x22:*************************** Core22 Folding@home Core ***************************
11:35:42:WU00:FS01:0x22:       Core: Core22
11:35:42:WU00:FS01:0x22:       Type: 0x22
11:35:42:WU00:FS01:0x22:    Version: 0.0.11
11:35:42:WU00:FS01:0x22:     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
11:35:42:WU00:FS01:0x22:  Copyright: 2020 foldingathome.org
11:35:42:WU00:FS01:0x22:   Homepage: https://foldingathome.org/
11:35:42:WU00:FS01:0x22:       Date: Jun 27 2020
11:35:42:WU00:FS01:0x22:       Time: 22:50:00
11:35:42:WU00:FS01:0x22:   Revision: cfc2940c5dd1aa80f60daa6e28d4a2a417f74edb
11:35:42:WU00:FS01:0x22:     Branch: core22-0.0.11
11:35:42:WU00:FS01:0x22:   Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
11:35:42:WU00:FS01:0x22:    Options: -std=c++11 -fsigned-char -ffunction-sections -fdata-sections -O3
11:35:42:WU00:FS01:0x22:             -funroll-loops
11:35:42:WU00:FS01:0x22:   Platform: linux2 4.19.76-linuxkit
11:35:42:WU00:FS01:0x22:       Bits: 64
11:35:42:WU00:FS01:0x22:       Mode: Release
11:35:42:WU00:FS01:0x22:Maintainers: John Chodera <john.chodera@choderalab.org> and Peter Eastman
11:35:42:WU00:FS01:0x22:             <peastman@stanford.edu>
11:35:42:WU00:FS01:0x22:       Args: -dir 00 -suffix 01 -version 706 -lifeline 10876 -checkpoint 15
11:35:42:WU00:FS01:0x22:             -gpu-vendor amd -opencl-platform 0 -opencl-device 0 -gpu 0
11:35:42:WU00:FS01:0x22:************************************ libFAH ************************************
11:35:42:WU00:FS01:0x22:       Date: Jun 27 2020
11:35:42:WU00:FS01:0x22:       Time: 22:11:04
11:35:42:WU00:FS01:0x22:   Revision: 2b383f4f04f38511dff592885d7c0400e72bdf43
11:35:42:WU00:FS01:0x22:     Branch: HEAD
11:35:42:WU00:FS01:0x22:   Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
11:35:42:WU00:FS01:0x22:    Options: -std=c++11 -fsigned-char -ffunction-sections -fdata-sections -O3
11:35:42:WU00:FS01:0x22:             -funroll-loops
11:35:42:WU00:FS01:0x22:   Platform: linux2 4.19.76-linuxkit
11:35:42:WU00:FS01:0x22:       Bits: 64
11:35:42:WU00:FS01:0x22:       Mode: Release
11:35:42:WU00:FS01:0x22:************************************ CBang *************************************
11:35:42:WU00:FS01:0x22:       Date: Jun 27 2020
11:35:42:WU00:FS01:0x22:       Time: 22:10:11
11:35:42:WU00:FS01:0x22:   Revision: f8529962055b0e7bde23e429f5072ff758089dee
11:35:42:WU00:FS01:0x22:     Branch: HEAD
11:35:42:WU00:FS01:0x22:   Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
11:35:42:WU00:FS01:0x22:    Options: -std=c++11 -fsigned-char -ffunction-sections -fdata-sections -O3
11:35:42:WU00:FS01:0x22:             -funroll-loops -fPIC
11:35:42:WU00:FS01:0x22:   Platform: linux2 4.19.76-linuxkit
11:35:42:WU00:FS01:0x22:       Bits: 64
11:35:42:WU00:FS01:0x22:       Mode: Release
11:35:42:WU00:FS01:0x22:************************************ System ************************************
11:35:42:WU00:FS01:0x22:        CPU: AMD Phenom(tm) II X6 1055T Processor
11:35:42:WU00:FS01:0x22:     CPU ID: AuthenticAMD Family 16 Model 10 Stepping 0
11:35:42:WU00:FS01:0x22:       CPUs: 6
11:35:42:WU00:FS01:0x22:     Memory: 7.78GiB
11:35:42:WU00:FS01:0x22:Free Memory: 4.31GiB
11:35:42:WU00:FS01:0x22:    Threads: POSIX_THREADS
11:35:42:WU00:FS01:0x22: OS Version: 5.6
11:35:42:WU00:FS01:0x22:Has Battery: false
11:35:42:WU00:FS01:0x22: On Battery: false
11:35:42:WU00:FS01:0x22: UTC Offset: 2
11:35:42:WU00:FS01:0x22:        PID: 10880
11:35:42:WU00:FS01:0x22:        CWD: /home/<user>/work
11:35:42:WU00:FS01:0x22:********************************************************************************
11:35:42:WU00:FS01:0x22:Project: 13424 (Run 2247, Clone 25, Gen 1)
11:35:42:WU00:FS01:0x22:Unit: 0x0000000112bc7d9a5f4ef988abf63ab6
11:35:42:WU00:FS01:0x22:Reading tar file core.xml
11:35:42:WU00:FS01:0x22:Reading tar file integrator.xml
11:35:42:WU00:FS01:0x22:Reading tar file state.xml.bz2
11:35:42:WU00:FS01:0x22:Reading tar file system.xml.bz2
11:35:42:WU00:FS01:0x22:Digital signatures verified
11:35:42:WU00:FS01:0x22:Folding@home GPU Core22 Folding@home Core
11:35:42:WU00:FS01:0x22:Version 0.0.11
11:35:43:WU00:FS01:0x22:  Checkpoint write interval: 50000 steps (5%) [20 total]
11:35:43:WU00:FS01:0x22:  JSON viewer frame write interval: 10000 steps (1%) [100 total]
11:35:43:WU00:FS01:0x22:  XTC frame write interval: 250000 steps (25%) [4 total]
11:35:43:WU00:FS01:0x22:  Global context and integrator variables write interval: 25000 steps (2.5%) [40 total]
11:35:48:WU01:FS01:Upload 99.27%
11:35:49:WU01:FS01:Upload complete
11:35:49:WU01:FS01:Server responded WORK_ACK (400)
11:35:49:WU01:FS01:Cleaning up
11:36:11:WU02:FS00:0xa7:Completed 6250 out of 125000 steps (5%)
11:37:04:WU00:FS01:0x22:Completed 0 out of 1000000 steps (0%)
11:41:14:WU02:FS00:0xa7:Completed 7500 out of 125000 steps (6%)
11:43:51:WU00:FS01:0x22:An exception occurred at step 5521: Particle coordinate is nan
11:43:51:WU00:FS01:0x22:ERROR:98: Attempting to restart from last good checkpoint by restarting core.
11:43:51:WU00:FS01:0x22:Folding@home Core Shutdown: CORE_RESTART
11:43:51:WARNING:WU00:FS01:FahCore returned: CORE_RESTART (98 = 0x62)
11:43:51:WU00:FS01:Starting
11:43:51:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /home/<user>/cores/cores.foldingathome.org/lin/64bit/22-0.0.11/Core_22.fah/FahCore_22 -dir 00 -suffix 01 -version 706 -lifeline 4307 -checkpoint 15 -gpu-vendor amd -opencl-platform 0 -opencl-device 0 -gpu 0
11:43:51:WU00:FS01:Started FahCore on PID 22056
11:43:51:WU00:FS01:Core PID:22060
11:43:51:WU00:FS01:FahCore 0x22 started
11:43:52:WU00:FS01:0x22:*********************** Log Started 2020-09-03T11:43:51Z ***********************
11:43:52:WU00:FS01:0x22:*************************** Core22 Folding@home Core ***************************
11:43:52:WU00:FS01:0x22:       Core: Core22
11:43:52:WU00:FS01:0x22:       Type: 0x22
11:43:52:WU00:FS01:0x22:    Version: 0.0.11
11:43:52:WU00:FS01:0x22:     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
11:43:52:WU00:FS01:0x22:  Copyright: 2020 foldingathome.org
11:43:52:WU00:FS01:0x22:   Homepage: https://foldingathome.org/
11:43:52:WU00:FS01:0x22:       Date: Jun 27 2020
11:43:52:WU00:FS01:0x22:       Time: 22:50:00
11:43:52:WU00:FS01:0x22:   Revision: cfc2940c5dd1aa80f60daa6e28d4a2a417f74edb
11:43:52:WU00:FS01:0x22:     Branch: core22-0.0.11
11:43:52:WU00:FS01:0x22:   Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
11:43:52:WU00:FS01:0x22:    Options: -std=c++11 -fsigned-char -ffunction-sections -fdata-sections -O3
11:43:52:WU00:FS01:0x22:             -funroll-loops
11:43:52:WU00:FS01:0x22:   Platform: linux2 4.19.76-linuxkit
11:43:52:WU00:FS01:0x22:       Bits: 64
11:43:52:WU00:FS01:0x22:       Mode: Release
11:43:52:WU00:FS01:0x22:Maintainers: John Chodera <john.chodera@choderalab.org> and Peter Eastman
11:43:52:WU00:FS01:0x22:             <peastman@stanford.edu>
11:43:52:WU00:FS01:0x22:       Args: -dir 00 -suffix 01 -version 706 -lifeline 22056 -checkpoint 15
11:43:52:WU00:FS01:0x22:             -gpu-vendor amd -opencl-platform 0 -opencl-device 0 -gpu 0
11:43:52:WU00:FS01:0x22:************************************ libFAH ************************************
11:43:52:WU00:FS01:0x22:       Date: Jun 27 2020
11:43:52:WU00:FS01:0x22:       Time: 22:11:04
11:43:52:WU00:FS01:0x22:   Revision: 2b383f4f04f38511dff592885d7c0400e72bdf43
11:43:52:WU00:FS01:0x22:     Branch: HEAD
11:43:52:WU00:FS01:0x22:   Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
11:43:52:WU00:FS01:0x22:    Options: -std=c++11 -fsigned-char -ffunction-sections -fdata-sections -O3
11:43:52:WU00:FS01:0x22:             -funroll-loops
11:43:52:WU00:FS01:0x22:   Platform: linux2 4.19.76-linuxkit
11:43:52:WU00:FS01:0x22:       Bits: 64
11:43:52:WU00:FS01:0x22:       Mode: Release
11:43:52:WU00:FS01:0x22:************************************ CBang *************************************
11:43:52:WU00:FS01:0x22:       Date: Jun 27 2020
11:43:52:WU00:FS01:0x22:       Time: 22:10:11
11:43:52:WU00:FS01:0x22:   Revision: f8529962055b0e7bde23e429f5072ff758089dee
11:43:52:WU00:FS01:0x22:     Branch: HEAD
11:43:52:WU00:FS01:0x22:   Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
11:43:52:WU00:FS01:0x22:    Options: -std=c++11 -fsigned-char -ffunction-sections -fdata-sections -O3
11:43:52:WU00:FS01:0x22:             -funroll-loops -fPIC
11:43:52:WU00:FS01:0x22:   Platform: linux2 4.19.76-linuxkit
11:43:52:WU00:FS01:0x22:       Bits: 64
11:43:52:WU00:FS01:0x22:       Mode: Release
11:43:52:WU00:FS01:0x22:************************************ System ************************************
11:43:52:WU00:FS01:0x22:        CPU: AMD Phenom(tm) II X6 1055T Processor
11:43:52:WU00:FS01:0x22:     CPU ID: AuthenticAMD Family 16 Model 10 Stepping 0
11:43:52:WU00:FS01:0x22:       CPUs: 6
11:43:52:WU00:FS01:0x22:     Memory: 7.78GiB
11:43:52:WU00:FS01:0x22:Free Memory: 4.21GiB
11:43:52:WU00:FS01:0x22:    Threads: POSIX_THREADS
11:43:52:WU00:FS01:0x22: OS Version: 5.6
11:43:52:WU00:FS01:0x22:Has Battery: false
11:43:52:WU00:FS01:0x22: On Battery: false
11:43:52:WU00:FS01:0x22: UTC Offset: 2
11:43:52:WU00:FS01:0x22:        PID: 22060
11:43:52:WU00:FS01:0x22:        CWD: /home/<user>/work
11:43:52:WU00:FS01:0x22:********************************************************************************
11:43:52:WU00:FS01:0x22:Project: 13424 (Run 2247, Clone 25, Gen 1)
11:43:52:WU00:FS01:0x22:Unit: 0x0000000112bc7d9a5f4ef988abf63ab6
11:43:52:WU00:FS01:0x22:Digital signatures verified
11:43:52:WU00:FS01:0x22:Folding@home GPU Core22 Folding@home Core
11:43:52:WU00:FS01:0x22:Version 0.0.11
11:43:52:WU00:FS01:0x22:  Checkpoint write interval: 50000 steps (5%) [20 total]
11:43:52:WU00:FS01:0x22:  JSON viewer frame write interval: 10000 steps (1%) [100 total]
11:43:52:WU00:FS01:0x22:  XTC frame write interval: 250000 steps (25%) [4 total]
11:43:52:WU00:FS01:0x22:  Global context and integrator variables write interval: 25000 steps (2.5%) [40 total]
11:44:46:WU00:FS01:0x22:ERROR:Discrepancy: Forces are blowing up! 0 0
11:44:46:WU00:FS01:0x22:Saving result file ../logfile_01.txt
11:44:46:WU00:FS01:0x22:Saving result file science.log
11:44:46:WU00:FS01:0x22:Saving result file state.xml.bz2
11:44:47:WU00:FS01:0x22:Folding@home Core Shutdown: BAD_WORK_UNIT
11:44:47:WARNING:WU00:FS01:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
11:44:47:WU00:FS01:Sending unit results: id:00 state:SEND error:FAULTY project:13424 run:2247 clone:25 gen:1 core:0x22 unit:0x0000000112bc7d9a5f4ef988abf63ab6
11:44:47:WU00:FS01:Uploading 5.62MiB to 18.188.125.154
11:44:47:WU00:FS01:Connecting to 18.188.125.154:8080
11:44:48:WU01:FS01:Connecting to assign1.foldingathome.org:80
11:44:48:WU01:FS01:Assigned to work server 18.188.125.154
11:44:48:WU01:FS01:Requesting new work unit for slot 01: READY gpu:0:Pitcairn [Radeon HD 7800] from 18.188.125.154
11:44:48:WU01:FS01:Connecting to 18.188.125.154:8080
11:44:50:WU01:FS01:Downloading 6.19MiB
11:44:53:WU00:FS01:Upload 45.61%
11:44:53:WU01:FS01:Download complete
11:44:53:WU01:FS01:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:13424 run:1934 clone:26 gen:4 core:0x22 unit:0x0000000512bc7d9a5f4ef9814859623a
11:44:53:WU01:FS01:Starting
11:44:53:WU01:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /home/<user>/cores/cores.foldingathome.org/lin/64bit/22-0.0.11/Core_22.fah/FahCore_22 -dir 01 -suffix 01 -version 706 -lifeline 4307 -checkpoint 15 -gpu-vendor amd -opencl-platform 0 -opencl-device 0 -gpu 0
11:44:53:WU01:FS01:Started FahCore on PID 23526
11:44:53:WU01:FS01:Core PID:23530
11:44:53:WU01:FS01:FahCore 0x22 started
11:44:54:WU01:FS01:0x22:*********************** Log Started 2020-09-03T11:44:53Z ***********************
11:44:54:WU01:FS01:0x22:*************************** Core22 Folding@home Core ***************************
11:44:54:WU01:FS01:0x22:       Core: Core22
11:44:54:WU01:FS01:0x22:       Type: 0x22
11:44:54:WU01:FS01:0x22:    Version: 0.0.11
11:44:54:WU01:FS01:0x22:     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
11:44:54:WU01:FS01:0x22:  Copyright: 2020 foldingathome.org
11:44:54:WU01:FS01:0x22:   Homepage: https://foldingathome.org/
11:44:54:WU01:FS01:0x22:       Date: Jun 27 2020
11:44:54:WU01:FS01:0x22:       Time: 22:50:00
11:44:54:WU01:FS01:0x22:   Revision: cfc2940c5dd1aa80f60daa6e28d4a2a417f74edb
11:44:54:WU01:FS01:0x22:     Branch: core22-0.0.11
11:44:54:WU01:FS01:0x22:   Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
11:44:54:WU01:FS01:0x22:    Options: -std=c++11 -fsigned-char -ffunction-sections -fdata-sections -O3
11:44:54:WU01:FS01:0x22:             -funroll-loops
11:44:54:WU01:FS01:0x22:   Platform: linux2 4.19.76-linuxkit
11:44:54:WU01:FS01:0x22:       Bits: 64
11:44:54:WU01:FS01:0x22:       Mode: Release
11:44:54:WU01:FS01:0x22:Maintainers: John Chodera <john.chodera@choderalab.org> and Peter Eastman
11:44:54:WU01:FS01:0x22:             <peastman@stanford.edu>
11:44:54:WU01:FS01:0x22:       Args: -dir 01 -suffix 01 -version 706 -lifeline 23526 -checkpoint 15
11:44:54:WU01:FS01:0x22:             -gpu-vendor amd -opencl-platform 0 -opencl-device 0 -gpu 0
11:44:54:WU01:FS01:0x22:************************************ libFAH ************************************
11:44:54:WU01:FS01:0x22:       Date: Jun 27 2020
11:44:54:WU01:FS01:0x22:       Time: 22:11:04
11:44:54:WU01:FS01:0x22:   Revision: 2b383f4f04f38511dff592885d7c0400e72bdf43
11:44:54:WU01:FS01:0x22:     Branch: HEAD
11:44:54:WU01:FS01:0x22:   Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
11:44:54:WU01:FS01:0x22:    Options: -std=c++11 -fsigned-char -ffunction-sections -fdata-sections -O3
11:44:54:WU01:FS01:0x22:             -funroll-loops
11:44:54:WU01:FS01:0x22:   Platform: linux2 4.19.76-linuxkit
11:44:54:WU01:FS01:0x22:       Bits: 64
11:44:54:WU01:FS01:0x22:       Mode: Release
11:44:54:WU01:FS01:0x22:************************************ CBang *************************************
11:44:54:WU01:FS01:0x22:       Date: Jun 27 2020
11:44:54:WU01:FS01:0x22:       Time: 22:10:11
11:44:54:WU01:FS01:0x22:   Revision: f8529962055b0e7bde23e429f5072ff758089dee
11:44:54:WU01:FS01:0x22:     Branch: HEAD
11:44:54:WU01:FS01:0x22:   Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
11:44:54:WU01:FS01:0x22:    Options: -std=c++11 -fsigned-char -ffunction-sections -fdata-sections -O3
11:44:54:WU01:FS01:0x22:             -funroll-loops -fPIC
11:44:54:WU01:FS01:0x22:   Platform: linux2 4.19.76-linuxkit
11:44:54:WU01:FS01:0x22:       Bits: 64
11:44:54:WU01:FS01:0x22:       Mode: Release
11:44:54:WU01:FS01:0x22:************************************ System ************************************
11:44:54:WU01:FS01:0x22:        CPU: AMD Phenom(tm) II X6 1055T Processor
11:44:54:WU01:FS01:0x22:     CPU ID: AuthenticAMD Family 16 Model 10 Stepping 0
11:44:54:WU01:FS01:0x22:       CPUs: 6
11:44:54:WU01:FS01:0x22:     Memory: 7.78GiB
11:44:54:WU01:FS01:0x22:Free Memory: 3.74GiB
11:44:54:WU01:FS01:0x22:    Threads: POSIX_THREADS
11:44:54:WU01:FS01:0x22: OS Version: 5.6
11:44:54:WU01:FS01:0x22:Has Battery: false
11:44:54:WU01:FS01:0x22: On Battery: false
11:44:54:WU01:FS01:0x22: UTC Offset: 2
11:44:54:WU01:FS01:0x22:        PID: 23530
11:44:54:WU01:FS01:0x22:        CWD: /home/<user>/work
11:44:54:WU01:FS01:0x22:********************************************************************************
11:44:54:WU01:FS01:0x22:Project: 13424 (Run 1934, Clone 26, Gen 4)
11:44:54:WU01:FS01:0x22:Unit: 0x0000000512bc7d9a5f4ef9814859623a
11:44:54:WU01:FS01:0x22:Reading tar file core.xml
11:44:54:WU01:FS01:0x22:Reading tar file integrator.xml
11:44:54:WU01:FS01:0x22:Reading tar file state.xml.bz2
11:44:54:WU01:FS01:0x22:Reading tar file system.xml.bz2
11:44:54:WU01:FS01:0x22:Digital signatures verified
11:44:54:WU01:FS01:0x22:Folding@home GPU Core22 Folding@home Core
11:44:54:WU01:FS01:0x22:Version 0.0.11
11:44:55:WU01:FS01:0x22:  Checkpoint write interval: 50000 steps (5%) [20 total]
11:44:55:WU01:FS01:0x22:  JSON viewer frame write interval: 10000 steps (1%) [100 total]
11:44:55:WU01:FS01:0x22:  XTC frame write interval: 250000 steps (25%) [4 total]
11:44:55:WU01:FS01:0x22:  Global context and integrator variables write interval: 25000 steps (2.5%) [40 total]
11:44:59:WU00:FS01:Upload 95.68%
11:45:01:WU00:FS01:Upload complete
11:45:01:WU00:FS01:Server responded WORK_ACK (400)
11:45:01:WU00:FS01:Cleaning up
11:45:44:WU01:FS01:0x22:ERROR:NaNs detected in forces. 4656 0
11:45:44:WU01:FS01:0x22:Saving result file ../logfile_01.txt
11:45:44:WU01:FS01:0x22:Saving result file science.log
11:45:44:WU01:FS01:0x22:Saving result file state.xml.bz2
11:45:45:WU01:FS01:0x22:Folding@home Core Shutdown: BAD_WORK_UNIT
11:45:46:WARNING:WU01:FS01:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
11:45:46:WU01:FS01:Sending unit results: id:01 state:SEND error:FAULTY project:13424 run:1934 clone:26 gen:4 core:0x22 unit:0x0000000512bc7d9a5f4ef9814859623a
11:45:46:WU01:FS01:Uploading 5.61MiB to 18.188.125.154
11:45:46:WU01:FS01:Connecting to 18.188.125.154:8080
11:45:46:WU00:FS01:Connecting to assign1.foldingathome.org:80
11:45:47:WU00:FS01:Assigned to work server 18.188.125.154
11:45:47:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:Pitcairn [Radeon HD 7800] from 18.188.125.154
11:45:47:WU00:FS01:Connecting to 18.188.125.154:8080
11:45:49:WU00:FS01:Downloading 6.21MiB
11:45:52:WU01:FS01:Upload 46.76%
11:45:52:WU00:FS01:Download complete
11:45:52:WU00:FS01:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:13424 run:2226 clone:42 gen:2 core:0x22 unit:0x0000000412bc7d9a5f4ef98bb24f4b11
11:45:52:WU00:FS01:Starting
11:45:52:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /home/<user>/cores/cores.foldingathome.org/lin/64bit/22-0.0.11/Core_22.fah/FahCore_22 -dir 00 -suffix 01 -version 706 -lifeline 4307 -checkpoint 15 -gpu-vendor amd -opencl-platform 0 -opencl-device 0 -gpu 0
11:45:52:WU00:FS01:Started FahCore on PID 24769
11:45:52:WU00:FS01:Core PID:24773
11:45:52:WU00:FS01:FahCore 0x22 started
11:45:52:WU00:FS01:0x22:*********************** Log Started 2020-09-03T11:45:52Z ***********************
11:45:52:WU00:FS01:0x22:*************************** Core22 Folding@home Core ***************************
11:45:52:WU00:FS01:0x22:       Core: Core22
11:45:52:WU00:FS01:0x22:       Type: 0x22
11:45:52:WU00:FS01:0x22:    Version: 0.0.11
11:45:52:WU00:FS01:0x22:     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
11:45:52:WU00:FS01:0x22:  Copyright: 2020 foldingathome.org
11:45:52:WU00:FS01:0x22:   Homepage: https://foldingathome.org/
11:45:52:WU00:FS01:0x22:       Date: Jun 27 2020
11:45:52:WU00:FS01:0x22:       Time: 22:50:00
11:45:52:WU00:FS01:0x22:   Revision: cfc2940c5dd1aa80f60daa6e28d4a2a417f74edb
11:45:52:WU00:FS01:0x22:     Branch: core22-0.0.11
11:45:52:WU00:FS01:0x22:   Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
11:45:52:WU00:FS01:0x22:    Options: -std=c++11 -fsigned-char -ffunction-sections -fdata-sections -O3
11:45:52:WU00:FS01:0x22:             -funroll-loops
11:45:52:WU00:FS01:0x22:   Platform: linux2 4.19.76-linuxkit
11:45:52:WU00:FS01:0x22:       Bits: 64
11:45:52:WU00:FS01:0x22:       Mode: Release
11:45:52:WU00:FS01:0x22:Maintainers: John Chodera <john.chodera@choderalab.org> and Peter Eastman
11:45:52:WU00:FS01:0x22:             <peastman@stanford.edu>
11:45:52:WU00:FS01:0x22:       Args: -dir 00 -suffix 01 -version 706 -lifeline 24769 -checkpoint 15
11:45:52:WU00:FS01:0x22:             -gpu-vendor amd -opencl-platform 0 -opencl-device 0 -gpu 0
11:45:52:WU00:FS01:0x22:************************************ libFAH ************************************
11:45:52:WU00:FS01:0x22:       Date: Jun 27 2020
11:45:52:WU00:FS01:0x22:       Time: 22:11:04
11:45:52:WU00:FS01:0x22:   Revision: 2b383f4f04f38511dff592885d7c0400e72bdf43
11:45:52:WU00:FS01:0x22:     Branch: HEAD
11:45:52:WU00:FS01:0x22:   Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
11:45:52:WU00:FS01:0x22:    Options: -std=c++11 -fsigned-char -ffunction-sections -fdata-sections -O3
11:45:52:WU00:FS01:0x22:             -funroll-loops
11:45:52:WU00:FS01:0x22:   Platform: linux2 4.19.76-linuxkit
11:45:52:WU00:FS01:0x22:       Bits: 64
11:45:52:WU00:FS01:0x22:       Mode: Release
11:45:52:WU00:FS01:0x22:************************************ CBang *************************************
11:45:52:WU00:FS01:0x22:       Date: Jun 27 2020
11:45:52:WU00:FS01:0x22:       Time: 22:10:11
11:45:52:WU00:FS01:0x22:   Revision: f8529962055b0e7bde23e429f5072ff758089dee
11:45:52:WU00:FS01:0x22:     Branch: HEAD
11:45:52:WU00:FS01:0x22:   Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
11:45:52:WU00:FS01:0x22:    Options: -std=c++11 -fsigned-char -ffunction-sections -fdata-sections -O3
11:45:52:WU00:FS01:0x22:             -funroll-loops -fPIC
11:45:52:WU00:FS01:0x22:   Platform: linux2 4.19.76-linuxkit
11:45:52:WU00:FS01:0x22:       Bits: 64
11:45:52:WU00:FS01:0x22:       Mode: Release
11:45:52:WU00:FS01:0x22:************************************ System ************************************
11:45:52:WU00:FS01:0x22:        CPU: AMD Phenom(tm) II X6 1055T Processor
11:45:52:WU00:FS01:0x22:     CPU ID: AuthenticAMD Family 16 Model 10 Stepping 0
11:45:52:WU00:FS01:0x22:       CPUs: 6
11:45:52:WU00:FS01:0x22:     Memory: 7.78GiB
11:45:52:WU00:FS01:0x22:Free Memory: 3.74GiB
11:45:52:WU00:FS01:0x22:    Threads: POSIX_THREADS
11:45:52:WU00:FS01:0x22: OS Version: 5.6
11:45:52:WU00:FS01:0x22:Has Battery: false
11:45:52:WU00:FS01:0x22: On Battery: false
11:45:52:WU00:FS01:0x22: UTC Offset: 2
11:45:52:WU00:FS01:0x22:        PID: 24773
11:45:52:WU00:FS01:0x22:        CWD: /home/<user>/work
11:45:52:WU00:FS01:0x22:********************************************************************************
11:45:52:WU00:FS01:0x22:Project: 13424 (Run 2226, Clone 42, Gen 2)
11:45:52:WU00:FS01:0x22:Unit: 0x0000000412bc7d9a5f4ef98bb24f4b11
11:45:52:WU00:FS01:0x22:Reading tar file core.xml
11:45:52:WU00:FS01:0x22:Reading tar file integrator.xml
11:45:52:WU00:FS01:0x22:Reading tar file state.xml.bz2
11:45:53:WU00:FS01:0x22:Reading tar file system.xml.bz2
11:45:53:WU00:FS01:0x22:Digital signatures verified
11:45:53:WU00:FS01:0x22:Folding@home GPU Core22 Folding@home Core
11:45:53:WU00:FS01:0x22:Version 0.0.11
11:45:53:WU00:FS01:0x22:  Checkpoint write interval: 50000 steps (5%) [20 total]
11:45:53:WU00:FS01:0x22:  JSON viewer frame write interval: 10000 steps (1%) [100 total]
11:45:53:WU00:FS01:0x22:  XTC frame write interval: 250000 steps (25%) [4 total]
11:45:53:WU00:FS01:0x22:  Global context and integrator variables write interval: 25000 steps (2.5%) [40 total]
11:45:58:WU01:FS01:Upload 96.86%
11:46:00:WU01:FS01:Upload complete
11:46:00:WU01:FS01:Server responded WORK_ACK (400)
11:46:00:WU01:FS01:Cleaning up
11:46:17:WU02:FS00:0xa7:Completed 8750 out of 125000 steps (7%)
11:47:13:WU00:FS01:0x22:Completed 0 out of 1000000 steps (0%)
11:51:09:WU02:FS00:0xa7:Completed 10000 out of 125000 steps (8%)
11:51:54:WU00:FS01:0x22:An exception occurred at step 2258: Particle coordinate is nan
11:51:54:WU00:FS01:0x22:ERROR:98: Attempting to restart from last good checkpoint by restarting core.
11:51:54:WU00:FS01:0x22:Folding@home Core Shutdown: CORE_RESTART
11:51:55:WARNING:WU00:FS01:FahCore returned: CORE_RESTART (98 = 0x62)
11:51:55:WU00:FS01:Starting
11:51:55:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /home/<user>/cores/cores.foldingathome.org/lin/64bit/22-0.0.11/Core_22.fah/FahCore_22 -dir 00 -suffix 01 -version 706 -lifeline 4307 -checkpoint 15 -gpu-vendor amd -opencl-platform 0 -opencl-device 0 -gpu 0
11:51:55:WU00:FS01:Started FahCore on PID 31244
11:51:55:WU00:FS01:Core PID:31248
11:51:55:WU00:FS01:FahCore 0x22 started
11:51:55:WU00:FS01:0x22:*********************** Log Started 2020-09-03T11:51:55Z ***********************
11:51:55:WU00:FS01:0x22:*************************** Core22 Folding@home Core ***************************
11:51:55:WU00:FS01:0x22:       Core: Core22
11:51:55:WU00:FS01:0x22:       Type: 0x22
11:51:55:WU00:FS01:0x22:    Version: 0.0.11
11:51:55:WU00:FS01:0x22:     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
11:51:55:WU00:FS01:0x22:  Copyright: 2020 foldingathome.org
11:51:55:WU00:FS01:0x22:   Homepage: https://foldingathome.org/
11:51:55:WU00:FS01:0x22:       Date: Jun 27 2020
11:51:55:WU00:FS01:0x22:       Time: 22:50:00
11:51:55:WU00:FS01:0x22:   Revision: cfc2940c5dd1aa80f60daa6e28d4a2a417f74edb
11:51:55:WU00:FS01:0x22:     Branch: core22-0.0.11
11:51:55:WU00:FS01:0x22:   Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
11:51:55:WU00:FS01:0x22:    Options: -std=c++11 -fsigned-char -ffunction-sections -fdata-sections -O3
11:51:55:WU00:FS01:0x22:             -funroll-loops
11:51:55:WU00:FS01:0x22:   Platform: linux2 4.19.76-linuxkit
11:51:55:WU00:FS01:0x22:       Bits: 64
11:51:55:WU00:FS01:0x22:       Mode: Release
11:51:55:WU00:FS01:0x22:Maintainers: John Chodera <john.chodera@choderalab.org> and Peter Eastman
11:51:55:WU00:FS01:0x22:             <peastman@stanford.edu>
11:51:55:WU00:FS01:0x22:       Args: -dir 00 -suffix 01 -version 706 -lifeline 31244 -checkpoint 15
11:51:55:WU00:FS01:0x22:             -gpu-vendor amd -opencl-platform 0 -opencl-device 0 -gpu 0
11:51:55:WU00:FS01:0x22:************************************ libFAH ************************************
11:51:55:WU00:FS01:0x22:       Date: Jun 27 2020
11:51:55:WU00:FS01:0x22:       Time: 22:11:04
11:51:55:WU00:FS01:0x22:   Revision: 2b383f4f04f38511dff592885d7c0400e72bdf43
11:51:55:WU00:FS01:0x22:     Branch: HEAD
11:51:55:WU00:FS01:0x22:   Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
11:51:55:WU00:FS01:0x22:    Options: -std=c++11 -fsigned-char -ffunction-sections -fdata-sections -O3
11:51:55:WU00:FS01:0x22:             -funroll-loops
11:51:55:WU00:FS01:0x22:   Platform: linux2 4.19.76-linuxkit
11:51:55:WU00:FS01:0x22:       Bits: 64
11:51:55:WU00:FS01:0x22:       Mode: Release
11:51:55:WU00:FS01:0x22:************************************ CBang *************************************
11:51:55:WU00:FS01:0x22:       Date: Jun 27 2020
11:51:55:WU00:FS01:0x22:       Time: 22:10:11
11:51:55:WU00:FS01:0x22:   Revision: f8529962055b0e7bde23e429f5072ff758089dee
11:51:55:WU00:FS01:0x22:     Branch: HEAD
11:51:55:WU00:FS01:0x22:   Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
11:51:55:WU00:FS01:0x22:    Options: -std=c++11 -fsigned-char -ffunction-sections -fdata-sections -O3
11:51:55:WU00:FS01:0x22:             -funroll-loops -fPIC
11:51:55:WU00:FS01:0x22:   Platform: linux2 4.19.76-linuxkit
11:51:55:WU00:FS01:0x22:       Bits: 64
11:51:55:WU00:FS01:0x22:       Mode: Release
11:51:55:WU00:FS01:0x22:************************************ System ************************************
11:51:55:WU00:FS01:0x22:        CPU: AMD Phenom(tm) II X6 1055T Processor
11:51:55:WU00:FS01:0x22:     CPU ID: AuthenticAMD Family 16 Model 10 Stepping 0
11:51:55:WU00:FS01:0x22:       CPUs: 6
11:51:55:WU00:FS01:0x22:     Memory: 7.78GiB
11:51:55:WU00:FS01:0x22:Free Memory: 3.46GiB
11:51:55:WU00:FS01:0x22:    Threads: POSIX_THREADS
11:51:55:WU00:FS01:0x22: OS Version: 5.6
11:51:55:WU00:FS01:0x22:Has Battery: false
11:51:55:WU00:FS01:0x22: On Battery: false
11:51:55:WU00:FS01:0x22: UTC Offset: 2
11:51:55:WU00:FS01:0x22:        PID: 31248
11:51:55:WU00:FS01:0x22:        CWD: /home/<user>/work
11:51:55:WU00:FS01:0x22:********************************************************************************
11:51:55:WU00:FS01:0x22:Project: 13424 (Run 2226, Clone 42, Gen 2)
11:51:55:WU00:FS01:0x22:Unit: 0x0000000412bc7d9a5f4ef98bb24f4b11
11:51:55:WU00:FS01:0x22:Digital signatures verified
11:51:55:WU00:FS01:0x22:Folding@home GPU Core22 Folding@home Core
11:51:55:WU00:FS01:0x22:Version 0.0.11
11:51:56:WU00:FS01:0x22:  Checkpoint write interval: 50000 steps (5%) [20 total]
11:51:56:WU00:FS01:0x22:  JSON viewer frame write interval: 10000 steps (1%) [100 total]
11:51:56:WU00:FS01:0x22:  XTC frame write interval: 250000 steps (25%) [4 total]
11:51:56:WU00:FS01:0x22:  Global context and integrator variables write interval: 25000 steps (2.5%) [40 total]
11:52:46:WU00:FS01:0x22:ERROR:Discrepancy: Forces are blowing up! 4656 0
11:52:46:WU00:FS01:0x22:Saving result file ../logfile_01.txt
11:52:46:WU00:FS01:0x22:Saving result file science.log
11:52:46:WU00:FS01:0x22:Saving result file state.xml.bz2
11:52:48:WU00:FS01:0x22:Folding@home Core Shutdown: BAD_WORK_UNIT
11:52:48:WARNING:WU00:FS01:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
11:52:48:WU00:FS01:Sending unit results: id:00 state:SEND error:FAULTY project:13424 run:2226 clone:42 gen:2 core:0x22 unit:0x0000000412bc7d9a5f4ef98bb24f4b11
11:52:48:WU00:FS01:Uploading 5.62MiB to 18.188.125.154
11:52:48:WU00:FS01:Connecting to 18.188.125.154:8080
11:52:48:WU01:FS01:Connecting to assign1.foldingathome.org:80
11:52:49:WU01:FS01:Assigned to work server 155.247.166.220
11:52:49:WU01:FS01:Requesting new work unit for slot 01: READY gpu:0:Pitcairn [Radeon HD 7800] from 155.247.166.220
11:52:49:WU01:FS01:Connecting to 155.247.166.220:8080
11:52:49:WU01:FS01:Downloading 9.01MiB
11:52:54:WU00:FS01:Upload 50.06%
11:52:55:WU01:FS01:Download 73.52%
11:52:56:WU01:FS01:Download complete
11:52:57:WU01:FS01:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:16920 run:36 clone:15 gen:87 core:0x22 unit:0x000000740002894c5f2a13c71034115a
11:52:57:WU01:FS01:Starting
<GPU core failing again, CPU core continuing to work>
Top
JimboPalmer
Posts: 2522
Joined: Mon Feb 16, 2009 4:12 am
Location: Greenwood MS USA

Re: AMD Radeon HD 7870 Pitcairn cannot handle the force

Post by JimboPalmer »

Welcome to Folding@home!

https://www.techpowerup.com/gpu-specs/r ... ition.c339

Your card is old, but meets the minimum for F@H: OpenCL 1.2 and Double Precision floating point math (FP64).

11:32:20: GPUs: 1
11:32:20: GPU 0: Bus:1 Slot:0 Func:0 AMD:5 Pitcairn [Radeon HD 7800]
11:32:20: CUDA: Not detected: Failed to open dynamic library 'libcuda.so':
11:32:20: libcuda.so: cannot open shared object file: No such file or
11:32:20: directory
11:32:20:OpenCL Device 0: Platform:0 Device:0 Bus:1 Slot:0 Compute:1.2 Driver:3143.9
11:32:20:OpenCL Device 1: Platform:1 Device:0 Bus:NA Slot:NA Compute:1.1 Driver:18.3

The OpenCL Device 1 looks like an Intel software, I would try to remove it. (if you know why you do need the Intel driver, ignore me)

https://www.guru3d.com/files-details/di ... nload.html

Your AMD driver seems up to date.
Tsar of all the Rushers
I tried to remain childlike, all I achieved was childish.
A friend to those who want no friends
Top
JohnChodera
Pande Group Member
Posts: 467
Joined: Fri Feb 22, 2013 9:59 pm

Re: AMD Radeon HD 7870 Pitcairn cannot handle the force

Post by JohnChodera »

@haveyaseen: Thanks so much for joining us! We're actually working through a number of issues we found in OpenMM, the open source GPU-accelerated molecular simulation code http://openmm.org, that impact AMD cards. We think we've solved all the issues and are targeting a core22 0.0.12 release with these bugfixes roughly Sep 14-18. If you want to test it out ahead of time, shoot me a private message with your email and we can invite you to test a preview!

~ John Chodera // MSKCC
Top
haveyaseen
Posts: 4
Joined: Mon Mar 16, 2020 10:37 am

Re: AMD Radeon HD 7870 Pitcairn cannot handle the force

Post by haveyaseen »

Thanks for the warm welcomes and for the quick replies!

@JimboPalmer Thanks for the hint, I just removed the second device by uninstalling the free mesa-opencl-icd (https://packages.debian.org/buster/mesa-opencl-icd).

@JohnChodera Awesome, sure would love to try it out! I'll send you my email.
Top
Joe_H
Site Admin
Posts: 7986
Joined: Tue Apr 21, 2009 4:41 pm
Hardware configuration: Mac Pro 2.8 quad 12 GB smp4
MacBook Pro 2.9 i7 8 GB smp2
Location: W. MA

Re: AMD Radeon HD 7870 Pitcairn cannot handle the force

Post by Joe_H »

AMD support for older TeraScale and GCN 1st gen based GPUs on Linux is more or less non-existent, last drivers that fully support a card such as your HD 7870 are several years old. You might have better luck with a GCN 2nd gen or newer card on Linux.

I went looking for information on this type of problem earlier this year. To start, the Mesa drivers and its OpenCL support don't work for folding. Support for older cards was not included in the ROCm drivers. That leaves the AMD proprietary drivers.

During my searches back then I came across mention that the newest AMD drivers that supported the HD 7870 and similar cards on Linux only included video drivers, no support for OpenCL. Also mentioned in a post was that the last version that did include OpenCL support gave incorrect computational results, they had to drop back to a prior version. I don't recall the version number now, but think it was 18.something.

Beyond that were mentions of needing to use older Linux kernels, the drivers would not work with too new of a kernel.

With all of that, best I can recommend is moving the card to a Windows system. There AMD at least kept supporting the GPU with driver updates. I say this as a person who prefers Linux and OS X, and avoid using Windows when I can.
Image

iMac 2.8 i7 12 GB smp8, Mac Pro 2.8 quad 12 GB smp6
MacBook Pro 2.9 i7 8 GB smp3
Top
Post Reply

Return to “Problems with AMD/ATI drivers”