freeze during folding

Moderators: Site Moderators, FAHC Science Team

Post Reply
doune
Posts: 30
Joined: Fri Jul 13, 2018 11:37 pm

freeze during folding

Post by doune »

This problem only seemed to happen after my recent upgrade to a 2070. I don't know whether to blame the card or the drivers or both.

For the most part, folding runs fine for days on end. About four times since the 2070 was added to the GPUs.txt file, I've come to use my computer and it seems more-or-less frozen. The mouse appears disabled no matter where I plug it in. The keyboard does not wake the monitor. The fan noises are quieter than during usual folding. However, there is still intermittent hard drive activity.

Ctrl+alt+del does nothing. Windows key does nothing. Moving the mouse does nothing. Hitting the power button on the tower case, which would normally trigger a graceful shutdown, does nothing.

No matter what I do, the monitor stays asleep and none of the key presses seem to cause any change or additional activity from the PC (as measured by some change in lights or noise or anything else.)

My only fix is to just hit the Reset button on the case and reboot.

There is one log from before the forced reboot and one from right after:

Code: Select all

*********************** Log Started 2018-11-10T19:49:03Z ***********************
19:49:03:************************* Folding@home Client *************************
19:49:03:        Website: https://foldingathome.org/
19:49:03:      Copyright: (c) 2009-2018 foldingathome.org
19:49:03:         Author: Joseph Coffland <joseph@cauldrondevelopment.com>
19:49:03:           Args: 
19:49:03:         Config: C:\Users\doune\AppData\Roaming\FAHClient\config.xml
19:49:03:******************************** Build ********************************
19:49:03:        Version: 7.5.1
19:49:03:           Date: May 11 2018
19:49:03:           Time: 13:06:32
19:49:03:     Repository: Git
19:49:03:       Revision: 4705bf53c635f88b8fe85af7675557e15d491ff0
19:49:03:         Branch: master
19:49:03:       Compiler: Visual C++ 2008
19:49:03:        Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
19:49:03:       Platform: win32 10
19:49:03:           Bits: 32
19:49:03:           Mode: Release
19:49:03:******************************* System ********************************
19:49:03:            CPU: Intel(R) Core(TM) i7-4930K CPU @ 3.40GHz
19:49:03:         CPU ID: GenuineIntel Family 6 Model 62 Stepping 4
19:49:03:           CPUs: 12
19:49:03:         Memory: 15.94GiB
19:49:03:    Free Memory: 13.96GiB
19:49:03:        Threads: WINDOWS_THREADS
19:49:03:     OS Version: 6.2
19:49:03:    Has Battery: false
19:49:03:     On Battery: false
19:49:03:     UTC Offset: -6
19:49:03:            PID: 10568
19:49:03:            CWD: C:\Users\doune\AppData\Roaming\FAHClient
19:49:03:             OS: Windows 10 Enterprise
19:49:03:        OS Arch: AMD64
19:49:03:           GPUs: 1
19:49:03:          GPU 0: Bus:1 Slot:0 Func:0 NVIDIA:7 TU106-400A-A1 [GeForce RTX 2070
19:49:03:                 Rev. A] M 6497
19:49:03:  CUDA Device 0: Platform:0 Device:0 Bus:1 Slot:0 Compute:7.5 Driver:10.0
19:49:03:OpenCL Device 0: Platform:0 Device:0 Bus:1 Slot:0 Compute:1.2 Driver:416.34
19:49:03:  Win32 Service: false
19:49:03:***********************************************************************
19:49:03:<config>
19:49:03:  <!-- Slot Control -->
19:49:03:  <power v='FULL'/>
19:49:03:
19:49:03:  <!-- User Information -->
19:49:03:  <passkey v='********************************'/>
19:49:03:  <team v='233892'/>
19:49:03:  <user v='morphal'/>
19:49:03:
19:49:03:  <!-- Folding Slots -->
19:49:03:  <slot id='0' type='CPU'/>
19:49:03:  <slot id='1' type='GPU'/>
19:49:03:</config>
19:49:03:Trying to access database...
19:49:03:Successfully acquired database lock
19:49:03:Enabled folding slot 00: READY cpu:11
19:49:03:Enabled folding slot 01: READY gpu:0:TU106-400A-A1 [GeForce RTX 2070 Rev. A] M 6497
19:49:03:WU00:FS01:Starting
19:49:03:WU00:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\Users\doune\AppData\Roaming\FAHClient\cores/cores.foldingathome.org/Win32/AMD64/NVIDIA/Fermi/Core_21.fah/FahCore_21.exe -dir 00 -suffix 01 -version 705 -lifeline 10568 -checkpoint 15 -gpu-vendor nvidia -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu 0
19:49:03:WU00:FS01:Started FahCore on PID 11088
19:49:03:WU00:FS01:Core PID:11112
19:49:03:WU00:FS01:FahCore 0x21 started
19:49:03:WU02:FS00:Starting
19:49:03:WARNING:WU02:FS00:AS lowered CPUs from 11 to 10
19:49:03:WU02:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\Users\doune\AppData\Roaming\FAHClient\cores/cores.foldingathome.org/Win32/AMD64/AVX/Core_a7.fah/FahCore_a7.exe -dir 02 -suffix 01 -version 705 -lifeline 10568 -checkpoint 15 -np 10
19:49:03:WU02:FS00:Started FahCore on PID 11156
19:49:03:WU02:FS00:Core PID:11204
19:49:03:WU02:FS00:FahCore 0xa7 started
19:49:04:WU00:FS01:0x21:*********************** Log Started 2018-11-10T19:49:03Z ***********************
19:49:04:WU00:FS01:0x21:Project: 11718 (Run 0, Clone 722, Gen 68)
19:49:04:WU00:FS01:0x21:Unit: 0x0000005b8ca304e75b846acdc52200c4
19:49:04:WU00:FS01:0x21:CPU: 0x00000000000000000000000000000000
19:49:04:WU00:FS01:0x21:Machine: 1
19:49:04:WU00:FS01:0x21:Reading tar file core.xml
19:49:04:WU00:FS01:0x21:Reading tar file integrator.xml
19:49:04:WU00:FS01:0x21:Reading tar file state.xml
19:49:04:WARNING:FS01:Size of positions 5845 does not match topology 5844
19:49:04:WARNING:FS01:Size of positions 5845 does not match topology 5844
19:49:04:WU02:FS00:0xa7:*********************** Log Started 2018-11-10T19:49:04Z ***********************
19:49:04:WU02:FS00:0xa7:************************** Gromacs Folding@home Core ***************************
19:49:04:WU02:FS00:0xa7:       Type: 0xa7
19:49:04:WU02:FS00:0xa7:       Core: Gromacs
19:49:04:WU02:FS00:0xa7:    Website: https://foldingathome.org/
19:49:04:WU02:FS00:0xa7:  Copyright: (c) 2009-2018 foldingathome.org
19:49:04:WU02:FS00:0xa7:     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
19:49:04:WU02:FS00:0xa7:       Args: -dir 02 -suffix 01 -version 705 -lifeline 11156 -checkpoint 15 -np
19:49:04:WARNING:FS01:Size of positions 5845 does not match topology 5844
19:49:04:WU02:FS00:0xa7:             10
19:49:04:WU02:FS00:0xa7:     Config: <none>
19:49:04:WU02:FS00:0xa7:************************************ Build *************************************
19:49:04:WU02:FS00:0xa7:    Version: 0.0.17
19:49:04:WU02:FS00:0xa7:       Date: Apr 27 2018
19:49:04:WU02:FS00:0xa7:       Time: 16:19:36
19:49:04:WU02:FS00:0xa7: Repository: Git
19:49:04:WU02:FS00:0xa7:   Revision: 21359963583d09ec2063ef946399441c4df4ccd7
19:49:04:WU02:FS00:0xa7:     Branch: master
19:49:04:WU02:FS00:0xa7:   Compiler: Visual C++ 2008
19:49:04:WU02:FS00:0xa7:    Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
19:49:04:WU02:FS00:0xa7:   Platform: win32 10
19:49:04:WU02:FS00:0xa7:       Bits: 64
19:49:04:WU02:FS00:0xa7:       Mode: Release
19:49:04:WU02:FS00:0xa7:       SIMD: avx_256
19:49:04:WU02:FS00:0xa7:************************************ System ************************************
19:49:04:WU02:FS00:0xa7:        CPU: Unknown
19:49:04:WU02:FS00:0xa7:     CPU ID: 
19:49:04:WU02:FS00:0xa7:       CPUs: 12
19:49:04:WU02:FS00:0xa7:     Memory: 15.94GiB
19:49:04:WU02:FS00:0xa7:Free Memory: 13.90GiB
19:49:04:WU02:FS00:0xa7:    Threads: WINDOWS_THREADS
19:49:04:WU02:FS00:0xa7: OS Version: 6.2
19:49:04:WU02:FS00:0xa7:Has Battery: false
19:49:04:WU02:FS00:0xa7: On Battery: false
19:49:04:WU02:FS00:0xa7: UTC Offset: -6
19:49:04:WU02:FS00:0xa7:        PID: 11204
19:49:04:WU02:FS00:0xa7:        CWD: C:\Users\doune\AppData\Roaming\FAHClient\work
19:49:04:WU02:FS00:0xa7:         OS: Windows 10 Pro
19:49:04:WU02:FS00:0xa7:    OS Arch: AMD64
19:49:04:WU02:FS00:0xa7:********************************************************************************
19:49:04:WU02:FS00:0xa7:Project: 13789 (Run 1, Clone 48, Gen 36)
19:49:04:WU02:FS00:0xa7:Unit: 0x0000002d0002894c5b7ad9c09c7475f9
19:49:04:WU02:FS00:0xa7:Digital signatures verified
19:49:04:WU02:FS00:0xa7:Calling: mdrun -s frame36.tpr -o frame36.trr -cpi state.cpt -cpt 15 -nt 10
19:49:04:WU00:FS01:0x21:Reading tar file system.xml
19:49:04:WARNING:FS01:Size of positions 5845 does not match topology 5844
19:49:04:WARNING:FS01:Size of positions 5845 does not match topology 5844
19:49:04:WU02:FS00:0xa7:Steps: first=90000000 total=2500000
19:49:04:WARNING:FS01:Size of positions 5845 does not match topology 5844
19:49:04:WARNING:FS01:Size of positions 5845 does not match topology 5844
19:49:04:WARNING:FS01:Size of positions 5845 does not match topology 5844
19:49:04:WARNING:FS01:Size of positions 5845 does not match topology 5844
19:49:04:WARNING:FS01:Size of positions 5845 does not match topology 5844
19:49:04:WARNING:FS01:Size of positions 5845 does not match topology 5844
19:49:05:WU00:FS01:0x21:Digital signatures verified
19:49:05:WU00:FS01:0x21:Folding@home GPU Core21 Folding@home Core
19:49:05:WU00:FS01:0x21:Version 0.0.18
19:49:05:WARNING:FS01:Size of positions 5845 does not match topology 5844
19:49:05:WARNING:FS01:Size of positions 5845 does not match topology 5844
19:49:06:WU02:FS00:0xa7:Completed 1394901 out of 2500000 steps (55%)
19:49:17:WU00:FS01:0x21:Completed 0 out of 5000000 steps (0%)
19:49:17:WU00:FS01:0x21:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
19:50:28:WU02:FS00:0xa7:Completed 1400000 out of 2500000 steps (56%)
19:51:01:WU00:FS01:0x21:Completed 50000 out of 5000000 steps (1%)
19:52:44:WU00:FS01:0x21:Completed 100000 out of 5000000 steps (2%)
19:54:28:WU00:FS01:0x21:Completed 150000 out of 5000000 steps (3%)
19:56:02:WU02:FS00:0xa7:Completed 1425000 out of 2500000 steps (57%)
19:56:13:WU00:FS01:0x21:Completed 200000 out of 5000000 steps (4%)
19:57:58:WU00:FS01:0x21:Completed 250000 out of 5000000 steps (5%)
19:59:44:WU00:FS01:0x21:Completed 300000 out of 5000000 steps (6%)
20:01:27:WU00:FS01:0x21:Completed 350000 out of 5000000 steps (7%)
20:01:33:WU02:FS00:0xa7:Completed 1450000 out of 2500000 steps (58%)
...
22:38:22:WU00:FS01:0x21:Completed 4900000 out of 5000000 steps (98%)
22:40:05:WU00:FS01:0x21:Completed 4950000 out of 5000000 steps (99%)
22:40:05:WU01:FS01:Connecting to 65.254.110.245:8080
22:40:05:WU01:FS01:Assigned to work server 140.163.4.231
22:40:05:WU01:FS01:Requesting new work unit for slot 01: RUNNING gpu:0:TU106-400A-A1 [GeForce RTX 2070 Rev. A] M 6497 from 140.163.4.231
22:40:05:WU01:FS01:Connecting to 140.163.4.231:8080
22:40:06:WU01:FS01:Downloading 33.32MiB
22:40:12:WU01:FS01:Download 77.09%
22:40:13:WU01:FS01:Download complete
22:40:13:WU01:FS01:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:11730 run:3 clone:423 gen:85 core:0x21 unit:0x000000638ca304e75bcd13a8255228a7
22:40:50:WU02:FS00:0xa7:Completed 2175000 out of 2500000 steps (87%)
22:41:47:WU00:FS01:0x21:Completed 5000000 out of 5000000 steps (100%)
22:41:50:WU00:FS01:0x21:Saving result file logfile_01.txt
22:41:50:WU00:FS01:0x21:Saving result file checkpointState.xml
22:41:50:WU00:FS01:0x21:Saving result file checkpt.crc
22:41:50:WU00:FS01:0x21:Saving result file log.txt
22:41:50:WU00:FS01:0x21:Saving result file positions.xtc
22:41:50:WU00:FS01:0x21:Folding@home Core Shutdown: FINISHED_UNIT
22:41:51:WU00:FS01:FahCore returned: FINISHED_UNIT (100 = 0x64)
22:41:51:WU00:FS01:Sending unit results: id:00 state:SEND error:NO_ERROR project:11718 run:0 clone:722 gen:68 core:0x21 unit:0x0000005b8ca304e75b846acdc52200c4
22:41:51:WU00:FS01:Uploading 15.09MiB to 140.163.4.231
22:41:51:WU00:FS01:Connecting to 140.163.4.231:8080
22:41:51:WU01:FS01:Starting
22:41:51:WU01:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\Users\doune\AppData\Roaming\FAHClient\cores/cores.foldingathome.org/Win32/AMD64/NVIDIA/Fermi/Core_21.fah/FahCore_21.exe -dir 01 -suffix 01 -version 705 -lifeline 10568 -checkpoint 15 -gpu-vendor nvidia -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu 0
22:41:51:WU01:FS01:Started FahCore on PID 1844
22:41:51:WU01:FS01:Core PID:7184
22:41:51:WU01:FS01:FahCore 0x21 started
22:41:52:WU01:FS01:0x21:*********************** Log Started 2018-11-10T22:41:51Z ***********************
22:41:52:WU01:FS01:0x21:Project: 11730 (Run 3, Clone 423, Gen 85)
22:41:52:WU01:FS01:0x21:Unit: 0x000000638ca304e75bcd13a8255228a7
22:41:52:WU01:FS01:0x21:CPU: 0x00000000000000000000000000000000
22:41:52:WU01:FS01:0x21:Machine: 1
22:41:52:WU01:FS01:0x21:Reading tar file core.xml
22:41:52:WU01:FS01:0x21:Reading tar file integrator.xml
22:41:52:WU01:FS01:0x21:Reading tar file state.xml
22:41:52:WU01:FS01:0x21:Reading tar file system.xml
22:41:53:WU01:FS01:0x21:Digital signatures verified
22:41:53:WU01:FS01:0x21:Folding@home GPU Core21 Folding@home Core
22:41:53:WU01:FS01:0x21:Version 0.0.18
22:41:57:WU00:FS01:Upload 23.61%
22:42:03:WU00:FS01:Upload 42.67%
22:42:06:WU01:FS01:0x21:Completed 0 out of 2500000 steps (0%)
22:42:06:WU01:FS01:0x21:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
22:42:09:WU00:FS01:Upload 64.21%
22:42:15:WU00:FS01:Upload 88.64%
22:42:19:WU00:FS01:Upload complete
22:42:19:WU00:FS01:Server responded WORK_ACK (400)
22:42:19:WU00:FS01:Final credit estimate, 82507.00 points
22:42:19:WU00:FS01:Cleaning up
22:43:13:WU01:FS01:0x21:Completed 25000 out of 2500000 steps (1%)
22:44:20:WU01:FS01:0x21:Completed 50000 out of 2500000 steps (2%)
22:45:27:WU01:FS01:0x21:Completed 75000 out of 2500000 steps (3%)
22:46:19:WU02:FS00:0xa7:Completed 2200000 out of 2500000 steps (88%)
22:46:34:WU01:FS01:0x21:Completed 100000 out of 2500000 steps (4%)
22:47:41:WU01:FS01:0x21:Completed 125000 out of 2500000 steps (5%)
22:48:48:WU01:FS01:0x21:Completed 150000 out of 2500000 steps (6%)
22:49:56:WU01:FS01:0x21:Completed 175000 out of 2500000 steps (7%)
22:51:03:WU01:FS01:0x21:Completed 200000 out of 2500000 steps (8%)
22:51:50:WU02:FS00:0xa7:Completed 2225000 out of 2500000 steps (89%)
...
23:46:08:WU01:FS01:0x21:Completed 1425000 out of 2500000 steps (57%)
23:46:57:WU02:FS00:0xa7:Completed 2475000 out of 2500000 steps (99%)
23:46:58:WU00:FS00:Connecting to 65.254.110.245:8080
23:46:58:WU00:FS00:Assigned to work server 128.252.203.2
23:46:58:WU00:FS00:Requesting new work unit for slot 00: RUNNING cpu:11 from 128.252.203.2
23:46:58:WU00:FS00:Connecting to 128.252.203.2:8080
23:46:58:WU00:FS00:Downloading 4.53MiB
23:47:00:WU00:FS00:Download complete
23:47:00:WU00:FS00:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:13809 run:0 clone:1548 gen:232 core:0xa7 unit:0x0000012980fccb025a919051ecb1e8f5
23:47:15:WU01:FS01:0x21:Completed 1450000 out of 2500000 steps (58%)
23:48:22:WU01:FS01:0x21:Completed 1475000 out of 2500000 steps (59%)
23:49:29:WU01:FS01:0x21:Completed 1500000 out of 2500000 steps (60%)
23:50:40:WU01:FS01:0x21:Completed 1525000 out of 2500000 steps (61%)
23:51:47:WU01:FS01:0x21:Completed 1550000 out of 2500000 steps (62%)
23:52:28:WU02:FS00:0xa7:Completed 2500000 out of 2500000 steps (100%)
23:52:30:WU02:FS00:0xa7:Saving result file ..\logfile_01.txt
23:52:30:WU02:FS00:0xa7:Saving result file ener.edr
23:52:30:WU02:FS00:0xa7:Saving result file frame36.trr
23:52:31:WU02:FS00:0xa7:Saving result file md.log
23:52:31:WU02:FS00:0xa7:Saving result file pullf.xvg
23:52:31:WU02:FS00:0xa7:Saving result file pullx.xvg
23:52:31:WU02:FS00:0xa7:Saving result file science.log
23:52:31:WU02:FS00:0xa7:Saving result file traj_comp.xtc
23:52:32:WU02:FS00:0xa7:Folding@home Core Shutdown: FINISHED_UNIT
23:52:32:WU02:FS00:FahCore returned: FINISHED_UNIT (100 = 0x64)
23:52:32:WU02:FS00:Sending unit results: id:02 state:SEND error:NO_ERROR project:13789 run:1 clone:48 gen:36 core:0xa7 unit:0x0000002d0002894c5b7ad9c09c7475f9
23:52:32:WU02:FS00:Uploading 8.15MiB to 155.247.166.220
23:52:32:WU00:FS00:Starting
23:52:32:WU02:FS00:Connecting to 155.247.166.220:8080
23:52:32:WARNING:WU00:FS00:AS lowered CPUs from 11 to 10
23:52:32:WU00:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\Users\doune\AppData\Roaming\FAHClient\cores/cores.foldingathome.org/Win32/AMD64/AVX/Core_a7.fah/FahCore_a7.exe -dir 00 -suffix 01 -version 705 -lifeline 10568 -checkpoint 15 -np 10
23:52:32:WU00:FS00:Started FahCore on PID 10344
23:52:32:WU00:FS00:Core PID:5252
23:52:32:WU00:FS00:FahCore 0xa7 started
23:52:33:WU00:FS00:0xa7:*********************** Log Started 2018-11-10T23:52:32Z ***********************
23:52:33:WU00:FS00:0xa7:************************** Gromacs Folding@home Core ***************************
23:52:33:WU00:FS00:0xa7:       Type: 0xa7
23:52:33:WU00:FS00:0xa7:       Core: Gromacs
23:52:33:WU00:FS00:0xa7:    Website: https://foldingathome.org/
23:52:33:WU00:FS00:0xa7:  Copyright: (c) 2009-2018 foldingathome.org
23:52:33:WU00:FS00:0xa7:     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
23:52:33:WU00:FS00:0xa7:       Args: -dir 00 -suffix 01 -version 705 -lifeline 10344 -checkpoint 15 -np
23:52:33:WU00:FS00:0xa7:             10
23:52:33:WU00:FS00:0xa7:     Config: <none>
23:52:33:WU00:FS00:0xa7:************************************ Build *************************************
23:52:33:WU00:FS00:0xa7:    Version: 0.0.17
23:52:33:WU00:FS00:0xa7:       Date: Apr 27 2018
23:52:33:WU00:FS00:0xa7:       Time: 16:19:36
23:52:33:WU00:FS00:0xa7: Repository: Git
23:52:33:WU00:FS00:0xa7:   Revision: 21359963583d09ec2063ef946399441c4df4ccd7
23:52:33:WU00:FS00:0xa7:     Branch: master
23:52:33:WU00:FS00:0xa7:   Compiler: Visual C++ 2008
23:52:33:WU00:FS00:0xa7:    Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
23:52:33:WU00:FS00:0xa7:   Platform: win32 10
23:52:33:WU00:FS00:0xa7:       Bits: 64
23:52:33:WU00:FS00:0xa7:       Mode: Release
23:52:33:WU00:FS00:0xa7:       SIMD: avx_256
23:52:33:WU00:FS00:0xa7:************************************ System ************************************
23:52:33:WU00:FS00:0xa7:        CPU: Unknown
23:52:33:WU00:FS00:0xa7:     CPU ID: 
23:52:33:WU00:FS00:0xa7:       CPUs: 12
23:52:33:WU00:FS00:0xa7:     Memory: 15.94GiB
23:52:33:WU00:FS00:0xa7:Free Memory: 12.37GiB
23:52:33:WU00:FS00:0xa7:    Threads: WINDOWS_THREADS
23:52:33:WU00:FS00:0xa7: OS Version: 6.2
23:52:33:WU00:FS00:0xa7:Has Battery: false
23:52:33:WU00:FS00:0xa7: On Battery: false
23:52:33:WU00:FS00:0xa7: UTC Offset: -6
23:52:33:WU00:FS00:0xa7:        PID: 5252
23:52:33:WU00:FS00:0xa7:        CWD: C:\Users\doune\AppData\Roaming\FAHClient\work
23:52:33:WU00:FS00:0xa7:         OS: Windows 10 Pro
23:52:33:WU00:FS00:0xa7:    OS Arch: AMD64
23:52:33:WU00:FS00:0xa7:********************************************************************************
23:52:33:WU00:FS00:0xa7:Project: 13809 (Run 0, Clone 1548, Gen 232)
23:52:33:WU00:FS00:0xa7:Unit: 0x0000012980fccb025a919051ecb1e8f5
23:52:33:WU00:FS00:0xa7:Reading tar file core.xml
23:52:33:WU00:FS00:0xa7:Reading tar file frame232.tpr
23:52:33:WU00:FS00:0xa7:Digital signatures verified
23:52:33:WU00:FS00:0xa7:Calling: mdrun -s frame232.tpr -o frame232.trr -x frame232.xtc -cpt 15 -nt 10
23:52:33:WU00:FS00:0xa7:Steps: first=58000000 total=250000
23:52:36:WU00:FS00:0xa7:Completed 1 out of 250000 steps (0%)
23:52:38:WU02:FS00:Upload 41.40%
23:52:44:WU02:FS00:Upload 91.99%
23:52:48:WU02:FS00:Upload complete
23:52:48:WU02:FS00:Server responded WORK_ACK (400)
23:52:48:WU02:FS00:Final credit estimate, 15524.00 points
23:52:48:WU02:FS00:Cleaning up
23:52:54:WU01:FS01:0x21:Completed 1575000 out of 2500000 steps (63%)
23:54:01:WU01:FS01:0x21:Completed 1600000 out of 2500000 steps (64%)
23:55:08:WU01:FS01:0x21:Completed 1625000 out of 2500000 steps (65%)
23:55:15:WU00:FS00:0xa7:Completed 2500 out of 250000 steps (1%)
23:56:15:WU01:FS01:0x21:Completed 1650000 out of 2500000 steps (66%)
23:57:22:WU01:FS01:0x21:Completed 1675000 out of 2500000 steps (67%)
23:57:52:WU00:FS00:0xa7:Completed 5000 out of 250000 steps (2%)
23:58:29:WU01:FS01:0x21:Completed 1700000 out of 2500000 steps (68%)
23:59:36:WU01:FS01:0x21:Completed 1725000 out of 2500000 steps (69%)
00:00:29:WU00:FS00:0xa7:Completed 7500 out of 250000 steps (3%)
00:00:43:WU01:FS01:0x21:Completed 1750000 out of 2500000 steps (70%)
00:01:53:WU01:FS01:0x21:Completed 1775000 out of 2500000 steps (71%)
00:03:01:WU01:FS01:0x21:Completed 1800000 out of 2500000 steps (72%)
00:03:06:WU00:FS00:0xa7:Completed 10000 out of 250000 steps (4%)
00:04:07:WU01:FS01:0x21:Completed 1825000 out of 2500000 steps (73%)
00:05:14:WU01:FS01:0x21:Completed 1850000 out of 2500000 steps (74%)
00:05:44:WU00:FS00:0xa7:Completed 12500 out of 250000 steps (5%)
00:06:21:WU01:FS01:0x21:Completed 1875000 out of 2500000 steps (75%)
00:07:28:WU01:FS01:0x21:Completed 1900000 out of 2500000 steps (76%)
00:08:23:WU00:FS00:0xa7:Completed 15000 out of 250000 steps (6%)
00:08:35:WU01:FS01:0x21:Completed 1925000 out of 2500000 steps (77%)
00:09:42:WU01:FS01:0x21:Completed 1950000 out of 2500000 steps (78%)
00:10:49:WU01:FS01:0x21:Completed 1975000 out of 2500000 steps (79%)
00:11:01:WU00:FS00:0xa7:Completed 17500 out of 250000 steps (7%)
00:11:56:WU01:FS01:0x21:Completed 2000000 out of 2500000 steps (80%)
00:13:07:WU01:FS01:0x21:Completed 2025000 out of 2500000 steps (81%)
00:13:38:WU00:FS00:0xa7:Completed 20000 out of 250000 steps (8%)
00:14:14:WU01:FS01:0x21:Completed 2050000 out of 2500000 steps (82%)
00:15:21:WU01:FS01:0x21:Completed 2075000 out of 2500000 steps (83%)
00:16:16:WU00:FS00:0xa7:Completed 22500 out of 250000 steps (9%)
00:16:28:WU01:FS01:0x21:Completed 2100000 out of 2500000 steps (84%)
00:17:35:WU01:FS01:0x21:Completed 2125000 out of 2500000 steps (85%)
00:18:42:WU01:FS01:0x21:Completed 2150000 out of 2500000 steps (86%)
00:18:53:WU00:FS00:0xa7:Completed 25000 out of 250000 steps (10%)
00:19:49:WU01:FS01:0x21:Completed 2175000 out of 2500000 steps (87%)
00:20:56:WU01:FS01:0x21:Completed 2200000 out of 2500000 steps (88%)
00:21:31:WU00:FS00:0xa7:Completed 27500 out of 250000 steps (11%)
00:22:03:WU01:FS01:0x21:Completed 2225000 out of 2500000 steps (89%)
00:23:10:WU01:FS01:0x21:Completed 2250000 out of 2500000 steps (90%)
00:24:10:WU00:FS00:0xa7:Completed 30000 out of 250000 steps (12%)
00:24:21:WU01:FS01:0x21:Completed 2275000 out of 2500000 steps (91%)
00:25:28:WU01:FS01:0x21:Completed 2300000 out of 2500000 steps (92%)
00:26:35:WU01:FS01:0x21:Completed 2325000 out of 2500000 steps (93%)
00:26:48:WU00:FS00:0xa7:Completed 32500 out of 250000 steps (13%)
00:27:42:WU01:FS01:0x21:Completed 2350000 out of 2500000 steps (94%)
00:28:49:WU01:FS01:0x21:Completed 2375000 out of 2500000 steps (95%)
00:29:26:WU00:FS00:0xa7:Completed 35000 out of 250000 steps (14%)
00:29:56:WU01:FS01:0x21:Completed 2400000 out of 2500000 steps (96%)
00:31:03:WU01:FS01:0x21:Completed 2425000 out of 2500000 steps (97%)
00:32:04:WU00:FS00:0xa7:Completed 37500 out of 250000 steps (15%)
00:32:10:WU01:FS01:0x21:Completed 2450000 out of 2500000 steps (98%)
00:33:17:WU01:FS01:0x21:Completed 2475000 out of 2500000 steps (99%)
00:33:18:WU02:FS01:Connecting to 65.254.110.245:8080
00:33:18:WU02:FS01:Assigned to work server 140.163.4.231
00:33:18:WU02:FS01:Requesting new work unit for slot 01: RUNNING gpu:0:TU106-400A-A1 [GeForce RTX 2070 Rev. A] M 6497 from 140.163.4.231
00:33:18:WU02:FS01:Connecting to 140.163.4.231:8080
00:33:19:WU02:FS01:Downloading 25.28MiB
00:33:25:WU02:FS01:Download 97.43%
00:33:25:WU02:FS01:Download complete
00:33:25:WU02:FS01:Received Unit: id:02 state:DOWNLOAD error:NO_ERROR project:11718 run:0 clone:871 gen:155 core:0x21 unit:0x000000d28ca304e75b846ae0340f90c4
00:34:24:WU01:FS01:0x21:Completed 2500000 out of 2500000 steps (100%)
00:34:28:WU01:FS01:0x21:Saving result file logfile_01.txt
00:34:28:WU01:FS01:0x21:Saving result file checkpointState.xml
00:34:28:WU01:FS01:0x21:Saving result file checkpt.crc
00:34:28:WU01:FS01:0x21:Saving result file log.txt
00:34:28:WU01:FS01:0x21:Saving result file positions.xtc
00:34:28:WU01:FS01:0x21:Folding@home Core Shutdown: FINISHED_UNIT
00:34:29:WU01:FS01:FahCore returned: FINISHED_UNIT (100 = 0x64)
00:34:29:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:11730 run:3 clone:423 gen:85 core:0x21 unit:0x000000638ca304e75bcd13a8255228a7
00:34:29:WU01:FS01:Uploading 18.27MiB to 140.163.4.231
00:34:29:WU01:FS01:Connecting to 140.163.4.231:8080
00:34:29:WU02:FS01:Starting
00:34:29:WU02:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\Users\doune\AppData\Roaming\FAHClient\cores/cores.foldingathome.org/Win32/AMD64/NVIDIA/Fermi/Core_21.fah/FahCore_21.exe -dir 02 -suffix 01 -version 705 -lifeline 10568 -checkpoint 15 -gpu-vendor nvidia -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu 0
00:34:29:WU02:FS01:Started FahCore on PID 1932
00:34:29:WU02:FS01:Core PID:11212
00:34:29:WU02:FS01:FahCore 0x21 started
00:34:29:WU02:FS01:0x21:*********************** Log Started 2018-11-11T00:34:29Z ***********************
00:34:29:WU02:FS01:0x21:Project: 11718 (Run 0, Clone 871, Gen 155)
00:34:29:WU02:FS01:0x21:Unit: 0x000000d28ca304e75b846ae0340f90c4
00:34:29:WU02:FS01:0x21:CPU: 0x00000000000000000000000000000000
00:34:29:WU02:FS01:0x21:Machine: 1
00:34:29:WU02:FS01:0x21:Reading tar file core.xml
00:34:29:WU02:FS01:0x21:Reading tar file integrator.xml
00:34:29:WU02:FS01:0x21:Reading tar file state.xml
00:34:30:WU02:FS01:0x21:Reading tar file system.xml
00:34:30:WU02:FS01:0x21:Digital signatures verified
00:34:30:WU02:FS01:0x21:Folding@home GPU Core21 Folding@home Core
00:34:30:WU02:FS01:0x21:Version 0.0.18
00:34:35:WU01:FS01:Upload 20.87%
00:34:40:WU02:FS01:0x21:Completed 0 out of 5000000 steps (0%)
00:34:40:WU02:FS01:0x21:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
00:34:41:WU01:FS01:Upload 43.11%
00:34:41:WU00:FS00:0xa7:Completed 40000 out of 250000 steps (16%)
00:34:47:WU01:FS01:Upload 60.57%
00:34:53:WU01:FS01:Upload 83.15%
00:34:59:WU01:FS01:Upload complete
00:34:59:WU01:FS01:Server responded WORK_ACK (400)
00:34:59:WU01:FS01:Final credit estimate, 82059.00 points
00:34:59:WU01:FS01:Cleaning up
00:36:23:WU02:FS01:0x21:Completed 50000 out of 5000000 steps (1%)
00:37:18:WU00:FS00:0xa7:Completed 42500 out of 250000 steps (17%)
00:38:06:WU02:FS01:0x21:Completed 100000 out of 5000000 steps (2%)
00:39:49:WU02:FS01:0x21:Completed 150000 out of 5000000 steps (3%)
00:39:56:WU00:FS00:0xa7:Completed 45000 out of 250000 steps (18%)
00:41:32:WU02:FS01:0x21:Completed 200000 out of 5000000 steps (4%)
00:42:32:WU00:FS00:0xa7:Completed 47500 out of 250000 steps (19%)
00:43:15:WU02:FS01:0x21:Completed 250000 out of 5000000 steps (5%)
00:43:22:WARNING:FS01:Size of positions 5845 does not match topology 5844
00:45:00:WU02:FS01:0x21:Completed 300000 out of 5000000 steps (6%)
00:45:09:WU00:FS00:0xa7:Completed 50000 out of 250000 steps (20%)
00:46:43:WU02:FS01:0x21:Completed 350000 out of 5000000 steps (7%)
00:47:45:WU00:FS00:0xa7:Completed 52500 out of 250000 steps (21%)
00:48:26:WU02:FS01:0x21:Completed 400000 out of 5000000 steps (8%)
00:50:09:WU02:FS01:0x21:Completed 450000 out of 5000000 steps (9%)
00:50:22:WU00:FS00:0xa7:Completed 55000 out of 250000 steps (22%)
00:51:51:WU02:FS01:0x21:Completed 500000 out of 5000000 steps (10%)
00:51:59:WARNING:FS01:Size of positions 5845 does not match topology 5844
00:53:00:WU00:FS00:0xa7:Completed 57500 out of 250000 steps (23%)
00:53:37:WU02:FS01:0x21:Completed 550000 out of 5000000 steps (11%)
00:55:20:WU02:FS01:0x21:Completed 600000 out of 5000000 steps (12%)
00:55:36:WU00:FS00:0xa7:Completed 60000 out of 250000 steps (24%)
00:57:03:WU02:FS01:0x21:Completed 650000 out of 5000000 steps (13%)
00:58:13:WU00:FS00:0xa7:Completed 62500 out of 250000 steps (25%)
00:58:46:WU02:FS01:0x21:Completed 700000 out of 5000000 steps (14%)
01:00:28:WU02:FS01:0x21:Completed 750000 out of 5000000 steps (15%)
01:00:36:WARNING:FS01:Size of positions 5845 does not match topology 5844
01:00:49:WU00:FS00:0xa7:Completed 65000 out of 250000 steps (26%)
01:02:14:WU02:FS01:0x21:Completed 800000 out of 5000000 steps (16%)
01:03:26:WU00:FS00:0xa7:Completed 67500 out of 250000 steps (27%)
01:03:57:WU02:FS01:0x21:Completed 850000 out of 5000000 steps (17%)
01:05:40:WU02:FS01:0x21:Completed 900000 out of 5000000 steps (18%)
01:06:03:WU00:FS00:0xa7:Completed 70000 out of 250000 steps (28%)
01:07:23:WU02:FS01:0x21:Completed 950000 out of 5000000 steps (19%)
01:08:41:WU00:FS00:0xa7:Completed 72500 out of 250000 steps (29%)
01:09:06:WU02:FS01:0x21:Completed 1000000 out of 5000000 steps (20%)
01:09:13:WARNING:FS01:Size of positions 5845 does not match topology 5844
01:10:51:WU02:FS01:0x21:Completed 1050000 out of 5000000 steps (21%)
01:11:18:WU00:FS00:0xa7:Completed 75000 out of 250000 steps (30%)
01:12:34:WU02:FS01:0x21:Completed 1100000 out of 5000000 steps (22%)
01:13:54:WU00:FS00:0xa7:Completed 77500 out of 250000 steps (31%)
01:14:17:WU02:FS01:0x21:Completed 1150000 out of 5000000 steps (23%)
01:16:00:WU02:FS01:0x21:Completed 1200000 out of 5000000 steps (24%)
01:16:31:WU00:FS00:0xa7:Completed 80000 out of 250000 steps (32%)
01:17:43:WU02:FS01:0x21:Completed 1250000 out of 5000000 steps (25%)
01:17:51:WARNING:FS01:Size of positions 5845 does not match topology 5844
01:19:08:WU00:FS00:0xa7:Completed 82500 out of 250000 steps (33%)
01:19:29:WU02:FS01:0x21:Completed 1300000 out of 5000000 steps (26%)
01:21:12:WU02:FS01:0x21:Completed 1350000 out of 5000000 steps (27%)
01:21:45:WU00:FS00:0xa7:Completed 85000 out of 250000 steps (34%)
01:22:55:WU02:FS01:0x21:Completed 1400000 out of 5000000 steps (28%)
01:24:23:WU00:FS00:0xa7:Completed 87500 out of 250000 steps (35%)
01:24:37:WU02:FS01:0x21:Completed 1450000 out of 5000000 steps (29%)
01:26:21:WU02:FS01:0x21:Completed 1500000 out of 5000000 steps (30%)
01:26:28:WARNING:FS01:Size of positions 5845 does not match topology 5844
01:27:00:WU00:FS00:0xa7:Completed 90000 out of 250000 steps (36%)
01:28:06:WU02:FS01:0x21:Completed 1550000 out of 5000000 steps (31%)
01:29:36:WU00:FS00:0xa7:Completed 92500 out of 250000 steps (37%)
01:29:49:WU02:FS01:0x21:Completed 1600000 out of 5000000 steps (32%)
01:31:32:WU02:FS01:0x21:Completed 1650000 out of 5000000 steps (33%)
01:32:13:WU00:FS00:0xa7:Completed 95000 out of 250000 steps (38%)
01:33:15:WU02:FS01:0x21:Completed 1700000 out of 5000000 steps (34%)
01:34:50:WU00:FS00:0xa7:Completed 97500 out of 250000 steps (39%)
01:34:57:WU02:FS01:0x21:Completed 1750000 out of 5000000 steps (35%)
01:35:05:WARNING:FS01:Size of positions 5845 does not match topology 5844
01:36:43:WU02:FS01:0x21:Completed 1800000 out of 5000000 steps (36%)
01:37:26:WU00:FS00:0xa7:Completed 100000 out of 250000 steps (40%)
01:38:26:WU02:FS01:0x21:Completed 1850000 out of 5000000 steps (37%)
01:40:05:WU00:FS00:0xa7:Completed 102500 out of 250000 steps (41%)
01:40:09:WU02:FS01:0x21:Completed 1900000 out of 5000000 steps (38%)
01:41:52:WU02:FS01:0x21:Completed 1950000 out of 5000000 steps (39%)
01:42:42:WU00:FS00:0xa7:Completed 105000 out of 250000 steps (42%)
01:43:35:WU02:FS01:0x21:Completed 2000000 out of 5000000 steps (40%)
01:43:42:WARNING:FS01:Size of positions 5845 does not match topology 5844
01:45:19:WU00:FS00:0xa7:Completed 107500 out of 250000 steps (43%)
01:45:20:WU02:FS01:0x21:Completed 2050000 out of 5000000 steps (41%)
01:47:04:WU02:FS01:0x21:Completed 2100000 out of 5000000 steps (42%)
01:47:56:WU00:FS00:0xa7:Completed 110000 out of 250000 steps (44%)
01:48:46:WU02:FS01:0x21:Completed 2150000 out of 5000000 steps (43%)
******************************* Date: 2018-11-11 *******************************
01:50:29:WU02:FS01:0x21:Completed 2200000 out of 5000000 steps (44%)
01:50:32:WU00:FS00:0xa7:Completed 112500 out of 250000 steps (45%)
01:52:12:WU02:FS01:0x21:Completed 2250000 out of 5000000 steps (45%)
01:52:20:WARNING:FS01:Size of positions 5845 does not match topology 5844
01:53:11:WU00:FS00:0xa7:Completed 115000 out of 250000 steps (46%)
01:53:58:WU02:FS01:0x21:Completed 2300000 out of 5000000 steps (46%)
01:55:41:WU02:FS01:0x21:Completed 2350000 out of 5000000 steps (47%)
01:55:48:WU00:FS00:0xa7:Completed 117500 out of 250000 steps (47%)
01:57:24:WU02:FS01:0x21:Completed 2400000 out of 5000000 steps (48%)
01:58:25:WU00:FS00:0xa7:Completed 120000 out of 250000 steps (48%)
01:59:07:WU02:FS01:0x21:Completed 2450000 out of 5000000 steps (49%)
02:00:50:WU02:FS01:0x21:Completed 2500000 out of 5000000 steps (50%)
02:00:57:WARNING:FS01:Size of positions 5845 does not match topology 5844
02:01:01:WU00:FS00:0xa7:Completed 122500 out of 250000 steps (49%)
02:02:36:WU02:FS01:0x21:Completed 2550000 out of 5000000 steps (51%)
02:03:38:WU00:FS00:0xa7:Completed 125000 out of 250000 steps (50%)
02:04:18:WU02:FS01:0x21:Completed 2600000 out of 5000000 steps (52%)
02:06:01:WU02:FS01:0x21:Completed 2650000 out of 5000000 steps (53%)
02:06:14:WU00:FS00:0xa7:Completed 127500 out of 250000 steps (51%)
02:07:44:WU02:FS01:0x21:Completed 2700000 out of 5000000 steps (54%)
02:08:53:WU00:FS00:0xa7:Completed 130000 out of 250000 steps (52%)
02:09:27:WU02:FS01:0x21:Completed 2750000 out of 5000000 steps (55%)
02:09:35:WARNING:FS01:Size of positions 5845 does not match topology 5844
02:11:13:WU02:FS01:0x21:Completed 2800000 out of 5000000 steps (56%)
02:11:30:WU00:FS00:0xa7:Completed 132500 out of 250000 steps (53%)
02:12:56:WU02:FS01:0x21:Completed 2850000 out of 5000000 steps (57%)
02:14:06:WU00:FS00:0xa7:Completed 135000 out of 250000 steps (54%)
02:14:39:WU02:FS01:0x21:Completed 2900000 out of 5000000 steps (58%)
02:16:22:WU02:FS01:0x21:Completed 2950000 out of 5000000 steps (59%)
02:16:43:WU00:FS00:0xa7:Completed 137500 out of 250000 steps (55%)
02:18:05:WU02:FS01:0x21:Completed 3000000 out of 5000000 steps (60%)
02:18:12:WARNING:FS01:Size of positions 5845 does not match topology 5844
02:19:19:WU00:FS00:0xa7:Completed 140000 out of 250000 steps (56%)
02:19:50:WU02:FS01:0x21:Completed 3050000 out of 5000000 steps (61%)
02:21:33:WU02:FS01:0x21:Completed 3100000 out of 5000000 steps (62%)
02:21:55:WU00:FS00:0xa7:Completed 142500 out of 250000 steps (57%)
02:23:16:WU02:FS01:0x21:Completed 3150000 out of 5000000 steps (63%)
02:24:34:WU00:FS00:0xa7:Completed 145000 out of 250000 steps (58%)
02:24:59:WU02:FS01:0x21:Completed 3200000 out of 5000000 steps (64%)
02:26:42:WU02:FS01:0x21:Completed 3250000 out of 5000000 steps (65%)
02:26:50:WARNING:FS01:Size of positions 5845 does not match topology 5844
02:27:10:WU00:FS00:0xa7:Completed 147500 out of 250000 steps (59%)
02:28:28:WU02:FS01:0x21:Completed 3300000 out of 5000000 steps (66%)
02:29:47:WU00:FS00:0xa7:Completed 150000 out of 250000 steps (60%)
02:30:11:WU02:FS01:0x21:Completed 3350000 out of 5000000 steps (67%)
02:31:53:WU02:FS01:0x21:Completed 3400000 out of 5000000 steps (68%)
02:32:24:WU00:FS00:0xa7:Completed 152500 out of 250000 steps (61%)
02:33:36:WU02:FS01:0x21:Completed 3450000 out of 5000000 steps (69%)
02:35:00:WU00:FS00:0xa7:Completed 155000 out of 250000 steps (62%)
02:35:19:WU02:FS01:0x21:Completed 3500000 out of 5000000 steps (70%)
02:35:27:WARNING:FS01:Size of positions 5845 does not match topology 5844
02:37:05:WU02:FS01:0x21:Completed 3550000 out of 5000000 steps (71%)
02:37:38:WU00:FS00:0xa7:Completed 157500 out of 250000 steps (63%)
02:38:48:WU02:FS01:0x21:Completed 3600000 out of 5000000 steps (72%)
02:40:15:WU00:FS00:0xa7:Completed 160000 out of 250000 steps (64%)
02:40:31:WU02:FS01:0x21:Completed 3650000 out of 5000000 steps (73%)
02:42:14:WU02:FS01:0x21:Completed 3700000 out of 5000000 steps (74%)
02:42:51:WU00:FS00:0xa7:Completed 162500 out of 250000 steps (65%)
02:43:57:WU02:FS01:0x21:Completed 3750000 out of 5000000 steps (75%)
02:44:04:WARNING:FS01:Size of positions 5845 does not match topology 5844
02:45:28:WU00:FS00:0xa7:Completed 165000 out of 250000 steps (66%)
02:45:42:WU02:FS01:0x21:Completed 3800000 out of 5000000 steps (76%)
02:47:25:WU02:FS01:0x21:Completed 3850000 out of 5000000 steps (77%)
02:48:05:WU00:FS00:0xa7:Completed 167500 out of 250000 steps (67%)
02:49:08:WU02:FS01:0x21:Completed 3900000 out of 5000000 steps (78%)
02:50:41:WU00:FS00:0xa7:Completed 170000 out of 250000 steps (68%)
02:50:51:WU02:FS01:0x21:Completed 3950000 out of 5000000 steps (79%)
02:52:34:WU02:FS01:0x21:Completed 4000000 out of 5000000 steps (80%)
02:52:42:WARNING:FS01:Size of positions 5845 does not match topology 5844
02:53:20:WU00:FS00:0xa7:Completed 172500 out of 250000 steps (69%)
02:54:20:WU02:FS01:0x21:Completed 4050000 out of 5000000 steps (81%)
02:55:56:WU00:FS00:0xa7:Completed 175000 out of 250000 steps (70%)
02:56:03:WU02:FS01:0x21:Completed 4100000 out of 5000000 steps (82%)
02:57:46:WU02:FS01:0x21:Completed 4150000 out of 5000000 steps (83%)
02:58:33:WU00:FS00:0xa7:Completed 177500 out of 250000 steps (71%)
02:59:29:WU02:FS01:0x21:Completed 4200000 out of 5000000 steps (84%)
03:00:18:WU02:FS01:0x21:ERROR:exception: Error downloading array energyBuffer: clEnqueueReadBuffer (-5)
03:00:18:WU02:FS01:0x21:Saving result file logfile_01.txt
03:00:18:WU02:FS01:0x21:Saving result file log.txt
03:00:18:WU02:FS01:0x21:Folding@home Core Shutdown: BAD_WORK_UNIT
03:01:09:WU00:FS00:0xa7:Completed 180000 out of 250000 steps (72%)
03:03:41:WU00:FS00:0xa7:Completed 182500 out of 250000 steps (73%)
03:06:13:WU00:FS00:0xa7:Completed 185000 out of 250000 steps (74%)
03:08:47:WU00:FS00:0xa7:Completed 187500 out of 250000 steps (75%)
03:11:19:WU00:FS00:0xa7:Completed 190000 out of 250000 steps (76%)
03:13:52:WU00:FS00:0xa7:Completed 192500 out of 250000 steps (77%)
03:16:24:WU00:FS00:0xa7:Completed 195000 out of 250000 steps (78%)
03:18:56:WU00:FS00:0xa7:Completed 197500 out of 250000 steps (79%)
03:21:28:WU00:FS00:0xa7:Completed 200000 out of 250000 steps (80%)
03:24:02:WU00:FS00:0xa7:Completed 202500 out of 250000 steps (81%)
03:26:33:WU00:FS00:0xa7:Completed 205000 out of 250000 steps (82%)
03:29:06:WU00:FS00:0xa7:Completed 207500 out of 250000 steps (83%)
03:31:37:WU00:FS00:0xa7:Completed 210000 out of 250000 steps (84%)
03:34:09:WU00:FS00:0xa7:Completed 212500 out of 250000 steps (85%)
03:36:41:WU00:FS00:0xa7:Completed 215000 out of 250000 steps (86%)
03:39:15:WU00:FS00:0xa7:Completed 217500 out of 250000 steps (87%)
03:41:47:WU00:FS00:0xa7:Completed 220000 out of 250000 steps (88%)
03:44:19:WU00:FS00:0xa7:Completed 222500 out of 250000 steps (89%)
03:46:51:WU00:FS00:0xa7:Completed 225000 out of 250000 steps (90%)
03:49:23:WU00:FS00:0xa7:Completed 227500 out of 250000 steps (91%)
03:51:55:WU00:FS00:0xa7:Completed 230000 out of 250000 steps (92%)

Code: Select all

*********************** Log Started 2018-11-11T03:54:55Z ***********************
03:54:55:************************* Folding@home Client *************************
03:54:55:        Website: https://foldingathome.org/
03:54:55:      Copyright: (c) 2009-2018 foldingathome.org
03:54:55:         Author: Joseph Coffland <joseph@cauldrondevelopment.com>
03:54:55:           Args: 
03:54:55:         Config: C:\Users\doune\AppData\Roaming\FAHClient\config.xml
03:54:55:******************************** Build ********************************
03:54:55:        Version: 7.5.1
03:54:55:           Date: May 11 2018
03:54:55:           Time: 13:06:32
03:54:55:     Repository: Git
03:54:55:       Revision: 4705bf53c635f88b8fe85af7675557e15d491ff0
03:54:55:         Branch: master
03:54:55:       Compiler: Visual C++ 2008
03:54:55:        Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
03:54:55:       Platform: win32 10
03:54:55:           Bits: 32
03:54:55:           Mode: Release
03:54:55:******************************* System ********************************
03:54:55:            CPU: Intel(R) Core(TM) i7-4930K CPU @ 3.40GHz
03:54:55:         CPU ID: GenuineIntel Family 6 Model 62 Stepping 4
03:54:55:           CPUs: 12
03:54:55:         Memory: 15.94GiB
03:54:55:    Free Memory: 13.99GiB
03:54:55:        Threads: WINDOWS_THREADS
03:54:55:     OS Version: 6.2
03:54:55:    Has Battery: false
03:54:55:     On Battery: false
03:54:55:     UTC Offset: -6
03:54:55:            PID: 10528
03:54:55:            CWD: C:\Users\doune\AppData\Roaming\FAHClient
03:54:55:             OS: Windows 10 Enterprise
03:54:55:        OS Arch: AMD64
03:54:55:           GPUs: 1
03:54:55:          GPU 0: Bus:1 Slot:0 Func:0 NVIDIA:7 TU106-400A-A1 [GeForce RTX 2070
03:54:55:                 Rev. A] M 6497
03:54:55:  CUDA Device 0: Platform:0 Device:0 Bus:1 Slot:0 Compute:7.5 Driver:10.0
03:54:55:OpenCL Device 0: Platform:0 Device:0 Bus:1 Slot:0 Compute:1.2 Driver:416.34
03:54:55:  Win32 Service: false
03:54:55:***********************************************************************
03:54:55:<config>
03:54:55:  <!-- Slot Control -->
03:54:55:  <power v='FULL'/>
03:54:55:
03:54:55:  <!-- User Information -->
03:54:55:  <passkey v='********************************'/>
03:54:55:  <team v='233892'/>
03:54:55:  <user v='morphal'/>
03:54:55:
03:54:55:  <!-- Folding Slots -->
03:54:55:  <slot id='0' type='CPU'/>
03:54:55:  <slot id='1' type='GPU'/>
03:54:55:</config>
03:54:55:Trying to access database...
03:54:55:Successfully acquired database lock
03:54:55:Enabled folding slot 00: READY cpu:11
03:54:55:Enabled folding slot 01: READY gpu:0:TU106-400A-A1 [GeForce RTX 2070 Rev. A] M 6497
03:54:55:WU02:FS01:Starting
03:54:55:WU02:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\Users\doune\AppData\Roaming\FAHClient\cores/cores.foldingathome.org/Win32/AMD64/NVIDIA/Fermi/Core_21.fah/FahCore_21.exe -dir 02 -suffix 01 -version 705 -lifeline 10528 -checkpoint 15 -gpu-vendor nvidia -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu 0
03:54:55:WU02:FS01:Started FahCore on PID 11184
03:54:55:WU02:FS01:Core PID:11224
03:54:55:WU02:FS01:FahCore 0x21 started
03:54:55:WU00:FS00:Starting
03:54:55:WARNING:WU00:FS00:AS lowered CPUs from 11 to 10
03:54:55:WU00:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\Users\doune\AppData\Roaming\FAHClient\cores/cores.foldingathome.org/Win32/AMD64/AVX/Core_a7.fah/FahCore_a7.exe -dir 00 -suffix 01 -version 705 -lifeline 10528 -checkpoint 15 -np 10
03:54:55:WU00:FS00:Started FahCore on PID 10268
03:54:56:WU02:FS01:0x21:*********************** Log Started 2018-11-11T03:54:55Z ***********************
03:54:56:WU02:FS01:0x21:Project: 11718 (Run 0, Clone 871, Gen 155)
03:54:56:WU02:FS01:0x21:Unit: 0x000000d28ca304e75b846ae0340f90c4
03:54:56:WU02:FS01:0x21:CPU: 0x00000000000000000000000000000000
03:54:56:WU02:FS01:0x21:Machine: 1
03:54:56:WU02:FS01:0x21:Reading tar file core.xml
03:54:56:WU02:FS01:0x21:Reading tar file integrator.xml
03:54:56:WU02:FS01:0x21:Reading tar file state.xml
03:54:56:WU02:FS01:0x21:Reading tar file system.xml
03:54:56:WU00:FS00:Core PID:10660
03:54:56:WU00:FS00:FahCore 0xa7 started
03:54:56:WU02:FS01:0x21:Digital signatures verified
03:54:56:WU02:FS01:0x21:Folding@home GPU Core21 Folding@home Core
03:54:56:WU02:FS01:0x21:Version 0.0.18
03:54:56:WARNING:FS01:Size of positions 5845 does not match topology 5844
03:54:56:WARNING:FS01:Size of positions 5845 does not match topology 5844
03:54:56:WARNING:FS01:Size of positions 5845 does not match topology 5844
03:54:57:WU00:FS00:0xa7:*********************** Log Started 2018-11-11T03:54:56Z ***********************
03:54:57:WU00:FS00:0xa7:************************** Gromacs Folding@home Core ***************************
03:54:57:WU00:FS00:0xa7:       Type: 0xa7
03:54:57:WU00:FS00:0xa7:       Core: Gromacs
03:54:57:WU00:FS00:0xa7:    Website: https://foldingathome.org/
03:54:57:WARNING:FS01:Size of positions 5845 does not match topology 5844
03:54:57:WU00:FS00:0xa7:  Copyright: (c) 2009-2018 foldingathome.org
03:54:57:WU00:FS00:0xa7:     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
03:54:57:WU00:FS00:0xa7:       Args: -dir 00 -suffix 01 -version 705 -lifeline 10268 -checkpoint 15 -np
03:54:57:WU00:FS00:0xa7:             10
03:54:57:WU00:FS00:0xa7:     Config: <none>
03:54:57:WU00:FS00:0xa7:************************************ Build *************************************
03:54:57:WU00:FS00:0xa7:    Version: 0.0.17
03:54:57:WU00:FS00:0xa7:       Date: Apr 27 2018
03:54:57:WU00:FS00:0xa7:       Time: 16:19:36
03:54:57:WU00:FS00:0xa7: Repository: Git
03:54:57:WU00:FS00:0xa7:   Revision: 21359963583d09ec2063ef946399441c4df4ccd7
03:54:57:WU00:FS00:0xa7:     Branch: master
03:54:57:WU00:FS00:0xa7:   Compiler: Visual C++ 2008
03:54:57:WU00:FS00:0xa7:    Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
03:54:57:WU00:FS00:0xa7:   Platform: win32 10
03:54:57:WU00:FS00:0xa7:       Bits: 64
03:54:57:WU00:FS00:0xa7:       Mode: Release
03:54:57:WU00:FS00:0xa7:       SIMD: avx_256
03:54:57:WU00:FS00:0xa7:************************************ System ************************************
03:54:57:WU00:FS00:0xa7:        CPU: Unknown
03:54:57:WU00:FS00:0xa7:     CPU ID: 
03:54:57:WU00:FS00:0xa7:       CPUs: 12
03:54:57:WU00:FS00:0xa7:     Memory: 15.94GiB
03:54:57:WU00:FS00:0xa7:Free Memory: 13.80GiB
03:54:57:WU00:FS00:0xa7:    Threads: WINDOWS_THREADS
03:54:57:WU00:FS00:0xa7: OS Version: 6.2
03:54:57:WU00:FS00:0xa7:Has Battery: false
03:54:57:WU00:FS00:0xa7: On Battery: false
03:54:57:WU00:FS00:0xa7: UTC Offset: -6
03:54:57:WU00:FS00:0xa7:        PID: 10660
03:54:57:WU00:FS00:0xa7:        CWD: C:\Users\doune\AppData\Roaming\FAHClient\work
03:54:57:WU00:FS00:0xa7:         OS: Windows 10 Pro
03:54:57:WU00:FS00:0xa7:    OS Arch: AMD64
03:54:57:WU00:FS00:0xa7:********************************************************************************
03:54:57:WU00:FS00:0xa7:Project: 13809 (Run 0, Clone 1548, Gen 232)
03:54:57:WU00:FS00:0xa7:Unit: 0x0000012980fccb025a919051ecb1e8f5
03:54:57:WU00:FS00:0xa7:Digital signatures verified
03:54:57:WU00:FS00:0xa7:Calling: mdrun -s frame232.tpr -o frame232.trr -x frame232.xtc -cpi state.cpt -cpt 15 -nt 10
03:54:57:WARNING:FS01:Size of positions 5845 does not match topology 5844
03:54:57:WARNING:FS01:Size of positions 5845 does not match topology 5844
03:54:57:WU00:FS00:0xa7:Steps: first=58000000 total=250000
03:54:57:WARNING:FS01:Size of positions 5845 does not match topology 5844
03:54:57:WARNING:FS01:Size of positions 5845 does not match topology 5844
03:54:57:WARNING:FS01:Size of positions 5845 does not match topology 5844
03:54:57:WARNING:FS01:Size of positions 5845 does not match topology 5844
03:54:57:WARNING:FS01:Size of positions 5845 does not match topology 5844
03:54:57:WARNING:FS01:Size of positions 5845 does not match topology 5844
03:54:57:WARNING:FS01:Size of positions 5845 does not match topology 5844
03:54:57:WARNING:FS01:Size of positions 5845 does not match topology 5844
03:54:57:WARNING:FS01:Size of positions 5845 does not match topology 5844
03:54:57:WARNING:FS01:Size of positions 5845 does not match topology 5844
03:54:59:WU00:FS00:0xa7:Completed 230641 out of 250000 steps (92%)
03:55:09:WU02:FS01:0x21:Completed 0 out of 5000000 steps (0%)
03:55:09:WU02:FS01:0x21:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
03:56:52:WU02:FS01:0x21:Completed 50000 out of 5000000 steps (1%)
03:57:05:WU00:FS00:0xa7:Completed 232500 out of 250000 steps (93%)
03:58:37:WU02:FS01:0x21:Completed 100000 out of 5000000 steps (2%)
03:59:48:WU00:FS00:0xa7:Completed 235000 out of 250000 steps (94%)
04:00:23:WU02:FS01:0x21:Completed 150000 out of 5000000 steps (3%)
04:02:08:WU02:FS01:0x21:Completed 200000 out of 5000000 steps (4%)
04:02:35:WU00:FS00:0xa7:Completed 237500 out of 250000 steps (95%)
04:03:53:WU02:FS01:0x21:Completed 250000 out of 5000000 steps (5%)
04:04:42:WU02:FS01:0x21:ERROR:exception: Error downloading array energyBuffer: clEnqueueReadBuffer (-5)
04:04:43:WU02:FS01:0x21:Saving result file logfile_01.txt
04:04:43:WU02:FS01:0x21:Saving result file log.txt
04:04:43:WU02:FS01:0x21:Folding@home Core Shutdown: BAD_WORK_UNIT
04:04:43:WARNING:WU02:FS01:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
04:04:43:WU02:FS01:Sending unit results: id:02 state:SEND error:FAULTY project:11718 run:0 clone:871 gen:155 core:0x21 unit:0x000000d28ca304e75b846ae0340f90c4
04:04:43:WU02:FS01:Uploading 8.00KiB to 140.163.4.231
04:04:43:WU02:FS01:Connecting to 140.163.4.231:8080
04:04:44:WU01:FS01:Connecting to 65.254.110.245:8080
04:04:44:WU02:FS01:Upload complete
04:04:44:WU02:FS01:Server responded WORK_ACK (400)
04:04:44:WU02:FS01:Cleaning up
04:04:44:WU01:FS01:Assigned to work server 140.163.4.231
04:04:44:WU01:FS01:Requesting new work unit for slot 01: READY gpu:0:TU106-400A-A1 [GeForce RTX 2070 Rev. A] M 6497 from 140.163.4.231
04:04:44:WU01:FS01:Connecting to 140.163.4.231:8080
04:04:45:WU01:FS01:Downloading 25.27MiB
04:04:51:WU01:FS01:Download 71.72%
04:04:51:WU01:FS01:Download complete
04:04:52:WU01:FS01:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:11718 run:0 clone:2139 gen:30 core:0x21 unit:0x0000002a8ca304e75bbce89e62e29667
04:04:52:WU01:FS01:Starting
04:04:52:WU01:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\Users\doune\AppData\Roaming\FAHClient\cores/cores.foldingathome.org/Win32/AMD64/NVIDIA/Fermi/Core_21.fah/FahCore_21.exe -dir 01 -suffix 01 -version 705 -lifeline 10528 -checkpoint 15 -gpu-vendor nvidia -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu 0
04:04:52:WU01:FS01:Started FahCore on PID 11984
04:04:52:WU01:FS01:Core PID:7576
04:04:52:WU01:FS01:FahCore 0x21 started
04:05:02:WARNING:WU01:FS01:FahCore returned an unknown error code which probably indicates that it crashed
04:05:02:WARNING:WU01:FS01:FahCore returned: UNKNOWN_ENUM (-1073741515 = 0xc0000135)
04:05:02:WU01:FS01:Starting
04:05:02:WU01:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\Users\doune\AppData\Roaming\FAHClient\cores/cores.foldingathome.org/Win32/AMD64/NVIDIA/Fermi/Core_21.fah/FahCore_21.exe -dir 01 -suffix 01 -version 705 -lifeline 10528 -checkpoint 15 -gpu-vendor nvidia -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu 0
04:05:02:WU01:FS01:Started FahCore on PID 9068
04:05:03:WU01:FS01:Core PID:9652
04:05:03:WU01:FS01:FahCore 0x21 started
04:05:06:WARNING:WU01:FS01:FahCore returned an unknown error code which probably indicates that it crashed
04:05:06:WARNING:WU01:FS01:FahCore returned: UNKNOWN_ENUM (-1073741515 = 0xc0000135)
04:05:18:WU00:FS00:0xa7:Completed 240000 out of 250000 steps (96%)
04:05:19:FS00:Paused
04:05:19:FS01:Paused
04:05:20:FS00:Shutting core down
04:05:20:WU00:FS00:0xa7:WARNING:Console control signal 1 on PID 10660
04:05:20:WU00:FS00:0xa7:Exiting, please wait. . .
04:05:23:WU00:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
04:05:30:Saving configuration to config.xml
04:05:30:<config>
04:05:30:  <!-- Slot Control -->
04:05:30:  <power v='FULL'/>
04:05:30:
04:05:30:  <!-- User Information -->
04:05:30:  <passkey v='********************************'/>
04:05:30:  <team v='233892'/>
04:05:30:  <user v='morphal'/>
04:05:30:
04:05:30:  <!-- Folding Slots -->
04:05:30:  <slot id='0' type='CPU'>
04:05:30:    <paused v='true'/>
04:05:30:  </slot>
04:05:30:  <slot id='1' type='GPU'>
04:05:30:    <paused v='true'/>
04:05:30:  </slot>
04:05:30:</config>
04:05:30:Clean exit
Looking back through Event Viewer, I see a repeating warning every 3-4 seconds. There are so many of these that digging for any other warning is nearly impossible:

Code: Select all

Display driver nvlddmkm stopped responding and has successfully recovered.
I looked back to find when these warnings started and see nothing alarming at the beginning of them. In one example, there's an error dealing with Microsoft.Messaging around 12:29 and these driver warnings don't start until 3:38. There are thousands of these warnings but I do see the occasional error mixed in with them

Code: Select all

The server {1F87137D-0E7C-44D5-8C73-4EFFB68962F2} did not register with DCOM within the required timeout.
This general pattern repeats in Event Viewer until the error about an unexpected shutdown.

I've looked through the forum and I see some people had a similar problem a couple years ago but the suggestions were a little vague and ended abruptly when pointing to an even older thread about driver versions. Since the 2070 is new, I can't go back very far version-wise.
Top
bruce
Posts: 20824
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: freeze during folding

Post by bruce »

The code for nvlddmkm is a component of the NVidia drivers.

The FIRST thing I would recommend is that you download the latest driver package from Nvidia.com and install it. (FAH does not require the optional features so it's your choice whether to include them or not.)
I see you have CUDA Device Driver:10.0 and OpenCL Device Driver:416.34
Their site recommends Version: 416.81 - Release Date: Thu Nov 08, 2018
(My system is Linux with older GPUs, so I can't test your configuration. You'll need to ask them for help if it still doesn't work.)

FYI The following discussion was found on the NVidia support site:
https://forums.geforce.com/default/topi ... his-issue/
Top
toTOW
Site Moderator
Posts: 6349
Joined: Sun Dec 02, 2007 10:38 am
Location: Bordeaux, France
Contact:
Contact toTOW

Re: freeze during folding

Post by toTOW »

Yes, something failed, and the following message is typical of a NV driver reset :
03:00:18:WU02:FS01:0x21:ERROR:exception: Error downloading array energyBuffer: clEnqueueReadBuffer (-5)
There should be an event like the one you quoted at the same time in event viewer.

There could be many causes to this ... start with a driver update, then check that everything is fine with your hardware (GPU temperatures, 12V power source stability, ...). It would be good to test the GPU with some other loads (3D game or benchmark). You can also disable your CPU slot for a while to see if stability improves ...
Image

Folding@Home beta tester since 2002. Folding Forum moderator since July 2008.
Top
doune
Posts: 30
Joined: Fri Jul 13, 2018 11:37 pm

Re: freeze during folding

Post by doune »

I've already updated the drivers since the last failure so I suppose it's possible that the new OpenCL Device Driver magically fixes the issue and this post is fruitless. I paused folding while I did some research but I can start it again to see what happens. I suppose that's the first and most practical test.

I've played several games since I got the card and I've had no problems but I don't think any game is going generate nearly the type of load that the folding does. That might help determine if it's a hardware/voltage/temperature issue or an issue with the OpenCL driver... maybe?

Should it become necessary, is there a particularly helpful resource for how to test and determine stable values for the various settings and components?
Top
bruce
Posts: 20824
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: freeze during folding

Post by bruce »

Games probably won't load the GPU as much as FAH does. Benchmark programs tend to use more power and dissipate more heat (and FAH tends to be as challenging a workload as the well-known benchmarks)

Maybe boosting your fan profile will help, too, or making other changes to handle excess heat.
Top
Post Reply

Return to “V7.5.1 Public Release Windows/Linux/MacOS X [deprecated]”