GPU folding fails -- clEnqueueReadBuffer

If you're new to FAH and need help getting started or you have very basic questions, start here.

Moderators: Site Moderators, FAHC Science Team

Static Stripes
Posts: 6
Joined: Wed Oct 05, 2016 2:44 am

GPU folding fails -- clEnqueueReadBuffer

Post by Static Stripes »

Howdy, I've noticed that my GPU no longer folds. It will attempt it a few times before labeling it as "Failed" and ignoring it.
CPU still works fine.

Code: Select all

*********************** Log Started 2016-11-25T04:51:04Z ***********************
04:51:04:************************* Folding@home Client *************************
04:51:04:      Website: http://folding.stanford.edu/
04:51:04:    Copyright: (c) 2009-2014 Stanford University
04:51:04:       Author: Joseph Coffland <joseph@cauldrondevelopment.com>
04:51:04:         Args: --open-web-control
04:51:04:       Config: <none>
04:51:04:******************************** Build ********************************
04:51:04:      Version: 7.4.4
04:51:04:         Date: Mar 4 2014
04:51:04:         Time: 20:26:54
04:51:04:      SVN Rev: 4130
04:51:04:       Branch: fah/trunk/client
04:51:04:     Compiler: Intel(R) C++ MSVC 1500 mode 1200
04:51:04:      Options: /TP /nologo /EHa /Qdiag-disable:4297,4103,1786,279 /Ox -arch:SSE
04:51:04:               /QaxSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 /Qopenmp /Qrestrict /MT /Qmkl
04:51:04:     Platform: win32 XP
04:51:04:         Bits: 32
04:51:04:         Mode: Release
04:51:04:******************************* System ********************************
04:51:04:          CPU: Intel(R) Core(TM) i5-4670K CPU @ 3.40GHz
04:51:04:       CPU ID: GenuineIntel Family 6 Model 60 Stepping 3
04:51:04:         CPUs: 4
04:51:04:       Memory: 15.95GiB
04:51:04:  Free Memory: 12.05GiB
04:51:04:      Threads: WINDOWS_THREADS
04:51:04:   OS Version: 6.2
04:51:04:  Has Battery: false
04:51:04:   On Battery: false
04:51:04:   UTC Offset: -6
04:51:04:          PID: 11108
04:51:04:          CWD: C:/Users/Static/AppData/Roaming/FAHClient
04:51:04:           OS: Windows 10 Pro
04:51:04:      OS Arch: AMD64
04:51:04:         GPUs: 0
04:51:04:         CUDA: 5.2
04:51:04:  CUDA Driver: 8000
04:51:04:Win32 Service: false
04:51:04:***********************************************************************
04:51:04:<config>
04:51:04:  <!-- Folding Slots -->
04:51:04:</config>
04:51:04:Connecting to assign-GPU.stanford.edu:80
04:51:04:Updated GPUs.txt
04:51:04:Read GPUs.txt
04:51:04:Trying to access database...
04:51:04:Successfully acquired database lock
04:51:04:Enabled folding slot 00: PAUSED cpu:2 (not configured)
04:51:04:Enabled folding slot 01: PAUSED gpu:0:GM204 [GeForce GTX 970] (not configured)
04:51:08:16:127.0.0.1:New Web connection
04:52:05:Saving configuration to config.xml
04:52:05:<config>
04:52:05:  <!-- Folding Slots -->
04:52:05:  <slot id='0' type='CPU'/>
04:52:05:  <slot id='1' type='GPU'/>
04:52:05:</config>
04:52:05:Set client configured
04:52:05:WU00:FS00:Connecting to 171.67.108.45:8080
04:52:05:WU01:FS01:Connecting to 171.67.108.45:8080
04:52:05:WU00:FS00:Connecting to 171.67.108.45:8080
04:52:06:WU01:FS01:Connecting to 171.67.108.45:80
04:52:06:WU00:FS00:Assigned to work server 171.64.65.41
04:52:06:WU00:FS00:Requesting new work unit for slot 00: READY cpu:2 from 171.64.65.41
04:52:06:WU00:FS00:Connecting to 171.64.65.41:8080
04:52:06:WU00:FS00:Downloading 20.96MiB
04:52:09:WU01:FS01:Assigned to work server 140.163.4.243
04:52:09:WU01:FS01:Requesting new work unit for slot 01: READY gpu:0:GM204 [GeForce GTX 970] from 140.163.4.243
04:52:09:WU01:FS01:Connecting to 140.163.4.243:8080
04:52:10:WU01:FS01:Downloading 2.67MiB
04:52:10:WU00:FS00:Download complete
04:52:10:WU00:FS00:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:11920 run:96 clone:5 gen:77 core:0xa7 unit:0x0000005eab4041295809d17d9c432b19
04:52:10:WU00:FS00:Downloading core from http://web.stanford.edu/~pande/Win32/AMD64/AVX/Core_a7.fah
04:52:10:WU00:FS00:Connecting to web.stanford.edu:80
04:52:11:WU00:FS00:FahCore a7: Downloading 7.35MiB
04:52:12:WU01:FS01:Download complete
04:52:12:WU01:FS01:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:11707 run:37 clone:1 gen:65 core:0x21 unit:0x0000005a8ca304f357a9e55fd2f7c8cf
04:52:12:WU01:FS01:Downloading core from http://web.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_21.fah
04:52:12:WU01:FS01:Connecting to web.stanford.edu:80
04:52:13:WU01:FS01:FahCore 21: Downloading 3.47MiB
04:52:17:WU00:FS00:FahCore a7: 52.74%
04:52:19:WU01:FS01:FahCore 21: 100.00%
04:52:19:WU01:FS01:FahCore 21: Download complete
04:52:19:WU01:FS01:Valid core signature
04:52:21:WU01:FS01:Unpacked 11.81MiB to cores/web.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_21.fah/FahCore_21.exe
04:52:21:WU01:FS01:Starting
04:52:21:WU01:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Static/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_21.fah/FahCore_21.exe -dir 01 -suffix 01 -version 704 -lifeline 11108 -checkpoint 15 -gpu 0 -gpu-vendor nvidia
04:52:21:WU01:FS01:Started FahCore on PID 5316
04:52:21:WU01:FS01:Core PID:6776
04:52:21:WU01:FS01:FahCore 0x21 started
04:52:21:WU01:FS01:0x21:*********************** Log Started 2016-11-25T04:52:21Z ***********************
04:52:21:WU01:FS01:0x21:Project: 11707 (Run 37, Clone 1, Gen 65)
04:52:21:WU01:FS01:0x21:Unit: 0x0000005a8ca304f357a9e55fd2f7c8cf
04:52:21:WU01:FS01:0x21:CPU: 0x00000000000000000000000000000000
04:52:21:WU01:FS01:0x21:Machine: 1
04:52:21:WU01:FS01:0x21:Reading tar file core.xml
04:52:21:WU01:FS01:0x21:Reading tar file system.xml
04:52:21:WU01:FS01:0x21:Reading tar file integrator.xml
04:52:21:WU01:FS01:0x21:Reading tar file state.xml
04:52:21:WU01:FS01:0x21:Digital signatures verified
04:52:21:WU01:FS01:0x21:Folding@home GPU Core21 Folding@home Core
04:52:21:WU01:FS01:0x21:Version 0.0.17
04:52:22:WU00:FS00:FahCore a7: Download complete
04:52:23:WU00:FS00:Valid core signature
04:52:23:WU00:FS00:Unpacked 13.14MiB to cores/web.stanford.edu/~pande/Win32/AMD64/AVX/Core_a7.fah/FahCore_a7.exe
04:52:23:WU00:FS00:Unpacked 72.16KiB to cores/web.stanford.edu/~pande/Win32/AMD64/AVX/Core_a7.fah/libbz2-1.dll
04:52:23:WU00:FS00:Unpacked 2.17MiB to cores/web.stanford.edu/~pande/Win32/AMD64/AVX/Core_a7.fah/libeay32.dll
04:52:23:WU00:FS00:Unpacked 154.79KiB to cores/web.stanford.edu/~pande/Win32/AMD64/AVX/Core_a7.fah/libexpat-1.dll
04:52:23:WU00:FS00:Unpacked 2.11MiB to cores/web.stanford.edu/~pande/Win32/AMD64/AVX/Core_a7.fah/libfftw3f-3.dll
04:52:23:WU00:FS00:Unpacked 406.90KiB to cores/web.stanford.edu/~pande/Win32/AMD64/AVX/Core_a7.fah/ssleay32.dll
04:52:23:WU00:FS00:Unpacked 88.13KiB to cores/web.stanford.edu/~pande/Win32/AMD64/AVX/Core_a7.fah/zlib1.dll
04:52:23:WU00:FS00:Unpacked 81.28KiB to cores/web.stanford.edu/~pande/Win32/AMD64/AVX/Core_a7.fah/libgcc_s_seh-1.dll
04:52:23:WU00:FS00:Unpacked 55.64KiB to cores/web.stanford.edu/~pande/Win32/AMD64/AVX/Core_a7.fah/libwinpthread-1.dll
04:52:23:WU00:FS00:Unpacked 1.35MiB to cores/web.stanford.edu/~pande/Win32/AMD64/AVX/Core_a7.fah/libstdc++-6.dll
04:52:23:WU00:FS00:Starting
04:52:23:WU00:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Static/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/AVX/Core_a7.fah/FahCore_a7.exe -dir 00 -suffix 01 -version 704 -lifeline 11108 -checkpoint 15 -np 2
04:52:23:WU00:FS00:Started FahCore on PID 10848
04:52:23:WU00:FS00:Core PID:6416
04:52:23:WU00:FS00:FahCore 0xa7 started
04:52:24:WU00:FS00:0xa7:*********************** Log Started 2016-11-25T04:52:23Z ***********************
04:52:24:WU00:FS00:0xa7:************************** Gromacs Folding@home Core ***************************
04:52:24:WU00:FS00:0xa7:       Type: 0xa7
04:52:24:WU00:FS00:0xa7:       Core: Gromacs
04:52:24:WU00:FS00:0xa7:    Website: http://folding.stanford.edu/
04:52:24:WU00:FS00:0xa7:  Copyright: (c) 2009-2016 Stanford University
04:52:24:WU00:FS00:0xa7:     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
04:52:24:WU00:FS00:0xa7:       Args: -dir 00 -suffix 01 -version 704 -lifeline 10848 -checkpoint 15 -np
04:52:24:WU00:FS00:0xa7:             2
04:52:24:WU00:FS00:0xa7:     Config: <none>
04:52:24:WU00:FS00:0xa7:************************************ Build *************************************
04:52:24:WU00:FS00:0xa7:    Version: 0.0.11
04:52:24:WU00:FS00:0xa7:       Date: Sep 21 2016
04:52:24:WU00:FS00:0xa7:       Time: 01:43:48
04:52:24:WU00:FS00:0xa7: Repository: Git
04:52:24:WU00:FS00:0xa7:   Revision: 957bd90e68d95ddcf1594dc15ff6c64cc4555146
04:52:24:WU00:FS00:0xa7:     Branch: master
04:52:24:WU00:FS00:0xa7:   Compiler: GNU 4.2.1 Compatible Clang 3.9.0 (trunk 274080)
04:52:24:WU00:FS00:0xa7:    Options: -std=gnu++98 -O3 -funroll-loops -ffast-math -mfpmath=sse
04:52:24:WU00:FS00:0xa7:             -fno-unsafe-math-optimizations -msse2 -I/mingw64/include
04:52:24:WU00:FS00:0xa7:             -Wno-inconsistent-dllimport -Wno-parentheses-equality
04:52:24:WU00:FS00:0xa7:             -Wno-deprecated-register -Wno-unused-local-typedef
04:52:24:WU00:FS00:0xa7:   Platform: linux2 4.6.0-1-amd64
04:52:24:WU00:FS00:0xa7:       Bits: 64
04:52:24:WU00:FS00:0xa7:       Mode: Release
04:52:24:WU00:FS00:0xa7:       SIMD: avx_256
04:52:24:WU00:FS00:0xa7:************************************ System ************************************
04:52:24:WU00:FS00:0xa7:        CPU: Intel(R) Core(TM) i5-4670K CPU @ 3.40GHz
04:52:24:WU00:FS00:0xa7:     CPU ID: GenuineIntel Family 6 Model 60 Stepping 3
04:52:24:WU00:FS00:0xa7:       CPUs: 4
04:52:24:WU00:FS00:0xa7:     Memory: 15.95GiB
04:52:24:WU00:FS00:0xa7:Free Memory: 11.61GiB
04:52:24:WU00:FS00:0xa7:    Threads: WINDOWS_THREADS
04:52:24:WU00:FS00:0xa7: OS Version: 6.2
04:52:24:WU00:FS00:0xa7:Has Battery: false
04:52:24:WU00:FS00:0xa7: On Battery: false
04:52:24:WU00:FS00:0xa7: UTC Offset: -6
04:52:24:WU00:FS00:0xa7:        PID: 6416
04:52:24:WU00:FS00:0xa7:        CWD: C:\Users\Static\AppData\Roaming\FAHClient\work
04:52:24:WU00:FS00:0xa7:         OS: Windows 10 Pro
04:52:24:WU00:FS00:0xa7:    OS Arch: AMD64
04:52:24:WU00:FS00:0xa7:********************************************************************************
04:52:24:WU00:FS00:0xa7:Project: 11920 (Run 96, Clone 5, Gen 77)
04:52:24:WU00:FS00:0xa7:Unit: 0x0000005eab4041295809d17d9c432b19
04:52:24:WU00:FS00:0xa7:Reading tar file core.xml
04:52:24:WU00:FS00:0xa7:Reading tar file frame77.tpr
04:52:24:WU00:FS00:0xa7:Digital signatures verified
04:52:24:WU00:FS00:0xa7:Calling: mdrun -s frame77.tpr -o frame77.trr -cpt 15 -nt 2
04:52:24:WU00:FS00:0xa7:Steps: first=6160000 total=80000
04:52:24:WU01:FS01:0x21:ERROR:exception: Error downloading array interactionCount: clEnqueueReadBuffer (-5)
04:52:24:WU01:FS01:0x21:Saving result file logfile_01.txt
04:52:24:WU01:FS01:0x21:Saving result file log.txt
04:52:24:WU01:FS01:0x21:Folding@home Core Shutdown: BAD_WORK_UNIT
04:52:26:WU00:FS00:0xa7:Completed 1 out of 80000 steps (0%)
04:52:27:WARNING:WU01:FS01:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
04:52:27:WU01:FS01:Sending unit results: id:01 state:SEND error:FAULTY project:11707 run:37 clone:1 gen:65 core:0x21 unit:0x0000005a8ca304f357a9e55fd2f7c8cf
04:52:27:WU01:FS01:Uploading 2.39KiB to 140.163.4.243
04:52:27:WU01:FS01:Connecting to 140.163.4.243:8080
04:52:27:WU02:FS01:Connecting to 171.67.108.45:80
04:52:27:WU01:FS01:Upload complete
04:52:27:WU01:FS01:Server responded WORK_ACK (400)
04:52:27:WU01:FS01:Cleaning up
04:52:28:WU02:FS01:Assigned to work server 140.163.4.245
04:52:28:WU02:FS01:Requesting new work unit for slot 01: READY gpu:0:GM204 [GeForce GTX 970] from 140.163.4.245
04:52:28:WU02:FS01:Connecting to 140.163.4.245:8080
04:52:28:WU02:FS01:Downloading 6.04MiB
04:52:32:WU02:FS01:Download complete
04:52:32:WU02:FS01:Received Unit: id:02 state:DOWNLOAD error:NO_ERROR project:10493 run:5 clone:8 gen:247 core:0x21 unit:0x000001538ca304f555d6169ad0a69645
04:52:32:WU02:FS01:Starting
04:52:32:WU02:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Static/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_21.fah/FahCore_21.exe -dir 02 -suffix 01 -version 704 -lifeline 11108 -checkpoint 15 -gpu 0 -gpu-vendor nvidia
04:52:32:WU02:FS01:Started FahCore on PID 11252
04:52:32:WU02:FS01:Core PID:2144
04:52:32:WU02:FS01:FahCore 0x21 started
04:52:33:WU02:FS01:0x21:*********************** Log Started 2016-11-25T04:52:32Z ***********************
04:52:33:WU02:FS01:0x21:Project: 10493 (Run 5, Clone 8, Gen 247)
04:52:33:WU02:FS01:0x21:Unit: 0x000001538ca304f555d6169ad0a69645
04:52:33:WU02:FS01:0x21:CPU: 0x00000000000000000000000000000000
04:52:33:WU02:FS01:0x21:Machine: 1
04:52:33:WU02:FS01:0x21:Reading tar file core.xml
04:52:33:WU02:FS01:0x21:Reading tar file system.xml
04:52:33:WU02:FS01:0x21:Reading tar file integrator.xml
04:52:33:WU02:FS01:0x21:Reading tar file state.xml
04:52:33:WU02:FS01:0x21:Digital signatures verified
04:52:33:WU02:FS01:0x21:Folding@home GPU Core21 Folding@home Core
04:52:33:WU02:FS01:0x21:Version 0.0.17
04:52:41:WU02:FS01:0x21:ERROR:exception: Error downloading array interactionCount: clEnqueueReadBuffer (-5)
04:52:41:WU02:FS01:0x21:Saving result file logfile_01.txt
04:52:41:WU02:FS01:0x21:Saving result file log.txt
04:52:41:WU02:FS01:0x21:Folding@home Core Shutdown: BAD_WORK_UNIT
04:52:43:WARNING:WU02:FS01:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
04:52:43:WU02:FS01:Sending unit results: id:02 state:SEND error:FAULTY project:10493 run:5 clone:8 gen:247 core:0x21 unit:0x000001538ca304f555d6169ad0a69645
04:52:43:WU02:FS01:Uploading 2.38KiB to 140.163.4.245
04:52:43:WU02:FS01:Connecting to 140.163.4.245:8080
04:52:44:WU01:FS01:Connecting to 171.67.108.45:80
04:52:44:WU02:FS01:Upload complete
04:52:44:WU02:FS01:Server responded WORK_ACK (400)
04:52:44:WU02:FS01:Cleaning up
04:52:44:WU01:FS01:Assigned to work server 171.67.108.102
04:52:44:WU01:FS01:Requesting new work unit for slot 01: READY gpu:0:GM204 [GeForce GTX 970] from 171.67.108.102
04:52:44:WU01:FS01:Connecting to 171.67.108.102:8080
04:52:55:WU01:FS01:Downloading 6.86MiB
04:52:57:WU01:FS01:Download complete
04:52:58:WU01:FS01:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:13201 run:25 clone:6 gen:58 core:0x21 unit:0x00000032ab436c66577fedfe25fe5c24
04:52:58:WU01:FS01:Starting
04:52:58:WU01:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Static/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_21.fah/FahCore_21.exe -dir 01 -suffix 01 -version 704 -lifeline 11108 -checkpoint 15 -gpu 0 -gpu-vendor nvidia
04:52:58:WU01:FS01:Started FahCore on PID 5696
04:52:58:WU01:FS01:Core PID:10032
04:52:58:WU01:FS01:FahCore 0x21 started
04:52:58:WU01:FS01:0x21:*********************** Log Started 2016-11-25T04:52:58Z ***********************
04:52:58:WU01:FS01:0x21:Project: 13201 (Run 25, Clone 6, Gen 58)
04:52:58:WU01:FS01:0x21:Unit: 0x00000032ab436c66577fedfe25fe5c24
04:52:58:WU01:FS01:0x21:CPU: 0x00000000000000000000000000000000
04:52:58:WU01:FS01:0x21:Machine: 1
04:52:58:WU01:FS01:0x21:Reading tar file core.xml
04:52:58:WU01:FS01:0x21:Reading tar file integrator.xml
04:52:58:WU01:FS01:0x21:Reading tar file state.xml
04:52:59:WU01:FS01:0x21:Reading tar file system.xml
04:53:00:WU01:FS01:0x21:Digital signatures verified
04:53:00:WU01:FS01:0x21:Folding@home GPU Core21 Folding@home Core
04:53:00:WU01:FS01:0x21:Version 0.0.17
04:53:44:WU01:FS01:0x21:ERROR:exception: Error downloading array interactionCount: clEnqueueReadBuffer (-5)
04:53:44:WU01:FS01:0x21:Saving result file logfile_01.txt
04:53:44:WU01:FS01:0x21:Saving result file log.txt
04:53:44:WU01:FS01:0x21:Folding@home Core Shutdown: BAD_WORK_UNIT
04:53:47:WARNING:WU01:FS01:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
04:53:47:WU01:FS01:Sending unit results: id:01 state:SEND error:FAULTY project:13201 run:25 clone:6 gen:58 core:0x21 unit:0x00000032ab436c66577fedfe25fe5c24
04:53:47:WU01:FS01:Uploading 2.33KiB to 171.67.108.102
04:53:47:WU01:FS01:Connecting to 171.67.108.102:8080
04:53:47:WU01:FS01:Upload complete
04:53:47:WU01:FS01:Server responded WORK_ACK (400)
04:53:47:WU01:FS01:Cleaning up
04:53:47:WU02:FS01:Connecting to 171.67.108.45:80
04:53:48:WU02:FS01:Assigned to work server 140.163.4.231
04:53:48:WU02:FS01:Requesting new work unit for slot 01: READY gpu:0:GM204 [GeForce GTX 970] from 140.163.4.231
04:53:48:WU02:FS01:Connecting to 140.163.4.231:8080
04:53:48:WU02:FS01:Downloading 16.73MiB
04:53:54:WU02:FS01:Download 60.16%
04:53:55:WU02:FS01:Download complete
04:53:55:WU02:FS01:Received Unit: id:02 state:DOWNLOAD error:NO_ERROR project:11710 run:0 clone:149 gen:48 core:0x21 unit:0x000000528ca304e75814df2f5e777501
04:53:55:WU02:FS01:Starting
04:53:55:WU02:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Static/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_21.fah/FahCore_21.exe -dir 02 -suffix 01 -version 704 -lifeline 11108 -checkpoint 15 -gpu 0 -gpu-vendor nvidia
04:53:55:WU02:FS01:Started FahCore on PID 5844
04:53:55:WU02:FS01:Core PID:1356
04:53:55:WU02:FS01:FahCore 0x21 started
04:53:56:WU02:FS01:0x21:*********************** Log Started 2016-11-25T04:53:56Z ***********************
04:53:56:WU02:FS01:0x21:Project: 11710 (Run 0, Clone 149, Gen 48)
04:53:56:WU02:FS01:0x21:Unit: 0x000000528ca304e75814df2f5e777501
04:53:56:WU02:FS01:0x21:CPU: 0x00000000000000000000000000000000
04:53:56:WU02:FS01:0x21:Machine: 1
04:53:56:WU02:FS01:0x21:Reading tar file core.xml
04:53:56:WU02:FS01:0x21:Reading tar file integrator.xml
04:53:56:WU02:FS01:0x21:Reading tar file state.xml
04:53:56:WU02:FS01:0x21:Reading tar file system.xml
04:53:56:WU02:FS01:0x21:Digital signatures verified
04:53:56:WU02:FS01:0x21:Folding@home GPU Core21 Folding@home Core
04:53:56:WU02:FS01:0x21:Version 0.0.17
04:54:00:WU02:FS01:0x21:ERROR:exception: Error downloading array interactionCount: clEnqueueReadBuffer (-5)
04:54:00:WU02:FS01:0x21:Saving result file logfile_01.txt
04:54:00:WU02:FS01:0x21:Saving result file log.txt
04:54:00:WU02:FS01:0x21:Folding@home Core Shutdown: BAD_WORK_UNIT
04:54:03:WARNING:WU02:FS01:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
04:54:03:WU02:FS01:Sending unit results: id:02 state:SEND error:FAULTY project:11710 run:0 clone:149 gen:48 core:0x21 unit:0x000000528ca304e75814df2f5e777501
04:54:03:WU02:FS01:Uploading 7.00KiB to 140.163.4.231
04:54:03:WU02:FS01:Connecting to 140.163.4.231:8080
04:54:03:WU01:FS01:Connecting to 171.67.108.45:80
04:54:03:WU02:FS01:Upload complete
04:54:03:WU02:FS01:Server responded WORK_ACK (400)
04:54:03:WU02:FS01:Cleaning up
04:54:04:WU01:FS01:Assigned to work server 171.67.108.105
04:54:04:WU01:FS01:Requesting new work unit for slot 01: READY gpu:0:GM204 [GeForce GTX 970] from 171.67.108.105
04:54:04:WU01:FS01:Connecting to 171.67.108.105:8080
04:54:04:WU01:FS01:Downloading 20.31MiB
04:54:08:WU01:FS01:Download complete
04:54:08:WU01:FS01:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:9176 run:6 clone:16 gen:161 core:0x21 unit:0x000000dfab436c6957b24c287c4839ab
04:54:08:WU01:FS01:Starting
04:54:08:WU01:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Static/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_21.fah/FahCore_21.exe -dir 01 -suffix 01 -version 704 -lifeline 11108 -checkpoint 15 -gpu 0 -gpu-vendor nvidia
04:54:08:WU01:FS01:Started FahCore on PID 8332
04:54:08:WU01:FS01:Core PID:9840
04:54:08:WU01:FS01:FahCore 0x21 started
04:54:09:WU01:FS01:0x21:*********************** Log Started 2016-11-25T04:54:08Z ***********************
04:54:09:WU01:FS01:0x21:Project: 9176 (Run 6, Clone 16, Gen 161)
04:54:09:WU01:FS01:0x21:Unit: 0x000000dfab436c6957b24c287c4839ab
04:54:09:WU01:FS01:0x21:CPU: 0x00000000000000000000000000000000
04:54:09:WU01:FS01:0x21:Machine: 1
04:54:09:WU01:FS01:0x21:Reading tar file core.xml
04:54:09:WU01:FS01:0x21:Reading tar file integrator.xml
04:54:09:WU01:FS01:0x21:Reading tar file state.xml
04:54:09:WU01:FS01:0x21:Reading tar file system.xml
04:54:09:WU01:FS01:0x21:Digital signatures verified
04:54:09:WU01:FS01:0x21:Folding@home GPU Core21 Folding@home Core
04:54:09:WU01:FS01:0x21:Version 0.0.17
04:54:09:WU00:FS00:0xa7:Completed 800 out of 80000 steps (1%)
04:54:14:WU01:FS01:0x21:ERROR:exception: Error downloading array interactionCount: clEnqueueReadBuffer (-5)
04:54:14:WU01:FS01:0x21:Saving result file logfile_01.txt
04:54:14:WU01:FS01:0x21:Saving result file log.txt
04:54:14:WU01:FS01:0x21:Folding@home Core Shutdown: BAD_WORK_UNIT
04:54:17:WARNING:WU01:FS01:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
04:54:17:WU01:FS01:Sending unit results: id:01 state:SEND error:FAULTY project:9176 run:6 clone:16 gen:161 core:0x21 unit:0x000000dfab436c6957b24c287c4839ab
04:54:17:WU01:FS01:Uploading 7.00KiB to 171.67.108.105
04:54:17:WU01:FS01:Connecting to 171.67.108.105:8080
04:54:17:WU01:FS01:Upload complete
04:54:17:WU01:FS01:Server responded WORK_ACK (400)
04:54:17:WU01:FS01:Cleaning up
04:54:17:WU02:FS01:Connecting to 171.67.108.45:80
04:54:17:WU02:FS01:Assigned to work server 171.67.108.104
04:54:17:WU02:FS01:Requesting new work unit for slot 01: READY gpu:0:GM204 [GeForce GTX 970] from 171.67.108.104
04:54:17:WU02:FS01:Connecting to 171.67.108.104:8080
04:54:18:WU02:FS01:Downloading 80.24MiB
04:54:24:WU02:FS01:Download 32.64%
04:54:30:WU02:FS01:Download 69.24%
04:54:35:WU02:FS01:Download complete
04:54:35:WU02:FS01:Received Unit: id:02 state:DOWNLOAD error:NO_ERROR project:9211 run:16 clone:44 gen:2 core:0x21 unit:0x0000000aab436c685796c0f51b5e4566
04:54:35:WU02:FS01:Starting
04:54:35:WU02:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Static/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_21.fah/FahCore_21.exe -dir 02 -suffix 01 -version 704 -lifeline 11108 -checkpoint 15 -gpu 0 -gpu-vendor nvidia
04:54:35:WU02:FS01:Started FahCore on PID 9692
04:54:35:WU02:FS01:Core PID:10832
04:54:35:WU02:FS01:FahCore 0x21 started
04:54:36:WU02:FS01:0x21:*********************** Log Started 2016-11-25T04:54:35Z ***********************
04:54:36:WU02:FS01:0x21:Project: 9211 (Run 16, Clone 44, Gen 2)
04:54:36:WU02:FS01:0x21:Unit: 0x0000000aab436c685796c0f51b5e4566
04:54:36:WU02:FS01:0x21:CPU: 0x00000000000000000000000000000000
04:54:36:WU02:FS01:0x21:Machine: 1
04:54:36:WU02:FS01:0x21:Reading tar file core.xml
04:54:36:WU02:FS01:0x21:Reading tar file integrator.xml
04:54:36:WU02:FS01:0x21:Reading tar file state.xml
04:54:36:WU02:FS01:0x21:Reading tar file system.xml
04:54:36:WU02:FS01:0x21:Digital signatures verified
04:54:36:WU02:FS01:0x21:Folding@home GPU Core21 Folding@home Core
04:54:36:WU02:FS01:0x21:Version 0.0.17
04:54:57:WU02:FS01:0x21:ERROR:exception: Error downloading array interactionCount: clEnqueueReadBuffer (-5)
04:54:57:WU02:FS01:0x21:Saving result file logfile_01.txt
04:54:57:WU02:FS01:0x21:Saving result file log.txt
04:54:58:WU02:FS01:0x21:Folding@home Core Shutdown: BAD_WORK_UNIT
04:55:00:WARNING:WU02:FS01:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
04:55:00:WU02:FS01:Sending unit results: id:02 state:SEND error:FAULTY project:9211 run:16 clone:44 gen:2 core:0x21 unit:0x0000000aab436c685796c0f51b5e4566
04:55:00:WU02:FS01:Uploading 7.00KiB to 171.67.108.104
04:55:00:WU02:FS01:Connecting to 171.67.108.104:8080
04:55:00:WU02:FS01:Upload complete
04:55:00:WU02:FS01:Server responded WORK_ACK (400)
04:55:00:WU02:FS01:Cleaning up
04:55:00:WU01:FS01:Connecting to 171.67.108.45:80
04:55:01:WU01:FS01:Assigned to work server 171.64.65.84
04:55:01:WU01:FS01:Requesting new work unit for slot 01: READY gpu:0:GM204 [GeForce GTX 970] from 171.64.65.84
04:55:01:WU01:FS01:Connecting to 171.64.65.84:8080
04:55:01:WU01:FS01:Downloading 3.18MiB
04:55:05:WU01:FS01:Download complete
04:55:05:WU01:FS01:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:9192 run:1 clone:76 gen:107 core:0x21 unit:0x000000a5ab40415457cb2d58e3a54cac
04:55:05:WU01:FS01:Starting
04:55:05:WU01:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Static/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_21.fah/FahCore_21.exe -dir 01 -suffix 01 -version 704 -lifeline 11108 -checkpoint 15 -gpu 0 -gpu-vendor nvidia
04:55:05:WU01:FS01:Started FahCore on PID 8184
04:55:05:WU01:FS01:Core PID:9564
04:55:05:WU01:FS01:FahCore 0x21 started
04:55:06:WU01:FS01:0x21:*********************** Log Started 2016-11-25T04:55:05Z ***********************
04:55:06:WU01:FS01:0x21:Project: 9192 (Run 1, Clone 76, Gen 107)
04:55:06:WU01:FS01:0x21:Unit: 0x000000a5ab40415457cb2d58e3a54cac
04:55:06:WU01:FS01:0x21:CPU: 0x00000000000000000000000000000000
04:55:06:WU01:FS01:0x21:Machine: 1
04:55:06:WU01:FS01:0x21:Reading tar file core.xml
04:55:06:WU01:FS01:0x21:Reading tar file system.xml
04:55:06:WU01:FS01:0x21:Reading tar file integrator.xml
04:55:06:WU01:FS01:0x21:Reading tar file state.xml
04:55:06:WU01:FS01:0x21:Digital signatures verified
04:55:06:WU01:FS01:0x21:Folding@home GPU Core21 Folding@home Core
04:55:06:WU01:FS01:0x21:Version 0.0.17
04:55:10:WU01:FS01:0x21:ERROR:exception: Error downloading array interactionCount: clEnqueueReadBuffer (-5)
04:55:10:WU01:FS01:0x21:Saving result file logfile_01.txt
04:55:10:WU01:FS01:0x21:Saving result file log.txt
04:55:10:WU01:FS01:0x21:Folding@home Core Shutdown: BAD_WORK_UNIT
04:55:13:WARNING:WU01:FS01:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
04:55:13:WU01:FS01:Sending unit results: id:01 state:SEND error:FAULTY project:9192 run:1 clone:76 gen:107 core:0x21 unit:0x000000a5ab40415457cb2d58e3a54cac
04:55:13:WU01:FS01:Uploading 2.39KiB to 171.64.65.84
04:55:13:WU01:FS01:Connecting to 171.64.65.84:8080
04:55:13:WU02:FS01:Connecting to 171.67.108.45:80
04:55:13:WU01:FS01:Upload complete
04:55:13:WU01:FS01:Server responded WORK_ACK (400)
04:55:13:WU01:FS01:Cleaning up
04:55:14:WU02:FS01:Assigned to work server 171.67.108.105
04:55:14:WU02:FS01:Requesting new work unit for slot 01: READY gpu:0:GM204 [GeForce GTX 970] from 171.67.108.105
04:55:14:WU02:FS01:Connecting to 171.67.108.105:8080
04:55:14:WU02:FS01:Downloading 20.50MiB
04:55:20:WU02:FS01:Download 92.36%
04:55:20:WU02:FS01:Download complete
04:55:20:WU02:FS01:Received Unit: id:02 state:DOWNLOAD error:NO_ERROR project:9176 run:29 clone:2 gen:76 core:0x21 unit:0x0000006fab436c6957b24c299caa45d6
04:55:20:WU02:FS01:Starting
04:55:20:WU02:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Static/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_21.fah/FahCore_21.exe -dir 02 -suffix 01 -version 704 -lifeline 11108 -checkpoint 15 -gpu 0 -gpu-vendor nvidia
04:55:20:WU02:FS01:Started FahCore on PID 5316
04:55:20:WU02:FS01:Core PID:2096
04:55:20:WU02:FS01:FahCore 0x21 started
04:55:20:WU02:FS01:0x21:*********************** Log Started 2016-11-25T04:55:20Z ***********************
04:55:20:WU02:FS01:0x21:Project: 9176 (Run 29, Clone 2, Gen 76)
04:55:20:WU02:FS01:0x21:Unit: 0x0000006fab436c6957b24c299caa45d6
04:55:20:WU02:FS01:0x21:CPU: 0x00000000000000000000000000000000
04:55:20:WU02:FS01:0x21:Machine: 1
04:55:20:WU02:FS01:0x21:Reading tar file core.xml
04:55:20:WU02:FS01:0x21:Reading tar file integrator.xml
04:55:20:WU02:FS01:0x21:Reading tar file state.xml
04:55:20:WU02:FS01:0x21:Reading tar file system.xml
04:55:20:WU02:FS01:0x21:Digital signatures verified
04:55:20:WU02:FS01:0x21:Folding@home GPU Core21 Folding@home Core
04:55:20:WU02:FS01:0x21:Version 0.0.17
04:55:25:WU02:FS01:0x21:ERROR:exception: Error downloading array interactionCount: clEnqueueReadBuffer (-5)
04:55:25:WU02:FS01:0x21:Saving result file logfile_01.txt
04:55:25:WU02:FS01:0x21:Saving result file log.txt
04:55:25:WU02:FS01:0x21:Folding@home Core Shutdown: BAD_WORK_UNIT
04:55:27:WARNING:WU02:FS01:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
04:55:27:WU02:FS01:Sending unit results: id:02 state:SEND error:FAULTY project:9176 run:29 clone:2 gen:76 core:0x21 unit:0x0000006fab436c6957b24c299caa45d6
04:55:27:WU02:FS01:Uploading 7.00KiB to 171.67.108.105
04:55:27:WU02:FS01:Connecting to 171.67.108.105:8080
04:55:27:WU02:FS01:Upload complete
04:55:27:WU02:FS01:Server responded WORK_ACK (400)
04:55:28:WU02:FS01:Cleaning up
04:55:28:WU01:FS01:Connecting to 171.67.108.45:80
04:55:28:WU01:FS01:Assigned to work server 171.67.108.104
04:55:28:WU01:FS01:Requesting new work unit for slot 01: READY gpu:0:GM204 [GeForce GTX 970] from 171.67.108.104
04:55:28:WU01:FS01:Connecting to 171.67.108.104:8080
04:55:29:WU01:FS01:Downloading 80.33MiB
04:55:35:WU01:FS01:Download 29.95%
04:55:41:WU01:FS01:Download 65.20%
04:55:46:WU01:FS01:Download complete
04:55:46:WU01:FS01:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:9205 run:56 clone:37 gen:5 core:0x21 unit:0x0000000cab436c685796c0b51e898eef
04:55:46:WU01:FS01:Starting
04:55:46:WU01:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Static/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_21.fah/FahCore_21.exe -dir 01 -suffix 01 -version 704 -lifeline 11108 -checkpoint 15 -gpu 0 -gpu-vendor nvidia
04:55:46:WU01:FS01:Started FahCore on PID 11160
04:55:46:WU01:FS01:Core PID:8848
04:55:46:WU01:FS01:FahCore 0x21 started
04:55:47:WU01:FS01:0x21:*********************** Log Started 2016-11-25T04:55:46Z ***********************
04:55:47:WU01:FS01:0x21:Project: 9205 (Run 56, Clone 37, Gen 5)
04:55:47:WU01:FS01:0x21:Unit: 0x0000000cab436c685796c0b51e898eef
04:55:47:WU01:FS01:0x21:CPU: 0x00000000000000000000000000000000
04:55:47:WU01:FS01:0x21:Machine: 1
04:55:47:WU01:FS01:0x21:Reading tar file core.xml
04:55:47:WU01:FS01:0x21:Reading tar file integrator.xml
04:55:47:WU01:FS01:0x21:Reading tar file state.xml
04:55:47:WU01:FS01:0x21:Reading tar file system.xml
04:55:47:WU01:FS01:0x21:Digital signatures verified
04:55:47:WU01:FS01:0x21:Folding@home GPU Core21 Folding@home Core
04:55:47:WU01:FS01:0x21:Version 0.0.17
04:55:50:WU00:FS00:0xa7:Completed 1600 out of 80000 steps (2%)
04:56:09:Saving configuration to config.xml
04:56:09:<config>
04:56:09:  <!-- User Information -->
04:56:09:  <passkey v='********************************'/>
04:56:09:  <user v='Static_Stripes'/>
04:56:09:
04:56:09:  <!-- Folding Slots -->
04:56:09:  <slot id='0' type='CPU'/>
04:56:09:  <slot id='1' type='GPU'/>
04:56:09:</config>
04:56:09:WU01:FS01:0x21:ERROR:exception: Error downloading array interactionCount: clEnqueueReadBuffer (-5)
04:56:09:WU01:FS01:0x21:Saving result file logfile_01.txt
04:56:09:WU01:FS01:0x21:Saving result file log.txt
04:56:09:WU01:FS01:0x21:Folding@home Core Shutdown: BAD_WORK_UNIT
04:56:12:WARNING:WU01:FS01:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
04:56:12:WU01:FS01:Sending unit results: id:01 state:SEND error:FAULTY project:9205 run:56 clone:37 gen:5 core:0x21 unit:0x0000000cab436c685796c0b51e898eef
04:56:12:WU01:FS01:Uploading 7.00KiB to 171.67.108.104
04:56:12:WU01:FS01:Connecting to 171.67.108.104:8080
04:56:12:WU01:FS01:Upload complete
04:56:12:WU01:FS01:Server responded WORK_ACK (400)
04:56:12:WU01:FS01:Cleaning up
04:56:12:WU02:FS01:Connecting to 171.67.108.45:80
04:56:12:WU02:FS01:Assigned to work server 140.163.4.244
04:56:12:WU02:FS01:Requesting new work unit for slot 01: READY gpu:0:GM204 [GeForce GTX 970] from 140.163.4.244
04:56:12:WU02:FS01:Connecting to 140.163.4.244:8080
04:56:13:WU02:FS01:Downloading 2.77MiB
04:56:16:WU02:FS01:Download complete
04:56:16:WU02:FS01:Received Unit: id:02 state:DOWNLOAD error:NO_ERROR project:13500 run:1 clone:138 gen:49 core:0x21 unit:0x0000003f8ca304f457a358fed75d341c
04:56:16:WU02:FS01:Starting
04:56:16:WU02:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Static/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_21.fah/FahCore_21.exe -dir 02 -suffix 01 -version 704 -lifeline 11108 -checkpoint 15 -gpu 0 -gpu-vendor nvidia
04:56:16:WU02:FS01:Started FahCore on PID 10680
04:56:16:WU02:FS01:Core PID:10932
04:56:16:WU02:FS01:FahCore 0x21 started
04:56:16:WU02:FS01:0x21:*********************** Log Started 2016-11-25T04:56:16Z ***********************
04:56:16:WU02:FS01:0x21:Project: 13500 (Run 1, Clone 138, Gen 49)
04:56:16:WU02:FS01:0x21:Unit: 0x0000003f8ca304f457a358fed75d341c
04:56:16:WU02:FS01:0x21:CPU: 0x00000000000000000000000000000000
04:56:16:WU02:FS01:0x21:Machine: 1
04:56:16:WU02:FS01:0x21:Reading tar file core.xml
04:56:16:WU02:FS01:0x21:Reading tar file system.xml
04:56:16:WU02:FS01:0x21:Reading tar file integrator.xml
04:56:16:WU02:FS01:0x21:Reading tar file state.xml
04:56:17:WU02:FS01:0x21:Digital signatures verified
04:56:17:WU02:FS01:0x21:Folding@home GPU Core21 Folding@home Core
04:56:17:WU02:FS01:0x21:Version 0.0.17
04:56:20:WU02:FS01:0x21:ERROR:exception: Error downloading array interactionCount: clEnqueueReadBuffer (-5)
04:56:20:WU02:FS01:0x21:Saving result file logfile_01.txt
04:56:20:WU02:FS01:0x21:Saving result file log.txt
04:56:20:WU02:FS01:0x21:Folding@home Core Shutdown: BAD_WORK_UNIT
04:56:22:WARNING:WU02:FS01:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
04:56:22:WU02:FS01:Sending unit results: id:02 state:SEND error:FAULTY project:13500 run:1 clone:138 gen:49 core:0x21 unit:0x0000003f8ca304f457a358fed75d341c
04:56:22:WU02:FS01:Uploading 2.39KiB to 140.163.4.244
04:56:22:WU02:FS01:Connecting to 140.163.4.244:8080
04:56:23:WU02:FS01:Upload complete
04:56:23:WU02:FS01:Server responded WORK_ACK (400)
04:56:23:WU02:FS01:Cleaning up
04:57:28:WU00:FS00:0xa7:Completed 2400 out of 80000 steps (3%)
04:59:06:WU00:FS00:0xa7:Completed 3200 out of 80000 steps (4%)
05:00:24:FS00:Paused
05:00:24:FS01:Paused
05:00:24:FS00:Shutting core down
05:00:24:WU00:FS00:0xa7:WARNING:Console control signal 1 on PID 6416
05:00:24:WU00:FS00:0xa7:Exiting, please wait. . .
05:00:26:WU00:FS00:0xa7:Folding@home Core Shutdown: INTERRUPTED
05:00:26:WU00:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
05:01:14:Saving configuration to config.xml
05:01:14:<config>
05:01:14:  <!-- User Information -->
05:01:14:  <passkey v='********************************'/>
05:01:14:  <user v='Static_Stripes'/>
05:01:14:
05:01:14:  <!-- Folding Slots -->
05:01:14:  <slot id='0' type='CPU'>
05:01:14:    <paused v='true'/>
05:01:14:  </slot>
05:01:14:  <slot id='1' type='GPU'>
05:01:14:    <paused v='true'/>
05:01:14:  </slot>
05:01:14:</config>
Joe_H
Site Admin
Posts: 7937
Joined: Tue Apr 21, 2009 4:41 pm
Hardware configuration: Mac Pro 2.8 quad 12 GB smp4
MacBook Pro 2.9 i7 8 GB smp2
Location: W. MA

Re: GPU folding fails

Post by Joe_H »

Code: Select all

04:52:41:WU02:FS01:0x21:ERROR:exception: Error downloading array interactionCount: clEnqueueReadBuffer (-5)
This error in your log for the GPU slot probably means you recently updated to one of the version 375 drivers from nVidia. They apparently managed to break part of the OpenCL implementation, these drivers do not work with Core_21 projects. The latest version of the driver reported to work is 373.06.

A blog message about this has been posted by PG, there is a topic here following the issue.
Image

iMac 2.8 i7 12 GB smp8, Mac Pro 2.8 quad 12 GB smp6
MacBook Pro 2.9 i7 8 GB smp3
Static Stripes
Posts: 6
Joined: Wed Oct 05, 2016 2:44 am

Re: GPU folding fails

Post by Static Stripes »

Got it working again, thanks!
RABishop
Posts: 73
Joined: Thu May 07, 2015 2:42 am

Re: GPU folding fails -- clEnqueueReadBuffer

Post by RABishop »

Early yesterday I saw my point totals dropping, so I checked each of my machines. The machine I call #2 had a problem, which I was hoping would workout; but it hasn't yet. There's always been something odd about that machine. There is an X99 mobo in it, and a 5930k processor, along with 3 NVIDIA graphics cards. I noticed a LONG time ago that the cards were being misidentified by the version of Mint 17.3 that I am running on that system. In the top slot, I have a 980Ti. In the middle slot, I have a 1080; and in the bottom slot, I have a 1070. I am NOT using that machine right now to report on this problem. That machine also has graphics issues which I have not been able to solve. So I have to do this from another machine. The point is that the newest of NVIDIA cards, the 1080s and 1070s aren't even identified, in LINUX, with any more than a code designation.

On the machine in question, it says I have a 980TI, in the top slot. It says I have an 1080 in the mid-slot. This is true, although they might be reversed in order from top to bottom. Either way it is, the fact that I have a 1080 in the middle is undeniable. But, what I have in the bottom slot is NOT a 1080, but a 1070. And I now suspect that a 1080 job has been dropped into that 1070, which doesn't have enough cuda cores to process the job. So, since early yesterday, that slot has been saying it's going to take 21.33 days, or something like that, to finish. It just keeps saying for a while that it is running, then reverts AGAIN to a non running state.

As I said, that system has graphical interface problems. Things like boxes sometimes show up in it (like the box I'm typing in, NOW) only partially formed, or with no borders at all. I have no idea why the 1070, in the bottom slot, was ever identified as a 1080. I'm getting graphics from the 980TI, in the top slot. But those graphics are so poor, it's nearly impossible to use the system in question for communications purposes. I know this is weird, but my memory isn't so perfect that I can go to the defective system, and memorize everything about it, in order to use it, to communicate here.

HELP.
RABishop
Posts: 73
Joined: Thu May 07, 2015 2:42 am

Re: GPU folding fails -- clEnqueueReadBuffer

Post by RABishop »

OK, well it seems there have been no replies to this. I knew it was a complicated mess when I tried to describe the problem. I guess the best thing I can do is just shut the whole thing down after uninstalling FAH, then try a reinstall, going card-by-card, and hope the machine will correctly recognize the 1070 card as a 1070 and not a 1080.
RABishop
Posts: 73
Joined: Thu May 07, 2015 2:42 am

Many, many failed GPU jobs

Post by RABishop »

I just checked one of my machines (this one) after having made sure, about a week ago that it was functioning properly over a couple of hours, precisely because of many previously failed GPU jobs, on this machine. Three GPUs, not in use due to failed jobs. Who knows how stinking long that was going on! ON the previous occasion, I uninstalled and reinstalled the Client using the terminal. I just did it again this time, and I managed to get 2 out of the 3 GPUs to install correctly. Although the 3rd seems to install, now it gets nothing but failed jobs again. I uninstall and reinjstall that card, and the Client just keeps giving me back the same job, over and over. I just looked at the Control Window. The last thing I did was uninstall the Card, so I can't say now what job and server might be involved. I DID copy the log file, perhaps (or not) before I uninstalled the card.

Here is that information:


22:16:13:Removing old file 'configs/config-20170131-213250.xml'
22:16:13:Saving configuration to /etc/fahclient/config.xml
22:16:13:<config>
22:16:13: <!-- Network -->
22:16:13: <proxy v=':8080'/>
22:16:13:
22:16:13: <!-- Slot Control -->
22:16:13: <power v='full'/>
22:16:13:
22:16:13: <!-- User Information -->
22:16:13: <passkey v='********************************'/>
22:16:13: <user v='RABishop'/>
22:16:13:
22:16:13: <!-- Folding Slots -->
22:16:13: <slot id='0' type='CPU'>
22:16:13: <cpus v='12'/>
22:16:13: <next-unit-percentage v='100'/>
22:16:13: </slot>
22:16:13: <slot id='1' type='GPU'>
22:16:13: <next-unit-percentage v='100'/>
22:16:13: </slot>
22:16:13: <slot id='2' type='GPU'>
22:16:13: <next-unit-percentage v='100'/>
22:16:13: </slot>
22:16:13:</config>
22:16:14:WU03:FS00:0xa4:Completed 75000 out of 250000 steps (30%)
22:16:26:FS00:Paused
22:16:26:FS01:Paused
22:16:26:FS02:Paused
22:16:26:FS00:Shutting core down
22:16:26:FS01:Shutting core down
22:16:26:FS02:Shutting core down
22:16:27:WU01:FS01:0x18:Caught signal SIGINT(2) on PID 1515
22:16:27:WU01:FS01:0x18:Exiting, please wait. . .
22:16:27:WU00:FS02:0x18:Caught signal SIGINT(2) on PID 1521
22:16:27:WU00:FS02:0x18:Exiting, please wait. . .
22:16:27:WU01:FS01:0x18:Folding@home Core Shutdown: INTERRUPTED
22:16:27:WU00:FS02:0x18:Folding@home Core Shutdown: INTERRUPTED
22:16:27:WU01:FS01:FahCore returned: INTERRUPTED (102 = 0x66)
22:16:27:WU00:FS02:FahCore returned: INTERRUPTED (102 = 0x66)
22:16:30:WU03:FS00:0xa4:Client no longer detected. Shutting down core.
22:16:30:WU03:FS00:0xa4:
22:16:30:WU03:FS00:0xa4:Folding@home Core Shutdown: CLIENT_DIED
22:16:31:WU03:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
22:16:49:Removing old file 'configs/config-20170131-213333.xml'
22:16:49:Saving configuration to /etc/fahclient/config.xml
22:16:49:<config>
22:16:49: <!-- Network -->
22:16:49: <proxy v=':8080'/>
22:16:49:
22:16:49: <!-- Slot Control -->
22:16:49: <power v='full'/>
22:16:49:
22:16:49: <!-- User Information -->
22:16:49: <passkey v='********************************'/>
22:16:49: <user v='RABishop'/>
22:16:49:
22:16:49: <!-- Folding Slots -->
22:16:49: <slot id='0' type='CPU'>
22:16:49: <cpus v='12'/>
22:16:49: <next-unit-percentage v='100'/>
22:16:49: <paused v='true'/>
22:16:49: </slot>
22:16:49: <slot id='1' type='GPU'>
22:16:49: <next-unit-percentage v='100'/>
22:16:49: <paused v='true'/>
22:16:49: </slot>
22:16:49: <slot id='2' type='GPU'>
22:16:49: <next-unit-percentage v='100'/>
22:16:49: <paused v='true'/>
22:16:49: </slot>
22:16:49:</config>
____________________________________________________________________________________

It looks as though I had uninstalled the 3rd GPU before copying the log. I'm not going to make any major changes until after I hear from someone that I ought to do so. At least the thing is making contributions with two of three cards. I'm using Mint 17,3 Cinnamon as an OS. System info recognizes all three cards, and NVIDIA X Server recognizes and allows me to control each of the cards' behavior. What I CAN'T understand is why the Client would continue to try feeding a jobs that ALWAYS is, inevitably, going to fail back to the same slot. And, even less, do I have any clue as to how to stop this junk without doing a complete uninstall and reinstall. Maybe someone can help me with that.

Thanks.
bruce
Posts: 20824
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: GPU folding fails -- clEnqueueReadBuffer

Post by bruce »

Your post doesn't tell us which drivers are in use, nor do they tell us which projects are failing. Using FAHClient, view the log. Press "refresh" and scroll back to the beginning. Post the section before a segment like the part you just posted. Then select the slot associated with the failed WUs (slot 3?) and paste enough of that log here so we can see what's happening.
foldinghomealone
Posts: 128
Joined: Wed Feb 01, 2017 7:07 pm

GPU folding fails - clEnqueueReadBuffer (-5)

Post by foldinghomealone »

Hi there,

I'm folding since Oct. 2016 but new to this forum. For a few weeks now, approx. 10-15% of my WUs fail. Before that it basically didn't happen.
I'm using FAH 7.4.4 on Win10 64bit and a GTX 1070 with driver 372.70.

This is the log from my last WU from today morning.

Code: Select all

03:45:33:WU01:FS00:0x21:Completed 2000000 out of 2000000 steps (100%)
03:45:33:WU00:FS00:Connecting to 171.67.108.45:80
03:45:34:WU00:FS00:Assigned to work server 140.163.4.245
03:45:34:WU00:FS00:Requesting new work unit for slot 00: RUNNING gpu:1:GP104 [GeForce GTX 1070] from 140.163.4.245
03:45:34:WU00:FS00:Connecting to 140.163.4.245:8080
03:45:34:WU00:FS00:Downloading 14.50MiB
03:45:37:WU01:FS00:0x21:Saving result file logfile_01.txt
03:45:37:WU01:FS00:0x21:Saving result file checkpointState.xml
03:45:40:WU00:FS00:Download 62.49%
03:45:41:WU01:FS00:0x21:Saving result file checkpt.crc
03:45:41:WU01:FS00:0x21:Saving result file log.txt
03:45:41:WU01:FS00:0x21:Saving result file positions.xtc
03:45:42:WU01:FS00:0x21:Folding@home Core Shutdown: FINISHED_UNIT
03:45:42:WU00:FS00:Download complete
03:45:42:WU00:FS00:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:10496 run:144 clone:3 gen:5 core:0x21 unit:0x000000068ca304f556bbb041333f9b30
03:45:42:WU01:FS00:FahCore returned: FINISHED_UNIT (100 = 0x64)
03:45:42:WU01:FS00:Sending unit results: id:01 state:SEND error:NO_ERROR project:10496 run:158 clone:5 gen:3 core:0x21 unit:0x000000058ca304f556bbb1407cb13b1b
03:45:42:WU01:FS00:Uploading 21.88MiB to 140.163.4.245
03:45:42:WU00:FS00:Starting
03:45:42:WU01:FS00:Connecting to 140.163.4.245:8080
03:45:42:WU00:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Kälker/AppData/Roaming/FAHClient/cores/fahwebx.stanford.edu/cores/Win32/AMD64/NVIDIA/Fermi/Core_21.fah/FahCore_21.exe -dir 00 -suffix 01 -version 704 -lifeline 6372 -checkpoint 15 -gpu 0 -gpu-vendor nvidia
03:45:43:WU00:FS00:Started FahCore on PID 5764
03:45:43:WU00:FS00:Core PID:3496
03:45:43:WU00:FS00:FahCore 0x21 started
03:45:43:WU00:FS00:0x21:*********************** Log Started 2017-02-01T03:45:43Z ***********************
03:45:43:WU00:FS00:0x21:Project: 10496 (Run 144, Clone 3, Gen 5)
03:45:43:WU00:FS00:0x21:Unit: 0x000000068ca304f556bbb041333f9b30
03:45:43:WU00:FS00:0x21:CPU: 0x00000000000000000000000000000000
03:45:43:WU00:FS00:0x21:Machine: 0
03:45:43:WU00:FS00:0x21:Reading tar file core.xml
03:45:43:WU00:FS00:0x21:Reading tar file system.xml
03:45:44:WU00:FS00:0x21:Reading tar file integrator.xml
03:45:44:WU00:FS00:0x21:Reading tar file state.xml
03:45:45:WU00:FS00:0x21:Digital signatures verified
03:45:45:WU00:FS00:0x21:Folding@home GPU Core21 Folding@home Core
03:45:45:WU00:FS00:0x21:Version 0.0.17
03:45:49:WU01:FS00:Upload 3.14%
03:45:55:WU01:FS00:Upload 6.00%
03:46:01:WU01:FS00:Upload 8.85%
03:46:04:WU00:FS00:0x21:Completed 0 out of 2000000 steps (0%)
03:46:04:WU00:FS00:0x21:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
03:46:07:WU01:FS00:Upload 11.71%
03:46:13:WU01:FS00:Upload 14.85%
03:46:19:WU01:FS00:Upload 17.71%
03:46:25:WU01:FS00:Upload 20.28%
03:46:31:WU01:FS00:Upload 23.42%
03:46:37:WU01:FS00:Upload 26.56%
03:46:43:WU01:FS00:Upload 29.42%
03:46:49:WU01:FS00:Upload 32.27%
03:46:55:WU01:FS00:Upload 35.13%
03:47:01:WU01:FS00:Upload 38.27%
03:47:07:WU01:FS00:Upload 41.13%
03:47:13:WU01:FS00:Upload 43.98%
03:47:19:WU01:FS00:Upload 46.84%
03:47:25:WU01:FS00:Upload 49.69%
03:47:31:WU01:FS00:Upload 52.83%
03:47:37:WU01:FS00:Upload 55.69%
03:47:43:WU01:FS00:Upload 58.55%
03:47:49:WU01:FS00:Upload 61.40%
03:47:55:WU01:FS00:Upload 64.54%
03:47:59:WU00:FS00:0x21:Completed 20000 out of 2000000 steps (1%)
03:48:01:WU01:FS00:Upload 67.40%
03:48:07:WU01:FS00:Upload 70.26%
03:48:13:WU01:FS00:Upload 73.40%
03:48:19:WU01:FS00:Upload 76.25%
03:48:25:WU01:FS00:Upload 79.11%
03:48:31:WU01:FS00:Upload 81.96%
03:48:37:WU01:FS00:Upload 84.82%
03:48:43:WU01:FS00:Upload 87.96%
03:48:49:WU01:FS00:Upload 90.82%
03:48:55:WU01:FS00:Upload 93.67%
03:49:01:WU01:FS00:Upload 96.53%
03:49:07:WU01:FS00:Upload 99.67%
03:49:29:WU01:FS00:Upload complete
03:49:29:WU01:FS00:Server responded WORK_ACK (400)
03:49:29:WU01:FS00:Final credit estimate, 89803.00 points
03:49:29:WU01:FS00:Cleaning up
03:49:54:WU00:FS00:0x21:Completed 40000 out of 2000000 steps (2%)
03:50:02:FS00:Finishing
03:51:50:WU00:FS00:0x21:Completed 60000 out of 2000000 steps (3%)
03:53:45:WU00:FS00:0x21:Completed 80000 out of 2000000 steps (4%)
03:55:41:WU00:FS00:0x21:Completed 100000 out of 2000000 steps (5%)
03:57:36:WU00:FS00:0x21:Completed 120000 out of 2000000 steps (6%)
03:59:35:WU00:FS00:0x21:Completed 140000 out of 2000000 steps (7%)
04:01:31:WU00:FS00:0x21:Completed 160000 out of 2000000 steps (8%)
04:03:26:WU00:FS00:0x21:Completed 180000 out of 2000000 steps (9%)
04:05:22:WU00:FS00:0x21:Completed 200000 out of 2000000 steps (10%)
04:07:17:WU00:FS00:0x21:Completed 220000 out of 2000000 steps (11%)
04:09:13:WU00:FS00:0x21:Completed 240000 out of 2000000 steps (12%)
04:11:13:WU00:FS00:0x21:Completed 260000 out of 2000000 steps (13%)
04:13:08:WU00:FS00:0x21:Completed 280000 out of 2000000 steps (14%)
04:15:04:WU00:FS00:0x21:Completed 300000 out of 2000000 steps (15%)
04:17:00:WU00:FS00:0x21:Completed 320000 out of 2000000 steps (16%)
04:18:56:WU00:FS00:0x21:Completed 340000 out of 2000000 steps (17%)
04:20:51:WU00:FS00:0x21:Completed 360000 out of 2000000 steps (18%)
04:22:51:WU00:FS00:0x21:Completed 380000 out of 2000000 steps (19%)
04:24:47:WU00:FS00:0x21:Completed 400000 out of 2000000 steps (20%)
04:26:42:WU00:FS00:0x21:Completed 420000 out of 2000000 steps (21%)
04:28:38:WU00:FS00:0x21:Completed 440000 out of 2000000 steps (22%)
04:30:34:WU00:FS00:0x21:Completed 460000 out of 2000000 steps (23%)
04:32:29:WU00:FS00:0x21:Completed 480000 out of 2000000 steps (24%)
04:34:25:WU00:FS00:0x21:Completed 500000 out of 2000000 steps (25%)
04:36:24:WU00:FS00:0x21:Completed 520000 out of 2000000 steps (26%)
04:38:20:WU00:FS00:0x21:Completed 540000 out of 2000000 steps (27%)
04:40:15:WU00:FS00:0x21:Completed 560000 out of 2000000 steps (28%)
04:42:11:WU00:FS00:0x21:Completed 580000 out of 2000000 steps (29%)
04:44:06:WU00:FS00:0x21:Completed 600000 out of 2000000 steps (30%)
04:46:02:WU00:FS00:0x21:Completed 620000 out of 2000000 steps (31%)
04:48:02:WU00:FS00:0x21:Completed 640000 out of 2000000 steps (32%)
04:49:58:WU00:FS00:0x21:Completed 660000 out of 2000000 steps (33%)
04:51:53:WU00:FS00:0x21:Completed 680000 out of 2000000 steps (34%)
04:53:49:WU00:FS00:0x21:Completed 700000 out of 2000000 steps (35%)
04:55:44:WU00:FS00:0x21:Completed 720000 out of 2000000 steps (36%)
04:56:06:WU00:FS00:0x21:ERROR:exception: Error downloading array energyBuffer: clEnqueueReadBuffer (-5)
04:56:06:WU00:FS00:0x21:Saving result file logfile_01.txt
04:56:06:WU00:FS00:0x21:Saving result file log.txt
04:56:06:WU00:FS00:0x21:Folding@home Core Shutdown: BAD_WORK_UNIT
04:56:08:WARNING:WU00:FS00:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
04:56:08:WU00:FS00:Sending unit results: id:00 state:SEND error:FAULTY project:10496 run:144 clone:3 gen:5 core:0x21 unit:0x000000068ca304f556bbb041333f9b30
04:56:08:WU00:FS00:Uploading 3.01KiB to 140.163.4.245
04:56:08:WU00:FS00:Connecting to 140.163.4.245:8080
04:56:09:WU00:FS00:Upload complete
04:56:09:WU00:FS00:Server responded WORK_ACK (400)
04:56:09:WU00:FS00:Cleaning up
Does it mean that the WU is damaged or is something wrong with my system?

Losing 10% of WUs is not really encouraging. Thank you for your help.

Mod edit: changed Quote tags to Code for log
bruce
Posts: 20824
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: GPU folding fails - clEnqueueReadBuffer (-5)

Post by bruce »

There has been an update to FAHCore_21 which fixes this problem. The new version (0.0.18 or later) is in the process of being pushed out automatically. Many projects have been adjusted to require the new version but some still accept the old version 0.0.17.

I'll speak to the owner of project:10496.

You can fix this manually or you can wait to be assigned a project that has been updated.

delete C:/Users/Kälker/AppData/Roaming/FAHClient/cores/fahwebx.stanford.edu/cores/Win32/AMD64/NVIDIA/Fermi/Core_21.fah/FahCore_21.exe
foldinghomealone
Posts: 128
Joined: Wed Feb 01, 2017 7:07 pm

Re: GPU folding fails - clEnqueueReadBuffer (-5)

Post by foldinghomealone »

Unfortunately the same happened with version 0.0.18 just now:

Code: Select all

06:34:31:WU00:FS00:0x21:Completed 2000000 out of 2000000 steps (100%)
06:34:32:WU01:FS00:Connecting to 171.67.108.45:80
06:34:33:WU01:FS00:Assigned to work server 140.163.4.231
06:34:33:WU01:FS00:Requesting new work unit for slot 00: RUNNING gpu:1:GP104 [GeForce GTX 1070] from 140.163.4.231
06:34:33:WU01:FS00:Connecting to 140.163.4.231:8080
06:34:33:WU01:FS00:Downloading 14.94MiB
06:34:36:WU00:FS00:0x21:Saving result file logfile_01.txt
06:34:36:WU00:FS00:0x21:Saving result file badstate-0.xml
06:34:39:WU01:FS00:Download 57.31%
06:34:40:WU00:FS00:0x21:Saving result file checkpointState.xml
06:34:41:WU01:FS00:Download complete
06:34:42:WU01:FS00:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:11711 run:10 clone:30 gen:54 core:0x21 unit:0x0000004d8ca304e758332b4e0d9e0abe
06:34:44:WU00:FS00:0x21:Saving result file checkpt.crc
06:34:44:WU00:FS00:0x21:Saving result file log.txt
06:34:44:WU00:FS00:0x21:Saving result file positions.xtc
06:34:45:WU00:FS00:0x21:Folding@home Core Shutdown: FINISHED_UNIT
06:34:46:WU00:FS00:FahCore returned: FINISHED_UNIT (100 = 0x64)
06:34:46:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:10496 run:1 clone:49 gen:1 core:0x21 unit:0x000000018ca304f5588956be14d9186d
06:34:46:WU00:FS00:Uploading 21.90MiB to 140.163.4.245
06:34:46:WU01:FS00:Starting
06:34:46:WU00:FS00:Connecting to 140.163.4.245:8080
06:34:47:WU01:FS00:0x21:*********************** Log Started 2017-02-02T06:34:46Z ***********************
06:34:47:WU01:FS00:0x21:Project: 11711 (Run 10, Clone 30, Gen 54)
06:34:47:WU01:FS00:0x21:Unit: 0x0000004d8ca304e758332b4e0d9e0abe
06:34:47:WU01:FS00:0x21:CPU: 0x00000000000000000000000000000000
06:34:47:WU01:FS00:0x21:Machine: 0
06:34:47:WU01:FS00:0x21:Reading tar file core.xml
06:34:47:WU01:FS00:0x21:Reading tar file integrator.xml
06:34:47:WU01:FS00:0x21:Reading tar file state.xml
06:34:47:WU01:FS00:0x21:Reading tar file system.xml
06:34:47:WU01:FS00:0x21:Digital signatures verified
06:34:47:WU01:FS00:0x21:Folding@home GPU Core21 Folding@home Core
06:34:47:WU01:FS00:0x21:Version 0.0.18
06:34:52:WU01:FS00:0x21:Completed 0 out of 7500000 steps (0%)
06:34:52:WU01:FS00:0x21:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
06:34:53:WU00:FS00:Upload 3.14%
06:34:59:WU00:FS00:Upload 6.28%
06:35:05:WU00:FS00:Upload 9.13%
06:35:11:WU00:FS00:Upload 11.99%
06:35:17:WU00:FS00:Upload 15.13%
06:35:23:WU00:FS00:Upload 17.98%
06:35:29:WU00:FS00:Upload 20.84%
06:35:35:WU00:FS00:Upload 23.69%
06:35:41:WU00:FS00:Upload 26.83%
06:35:47:WU00:FS00:Upload 29.69%
06:35:53:WU00:FS00:Upload 32.54%
06:35:59:WU00:FS00:Upload 35.40%
06:36:05:WU00:FS00:Upload 38.54%
06:36:11:WU00:FS00:Upload 41.39%
06:36:17:WU00:FS00:Upload 44.24%
06:36:23:WU00:FS00:Upload 47.10%
06:36:29:WU00:FS00:Upload 50.24%
06:36:35:WU00:FS00:Upload 53.09%
06:36:41:WU00:FS00:Upload 55.95%
06:36:47:WU00:FS00:Upload 59.09%
06:36:53:WU00:FS00:Upload 61.94%
06:36:59:WU00:FS00:Upload 64.80%
06:37:01:WU01:FS00:0x21:Completed 75000 out of 7500000 steps (1%)
06:37:05:WU00:FS00:Upload 67.65%
06:37:11:WU00:FS00:Upload 70.51%
06:37:18:WU00:FS00:Upload 73.07%
06:37:24:WU00:FS00:Upload 75.93%
06:37:30:WU00:FS00:Upload 79.07%
06:37:36:WU00:FS00:Upload 81.92%
06:37:42:WU00:FS00:Upload 84.78%
06:37:48:WU00:FS00:Upload 87.63%
06:37:54:WU00:FS00:Upload 90.49%
06:38:00:WU00:FS00:Upload 93.63%
06:38:06:WU00:FS00:Upload 96.48%
06:38:12:WU00:FS00:Upload 99.34%
06:38:42:WU00:FS00:Upload complete
06:38:42:WU00:FS00:Server responded WORK_ACK (400)
06:38:42:WU00:FS00:Final credit estimate, 85376.00 points
06:38:42:WU00:FS00:Cleaning up
06:39:11:WU01:FS00:0x21:Completed 150000 out of 7500000 steps (2%)
06:41:20:WU01:FS00:0x21:Completed 225000 out of 7500000 steps (3%)
06:43:30:WU01:FS00:0x21:Completed 300000 out of 7500000 steps (4%)
06:45:39:WU01:FS00:0x21:Completed 375000 out of 7500000 steps (5%)
06:47:49:WU01:FS00:0x21:Completed 450000 out of 7500000 steps (6%)
06:49:58:WU01:FS00:0x21:Completed 525000 out of 7500000 steps (7%)
06:52:07:WU01:FS00:0x21:Completed 600000 out of 7500000 steps (8%)
06:54:16:WU01:FS00:0x21:Completed 675000 out of 7500000 steps (9%)
06:56:26:WU01:FS00:0x21:Completed 750000 out of 7500000 steps (10%)
06:58:35:WU01:FS00:0x21:Completed 825000 out of 7500000 steps (11%)
07:00:44:WU01:FS00:0x21:Completed 900000 out of 7500000 steps (12%)
07:02:57:WU01:FS00:0x21:Completed 975000 out of 7500000 steps (13%)
07:05:10:WU01:FS00:0x21:Completed 1050000 out of 7500000 steps (14%)
07:07:23:WU01:FS00:0x21:Completed 1125000 out of 7500000 steps (15%)
07:09:36:WU01:FS00:0x21:Completed 1200000 out of 7500000 steps (16%)
07:11:50:WU01:FS00:0x21:Completed 1275000 out of 7500000 steps (17%)
07:14:03:WU01:FS00:0x21:Completed 1350000 out of 7500000 steps (18%)
07:16:16:WU01:FS00:0x21:Completed 1425000 out of 7500000 steps (19%)
07:18:29:WU01:FS00:0x21:Completed 1500000 out of 7500000 steps (20%)
07:20:43:WU01:FS00:0x21:Completed 1575000 out of 7500000 steps (21%)
07:22:56:WU01:FS00:0x21:Completed 1650000 out of 7500000 steps (22%)
07:25:09:WU01:FS00:0x21:Completed 1725000 out of 7500000 steps (23%)
07:27:23:WU01:FS00:0x21:Completed 1800000 out of 7500000 steps (24%)
07:29:36:WU01:FS00:0x21:Completed 1875000 out of 7500000 steps (25%)
07:31:50:WU01:FS00:0x21:Completed 1950000 out of 7500000 steps (26%)
07:34:04:WU01:FS00:0x21:Completed 2025000 out of 7500000 steps (27%)
07:36:17:WU01:FS00:0x21:Completed 2100000 out of 7500000 steps (28%)
07:38:31:WU01:FS00:0x21:Completed 2175000 out of 7500000 steps (29%)
07:40:42:WU01:FS00:0x21:Completed 2250000 out of 7500000 steps (30%)
07:40:43:WU01:FS00:0x21:ERROR:exception: Error downloading array energyBuffer: clEnqueueReadBuffer (-5)
07:40:43:WU01:FS00:0x21:Saving result file logfile_01.txt
07:40:43:WU01:FS00:0x21:Saving result file log.txt
07:40:43:WU01:FS00:0x21:Folding@home Core Shutdown: BAD_WORK_UNIT
07:40:51:WARNING:WU01:FS00:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
07:40:51:WU01:FS00:Sending unit results: id:01 state:SEND error:FAULTY project:11711 run:10 clone:30 gen:54 core:0x21 unit:0x0000004d8ca304e758332b4e0d9e0abe
07:40:51:WU01:FS00:Uploading 11.00KiB to 140.163.4.231
07:40:51:WU01:FS00:Connecting to 140.163.4.231:8080
07:40:51:WU00:FS00:Connecting to 171.67.108.45:80
07:40:51:WU01:FS00:Upload complete
07:40:51:WU01:FS00:Server responded WORK_ACK (400)
07:40:51:WU01:FS00:Cleaning up
07:40:52:WU00:FS00:Assigned to work server 140.163.4.231
07:40:52:WU00:FS00:Requesting new work unit for slot 00: READY gpu:1:GP104 [GeForce GTX 1070] from 140.163.4.231
07:40:52:WU00:FS00:Connecting to 140.163.4.231:8080
Mod edit: Please use Code tags around log listings, not Quote
bruce
Posts: 20824
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: GPU folding fails - clEnqueueReadBuffer (-5)

Post by bruce »

Unfortunately I don't know what to tell you.

I did check, and you got partial credit for your efforts. It was reassigned and somebody else who completed it. I have no ideal what might be different about their system.

Hi foldinghomealone (team 70335),
Your WU (P10496 R144 C3 G5) was added to the stats database on 2017-01-31 21:07:04 for 4385.88 points of credit. (partial)
Hi ***** (team ******),
Your WU (P10496 R144 C3 G5) was added to the stats database on 2017-02-02 03:08:37 for 12183 points of credit.
-----------------
Hi foldinghomealone (team 70335),
Your WU (P11711 R10 C30 G54) was added to the stats database on 2017-02-02 00:07:14 for 4334.4 points of credit. (partial)
(... reassigned but not yet returned.)
foldinghomealone
Posts: 128
Joined: Wed Feb 01, 2017 7:07 pm

Re: GPU folding fails - clEnqueueReadBuffer (-5)

Post by foldinghomealone »

Then it seem (at least for me) that my system is somehow not stable.
Ok, thanks for your answer.
JohnChodera
Pande Group Member
Posts: 467
Joined: Fri Feb 22, 2013 9:59 pm

Re: GPU folding fails - clEnqueueReadBuffer (-5)

Post by JohnChodera »

I think clEnqueueReadBuffer (-5) is an CL_OUT_OF_RESOURCES error. Is it possible that other activity on the system caused all memory to be exhausted?
foldinghomealone
Posts: 128
Joined: Wed Feb 01, 2017 7:07 pm

Re: GPU folding fails - clEnqueueReadBuffer (-5)

Post by foldinghomealone »

I don't think so, right now, the GPU is used only for folding purposes.
I use CPU and iGPU for internet browsing from time to time.
toTOW
Site Moderator
Posts: 6359
Joined: Sun Dec 02, 2007 10:38 am
Location: Bordeaux, France
Contact:

Re: GPU folding fails - clEnqueueReadBuffer (-5)

Post by toTOW »

Unfortunately, clEnqueueReadBuffer (-5) error happening during the processing of a WU is a sign of an error with hardware. This error usually happen when GPU crashes, and that the OS has reset the drivers to recover the issue. Usually, after this issue, the GPU will stay in safe mode (reduced clocks and voltages) until you reboot your system.

Does it happen while you're using your machine, or also when nothing but FAH is running ?

I see also from you log that you got some spurious Bad State, which could also be the sign of an unstable GPU ...

Check if everything is OK with your GPU (are fan spinning fine ?), monitor temperatures, voltages and clocks with GPUZ ...

You can also try to run FurMark on the GPU to stresstest it.
Image

Folding@Home beta tester since 2002. Folding Forum moderator since July 2008.
Post Reply