Issue with gpu submitting invalid work and not connecting

[Mastermine]

Code: Select all

20:54:41:Removing old file 'configs/config-20140513-081519.xml'
20:54:41:Saving configuration to config.xml
20:54:41:<config>
20:54:41:  <!-- Folding Core -->
20:54:41:  <core-priority v='low'/>
20:54:41:
20:54:41:  <!-- Network -->
20:54:41:  <proxy v=':8080'/>
20:54:41:
20:54:41:  <!-- Slot Control -->
20:54:41:  <power v='full'/>
20:54:41:
20:54:41:  <!-- User Information -->
20:54:41:  <passkey v='********************************'/>
20:54:41:  <team v='224497'/>
20:54:41:  <user v='Mastermine'/>
20:54:41:
20:54:41:  <!-- Folding Slots -->
20:54:41:  <slot id='0' type='CPU'>
20:54:41:    <cpus v='2'/>
20:54:41:    <paused v='true'/>
20:54:41:  </slot>
20:54:41:  <slot id='1' type='GPU'>
20:54:41:    <paused v='true'/>
20:54:41:  </slot>
20:54:41:  <slot id='2' type='GPU'/>
20:54:41:  <slot id='3' type='GPU'/>
20:54:41:</config>
20:54:43:WU03:FS03:0x17:Completed 850000 out of 5000000 steps (17%)
20:55:10:Removing old file 'configs/config-20140513-081543.xml'
20:55:10:Saving configuration to config.xml
20:55:10:<config>
20:55:10:  <!-- Folding Core -->
20:55:10:  <core-priority v='low'/>
20:55:10:
20:55:10:  <!-- Network -->
20:55:10:  <proxy v=':8080'/>
20:55:10:
20:55:10:  <!-- Slot Control -->
20:55:10:  <power v='full'/>
20:55:10:
20:55:10:  <!-- User Information -->
20:55:10:  <passkey v='********************************'/>
20:55:10:  <team v='224497'/>
20:55:10:  <user v='Mastermine'/>
20:55:10:
20:55:10:  <!-- Folding Slots -->
20:55:10:  <slot id='0' type='CPU'>
20:55:10:    <cpus v='2'/>
20:55:10:    <paused v='true'/>
20:55:10:  </slot>
20:55:10:  <slot id='1' type='GPU'>
20:55:10:    <paused v='true'/>
20:55:10:  </slot>
20:55:10:  <slot id='2' type='GPU'/>
20:55:10:  <slot id='3' type='GPU'/>
20:55:10:</config>
20:55:19:FS01:Unpaused
20:55:19:WU04:FS01:Starting
20:55:19:WU04:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/KaZ/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/ATI/R600/Core_17.fah/FahCore_17.exe -dir 04 -suffix 01 -version 704 -lifeline 6008 -checkpoint 15 -gpu 0 -gpu-vendor ati
20:55:19:WU04:FS01:Started FahCore on PID 5384
20:55:19:WU04:FS01:Core PID:6452
20:55:19:WU04:FS01:FahCore 0x17 started
20:55:19:WU04:FS01:0x17:*********************** Log Started 2014-05-13T20:55:19Z ***********************
20:55:19:WU04:FS01:0x17:Project: 13000 (Run 42, Clone 0, Gen 14)
20:55:19:WU04:FS01:0x17:Unit: 0x0000001f538b3db7530fa7ee158b9da7
20:55:19:WU04:FS01:0x17:CPU: 0x00000000000000000000000000000000
20:55:19:WU04:FS01:0x17:Machine: 1
20:55:19:WU04:FS01:0x17:Reading tar file state.xml
20:55:20:WU04:FS01:0x17:Reading tar file system.xml
20:55:20:WU04:FS01:0x17:Reading tar file integrator.xml
20:55:20:WU04:FS01:0x17:Reading tar file core.xml
20:55:20:WU04:FS01:0x17:Digital signatures verified
20:55:20:WU04:FS01:0x17:Folding@home GPU core17
20:55:20:WU04:FS01:0x17:Version 0.0.52
20:56:11:Removing old file 'configs/config-20140513-082250.xml'
20:56:11:Saving configuration to config.xml
20:56:11:<config>
20:56:11:  <!-- Folding Core -->
20:56:11:  <core-priority v='low'/>
20:56:11:
20:56:11:  <!-- Network -->
20:56:11:  <proxy v=':8080'/>
20:56:11:
20:56:11:  <!-- Slot Control -->
20:56:11:  <power v='full'/>
20:56:11:
20:56:11:  <!-- User Information -->
20:56:11:  <passkey v='********************************'/>
20:56:11:  <team v='224497'/>
20:56:11:  <user v='Mastermine'/>
20:56:11:
20:56:11:  <!-- Folding Slots -->
20:56:11:  <slot id='0' type='CPU'>
20:56:11:    <cpus v='2'/>
20:56:11:    <paused v='true'/>
20:56:11:  </slot>
20:56:11:  <slot id='1' type='GPU'/>
20:56:11:  <slot id='2' type='GPU'/>
20:56:11:  <slot id='3' type='GPU'/>
20:56:11:</config>
20:57:05:FS01:Paused
20:57:05:FS01:Shutting core down
20:57:05:WU04:FS01:0x17:WARNING:Console control signal 1 on PID 6452
20:57:05:WU04:FS01:0x17:Exiting, please wait. . .
20:57:12:Removing old file 'configs/config-20140513-164706.xml'
20:57:12:Saving configuration to config.xml
20:57:12:<config>
20:57:12:  <!-- Folding Core -->
20:57:12:  <core-priority v='low'/>
20:57:12:
20:57:12:  <!-- Network -->
20:57:12:  <proxy v=':8080'/>
20:57:12:
20:57:12:  <!-- Slot Control -->
20:57:12:  <power v='full'/>
20:57:12:
20:57:12:  <!-- User Information -->
20:57:12:  <passkey v='********************************'/>
20:57:12:  <team v='224497'/>
20:57:12:  <user v='Mastermine'/>
20:57:12:
20:57:12:  <!-- Folding Slots -->
20:57:12:  <slot id='0' type='CPU'>
20:57:12:    <cpus v='2'/>
20:57:12:    <paused v='true'/>
20:57:12:  </slot>
20:57:12:  <slot id='1' type='GPU'>
20:57:12:    <paused v='true'/>
20:57:12:  </slot>
20:57:12:  <slot id='2' type='GPU'/>
20:57:12:  <slot id='3' type='GPU'/>
20:57:12:</config>
20:57:41:WU04:FS01:0x17:ERROR:exception: Force RMSE error of 220.908 with threshold of 5
20:57:41:WU04:FS01:0x17:Saving result file logfile_01.txt
20:57:41:WU04:FS01:0x17:Saving result file log.txt
20:57:41:WU04:FS01:0x17:Folding@home Core Shutdown: BAD_WORK_UNIT
20:57:42:WU04:FS01:FahCore returned: INTERRUPTED (102 = 0x66)
21:00:36:WU02:FS02:0x17:Completed 850000 out of 5000000 steps (17%)
21:07:20:WU03:FS03:0x17:Completed 900000 out of 5000000 steps (18%)
21:13:35:WU02:FS02:0x17:Completed 900000 out of 5000000 steps (18%)
21:19:43:WU03:FS03:0x17:Completed 950000 out of 5000000 steps (19%)
21:26:20:WU02:FS02:0x17:Completed 950000 out of 5000000 steps (19%)
21:32:05:WU03:FS03:0x17:Completed 1000000 out of 5000000 steps (20%)
21:39:07:WU02:FS02:0x17:Completed 1000000 out of 5000000 steps (20%)
21:44:42:WU03:FS03:0x17:Completed 1050000 out of 5000000 steps (21%)
21:52:06:WU02:FS02:0x17:Completed 1050000 out of 5000000 steps (21%)
21:57:05:WU03:FS03:0x17:Completed 1100000 out of 5000000 steps (22%)
21:58:12:FS01:Unpaused
21:58:12:WU04:FS01:Starting
21:58:12:WU04:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/KaZ/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/ATI/R600/Core_17.fah/FahCore_17.exe -dir 04 -suffix 01 -version 704 -lifeline 6008 -checkpoint 15 -gpu 0 -gpu-vendor ati
21:58:12:WU04:FS01:Started FahCore on PID 7736
21:58:12:WU04:FS01:Core PID:7904
21:58:12:WU04:FS01:FahCore 0x17 started
21:58:13:WU04:FS01:0x17:*********************** Log Started 2014-05-13T21:58:12Z ***********************
21:58:13:WU04:FS01:0x17:Project: 13000 (Run 42, Clone 0, Gen 14)
21:58:13:WU04:FS01:0x17:Unit: 0x0000001f538b3db7530fa7ee158b9da7
21:58:13:WU04:FS01:0x17:CPU: 0x00000000000000000000000000000000
21:58:13:WU04:FS01:0x17:Machine: 1
21:58:13:WU04:FS01:0x17:Reading tar file state.xml
21:58:13:WU04:FS01:0x17:Reading tar file system.xml
21:58:14:WU04:FS01:0x17:Reading tar file integrator.xml
21:58:14:WU04:FS01:0x17:Reading tar file core.xml
21:58:14:WU04:FS01:0x17:Digital signatures verified
21:58:14:WU04:FS01:0x17:Folding@home GPU core17
21:58:14:WU04:FS01:0x17:Version 0.0.52
21:59:13:Removing old file 'configs/config-20140513-171029.xml'
21:59:13:Saving configuration to config.xml
21:59:13:<config>
21:59:13:  <!-- Folding Core -->
21:59:13:  <core-priority v='low'/>
21:59:13:
21:59:13:  <!-- Network -->
21:59:13:  <proxy v=':8080'/>
21:59:13:
21:59:13:  <!-- Slot Control -->
21:59:13:  <power v='full'/>
21:59:13:
21:59:13:  <!-- User Information -->
21:59:13:  <passkey v='********************************'/>
21:59:13:  <team v='224497'/>
21:59:13:  <user v='Mastermine'/>
21:59:13:
21:59:13:  <!-- Folding Slots -->
21:59:13:  <slot id='0' type='CPU'>
21:59:13:    <cpus v='2'/>
21:59:13:    <paused v='true'/>
21:59:13:  </slot>
21:59:13:  <slot id='1' type='GPU'/>
21:59:13:  <slot id='2' type='GPU'/>
21:59:13:  <slot id='3' type='GPU'/>
21:59:13:</config>
22:00:37:WU04:FS01:0x17:ERROR:exception: Force RMSE error of 221.131 with threshold of 5
22:00:37:WU04:FS01:0x17:Saving result file logfile_01.txt
22:00:37:WU04:FS01:0x17:Saving result file log.txt
22:00:37:WU04:FS01:0x17:Folding@home Core Shutdown: BAD_WORK_UNIT
22:00:37:WARNING:WU04:FS01:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
22:00:37:WU04:FS01:Sending unit results: id:04 state:SEND error:FAULTY project:13000 run:42 clone:0 gen:14 core:0x17 unit:0x0000001f538b3db7530fa7ee158b9da7
22:00:37:WU04:FS01:Uploading 2.38KiB to 140.163.4.231
22:00:37:WU04:FS01:Connecting to 140.163.4.231:8080
22:00:37:WU04:FS01:Upload complete
22:00:38:WU04:FS01:Server responded WORK_ACK (400)
22:00:38:WU04:FS01:Cleaning up
22:00:38:WU01:FS01:Connecting to 171.67.108.201:80
22:00:38:WU01:FS01:Assigned to work server 140.163.4.231
22:00:38:WU01:FS01:Requesting new work unit for slot 01: READY gpu:0:Hawaii [Radeon R9 200 Series] from 140.163.4.231
22:00:38:WU01:FS01:Connecting to 140.163.4.231:8080
22:00:38:WU01:FS01:Downloading 4.83MiB
22:00:43:WU01:FS01:Download complete
22:00:43:WU01:FS01:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:13001 run:491 clone:6 gen:28 core:0x17 unit:0x00000040538b3db7532c876cfd348969
22:00:43:WU01:FS01:Starting
22:00:43:WU01:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/KaZ/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/ATI/R600/Core_17.fah/FahCore_17.exe -dir 01 -suffix 01 -version 704 -lifeline 6008 -checkpoint 15 -gpu 0 -gpu-vendor ati
22:00:43:WU01:FS01:Started FahCore on PID 7676
22:00:43:WU01:FS01:Core PID:7368
22:00:43:WU01:FS01:FahCore 0x17 started
22:00:44:WU01:FS01:0x17:*********************** Log Started 2014-05-13T22:00:43Z ***********************
22:00:44:WU01:FS01:0x17:Project: 13001 (Run 491, Clone 6, Gen 28)
22:00:44:WU01:FS01:0x17:Unit: 0x00000040538b3db7532c876cfd348969
22:00:44:WU01:FS01:0x17:CPU: 0x00000000000000000000000000000000
22:00:44:WU01:FS01:0x17:Machine: 1
22:00:44:WU01:FS01:0x17:Reading tar file state.xml
22:00:44:WU01:FS01:0x17:Reading tar file system.xml
22:00:45:WU01:FS01:0x17:Reading tar file integrator.xml
22:00:45:WU01:FS01:0x17:Reading tar file core.xml
22:00:45:WU01:FS01:0x17:Digital signatures verified
22:00:45:WU01:FS01:0x17:Folding@home GPU core17
22:00:45:WU01:FS01:0x17:Version 0.0.52
22:01:20:FS01:Paused
22:01:20:FS01:Shutting core down
22:01:20:WU01:FS01:0x17:WARNING:Console control signal 1 on PID 7368
22:01:20:WU01:FS01:0x17:Exiting, please wait. . .
22:02:16:Removing old file 'configs/config-20140513-171534.xml'
22:02:16:Saving configuration to config.xml
22:02:16:<config>
22:02:16:  <!-- Folding Core -->
22:02:16:  <core-priority v='low'/>
22:02:16:
22:02:16:  <!-- Network -->
22:02:16:  <proxy v=':8080'/>
22:02:16:
22:02:16:  <!-- Slot Control -->
22:02:16:  <power v='full'/>
22:02:16:
22:02:16:  <!-- User Information -->
22:02:16:  <passkey v='********************************'/>
22:02:16:  <team v='224497'/>
22:02:16:  <user v='Mastermine'/>
22:02:16:
22:02:16:  <!-- Folding Slots -->
22:02:16:  <slot id='0' type='CPU'>
22:02:16:    <cpus v='2'/>
22:02:16:    <paused v='true'/>
22:02:16:  </slot>
22:02:16:  <slot id='1' type='GPU'>
22:02:16:    <paused v='true'/>
22:02:16:  </slot>
22:02:16:  <slot id='2' type='GPU'/>
22:02:16:  <slot id='3' type='GPU'/>
22:02:16:</config>
22:02:21:WARNING:FS01:Killing WU01
22:02:21:WU01:FS01:FahCore returned: INTERRUPTED (102 = 0x66)
22:04:52:WU02:FS02:0x17:Completed 1100000 out of 5000000 steps (22%)
22:05:52:52:127.0.0.1:New Web connection
22:09:43:WU03:FS03:0x17:Completed 1150000 out of 5000000 steps (23%)

keep getting bad submissions.. it wont even start folding on this gpu.. it keeps re-downloading and failing to start then submitting and doing it all over again. I now have over 7 faulty work unit submissions making it near impossible to reach the 90% threshold in the next month :/.

GPU: 270x giga 4gb
driver version: 13.12
Windows 7 x64 fully updated
f@h build 7.4.4
amd fx-8350 (paused)

[Joe_H]

Please also post the initial section of the log that gives system and configuration information. Which version of the video drivers and OpenCL support do you have installed on your system?

[PantherX]

Welcome to the F@H Forum Mastermine,

Could you please update to 14.4 WHQL and see if it resolves your issue or not (https://folding.stanford.edu/home/new-a ... rformance/)? Moreover, I do hope that you are using the latest V7 public release which is V7.4.4

If updating to 14.4 fails, then you could try lowering your GPU frequencies by using Afterburner or PrecisionX to see if the errors were caused by an unstable overclock. Please note that it has been reported that some factory overclocked GPUs were unstable for F@H and the donors had to reduce its frequencies to be able to fold without any issues. Also, I hope that the GPU is operating within normal temperature range.

[Mastermine]

i have a long log i guess... my apologies i cant paste the entire thing heres the top:

Code: Select all

*********************** Log Started 2014-05-10T22:03:28Z ***********************
22:03:28:************************* Folding@home Client *************************
22:03:28:      Website: http://folding.stanford.edu/
22:03:28:    Copyright: (c) 2009-2014 Stanford University
22:03:28:       Author: Joseph Coffland <joseph@cauldrondevelopment.com>
22:03:28:         Args: --open-web-control
22:03:28:       Config: <none>
22:03:28:******************************** Build ********************************
22:03:28:      Version: 7.4.4
22:03:28:         Date: Mar 4 2014
22:03:28:         Time: 20:26:54
22:03:28:      SVN Rev: 4130
22:03:28:       Branch: fah/trunk/client
22:03:28:     Compiler: Intel(R) C++ MSVC 1500 mode 1200
22:03:28:      Options: /TP /nologo /EHa /Qdiag-disable:4297,4103,1786,279 /Ox -arch:SSE
22:03:28:               /QaxSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 /Qopenmp /Qrestrict /MT /Qmkl
22:03:28:     Platform: win32 XP
22:03:28:         Bits: 32
22:03:28:         Mode: Release
22:03:28:******************************* System ********************************
22:03:28:          CPU: AMD FX(tm)-8350 Eight-Core Processor
22:03:28:       CPU ID: AuthenticAMD Family 21 Model 2 Stepping 0
22:03:28:         CPUs: 8
22:03:28:       Memory: 8.00GiB
22:03:28:  Free Memory: 5.42GiB
22:03:28:      Threads: WINDOWS_THREADS
22:03:28:   OS Version: 6.1
22:03:28:  Has Battery: false
22:03:28:   On Battery: false
22:03:28:   UTC Offset: -7
22:03:28:          PID: 6008
22:03:28:          CWD: C:/Users/KaZ/AppData/Roaming/FAHClient
22:03:28:           OS: Windows 7 Professional
22:03:28:      OS Arch: AMD64
22:03:28:         GPUs: 3
22:03:28:        GPU 0: ATI:5 Hawaii [Radeon R9 200 Series]
22:03:28:        GPU 1: ATI:5 Pitcairn []
22:03:28:        GPU 2: ATI:5 Pitcairn [Radeon HD 7800]
22:03:28:         CUDA: Not detected
22:03:28:Win32 Service: false
22:03:28:***********************************************************************

I have trouble with the newer drivers and mining coins (not that its related). but i will try it none the less. thanks for the tip get back to you fairly soon with results.

**edit sorry for confusing replies just saw part of your message.. i do indeed have 3 gpu and 1300w of power. i have not had a power issue with running all 3 gpus mining coins for months now. i have had this gpu working folding just yesterday. I am not sure what has happened since then but this 270x is the only one having problems at the moment. Also i have tried playing with clocks to no change. the 270 is a crap version of this 270x and its doing fine on a crappy 1x pcie riser.

[PantherX]

Can you please clarify as to how many GPUs you have installed in your system? The reason is that you have stated a single 270x but FAHClient detects 3 GPUs:
22:03:28: GPUs: 3
22:03:28: GPU 0: ATI:5 Hawaii [Radeon R9 200 Series]
22:03:28: GPU 1: ATI:5 Pitcairn []
22:03:28: GPU 2: ATI:5 Pitcairn [Radeon HD 7800]

If you indeed have 3 GPUs, then do you have a sufficient PSU (power supply unit)?

[PantherX]

Okay, just to see if it is a PCI-E issue, can you swap the GPU with another one and see if the issue is present or not? Do note that if you do change the position of the GPU, it is recommended that you do a fresh installation of F@H so that auto-detection will work properly. In this case, it means that you will have to set the WUs to finish and once all WUs have folded, you can then uninstall F@H (including the data folder) and then swap the GPUs and then reinstall F@H.

BTW, have you checked if the PCI-E riser is causing the issue or not? Maybe a loose connection?

[Mastermine]

it appears the driver was the issue i guess. installing new drivers seems to have fixed the problem. although my gpu are running extremely sluggish now. Ill just assume thats due to the reset wait for the next work units see how it goes. Thanks for the help, hopefully all is well . Will update if things change for the worse or I cant get speeds up to par. I was getting about 60k pp per gpu does that seem normal range for a 270x/7870/270?

[bruce]

My first guess about what you mean by "running...sluggish" is that you evaluate the sluggisheness based on how long it takes to update your screen rather than some other method of evaluating throughput. (If that's not what you mean, please correct me.)

FAH does it's best to keep all GPUs folding at maximum throughput. Depending on a number of factors, this can cause what we call "screen lag" because your operating system assumes that the GPU is never busy whereas FAH makes sure it is always busy. Work for a GPU is not managed by an operating system which understands priority settings and the screen-update cannot interrupt the work being done by FAH until a certain block of processing is completed -- which often takes long enough for a perceptible pause.

The current method of dealing with that problem is to configure the GPU to which the monitor is attached so that it "folds on idle," This will suspend FAH processing on that GPU whenever you are using the keyboard/mouse and will resume folding whenever you screensaver would have kicked in. That allows you free use of the computer but hides the screen-lag by making sure it only happens when you're not using the computer.

Folding on other GPUs and on your CPU can continue both while you're active and while you're away from the keyboard.

On the other hand, this might become imperceptible if you move your monitor to the fastest GPU.

[Mastermine]

the sluggishness would be the PPD.. there is 72k ppd coming from 270x now? go figure its the fastest one but at least its showing it now... the 270 is at 40k and the 7870 is at 39k? seems strange they were at 60k each earlier.. now i cant get them over 40k.. all 3 gpu are working at 98% useage now and no issues as far as lag.. just bad ppd

[Mastermine]

I guess I was just impatient. 7870 now running at 73k and the 270 running at 69k.. finally doing well thanks for your help and pointing out my stupidity .

[PantherX]

Please note that with a fresh installation, a possible reason for the low PPD is there isn't any historical data for the Client to use to display the information. Thus, it uses the "worst case" information for the initial display (the previous releases wouldn't display any information which made donors think that nothing was working and attempted to "fix" it). However, once it finishes a few frames (IIRC, between 1% to 5%), then the values will be automatically reflected with actual data. While this approach is better than the previous releases, hopefully it can be further improved (https://fah.stanford.edu/projects/FAHClient/ticket/1129).

Do note that on my systems with GTX 660 Ti and GTX 650, what happens is when both GPUs get assigned WUs from the same Project for the first time, the estimates for GTX 660 is used for GTX 650 which leads to an artificial inflation. However, once GTX 650 has folded few frames, the actual estimates are then used. Moreover, if the system is restarted, the estimates for my GPUs are incorrect until the first few frames are folded.

Thus, the solution to the PPD issue is to let the system fold overnight and see what the results are in the morning which would be pretty accurate assuming nothing bad happened overnight (failed Slot, driver reset, etc).

[Mastermine]

makes much more sense now thanks! i really was confused where it was getting its information and how it led to that average. Would certainly be nice to see it be more accurate.. even if its just based on the time between the last 0.01 percent averaged over 0.1 percent or something.. that would at least show results faster and give something to stare at during the first few seconds of wildly adjusting average . to even make a reference model and do this and then set up some sort of "speedometer" of sorts. you are officially going 2.5 750ti's per second or something lol

[bruce]

The client makes the estimates based on the information provided by the FahCore. Since there are quite a number of different FahCores, developed by different authors, the FahClient must use a standardized methodology that is consistent with all FahCores. There is no information available after 0.01 percent or even 0.1 percent. The first reliable information is available after a few frames have been completed, as PantherX has stated and assuming otherwise will lead you to false conclusions, as you've discovered.

The most recently developed client and core can be found under the names NaCl or Chrome and it is more closely integrated and can make more accurate estimates more quickly. The historic client and cores just takes longer for the estimate to stabilize.