Titan Blacks won't fold

[blacckbox]

I've got two brand new cards that the client recognizes. They say they're ready for a few seconds, but then they die. I'm using the EVGA drives that came with the card and did a fresh install of the F@H software. Here's the warnings and errors from the log, any ideas?


*********************** Log Started 2014-07-11T19:15:04Z ***********************
19:16:17:WARNING:WU00:FS02:Changed SMP threads from 7 to 8 this can cause some work units to fail
19:16:18:WU01:FS00:0x17:ERROR:exception: Bad platformId size.
19:16:18:WU02:FS01:0x17:ERROR:exception: Bad platformId size.
19:16:18:WARNING:WU01:FS00:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
19:16:19:WARNING:WU02:FS01:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
19:16:21:WU04:FS01:0x17:ERROR:exception: Bad platformId size.
19:16:21:WARNING:WU04:FS01:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
19:16:22:WU03:FS00:0x17:ERROR:exception: Bad platformId size.
19:16:22:WARNING:WU03:FS00:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
19:16:24:WU01:FS01:0x17:ERROR:exception: Bad platformId size.
19:16:24:WARNING:WU01:FS01:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
19:16:25:WU02:FS00:0x17:ERROR:exception: Bad platformId size.
19:16:25:WARNING:WU02:FS00:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
19:16:26:WU03:FS01:0x17:ERROR:exception: Bad platformId size.
19:16:27:WARNING:WU03:FS01:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
19:16:28:WU01:FS00:0x17:ERROR:exception: Bad platformId size.
19:16:29:WARNING:WU01:FS00:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
19:16:29:WU02:FS01:0x17:ERROR:exception: Bad platformId size.
19:16:29:WARNING:WU02:FS01:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
19:16:31:WU03:FS00:0x17:ERROR:exception: Bad platformId size.
19:16:32:WARNING:WU03:FS00:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
19:16:39:WARNING:Exception: 19:127.0.0.1: Send error: 10053: An established connection was aborted by the software in your host machine.

[bruce]

"Bad platformId size" means that the EVGA drivers do not contain the necessary support that hardware. Go to NVidia's site and download newer drivers.

[P5-133XL]

Two possible causes: You don't have OpenCL included in your video drivers, or You have non-foldable OpenCL device(s) with OpenCL drivers installed (like a CPU or an onboard GPU). The first is solved by downloading and installing drivers directly from Nvidia (not from Microsoft or an OEM driver). The second possible cause is fixed by manually assigning the opencl-index values to point to the correct video cards.

[7im]

And after you get it working, you'll need to reconfigure the SMP slot to reserve one CPU core per GPU slot for best performance. (8 -> 6)

[blacckbox]

Soon after my op I bsod. I updated my bios and windows refused to boot afterward. After reinstalling windows things seem to be ok. I'll install fresh drivers & F@H in the morning and see how it goes. Thanks for the replies.

[blacckbox]

So far things seem to be ok. I'm currently getting just north of 250k ppd. I used 337.88 and not the EVGA drivers that shipped with the cards. Is there a preferred driver version for Titan Blacks?

One weird issue is that the latest GPUZ release reports no OpenCL, CUDA, PhysX, or DirectCompute. I have a GTX580 in another machine and GPUZ checks off all of these... Also, one Titan seems to be lagging behind the other for whatever reason. This is my first dual card set up so I don't know if this is normal or not.

>>reserve one CPU core per GPU slot for best performance. (8 -> 6)
I've been folding for over a decade, but I've never needed to do this.
How does I reduce core count in 7.3.6?

[bruce]

The core count can be adjusted by opening the Advanced Control (FAHControl), selecting Configure + slots + CPU + Edit and changing the number. There's a default already defined which is selected by "-1" but the actual value used depends on several factors. As far as the CPU count is concerned, each Black counts as one GPU.

[blacckbox]

Thanks Bruce. I'm at 6 cores now, but the one Titan doesn't seem to be doing much of anything and ppd has dropped off a bit. This situation might work itself out, but if not I'll check back and pester foldingforum again.

[bruce]

You've probably been assigned a different project. The PPD cannot be accuratedly aligned, particularly for high-end GPUs.

[blacckbox]

Good to know. It seemed natural to me that they would do the exact same thing. One card isn't doing anything at the moment...

[7im]

How about a new log file?

[blacckbox]

This is my first dual card machine and my first liquid cooled setup. It's been a nerve-wracking weekend. After discovering a leak late last night I shut it all down. This morning the leak has been stopped and things seem to be working properly. My ppd was clocking in around 260k last night, now I'm at 150k. GPUZ is still giving me an OpenCL error message, but it might be related to my ivy bridge... *shrugs*

Code: Select all

*********************** Log Started 2014-07-13T13:06:56Z ***********************
13:06:56:************************* Folding@home Client *************************
13:06:56:      Website: http://folding.stanford.edu/
13:06:56:    Copyright: (c) 2009-2013 Stanford University
13:06:56:       Author: Joseph Coffland <joseph@cauldrondevelopment.com>
13:06:56:         Args: 
13:06:56:       Config: C:/ProgramData/FAHClient/config.xml
13:06:56:******************************** Build ********************************
13:06:56:      Version: 7.3.6
13:06:56:         Date: Feb 18 2013
13:06:56:         Time: 15:25:17
13:06:56:      SVN Rev: 3923
13:06:56:       Branch: fah/trunk/client
13:06:56:     Compiler: Intel(R) C++ MSVC 1500 mode 1200
13:06:56:      Options: /TP /nologo /EHa /Qdiag-disable:4297,4103,1786,279 /Ox -arch:SSE
13:06:56:               /QaxSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 /Qopenmp /Qrestrict /MT /Qmkl
13:06:56:     Platform: win32 XP
13:06:56:         Bits: 32
13:06:56:         Mode: Release
13:06:56:******************************* System ********************************
13:06:56:          CPU: Intel(R) Core(TM) i7-3770K CPU @ 3.50GHz
13:06:56:       CPU ID: GenuineIntel Family 6 Model 58 Stepping 9
13:06:56:         CPUs: 8
13:06:56:       Memory: 15.84GiB
13:06:56:  Free Memory: 14.63GiB
13:06:56:      Threads: WINDOWS_THREADS
13:06:56:  Has Battery: false
13:06:56:   On Battery: false
13:06:56:   UTC offset: -7
13:06:56:          PID: 2536
13:06:56:          CWD: C:/ProgramData/FAHClient
13:06:56:           OS: Windows 7 Ultimate
13:06:56:      OS Arch: AMD64
13:06:56:         GPUs: 2
13:06:56:        GPU 0: NVIDIA:3 GK110 [GeForce GTX Titan Black]
13:06:56:        GPU 1: NVIDIA:3 GK110 [GeForce GTX Titan Black]
13:06:56:         CUDA: 3.5
13:06:56:  CUDA Driver: 6000
13:06:56:Win32 Service: false
13:06:56:***********************************************************************
13:06:56:<config>
13:06:56:  <!-- Folding Slot Configuration -->
13:06:56:  <power v='full'/>
13:06:56:
13:06:56:  <!-- Network -->
13:06:56:  <proxy v=':8080'/>
13:06:56:
13:06:56:  <!-- User Information -->
13:06:56:  <passkey v='********************************'/>
13:06:56:  <user v='lottab'/>
13:06:56:
13:06:56:  <!-- Folding Slots -->
13:06:56:  <slot id='0' type='GPU'/>
13:06:56:  <slot id='1' type='GPU'/>
13:06:56:  <slot id='2' type='CPU'>
13:06:56:    <cpus v='6'/>
13:06:56:  </slot>
13:06:56:</config>
13:06:56:Trying to access database...
13:06:56:Successfully acquired database lock
13:06:56:Enabled folding slot 00: READY gpu:0:GK110 [GeForce GTX Titan Black]
13:06:56:Enabled folding slot 01: READY gpu:1:GK110 [GeForce GTX Titan Black]
13:06:56:Enabled folding slot 02: READY cpu:6
13:06:56:WU02:FS02:Starting
13:06:56:WU02:FS02:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/ProgramData/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/Core_a3.fah/FahCore_a3.exe -dir 02 -suffix 01 -version 703 -lifeline 2536 -checkpoint 15 -np 6
13:06:56:WU02:FS02:Started FahCore on PID 3208
13:06:56:WU02:FS02:Core PID:3228
13:06:56:WU02:FS02:FahCore 0xa3 started
13:06:56:WU00:FS02:Sending unit results: id:00 state:SEND error:NO_ERROR project:9008 run:551 clone:0 gen:71 core:0xa4 unit:0x0000004b664f2de453826cb74f48b28a
13:06:56:WU00:FS02:Uploading 1.57MiB to 171.64.65.124
13:06:56:WU03:FS01:Starting
13:06:56:WU00:FS02:Connecting to 171.64.65.124:8080
13:06:56:WU03:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/ProgramData/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_17.fah/FahCore_17.exe -dir 03 -suffix 01 -version 703 -lifeline 2536 -checkpoint 15 -gpu 1 -gpu-vendor nvidia
13:06:56:WU03:FS01:Started FahCore on PID 3248
13:06:56:WU03:FS01:Core PID:3260
13:06:56:WU03:FS01:FahCore 0x17 started
13:06:56:WU01:FS00:Starting
13:06:56:WU01:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/ProgramData/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_17.fah/FahCore_17.exe -dir 01 -suffix 01 -version 703 -lifeline 2536 -checkpoint 15 -gpu 0 -gpu-vendor nvidia
13:06:56:WU01:FS00:Started FahCore on PID 3276
13:06:56:WU01:FS00:Core PID:3292
13:06:56:WU01:FS00:FahCore 0x17 started
13:06:56:WU02:FS02:0xa3:
13:06:56:WU02:FS02:0xa3:*------------------------------*
13:06:56:WU02:FS02:0xa3:Folding@Home Gromacs SMP Core
13:06:56:WU02:FS02:0xa3:Version 2.27 (Dec. 15, 2010)
13:06:56:WU02:FS02:0xa3:
13:06:56:WU02:FS02:0xa3:Preparing to commence simulation
13:06:56:WU02:FS02:0xa3:- Looking at optimizations...
13:06:56:WU02:FS02:0xa3:- Created dyn
13:06:56:WU02:FS02:0xa3:- Files status OK
13:06:56:WU02:FS02:0xa3:- Expanded 3847389 -> 4383072 (decompressed 113.9 percent)
13:06:56:WU02:FS02:0xa3:Called DecompressByteArray: compressed_data_size=3847389 data_size=4383072, decompressed_data_size=4383072 diff=0
13:06:56:WU02:FS02:0xa3:- Digital signature verified
13:06:56:WU02:FS02:0xa3:
13:06:56:WU02:FS02:0xa3:Project: 8567 (Run 1, Clone 5, Gen 491)
13:06:56:WU02:FS02:0xa3:
13:06:56:WU02:FS02:0xa3:Assembly optimizations on if available.
13:06:56:WU02:FS02:0xa3:Entering M.D.
13:06:57:WU03:FS01:0x17:*********************** Log Started 2014-07-13T13:06:56Z ***********************
13:06:57:WU03:FS01:0x17:Project: 10467 (Run 0, Clone 446, Gen 8)
13:06:57:WU03:FS01:0x17:Unit: 0x00000013538b3db9538bc4bca4a0116d
13:06:57:WU03:FS01:0x17:CPU: 0x00000000000000000000000000000000
13:06:57:WU03:FS01:0x17:Machine: 1
13:06:57:WU03:FS01:0x17:Digital signatures verified
13:06:57:WU03:FS01:0x17:Folding@home GPU core17
13:06:57:WU03:FS01:0x17:Version 0.0.52
13:06:57:WU03:FS01:0x17:  Found a checkpoint file
13:06:57:WU01:FS00:0x17:*********************** Log Started 2014-07-13T13:06:56Z ***********************
13:06:57:WU01:FS00:0x17:Project: 9408 (Run 2097, Clone 0, Gen 49)
13:06:57:WU01:FS00:0x17:Unit: 0x000000490a3b1e5c534339de09b8d406
13:06:57:WU01:FS00:0x17:CPU: 0x00000000000000000000000000000000
13:06:57:WU01:FS00:0x17:Machine: 0
13:06:57:WU01:FS00:0x17:Digital signatures verified
13:06:57:WU01:FS00:0x17:Folding@home GPU core17
13:06:57:WU01:FS00:0x17:Version 0.0.52
13:06:57:WU01:FS00:0x17:  Found a checkpoint file
13:07:02:WU00:FS02:Upload 47.76%
13:07:03:WU02:FS02:0xa3:Mapping NT from 6 to 6 
13:07:03:WU02:FS02:0xa3:Completed 0 out of 500000 steps  (0%)
13:07:08:WU00:FS02:Upload 95.52%
13:07:09:WU00:FS02:Upload complete
13:07:09:WU00:FS02:Server responded GOT_ALREADY (434)
13:07:09:WARNING:WU00:FS02:Server did not like results, dumping
13:07:09:WU00:FS02:Cleaning up
13:08:51:WU01:FS00:0x17:Completed 1850000 out of 5000000 steps (37%)
13:08:51:WU01:FS00:0x17:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
13:09:44:WU03:FS01:0x17:Completed 625000 out of 5000000 steps (12%)
13:09:44:WU03:FS01:0x17:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
13:13:40:WU03:FS01:0x17:Completed 650000 out of 5000000 steps (13%)
13:16:44:WU01:FS00:0x17:Completed 1900000 out of 5000000 steps (38%)
13:18:48:WU02:FS02:0xa3:Completed 5000 out of 500000 steps  (1%)

[PantherX]

...My ppd was clocking in around 260k last night, now I'm at 150k...

Please note that if FAHClient has been restarted or a WU from a Project which you haven't previously folded is downloaded, it would need to complete 3% to 5% of the WU before the PPD/TPF is accurate enough to give you a real world approximation.

[7im]

Are the frame times on one slot still double that of the other slot as shown above in the log? What kind of motherboard do you have? What speed are the PCIe slots? How big is the power supply?

[blacckbox]

After a week of chronic leaks I'm eagerly awaiting the arrival of compression tube fittings. Barbs & hose clamps are not the way to go.

When not leaking everywhere the titans seem to be ok, but there are minor differences between them. I would upload a screen cap from gpuz, but my ignorance in this forum is astounding.

Regarding the PCIe slot speeds, they are similar at 16x8 but one is v1.1 and one is v3.0. This makes little sense to me; why would two versions be on one board? Could this significantly slow one down? The TPFs between the Titans are still not equivalent, but they do seem to have converged a bit. At some point I'll have them finish their respective WUs and then reboot in the hopes that they download WUs from the same project. It would be interesting to compare them working on similar WUs.

In my titan box I'm running an ivy bridge 3770k at stock speed on a Intel DZ77GA-70K. This Intel board has always been a bit buggy, but after I updated the bios things seem smoother. However, my 1866 memory sticks are still clocked at 1333. I should have picked up an Asus board.

I've got a pcpower & cooling MkIII 1200 pushing everything in the Titan box, and a pcpower & cooling MkIII 850 in the gtx580 box. These units have worked very well for me.

Why doesn't the software automatically drop a cpu core for each gpu detected?! Per 7im's suggestion a few days back, I subtracted a core in my older Core Q8300 machine. Holy shits: PPD increased 40-50%. I've had a gtx580 folding in that box for the last 3 years or so, unaware that it was essentially crippled. How many other gpus out there are not folding to their maximum ability? I float through the folding forum several times a year, yet this valuable tip slipped past me.