Project: 5766 (Run 0, Clone 49, Gen 406)

[ChrissyT88]

Hi,

Had an EUE problem with this unit - instant NANs detected on GPU and shutdown + pause for 24 hours. Lucky i checked when i did! The log file is shown below.

Code: Select all

[16:22:48] + Processing work unit
[16:22:48] Core required: FahCore_11.exe
[16:22:48] Core found.
[16:22:48] Working on queue slot 03 [April 16 16:22:48 UTC]
[16:22:48] + Working ...
[16:22:48] - Calling '.\FahCore_11.exe -dir work/ -suffix 03 -priority 96 -nocpulock -checkpoint 15 -verbose -lifeline 4496 -version 623'

[16:22:48] 
[16:22:48] *------------------------------*
[16:22:48] Folding@Home GPU Core - Beta
[16:22:48] Version 1.19 (Mon Nov 3 09:34:13 PST 2008)
[16:22:48] 
[16:22:48] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[16:22:48] Build host: amoeba
[16:22:48] Board Type: Nvidia
[16:22:48] Core      : 
[16:22:48] Preparing to commence simulation
[16:22:48] - Looking at optimizations...
[16:22:48] - Created dyn
[16:22:48] - Files status OK
[16:22:48] - Expanded 46735 -> 252912 (decompressed 541.1 percent)
[16:22:48] Called DecompressByteArray: compressed_data_size=46735 data_size=252912, decompressed_data_size=252912 diff=0
[16:22:48] - Digital signature verified
[16:22:48] 
[16:22:48] Project: 5766 (Run 0, Clone 49, Gen 406)
[16:22:48] 
[16:22:48] Assembly optimizations on if available.
[16:22:48] Entering M.D.
[16:22:54] Working on Protein
[16:22:55] Client config found, loading data.
[16:22:55] mdrun_gpu returned 
[16:22:55] NANs detected on GPU
[16:22:55] 
[16:22:55] Folding@home Core Shutdown: UNSTABLE_MACHINE
[16:22:58] CoreStatus = 7A (122)
[16:22:58] Sending work to server
[16:22:58] Project: 5766 (Run 0, Clone 49, Gen 406)
[16:22:58] - Error: Could not get length of results file work/wuresults_03.dat
[16:22:58] - Error: Could not read unit 03 file. Removing from queue.
[16:22:58] Trying to send all finished work units
[16:22:58] + No unsent completed units remaining.
[16:22:58] - Preparing to get new work unit...
[16:22:58] + Attempting to get work packet
[16:22:58] - Will indicate memory of 4094 MB
[16:22:58] - Connecting to assignment server
[16:22:58] Connecting to http://assign-GPU.stanford.edu:8080/
[16:22:59] Posted data.
[16:22:59] Initial: 43AB; - Successful: assigned to (171.67.108.11).
[16:22:59] + News From Folding@Home: GPU folding beta
[16:22:59] Loaded queue successfully.
[16:22:59] Connecting to http://171.67.108.11:8080/
[16:23:00] Posted data.
[16:23:00] Initial: 0000; - Receiving payload (expected size: 47247)
[16:23:00] Conversation time very short, giving reduced weight in bandwidth avg
[16:23:00] - Downloaded at ~92 kB/s
[16:23:00] - Averaged speed for that direction ~115 kB/s
[16:23:00] + Received work.
[16:23:00] Trying to send all finished work units
[16:23:00] + No unsent completed units remaining.
[16:23:00] + Closed connections
[16:23:05] 
[16:23:05] + Processing work unit
[16:23:05] Core required: FahCore_11.exe
[16:23:05] Core found.
[16:23:05] Working on queue slot 04 [April 16 16:23:05 UTC]
[16:23:05] + Working ...
[16:23:05] - Calling '.\FahCore_11.exe -dir work/ -suffix 04 -priority 96 -nocpulock -checkpoint 15 -verbose -lifeline 4496 -version 623'

[16:23:05] 
[16:23:05] *------------------------------*
[16:23:05] Folding@Home GPU Core - Beta
[16:23:05] Version 1.19 (Mon Nov 3 09:34:13 PST 2008)
[16:23:05] 
[16:23:05] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[16:23:05] Build host: amoeba
[16:23:05] Board Type: Nvidia
[16:23:05] Core      : 
[16:23:05] Preparing to commence simulation
[16:23:05] - Looking at optimizations...
[16:23:05] - Created dyn
[16:23:05] - Files status OK
[16:23:05] - Expanded 46735 -> 252912 (decompressed 541.1 percent)
[16:23:05] Called DecompressByteArray: compressed_data_size=46735 data_size=252912, decompressed_data_size=252912 diff=0
[16:23:05] - Digital signature verified
[16:23:05] 
[16:23:05] Project: 5766 (Run 0, Clone 49, Gen 406)
[16:23:05] 
[16:23:05] Assembly optimizations on if available.
[16:23:05] Entering M.D.
[16:23:12] Working on Protein
[16:23:13] Client config found, loading data.
[16:23:13] mdrun_gpu returned 
[16:23:13] NANs detected on GPU
[16:23:13] 
[16:23:13] Folding@home Core Shutdown: UNSTABLE_MACHINE
[16:23:13] Starting GUI Server
[16:23:16] CoreStatus = 7A (122)
[16:23:16] Sending work to server
[16:23:16] Project: 5766 (Run 0, Clone 49, Gen 406)
[16:23:16] - Error: Could not get length of results file work/wuresults_04.dat
[16:23:16] - Error: Could not read unit 04 file. Removing from queue.
[16:23:16] Trying to send all finished work units
[16:23:16] + No unsent completed units remaining.
[16:23:16] - Preparing to get new work unit...
[16:23:16] + Attempting to get work packet
[16:23:16] - Will indicate memory of 4094 MB
[16:23:16] - Connecting to assignment server
[16:23:16] Connecting to http://assign-GPU.stanford.edu:8080/
[16:23:17] Posted data.
[16:23:17] Initial: 43AB; - Successful: assigned to (171.67.108.11).
[16:23:17] + News From Folding@Home: GPU folding beta
[16:23:17] Loaded queue successfully.
[16:23:17] Connecting to http://171.67.108.11:8080/
[16:23:18] Posted data.
[16:23:18] Initial: 0000; - Receiving payload (expected size: 47247)
[16:23:18] Conversation time very short, giving reduced weight in bandwidth avg
[16:23:18] - Downloaded at ~92 kB/s
[16:23:18] - Averaged speed for that direction ~112 kB/s
[16:23:18] + Received work.
[16:23:18] Trying to send all finished work units
[16:23:18] + No unsent completed units remaining.
[16:23:18] + Closed connections
[16:23:23] 
[16:23:23] + Processing work unit
[16:23:23] Core required: FahCore_11.exe
[16:23:23] Core found.
[16:23:23] Working on queue slot 05 [April 16 16:23:23 UTC]
[16:23:23] + Working ...
[16:23:23] - Calling '.\FahCore_11.exe -dir work/ -suffix 05 -priority 96 -nocpulock -checkpoint 15 -verbose -lifeline 4496 -version 623'

[16:23:24] 
[16:23:24] *------------------------------*
[16:23:24] Folding@Home GPU Core - Beta
[16:23:24] Version 1.19 (Mon Nov 3 09:34:13 PST 2008)
[16:23:24] 
[16:23:24] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[16:23:24] Build host: amoeba
[16:23:24] Board Type: Nvidia
[16:23:24] Core      : 
[16:23:24] Preparing to commence simulation
[16:23:24] - Looking at optimizations...
[16:23:24] - Created dyn
[16:23:24] - Files status OK
[16:23:24] - Expanded 46735 -> 252912 (decompressed 541.1 percent)
[16:23:24] Called DecompressByteArray: compressed_data_size=46735 data_size=252912, decompressed_data_size=252912 diff=0
[16:23:24] - Digital signature verified
[16:23:24] 
[16:23:24] Project: 5766 (Run 0, Clone 49, Gen 406)
[16:23:24] 
[16:23:24] Assembly optimizations on if available.
[16:23:24] Entering M.D.
[16:23:30] Working on Protein
[16:23:31] Client config found, loading data.
[16:23:31] mdrun_gpu returned 
[16:23:31] NANs detected on GPU
[16:23:31] 
[16:23:31] Folding@home Core Shutdown: UNSTABLE_MACHINE
[16:23:34] CoreStatus = 7A (122)
[16:23:34] Sending work to server
[16:23:34] Project: 5766 (Run 0, Clone 49, Gen 406)
[16:23:34] - Error: Could not get length of results file work/wuresults_05.dat
[16:23:34] - Error: Could not read unit 05 file. Removing from queue.
[16:23:34] Trying to send all finished work units
[16:23:34] + No unsent completed units remaining.
[16:23:34] - Preparing to get new work unit...
[16:23:34] + Attempting to get work packet
[16:23:34] - Will indicate memory of 4094 MB
[16:23:34] - Connecting to assignment server
[16:23:34] Connecting to http://assign-GPU.stanford.edu:8080/
[16:23:34] Posted data.
[16:23:34] Initial: 43AB; - Successful: assigned to (171.67.108.11).
[16:23:34] + News From Folding@Home: GPU folding beta
[16:23:35] Loaded queue successfully.
[16:23:35] Connecting to http://171.67.108.11:8080/
[16:23:36] Posted data.
[16:23:36] Initial: 0000; - Receiving payload (expected size: 47247)
[16:23:36] Conversation time very short, giving reduced weight in bandwidth avg
[16:23:36] - Downloaded at ~92 kB/s
[16:23:36] - Averaged speed for that direction ~110 kB/s
[16:23:36] + Received work.
[16:23:36] Trying to send all finished work units
[16:23:36] + No unsent completed units remaining.
[16:23:36] + Closed connections
[16:23:41] 
[16:23:41] + Processing work unit
[16:23:41] Core required: FahCore_11.exe
[16:23:41] Core found.
[16:23:41] Working on queue slot 06 [April 16 16:23:41 UTC]
[16:23:41] + Working ...
[16:23:41] - Calling '.\FahCore_11.exe -dir work/ -suffix 06 -priority 96 -nocpulock -checkpoint 15 -verbose -lifeline 4496 -version 623'

[16:23:41] 
[16:23:41] *------------------------------*
[16:23:41] Folding@Home GPU Core - Beta
[16:23:41] Version 1.19 (Mon Nov 3 09:34:13 PST 2008)
[16:23:41] 
[16:23:41] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[16:23:41] Build host: amoeba
[16:23:41] Board Type: Nvidia
[16:23:41] Core      : 
[16:23:41] Preparing to commence simulation
[16:23:41] - Looking at optimizations...
[16:23:41] - Created dyn
[16:23:41] - Files status OK
[16:23:41] - Expanded 46735 -> 252912 (decompressed 541.1 percent)
[16:23:41] Called DecompressByteArray: compressed_data_size=46735 data_size=252912, decompressed_data_size=252912 diff=0
[16:23:41] - Digital signature verified
[16:23:41] 
[16:23:41] Project: 5766 (Run 0, Clone 49, Gen 406)
[16:23:41] 
[16:23:41] Assembly optimizations on if available.
[16:23:41] Entering M.D.
[16:23:48] Working on Protein
[16:23:49] Client config found, loading data.
[16:23:49] mdrun_gpu returned 
[16:23:49] NANs detected on GPU
[16:23:49] 
[16:23:49] Folding@home Core Shutdown: UNSTABLE_MACHINE
[16:23:49] Starting GUI Server
[16:23:51] CoreStatus = 7A (122)
[16:23:51] Sending work to server
[16:23:51] Project: 5766 (Run 0, Clone 49, Gen 406)
[16:23:51] - Error: Could not get length of results file work/wuresults_06.dat
[16:23:51] - Error: Could not read unit 06 file. Removing from queue.
[16:23:51] Trying to send all finished work units
[16:23:51] + No unsent completed units remaining.
[16:23:51] - Preparing to get new work unit...
[16:23:51] + Attempting to get work packet
[16:23:51] - Will indicate memory of 4094 MB
[16:23:51] - Connecting to assignment server
[16:23:51] Connecting to http://assign-GPU.stanford.edu:8080/
[16:23:52] Posted data.
[16:23:52] Initial: 43AB; - Successful: assigned to (171.67.108.11).
[16:23:52] + News From Folding@Home: GPU folding beta
[16:23:53] Loaded queue successfully.
[16:23:53] Connecting to http://171.67.108.11:8080/
[16:23:54] Posted data.
[16:23:54] Initial: 0000; - Receiving payload (expected size: 47247)
[16:23:54] Conversation time very short, giving reduced weight in bandwidth avg
[16:23:54] - Downloaded at ~92 kB/s
[16:23:54] - Averaged speed for that direction ~108 kB/s
[16:23:54] + Received work.
[16:23:54] Trying to send all finished work units
[16:23:54] + No unsent completed units remaining.
[16:23:54] + Closed connections
[16:23:59] 
[16:23:59] + Processing work unit
[16:23:59] Core required: FahCore_11.exe
[16:23:59] Core found.
[16:23:59] Working on queue slot 07 [April 16 16:23:59 UTC]
[16:23:59] + Working ...
[16:23:59] - Calling '.\FahCore_11.exe -dir work/ -suffix 07 -priority 96 -nocpulock -checkpoint 15 -verbose -lifeline 4496 -version 623'

[16:23:59] 
[16:23:59] *------------------------------*
[16:23:59] Folding@Home GPU Core - Beta
[16:23:59] Version 1.19 (Mon Nov 3 09:34:13 PST 2008)
[16:23:59] 
[16:23:59] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[16:23:59] Build host: amoeba
[16:23:59] Board Type: Nvidia
[16:23:59] Core      : 
[16:23:59] Preparing to commence simulation
[16:23:59] - Looking at optimizations...
[16:23:59] - Created dyn
[16:23:59] - Files status OK
[16:24:00] - Expanded 46735 -> 252912 (decompressed 541.1 percent)
[16:24:00] Called DecompressByteArray: compressed_data_size=46735 data_size=252912, decompressed_data_size=252912 diff=0
[16:24:00] - Digital signature verified
[16:24:00] 
[16:24:00] Project: 5766 (Run 0, Clone 49, Gen 406)
[16:24:00] 
[16:24:00] Assembly optimizations on if available.
[16:24:00] Entering M.D.
[16:24:06] Working on Protein
[16:24:07] Client config found, loading data.
[16:24:07] mdrun_gpu returned 
[16:24:07] NANs detected on GPU
[16:24:07] 
[16:24:07] Folding@home Core Shutdown: UNSTABLE_MACHINE
[16:24:07] Starting GUI Server
[16:24:09] CoreStatus = 7A (122)
[16:24:09] Sending work to server
[16:24:09] Project: 5766 (Run 0, Clone 49, Gen 406)
[16:24:09] - Error: Could not get length of results file work/wuresults_07.dat
[16:24:09] - Error: Could not read unit 07 file. Removing from queue.
[16:24:09] EUE limit exceeded. Pausing 24 hours.
[16:38:25] ***** Got a SIGTERM signal (2)
[16:38:25] Killing all core threads

Folding@Home Client Shutdown.

Havent seen an EUE in ages, so this is a bit out of the blue. I will delete the WU and carry on. What with Windows update restarting the machine this morning by itself and now an EUE, today has not been a productive day!

Cheers,

Chris

EDIT: Forgot to mention system specs. GTX280 w/ 50Mhz overclock on the shaders (never been a problem), Q6600 @ 3.3Ghz, Vista 64, 181.20 CUDA driver

[toTOW]

There's no data for this WU in the DB yet ...

[ChrissyT88]

Thanks for looking toTow.

[toTOW]

I've just checked again, and the WU had been completed successfully by two donors.

[ChrissyT88]

Hmmm... Thats a bit of a spanner in the works. I'll keep an eye on the client - hopefully it will be a one off, but i have no idea what caused it. Thanks again.

[ChrissyT88]

Not so sure this is a one off now. Got another EUE this morning not long after typing that message above. Heres the lgo:

Code: Select all

[08:56:16] Completed 88%
[08:57:03] Completed 89%
[08:57:50] Completed 90%
[08:58:37] Completed 91%
[08:59:23] Completed 92%
[09:00:09] Completed 93%
[09:00:54] Completed 94%
[09:01:39] Completed 95%
[09:02:25] Completed 96%
[09:03:11] Completed 97%
[09:03:57] Completed 98%
[09:04:43] Completed 99%
[09:05:29] Completed 100%
[09:05:29] Successful run
[09:05:29] DynamicWrapper: Finished Work Unit: sleep=10000
[09:05:39] Reserved 78644 bytes for xtc file; Cosm status=0
[09:05:39] Allocated 78644 bytes for xtc file
[09:05:39] - Reading up to 78644 from "work/wudata_04.xtc": Read 78644
[09:05:39] Read 78644 bytes from xtc file; available packet space=786351820
[09:05:39] xtc file hash check passed.
[09:05:39] Reserved 23472 23472 786351820 bytes for arc file=<work/wudata_04.trr> Cosm status=0
[09:05:39] Allocated 23472 bytes for arc file
[09:05:39] - Reading up to 23472 from "work/wudata_04.trr": Read 23472
[09:05:39] Read 23472 bytes from arc file; available packet space=786328348
[09:05:39] trr file hash check passed.
[09:05:39] Allocated 560 bytes for edr file
[09:05:39] Read bedfile
[09:05:39] edr file hash check passed.
[09:05:39] Allocated 16663 bytes for logfile
[09:05:39] Read logfile
[09:05:39] GuardedRun: success in DynamicWrapper
[09:05:39] GuardedRun: done
[09:05:39] Run: GuardedRun completed.
[09:05:43] - Writing 119851 bytes of core data to disk...
[09:05:43] Done: 119339 -> 108456 (compressed to 90.8 percent)
[09:05:43]   ... Done.
[09:05:43] - Shutting down core 
[09:05:43] 
[09:05:43] Folding@home Core Shutdown: FINISHED_UNIT
[09:05:47] CoreStatus = 64 (100)
[09:05:47] Unit 4 finished with 86 percent of time to deadline remaining.
[09:05:47] Updated performance fraction: 0.947859
[09:05:47] Sending work to server
[09:05:47] Project: 5757 (Run 12, Clone 96, Gen 133)


[09:05:47] + Attempting to send results [April 17 09:05:47 UTC]
[09:05:47] - Reading file work/wuresults_04.dat from core
[09:05:47]   (Read 108968 bytes from disk)
[09:05:47] Connecting to http://171.64.65.106:8080/
[09:05:53] Posted data.
[09:05:53] Initial: 0000; - Uploaded at ~17 kB/s
[09:05:53] - Averaged speed for that direction ~18 kB/s
[09:05:53] + Results successfully sent
[09:05:53] Thank you for your contribution to Folding@Home.
[09:05:53] + Number of Units Completed: 96

[09:05:57] Trying to send all finished work units
[09:05:57] + No unsent completed units remaining.
[09:05:57] - Preparing to get new work unit...
[09:05:57] + Attempting to get work packet
[09:05:57] - Will indicate memory of 4094 MB
[09:05:57] - Detect CPU. Vendor: GenuineIntel, Family: 6, Model: 15, Stepping: 11
[09:05:57] - Connecting to assignment server
[09:05:57] Connecting to http://assign-GPU.stanford.edu:8080/
[09:05:58] Posted data.
[09:05:58] Initial: 43AB; - Successful: assigned to (171.67.108.11).
[09:05:58] + News From Folding@Home: GPU folding beta
[09:05:58] Loaded queue successfully.
[09:05:58] Connecting to http://171.67.108.11:8080/
[09:05:59] Posted data.
[09:05:59] Initial: 0000; - Receiving payload (expected size: 47202)
[09:05:59] Conversation time very short, giving reduced weight in bandwidth avg
[09:05:59] - Downloaded at ~92 kB/s
[09:05:59] - Averaged speed for that direction ~54 kB/s
[09:05:59] + Received work.
[09:05:59] Trying to send all finished work units
[09:05:59] + No unsent completed units remaining.
[09:05:59] + Closed connections
[09:05:59] 
[09:05:59] + Processing work unit
[09:05:59] Core required: FahCore_11.exe
[09:05:59] Core found.
[09:05:59] Working on queue slot 05 [April 17 09:05:59 UTC]
[09:05:59] + Working ...
[09:05:59] - Calling '.\FahCore_11.exe -dir work/ -suffix 05 -priority 96 -nocpulock -checkpoint 15 -verbose -lifeline 3656 -version 623'

[09:05:59] 
[09:05:59] *------------------------------*
[09:05:59] Folding@Home GPU Core - Beta
[09:05:59] Version 1.19 (Mon Nov 3 09:34:13 PST 2008)
[09:05:59] 
[09:05:59] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[09:05:59] Build host: amoeba
[09:05:59] Board Type: Nvidia
[09:05:59] Core      : 
[09:05:59] Preparing to commence simulation
[09:05:59] - Looking at optimizations...
[09:05:59] - Created dyn
[09:05:59] - Files status OK
[09:05:59] - Expanded 46690 -> 252912 (decompressed 541.6 percent)
[09:05:59] Called DecompressByteArray: compressed_data_size=46690 data_size=252912, decompressed_data_size=252912 diff=0
[09:05:59] - Digital signature verified
[09:05:59] 
[09:05:59] Project: 5767 (Run 4, Clone 240, Gen 343)
[09:05:59] 
[09:05:59] Assembly optimizations on if available.
[09:05:59] Entering M.D.
[09:06:06] Working on Protein
[09:06:06] Client config found, loading data.
[09:06:06] Starting GUI Server
[09:06:07] mdrun_gpu returned 
[09:06:07] NANs detected on GPU
[09:06:07] 
[09:06:07] Folding@home Core Shutdown: UNSTABLE_MACHINE
[09:06:10] CoreStatus = 7A (122)
[09:06:10] Sending work to server
[09:06:10] Project: 5767 (Run 4, Clone 240, Gen 343)
[09:06:10] - Error: Could not get length of results file work/wuresults_05.dat
[09:06:10] - Error: Could not read unit 05 file. Removing from queue.
[09:06:10] Trying to send all finished work units
[09:06:10] + No unsent completed units remaining.
[09:06:10] - Preparing to get new work unit...
[09:06:10] + Attempting to get work packet
[09:06:10] - Will indicate memory of 4094 MB
[09:06:10] - Connecting to assignment server
[09:06:10] Connecting to http://assign-GPU.stanford.edu:8080/
[09:06:11] Posted data.
[09:06:11] Initial: 43AB; - Successful: assigned to (171.67.108.11).
[09:06:11] + News From Folding@Home: GPU folding beta
[09:06:11] Loaded queue successfully.
[09:06:11] Connecting to http://171.67.108.11:8080/
[09:06:12] Posted data.
[09:06:12] Initial: 0000; - Receiving payload (expected size: 47202)
[09:06:12] Conversation time very short, giving reduced weight in bandwidth avg
[09:06:12] - Downloaded at ~92 kB/s
[09:06:12] - Averaged speed for that direction ~59 kB/s
[09:06:12] + Received work.
[09:06:12] Trying to send all finished work units
[09:06:12] + No unsent completed units remaining.
[09:06:12] + Closed connections
[09:06:17] 
[09:06:17] + Processing work unit
[09:06:17] Core required: FahCore_11.exe
[09:06:17] Core found.
[09:06:17] Working on queue slot 06 [April 17 09:06:17 UTC]
[09:06:17] + Working ...
[09:06:17] - Calling '.\FahCore_11.exe -dir work/ -suffix 06 -priority 96 -nocpulock -checkpoint 15 -verbose -lifeline 3656 -version 623'

[09:06:17] 
[09:06:17] *------------------------------*
[09:06:17] Folding@Home GPU Core - Beta
[09:06:17] Version 1.19 (Mon Nov 3 09:34:13 PST 2008)
[09:06:17] 
[09:06:17] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[09:06:17] Build host: amoeba
[09:06:17] Board Type: Nvidia
[09:06:17] Core      : 
[09:06:17] Preparing to commence simulation
[09:06:17] - Looking at optimizations...
[09:06:17] - Created dyn
[09:06:17] - Files status OK
[09:06:17] - Expanded 46690 -> 252912 (decompressed 541.6 percent)
[09:06:17] Called DecompressByteArray: compressed_data_size=46690 data_size=252912, decompressed_data_size=252912 diff=0
[09:06:17] - Digital signature verified
[09:06:17] 
[09:06:17] Project: 5767 (Run 4, Clone 240, Gen 343)
[09:06:17] 
[09:06:17] Assembly optimizations on if available.
[09:06:17] Entering M.D.
[09:06:24] Working on Protein
[09:06:24] Client config found, loading data.
[09:06:24] mdrun_gpu returned 
[09:06:24] NANs detected on GPU
[09:06:24] 
[09:06:24] Folding@home Core Shutdown: UNSTABLE_MACHINE
[09:06:24] Starting GUI Server
[09:06:28] CoreStatus = 7A (122)
[09:06:28] Sending work to server
[09:06:28] Project: 5767 (Run 4, Clone 240, Gen 343)
[09:06:28] - Error: Could not get length of results file work/wuresults_06.dat
[09:06:28] - Error: Could not read unit 06 file. Removing from queue.
[09:06:28] Trying to send all finished work units
[09:06:28] + No unsent completed units remaining.
[09:06:28] - Preparing to get new work unit...
[09:06:28] + Attempting to get work packet
[09:06:28] - Will indicate memory of 4094 MB
[09:06:28] - Connecting to assignment server
[09:06:28] Connecting to http://assign-GPU.stanford.edu:8080/
[09:06:28] Posted data.
[09:06:28] Initial: 43AB; - Successful: assigned to (171.67.108.11).
[09:06:28] + News From Folding@Home: GPU folding beta
[09:06:29] Loaded queue successfully.
[09:06:29] Connecting to http://171.67.108.11:8080/
[09:06:30] Posted data.
[09:06:30] Initial: 0000; - Receiving payload (expected size: 47202)
[09:06:30] Conversation time very short, giving reduced weight in bandwidth avg
[09:06:30] - Downloaded at ~92 kB/s
[09:06:30] - Averaged speed for that direction ~62 kB/s
[09:06:30] + Received work.
[09:06:30] Trying to send all finished work units
[09:06:30] + No unsent completed units remaining.
[09:06:30] + Closed connections
[09:06:35] 
[09:06:35] + Processing work unit
[09:06:35] Core required: FahCore_11.exe
[09:06:35] Core found.
[09:06:35] Working on queue slot 07 [April 17 09:06:35 UTC]
[09:06:35] + Working ...
[09:06:35] - Calling '.\FahCore_11.exe -dir work/ -suffix 07 -priority 96 -nocpulock -checkpoint 15 -verbose -lifeline 3656 -version 623'

[09:06:35] 
[09:06:35] *------------------------------*
[09:06:35] Folding@Home GPU Core - Beta
[09:06:35] Version 1.19 (Mon Nov 3 09:34:13 PST 2008)
[09:06:35] 
[09:06:35] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[09:06:35] Build host: amoeba
[09:06:35] Board Type: Nvidia
[09:06:35] Core      : 
[09:06:35] Preparing to commence simulation
[09:06:35] - Looking at optimizations...
[09:06:35] - Created dyn
[09:06:35] - Files status OK
[09:06:35] - Expanded 46690 -> 252912 (decompressed 541.6 percent)
[09:06:35] Called DecompressByteArray: compressed_data_size=46690 data_size=252912, decompressed_data_size=252912 diff=0
[09:06:35] - Digital signature verified
[09:06:35] 
[09:06:35] Project: 5767 (Run 4, Clone 240, Gen 343)
[09:06:35] 
[09:06:35] Assembly optimizations on if available.
[09:06:35] Entering M.D.
[09:06:42] Working on Protein
[09:06:42] Client config found, loading data.
[09:06:42] mdrun_gpu returned 
[09:06:42] NANs detected on GPU
[09:06:42] 
[09:06:42] Folding@home Core Shutdown: UNSTABLE_MACHINE
[09:06:45] CoreStatus = 7A (122)
[09:06:45] Sending work to server
[09:06:45] Project: 5767 (Run 4, Clone 240, Gen 343)
[09:06:45] - Error: Could not get length of results file work/wuresults_07.dat
[09:06:45] - Error: Could not read unit 07 file. Removing from queue.
[09:06:45] Trying to send all finished work units
[09:06:45] + No unsent completed units remaining.
[09:06:45] - Preparing to get new work unit...
[09:06:45] + Attempting to get work packet
[09:06:45] - Will indicate memory of 4094 MB
[09:06:45] - Connecting to assignment server
[09:06:45] Connecting to http://assign-GPU.stanford.edu:8080/
[09:06:46] Posted data.
[09:06:46] Initial: 43AB; - Successful: assigned to (171.67.108.11).
[09:06:46] + News From Folding@Home: GPU folding beta
[09:06:46] Loaded queue successfully.
[09:06:46] Connecting to http://171.67.108.11:8080/
[09:06:48] Posted data.
[09:06:48] Initial: 0000; - Receiving payload (expected size: 47202)
[09:06:48] Conversation time very short, giving reduced weight in bandwidth avg
[09:06:48] - Downloaded at ~92 kB/s
[09:06:48] - Averaged speed for that direction ~66 kB/s
[09:06:48] + Received work.
[09:06:48] Trying to send all finished work units
[09:06:48] + No unsent completed units remaining.
[09:06:48] + Closed connections
[09:06:53] 
[09:06:53] + Processing work unit
[09:06:53] Core required: FahCore_11.exe
[09:06:53] Core found.
[09:06:53] Working on queue slot 08 [April 17 09:06:53 UTC]
[09:06:53] + Working ...
[09:06:53] - Calling '.\FahCore_11.exe -dir work/ -suffix 08 -priority 96 -nocpulock -checkpoint 15 -verbose -lifeline 3656 -version 623'

[09:06:53] 
[09:06:53] *------------------------------*
[09:06:53] Folding@Home GPU Core - Beta
[09:06:53] Version 1.19 (Mon Nov 3 09:34:13 PST 2008)
[09:06:53] 
[09:06:53] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[09:06:53] Build host: amoeba
[09:06:53] Board Type: Nvidia
[09:06:53] Core      : 
[09:06:53] Preparing to commence simulation
[09:06:53] - Looking at optimizations...
[09:06:53] - Created dyn
[09:06:53] - Files status OK
[09:06:53] - Expanded 46690 -> 252912 (decompressed 541.6 percent)
[09:06:53] Called DecompressByteArray: compressed_data_size=46690 data_size=252912, decompressed_data_size=252912 diff=0
[09:06:53] - Digital signature verified
[09:06:53] 
[09:06:53] Project: 5767 (Run 4, Clone 240, Gen 343)
[09:06:53] 
[09:06:53] Assembly optimizations on if available.
[09:06:53] Entering M.D.
[09:07:00] Working on Protein
[09:07:00] Client config found, loading data.
[09:07:01] Starting GUI Server
[09:07:01] mdrun_gpu returned 
[09:07:01] NANs detected on GPU
[09:07:01] 
[09:07:01] Folding@home Core Shutdown: UNSTABLE_MACHINE
[09:07:03] CoreStatus = 7A (122)
[09:07:03] Sending work to server
[09:07:03] Project: 5767 (Run 4, Clone 240, Gen 343)
[09:07:03] - Error: Could not get length of results file work/wuresults_08.dat
[09:07:03] - Error: Could not read unit 08 file. Removing from queue.
[09:07:03] Trying to send all finished work units
[09:07:03] + No unsent completed units remaining.
[09:07:03] - Preparing to get new work unit...
[09:07:03] + Attempting to get work packet
[09:07:03] - Will indicate memory of 4094 MB
[09:07:03] - Connecting to assignment server
[09:07:03] Connecting to http://assign-GPU.stanford.edu:8080/
[09:07:04] Posted data.
[09:07:04] Initial: 43AB; - Successful: assigned to (171.67.108.11).
[09:07:04] + News From Folding@Home: GPU folding beta
[09:07:04] Loaded queue successfully.
[09:07:04] Connecting to http://171.67.108.11:8080/
[09:07:07] Posted data.
[09:07:07] Initial: 0000; - Receiving payload (expected size: 47202)
[09:07:07] Conversation time very short, giving reduced weight in bandwidth avg
[09:07:07] - Downloaded at ~92 kB/s
[09:07:07] - Averaged speed for that direction ~68 kB/s
[09:07:07] + Received work.
[09:07:07] Trying to send all finished work units
[09:07:07] + No unsent completed units remaining.
[09:07:07] + Closed connections
[09:07:12] 
[09:07:12] + Processing work unit
[09:07:12] Core required: FahCore_11.exe
[09:07:12] Core found.
[09:07:12] Working on queue slot 09 [April 17 09:07:12 UTC]
[09:07:12] + Working ...
[09:07:12] - Calling '.\FahCore_11.exe -dir work/ -suffix 09 -priority 96 -nocpulock -checkpoint 15 -verbose -lifeline 3656 -version 623'

[09:07:12] 
[09:07:12] *------------------------------*
[09:07:12] Folding@Home GPU Core - Beta
[09:07:12] Version 1.19 (Mon Nov 3 09:34:13 PST 2008)
[09:07:12] 
[09:07:12] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[09:07:12] Build host: amoeba
[09:07:12] Board Type: Nvidia
[09:07:12] Core      : 
[09:07:12] Preparing to commence simulation
[09:07:12] - Looking at optimizations...
[09:07:12] - Created dyn
[09:07:12] - Files status OK
[09:07:12] - Expanded 46690 -> 252912 (decompressed 541.6 percent)
[09:07:12] Called DecompressByteArray: compressed_data_size=46690 data_size=252912, decompressed_data_size=252912 diff=0
[09:07:12] - Digital signature verified
[09:07:12] 
[09:07:12] Project: 5767 (Run 4, Clone 240, Gen 343)
[09:07:12] 
[09:07:12] Assembly optimizations on if available.
[09:07:12] Entering M.D.
[09:07:18] Working on Protein
[09:07:19] Client config found, loading data.
[09:07:19] mdrun_gpu returned 
[09:07:19] NANs detected on GPU
[09:07:19] 
[09:07:19] Folding@home Core Shutdown: UNSTABLE_MACHINE
[09:07:22] CoreStatus = 7A (122)
[09:07:22] Sending work to server
[09:07:22] Project: 5767 (Run 4, Clone 240, Gen 343)
[09:07:22] - Error: Could not get length of results file work/wuresults_09.dat
[09:07:22] - Error: Could not read unit 09 file. Removing from queue.
[09:07:22] EUE limit exceeded. Pausing 24 hours.

I couldnt see this project number entered elsewhere in this section either. I'll continue to keep a close eye on the client and watch out for errors. Its odd as its been EUE free since just after Christmas when the card was installed.

EDIT: And another. I will reset the card back to stock speeds now - seems to be a bit too much of a coincidence for three to fail. Its strange as the 353 point/384 point/511 point units were all stable at a higher clockspeed before the introduction of the 5903/4 etc. Maybe these newer units are more OC sensitive?

Code: Select all

[11:37:35] Completed 92%
[11:38:26] Completed 93%
[11:39:18] Completed 94%
[11:40:10] Completed 95%
[11:41:02] Completed 96%
[11:41:54] Completed 97%
[11:42:46] Completed 98%
[11:43:38] Completed 99%
[11:44:30] Completed 100%
[11:44:30] Successful run
[11:44:30] DynamicWrapper: Finished Work Unit: sleep=10000
[11:44:40] Reserved 79264 bytes for xtc file; Cosm status=0
[11:44:40] Allocated 79264 bytes for xtc file
[11:44:40] - Reading up to 79264 from "work/wudata_01.xtc": Read 79264
[11:44:40] Read 79264 bytes from xtc file; available packet space=786351200
[11:44:40] xtc file hash check passed.
[11:44:40] Reserved 23472 23472 786351200 bytes for arc file=<work/wudata_01.trr> Cosm status=0
[11:44:40] Allocated 23472 bytes for arc file
[11:44:40] - Reading up to 23472 from "work/wudata_01.trr": Read 23472
[11:44:40] Read 23472 bytes from arc file; available packet space=786327728
[11:44:40] trr file hash check passed.
[11:44:40] Allocated 560 bytes for edr file
[11:44:40] Read bedfile
[11:44:40] edr file hash check passed.
[11:44:40] Allocated 15659 bytes for logfile
[11:44:40] Read logfile
[11:44:40] GuardedRun: success in DynamicWrapper
[11:44:40] GuardedRun: done
[11:44:40] Run: GuardedRun completed.
[11:44:44] - Writing 119467 bytes of core data to disk...
[11:44:44] Done: 118955 -> 108893 (compressed to 91.5 percent)
[11:44:44]   ... Done.
[11:44:44] - Shutting down core 
[11:44:44] 
[11:44:44] Folding@home Core Shutdown: FINISHED_UNIT
[11:44:48] CoreStatus = 64 (100)
[11:44:48] Unit 1 finished with 98 percent of time to deadline remaining.
[11:44:48] Updated performance fraction: 0.982045
[11:44:48] Sending work to server
[11:44:48] Project: 5758 (Run 8, Clone 50, Gen 132)


[11:44:48] + Attempting to send results [April 17 11:44:48 UTC]
[11:44:48] - Reading file work/wuresults_01.dat from core
[11:44:48]   (Read 109405 bytes from disk)
[11:44:48] Connecting to http://171.64.65.106:8080/
[11:44:57] Posted data.
[11:44:57] Initial: 0000; - Uploaded at ~11 kB/s
[11:44:57] - Averaged speed for that direction ~11 kB/s
[11:44:57] + Results successfully sent
[11:44:57] Thank you for your contribution to Folding@Home.
[11:44:57] + Number of Units Completed: 97

[11:45:01] Trying to send all finished work units
[11:45:01] + No unsent completed units remaining.
[11:45:01] - Preparing to get new work unit...
[11:45:01] + Attempting to get work packet
[11:45:01] - Will indicate memory of 4094 MB
[11:45:01] - Connecting to assignment server
[11:45:01] Connecting to http://assign-GPU.stanford.edu:8080/
[11:45:04] Posted data.
[11:45:04] Initial: 43AB; - Successful: assigned to (171.67.108.11).
[11:45:04] + News From Folding@Home: GPU folding beta
[11:45:04] Loaded queue successfully.
[11:45:04] Connecting to http://171.67.108.11:8080/
[11:45:10] Posted data.
[11:45:10] Initial: 0000; - Receiving payload (expected size: 47202)
[11:45:10] Conversation time very short, giving reduced weight in bandwidth avg
[11:45:10] - Downloaded at ~92 kB/s
[11:45:10] - Averaged speed for that direction ~122 kB/s
[11:45:10] + Received work.
[11:45:10] Trying to send all finished work units
[11:45:10] + No unsent completed units remaining.
[11:45:10] + Closed connections
[11:45:10] 
[11:45:10] + Processing work unit
[11:45:10] Core required: FahCore_11.exe
[11:45:10] Core found.
[11:45:10] Working on queue slot 02 [April 17 11:45:10 UTC]
[11:45:10] + Working ...
[11:45:10] - Calling '.\FahCore_11.exe -dir work/ -suffix 02 -priority 96 -nocpulock -checkpoint 15 -verbose -lifeline 3492 -version 623'

[11:45:10] 
[11:45:10] *------------------------------*
[11:45:10] Folding@Home GPU Core - Beta
[11:45:10] Version 1.19 (Mon Nov 3 09:34:13 PST 2008)
[11:45:10] 
[11:45:10] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[11:45:10] Build host: amoeba
[11:45:10] Board Type: Nvidia
[11:45:10] Core      : 
[11:45:10] Preparing to commence simulation
[11:45:10] - Looking at optimizations...
[11:45:10] - Created dyn
[11:45:10] - Files status OK
[11:45:10] - Expanded 46690 -> 252912 (decompressed 541.6 percent)
[11:45:10] Called DecompressByteArray: compressed_data_size=46690 data_size=252912, decompressed_data_size=252912 diff=0
[11:45:10] - Digital signature verified
[11:45:10] 
[11:45:10] Project: 5767 (Run 4, Clone 240, Gen 343)
[11:45:10] 
[11:45:10] Assembly optimizations on if available.
[11:45:10] Entering M.D.
[11:45:16] Working on Protein
[11:45:17] Client config found, loading data.
[11:45:17] Starting GUI Server
[11:45:17] mdrun_gpu returned 
[11:45:17] NANs detected on GPU
[11:45:17] 
[11:45:17] Folding@home Core Shutdown: UNSTABLE_MACHINE
[11:45:20] CoreStatus = 7A (122)
[11:45:20] Sending work to server
[11:45:20] Project: 5767 (Run 4, Clone 240, Gen 343)
[11:45:20] - Error: Could not get length of results file work/wuresults_02.dat
[11:45:20] - Error: Could not read unit 02 file. Removing from queue.
[11:45:20] Trying to send all finished work units
[11:45:20] + No unsent completed units remaining.
[11:45:20] - Preparing to get new work unit...
[11:45:20] + Attempting to get work packet
[11:45:20] - Will indicate memory of 4094 MB
[11:45:20] - Connecting to assignment server
[11:45:20] Connecting to http://assign-GPU.stanford.edu:8080/
[11:45:25] Posted data.
[11:45:25] Initial: 43AB; - Successful: assigned to (171.67.108.11).
[11:45:25] + News From Folding@Home: GPU folding beta
[11:45:25] Loaded queue successfully.
[11:45:25] Connecting to http://171.67.108.11:8080/
[11:45:28] Posted data.
[11:45:28] Initial: 0000; - Error: Bad packet type from server, expected work assignment
[11:45:28] - Attempt #1  to get work failed, and no other work to do.
Waiting before retry.
[11:45:41] + Attempting to get work packet
[11:45:41] - Will indicate memory of 4094 MB
[11:45:41] - Connecting to assignment server
[11:45:41] Connecting to http://assign-GPU.stanford.edu:8080/
[11:45:44] Posted data.
[11:45:44] Initial: 43AB; - Successful: assigned to (171.67.108.11).
[11:45:44] + News From Folding@Home: GPU folding beta
[11:45:44] Loaded queue successfully.
[11:45:44] Connecting to http://171.67.108.11:8080/
[11:45:49] Posted data.
[11:45:49] Initial: 0000; - Receiving payload (expected size: 47270)
[11:45:49] Conversation time very short, giving reduced weight in bandwidth avg
[11:45:49] - Downloaded at ~92 kB/s
[11:45:49] - Averaged speed for that direction ~116 kB/s
[11:45:49] + Received work.
[11:45:49] Trying to send all finished work units
[11:45:49] + No unsent completed units remaining.
[11:45:49] + Closed connections
[11:45:54] 
[11:45:54] + Processing work unit
[11:45:54] Core required: FahCore_11.exe
[11:45:54] Core found.
[11:45:54] Working on queue slot 03 [April 17 11:45:54 UTC]
[11:45:54] + Working ...
[11:45:54] - Calling '.\FahCore_11.exe -dir work/ -suffix 03 -priority 96 -nocpulock -checkpoint 15 -verbose -lifeline 3492 -version 623'

[11:45:54] 
[11:45:54] *------------------------------*
[11:45:54] Folding@Home GPU Core - Beta
[11:45:54] Version 1.19 (Mon Nov 3 09:34:13 PST 2008)
[11:45:54] 
[11:45:54] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[11:45:54] Build host: amoeba
[11:45:54] Board Type: Nvidia
[11:45:54] Core      : 
[11:45:54] Preparing to commence simulation
[11:45:54] - Looking at optimizations...
[11:45:54] - Created dyn
[11:45:54] - Files status OK
[11:45:54] - Expanded 46758 -> 252912 (decompressed 540.8 percent)
[11:45:54] Called DecompressByteArray: compressed_data_size=46758 data_size=252912, decompressed_data_size=252912 diff=0
[11:45:54] - Digital signature verified
[11:45:54] 
[11:45:54] Project: 5768 (Run 14, Clone 165, Gen 364)
[11:45:54] 
[11:45:54] Assembly optimizations on if available.
[11:45:54] Entering M.D.
[11:46:01] Working on Protein
[11:46:01] Client config found, loading data.
[11:46:01] mdrun_gpu returned 
[11:46:01] NANs detected on GPU
[11:46:01] 
[11:46:01] Folding@home Core Shutdown: UNSTABLE_MACHINE
[11:46:04] CoreStatus = 7A (122)
[11:46:04] Sending work to server
[11:46:04] Project: 5768 (Run 14, Clone 165, Gen 364)
[11:46:04] - Error: Could not get length of results file work/wuresults_03.dat
[11:46:04] - Error: Could not read unit 03 file. Removing from queue.
[11:46:04] Trying to send all finished work units
[11:46:04] + No unsent completed units remaining.
[11:46:04] - Preparing to get new work unit...
[11:46:04] + Attempting to get work packet
[11:46:04] - Will indicate memory of 4094 MB
[11:46:04] - Connecting to assignment server
[11:46:04] Connecting to http://assign-GPU.stanford.edu:8080/
[11:46:05] Posted data.
[11:46:05] Initial: 43AB; - Successful: assigned to (171.67.108.11).
[11:46:05] + News From Folding@Home: GPU folding beta
[11:46:06] Loaded queue successfully.
[11:46:06] Connecting to http://171.67.108.11:8080/
[11:46:12] Posted data.
[11:46:12] Initial: 0000; - Receiving payload (expected size: 47270)
[11:46:12] Conversation time very short, giving reduced weight in bandwidth avg
[11:46:12] - Downloaded at ~92 kB/s
[11:46:12] - Averaged speed for that direction ~113 kB/s
[11:46:12] + Received work.
[11:46:12] Trying to send all finished work units
[11:46:12] + No unsent completed units remaining.
[11:46:12] + Closed connections
[11:46:17] 
[11:46:17] + Processing work unit
[11:46:17] Core required: FahCore_11.exe
[11:46:17] Core found.
[11:46:17] Working on queue slot 04 [April 17 11:46:17 UTC]
[11:46:17] + Working ...
[11:46:17] - Calling '.\FahCore_11.exe -dir work/ -suffix 04 -priority 96 -nocpulock -checkpoint 15 -verbose -lifeline 3492 -version 623'

[11:46:17] 
[11:46:17] *------------------------------*
[11:46:17] Folding@Home GPU Core - Beta
[11:46:17] Version 1.19 (Mon Nov 3 09:34:13 PST 2008)
[11:46:17] 
[11:46:17] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[11:46:17] Build host: amoeba
[11:46:17] Board Type: Nvidia
[11:46:17] Core      : 
[11:46:17] Preparing to commence simulation
[11:46:17] - Looking at optimizations...
[11:46:17] - Created dyn
[11:46:17] - Files status OK
[11:46:17] - Expanded 46758 -> 252912 (decompressed 540.8 percent)
[11:46:17] Called DecompressByteArray: compressed_data_size=46758 data_size=252912, decompressed_data_size=252912 diff=0
[11:46:17] - Digital signature verified
[11:46:17] 
[11:46:17] Project: 5768 (Run 14, Clone 165, Gen 364)
[11:46:17] 
[11:46:17] Assembly optimizations on if available.
[11:46:17] Entering M.D.
[11:46:23] Working on Protein
[11:46:24] Client config found, loading data.
[11:46:24] mdrun_gpu returned 
[11:46:24] NANs detected on GPU
[11:46:24] 
[11:46:24] Folding@home Core Shutdown: UNSTABLE_MACHINE
[11:46:27] CoreStatus = 7A (122)
[11:46:27] Sending work to server
[11:46:27] Project: 5768 (Run 14, Clone 165, Gen 364)
[11:46:27] - Error: Could not get length of results file work/wuresults_04.dat
[11:46:27] - Error: Could not read unit 04 file. Removing from queue.
[11:46:27] Trying to send all finished work units
[11:46:27] + No unsent completed units remaining.
[11:46:27] - Preparing to get new work unit...
[11:46:27] + Attempting to get work packet
[11:46:27] - Will indicate memory of 4094 MB
[11:46:27] - Connecting to assignment server
[11:46:27] Connecting to http://assign-GPU.stanford.edu:8080/
[11:46:31] Posted data.
[11:46:31] Initial: 43AB; - Successful: assigned to (171.67.108.11).
[11:46:31] + News From Folding@Home: GPU folding beta
[11:46:31] Loaded queue successfully.
[11:46:31] Connecting to http://171.67.108.11:8080/
[11:46:38] Posted data.
[11:46:38] Initial: 0000; - Receiving payload (expected size: 47270)
[11:46:38] Conversation time very short, giving reduced weight in bandwidth avg
[11:46:38] - Downloaded at ~92 kB/s
[11:46:38] - Averaged speed for that direction ~110 kB/s
[11:46:38] + Received work.
[11:46:38] Trying to send all finished work units
[11:46:38] + No unsent completed units remaining.
[11:46:38] + Closed connections
[11:46:43] 
[11:46:43] + Processing work unit
[11:46:43] Core required: FahCore_11.exe
[11:46:43] Core found.
[11:46:43] Working on queue slot 05 [April 17 11:46:43 UTC]
[11:46:43] + Working ...
[11:46:43] - Calling '.\FahCore_11.exe -dir work/ -suffix 05 -priority 96 -nocpulock -checkpoint 15 -verbose -lifeline 3492 -version 623'

[11:46:43] 
[11:46:43] *------------------------------*
[11:46:43] Folding@Home GPU Core - Beta
[11:46:43] Version 1.19 (Mon Nov 3 09:34:13 PST 2008)
[11:46:43] 
[11:46:43] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[11:46:43] Build host: amoeba
[11:46:43] Board Type: Nvidia
[11:46:43] Core      : 
[11:46:43] Preparing to commence simulation
[11:46:43] - Looking at optimizations...
[11:46:43] - Created dyn
[11:46:43] - Files status OK
[11:46:43] - Expanded 46758 -> 252912 (decompressed 540.8 percent)
[11:46:43] Called DecompressByteArray: compressed_data_size=46758 data_size=252912, decompressed_data_size=252912 diff=0
[11:46:43] - Digital signature verified
[11:46:43] 
[11:46:43] Project: 5768 (Run 14, Clone 165, Gen 364)
[11:46:43] 
[11:46:43] Assembly optimizations on if available.
[11:46:43] Entering M.D.
[11:46:49] Working on Protein
[11:46:50] Client config found, loading data.
[11:46:50] mdrun_gpu returned 
[11:46:50] NANs detected on GPU
[11:46:50] 
[11:46:50] Folding@home Core Shutdown: UNSTABLE_MACHINE
[11:46:50] Starting GUI Server
[11:46:53] CoreStatus = 7A (122)
[11:46:53] Sending work to server
[11:46:53] Project: 5768 (Run 14, Clone 165, Gen 364)
[11:46:53] - Error: Could not get length of results file work/wuresults_05.dat
[11:46:53] - Error: Could not read unit 05 file. Removing from queue.
[11:46:53] Trying to send all finished work units
[11:46:53] + No unsent completed units remaining.
[11:46:53] - Preparing to get new work unit...
[11:46:53] + Attempting to get work packet
[11:46:53] - Will indicate memory of 4094 MB
[11:46:53] - Connecting to assignment server
[11:46:53] Connecting to http://assign-GPU.stanford.edu:8080/
[11:46:58] Posted data.
[11:46:58] Initial: 43AB; - Successful: assigned to (171.67.108.11).
[11:46:58] + News From Folding@Home: GPU folding beta
[11:46:59] Loaded queue successfully.
[11:46:59] Connecting to http://171.67.108.11:8080/
[11:47:04] Posted data.
[11:47:04] Initial: 0000; - Receiving payload (expected size: 47270)
[11:47:04] Conversation time very short, giving reduced weight in bandwidth avg
[11:47:04] - Downloaded at ~92 kB/s
[11:47:04] - Averaged speed for that direction ~108 kB/s
[11:47:04] + Received work.
[11:47:04] Trying to send all finished work units
[11:47:04] + No unsent completed units remaining.
[11:47:04] + Closed connections
[11:47:09] 
[11:47:09] + Processing work unit
[11:47:09] Core required: FahCore_11.exe
[11:47:09] Core found.
[11:47:09] Working on queue slot 06 [April 17 11:47:09 UTC]
[11:47:09] + Working ...
[11:47:09] - Calling '.\FahCore_11.exe -dir work/ -suffix 06 -priority 96 -nocpulock -checkpoint 15 -verbose -lifeline 3492 -version 623'

[11:47:09] 
[11:47:09] *------------------------------*
[11:47:09] Folding@Home GPU Core - Beta
[11:47:09] Version 1.19 (Mon Nov 3 09:34:13 PST 2008)
[11:47:09] 
[11:47:09] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[11:47:09] Build host: amoeba
[11:47:09] Board Type: Nvidia
[11:47:09] Core      : 
[11:47:09] Preparing to commence simulation
[11:47:09] - Looking at optimizations...
[11:47:09] - Created dyn
[11:47:09] - Files status OK
[11:47:09] - Expanded 46758 -> 252912 (decompressed 540.8 percent)
[11:47:09] Called DecompressByteArray: compressed_data_size=46758 data_size=252912, decompressed_data_size=252912 diff=0
[11:47:09] - Digital signature verified
[11:47:09] 
[11:47:09] Project: 5768 (Run 14, Clone 165, Gen 364)
[11:47:09] 
[11:47:09] Assembly optimizations on if available.
[11:47:09] Entering M.D.
[11:47:16] Working on Protein
[11:47:16] Client config found, loading data.
[11:47:16] Starting GUI Server
[11:47:16] mdrun_gpu returned 
[11:47:16] NANs detected on GPU
[11:47:16] 
[11:47:16] Folding@home Core Shutdown: UNSTABLE_MACHINE
[11:47:19] CoreStatus = 7A (122)
[11:47:19] Sending work to server
[11:47:19] Project: 5768 (Run 14, Clone 165, Gen 364)
[11:47:19] - Error: Could not get length of results file work/wuresults_06.dat
[11:47:19] - Error: Could not read unit 06 file. Removing from queue.
[11:47:19] EUE limit exceeded. Pausing 24 hours.

[toTOW]

There's no data for Project: 5767 (Run 4, Clone 240, Gen 343) and Project: 5768 (Run 14, Clone 165, Gen 364) yet ...

You might want to run an OCCT test to check your board stability ...

[ChrissyT88]

Hi toTow - i did as you suggested and ran the OCCT GPU tests. I ran 10 passes of the CUDA based memory test with no errors, and then gave the card an hour long blast of the graphics test at a resolution of 1024x768 with error checking turned on, which again returned with no errors. It didnt get anywhere near as hot as it did when folding, although the temperatures when folding dont trigger the fans to go above 50%.

The rest of the system was stressed a few weeks ago with OCCT and LinX as i was trying to iron out instability. Memtest also passed, so im not sure what the problem is here? If this is an inappropriate place to discuss the issue, i'll post a thread in the nVidia section if you would prefer.

Thanks again

[toTOW]

So the board seems fine (but OCCT doesn't test everything) ... keep watching your stability at stock clocks ...