Folding Forum

Community driven support forum for Folding@home
https://foldingforum.org/

Get New GPU Task After Failed GPU Task

https://foldingforum.org/viewtopic.php?t=23820

Page 1 of 3

Get New GPU Task After Failed GPU Task

Posted: Fri Mar 01, 2013 11:20 pm
by Space45
The GPU slot of my Folding@home client has had a status of "Failed" for the last week. I presume this means that no new tasks have been running in that slot. How do I get a new GPU task?

Re: Get New GPU Task After Failed GPU Task

Posted: Fri Mar 01, 2013 11:31 pm
by Jesse_V
Welcome to the forum Space45.
The GPU slot is marked as "Failed" because a number of tasks, known as Work Units, failed during processing. Usually this is quite rare, so it indicates that there is a problem somewhere on your end. It turned off the slot to prevent further problems and delays to the project overall. I suspect that simply getting new Work Units will just continue the problem, so it's first important to figure out what is going on.

There are a number of ways of starting the diagnosis process. Are you using Web Control, or are you familiar with the more advanced FAHControl?

Re: Get New GPU Task After Failed GPU Task

Posted: Fri Mar 01, 2013 11:32 pm
by bruce
Welcome to foldingforum.org, Space45.

A GPU slot is set to the "FAILED" status when there have been a series of WUs that have failed. Typically that happens because the GPU is overheating or has been overclocked too much or the hardware itself is starting to fail. It should be noted that if the GPU has a bad memory chip or starts making occasional computational errors while being used to generate graphics, an occasional pixel that's not quite the right color will often not be seen, whereas when the GPU is being used for scientific calculations, there's a much higher need for accurate results.

Pause the GPU slot, reduce your overclock, and restart it. If it fails again, return it to the vendor for repair.

Also check that all your fans are operating as expected. Adjusting the fan profile might help, too.

Re: Get New GPU Task After Failed GPU Task

Posted: Fri Mar 01, 2013 11:39 pm
by Space45
Jesse_V wrote:Welcome to the forum Space45.
The GPU slot is marked as "Failed" because a number of tasks, known as Work Units, failed during processing. Usually this is quite rare, so it indicates that there is a problem somewhere on your end. It turned off the slot to prevent further problems and delays to the project overall. I suspect that simply getting new Work Units will just continue the problem, so it's first important to figure out what is going on.

There are a number of ways of starting the diagnosis process. Are you using Web Control, or are you familiar with the more advanced FAHControl?
I use the Advanced Control instead of the Web Control.

In response to Bruce: I ran a test tool from EVGA, the manufacturer of my video card, for 30 minutes under full load and it listed no errors during that time.

Re: Get New GPU Task After Failed GPU Task

Posted: Fri Mar 01, 2013 11:41 pm
by Space45
I was thinking, maybe the GPU slot is conflicting with BOINC?

Re: Get New GPU Task After Failed GPU Task

Posted: Fri Mar 01, 2013 11:48 pm
by Jesse_V
Space45 wrote:I use the Advanced Control instead of the Web Control.
Ok. Under Folding Slots, it should show you what the ID is your GPU slot. Then click the Log tab, and filter by that ID. Clicking Refresh at the bottom might be helpful. Paste the results in here in between Code tags (

Code: Select all

). It'd also be helpful to see the System section at the top of the log. Please remove the filter and paste that here in between Code tags as well.
Space45 wrote:I was thinking, maybe the GPU slot is conflicting with BOINC?
What BOINC project are you contributing to that uses the GPU?

Re: Get New GPU Task After Failed GPU Task

Posted: Fri Mar 01, 2013 11:58 pm
by Space45
I actively contribute to about five projects, of which I know SETI@home, SETI@home Beta, and Einstein@Home use the GPU.

Here's the log. I couldn't fit it all in one post, so I'll post a second one soon with the remainder.

Code: Select all

*********************** Log Started 2013-03-01T01:39:23Z ***********************
******************************* Date: 2013-03-01 *******************************
13:33:30:WU00:FS00:Connecting to assign-GPU.stanford.edu:80
13:33:31:WU00:FS00:News: Welcome to Folding@Home
13:33:31:WU00:FS00:Assigned to work server 171.64.65.105
13:33:31:WU00:FS00:Requesting new work unit for slot 00: READY gpu:0:GF100 [GeForce GTX 470] from 171.64.65.105
13:33:31:WU00:FS00:Connecting to 171.64.65.105:8080
13:33:31:WU00:FS00:Downloading 77.98KiB
13:33:32:WU00:FS00:Download complete
13:33:32:WU00:FS00:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:7660 run:1211 clone:0 gen:12 core:0x15 unit:0x0000000e664f2dd150f83fcdae21d7b4
13:33:32:WU00:FS00:Downloading core from http://www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah
13:33:32:WU00:FS00:Connecting to www.stanford.edu:80
13:33:33:WU00:FS00:FahCore 15: Downloading 1.88MiB
13:33:39:WU00:FS00:FahCore 15: 29.99%
13:33:45:WU00:FS00:FahCore 15: 63.31%
13:33:51:WU00:FS00:FahCore 15: 96.63%
13:33:52:WU00:FS00:FahCore 15: Download complete
13:33:53:WU00:FS00:Valid core signature
13:33:53:WARNING:WU00:FS00:FahCore has not changed since last download, aborting core update
13:33:53:WU00:FS00:Starting
13:33:53:WU00:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" "C:/Users/Space45/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe" -dir 00 -suffix 01 -version 703 -lifeline 4192 -checkpoint 5 -gpu 0 -gpu-vendor nvidia
13:33:53:WU00:FS00:Started FahCore on PID 3360
13:33:53:WU00:FS00:Core PID:4520
13:33:53:WU00:FS00:FahCore 0x15 started
13:33:53:WU00:FS00:0x15:
13:33:53:WU00:FS00:0x15:*------------------------------*
13:33:53:WU00:FS00:0x15:Folding@Home GPU Core
13:33:53:WU00:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
13:33:53:WU00:FS00:0x15:Build host             AmoebaRemote
13:33:53:WU00:FS00:0x15:Board Type             NVIDIA/CUDA
13:33:53:WU00:FS00:0x15:Core                   15
13:33:53:WU00:FS00:0x15:
13:33:53:WU00:FS00:0x15:Window's signal control handler registered.
13:33:53:WU00:FS00:0x15:Preparing to commence simulation
13:33:53:WU00:FS00:0x15:- Looking at optimizations...
13:33:53:WU00:FS00:0x15:DeleteFrameFiles: successfully deleted file=00/wudata_01.ckp
13:33:53:WU00:FS00:0x15:- Created dyn
13:33:53:WU00:FS00:0x15:- Files status OK
13:33:53:WU00:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
13:33:53:WU00:FS00:0x15:- Expanded 79343 -> 307810 (decompressed 387.9 percent)
13:33:53:WU00:FS00:0x15:Called DecompressByteArray: compressed_data_size=79343 data_size=307810, decompressed_data_size=307810 diff=0
13:33:53:WU00:FS00:0x15:- Digital signature verified
13:33:53:WU00:FS00:0x15:
13:33:53:WU00:FS00:0x15:Project: 7660 (Run 1211, Clone 0, Gen 12)
13:33:53:WU00:FS00:0x15:
13:33:53:WU00:FS00:0x15:Assembly optimizations on if available.
13:33:53:WU00:FS00:0x15:Entering M.D.
13:33:55:WU00:FS00:0x15:Tpr hash 00/wudata_01.tpr:  3474743481 1040523797 4034662644 1170707484 2660897799
13:33:55:WU00:FS00:0x15:GPU device id=0
13:33:55:WU00:FS00:0x15:Working on Protein
13:33:55:WU00:FS00:0x15:Client config unavailable.
13:33:55:WU00:FS00:0x15:Finished fah_main status=59
13:33:55:WU00:FS00:0x15:mdrun_gpu returned 59
13:33:55:WU00:FS00:0x15:GPU memtest failure
13:33:55:WU00:FS00:0x15:
13:33:55:WU00:FS00:0x15:Folding@home Core Shutdown: GPU_MEMTEST_ERROR
13:33:55:WARNING:WU00:FS00:FahCore returned: GPU_MEMTEST_ERROR (124 = 0x7c)
13:33:56:WU00:FS00:Starting
13:33:56:WU00:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" "C:/Users/Space45/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe" -dir 00 -suffix 01 -version 703 -lifeline 4192 -checkpoint 5 -gpu 0 -gpu-vendor nvidia
13:33:56:WU00:FS00:Started FahCore on PID 8860
13:33:56:WU00:FS00:Core PID:7228
13:33:56:WU00:FS00:FahCore 0x15 started
13:33:56:WU00:FS00:0x15:
13:33:56:WU00:FS00:0x15:*------------------------------*
13:33:56:WU00:FS00:0x15:Folding@Home GPU Core
13:33:56:WU00:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
13:33:56:WU00:FS00:0x15:Build host             AmoebaRemote
13:33:56:WU00:FS00:0x15:Board Type             NVIDIA/CUDA
13:33:56:WU00:FS00:0x15:Core                   15
13:33:56:WU00:FS00:0x15:
13:33:56:WU00:FS00:0x15:Window's signal control handler registered.
13:33:56:WU00:FS00:0x15:Preparing to commence simulation
13:33:56:WU00:FS00:0x15:- Looking at optimizations...
13:33:56:WU00:FS00:0x15:DeleteFrameFiles: successfully deleted file=00/wudata_01.ckp
13:33:56:WU00:FS00:0x15:- Created dyn
13:33:56:WU00:FS00:0x15:- Files status OK
13:33:56:WU00:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
13:33:56:WU00:FS00:0x15:- Expanded 79343 -> 307810 (decompressed 387.9 percent)
13:33:56:WU00:FS00:0x15:Called DecompressByteArray: compressed_data_size=79343 data_size=307810, decompressed_data_size=307810 diff=0
13:33:56:WU00:FS00:0x15:- Digital signature verified
13:33:56:WU00:FS00:0x15:
13:33:56:WU00:FS00:0x15:Project: 7660 (Run 1211, Clone 0, Gen 12)
13:33:56:WU00:FS00:0x15:
13:33:56:WU00:FS00:0x15:Assembly optimizations on if available.
13:33:56:WU00:FS00:0x15:Entering M.D.
13:33:58:WU00:FS00:0x15:Tpr hash 00/wudata_01.tpr:  3474743481 1040523797 4034662644 1170707484 2660897799
13:33:58:WU00:FS00:0x15:GPU device id=0
13:33:58:WU00:FS00:0x15:Working on Protein
13:33:58:WU00:FS00:0x15:Client config unavailable.
13:33:58:WU00:FS00:0x15:Finished fah_main status=59
13:33:58:WU00:FS00:0x15:mdrun_gpu returned 59
13:33:58:WU00:FS00:0x15:GPU memtest failure
13:33:58:WU00:FS00:0x15:
13:33:58:WU00:FS00:0x15:Folding@home Core Shutdown: GPU_MEMTEST_ERROR
13:33:58:WARNING:WU00:FS00:FahCore returned: GPU_MEMTEST_ERROR (124 = 0x7c)
13:34:56:WU00:FS00:Starting
13:34:56:WU00:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" "C:/Users/Space45/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe" -dir 00 -suffix 01 -version 703 -lifeline 4192 -checkpoint 5 -gpu 0 -gpu-vendor nvidia
13:34:56:WU00:FS00:Started FahCore on PID 8304
13:34:56:WU00:FS00:Core PID:4032
13:34:56:WU00:FS00:FahCore 0x15 started
13:34:56:WU00:FS00:0x15:
13:34:56:WU00:FS00:0x15:*------------------------------*
13:34:56:WU00:FS00:0x15:Folding@Home GPU Core
13:34:56:WU00:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
13:34:56:WU00:FS00:0x15:Build host             AmoebaRemote
13:34:56:WU00:FS00:0x15:Board Type             NVIDIA/CUDA
13:34:56:WU00:FS00:0x15:Core                   15
13:34:56:WU00:FS00:0x15:
13:34:56:WU00:FS00:0x15:Window's signal control handler registered.
13:34:56:WU00:FS00:0x15:Preparing to commence simulation
13:34:56:WU00:FS00:0x15:- Looking at optimizations...
13:34:56:WU00:FS00:0x15:DeleteFrameFiles: successfully deleted file=00/wudata_01.ckp
13:34:56:WU00:FS00:0x15:- Created dyn
13:34:56:WU00:FS00:0x15:- Files status OK
13:34:56:WU00:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
13:34:56:WU00:FS00:0x15:- Expanded 79343 -> 307810 (decompressed 387.9 percent)
13:34:56:WU00:FS00:0x15:Called DecompressByteArray: compressed_data_size=79343 data_size=307810, decompressed_data_size=307810 diff=0
13:34:56:WU00:FS00:0x15:- Digital signature verified
13:34:56:WU00:FS00:0x15:
13:34:56:WU00:FS00:0x15:Project: 7660 (Run 1211, Clone 0, Gen 12)
13:34:56:WU00:FS00:0x15:
13:34:56:WU00:FS00:0x15:Assembly optimizations on if available.
13:34:56:WU00:FS00:0x15:Entering M.D.
13:34:58:WU00:FS00:0x15:Tpr hash 00/wudata_01.tpr:  3474743481 1040523797 4034662644 1170707484 2660897799
13:34:58:WU00:FS00:0x15:GPU device id=0
13:34:58:WU00:FS00:0x15:Working on Protein
13:34:58:WU00:FS00:0x15:Client config unavailable.
13:34:58:WU00:FS00:0x15:Finished fah_main status=59
13:34:58:WU00:FS00:0x15:mdrun_gpu returned 59
13:34:58:WU00:FS00:0x15:GPU memtest failure
13:34:58:WU00:FS00:0x15:
13:34:58:WU00:FS00:0x15:Folding@home Core Shutdown: GPU_MEMTEST_ERROR
13:34:59:WARNING:WU00:FS00:FahCore returned: GPU_MEMTEST_ERROR (124 = 0x7c)
13:35:56:WU00:FS00:Starting
13:35:56:WU00:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" "C:/Users/Space45/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe" -dir 00 -suffix 01 -version 703 -lifeline 4192 -checkpoint 5 -gpu 0 -gpu-vendor nvidia
13:35:56:WU00:FS00:Started FahCore on PID 2628
13:35:56:WU00:FS00:Core PID:6420
13:35:56:WU00:FS00:FahCore 0x15 started
13:35:56:WU00:FS00:0x15:
13:35:56:WU00:FS00:0x15:*------------------------------*
13:35:56:WU00:FS00:0x15:Folding@Home GPU Core
13:35:56:WU00:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
13:35:56:WU00:FS00:0x15:Build host             AmoebaRemote
13:35:56:WU00:FS00:0x15:Board Type             NVIDIA/CUDA
13:35:56:WU00:FS00:0x15:Core                   15
13:35:56:WU00:FS00:0x15:
13:35:56:WU00:FS00:0x15:Window's signal control handler registered.
13:35:56:WU00:FS00:0x15:Preparing to commence simulation
13:35:56:WU00:FS00:0x15:- Looking at optimizations...
13:35:56:WU00:FS00:0x15:DeleteFrameFiles: successfully deleted file=00/wudata_01.ckp
13:35:56:WU00:FS00:0x15:- Created dyn
13:35:56:WU00:FS00:0x15:- Files status OK
13:35:56:WU00:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
13:35:56:WU00:FS00:0x15:- Expanded 79343 -> 307810 (decompressed 387.9 percent)
13:35:56:WU00:FS00:0x15:Called DecompressByteArray: compressed_data_size=79343 data_size=307810, decompressed_data_size=307810 diff=0
13:35:56:WU00:FS00:0x15:- Digital signature verified
13:35:56:WU00:FS00:0x15:
13:35:56:WU00:FS00:0x15:Project: 7660 (Run 1211, Clone 0, Gen 12)
13:35:56:WU00:FS00:0x15:
13:35:56:WU00:FS00:0x15:Assembly optimizations on if available.
13:35:56:WU00:FS00:0x15:Entering M.D.
13:35:58:WU00:FS00:0x15:Tpr hash 00/wudata_01.tpr:  3474743481 1040523797 4034662644 1170707484 2660897799
13:35:58:WU00:FS00:0x15:GPU device id=0
13:35:58:WU00:FS00:0x15:Working on Protein
13:35:58:WU00:FS00:0x15:Client config unavailable.
13:35:58:WU00:FS00:0x15:Finished fah_main status=59
13:35:58:WU00:FS00:0x15:mdrun_gpu returned 59
13:35:58:WU00:FS00:0x15:GPU memtest failure
13:35:58:WU00:FS00:0x15:
13:35:58:WU00:FS00:0x15:Folding@home Core Shutdown: GPU_MEMTEST_ERROR
13:35:59:WARNING:WU00:FS00:FahCore returned: GPU_MEMTEST_ERROR (124 = 0x7c)
13:36:56:WU00:FS00:Starting
13:36:56:WU00:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" "C:/Users/Space45/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe" -dir 00 -suffix 01 -version 703 -lifeline 4192 -checkpoint 5 -gpu 0 -gpu-vendor nvidia
13:36:56:WU00:FS00:Started FahCore on PID 4848
13:36:56:WU00:FS00:Core PID:7372
13:36:56:WU00:FS00:FahCore 0x15 started
13:36:56:WU00:FS00:0x15:
13:36:56:WU00:FS00:0x15:*------------------------------*
13:36:56:WU00:FS00:0x15:Folding@Home GPU Core
13:36:56:WU00:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
13:36:56:WU00:FS00:0x15:Build host             AmoebaRemote
13:36:56:WU00:FS00:0x15:Board Type             NVIDIA/CUDA
13:36:56:WU00:FS00:0x15:Core                   15
13:36:56:WU00:FS00:0x15:
13:36:56:WU00:FS00:0x15:Window's signal control handler registered.
13:36:56:WU00:FS00:0x15:Preparing to commence simulation
13:36:56:WU00:FS00:0x15:- Looking at optimizations...
13:36:56:WU00:FS00:0x15:DeleteFrameFiles: successfully deleted file=00/wudata_01.ckp
13:36:56:WU00:FS00:0x15:- Created dyn
13:36:56:WU00:FS00:0x15:- Files status OK
13:36:56:WU00:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
13:36:56:WU00:FS00:0x15:- Expanded 79343 -> 307810 (decompressed 387.9 percent)
13:36:56:WU00:FS00:0x15:Called DecompressByteArray: compressed_data_size=79343 data_size=307810, decompressed_data_size=307810 diff=0
13:36:56:WU00:FS00:0x15:- Digital signature verified
13:36:56:WU00:FS00:0x15:
13:36:56:WU00:FS00:0x15:Project: 7660 (Run 1211, Clone 0, Gen 12)
13:36:56:WU00:FS00:0x15:
13:36:56:WU00:FS00:0x15:Assembly optimizations on if available.
13:36:56:WU00:FS00:0x15:Entering M.D.
13:36:58:WU00:FS00:0x15:Tpr hash 00/wudata_01.tpr:  3474743481 1040523797 4034662644 1170707484 2660897799
13:36:58:WU00:FS00:0x15:GPU device id=0
13:36:58:WU00:FS00:0x15:Working on Protein
13:36:58:WU00:FS00:0x15:Client config unavailable.
13:36:58:WU00:FS00:0x15:Finished fah_main status=59
13:36:58:WU00:FS00:0x15:mdrun_gpu returned 59
13:36:58:WU00:FS00:0x15:GPU memtest failure
13:36:58:WU00:FS00:0x15:
13:36:58:WU00:FS00:0x15:Folding@home Core Shutdown: GPU_MEMTEST_ERROR
13:36:58:WARNING:WU00:FS00:FahCore returned: GPU_MEMTEST_ERROR (124 = 0x7c)
13:36:58:WARNING:WU00:FS00:Too many errors, failing
13:36:59:WU00:FS00:Sending unit results: id:00 state:SEND error:FAILED project:7660 run:1211 clone:0 gen:12 core:0x15 unit:0x0000000e664f2dd150f83fcdae21d7b4
13:36:59:WU00:FS00:Connecting to 171.64.65.105:8080
13:36:59:WU02:FS00:Connecting to assign-GPU.stanford.edu:80
13:36:59:WU00:FS00:Server responded WORK_QUIT (404)
13:36:59:WARNING:WU00:FS00:Server did not like results, dumping
13:36:59:WU00:FS00:Cleaning up
13:37:01:WU02:FS00:News: Welcome to Folding@Home
13:37:01:WU02:FS00:Assigned to work server 171.67.108.36
13:37:01:WU02:FS00:Requesting new work unit for slot 00: READY gpu:0:GF100 [GeForce GTX 470] from 171.67.108.36
13:37:01:WU02:FS00:Connecting to 171.67.108.36:8080
13:37:02:WU02:FS00:Downloading 58.00KiB
13:37:02:WU02:FS00:Download complete
13:37:03:WU02:FS00:Received Unit: id:02 state:DOWNLOAD error:NO_ERROR project:8071 run:61 clone:5 gen:8 core:0x15 unit:0x000000096652edb45122e1045b0ff210
13:37:03:WU02:FS00:Starting
13:37:03:WU02:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" "C:/Users/Space45/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe" -dir 02 -suffix 01 -version 703 -lifeline 4192 -checkpoint 5 -gpu 0 -gpu-vendor nvidia
13:37:03:WU02:FS00:Started FahCore on PID 4468
13:37:03:WU02:FS00:Core PID:7196
13:37:03:WU02:FS00:FahCore 0x15 started
13:37:03:WU02:FS00:0x15:
13:37:03:WU02:FS00:0x15:*------------------------------*
13:37:03:WU02:FS00:0x15:Folding@Home GPU Core
13:37:03:WU02:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
13:37:03:WU02:FS00:0x15:Build host             AmoebaRemote
13:37:03:WU02:FS00:0x15:Board Type             NVIDIA/CUDA
13:37:03:WU02:FS00:0x15:Core                   15
13:37:03:WU02:FS00:0x15:
13:37:03:WU02:FS00:0x15:Window's signal control handler registered.
13:37:03:WU02:FS00:0x15:Preparing to commence simulation
13:37:03:WU02:FS00:0x15:- Looking at optimizations...
13:37:03:WU02:FS00:0x15:DeleteFrameFiles: successfully deleted file=02/wudata_01.ckp
13:37:03:WU02:FS00:0x15:- Created dyn
13:37:03:WU02:FS00:0x15:- Files status OK
13:37:03:WU02:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
13:37:03:WU02:FS00:0x15:- Expanded 58875 -> 259314 (decompressed 440.4 percent)
13:37:03:WU02:FS00:0x15:Called DecompressByteArray: compressed_data_size=58875 data_size=259314, decompressed_data_size=259314 diff=0
13:37:03:WU02:FS00:0x15:- Digital signature verified
13:37:03:WU02:FS00:0x15:
13:37:03:WU02:FS00:0x15:Project: 8071 (Run 61, Clone 5, Gen 8)
13:37:03:WU02:FS00:0x15:
13:37:03:WU02:FS00:0x15:Assembly optimizations on if available.
13:37:03:WU02:FS00:0x15:Entering M.D.
13:37:05:WU02:FS00:0x15:Tpr hash 02/wudata_01.tpr:  1768724696 3563293753 1457912165 911357650 2199223979
13:37:05:WU02:FS00:0x15:GPU device id=0
13:37:05:WU02:FS00:0x15:Working on Glycine aRginine prOline Methionine Alanine Cystine Serine t=  61.00000
13:37:05:WU02:FS00:0x15:Client config unavailable.
13:37:05:WU02:FS00:0x15:Finished fah_main status=59
13:37:05:WU02:FS00:0x15:mdrun_gpu returned 59
13:37:05:WU02:FS00:0x15:GPU memtest failure
13:37:05:WU02:FS00:0x15:
13:37:05:WU02:FS00:0x15:Folding@home Core Shutdown: GPU_MEMTEST_ERROR
13:37:05:WARNING:WU02:FS00:FahCore returned: GPU_MEMTEST_ERROR (124 = 0x7c)
13:37:06:WU02:FS00:Starting
13:37:06:WU02:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" "C:/Users/Space45/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe" -dir 02 -suffix 01 -version 703 -lifeline 4192 -checkpoint 5 -gpu 0 -gpu-vendor nvidia
13:37:06:WU02:FS00:Started FahCore on PID 5716
13:37:06:WU02:FS00:Core PID:8976
13:37:06:WU02:FS00:FahCore 0x15 started
13:37:07:WU02:FS00:0x15:
13:37:07:WU02:FS00:0x15:*------------------------------*
13:37:07:WU02:FS00:0x15:Folding@Home GPU Core
13:37:07:WU02:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
13:37:07:WU02:FS00:0x15:Build host             AmoebaRemote
13:37:07:WU02:FS00:0x15:Board Type             NVIDIA/CUDA
13:37:07:WU02:FS00:0x15:Core                   15
13:37:07:WU02:FS00:0x15:
13:37:07:WU02:FS00:0x15:Window's signal control handler registered.
13:37:07:WU02:FS00:0x15:Preparing to commence simulation
13:37:07:WU02:FS00:0x15:- Looking at optimizations...
13:37:07:WU02:FS00:0x15:DeleteFrameFiles: successfully deleted file=02/wudata_01.ckp
13:37:07:WU02:FS00:0x15:- Created dyn
13:37:07:WU02:FS00:0x15:- Files status OK
13:37:07:WU02:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
13:37:07:WU02:FS00:0x15:- Expanded 58875 -> 259314 (decompressed 440.4 percent)
13:37:07:WU02:FS00:0x15:Called DecompressByteArray: compressed_data_size=58875 data_size=259314, decompressed_data_size=259314 diff=0
13:37:07:WU02:FS00:0x15:- Digital signature verified
13:37:07:WU02:FS00:0x15:
13:37:07:WU02:FS00:0x15:Project: 8071 (Run 61, Clone 5, Gen 8)
13:37:07:WU02:FS00:0x15:
13:37:07:WU02:FS00:0x15:Assembly optimizations on if available.
13:37:07:WU02:FS00:0x15:Entering M.D.
13:37:08:WU02:FS00:0x15:Tpr hash 02/wudata_01.tpr:  1768724696 3563293753 1457912165 911357650 2199223979
13:37:08:WU02:FS00:0x15:GPU device id=0
13:37:08:WU02:FS00:0x15:Working on Glycine aRginine prOline Methionine Alanine Cystine Serine t=  61.00000
13:37:08:WU02:FS00:0x15:Client config unavailable.
13:37:08:WU02:FS00:0x15:Finished fah_main status=59
13:37:08:WU02:FS00:0x15:mdrun_gpu returned 59
13:37:08:WU02:FS00:0x15:GPU memtest failure
13:37:08:WU02:FS00:0x15:
13:37:08:WU02:FS00:0x15:Folding@home Core Shutdown: GPU_MEMTEST_ERROR
13:37:09:WARNING:WU02:FS00:FahCore returned: GPU_MEMTEST_ERROR (124 = 0x7c)
13:38:06:WU02:FS00:Starting
13:38:06:WU02:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" "C:/Users/Space45/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe" -dir 02 -suffix 01 -version 703 -lifeline 4192 -checkpoint 5 -gpu 0 -gpu-vendor nvidia
13:38:06:WU02:FS00:Started FahCore on PID 5164
13:38:06:WU02:FS00:Core PID:644
13:38:06:WU02:FS00:FahCore 0x15 started
13:38:07:WU02:FS00:0x15:
13:38:07:WU02:FS00:0x15:*------------------------------*
13:38:07:WU02:FS00:0x15:Folding@Home GPU Core
13:38:07:WU02:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
13:38:07:WU02:FS00:0x15:Build host             AmoebaRemote
13:38:07:WU02:FS00:0x15:Board Type             NVIDIA/CUDA
13:38:07:WU02:FS00:0x15:Core                   15
13:38:07:WU02:FS00:0x15:
13:38:07:WU02:FS00:0x15:Window's signal control handler registered.
13:38:07:WU02:FS00:0x15:Preparing to commence simulation
13:38:07:WU02:FS00:0x15:- Looking at optimizations...
13:38:07:WU02:FS00:0x15:DeleteFrameFiles: successfully deleted file=02/wudata_01.ckp
13:38:07:WU02:FS00:0x15:- Created dyn
13:38:07:WU02:FS00:0x15:- Files status OK
13:38:07:WU02:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
13:38:07:WU02:FS00:0x15:- Expanded 58875 -> 259314 (decompressed 440.4 percent)
13:38:07:WU02:FS00:0x15:Called DecompressByteArray: compressed_data_size=58875 data_size=259314, decompressed_data_size=259314 diff=0
13:38:07:WU02:FS00:0x15:- Digital signature verified
13:38:07:WU02:FS00:0x15:
13:38:07:WU02:FS00:0x15:Project: 8071 (Run 61, Clone 5, Gen 8)
13:38:07:WU02:FS00:0x15:
13:38:07:WU02:FS00:0x15:Assembly optimizations on if available.
13:38:07:WU02:FS00:0x15:Entering M.D.
13:38:09:WU02:FS00:0x15:Tpr hash 02/wudata_01.tpr:  1768724696 3563293753 1457912165 911357650 2199223979
13:38:09:WU02:FS00:0x15:GPU device id=0
13:38:09:WU02:FS00:0x15:Working on Glycine aRginine prOline Methionine Alanine Cystine Serine t=  61.00000
13:38:09:WU02:FS00:0x15:Client config unavailable.
13:38:09:WU02:FS00:0x15:Finished fah_main status=59
13:38:09:WU02:FS00:0x15:mdrun_gpu returned 59
13:38:09:WU02:FS00:0x15:GPU memtest failure
13:38:09:WU02:FS00:0x15:
13:38:09:WU02:FS00:0x15:Folding@home Core Shutdown: GPU_MEMTEST_ERROR
13:38:09:WARNING:WU02:FS00:FahCore returned: GPU_MEMTEST_ERROR (124 = 0x7c)
13:39:06:WU02:FS00:Starting
13:39:06:WU02:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" "C:/Users/Space45/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe" -dir 02 -suffix 01 -version 703 -lifeline 4192 -checkpoint 5 -gpu 0 -gpu-vendor nvidia
13:39:06:WU02:FS00:Started FahCore on PID 3872
13:39:06:WU02:FS00:Core PID:4372
13:39:06:WU02:FS00:FahCore 0x15 started
13:39:07:WU02:FS00:0x15:
13:39:07:WU02:FS00:0x15:*------------------------------*
13:39:07:WU02:FS00:0x15:Folding@Home GPU Core
13:39:07:WU02:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
13:39:07:WU02:FS00:0x15:Build host             AmoebaRemote
13:39:07:WU02:FS00:0x15:Board Type             NVIDIA/CUDA
13:39:07:WU02:FS00:0x15:Core                   15
13:39:07:WU02:FS00:0x15:
13:39:07:WU02:FS00:0x15:Window's signal control handler registered.
13:39:07:WU02:FS00:0x15:Preparing to commence simulation
13:39:07:WU02:FS00:0x15:- Looking at optimizations...
13:39:07:WU02:FS00:0x15:DeleteFrameFiles: successfully deleted file=02/wudata_01.ckp
13:39:07:WU02:FS00:0x15:- Created dyn
13:39:07:WU02:FS00:0x15:- Files status OK
13:39:07:WU02:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
13:39:07:WU02:FS00:0x15:- Expanded 58875 -> 259314 (decompressed 440.4 percent)
13:39:07:WU02:FS00:0x15:Called DecompressByteArray: compressed_data_size=58875 data_size=259314, decompressed_data_size=259314 diff=0
13:39:07:WU02:FS00:0x15:- Digital signature verified
13:39:07:WU02:FS00:0x15:
13:39:07:WU02:FS00:0x15:Project: 8071 (Run 61, Clone 5, Gen 8)
13:39:07:WU02:FS00:0x15:
13:39:07:WU02:FS00:0x15:Assembly optimizations on if available.
13:39:07:WU02:FS00:0x15:Entering M.D.
13:39:08:WU02:FS00:0x15:Tpr hash 02/wudata_01.tpr:  1768724696 3563293753 1457912165 911357650 2199223979
13:39:08:WU02:FS00:0x15:GPU device id=0
13:39:09:WU02:FS00:0x15:Working on Glycine aRginine prOline Methionine Alanine Cystine Serine t=  61.00000
13:39:09:WU02:FS00:0x15:Client config unavailable.
13:39:09:WU02:FS00:0x15:Finished fah_main status=59
13:39:09:WU02:FS00:0x15:mdrun_gpu returned 59
13:39:09:WU02:FS00:0x15:GPU memtest failure
13:39:09:WU02:FS00:0x15:
13:39:09:WU02:FS00:0x15:Folding@home Core Shutdown: GPU_MEMTEST_ERROR
13:39:09:WARNING:WU02:FS00:FahCore returned: GPU_MEMTEST_ERROR (124 = 0x7c)
13:40:06:WU02:FS00:Starting
13:40:06:WU02:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" "C:/Users/Space45/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe" -dir 02 -suffix 01 -version 703 -lifeline 4192 -checkpoint 5 -gpu 0 -gpu-vendor nvidia
13:40:06:WU02:FS00:Started FahCore on PID 8476
13:40:06:WU02:FS00:Core PID:8752
13:40:06:WU02:FS00:FahCore 0x15 started
13:40:07:WU02:FS00:0x15:
13:40:07:WU02:FS00:0x15:*------------------------------*
13:40:07:WU02:FS00:0x15:Folding@Home GPU Core
13:40:07:WU02:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
13:40:07:WU02:FS00:0x15:Build host             AmoebaRemote
13:40:07:WU02:FS00:0x15:Board Type             NVIDIA/CUDA
13:40:07:WU02:FS00:0x15:Core                   15
13:40:07:WU02:FS00:0x15:
13:40:07:WU02:FS00:0x15:Window's signal control handler registered.
13:40:07:WU02:FS00:0x15:Preparing to commence simulation
13:40:07:WU02:FS00:0x15:- Looking at optimizations...
13:40:07:WU02:FS00:0x15:DeleteFrameFiles: successfully deleted file=02/wudata_01.ckp
13:40:07:WU02:FS00:0x15:- Created dyn
13:40:07:WU02:FS00:0x15:- Files status OK
13:40:07:WU02:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
13:40:07:WU02:FS00:0x15:- Expanded 58875 -> 259314 (decompressed 440.4 percent)
13:40:07:WU02:FS00:0x15:Called DecompressByteArray: compressed_data_size=58875 data_size=259314, decompressed_data_size=259314 diff=0
13:40:07:WU02:FS00:0x15:- Digital signature verified
13:40:07:WU02:FS00:0x15:
13:40:07:WU02:FS00:0x15:Project: 8071 (Run 61, Clone 5, Gen 8)
13:40:07:WU02:FS00:0x15:
13:40:07:WU02:FS00:0x15:Assembly optimizations on if available.
13:40:07:WU02:FS00:0x15:Entering M.D.
13:40:08:WU02:FS00:0x15:Tpr hash 02/wudata_01.tpr:  1768724696 3563293753 1457912165 911357650 2199223979
13:40:08:WU02:FS00:0x15:GPU device id=0
13:40:08:WU02:FS00:0x15:Working on Glycine aRginine prOline Methionine Alanine Cystine Serine t=  61.00000
13:40:08:WU02:FS00:0x15:Client config unavailable.
13:40:08:WU02:FS00:0x15:Finished fah_main status=59
13:40:08:WU02:FS00:0x15:mdrun_gpu returned 59
13:40:08:WU02:FS00:0x15:GPU memtest failure
13:40:08:WU02:FS00:0x15:
13:40:08:WU02:FS00:0x15:Folding@home Core Shutdown: GPU_MEMTEST_ERROR
13:40:09:WARNING:WU02:FS00:FahCore returned: GPU_MEMTEST_ERROR (124 = 0x7c)
13:40:09:WARNING:WU02:FS00:Too many errors, failing
13:40:09:WU02:FS00:Sending unit results: id:02 state:SEND error:FAILED project:8071 run:61 clone:5 gen:8 core:0x15 unit:0x000000096652edb45122e1045b0ff210
13:40:09:WU02:FS00:Connecting to 171.67.108.36:8080
13:40:09:WU00:FS00:Connecting to assign-GPU.stanford.edu:80
13:40:10:WU02:FS00:Server responded WORK_QUIT (404)
13:40:10:WARNING:WU02:FS00:Server did not like results, dumping
13:40:10:WU02:FS00:Cleaning up
13:40:10:WU00:FS00:News: Welcome to Folding@Home
13:40:10:WU00:FS00:Assigned to work server 171.64.65.105
13:40:10:WU00:FS00:Requesting new work unit for slot 00: READY gpu:0:GF100 [GeForce GTX 470] from 171.64.65.105
13:40:10:WU00:FS00:Connecting to 171.64.65.105:8080
13:40:11:WU00:FS00:Downloading 78.23KiB
13:40:12:WU00:FS00:Download complete
13:40:12:WU00:FS00:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:7660 run:963 clone:0 gen:8 core:0x15 unit:0x0000000c664f2dd150f83fab68c248f7
13:40:12:WU00:FS00:Starting
13:40:12:WU00:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" "C:/Users/Space45/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe" -dir 00 -suffix 01 -version 703 -lifeline 4192 -checkpoint 5 -gpu 0 -gpu-vendor nvidia
13:40:12:WU00:FS00:Started FahCore on PID 7408
13:40:12:WU00:FS00:Core PID:9080
13:40:12:WU00:FS00:FahCore 0x15 started
13:40:13:WU00:FS00:0x15:
13:40:13:WU00:FS00:0x15:*------------------------------*
13:40:13:WU00:FS00:0x15:Folding@Home GPU Core
13:40:13:WU00:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
13:40:13:WU00:FS00:0x15:Build host             AmoebaRemote
13:40:13:WU00:FS00:0x15:Board Type             NVIDIA/CUDA
13:40:13:WU00:FS00:0x15:Core                   15
13:40:13:WU00:FS00:0x15:
13:40:13:WU00:FS00:0x15:Window's signal control handler registered.
13:40:13:WU00:FS00:0x15:Preparing to commence simulation
13:40:13:WU00:FS00:0x15:- Looking at optimizations...
13:40:13:WU00:FS00:0x15:DeleteFrameFiles: successfully deleted file=00/wudata_01.ckp
13:40:13:WU00:FS00:0x15:- Created dyn
13:40:13:WU00:FS00:0x15:- Files status OK
13:40:13:WU00:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
13:40:13:WU00:FS00:0x15:- Expanded 79600 -> 307810 (decompressed 386.6 percent)
13:40:13:WU00:FS00:0x15:Called DecompressByteArray: compressed_data_size=79600 data_size=307810, decompressed_data_size=307810 diff=0
13:40:13:WU00:FS00:0x15:- Digital signature verified
13:40:13:WU00:FS00:0x15:
13:40:13:WU00:FS00:0x15:Project: 7660 (Run 963, Clone 0, Gen 8)
13:40:13:WU00:FS00:0x15:
13:40:13:WU00:FS00:0x15:Assembly optimizations on if available.
13:40:13:WU00:FS00:0x15:Entering M.D.
13:40:15:WU00:FS00:0x15:Tpr hash 00/wudata_01.tpr:  2327992468 1308474764 3265566474 1012946672 257706942
13:40:15:WU00:FS00:0x15:GPU device id=0
13:40:15:WU00:FS00:0x15:Working on Protein
13:40:15:WU00:FS00:0x15:Client config unavailable.
13:40:15:WU00:FS00:0x15:Finished fah_main status=59
13:40:15:WU00:FS00:0x15:mdrun_gpu returned 59
13:40:15:WU00:FS00:0x15:GPU memtest failure
13:40:15:WU00:FS00:0x15:
13:40:15:WU00:FS00:0x15:Folding@home Core Shutdown: GPU_MEMTEST_ERROR
13:40:15:WARNING:WU00:FS00:FahCore returned: GPU_MEMTEST_ERROR (124 = 0x7c)
13:40:15:WU00:FS00:Starting
13:40:15:WU00:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" "C:/Users/Space45/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe" -dir 00 -suffix 01 -version 703 -lifeline 4192 -checkpoint 5 -gpu 0 -gpu-vendor nvidia
13:40:15:WU00:FS00:Started FahCore on PID 1564
13:40:15:WU00:FS00:Core PID:5084
13:40:15:WU00:FS00:FahCore 0x15 started
13:40:16:WU00:FS00:0x15:
13:40:16:WU00:FS00:0x15:*------------------------------*
13:40:16:WU00:FS00:0x15:Folding@Home GPU Core
13:40:16:WU00:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
13:40:16:WU00:FS00:0x15:Build host             AmoebaRemote
13:40:16:WU00:FS00:0x15:Board Type             NVIDIA/CUDA
13:40:16:WU00:FS00:0x15:Core                   15
13:40:16:WU00:FS00:0x15:
13:40:16:WU00:FS00:0x15:Window's signal control handler registered.
13:40:16:WU00:FS00:0x15:Preparing to commence simulation
13:40:16:WU00:FS00:0x15:- Looking at optimizations...
13:40:16:WU00:FS00:0x15:DeleteFrameFiles: successfully deleted file=00/wudata_01.ckp
13:40:16:WU00:FS00:0x15:- Created dyn
13:40:16:WU00:FS00:0x15:- Files status OK
13:40:16:WU00:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
13:40:16:WU00:FS00:0x15:- Expanded 79600 -> 307810 (decompressed 386.6 percent)
13:40:16:WU00:FS00:0x15:Called DecompressByteArray: compressed_data_size=79600 data_size=307810, decompressed_data_size=307810 diff=0
13:40:16:WU00:FS00:0x15:- Digital signature verified
13:40:16:WU00:FS00:0x15:
13:40:16:WU00:FS00:0x15:Project: 7660 (Run 963, Clone 0, Gen 8)
13:40:16:WU00:FS00:0x15:
13:40:16:WU00:FS00:0x15:Assembly optimizations on if available.
13:40:16:WU00:FS00:0x15:Entering M.D.
13:40:18:WU00:FS00:0x15:Tpr hash 00/wudata_01.tpr:  2327992468 1308474764 3265566474 1012946672 257706942
13:40:18:WU00:FS00:0x15:GPU device id=0
13:40:18:WU00:FS00:0x15:Working on Protein
13:40:18:WU00:FS00:0x15:Client config unavailable.
13:40:18:WU00:FS00:0x15:Finished fah_main status=59
13:40:18:WU00:FS00:0x15:mdrun_gpu returned 59
13:40:18:WU00:FS00:0x15:GPU memtest failure
13:40:18:WU00:FS00:0x15:
13:40:18:WU00:FS00:0x15:Folding@home Core Shutdown: GPU_MEMTEST_ERROR
13:40:18:WARNING:WU00:FS00:FahCore returned: GPU_MEMTEST_ERROR (124 = 0x7c)
13:41:15:WU00:FS00:Starting
13:41:15:WU00:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" "C:/Users/Space45/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe" -dir 00 -suffix 01 -version 703 -lifeline 4192 -checkpoint 5 -gpu 0 -gpu-vendor nvidia
13:41:15:WU00:FS00:Started FahCore on PID 7776
13:41:15:WU00:FS00:Core PID:8624
13:41:15:WU00:FS00:FahCore 0x15 started
13:41:16:WU00:FS00:0x15:
13:41:16:WU00:FS00:0x15:*------------------------------*
13:41:16:WU00:FS00:0x15:Folding@Home GPU Core
13:41:16:WU00:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
13:41:16:WU00:FS00:0x15:Build host             AmoebaRemote
13:41:16:WU00:FS00:0x15:Board Type             NVIDIA/CUDA
13:41:16:WU00:FS00:0x15:Core                   15
13:41:16:WU00:FS00:0x15:
13:41:16:WU00:FS00:0x15:Window's signal control handler registered.
13:41:16:WU00:FS00:0x15:Preparing to commence simulation
13:41:16:WU00:FS00:0x15:- Looking at optimizations...
13:41:16:WU00:FS00:0x15:DeleteFrameFiles: successfully deleted file=00/wudata_01.ckp
13:41:16:WU00:FS00:0x15:- Created dyn
13:41:16:WU00:FS00:0x15:- Files status OK
13:41:16:WU00:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
13:41:16:WU00:FS00:0x15:- Expanded 79600 -> 307810 (decompressed 386.6 percent)
13:41:16:WU00:FS00:0x15:Called DecompressByteArray: compressed_data_size=79600 data_size=307810, decompressed_data_size=307810 diff=0
13:41:16:WU00:FS00:0x15:- Digital signature verified
13:41:16:WU00:FS00:0x15:
13:41:16:WU00:FS00:0x15:Project: 7660 (Run 963, Clone 0, Gen 8)
13:41:16:WU00:FS00:0x15:
13:41:16:WU00:FS00:0x15:Assembly optimizations on if available.
13:41:16:WU00:FS00:0x15:Entering M.D.
13:41:17:WU00:FS00:0x15:Tpr hash 00/wudata_01.tpr:  2327992468 1308474764 3265566474 1012946672 257706942
13:41:17:WU00:FS00:0x15:GPU device id=0
13:41:17:WU00:FS00:0x15:Working on Protein
13:41:17:WU00:FS00:0x15:Client config unavailable.
13:41:18:WU00:FS00:0x15:Finished fah_main status=59
13:41:18:WU00:FS00:0x15:mdrun_gpu returned 59
13:41:18:WU00:FS00:0x15:GPU memtest failure
13:41:18:WU00:FS00:0x15:
13:41:18:WU00:FS00:0x15:Folding@home Core Shutdown: GPU_MEMTEST_ERROR
13:41:18:WARNING:WU00:FS00:FahCore returned: GPU_MEMTEST_ERROR (124 = 0x7c)
13:42:15:WU00:FS00:Starting
13:42:15:WU00:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" "C:/Users/Space45/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe" -dir 00 -suffix 01 -version 703 -lifeline 4192 -checkpoint 5 -gpu 0 -gpu-vendor nvidia
13:42:15:WU00:FS00:Started FahCore on PID 8584
13:42:15:WU00:FS00:Core PID:7996
13:42:15:WU00:FS00:FahCore 0x15 started
13:42:16:WU00:FS00:0x15:
13:42:16:WU00:FS00:0x15:*------------------------------*
13:42:16:WU00:FS00:0x15:Folding@Home GPU Core
13:42:16:WU00:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
13:42:16:WU00:FS00:0x15:Build host             AmoebaRemote
13:42:16:WU00:FS00:0x15:Board Type             NVIDIA/CUDA
13:42:16:WU00:FS00:0x15:Core                   15
13:42:16:WU00:FS00:0x15:
13:42:16:WU00:FS00:0x15:Window's signal control handler registered.
13:42:16:WU00:FS00:0x15:Preparing to commence simulation
13:42:16:WU00:FS00:0x15:- Looking at optimizations...
13:42:16:WU00:FS00:0x15:DeleteFrameFiles: successfully deleted file=00/wudata_01.ckp
13:42:16:WU00:FS00:0x15:- Created dyn
13:42:16:WU00:FS00:0x15:- Files status OK
13:42:16:WU00:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
13:42:16:WU00:FS00:0x15:- Expanded 79600 -> 307810 (decompressed 386.6 percent)
13:42:16:WU00:FS00:0x15:Called DecompressByteArray: compressed_data_size=79600 data_size=307810, decompressed_data_size=307810 diff=0
13:42:16:WU00:FS00:0x15:- Digital signature verified
13:42:16:WU00:FS00:0x15:
13:42:16:WU00:FS00:0x15:Project: 7660 (Run 963, Clone 0, Gen 8)
13:42:16:WU00:FS00:0x15:
13:42:16:WU00:FS00:0x15:Assembly optimizations on if available.
13:42:16:WU00:FS00:0x15:Entering M.D.
13:42:17:WU00:FS00:0x15:Tpr hash 00/wudata_01.tpr:  2327992468 1308474764 3265566474 1012946672 257706942
13:42:17:WU00:FS00:0x15:GPU device id=0
13:42:18:WU00:FS00:0x15:Working on Protein
13:42:18:WU00:FS00:0x15:Client config unavailable.
13:42:18:WU00:FS00:0x15:Finished fah_main status=59
13:42:18:WU00:FS00:0x15:mdrun_gpu returned 59
13:42:18:WU00:FS00:0x15:GPU memtest failure
13:42:18:WU00:FS00:0x15:
13:42:18:WU00:FS00:0x15:Folding@home Core Shutdown: GPU_MEMTEST_ERROR
13:42:18:WARNING:WU00:FS00:FahCore returned: GPU_MEMTEST_ERROR (124 = 0x7c)
13:43:15:WU00:FS00:Starting
13:43:15:WU00:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" "C:/Users/Space45/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe" -dir 00 -suffix 01 -version 703 -lifeline 4192 -checkpoint 5 -gpu 0 -gpu-vendor nvidia
13:43:15:WU00:FS00:Started FahCore on PID 1196
13:43:15:WU00:FS00:Core PID:7500
13:43:15:WU00:FS00:FahCore 0x15 started
13:43:16:WU00:FS00:0x15:
13:43:16:WU00:FS00:0x15:*------------------------------*
13:43:16:WU00:FS00:0x15:Folding@Home GPU Core
13:43:16:WU00:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
13:43:16:WU00:FS00:0x15:Build host             AmoebaRemote
13:43:16:WU00:FS00:0x15:Board Type             NVIDIA/CUDA
13:43:16:WU00:FS00:0x15:Core                   15
13:43:16:WU00:FS00:0x15:
13:43:16:WU00:FS00:0x15:Window's signal control handler registered.
13:43:16:WU00:FS00:0x15:Preparing to commence simulation
13:43:16:WU00:FS00:0x15:- Looking at optimizations...
13:43:16:WU00:FS00:0x15:DeleteFrameFiles: successfully deleted file=00/wudata_01.ckp
13:43:16:WU00:FS00:0x15:- Created dyn
13:43:16:WU00:FS00:0x15:- Files status OK
13:43:16:WU00:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
13:43:16:WU00:FS00:0x15:- Expanded 79600 -> 307810 (decompressed 386.6 percent)
13:43:16:WU00:FS00:0x15:Called DecompressByteArray: compressed_data_size=79600 data_size=307810, decompressed_data_size=307810 diff=0
13:43:16:WU00:FS00:0x15:- Digital signature verified
13:43:16:WU00:FS00:0x15:
13:43:16:WU00:FS00:0x15:Project: 7660 (Run 963, Clone 0, Gen 8)
13:43:16:WU00:FS00:0x15:
13:43:16:WU00:FS00:0x15:Assembly optimizations on if available.
13:43:16:WU00:FS00:0x15:Entering M.D.
13:43:17:WU00:FS00:0x15:Tpr hash 00/wudata_01.tpr:  2327992468 1308474764 3265566474 1012946672 257706942
13:43:17:WU00:FS00:0x15:GPU device id=0
13:43:18:WU00:FS00:0x15:Working on Protein
13:43:18:WU00:FS00:0x15:Client config unavailable.
13:43:18:WU00:FS00:0x15:Finished fah_main status=59
13:43:18:WU00:FS00:0x15:mdrun_gpu returned 59
13:43:18:WU00:FS00:0x15:GPU memtest failure
13:43:18:WU00:FS00:0x15:
13:43:18:WU00:FS00:0x15:Folding@home Core Shutdown: GPU_MEMTEST_ERROR
13:43:18:WARNING:WU00:FS00:FahCore returned: GPU_MEMTEST_ERROR (124 = 0x7c)
13:43:18:WARNING:WU00:FS00:Too many errors, failing
13:43:18:WU00:FS00:Sending unit results: id:00 state:SEND error:FAILED project:7660 run:963 clone:0 gen:8 core:0x15 unit:0x0000000c664f2dd150f83fab68c248f7
13:43:18:WU00:FS00:Connecting to 171.64.65.105:8080
13:43:19:WU02:FS00:Connecting to assign-GPU.stanford.edu:80
13:43:19:WU00:FS00:Server responded WORK_QUIT (404)
13:43:19:WARNING:WU00:FS00:Server did not like results, dumping
13:43:19:WU00:FS00:Cleaning up
13:43:20:WU02:FS00:News: Welcome to Folding@Home
13:43:20:WU02:FS00:Assigned to work server 171.64.65.105
13:43:20:WU02:FS00:Requesting new work unit for slot 00: READY gpu:0:GF100 [GeForce GTX 470] from 171.64.65.105
13:43:20:WU02:FS00:Connecting to 171.64.65.105:8080
13:43:21:WU02:FS00:Downloading 122.74KiB
13:43:22:WU02:FS00:Download complete
13:43:22:WU02:FS00:Received Unit: id:02 state:DOWNLOAD error:NO_ERROR project:7625 run:17 clone:0 gen:175 core:0x15 unit:0x000000ff664f2dd14fe6153b1b7fd235
13:43:22:WU02:FS00:Starting
13:43:22:WU02:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" "C:/Users/Space45/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe" -dir 02 -suffix 01 -version 703 -lifeline 4192 -checkpoint 5 -gpu 0 -gpu-vendor nvidia
13:43:22:WU02:FS00:Started FahCore on PID 5816
13:43:22:WU02:FS00:Core PID:4408
13:43:22:WU02:FS00:FahCore 0x15 started
13:43:23:WU02:FS00:0x15:
13:43:23:WU02:FS00:0x15:*------------------------------*
13:43:23:WU02:FS00:0x15:Folding@Home GPU Core
13:43:23:WU02:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
13:43:23:WU02:FS00:0x15:Build host             AmoebaRemote
13:43:23:WU02:FS00:0x15:Board Type             NVIDIA/CUDA
13:43:23:WU02:FS00:0x15:Core                   15
13:43:23:WU02:FS00:0x15:
13:43:23:WU02:FS00:0x15:Window's signal control handler registered.
13:43:23:WU02:FS00:0x15:Preparing to commence simulation
13:43:23:WU02:FS00:0x15:- Looking at optimizations...
13:43:23:WU02:FS00:0x15:DeleteFrameFiles: successfully deleted file=02/wudata_01.ckp
13:43:23:WU02:FS00:0x15:- Created dyn
13:43:23:WU02:FS00:0x15:- Files status OK
13:43:23:WU02:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
13:43:23:WU02:FS00:0x15:- Expanded 125169 -> 502918 (decompressed 401.7 percent)
13:43:23:WU02:FS00:0x15:Called DecompressByteArray: compressed_data_size=125169 data_size=502918, decompressed_data_size=502918 diff=0
13:43:23:WU02:FS00:0x15:- Digital signature verified
13:43:23:WU02:FS00:0x15:
13:43:23:WU02:FS00:0x15:Project: 7625 (Run 17, Clone 0, Gen 175)
13:43:23:WU02:FS00:0x15:
13:43:23:WU02:FS00:0x15:Assembly optimizations on if available.
13:43:23:WU02:FS00:0x15:Entering M.D.
13:43:24:WU02:FS00:0x15:Tpr hash 02/wudata_01.tpr:  3869150238 846582451 1486108105 3475610688 732002762
13:43:24:WU02:FS00:0x15:GPU device id=0
13:43:25:WU02:FS00:0x15:Working on Protein
13:43:25:WU02:FS00:0x15:Client config unavailable.
13:43:25:WU02:FS00:0x15:Finished fah_main status=59
13:43:25:WU02:FS00:0x15:mdrun_gpu returned 59
13:43:25:WU02:FS00:0x15:GPU memtest failure
13:43:25:WU02:FS00:0x15:
13:43:25:WU02:FS00:0x15:Folding@home Core Shutdown: GPU_MEMTEST_ERROR
13:43:25:WARNING:WU02:FS00:FahCore returned: GPU_MEMTEST_ERROR (124 = 0x7c)
13:43:25:WU02:FS00:Starting
13:43:25:WU02:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" "C:/Users/Space45/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe" -dir 02 -suffix 01 -version 703 -lifeline 4192 -checkpoint 5 -gpu 0 -gpu-vendor nvidia
13:43:25:WU02:FS00:Started FahCore on PID 5124
13:43:25:WU02:FS00:Core PID:8308
13:43:25:WU02:FS00:FahCore 0x15 started
13:43:26:WU02:FS00:0x15:
13:43:26:WU02:FS00:0x15:*------------------------------*
13:43:26:WU02:FS00:0x15:Folding@Home GPU Core
13:43:26:WU02:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
13:43:26:WU02:FS00:0x15:Build host             AmoebaRemote
13:43:26:WU02:FS00:0x15:Board Type             NVIDIA/CUDA
13:43:26:WU02:FS00:0x15:Core                   15
13:43:26:WU02:FS00:0x15:
13:43:26:WU02:FS00:0x15:Window's signal control handler registered.
13:43:26:WU02:FS00:0x15:Preparing to commence simulation
13:43:26:WU02:FS00:0x15:- Looking at optimizations...
13:43:26:WU02:FS00:0x15:DeleteFrameFiles: successfully deleted file=02/wudata_01.ckp
13:43:26:WU02:FS00:0x15:- Created dyn
13:43:26:WU02:FS00:0x15:- Files status OK
13:43:26:WU02:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
13:43:26:WU02:FS00:0x15:- Expanded 125169 -> 502918 (decompressed 401.7 percent)
13:43:26:WU02:FS00:0x15:Called DecompressByteArray: compressed_data_size=125169 data_size=502918, decompressed_data_size=502918 diff=0
13:43:26:WU02:FS00:0x15:- Digital signature verified
13:43:26:WU02:FS00:0x15:
13:43:26:WU02:FS00:0x15:Project: 7625 (Run 17, Clone 0, Gen 175)
13:43:26:WU02:FS00:0x15:
13:43:26:WU02:FS00:0x15:Assembly optimizations on if available.
13:43:26:WU02:FS00:0x15:Entering M.D.
13:43:28:WU02:FS00:0x15:Tpr hash 02/wudata_01.tpr:  3869150238 846582451 1486108105 3475610688 732002762
13:43:28:WU02:FS00:0x15:GPU device id=0
13:43:28:WU02:FS00:0x15:Working on Protein
13:43:28:WU02:FS00:0x15:Client config unavailable.
13:43:28:WU02:FS00:0x15:Finished fah_main status=59
13:43:28:WU02:FS00:0x15:mdrun_gpu returned 59
13:43:28:WU02:FS00:0x15:GPU memtest failure
13:43:28:WU02:FS00:0x15:
13:43:28:WU02:FS00:0x15:Folding@home Core Shutdown: GPU_MEMTEST_ERROR
13:43:28:WARNING:WU02:FS00:FahCore returned: GPU_MEMTEST_ERROR (124 = 0x7c)
13:44:25:WU02:FS00:Starting
13:44:25:WU02:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" "C:/Users/Space45/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe" -dir 02 -suffix 01 -version 703 -lifeline 4192 -checkpoint 5 -gpu 0 -gpu-vendor nvidia
13:44:25:WU02:FS00:Started FahCore on PID 8256
13:44:25:WU02:FS00:Core PID:6592
13:44:25:WU02:FS00:FahCore 0x15 started
******************************* Date: 2013-03-01 *******************************
13:44:26:WU02:FS00:0x15:
13:44:26:WU02:FS00:0x15:*------------------------------*
13:44:26:WU02:FS00:0x15:Folding@Home GPU Core
13:44:26:WU02:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
13:44:26:WU02:FS00:0x15:Build host             AmoebaRemote
13:44:26:WU02:FS00:0x15:Board Type             NVIDIA/CUDA
13:44:26:WU02:FS00:0x15:Core                   15
13:44:26:WU02:FS00:0x15:
13:44:26:WU02:FS00:0x15:Window's signal control handler registered.
13:44:26:WU02:FS00:0x15:Preparing to commence simulation
13:44:26:WU02:FS00:0x15:- Looking at optimizations...
13:44:26:WU02:FS00:0x15:DeleteFrameFiles: successfully deleted file=02/wudata_01.ckp
13:44:26:WU02:FS00:0x15:- Created dyn
13:44:26:WU02:FS00:0x15:- Files status OK
13:44:26:WU02:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
13:44:26:WU02:FS00:0x15:- Expanded 125169 -> 502918 (decompressed 401.7 percent)
13:44:26:WU02:FS00:0x15:Called DecompressByteArray: compressed_data_size=125169 data_size=502918, decompressed_data_size=502918 diff=0
13:44:26:WU02:FS00:0x15:- Digital signature verified
13:44:26:WU02:FS00:0x15:
13:44:26:WU02:FS00:0x15:Project: 7625 (Run 17, Clone 0, Gen 175)
13:44:26:WU02:FS00:0x15:
13:44:26:WU02:FS00:0x15:Assembly optimizations on if available.
13:44:26:WU02:FS00:0x15:Entering M.D.
13:44:28:WU02:FS00:0x15:Tpr hash 02/wudata_01.tpr:  3869150238 846582451 1486108105 3475610688 732002762
13:44:28:WU02:FS00:0x15:GPU device id=0
13:44:28:WU02:FS00:0x15:Working on Protein
13:44:28:WU02:FS00:0x15:Client config unavailable.
13:44:28:WU02:FS00:0x15:Finished fah_main status=59
13:44:28:WU02:FS00:0x15:mdrun_gpu returned 59
13:44:28:WU02:FS00:0x15:GPU memtest failure
13:44:28:WU02:FS00:0x15:
13:44:28:WU02:FS00:0x15:Folding@home Core Shutdown: GPU_MEMTEST_ERROR
13:44:28:WARNING:WU02:FS00:FahCore returned: GPU_MEMTEST_ERROR (124 = 0x7c)
13:45:25:WU02:FS00:Starting
13:45:25:WU02:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" "C:/Users/Space45/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe" -dir 02 -suffix 01 -version 703 -lifeline 4192 -checkpoint 5 -gpu 0 -gpu-vendor nvidia
13:45:25:WU02:FS00:Started FahCore on PID 3936
13:45:25:WU02:FS00:Core PID:8064
13:45:25:WU02:FS00:FahCore 0x15 started
13:45:26:WU02:FS00:0x15:
13:45:26:WU02:FS00:0x15:*------------------------------*
13:45:26:WU02:FS00:0x15:Folding@Home GPU Core
13:45:26:WU02:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
13:45:26:WU02:FS00:0x15:Build host             AmoebaRemote
13:45:26:WU02:FS00:0x15:Board Type             NVIDIA/CUDA
13:45:26:WU02:FS00:0x15:Core                   15
13:45:26:WU02:FS00:0x15:
13:45:26:WU02:FS00:0x15:Window's signal control handler registered.
13:45:26:WU02:FS00:0x15:Preparing to commence simulation
13:45:26:WU02:FS00:0x15:- Looking at optimizations...
13:45:26:WU02:FS00:0x15:DeleteFrameFiles: successfully deleted file=02/wudata_01.ckp
13:45:26:WU02:FS00:0x15:- Created dyn
13:45:26:WU02:FS00:0x15:- Files status OK
13:45:26:WU02:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
13:45:26:WU02:FS00:0x15:- Expanded 125169 -> 502918 (decompressed 401.7 percent)
13:45:26:WU02:FS00:0x15:Called DecompressByteArray: compressed_data_size=125169 data_size=502918, decompressed_data_size=502918 diff=0
13:45:26:WU02:FS00:0x15:- Digital signature verified
13:45:26:WU02:FS00:0x15:
13:45:26:WU02:FS00:0x15:Project: 7625 (Run 17, Clone 0, Gen 175)
13:45:26:WU02:FS00:0x15:
13:45:26:WU02:FS00:0x15:Assembly optimizations on if available.
13:45:26:WU02:FS00:0x15:Entering M.D.
13:45:27:WU02:FS00:0x15:Tpr hash 02/wudata_01.tpr:  3869150238 846582451 1486108105 3475610688 732002762
13:45:27:WU02:FS00:0x15:GPU device id=0
13:45:28:WU02:FS00:0x15:Working on Protein
13:45:28:WU02:FS00:0x15:Client config unavailable.
13:45:28:WU02:FS00:0x15:Finished fah_main status=59
13:45:28:WU02:FS00:0x15:mdrun_gpu returned 59
13:45:28:WU02:FS00:0x15:GPU memtest failure
13:45:28:WU02:FS00:0x15:
13:45:28:WU02:FS00:0x15:Folding@home Core Shutdown: GPU_MEMTEST_ERROR
13:45:28:WARNING:WU02:FS00:FahCore returned: GPU_MEMTEST_ERROR (124 = 0x7c)
13:46:25:WU02:FS00:Starting
13:46:25:WU02:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" "C:/Users/Space45/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe" -dir 02 -suffix 01 -version 703 -lifeline 4192 -checkpoint 5 -gpu 0 -gpu-vendor nvidia
:WU02:FS00:Started FahCore on PID 8320
13:46:25:WU02:FS00:Core PID:8548
13:46:25:WU02:FS00:FahCore 0x15 started
13:46:26:WU02:FS00:0x15:
13:46:26:WU02:FS00:0x15:*------------------------------*
13:46:26:WU02:FS00:0x15:Folding@Home GPU Core
13:46:26:WU02:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
13:46:26:WU02:FS00:0x15:Build host             AmoebaRemote
13:46:26:WU02:FS00:0x15:Board Type             NVIDIA/CUDA
13:46:26:WU02:FS00:0x15:Core                   15
13:46:26:WU02:FS00:0x15:
13:46:26:WU02:FS00:0x15:Window's signal control handler registered.
13:46:26:WU02:FS00:0x15:Preparing to commence simulation
13:46:26:WU02:FS00:0x15:- Looking at optimizations...
13:46:26:WU02:FS00:0x15:DeleteFrameFiles: successfully deleted file=02/wudata_01.ckp
13:46:26:WU02:FS00:0x15:- Created dyn
13:46:26:WU02:FS00:0x15:- Files status OK
13:46:26:WU02:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
13:46:26:WU02:FS00:0x15:- Expanded 125169 -> 502918 (decompressed 401.7 percent)
13:46:26:WU02:FS00:0x15:Called DecompressByteArray: compressed_data_size=125169 data_size=502918, decompressed_data_size=502918 diff=0
13:46:26:WU02:FS00:0x15:- Digital signature verified
13:46:26:WU02:FS00:0x15:
13:46:26:WU02:FS00:0x15:Project: 7625 (Run 17, Clone 0, Gen 175)
13:46:26:WU02:FS00:0x15:
13:46:26:WU02:FS00:0x15:Assembly optimizations on if available.
13:46:26:WU02:FS00:0x15:Entering M.D.
13:46:28:WU02:FS00:0x15:Tpr hash 02/wudata_01.tpr:  3869150238 846582451 1486108105 3475610688 732002762
13:46:28:WU02:FS00:0x15:GPU device id=0
13:46:28:WU02:FS00:0x15:Working on Protein
13:46:28:WU02:FS00:0x15:Client config unavailable.
13:46:28:WU02:FS00:0x15:Finished fah_main status=59
13:46:28:WU02:FS00:0x15:mdrun_gpu returned 59
13:46:28:WU02:FS00:0x15:GPU memtest failure
13:46:28:WU02:FS00:0x15:
13:46:28:WU02:FS00:0x15:Folding@home Core Shutdown: GPU_MEMTEST_ERROR
13:46:28:WARNING:WU02:FS00:FahCore returned: GPU_MEMTEST_ERROR (124 = 0x7c)
13:46:28:WARNING:WU02:FS00:Too many errors, failing
13:46:28:WU02:FS00:Sending unit results: id:02 state:SEND error:FAILED project:7625 run:17 clone:0 gen:175 core:0x15 unit:0x000000ff664f2dd14fe6153b1b7fd235
13:46:29:WU02:FS00:Connecting to 171.64.65.105:8080
13:46:29:WU00:FS00:Connecting to assign-GPU.stanford.edu:80
13:46:29:WU02:FS00:Server responded WORK_QUIT (404)
13:46:29:WARNING:WU02:FS00:Server did not like results, dumping
13:46:29:WU02:FS00:Cleaning up
13:46:30:WU00:FS00:News: Welcome to Folding@Home
13:46:30:WU00:FS00:Assigned to work server 171.64.65.105
13:46:30:WU00:FS00:Requesting new work unit for slot 00: READY gpu:0:GF100 [GeForce GTX 470] from 171.64.65.105
13:46:30:WU00:FS00:Connecting to 171.64.65.105:8080
13:46:31:WU00:FS00:Downloading 78.03KiB
13:46:31:WU00:FS00:Download complete
13:46:32:WU00:FS00:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:7660 run:916 clone:0 gen:20 core:0x15 unit:0x0000001a664f2dd150f83fa5f3288aac
13:46:32:WU00:FS00:Starting
13:46:32:WU00:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" "C:/Users/Space45/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe" -dir 00 -suffix 01 -version 703 -lifeline 4192 -checkpoint 5 -gpu 0 -gpu-vendor nvidia
13:46:32:WU00:FS00:Started FahCore on PID 5740
13:46:32:WU00:FS00:Core PID:7164
13:46:32:WU00:FS00:FahCore 0x15 started
13:46:33:WU00:FS00:0x15:
13:46:33:WU00:FS00:0x15:*------------------------------*
13:46:33:WU00:FS00:0x15:Folding@Home GPU Core
13:46:33:WU00:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
13:46:33:WU00:FS00:0x15:Build host             AmoebaRemote
13:46:33:WU00:FS00:0x15:Board Type             NVIDIA/CUDA
13:46:33:WU00:FS00:0x15:Core                   15
13:46:33:WU00:FS00:0x15:
13:46:33:WU00:FS00:0x15:Window's signal control handler registered.
13:46:33:WU00:FS00:0x15:Preparing to commence simulation
13:46:33:WU00:FS00:0x15:- Looking at optimizations...
13:46:33:WU00:FS00:0x15:DeleteFrameFiles: successfully deleted file=00/wudata_01.ckp
13:46:33:WU00:FS00:0x15:- Created dyn
13:46:33:WU00:FS00:0x15:- Files status OK
13:46:33:WU00:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
13:46:33:WU00:FS00:0x15:- Expanded 79389 -> 307810 (decompressed 387.7 percent)
13:46:33:WU00:FS00:0x15:Called DecompressByteArray: compressed_data_size=79389 data_size=307810, decompressed_data_size=307810 diff=0
13:46:33:WU00:FS00:0x15:- Digital signature verified
13:46:33:WU00:FS00:0x15:
13:46:33:WU00:FS00:0x15:Project: 7660 (Run 916, Clone 0, Gen 20)
13:46:33:WU00:FS00:0x15:
13:46:33:WU00:FS00:0x15:Assembly optimizations on if available.
13:46:33:WU00:FS00:0x15:Entering M.D.
13:46:34:WU00:FS00:0x15:Tpr hash 00/wudata_01.tpr:  3868380541 4278632758 3304340208 4155827756 4207721904
13:46:34:WU00:FS00:0x15:GPU device id=0
13:46:34:WU00:FS00:0x15:Working on Protein
13:46:34:WU00:FS00:0x15:Client config unavailable.
13:46:34:WU00:FS00:0x15:Finished fah_main status=59
13:46:34:WU00:FS00:0x15:mdrun_gpu returned 59
13:46:34:WU00:FS00:0x15:GPU memtest failure
13:46:34:WU00:FS00:0x15:
13:46:34:WU00:FS00:0x15:Folding@home Core Shutdown: GPU_MEMTEST_ERROR
13:46:35:WARNING:WU00:FS00:FahCore returned: GPU_MEMTEST_ERROR (124 = 0x7c)
13:46:35:WU00:FS00:Starting
13:46:35:WU00:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" "C:/Users/Space45/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe" -dir 00 -suffix 01 -version 703 -lifeline 4192 -checkpoint 5 -gpu 0 -gpu-vendor nvidia
13:46:35:WU00:FS00:Started FahCore on PID 8312
13:46:35:WU00:FS00:Core PID:1580
13:46:35:WU00:FS00:FahCore 0x15 started
13:46:36:WU00:FS00:0x15:
13:46:36:WU00:FS00:0x15:*------------------------------*
13:46:36:WU00:FS00:0x15:Folding@Home GPU Core
13:46:36:WU00:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
13:46:36:WU00:FS00:0x15:Build host             AmoebaRemote
13:46:36:WU00:FS00:0x15:Board Type             NVIDIA/CUDA
13:46:36:WU00:FS00:0x15:Core                   15
13:46:36:WU00:FS00:0x15:
13:46:36:WU00:FS00:0x15:Window's signal control handler registered.
13:46:36:WU00:FS00:0x15:Preparing to commence simulation
13:46:36:WU00:FS00:0x15:- Looking at optimizations...
13:46:36:WU00:FS00:0x15:DeleteFrameFiles: successfully deleted file=00/wudata_01.ckp
13:46:36:WU00:FS00:0x15:- Created dyn
13:46:36:WU00:FS00:0x15:- Files status OK
13:46:36:WU00:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
13:46:36:WU00:FS00:0x15:- Expanded 79389 -> 307810 (decompressed 387.7 percent)
13:46:36:WU00:FS00:0x15:Called DecompressByteArray: compressed_data_size=79389 data_size=307810, decompressed_data_size=307810 diff=0
13:46:36:WU00:FS00:0x15:- Digital signature verified
13:46:36:WU00:FS00:0x15:
13:46:36:WU00:FS00:0x15:Project: 7660 (Run 916, Clone 0, Gen 20)
13:46:36:WU00:FS00:0x15:
13:46:36:WU00:FS00:0x15:Assembly optimizations on if available.
13:46:36:WU00:FS00:0x15:Entering M.D.
13:46:37:WU00:FS00:0x15:Tpr hash 00/wudata_01.tpr:  3868380541 4278632758 3304340208 4155827756 4207721904
13:46:37:WU00:FS00:0x15:GPU device id=0
13:46:37:WU00:FS00:0x15:Working on Protein
13:46:37:WU00:FS00:0x15:Client config unavailable.
13:46:37:WU00:FS00:0x15:Finished fah_main status=59
13:46:37:WU00:FS00:0x15:mdrun_gpu returned 59
13:46:37:WU00:FS00:0x15:GPU memtest failure
13:46:37:WU00:FS00:0x15:
13:46:37:WU00:FS00:0x15:Folding@home Core Shutdown: GPU_MEMTEST_ERROR
13:46:38:WARNING:WU00:FS00:FahCore returned: GPU_MEMTEST_ERROR (124 = 0x7c)
13:47:35:WU00:FS00:Starting
13:47:35:WU00:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" "C:/Users/Space45/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe" -dir 00 -suffix 01 -version 703 -lifeline 4192 -checkpoint 5 -gpu 0 -gpu-vendor nvidia
13:47:35:WU00:FS00:Started FahCore on PID 4152
13:47:35:WU00:FS00:Core PID:7764
13:47:35:WU00:FS00:FahCore 0x15 started
13:47:36:WU00:FS00:0x15:
13:47:36:WU00:FS00:0x15:*------------------------------*

Re: Get New GPU Task After Failed GPU Task

Posted: Fri Mar 01, 2013 11:59 pm
by Space45
Here's the remainder of the log:

Code: Select all

13:47:36:WU00:FS00:0x15:Folding@Home GPU Core
13:47:36:WU00:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
13:47:36:WU00:FS00:0x15:Build host             AmoebaRemote
13:47:36:WU00:FS00:0x15:Board Type             NVIDIA/CUDA
13:47:36:WU00:FS00:0x15:Core                   15
13:47:36:WU00:FS00:0x15:
13:47:36:WU00:FS00:0x15:Window's signal control handler registered.
13:47:36:WU00:FS00:0x15:Preparing to commence simulation
13:47:36:WU00:FS00:0x15:- Looking at optimizations...
13:47:36:WU00:FS00:0x15:DeleteFrameFiles: successfully deleted file=00/wudata_01.ckp
13:47:36:WU00:FS00:0x15:- Created dyn
13:47:36:WU00:FS00:0x15:- Files status OK
13:47:36:WU00:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
13:47:36:WU00:FS00:0x15:- Expanded 79389 -> 307810 (decompressed 387.7 percent)
13:47:36:WU00:FS00:0x15:Called DecompressByteArray: compressed_data_size=79389 data_size=307810, decompressed_data_size=307810 diff=0
13:47:36:WU00:FS00:0x15:- Digital signature verified
13:47:36:WU00:FS00:0x15:
13:47:36:WU00:FS00:0x15:Project: 7660 (Run 916, Clone 0, Gen 20)
13:47:36:WU00:FS00:0x15:
13:47:36:WU00:FS00:0x15:Assembly optimizations on if available.
13:47:36:WU00:FS00:0x15:Entering M.D.
13:47:37:WU00:FS00:0x15:Tpr hash 00/wudata_01.tpr:  3868380541 4278632758 3304340208 4155827756 4207721904
13:47:37:WU00:FS00:0x15:GPU device id=0
13:47:37:WU00:FS00:0x15:Working on Protein
13:47:37:WU00:FS00:0x15:Client config unavailable.
13:47:37:WU00:FS00:0x15:Finished fah_main status=59
13:47:37:WU00:FS00:0x15:mdrun_gpu returned 59
13:47:37:WU00:FS00:0x15:GPU memtest failure
13:47:37:WU00:FS00:0x15:
13:47:37:WU00:FS00:0x15:Folding@home Core Shutdown: GPU_MEMTEST_ERROR
13:47:38:WARNING:WU00:FS00:FahCore returned: GPU_MEMTEST_ERROR (124 = 0x7c)
13:48:35:WU00:FS00:Starting
13:48:35:WU00:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" "C:/Users/Space45/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe" -dir 00 -suffix 01 -version 703 -lifeline 4192 -checkpoint 5 -gpu 0 -gpu-vendor nvidia
13:48:35:WU00:FS00:Started FahCore on PID 1120
13:48:35:WU00:FS00:Core PID:600
13:48:35:WU00:FS00:FahCore 0x15 started
13:48:36:WU00:FS00:0x15:
13:48:36:WU00:FS00:0x15:*------------------------------*
13:48:36:WU00:FS00:0x15:Folding@Home GPU Core
13:48:36:WU00:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
13:48:36:WU00:FS00:0x15:Build host             AmoebaRemote
13:48:36:WU00:FS00:0x15:Board Type             NVIDIA/CUDA
13:48:36:WU00:FS00:0x15:Core                   15
13:48:36:WU00:FS00:0x15:
13:48:36:WU00:FS00:0x15:Window's signal control handler registered.
13:48:36:WU00:FS00:0x15:Preparing to commence simulation
13:48:36:WU00:FS00:0x15:- Looking at optimizations...
13:48:36:WU00:FS00:0x15:DeleteFrameFiles: successfully deleted file=00/wudata_01.ckp
13:48:36:WU00:FS00:0x15:- Created dyn
13:48:36:WU00:FS00:0x15:- Files status OK
13:48:36:WU00:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
13:48:36:WU00:FS00:0x15:- Expanded 79389 -> 307810 (decompressed 387.7 percent)
13:48:36:WU00:FS00:0x15:Called DecompressByteArray: compressed_data_size=79389 data_size=307810, decompressed_data_size=307810 diff=0
13:48:36:WU00:FS00:0x15:- Digital signature verified
13:48:36:WU00:FS00:0x15:
13:48:36:WU00:FS00:0x15:Project: 7660 (Run 916, Clone 0, Gen 20)
13:48:36:WU00:FS00:0x15:
13:48:36:WU00:FS00:0x15:Assembly optimizations on if available.
13:48:36:WU00:FS00:0x15:Entering M.D.
13:48:37:WU00:FS00:0x15:Tpr hash 00/wudata_01.tpr:  3868380541 4278632758 3304340208 4155827756 4207721904
13:48:37:WU00:FS00:0x15:GPU device id=0
13:48:37:WU00:FS00:0x15:Working on Protein
13:48:37:WU00:FS00:0x15:Client config unavailable.
13:48:37:WU00:FS00:0x15:Finished fah_main status=59
13:48:37:WU00:FS00:0x15:mdrun_gpu returned 59
13:48:37:WU00:FS00:0x15:GPU memtest failure
13:48:37:WU00:FS00:0x15:
13:48:37:WU00:FS00:0x15:Folding@home Core Shutdown: GPU_MEMTEST_ERROR
13:48:38:WARNING:WU00:FS00:FahCore returned: GPU_MEMTEST_ERROR (124 = 0x7c)
13:49:35:WU00:FS00:Starting
13:49:35:WU00:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" "C:/Users/Space45/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe" -dir 00 -suffix 01 -version 703 -lifeline 4192 -checkpoint 5 -gpu 0 -gpu-vendor nvidia
13:49:35:WU00:FS00:Started FahCore on PID 1544
13:49:35:WU00:FS00:Core PID:6372
13:49:35:WU00:FS00:FahCore 0x15 started
13:49:36:WU00:FS00:0x15:
13:49:36:WU00:FS00:0x15:*------------------------------*
13:49:36:WU00:FS00:0x15:Folding@Home GPU Core
13:49:36:WU00:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
13:49:36:WU00:FS00:0x15:Build host             AmoebaRemote
13:49:36:WU00:FS00:0x15:Board Type             NVIDIA/CUDA
13:49:36:WU00:FS00:0x15:Core                   15
13:49:36:WU00:FS00:0x15:
13:49:36:WU00:FS00:0x15:Window's signal control handler registered.
13:49:36:WU00:FS00:0x15:Preparing to commence simulation
13:49:36:WU00:FS00:0x15:- Looking at optimizations...
13:49:36:WU00:FS00:0x15:DeleteFrameFiles: successfully deleted file=00/wudata_01.ckp
13:49:36:WU00:FS00:0x15:- Created dyn
13:49:36:WU00:FS00:0x15:- Files status OK
13:49:36:WU00:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
13:49:36:WU00:FS00:0x15:- Expanded 79389 -> 307810 (decompressed 387.7 percent)
13:49:36:WU00:FS00:0x15:Called DecompressByteArray: compressed_data_size=79389 data_size=307810, decompressed_data_size=307810 diff=0
13:49:36:WU00:FS00:0x15:- Digital signature verified
13:49:36:WU00:FS00:0x15:
13:49:36:WU00:FS00:0x15:Project: 7660 (Run 916, Clone 0, Gen 20)
13:49:36:WU00:FS00:0x15:
13:49:36:WU00:FS00:0x15:Assembly optimizations on if available.
13:49:36:WU00:FS00:0x15:Entering M.D.
13:49:37:WU00:FS00:0x15:Tpr hash 00/wudata_01.tpr:  3868380541 4278632758 3304340208 4155827756 4207721904
13:49:37:WU00:FS00:0x15:GPU device id=0
13:49:37:WU00:FS00:0x15:Working on Protein
13:49:37:WU00:FS00:0x15:Client config unavailable.
13:49:37:WU00:FS00:0x15:Finished fah_main status=59
13:49:37:WU00:FS00:0x15:mdrun_gpu returned 59
13:49:37:WU00:FS00:0x15:GPU memtest failure
13:49:37:WU00:FS00:0x15:
13:49:37:WU00:FS00:0x15:Folding@home Core Shutdown: GPU_MEMTEST_ERROR
13:49:38:WARNING:WU00:FS00:FahCore returned: GPU_MEMTEST_ERROR (124 = 0x7c)
13:49:38:WARNING:WU00:FS00:Too many errors, failing
13:49:38:WU00:FS00:Sending unit results: id:00 state:SEND error:FAILED project:7660 run:916 clone:0 gen:20 core:0x15 unit:0x0000001a664f2dd150f83fa5f3288aac
13:49:38:WU00:FS00:Connecting to 171.64.65.105:8080
13:49:39:WU00:FS00:Server responded WORK_QUIT (404)
13:49:39:WARNING:WU00:FS00:Server did not like results, dumping
13:49:39:WU00:FS00:Cleaning up
******************************* Date: 2013-03-01 *******************************
23:42:47:FS00:Paused

Re: Get New GPU Task After Failed GPU Task

Posted: Sat Mar 02, 2013 12:26 am
by Jesse_V
13:43:28:WU02:FS00:0x15:Client config unavailable.
13:43:28:WU02:FS00:0x15:Finished fah_main status=59
13:43:28:WU02:FS00:0x15:mdrun_gpu returned 59
13:43:28:WU02:FS00:0x15:GPU memtest failure
13:43:28:WU02:FS00:0x15:
13:43:28:WU02:FS00:0x15:Folding@home Core Shutdown: GPU_MEMTEST_ERROR
13:43:28:WARNING:WU02:FS00:FahCore returned: GPU_MEMTEST_ERROR (124 = 0x7c)
Seems to be the recurring problem. It looks to me like something is going on with the memory system of your GPU.

Re: Get New GPU Task After Failed GPU Task

Posted: Sat Mar 02, 2013 12:30 am
by Space45
Jesse_V wrote:
13:43:28:WU02:FS00:0x15:Client config unavailable.
13:43:28:WU02:FS00:0x15:Finished fah_main status=59
13:43:28:WU02:FS00:0x15:mdrun_gpu returned 59
13:43:28:WU02:FS00:0x15:GPU memtest failure
13:43:28:WU02:FS00:0x15:
13:43:28:WU02:FS00:0x15:Folding@home Core Shutdown: GPU_MEMTEST_ERROR
13:43:28:WARNING:WU02:FS00:FahCore returned: GPU_MEMTEST_ERROR (124 = 0x7c)
Seems to be the recurring problem. It looks to me like something is going on with the memory system of your GPU.
Are there any memory tests I could run?

Re: Get New GPU Task After Failed GPU Task

Posted: Sat Mar 02, 2013 12:33 am
by Jesse_V
Space45 wrote:Are there any memory tests I could run?
MemtestG80: http://folding.stanford.edu/English/DownloadUtils#ntoc2
That page should be sufficient, but the author's page for that is https://simtk.org/home/memtest

Re: Get New GPU Task After Failed GPU Task

Posted: Sat Mar 02, 2013 1:47 am
by Space45
I ran 1000 iterations of MemtestG80 using 750MB of memory on my GeForce GTX 470 with stock settings and it found no errors. Is there anything else that could be causing this error?

Re: Get New GPU Task After Failed GPU Task

Posted: Sat Mar 02, 2013 2:00 am
by P5-133XL
Bad driver versions have cause GPU-MemTest-Errors in the past (though typically in AMD) also getting the wrong project type for your GPU (Getting a GPU3 project for a GPU only capable of GPU2) have caused them.

Re: Get New GPU Task After Failed GPU Task

Posted: Sat Mar 02, 2013 2:09 am
by Space45
P5-133XL wrote:Bad driver versions have cause GPU-MemTest-Errors in the past (though typically in AMD) also getting the wrong project type for your GPU (Getting a GPU3 project for a GPU only capable of GPU2) have caused them.
I'm downloading the latest driver version (314.07) and I'll see if it works better than my current version. Are there any known issues with GeForce driver 310.70?

Re: Get New GPU Task After Failed GPU Task

Posted: Sat Mar 02, 2013 3:49 am
by Space45
I've updated the driver and a GPU task is running. Hopefully, I'll know whether it's succeeded soon.
All times are UTC
Page 1 of 3

Powered by phpBB® Forum Software © phpBB Limited