7808 (2,180,57) - ETEA 11.30 days!

Moderators: Site Moderators, FAHC Science Team

Post Reply
telecentricity
Posts: 7
Joined: Fri Mar 01, 2013 4:40 am

7808 (2,180,57) - ETEA 11.30 days!

Post by telecentricity »

Hi,
I'm currently running PRCG 7808 (2, 180, 57) on a quad core i7-2600K. I am getting TPF of 4h 08 min and an ETA of 11.30 days with PPD of 141. This is way out of line from my other runs (usually ETA 8-12h, TPF of 3-4min and PPD of 6k to 8k). I've tried pausing and also rebooting my system to no effect. My GPU folding slot is running fine at a reasonable speed. I read about another 7808 that had a bad WU and I wonder if this is a similar problem. I'm attaching my log and configuration (since my reboot). Thanks very much for your help,
Gary

Code: Select all

*********************** Log Started 2013-03-01T04:47:23Z ***********************
04:47:23:************************* Foldinghome Client *************************
04:47:23:      Website: http://folding.stanford.edu/
04:47:23:    Copyright: (c) 2009-2013 Stanford University
04:47:23:       Author: Joseph Coffland <josephcauldrondevelopment.com>
04:47:23:         Args: 
04:47:23:       Config: C:/ProgramData/FAHClient/config.xml
04:47:23:******************************** Build ********************************
04:47:23:      Version: 7.3.6
04:47:23:         Date: Feb 18 2013
04:47:23:         Time: 15:25:17
04:47:23:      SVN Rev: 3923
04:47:23:       Branch: fah/trunk/client
04:47:23:     Compiler: Intel(R) C++ MSVC 1500 mode 1200
04:47:23:      Options: /TP /nologo /EHa /Qdiag-disable:4297,4103,1786,279 /Ox -arch:SSE
04:47:23:               /QaxSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 /Qopenmp /Qrestrict /MT /Qmkl
04:47:23:     Platform: win32 XP
04:47:23:         Bits: 32
04:47:23:         Mode: Release
04:47:23:******************************* System ********************************
04:47:23:          CPU: Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz
04:47:23:       CPU ID: GenuineIntel Family 6 Model 42 Stepping 7
04:47:23:         CPUs: 8
04:47:23:       Memory: 31.90GiB
04:47:23:  Free Memory: 28.48GiB
04:47:23:      Threads: WINDOWS_THREADS
04:47:23:  Has Battery: false
04:47:23:   On Battery: false
04:47:23:   UTC offset: -6
04:47:23:          PID: 6592
04:47:23:          CWD: C:/ProgramData/FAHClient
04:47:23:           OS: Windows 8 Enterprise
04:47:23:      OS Arch: AMD64
04:47:23:         GPUs: 1
04:47:23:        GPU 0: NVIDIA:2 GF110 [GeForce GTX 570 HD]
04:47:23:         CUDA: 2.0
04:47:23:  CUDA Driver: 5000
04:47:23:Win32 Service: false
04:47:23:***********************************************************************
04:47:23:<config>
04:47:23:  <!-- Folding Slot Configuration -->
04:47:23:  <power v='full'/>
04:47:23:
04:47:23:  <!-- Network -->
04:47:23:  <proxy v=':8080'/>
04:47:23:
04:47:23:  <!-- User Information -->
04:47:23:  <passkey v='********************************'/>
04:47:23:  <team v='40051'/>
04:47:23:  <user v='telecentricity'/>
04:47:23:
04:47:23:  <!-- Folding Slots -->
04:47:23:  <slot id='0' type='GPU'/>
04:47:23:  <slot id='1' type='CPU'/>
04:47:23:</config>
04:47:23:Trying to access database...
*********************** Log Started 2013-03-01T04:47:23Z ***********************
04:47:23:************************* Foldinghome Client *************************
04:47:23:      Website: http://folding.stanford.edu/
04:47:23:    Copyright: (c) 2009-2013 Stanford University
04:47:23:       Author: Joseph Coffland <josephcauldrondevelopment.com>
04:47:23:         Args: 
04:47:23:       Config: C:/ProgramData/FAHClient/config.xml
04:47:23:******************************** Build ********************************
04:47:23:      Version: 7.3.6
04:47:23:         Date: Feb 18 2013
04:47:23:         Time: 15:25:17
04:47:23:      SVN Rev: 3923
04:47:23:       Branch: fah/trunk/client
04:47:23:     Compiler: Intel(R) C++ MSVC 1500 mode 1200
04:47:23:      Options: /TP /nologo /EHa /Qdiag-disable:4297,4103,1786,279 /Ox -arch:SSE
04:47:23:               /QaxSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 /Qopenmp /Qrestrict /MT /Qmkl
04:47:23:     Platform: win32 XP
04:47:23:         Bits: 32
04:47:23:         Mode: Release
04:47:23:******************************* System ********************************
04:47:23:          CPU: Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz
04:47:23:       CPU ID: GenuineIntel Family 6 Model 42 Stepping 7
04:47:23:         CPUs: 8
04:47:23:       Memory: 31.90GiB
04:47:23:  Free Memory: 28.48GiB
04:47:23:      Threads: WINDOWS_THREADS
04:47:23:  Has Battery: false
04:47:23:   On Battery: false
04:47:23:   UTC offset: -6
04:47:23:          PID: 6592
04:47:23:          CWD: C:/ProgramData/FAHClient
04:47:23:           OS: Windows 8 Enterprise
04:47:23:      OS Arch: AMD64
04:47:23:         GPUs: 1
04:47:23:        GPU 0: NVIDIA:2 GF110 [GeForce GTX 570 HD]
04:47:23:         CUDA: 2.0
04:47:23:  CUDA Driver: 5000
04:47:23:Win32 Service: false
04:47:23:***********************************************************************
04:47:23:<config>
04:47:23:  <!-- Folding Slot Configuration -->
04:47:23:  <power v='full'/>
04:47:23:
04:47:23:  <!-- Network -->
04:47:23:  <proxy v=':8080'/>
04:47:23:
04:47:23:  <!-- User Information -->
04:47:23:  <passkey v='********************************'/>
04:47:23:  <team v='40051'/>
04:47:23:  <user v='telecentricity'/>
04:47:23:
04:47:23:  <!-- Folding Slots -->
04:47:23:  <slot id='0' type='GPU'/>
04:47:23:  <slot id='1' type='CPU'/>
04:47:23:</config>
04:47:23:Trying to access database...
04:47:23:Successfully acquired database lock
04:47:23:Enabled folding slot 00: READY gpu:0:GF110 [GeForce GTX 570 HD]
04:47:23:Enabled folding slot 01: READY cpu:8
04:47:23:WU01:FS01:Starting
04:47:23:WU01:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/ProgramData/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/Core_a4.fah/FahCore_a4.exe -dir 01 -suffix 01 -version 703 -lifeline 6592 -checkpoint 15 -np 8
04:47:23:WU01:FS01:Started FahCore on PID 8188
04:47:23:WU01:FS01:Core PID:5956
04:47:23:WU01:FS01:FahCore 0xa4 started
04:47:23:WU00:FS00:Starting
04:47:23:WU00:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/ProgramData/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 00 -suffix 01 -version 703 -lifeline 6592 -checkpoint 15 -gpu 0 -gpu-vendor nvidia
04:47:23:WU00:FS00:Started FahCore on PID 6572
04:47:23:WU00:FS00:Core PID:6368
04:47:23:WU00:FS00:FahCore 0x15 started
04:47:24:WU01:FS01:0xa4:
04:47:24:WU01:FS01:0xa4:*------------------------------*
04:47:24:WU01:FS01:0xa4:Folding@Home Gromacs GB Core
04:47:24:WU01:FS01:0xa4:Version 2.27 (Dec. 15, 2010)
04:47:24:WU01:FS01:0xa4:
04:47:24:WU01:FS01:0xa4:Preparing to commence simulation
04:47:24:WU01:FS01:0xa4:- Ensuring status. Please wait.
04:47:24:WU00:FS00:0x15:
04:47:24:WU00:FS00:0x15:*------------------------------*
04:47:24:WU00:FS00:0x15:Folding@Home GPU Core
04:47:24:WU00:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
04:47:24:WU00:FS00:0x15:Build host             AmoebaRemote
04:47:24:WU00:FS00:0x15:Board Type             NVIDIA/CUDA
04:47:24:WU00:FS00:0x15:Core                   15
04:47:24:WU00:FS00:0x15:
04:47:24:WU00:FS00:0x15:Window's signal control handler registered.
04:47:24:WU00:FS00:0x15:Preparing to commence simulation
04:47:24:WU00:FS00:0x15:- Ensuring status. Please wait.
04:47:33:WU01:FS01:0xa4:- Looking at optimizations...
04:47:33:WU01:FS01:0xa4:- Working with standard loops on this execution.
04:47:33:WU01:FS01:0xa4:- Previous termination of core was improper.
04:47:33:WU01:FS01:0xa4:- Files status OK
04:47:33:WU00:FS00:0x15:- Looking at optimizations...
04:47:33:WU00:FS00:0x15:- Working with standard loops on this execution.
04:47:33:WU00:FS00:0x15:- Previous termination of core was improper.
04:47:33:WU00:FS00:0x15:- Files status OK
04:47:33:WU00:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
04:47:33:WU00:FS00:0x15:- Expanded 58812 -> 259314 (decompressed 440.9 percent)
04:47:33:WU00:FS00:0x15:Called DecompressByteArray: compressed_data_size=58812 data_size=259314, decompressed_data_size=259314 diff=0
04:47:33:WU00:FS00:0x15:- Digital signature verified
04:47:33:WU00:FS00:0x15:
04:47:33:WU00:FS00:0x15:Project: 8071 (Run 118, Clone 11, Gen 4)
04:47:33:WU00:FS00:0x15:
04:47:33:WU00:FS00:0x15:Entering M.D.
04:47:33:WU01:FS01:0xa4:- Expanded 2054096 -> 5365960 (decompressed 261.2 percent)
04:47:33:WU01:FS01:0xa4:Called DecompressByteArray: compressed_data_size=2054096 data_size=5365960, decompressed_data_size=5365960 diff=0
04:47:33:WU01:FS01:0xa4:- Digital signature verified
04:47:33:WU01:FS01:0xa4:
04:47:33:WU01:FS01:0xa4:Project: 7808 (Run 2, Clone 180, Gen 57)
04:47:33:WU01:FS01:0xa4:
04:47:33:WU01:FS01:0xa4:Entering M.D.
04:47:35:WU00:FS00:0x15:Will resume from checkpoint file 00/wudata_01.ckp
04:47:35:WU00:FS00:0x15:Tpr hash 00/wudata_01.tpr:  4137678891 2023613714 3735999060 4019548950 179042560
04:47:35:WU00:FS00:0x15:GPU device id=0
04:47:35:WU00:FS00:0x15:Working on Glycine aRginine prOline Methionine Alanine Cystine Serine t= 118.00000
04:47:35:WU00:FS00:0x15:Client config unavailable.
04:47:35:WU00:FS00:0x15:Starting GUI Server
04:47:39:WU01:FS01:0xa4:Using Gromacs checkpoints
04:47:39:WU01:FS01:0xa4:Mapping NT from 8 to 8 
04:47:40:WU01:FS01:0xa4:Resuming from checkpoint
04:47:40:WU01:FS01:0xa4:Verified 01/wudata_01.log
04:47:40:WU01:FS01:0xa4:Verified 01/wudata_01.trr
04:47:40:WU01:FS01:0xa4:Verified 01/wudata_01.xtc
04:47:40:WU01:FS01:0xa4:Verified 01/wudata_01.edr
04:47:40:WU01:FS01:0xa4:Completed 518840 out of 1500000 steps  (34%)
04:47:43:3:127.0.0.1:New Web connection
04:48:37:WARNING:WU00:FS00:Detected clock skew (1 mins 04 secs), adjusting time estimates
04:48:37:WU00:FS00:0x15:Resuming from checkpoint
04:48:37:WU00:FS00:0x15:fcCheckPointResume: retreived and current tpr file hash:
04:48:37:WU00:FS00:0x15:   0   4137678891   4137678891
04:48:37:WU00:FS00:0x15:   1   2023613714   2023613714
04:48:37:WU00:FS00:0x15:   2   3735999060   3735999060
04:48:37:WU00:FS00:0x15:   3   4019548950   4019548950
04:48:37:WU00:FS00:0x15:   4    179042560    179042560
04:48:37:WU00:FS00:0x15:fcCheckPointResume: file hashes same.
04:48:37:WU00:FS00:0x15:fcCheckPointResume: state restored.
04:48:37:WU00:FS00:0x15:fcCheckPointResume: name 00/wudata_01.log Verified 00/wudata_01.log
04:48:37:WU00:FS00:0x15:fcCheckPointResume: name 00/wudata_01.trr Verified 00/wudata_01.trr
04:48:37:WU00:FS00:0x15:fcCheckPointResume: name 00/wudata_01.xtc Verified 00/wudata_01.xtc
04:48:37:WU00:FS00:0x15:fcCheckPointResume: name 00/wudata_01.edr Verified 00/wudata_01.edr
04:48:37:WU00:FS00:0x15:fcCheckPointResume: state restored 2
04:48:37:WU00:FS00:0x15:Resumed from checkpoint
04:48:37:WU00:FS00:0x15:Setting checkpoint frequency: 500000
04:48:37:WU00:FS00:0x15:Completed  12000001 out of 50000000 steps (24%).
04:51:10:WU00:FS00:0x15:Completed  12500000 out of 50000000 steps (25%).
04:53:44:WU00:FS00:0x15:Completed  13000000 out of 50000000 steps (26%).
Top
bruce
Posts: 20824
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: 7808 (2,180,57) - ETEA 11.30 days!

Post by bruce »

The ETA doesn't match what I see in that log. Give it some time to make a reasonable estimation.

I see that you progressed from 25% to 26% in 2m34s. If that's a typical rate of progress, you can complete 100% in 4h16m40s. You've already completed 26% so it should be completed about the time you expect it to be completed. GPU Project 8071 has a preferred deadline of 10 days so 4 hours is really very fast.

Project 7808 has a preferred deadline of 25 days, so 11 days is not unreasonable for a 4c8t CPU.
Top
Napoleon
Posts: 887
Joined: Wed May 26, 2010 2:31 pm
Hardware configuration: Atom330 (overclocked):
Windows 7 Ultimate 64bit
Intel Atom330 dualcore (4 HyperThreads)
NVidia GT430, core_15 work
2x2GB Kingston KVR1333D3N9K2/4G 1333MHz memory kit
Asus AT3IONT-I Deluxe motherboard
Location: Finland

Re: 7808 (2,180,57) - ETA 11.30 days!

Post by Napoleon »

Welcome to the forum, telecentricity.

The 2m34s is for the GPU slot. One thing I've noticed about P807x GPU WUs is that they're quite CPU intensive in real life, no matter what e.g Task Manager might say - seriously unbalancing SMP work. Dropping back to CPU:6 should do the trick for the A4 WU (P7808). CPU:7 may fail for some projects/WUs, yet many people use it nevertheless. I'd recommend CPU:6, but ultimately it's your call.
Win7 64bit, FAH v7, OC'd
2C/4T Atom330 3x667MHz - GT430 2x832.5MHz - ION iGPU 3x466.7MHz
NaCl - Core_15 - display
Top
mmonnin
Posts: 324
Joined: Wed Dec 05, 2007 1:27 am

Re: 7808 (2,180,57) - ETEA 11.30 days!

Post by mmonnin »

This is the first time this project has reported 4-5 hour TPFs...
Top
Qinsp
Posts: 216
Joined: Sun Oct 17, 2010 2:34 pm

Re: 7808 (2,180,57) - ETEA 11.30 days!

Post by Qinsp »

I've the Status page estimated TPF off by huge amounts.

Is the TPF you are seeing in the log or estimate?
Quality Inspection - Corona, CA, USA
Dimensional Inspection Laboratory
Pat McSwain, President
Top
telecentricity
Posts: 7
Joined: Fri Mar 01, 2013 4:40 am

Re: 7808 (2,180,57) - ETEA 11.30 days!

Post by telecentricity »

Thanks for the fast replies. I dropped the CPU from 8 down to 6 per Napoleon. Now I'm getting TPF of 9:22 with ETA of 9h38m. Wow! Not sure why that worked but it did. I think this was real - it had stumbled along at TPF 4h for a couple of days before I rebooted. There is another thread below by ptschnack which reproduces the problem. Thanks for the help,
Gary
Top
bruce
Posts: 20824
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: 7808 (2,180,57) - ETEA 11.30 days!

Post by bruce »

Note that when you use all of your CPUs (in this case, 8) any interruptions from other applications will cause a major loss of performance since SMP runs at the speed of the slowest thread. By changing the setting to 6, you've left two virtual cores available for other things that might be running on your system. Check performance monitor. You should see the FahCore using a consistent total of 6/8 = 75% of the CPU time. You should also see essentially 0% for other applications. If you're seeing a few percent consistently going to anything else that's running, that's probably the complete explanation of the problem. SMP can never perform well unless all of its threads are balanced (aka "symmetric").

When you restart the client, the previous log is backed up to the subdirectory "logs" in the FAH data directory. Please post a segment section of one of them showing the 4h TPF. (Unfortunately there is no evidence in the log if the previous paragraph applies to your system.) The log posted above shows the project restarting at (34%) [Actually 34.72%] but does not show additional messages from which we can calculate a TPF.
Top
telecentricity
Posts: 7
Joined: Fri Mar 01, 2013 4:40 am

Re: 7808 (2,180,57) - ETEA 11.30 days!

Post by telecentricity »

Bruce,
Thanks very much for your explanation and your patience. Below is the first of two logs prior to the one I submitted, starting with the download. I hope they help you analyze the problem.
Kind regards,
Gary

Code: Select all

*********************** Log Started 2013-02-28T00:40:16Z ***********************
00:40:16:************************* Folding@home Client *************************
00:40:16:      Website: http://folding.stanford.edu/
00:40:16:    Copyright: (c) 2009-2013 Stanford University
00:40:16:       Author: Joseph Coffland <joseph@cauldrondevelopment.com>
00:40:16:         Args: 
00:40:16:       Config: C:/ProgramData/FAHClient/config.xml
00:40:16:******************************** Build ********************************
00:40:16:      Version: 7.3.6
00:40:16:         Date: Feb 18 2013
00:40:16:         Time: 15:25:17
00:40:16:      SVN Rev: 3923
00:40:16:       Branch: fah/trunk/client
00:40:16:     Compiler: Intel(R) C++ MSVC 1500 mode 1200
00:40:16:      Options: /TP /nologo /EHa /Qdiag-disable:4297,4103,1786,279 /Ox -arch:SSE
00:40:16:               /QaxSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 /Qopenmp /Qrestrict /MT /Qmkl
00:40:16:     Platform: win32 XP
00:40:16:         Bits: 32
00:40:16:         Mode: Release
00:40:16:******************************* System ********************************
00:40:16:          CPU: Intel(R) Core(TM) i7-2600K CPU @ 3.40GHz
00:40:16:       CPU ID: GenuineIntel Family 6 Model 42 Stepping 7
00:40:16:         CPUs: 8
00:40:16:       Memory: 31.90GiB
00:40:16:  Free Memory: 28.52GiB
00:40:16:      Threads: WINDOWS_THREADS
00:40:16:  Has Battery: false
00:40:16:   On Battery: false
00:40:16:   UTC offset: -6
00:40:16:          PID: 8388
00:40:16:          CWD: C:/ProgramData/FAHClient
00:40:16:           OS: Windows 8 Enterprise
00:40:16:      OS Arch: AMD64
00:40:16:         GPUs: 1
00:40:16:        GPU 0: NVIDIA:2 GF110 [GeForce GTX 570 HD]
00:40:16:         CUDA: 2.0
00:40:16:  CUDA Driver: 5000
00:40:16:Win32 Service: false
00:40:16:***********************************************************************
00:40:16:<config>
00:40:16:  <!-- Folding Slot Configuration -->
00:40:16:  <power v='full'/>
00:40:16:
00:40:16:  <!-- Network -->
00:40:16:  <proxy v=':8080'/>
00:40:16:
00:40:16:  <!-- User Information -->
00:40:16:  <passkey v='********************************'/>
00:40:16:  <team v='40051'/>
00:40:16:  <user v='telecentricity'/>
00:40:16:
00:40:16:  <!-- Folding Slots -->
00:40:16:  <slot id='0' type='GPU'/>
00:40:16:  <slot id='1' type='CPU'/>
00:40:16:</config>
00:40:16:Trying to access database...
00:40:16:Successfully acquired database lock
00:40:16:Enabled folding slot 00: READY gpu:0:GF110 [GeForce GTX 570 HD]
00:40:16:Enabled folding slot 01: READY cpu:8
00:40:16:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:7624 run:335 clone:3 gen:40 core:0x15 unit:0x00000030664f2dd14fe61276ae9eebf5
00:40:16:WU00:FS00:Uploading 807.72KiB to 171.64.65.105
00:40:16:WU01:FS01:Starting
00:40:16:WU00:FS00:Connecting to 171.64.65.105:8080
00:40:16:WU01:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/ProgramData/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/Core_a4.fah/FahCore_a4.exe -dir 01 -suffix 01 -version 703 -lifeline 8388 -checkpoint 15 -np 8
00:40:16:WU01:FS01:Started FahCore on PID 9300
00:40:16:WU01:FS01:Core PID:9312
00:40:16:WU01:FS01:FahCore 0xa4 started
00:40:16:WU02:FS00:Connecting to assign-GPU.stanford.edu:80
00:40:16:WU01:FS01:0xa4:
00:40:16:WU01:FS01:0xa4:*------------------------------*
00:40:16:WU01:FS01:0xa4:Folding@Home Gromacs GB Core
00:40:16:WU01:FS01:0xa4:Version 2.27 (Dec. 15, 2010)
00:40:16:WU01:FS01:0xa4:
00:40:16:WU01:FS01:0xa4:Preparing to commence simulation
00:40:16:WU01:FS01:0xa4:- Ensuring status. Please wait.
00:40:17:WU02:FS00:News: Welcome to Folding@Home
00:40:17:WU02:FS00:Assigned to work server 171.64.65.105
00:40:17:WU02:FS00:Requesting new work unit for slot 00: READY gpu:0:GF110 [GeForce GTX 570 HD] from 171.64.65.105
00:40:17:WU02:FS00:Connecting to 171.64.65.105:8080
00:40:17:WU00:FS00:Upload complete
00:40:17:WU00:FS00:Server responded WORK_ACK (400)
00:40:17:WU00:FS00:Final credit estimate, 14093.00 points
00:40:17:WU00:FS00:Cleaning up
00:40:18:WU02:FS00:Downloading 122.13KiB
00:40:18:WU02:FS00:Download complete
00:40:18:WU02:FS00:Received Unit: id:02 state:DOWNLOAD error:NO_ERROR project:7625 run:709 clone:1 gen:9 core:0x15 unit:0x0000000c664f2dd14fe619371e442d39
00:40:18:WU02:FS00:Starting
00:40:18:WU02:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/ProgramData/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 02 -suffix 01 -version 703 -lifeline 8388 -checkpoint 15 -gpu 0 -gpu-vendor nvidia
00:40:18:WU02:FS00:Started FahCore on PID 9464
00:40:18:WU02:FS00:Core PID:9476
00:40:18:WU02:FS00:FahCore 0x15 started
00:40:19:WU02:FS00:0x15:
00:40:19:WU02:FS00:0x15:*------------------------------*
00:40:19:WU02:FS00:0x15:Folding@Home GPU Core
00:40:19:WU02:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
00:40:19:WU02:FS00:0x15:Build host             AmoebaRemote
00:40:19:WU02:FS00:0x15:Board Type             NVIDIA/CUDA
00:40:19:WU02:FS00:0x15:Core                   15
00:40:19:WU02:FS00:0x15:
00:40:19:WU02:FS00:0x15:Window's signal control handler registered.
00:40:19:WU02:FS00:0x15:Preparing to commence simulation
00:40:19:WU02:FS00:0x15:- Looking at optimizations...
00:40:19:WU02:FS00:0x15:DeleteFrameFiles: successfully deleted file=02/wudata_01.ckp
00:40:19:WU02:FS00:0x15:- Created dyn
00:40:19:WU02:FS00:0x15:- Files status OK
00:40:19:WU02:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
00:40:19:WU02:FS00:0x15:- Expanded 124547 -> 502918 (decompressed 403.7 percent)
00:40:19:WU02:FS00:0x15:Called DecompressByteArray: compressed_data_size=124547 data_size=502918, decompressed_data_size=502918 diff=0
00:40:19:WU02:FS00:0x15:- Digital signature verified
00:40:19:WU02:FS00:0x15:
00:40:19:WU02:FS00:0x15:Project: 7625 (Run 709, Clone 1, Gen 9)
00:40:19:WU02:FS00:0x15:
00:40:19:WU02:FS00:0x15:Assembly optimizations on if available.
00:40:19:WU02:FS00:0x15:Entering M.D.
00:40:21:WU02:FS00:0x15:Tpr hash 02/wudata_01.tpr:  3275045972 1625723097 2082118532 2286997234 4269661401
00:40:21:WU02:FS00:0x15:GPU device id=0
00:40:21:WU02:FS00:0x15:Working on Protein
00:40:21:WU02:FS00:0x15:Client config unavailable.
00:40:21:WU02:FS00:0x15:Starting GUI Server
00:40:26:WU01:FS01:0xa4:- Looking at optimizations...
00:40:26:WU01:FS01:0xa4:- Working with standard loops on this execution.
00:40:26:WU01:FS01:0xa4:- Previous termination of core was improper.
00:40:26:WU01:FS01:0xa4:- Files status OK
00:40:26:WU01:FS01:0xa4:- Expanded 2054096 -> 5365960 (decompressed 261.2 percent)
00:40:26:WU01:FS01:0xa4:Called DecompressByteArray: compressed_data_size=2054096 data_size=5365960, decompressed_data_size=5365960 diff=0
00:40:26:WU01:FS01:0xa4:- Digital signature verified
00:40:26:WU01:FS01:0xa4:
00:40:26:WU01:FS01:0xa4:Project: 7808 (Run 2, Clone 180, Gen 57)
00:40:26:WU01:FS01:0xa4:
00:40:26:WU01:FS01:0xa4:Entering M.D.
00:40:26:3:127.0.0.1:New Web connection
00:40:32:WU01:FS01:0xa4:Using Gromacs checkpoints
00:40:32:WU01:FS01:0xa4:Mapping NT from 8 to 8 
00:40:32:WU01:FS01:0xa4:Resuming from checkpoint
00:40:32:WU01:FS01:0xa4:Verified 01/wudata_01.log
00:40:32:WU01:FS01:0xa4:Verified 01/wudata_01.trr
00:40:32:WU01:FS01:0xa4:Verified 01/wudata_01.xtc
00:40:32:WU01:FS01:0xa4:Verified 01/wudata_01.edr
00:40:32:WU01:FS01:0xa4:Completed 427320 out of 1500000 steps  (28%)
00:41:23:WU02:FS00:0x15:Setting checkpoint frequency: 400000
00:41:23:WU02:FS00:0x15:Completed         3 out of 40000000 steps (0%).
00:48:07:WU02:FS00:0x15:Completed    400000 out of 40000000 steps (1%).
00:54:51:WU02:FS00:0x15:Completed    800000 out of 40000000 steps (2%).
01:01:34:WU02:FS00:0x15:Completed   1200000 out of 40000000 steps (3%).
01:08:17:WU02:FS00:0x15:Completed   1600000 out of 40000000 steps (4%).
01:15:01:WU02:FS00:0x15:Completed   2000000 out of 40000000 steps (5%).
01:21:44:WU02:FS00:0x15:Completed   2400000 out of 40000000 steps (6%).
01:21:44:WARNING:WU01:FS01:Detected clock skew (3 mins 46 secs), adjusting time estimates
01:21:44:WARNING:WU02:FS00:Detected clock skew (3 mins 46 secs), adjusting time estimates
01:22:35:WU01:FS01:0xa4:Completed 435000 out of 1500000 steps  (29%)
01:28:28:WU02:FS00:0x15:Completed   2800000 out of 40000000 steps (7%).
01:35:13:WU02:FS00:0x15:Completed   3200000 out of 40000000 steps (8%).
01:41:57:WU02:FS00:0x15:Completed   3600000 out of 40000000 steps (9%).
01:48:40:WU02:FS00:0x15:Completed   4000000 out of 40000000 steps (10%).
01:55:23:WU02:FS00:0x15:Completed   4400000 out of 40000000 steps (11%).
02:02:06:WU02:FS00:0x15:Completed   4800000 out of 40000000 steps (12%).
02:08:52:WU02:FS00:0x15:Completed   5200000 out of 40000000 steps (13%).
02:15:36:WU02:FS00:0x15:Completed   5600000 out of 40000000 steps (14%).
02:22:20:WU02:FS00:0x15:Completed   6000000 out of 40000000 steps (15%).
02:29:04:WU02:FS00:0x15:Completed   6400000 out of 40000000 steps (16%).
02:35:47:WU02:FS00:0x15:Completed   6800000 out of 40000000 steps (17%).
02:42:31:WU02:FS00:0x15:Completed   7200000 out of 40000000 steps (18%).
02:49:14:WU02:FS00:0x15:Completed   7600000 out of 40000000 steps (19%).
02:55:58:WU02:FS00:0x15:Completed   8000000 out of 40000000 steps (20%).
03:02:41:WU02:FS00:0x15:Completed   8400000 out of 40000000 steps (21%).
03:09:25:WU02:FS00:0x15:Completed   8800000 out of 40000000 steps (22%).
03:16:08:WU02:FS00:0x15:Completed   9200000 out of 40000000 steps (23%).
03:22:52:WU02:FS00:0x15:Completed   9600000 out of 40000000 steps (24%).
03:29:35:WU02:FS00:0x15:Completed  10000000 out of 40000000 steps (25%).
03:36:19:WU02:FS00:0x15:Completed  10400000 out of 40000000 steps (26%).
03:43:02:WU02:FS00:0x15:Completed  10800000 out of 40000000 steps (27%).
03:49:46:WU02:FS00:0x15:Completed  11200000 out of 40000000 steps (28%).
03:56:29:WU02:FS00:0x15:Completed  11600000 out of 40000000 steps (29%).
04:03:13:WU02:FS00:0x15:Completed  12000000 out of 40000000 steps (30%).
04:09:56:WU02:FS00:0x15:Completed  12400000 out of 40000000 steps (31%).
04:16:40:WU02:FS00:0x15:Completed  12800000 out of 40000000 steps (32%).
04:23:23:WU02:FS00:0x15:Completed  13200000 out of 40000000 steps (33%).
04:30:07:WU02:FS00:0x15:Completed  13600000 out of 40000000 steps (34%).
04:36:50:WU02:FS00:0x15:Completed  14000000 out of 40000000 steps (35%).
04:43:33:WU02:FS00:0x15:Completed  14400000 out of 40000000 steps (36%).
04:50:17:WU02:FS00:0x15:Completed  14800000 out of 40000000 steps (37%).
04:57:00:WU02:FS00:0x15:Completed  15200000 out of 40000000 steps (38%).
05:03:43:WU02:FS00:0x15:Completed  15600000 out of 40000000 steps (39%).
05:10:27:WU02:FS00:0x15:Completed  16000000 out of 40000000 steps (40%).
05:17:10:WU02:FS00:0x15:Completed  16400000 out of 40000000 steps (41%).
05:23:53:WU02:FS00:0x15:Completed  16800000 out of 40000000 steps (42%).
05:30:36:WU02:FS00:0x15:Completed  17200000 out of 40000000 steps (43%).
05:37:19:WU02:FS00:0x15:Completed  17600000 out of 40000000 steps (44%).
05:44:03:WU02:FS00:0x15:Completed  18000000 out of 40000000 steps (45%).
05:50:46:WU02:FS00:0x15:Completed  18400000 out of 40000000 steps (46%).
05:57:29:WU02:FS00:0x15:Completed  18800000 out of 40000000 steps (47%).
06:04:12:WU02:FS00:0x15:Completed  19200000 out of 40000000 steps (48%).
06:10:56:WU02:FS00:0x15:Completed  19600000 out of 40000000 steps (49%).
06:17:39:WU02:FS00:0x15:Completed  20000000 out of 40000000 steps (50%).
06:24:22:WU02:FS00:0x15:Completed  20400000 out of 40000000 steps (51%).
06:31:05:WU02:FS00:0x15:Completed  20800000 out of 40000000 steps (52%).
06:37:49:WU02:FS00:0x15:Completed  21200000 out of 40000000 steps (53%).
******************************* Date: 2013-02-28 *******************************
06:44:32:WU02:FS00:0x15:Completed  21600000 out of 40000000 steps (54%).
06:51:15:WU02:FS00:0x15:Completed  22000000 out of 40000000 steps (55%).
06:57:58:WU02:FS00:0x15:Completed  22400000 out of 40000000 steps (56%).
07:04:42:WU02:FS00:0x15:Completed  22800000 out of 40000000 steps (57%).
07:11:25:WU02:FS00:0x15:Completed  23200000 out of 40000000 steps (58%).
07:18:08:WU02:FS00:0x15:Completed  23600000 out of 40000000 steps (59%).
07:24:52:WU02:FS00:0x15:Completed  24000000 out of 40000000 steps (60%).
07:31:35:WU02:FS00:0x15:Completed  24400000 out of 40000000 steps (61%).
07:31:49:WU01:FS01:0xa4:Completed 450000 out of 1500000 steps  (30%)
07:38:18:WU02:FS00:0x15:Completed  24800000 out of 40000000 steps (62%).
07:45:01:WU02:FS00:0x15:Completed  25200000 out of 40000000 steps (63%).
07:51:45:WU02:FS00:0x15:Completed  25600000 out of 40000000 steps (64%).
07:58:28:WU02:FS00:0x15:Completed  26000000 out of 40000000 steps (65%).
08:05:12:WU02:FS00:0x15:Completed  26400000 out of 40000000 steps (66%).
08:11:55:WU02:FS00:0x15:Completed  26800000 out of 40000000 steps (67%).
08:18:38:WU02:FS00:0x15:Completed  27200000 out of 40000000 steps (68%).
08:25:21:WU02:FS00:0x15:Completed  27600000 out of 40000000 steps (69%).
08:32:05:WU02:FS00:0x15:Completed  28000000 out of 40000000 steps (70%).
08:38:48:WU02:FS00:0x15:Completed  28400000 out of 40000000 steps (71%).
08:45:31:WU02:FS00:0x15:Completed  28800000 out of 40000000 steps (72%).
08:52:15:WU02:FS00:0x15:Completed  29200000 out of 40000000 steps (73%).
08:58:58:WU02:FS00:0x15:Completed  29600000 out of 40000000 steps (74%).
09:05:41:WU02:FS00:0x15:Completed  30000000 out of 40000000 steps (75%).
09:12:24:WU02:FS00:0x15:Completed  30400000 out of 40000000 steps (76%).
09:19:08:WU02:FS00:0x15:Completed  30800000 out of 40000000 steps (77%).
09:26:49:WU02:FS00:0x15:Completed  31200000 out of 40000000 steps (78%).
09:40:34:WU02:FS00:0x15:Completed  31600000 out of 40000000 steps (79%).
09:49:43:WU02:FS00:0x15:Completed  32000000 out of 40000000 steps (80%).
09:56:27:WU02:FS00:0x15:Completed  32400000 out of 40000000 steps (81%).
10:03:13:WU02:FS00:0x15:Completed  32800000 out of 40000000 steps (82%).
10:09:56:WU02:FS00:0x15:Completed  33200000 out of 40000000 steps (83%).
10:13:07:52:127.0.0.1:New Web connection
10:13:11:WARNING:Exception: 49:127.0.0.1: Send error: 10053: An established connection was aborted by the software in your host machine.
10:16:42:WU02:FS00:0x15:Completed  33600000 out of 40000000 steps (84%).
10:18:54:WARNING:Exception: 57:127.0.0.1: Send error: 10053: An established connection was aborted by the software in your host machine.
10:19:15:61:127.0.0.1:New Web connection
10:23:28:WU02:FS00:0x15:Completed  34000000 out of 40000000 steps (85%).
10:30:13:WU02:FS00:0x15:Completed  34400000 out of 40000000 steps (86%).
10:36:58:WU02:FS00:0x15:Completed  34800000 out of 40000000 steps (87%).
10:37:33:WU01:FS01:0xa4:Completed 465000 out of 1500000 steps  (31%)
10:43:42:WU02:FS00:0x15:Completed  35200000 out of 40000000 steps (88%).
10:50:27:WU02:FS00:0x15:Completed  35600000 out of 40000000 steps (89%).
10:57:11:WU02:FS00:0x15:Completed  36000000 out of 40000000 steps (90%).
11:03:56:WU02:FS00:0x15:Completed  36400000 out of 40000000 steps (91%).
11:10:39:WU02:FS00:0x15:Completed  36800000 out of 40000000 steps (92%).
11:17:23:WU02:FS00:0x15:Completed  37200000 out of 40000000 steps (93%).
11:24:06:WU02:FS00:0x15:Completed  37600000 out of 40000000 steps (94%).
11:30:50:WU02:FS00:0x15:Completed  38000000 out of 40000000 steps (95%).
11:37:33:WU02:FS00:0x15:Completed  38400000 out of 40000000 steps (96%).
11:44:17:WU02:FS00:0x15:Completed  38800000 out of 40000000 steps (97%).
11:51:00:WU02:FS00:0x15:Completed  39200000 out of 40000000 steps (98%).
11:51:01:WU00:FS00:Connecting to assign-GPU.stanford.edu:80
11:51:01:WU00:FS00:News: Welcome to Folding@Home
11:51:01:WU00:FS00:Assigned to work server 171.64.65.105
11:51:01:WU00:FS00:Requesting new work unit for slot 00: RUNNING gpu:0:GF110 [GeForce GTX 570 HD] from 171.64.65.105
11:51:01:WU00:FS00:Connecting to 171.64.65.105:8080
11:51:01:WU00:FS00:Downloading 124.36KiB
11:51:02:WU00:FS00:Download complete
11:51:02:WU00:FS00:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:7626 run:44 clone:0 gen:97 core:0x15 unit:0x00000080664f2dd14fe619fecfc47e9d
11:57:44:WU02:FS00:0x15:Completed  39600000 out of 40000000 steps (99%).
12:04:27:WU02:FS00:0x15:Completed  40000000 out of 40000000 steps (100%).
12:04:28:WU02:FS00:0x15:Finished fah_main status=0
12:04:28:WU02:FS00:0x15:Successful run
12:04:28:WU02:FS00:0x15:DynamicWrapper: Finished Work Unit: sleep=10000
12:04:38:WU02:FS00:0x15:Reserved 761532 bytes for xtc file; Cosm status=0
12:04:38:WU02:FS00:0x15:Allocated 761532 bytes for xtc file
12:04:38:WU02:FS00:0x15:- Reading up to 761532 from "02/wudata_01.xtc": Read 761532
12:04:38:WU02:FS00:0x15:Read 761532 bytes from xtc file; available packet space=785668932
12:04:38:WU02:FS00:0x15:xtc file hash check passed.
12:04:38:WU02:FS00:0x15:Reserved 47832 47832 785668932 bytes for arc file=<02/wudata_01.trr> Cosm status=0
12:04:38:WU02:FS00:0x15:Allocated 47832 bytes for arc file
12:04:38:WU02:FS00:0x15:- Reading up to 47832 from "02/wudata_01.trr": Read 47832
12:04:38:WU02:FS00:0x15:Read 47832 bytes from arc file; available packet space=785621100
12:04:38:WU02:FS00:0x15:trr file hash check passed.
12:04:38:WU02:FS00:0x15:Allocated 544 bytes for edr file
12:04:38:WU02:FS00:0x15:Read bedfile
12:04:38:WU02:FS00:0x15:edr file hash check passed.
12:04:38:WU02:FS00:0x15:Allocated 36746 bytes for logfile
12:04:38:WU02:FS00:0x15:Read logfile
12:04:38:WU02:FS00:0x15:GuardedRun: success in DynamicWrapper
12:04:38:WU02:FS00:0x15:GuardedRun: done
12:04:38:WU02:FS00:0x15:Run: GuardedRun completed.
12:04:41:WU02:FS00:0x15:+ Opened results file
12:04:41:WU02:FS00:0x15:- Writing 847166 bytes of core data to disk...
12:04:41:WU02:FS00:0x15:Done: 846654 -> 818598 (compressed to 96.6 percent)
12:04:41:WU02:FS00:0x15:  ... Done.
12:04:41:WU02:FS00:0x15:DeleteFrameFiles: successfully deleted file=02/wudata_01.ckp
12:04:42:WU02:FS00:0x15:Shutting down core 
12:04:42:WU02:FS00:0x15:
12:04:42:WU02:FS00:0x15:Folding@home Core Shutdown: FINISHED_UNIT
12:04:42:WU02:FS00:FahCore returned: FINISHED_UNIT (100 = 0x64)
12:04:42:WU02:FS00:Sending unit results: id:02 state:SEND error:NO_ERROR project:7625 run:709 clone:1 gen:9 core:0x15 unit:0x0000000c664f2dd14fe619371e442d39
12:04:42:WU02:FS00:Uploading 799.91KiB to 171.64.65.105
12:04:42:WU02:FS00:Connecting to 171.64.65.105:8080
12:04:42:WU00:FS00:Starting
12:04:42:WU00:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/ProgramData/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 00 -suffix 01 -version 703 -lifeline 8388 -checkpoint 15 -gpu 0 -gpu-vendor nvidia
12:04:42:WU00:FS00:Started FahCore on PID 10672
12:04:42:WU00:FS00:Core PID:6388
12:04:42:WU00:FS00:FahCore 0x15 started
12:04:43:WU00:FS00:0x15:
12:04:43:WU00:FS00:0x15:*------------------------------*
12:04:43:WU00:FS00:0x15:Folding@Home GPU Core
12:04:43:WU00:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
12:04:43:WU00:FS00:0x15:Build host             AmoebaRemote
12:04:43:WU00:FS00:0x15:Board Type             NVIDIA/CUDA
12:04:43:WU00:FS00:0x15:Core                   15
12:04:43:WU00:FS00:0x15:
12:04:43:WU00:FS00:0x15:Window's signal control handler registered.
12:04:43:WU00:FS00:0x15:Preparing to commence simulation
12:04:43:WU00:FS00:0x15:- Looking at optimizations...
12:04:43:WU00:FS00:0x15:DeleteFrameFiles: successfully deleted file=00/wudata_01.ckp
12:04:43:WU00:FS00:0x15:- Created dyn
12:04:43:WU00:FS00:0x15:- Files status OK
12:04:43:WU00:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
12:04:43:WU00:FS00:0x15:- Expanded 126828 -> 507182 (decompressed 399.8 percent)
12:04:43:WU00:FS00:0x15:Called DecompressByteArray: compressed_data_size=126828 data_size=507182, decompressed_data_size=507182 diff=0
12:04:43:WU00:FS00:0x15:- Digital signature verified
12:04:43:WU00:FS00:0x15:
12:04:43:WU00:FS00:0x15:Project: 7626 (Run 44, Clone 0, Gen 97)
12:04:43:WU00:FS00:0x15:
12:04:43:WU00:FS00:0x15:Assembly optimizations on if available.
12:04:43:WU00:FS00:0x15:Entering M.D.
12:04:44:WU02:FS00:Upload complete
12:04:44:WU02:FS00:Server responded WORK_ACK (400)
12:04:44:WU02:FS00:Final credit estimate, 14093.00 points
12:04:44:WU02:FS00:Cleaning up
12:04:44:WU00:FS00:0x15:Tpr hash 00/wudata_01.tpr:  2293198413 3759896912 710603028 1508577764 398535787
12:04:44:WU00:FS00:0x15:GPU device id=0
12:04:44:WU00:FS00:0x15:Working on Protein
12:04:44:WU00:FS00:0x15:Client config unavailable.
12:04:45:WU00:FS00:0x15:Starting GUI Server
12:05:46:WU00:FS00:0x15:Setting checkpoint frequency: 400000
12:05:46:WU00:FS00:0x15:Completed         3 out of 40000000 steps (0%).
12:12:29:WU00:FS00:0x15:Completed    400000 out of 40000000 steps (1%).
12:19:12:WU00:FS00:0x15:Completed    800000 out of 40000000 steps (2%).
12:25:56:WU00:FS00:0x15:Completed   1200000 out of 40000000 steps (3%).
12:32:40:WU00:FS00:0x15:Completed   1600000 out of 40000000 steps (4%).
12:39:23:WU00:FS00:0x15:Completed   2000000 out of 40000000 steps (5%).
******************************* Date: 2013-02-28 *******************************
12:46:06:WU00:FS00:0x15:Completed   2400000 out of 40000000 steps (6%).
12:52:50:WU00:FS00:0x15:Completed   2800000 out of 40000000 steps (7%).
12:59:34:WU00:FS00:0x15:Completed   3200000 out of 40000000 steps (8%).
13:06:17:WU00:FS00:0x15:Completed   3600000 out of 40000000 steps (9%).
13:13:01:WU00:FS00:0x15:Completed   4000000 out of 40000000 steps (10%).
13:19:44:WU00:FS00:0x15:Completed   4400000 out of 40000000 steps (11%).
13:26:28:WU00:FS00:0x15:Completed   4800000 out of 40000000 steps (12%).
13:33:11:WU00:FS00:0x15:Completed   5200000 out of 40000000 steps (13%).
13:39:55:WU00:FS00:0x15:Completed   5600000 out of 40000000 steps (14%).
13:46:38:WU00:FS00:0x15:Completed   6000000 out of 40000000 steps (15%).
13:53:21:WU00:FS00:0x15:Completed   6400000 out of 40000000 steps (16%).
14:00:05:WU00:FS00:0x15:Completed   6800000 out of 40000000 steps (17%).
14:06:48:WU00:FS00:0x15:Completed   7200000 out of 40000000 steps (18%).
14:13:31:WU00:FS00:0x15:Completed   7600000 out of 40000000 steps (19%).
14:20:15:WU00:FS00:0x15:Completed   8000000 out of 40000000 steps (20%).
14:26:58:WU00:FS00:0x15:Completed   8400000 out of 40000000 steps (21%).
14:33:41:WU00:FS00:0x15:Completed   8800000 out of 40000000 steps (22%).
14:40:24:WU00:FS00:0x15:Completed   9200000 out of 40000000 steps (23%).
14:47:08:WU00:FS00:0x15:Completed   9600000 out of 40000000 steps (24%).
14:53:51:WU00:FS00:0x15:Completed  10000000 out of 40000000 steps (25%).
15:00:34:WU00:FS00:0x15:Completed  10400000 out of 40000000 steps (26%).
15:07:18:WU00:FS00:0x15:Completed  10800000 out of 40000000 steps (27%).
15:14:01:WU00:FS00:0x15:Completed  11200000 out of 40000000 steps (28%).
15:20:44:WU00:FS00:0x15:Completed  11600000 out of 40000000 steps (29%).
15:27:28:WU00:FS00:0x15:Completed  12000000 out of 40000000 steps (30%).
15:34:11:WU00:FS00:0x15:Completed  12400000 out of 40000000 steps (31%).
15:40:54:WU00:FS00:0x15:Completed  12800000 out of 40000000 steps (32%).
15:47:37:WU00:FS00:0x15:Completed  13200000 out of 40000000 steps (33%).
15:54:21:WU00:FS00:0x15:Completed  13600000 out of 40000000 steps (34%).
16:01:04:WU00:FS00:0x15:Completed  14000000 out of 40000000 steps (35%).
16:07:47:WU00:FS00:0x15:Completed  14400000 out of 40000000 steps (36%).
16:14:31:WU00:FS00:0x15:Completed  14800000 out of 40000000 steps (37%).
16:21:14:WU00:FS00:0x15:Completed  15200000 out of 40000000 steps (38%).
16:27:57:WU00:FS00:0x15:Completed  15600000 out of 40000000 steps (39%).
16:34:41:WU00:FS00:0x15:Completed  16000000 out of 40000000 steps (40%).
16:41:24:WU00:FS00:0x15:Completed  16400000 out of 40000000 steps (41%).
16:44:20:WU01:FS01:0xa4:Completed 480000 out of 1500000 steps  (32%)
16:48:07:WU00:FS00:0x15:Completed  16800000 out of 40000000 steps (42%).
16:54:51:WU00:FS00:0x15:Completed  17200000 out of 40000000 steps (43%).
17:01:34:WU00:FS00:0x15:Completed  17600000 out of 40000000 steps (44%).
17:08:17:WU00:FS00:0x15:Completed  18000000 out of 40000000 steps (45%).
17:15:01:WU00:FS00:0x15:Completed  18400000 out of 40000000 steps (46%).
17:21:44:WU00:FS00:0x15:Completed  18800000 out of 40000000 steps (47%).
17:28:27:WU00:FS00:0x15:Completed  19200000 out of 40000000 steps (48%).
17:35:10:WU00:FS00:0x15:Completed  19600000 out of 40000000 steps (49%).
17:41:54:WU00:FS00:0x15:Completed  20000000 out of 40000000 steps (50%).
17:48:37:WU00:FS00:0x15:Completed  20400000 out of 40000000 steps (51%).
17:55:20:WU00:FS00:0x15:Completed  20800000 out of 40000000 steps (52%).
18:02:04:WU00:FS00:0x15:Completed  21200000 out of 40000000 steps (53%).
18:08:47:WU00:FS00:0x15:Completed  21600000 out of 40000000 steps (54%).
18:15:30:WU00:FS00:0x15:Completed  22000000 out of 40000000 steps (55%).
18:22:14:WU00:FS00:0x15:Completed  22400000 out of 40000000 steps (56%).
18:28:57:WU00:FS00:0x15:Completed  22800000 out of 40000000 steps (57%).
18:35:40:WU00:FS00:0x15:Completed  23200000 out of 40000000 steps (58%).
18:42:24:WU00:FS00:0x15:Completed  23600000 out of 40000000 steps (59%).
******************************* Date: 2013-02-28 *******************************
18:49:07:WU00:FS00:0x15:Completed  24000000 out of 40000000 steps (60%).
18:55:50:WU00:FS00:0x15:Completed  24400000 out of 40000000 steps (61%).
19:02:33:WU00:FS00:0x15:Completed  24800000 out of 40000000 steps (62%).
19:09:17:WU00:FS00:0x15:Completed  25200000 out of 40000000 steps (63%).
19:16:00:WU00:FS00:0x15:Completed  25600000 out of 40000000 steps (64%).
19:22:44:WU00:FS00:0x15:Completed  26000000 out of 40000000 steps (65%).
19:29:27:WU00:FS00:0x15:Completed  26400000 out of 40000000 steps (66%).
19:36:10:WU00:FS00:0x15:Completed  26800000 out of 40000000 steps (67%).
19:42:53:WU00:FS00:0x15:Completed  27200000 out of 40000000 steps (68%).
19:49:37:WU00:FS00:0x15:Completed  27600000 out of 40000000 steps (69%).
19:56:20:WU00:FS00:0x15:Completed  28000000 out of 40000000 steps (70%).
20:03:04:WU00:FS00:0x15:Completed  28400000 out of 40000000 steps (71%).
20:09:47:WU00:FS00:0x15:Completed  28800000 out of 40000000 steps (72%).
20:16:30:WU00:FS00:0x15:Completed  29200000 out of 40000000 steps (73%).
20:23:13:WU00:FS00:0x15:Completed  29600000 out of 40000000 steps (74%).
20:29:57:WU00:FS00:0x15:Completed  30000000 out of 40000000 steps (75%).
20:36:40:WU00:FS00:0x15:Completed  30400000 out of 40000000 steps (76%).
20:43:23:WU00:FS00:0x15:Completed  30800000 out of 40000000 steps (77%).
20:50:07:WU00:FS00:0x15:Completed  31200000 out of 40000000 steps (78%).
20:52:58:WU01:FS01:0xa4:Completed 495000 out of 1500000 steps  (33%)
20:56:51:WU00:FS00:0x15:Completed  31600000 out of 40000000 steps (79%).
21:03:35:WU00:FS00:0x15:Completed  32000000 out of 40000000 steps (80%).
21:10:19:WU00:FS00:0x15:Completed  32400000 out of 40000000 steps (81%).
21:17:03:WU00:FS00:0x15:Completed  32800000 out of 40000000 steps (82%).
21:23:47:WU00:FS00:0x15:Completed  33200000 out of 40000000 steps (83%).
21:30:31:WU00:FS00:0x15:Completed  33600000 out of 40000000 steps (84%).
21:37:14:WU00:FS00:0x15:Completed  34000000 out of 40000000 steps (85%).
21:43:58:WU00:FS00:0x15:Completed  34400000 out of 40000000 steps (86%).
21:50:41:WU00:FS00:0x15:Completed  34800000 out of 40000000 steps (87%).
21:57:34:WU00:FS00:0x15:Completed  35200000 out of 40000000 steps (88%).
22:04:17:WU00:FS00:0x15:Completed  35600000 out of 40000000 steps (89%).
22:11:03:WU00:FS00:0x15:Completed  36000000 out of 40000000 steps (90%).
22:17:46:WU00:FS00:0x15:Completed  36400000 out of 40000000 steps (91%).
22:24:30:WU00:FS00:0x15:Completed  36800000 out of 40000000 steps (92%).
22:31:19:WU00:FS00:0x15:Completed  37200000 out of 40000000 steps (93%).
22:38:06:WU00:FS00:0x15:Completed  37600000 out of 40000000 steps (94%).
22:44:52:WU00:FS00:0x15:Completed  38000000 out of 40000000 steps (95%).
22:51:36:WU00:FS00:0x15:Completed  38400000 out of 40000000 steps (96%).
22:58:29:WU00:FS00:0x15:Completed  38800000 out of 40000000 steps (97%).
23:05:13:WU00:FS00:0x15:Completed  39200000 out of 40000000 steps (98%).
23:05:14:WU02:FS00:Connecting to assign-GPU.stanford.edu:80
23:05:14:WU02:FS00:News: Welcome to Folding@Home
23:05:14:WU02:FS00:Assigned to work server 171.67.108.36
23:05:14:WU02:FS00:Requesting new work unit for slot 00: RUNNING gpu:0:GF110 [GeForce GTX 570 HD] from 171.67.108.36
23:05:14:WU02:FS00:Connecting to 171.67.108.36:8080
23:05:15:WU02:FS00:Downloading 57.49KiB
23:05:15:WU02:FS00:Download complete
23:05:15:WU02:FS00:Received Unit: id:02 state:DOWNLOAD error:NO_ERROR project:8071 run:241 clone:4 gen:10 core:0x15 unit:0x0000000c6652edb45122e225b8e119e7
23:12:02:WU00:FS00:0x15:Completed  39600000 out of 40000000 steps (99%).
23:18:46:WU00:FS00:0x15:Completed  40000000 out of 40000000 steps (100%).
23:18:47:WU00:FS00:0x15:Finished fah_main status=0
23:18:47:WU00:FS00:0x15:Successful run
23:18:47:WU00:FS00:0x15:DynamicWrapper: Finished Work Unit: sleep=10000
23:18:57:WU00:FS00:0x15:Reserved 767852 bytes for xtc file; Cosm status=0
23:18:57:WU00:FS00:0x15:Allocated 767852 bytes for xtc file
23:18:57:WU00:FS00:0x15:- Reading up to 767852 from "00/wudata_01.xtc": Read 767852
23:18:57:WU00:FS00:0x15:Read 767852 bytes from xtc file; available packet space=785662612
23:18:57:WU00:FS00:0x15:xtc file hash check passed.
23:18:57:WU00:FS00:0x15:Reserved 48264 48264 785662612 bytes for arc file=<00/wudata_01.trr> Cosm status=0
23:18:57:WU00:FS00:0x15:Allocated 48264 bytes for arc file
23:18:57:WU00:FS00:0x15:- Reading up to 48264 from "00/wudata_01.trr": Read 48264
23:18:57:WU00:FS00:0x15:Read 48264 bytes from arc file; available packet space=785614348
23:18:57:WU00:FS00:0x15:trr file hash check passed.
23:18:57:WU00:FS00:0x15:Allocated 544 bytes for edr file
23:18:57:WU00:FS00:0x15:Read bedfile
23:18:57:WU00:FS00:0x15:edr file hash check passed.
23:18:57:WU00:FS00:0x15:Allocated 36748 bytes for logfile
23:18:57:WU00:FS00:0x15:Read logfile
23:18:57:WU00:FS00:0x15:GuardedRun: success in DynamicWrapper
23:18:57:WU00:FS00:0x15:GuardedRun: done
23:18:57:WU00:FS00:0x15:Run: GuardedRun completed.
23:18:59:WU00:FS00:0x15:+ Opened results file
23:18:59:WU00:FS00:0x15:- Writing 853920 bytes of core data to disk...
23:18:59:WU00:FS00:0x15:Done: 853408 -> 825253 (compressed to 96.7 percent)
23:18:59:WU00:FS00:0x15:  ... Done.
23:18:59:WU00:FS00:0x15:DeleteFrameFiles: successfully deleted file=00/wudata_01.ckp
23:18:59:WU00:FS00:0x15:Shutting down core 
23:18:59:WU00:FS00:0x15:
23:18:59:WU00:FS00:0x15:Folding@home Core Shutdown: FINISHED_UNIT
23:18:59:WU00:FS00:FahCore returned: FINISHED_UNIT (100 = 0x64)
23:18:59:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:7626 run:44 clone:0 gen:97 core:0x15 unit:0x00000080664f2dd14fe619fecfc47e9d
23:18:59:WU00:FS00:Uploading 806.41KiB to 171.64.65.105
23:18:59:WU00:FS00:Connecting to 171.64.65.105:8080
23:19:00:WU02:FS00:Starting
23:19:00:WU02:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/ProgramData/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 02 -suffix 01 -version 703 -lifeline 8388 -checkpoint 15 -gpu 0 -gpu-vendor nvidia
23:19:00:WU02:FS00:Started FahCore on PID 1156
23:19:00:WU02:FS00:Core PID:11132
23:19:00:WU02:FS00:FahCore 0x15 started
23:19:00:WU02:FS00:0x15:
23:19:00:WU02:FS00:0x15:*------------------------------*
23:19:00:WU02:FS00:0x15:Folding@Home GPU Core
23:19:00:WU02:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
23:19:00:WU02:FS00:0x15:Build host             AmoebaRemote
23:19:00:WU02:FS00:0x15:Board Type             NVIDIA/CUDA
23:19:00:WU02:FS00:0x15:Core                   15
23:19:00:WU02:FS00:0x15:
23:19:00:WU02:FS00:0x15:Window's signal control handler registered.
23:19:00:WU02:FS00:0x15:Preparing to commence simulation
23:19:00:WU02:FS00:0x15:- Looking at optimizations...
23:19:00:WU02:FS00:0x15:DeleteFrameFiles: successfully deleted file=02/wudata_01.ckp
23:19:00:WU02:FS00:0x15:- Created dyn
23:19:00:WU02:FS00:0x15:- Files status OK
23:19:00:WU02:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
23:19:00:WU02:FS00:0x15:- Expanded 58354 -> 259314 (decompressed 444.3 percent)
23:19:00:WU02:FS00:0x15:Called DecompressByteArray: compressed_data_size=58354 data_size=259314, decompressed_data_size=259314 diff=0
23:19:00:WU02:FS00:0x15:- Digital signature verified
23:19:00:WU02:FS00:0x15:
23:19:00:WU02:FS00:0x15:Project: 8071 (Run 241, Clone 4, Gen 10)
23:19:00:WU02:FS00:0x15:
23:19:00:WU02:FS00:0x15:Assembly optimizations on if available.
23:19:00:WU02:FS00:0x15:Entering M.D.
23:19:01:WU00:FS00:Upload complete
23:19:01:WU00:FS00:Server responded WORK_ACK (400)
23:19:01:WU00:FS00:Final credit estimate, 14093.00 points
23:19:01:WU00:FS00:Cleaning up
23:19:02:WU02:FS00:0x15:Tpr hash 02/wudata_01.tpr:  2936282967 4001514755 1064928222 926171899 3862711347
23:19:02:WU02:FS00:0x15:GPU device id=0
23:19:02:WU02:FS00:0x15:Working on Glycine aRginine prOline Methionine Alanine Cystine Serine t= 241.00000
23:19:02:WU02:FS00:0x15:Client config unavailable.
23:19:02:WU02:FS00:0x15:Starting GUI Server
23:20:03:WU02:FS00:0x15:Setting checkpoint frequency: 500000
23:20:03:WU02:FS00:0x15:Completed         3 out of 50000000 steps (0%).
23:22:34:WU02:FS00:0x15:Completed    500000 out of 50000000 steps (1%).
23:25:06:WU02:FS00:0x15:Completed   1000000 out of 50000000 steps (2%).
23:27:37:WU02:FS00:0x15:Completed   1500000 out of 50000000 steps (3%).
23:30:09:WU02:FS00:0x15:Completed   2000000 out of 50000000 steps (4%).
23:32:40:WU02:FS00:0x15:Completed   2500000 out of 50000000 steps (5%).
23:35:12:WU02:FS00:0x15:Completed   3000000 out of 50000000 steps (6%).
23:37:44:WU02:FS00:0x15:Completed   3500000 out of 50000000 steps (7%).
23:40:15:WU02:FS00:0x15:Completed   4000000 out of 50000000 steps (8%).
23:42:47:WU02:FS00:0x15:Completed   4500000 out of 50000000 steps (9%).
23:45:19:WU02:FS00:0x15:Completed   5000000 out of 50000000 steps (10%).
23:47:50:WU02:FS00:0x15:Completed   5500000 out of 50000000 steps (11%).
23:50:22:WU02:FS00:0x15:Completed   6000000 out of 50000000 steps (12%).
23:52:54:WU02:FS00:0x15:Completed   6500000 out of 50000000 steps (13%).
23:55:25:WU02:FS00:0x15:Completed   7000000 out of 50000000 steps (14%).
23:58:19:WU02:FS00:0x15:Completed   7500000 out of 50000000 steps (15%).
00:00:51:WU02:FS00:0x15:Completed   8000000 out of 50000000 steps (16%).
00:03:22:WU02:FS00:0x15:Completed   8500000 out of 50000000 steps (17%).
00:05:54:WU02:FS00:0x15:Completed   9000000 out of 50000000 steps (18%).
00:08:25:WU02:FS00:0x15:Completed   9500000 out of 50000000 steps (19%).
00:10:57:WU02:FS00:0x15:Completed  10000000 out of 50000000 steps (20%).
00:13:29:WU02:FS00:0x15:Completed  10500000 out of 50000000 steps (21%).
00:16:00:WU02:FS00:0x15:Completed  11000000 out of 50000000 steps (22%).
00:18:32:WU02:FS00:0x15:Completed  11500000 out of 50000000 steps (23%).
00:21:03:WU02:FS00:0x15:Completed  12000000 out of 50000000 steps (24%).
00:23:35:WU02:FS00:0x15:Completed  12500000 out of 50000000 steps (25%).
00:26:06:WU02:FS00:0x15:Completed  13000000 out of 50000000 steps (26%).
00:28:38:WU02:FS00:0x15:Completed  13500000 out of 50000000 steps (27%).
00:31:10:WU02:FS00:0x15:Completed  14000000 out of 50000000 steps (28%).
00:33:41:WU02:FS00:0x15:Completed  14500000 out of 50000000 steps (29%).
00:36:12:WU02:FS00:0x15:Completed  15000000 out of 50000000 steps (30%).
00:38:44:WU02:FS00:0x15:Completed  15500000 out of 50000000 steps (31%).
00:41:16:WU02:FS00:0x15:Completed  16000000 out of 50000000 steps (32%).
00:43:47:WU02:FS00:0x15:Completed  16500000 out of 50000000 steps (33%).
00:46:19:WU02:FS00:0x15:Completed  17000000 out of 50000000 steps (34%).
00:48:50:WU02:FS00:0x15:Completed  17500000 out of 50000000 steps (35%).
******************************* Date: 2013-03-01 *******************************
00:51:22:WU02:FS00:0x15:Completed  18000000 out of 50000000 steps (36%).
00:53:53:WU02:FS00:0x15:Completed  18500000 out of 50000000 steps (37%).
00:56:25:WU02:FS00:0x15:Completed  19000000 out of 50000000 steps (38%).
00:58:57:WU02:FS00:0x15:Completed  19500000 out of 50000000 steps (39%).
01:01:28:WU02:FS00:0x15:Completed  20000000 out of 50000000 steps (40%).
01:04:00:WU02:FS00:0x15:Completed  20500000 out of 50000000 steps (41%).
01:06:31:WU02:FS00:0x15:Completed  21000000 out of 50000000 steps (42%).
01:09:03:WU02:FS00:0x15:Completed  21500000 out of 50000000 steps (43%).
01:11:34:WU02:FS00:0x15:Completed  22000000 out of 50000000 steps (44%).
01:14:06:WU02:FS00:0x15:Completed  22500000 out of 50000000 steps (45%).
01:16:38:WU02:FS00:0x15:Completed  23000000 out of 50000000 steps (46%).
01:19:09:WU02:FS00:0x15:Completed  23500000 out of 50000000 steps (47%).
01:21:41:WU02:FS00:0x15:Completed  24000000 out of 50000000 steps (48%).
01:24:12:WU02:FS00:0x15:Completed  24500000 out of 50000000 steps (49%).
01:26:44:WU02:FS00:0x15:Completed  25000000 out of 50000000 steps (50%).
01:29:15:WU02:FS00:0x15:Completed  25500000 out of 50000000 steps (51%).
01:31:47:WU02:FS00:0x15:Completed  26000000 out of 50000000 steps (52%).
01:34:19:WU02:FS00:0x15:Completed  26500000 out of 50000000 steps (53%).
01:36:50:WU02:FS00:0x15:Completed  27000000 out of 50000000 steps (54%).
01:39:22:WU02:FS00:0x15:Completed  27500000 out of 50000000 steps (55%).
01:41:53:WU02:FS00:0x15:Completed  28000000 out of 50000000 steps (56%).
01:44:25:WU02:FS00:0x15:Completed  28500000 out of 50000000 steps (57%).
01:46:56:WU02:FS00:0x15:Completed  29000000 out of 50000000 steps (58%).
01:49:28:WU02:FS00:0x15:Completed  29500000 out of 50000000 steps (59%).
01:52:00:WU02:FS00:0x15:Completed  30000000 out of 50000000 steps (60%).
01:54:31:WU02:FS00:0x15:Completed  30500000 out of 50000000 steps (61%).
01:57:03:WU02:FS00:0x15:Completed  31000000 out of 50000000 steps (62%).
01:59:34:WU02:FS00:0x15:Completed  31500000 out of 50000000 steps (63%).
02:02:06:WU02:FS00:0x15:Completed  32000000 out of 50000000 steps (64%).
02:04:37:WU02:FS00:0x15:Completed  32500000 out of 50000000 steps (65%).
02:07:09:WU02:FS00:0x15:Completed  33000000 out of 50000000 steps (66%).
02:09:41:WU02:FS00:0x15:Completed  33500000 out of 50000000 steps (67%).
02:12:12:WU02:FS00:0x15:Completed  34000000 out of 50000000 steps (68%).
02:14:44:WU02:FS00:0x15:Completed  34500000 out of 50000000 steps (69%).
02:17:15:WU02:FS00:0x15:Completed  35000000 out of 50000000 steps (70%).
02:19:47:WU02:FS00:0x15:Completed  35500000 out of 50000000 steps (71%).
02:22:19:WU02:FS00:0x15:Completed  36000000 out of 50000000 steps (72%).
02:24:51:WU02:FS00:0x15:Completed  36500000 out of 50000000 steps (73%).
02:27:22:WU02:FS00:0x15:Completed  37000000 out of 50000000 steps (74%).
02:29:54:WU02:FS00:0x15:Completed  37500000 out of 50000000 steps (75%).
02:32:26:WU02:FS00:0x15:Completed  38000000 out of 50000000 steps (76%).
02:33:06:WU01:FS01:0xa4:Completed 510000 out of 1500000 steps  (34%)
02:34:58:WU02:FS00:0x15:Completed  38500000 out of 50000000 steps (77%).
02:37:29:WU02:FS00:0x15:Completed  39000000 out of 50000000 steps (78%).
02:40:01:WU02:FS00:0x15:Completed  39500000 out of 50000000 steps (79%).
02:42:32:WU02:FS00:0x15:Completed  40000000 out of 50000000 steps (80%).
02:45:04:WU02:FS00:0x15:Completed  40500000 out of 50000000 steps (81%).
02:47:36:WU02:FS00:0x15:Completed  41000000 out of 50000000 steps (82%).
02:50:07:WU02:FS00:0x15:Completed  41500000 out of 50000000 steps (83%).
02:52:39:WU02:FS00:0x15:Completed  42000000 out of 50000000 steps (84%).
02:55:10:WU02:FS00:0x15:Completed  42500000 out of 50000000 steps (85%).
02:57:43:WU02:FS00:0x15:Completed  43000000 out of 50000000 steps (86%).
03:00:14:WU02:FS00:0x15:Completed  43500000 out of 50000000 steps (87%).
03:02:47:WU02:FS00:0x15:Completed  44000000 out of 50000000 steps (88%).
03:05:18:WU02:FS00:0x15:Completed  44500000 out of 50000000 steps (89%).
03:07:50:WU02:FS00:0x15:Completed  45000000 out of 50000000 steps (90%).
03:10:21:WU02:FS00:0x15:Completed  45500000 out of 50000000 steps (91%).
03:12:53:WU02:FS00:0x15:Completed  46000000 out of 50000000 steps (92%).
03:15:24:WU02:FS00:0x15:Completed  46500000 out of 50000000 steps (93%).
03:17:56:WU02:FS00:0x15:Completed  47000000 out of 50000000 steps (94%).
03:20:28:WU02:FS00:0x15:Completed  47500000 out of 50000000 steps (95%).
03:22:59:WU02:FS00:0x15:Completed  48000000 out of 50000000 steps (96%).
03:25:31:WU02:FS00:0x15:Completed  48500000 out of 50000000 steps (97%).
03:28:02:WU02:FS00:0x15:Completed  49000000 out of 50000000 steps (98%).
03:28:03:WU00:FS00:Connecting to assign-GPU.stanford.edu:80
03:28:03:WU00:FS00:News: Welcome to Folding@Home
03:28:03:WU00:FS00:Assigned to work server 171.67.108.36
03:28:03:WU00:FS00:Requesting new work unit for slot 00: RUNNING gpu:0:GF110 [GeForce GTX 570 HD] from 171.67.108.36
03:28:03:WU00:FS00:Connecting to 171.67.108.36:8080
03:28:04:WU00:FS00:Downloading 57.93KiB
03:28:04:WU00:FS00:Download complete
03:28:04:WU00:FS00:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:8071 run:118 clone:11 gen:4 core:0x15 unit:0x000000046652edb4512f2d6fa261427a
03:30:36:WU02:FS00:0x15:Completed  49500000 out of 50000000 steps (99%).
03:33:08:WU02:FS00:0x15:Completed  50000000 out of 50000000 steps (100%).
03:33:08:WU02:FS00:0x15:Finished fah_main status=0
03:33:08:WU02:FS00:0x15:Successful run
03:33:08:WU02:FS00:0x15:DynamicWrapper: Finished Work Unit: sleep=10000
03:33:18:WU02:FS00:0x15:Reserved 324800 bytes for xtc file; Cosm status=0
03:33:18:WU02:FS00:0x15:Allocated 324800 bytes for xtc file
03:33:18:WU02:FS00:0x15:- Reading up to 324800 from "02/wudata_01.xtc": Read 324800
03:33:18:WU02:FS00:0x15:Read 324800 bytes from xtc file; available packet space=786105664
03:33:18:WU02:FS00:0x15:xtc file hash check passed.
03:33:18:WU02:FS00:0x15:Reserved 20256 20256 786105664 bytes for arc file=<02/wudata_01.trr> Cosm status=0
03:33:18:WU02:FS00:0x15:Allocated 20256 bytes for arc file
03:33:18:WU02:FS00:0x15:- Reading up to 20256 from "02/wudata_01.trr": Read 20256
03:33:18:WU02:FS00:0x15:Read 20256 bytes from arc file; available packet space=786085408
03:33:18:WU02:FS00:0x15:trr file hash check passed.
03:33:18:WU02:FS00:0x15:Allocated 544 bytes for edr file
03:33:18:WU02:FS00:0x15:Read bedfile
03:33:18:WU02:FS00:0x15:edr file hash check passed.
03:33:18:WU02:FS00:0x15:Allocated 36804 bytes for logfile
03:33:18:WU02:FS00:0x15:Read logfile
03:33:18:WU02:FS00:0x15:GuardedRun: success in DynamicWrapper
03:33:18:WU02:FS00:0x15:GuardedRun: done
03:33:18:WU02:FS00:0x15:Run: GuardedRun completed.
03:33:22:WU02:FS00:0x15:+ Opened results file
03:33:22:WU02:FS00:0x15:- Writing 382916 bytes of core data to disk...
03:33:22:WU02:FS00:0x15:Done: 382404 -> 351452 (compressed to 91.9 percent)
03:33:22:WU02:FS00:0x15:  ... Done.
03:33:22:WU02:FS00:0x15:DeleteFrameFiles: successfully deleted file=02/wudata_01.ckp
03:33:22:WU02:FS00:0x15:Shutting down core 
03:33:22:WU02:FS00:0x15:
03:33:22:WU02:FS00:0x15:Folding@home Core Shutdown: FINISHED_UNIT
03:33:22:WU02:FS00:FahCore returned: FINISHED_UNIT (100 = 0x64)
03:33:22:WU02:FS00:Sending unit results: id:02 state:SEND error:NO_ERROR project:8071 run:241 clone:4 gen:10 core:0x15 unit:0x0000000c6652edb45122e225b8e119e7
03:33:22:WU02:FS00:Uploading 343.71KiB to 171.67.108.36
03:33:22:WU02:FS00:Connecting to 171.67.108.36:8080
03:33:23:WU00:FS00:Starting
03:33:23:WU00:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/ProgramData/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 00 -suffix 01 -version 703 -lifeline 8388 -checkpoint 15 -gpu 0 -gpu-vendor nvidia
03:33:23:WU00:FS00:Started FahCore on PID 10224
03:33:23:WU00:FS00:Core PID:9208
03:33:23:WU00:FS00:FahCore 0x15 started
03:33:23:WU00:FS00:0x15:
03:33:23:WU00:FS00:0x15:*------------------------------*
03:33:23:WU00:FS00:0x15:Folding@Home GPU Core
03:33:23:WU00:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
03:33:23:WU00:FS00:0x15:Build host             AmoebaRemote
03:33:23:WU00:FS00:0x15:Board Type             NVIDIA/CUDA
03:33:23:WU00:FS00:0x15:Core                   15
03:33:23:WU00:FS00:0x15:
03:33:23:WU00:FS00:0x15:Window's signal control handler registered.
03:33:23:WU00:FS00:0x15:Preparing to commence simulation
03:33:23:WU00:FS00:0x15:- Looking at optimizations...
03:33:23:WU00:FS00:0x15:DeleteFrameFiles: successfully deleted file=00/wudata_01.ckp
03:33:23:WU00:FS00:0x15:- Created dyn
03:33:23:WU00:FS00:0x15:- Files status OK
03:33:23:WU00:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
03:33:23:WU00:FS00:0x15:- Expanded 58812 -> 259314 (decompressed 440.9 percent)
03:33:23:WU00:FS00:0x15:Called DecompressByteArray: compressed_data_size=58812 data_size=259314, decompressed_data_size=259314 diff=0
03:33:23:WU00:FS00:0x15:- Digital signature verified
03:33:23:WU00:FS00:0x15:
03:33:23:WU00:FS00:0x15:Project: 8071 (Run 118, Clone 11, Gen 4)
03:33:23:WU00:FS00:0x15:
03:33:23:WU00:FS00:0x15:Assembly optimizations on if available.
03:33:23:WU00:FS00:0x15:Entering M.D.
03:33:24:WU02:FS00:Upload complete
03:33:24:WU02:FS00:Server responded WORK_ACK (400)
03:33:24:WU02:FS00:Final credit estimate, 3874.00 points
03:33:24:WU02:FS00:Cleaning up
03:33:25:WU00:FS00:0x15:Tpr hash 00/wudata_01.tpr:  4137678891 2023613714 3735999060 4019548950 179042560
03:33:25:WU00:FS00:0x15:GPU device id=0
03:33:25:WU00:FS00:0x15:Working on Glycine aRginine prOline Methionine Alanine Cystine Serine t= 118.00000
03:33:25:WU00:FS00:0x15:Client config unavailable.
03:33:25:WU00:FS00:0x15:Starting GUI Server
03:34:25:WU00:FS00:0x15:Setting checkpoint frequency: 500000
03:34:25:WU00:FS00:0x15:Completed         3 out of 50000000 steps (0%).
03:36:57:WU00:FS00:0x15:Completed    500000 out of 50000000 steps (1%).
03:39:28:WU00:FS00:0x15:Completed   1000000 out of 50000000 steps (2%).
03:42:00:WU00:FS00:0x15:Completed   1500000 out of 50000000 steps (3%).
03:44:31:WU00:FS00:0x15:Completed   2000000 out of 50000000 steps (4%).
03:47:03:WU00:FS00:0x15:Completed   2500000 out of 50000000 steps (5%).
03:49:35:WU00:FS00:0x15:Completed   3000000 out of 50000000 steps (6%).
03:52:06:WU00:FS00:0x15:Completed   3500000 out of 50000000 steps (7%).
03:54:38:WU00:FS00:0x15:Completed   4000000 out of 50000000 steps (8%).
03:57:09:WU00:FS00:0x15:Completed   4500000 out of 50000000 steps (9%).
03:59:41:WU00:FS00:0x15:Completed   5000000 out of 50000000 steps (10%).
04:02:12:WU00:FS00:0x15:Completed   5500000 out of 50000000 steps (11%).
04:04:44:WU00:FS00:0x15:Completed   6000000 out of 50000000 steps (12%).
04:07:16:WU00:FS00:0x15:Completed   6500000 out of 50000000 steps (13%).
04:09:47:WU00:FS00:0x15:Completed   7000000 out of 50000000 steps (14%).
04:12:19:WU00:FS00:0x15:Completed   7500000 out of 50000000 steps (15%).
04:14:50:WU00:FS00:0x15:Completed   8000000 out of 50000000 steps (16%).
04:17:22:WU00:FS00:0x15:Completed   8500000 out of 50000000 steps (17%).
04:19:33:FS00:Shutting core down
04:19:33:FS01:Shutting core down
04:19:40:WU01:FS01:0xa4:Client no longer detected. Shutting down core 
04:19:40:WU01:FS01:0xa4:
04:19:40:WU01:FS01:0xa4:Folding@home Core Shutdown: CLIENT_DIED
04:19:40:WU01:FS01:FahCore returned: INTERRUPTED (102 = 0x66)
04:19:40:WU01:FS01:Starting
04:19:40:WARNING:WU01:FS01:Changed SMP threads from 8 to 4 this can cause some work units to fail
04:19:40:WU01:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/ProgramData/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/Core_a4.fah/FahCore_a4.exe -dir 01 -suffix 01 -version 703 -lifeline 8388 -checkpoint 15 -np 4
04:19:40:WU01:FS01:Started FahCore on PID 7784
04:19:40:WU01:FS01:Core PID:10260
04:19:40:WU01:FS01:FahCore 0xa4 started
04:19:41:WU01:FS01:0xa4:
04:19:41:WU01:FS01:0xa4:*------------------------------*
04:19:41:WU01:FS01:0xa4:Folding@Home Gromacs GB Core
04:19:41:WU01:FS01:0xa4:Version 2.27 (Dec. 15, 2010)
04:19:41:WU01:FS01:0xa4:
04:19:41:WU01:FS01:0xa4:Preparing to commence simulation
04:19:41:WU01:FS01:0xa4:- Looking at optimizations...
04:19:41:WU01:FS01:0xa4:- Files status OK
04:19:41:WU01:FS01:0xa4:- Expanded 2054096 -> 5365960 (decompressed 261.2 percent)
04:19:41:WU01:FS01:0xa4:Called DecompressByteArray: compressed_data_size=2054096 data_size=5365960, decompressed_data_size=5365960 diff=0
04:19:41:WU01:FS01:0xa4:- Digital signature verified
04:19:41:WU01:FS01:0xa4:
04:19:41:WU01:FS01:0xa4:Project: 7808 (Run 2, Clone 180, Gen 57)
04:19:41:WU01:FS01:0xa4:
04:19:41:WU01:FS01:0xa4:Assembly optimizations on if available.
04:19:41:WU01:FS01:0xa4:Entering M.D.
04:19:43:WU00:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
04:19:47:WU01:FS01:0xa4:Using Gromacs checkpoints
04:19:47:WU01:FS01:0xa4:Mapping NT from 4 to 4 
04:19:47:WU01:FS01:0xa4:Resuming from checkpoint
04:19:47:WU01:FS01:0xa4:Verified 01/wudata_01.log
04:19:47:WU01:FS01:0xa4:Verified 01/wudata_01.trr
04:19:47:WU01:FS01:0xa4:Verified 01/wudata_01.xtc
04:19:47:WU01:FS01:0xa4:Verified 01/wudata_01.edr
04:19:47:WU01:FS01:0xa4:Completed 514750 out of 1500000 steps  (34%)
04:23:41:WARNING:Exception: 138:127.0.0.1: Send error: 10053: An established connection was aborted by the software in your host machine.
04:23:53:FS01:Shutting core down
04:23:53:WU00:FS00:Starting
04:23:53:WU00:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/ProgramData/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 00 -suffix 01 -version 703 -lifeline 8388 -checkpoint 15 -gpu 0 -gpu-vendor nvidia
04:23:53:WU00:FS00:Started FahCore on PID 9384
04:23:53:WU00:FS00:Core PID:2400
04:23:53:WU00:FS00:FahCore 0x15 started
04:23:54:WU00:FS00:0x15:
04:23:54:WU00:FS00:0x15:*------------------------------*
04:23:54:WU00:FS00:0x15:Folding@Home GPU Core
04:23:54:WU00:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
04:23:54:WU00:FS00:0x15:Build host             AmoebaRemote
04:23:54:WU00:FS00:0x15:Board Type             NVIDIA/CUDA
04:23:54:WU00:FS00:0x15:Core                   15
04:23:54:WU00:FS00:0x15:
04:23:54:WU00:FS00:0x15:Window's signal control handler registered.
04:23:54:WU00:FS00:0x15:Preparing to commence simulation
04:23:54:WU00:FS00:0x15:- Looking at optimizations...
04:23:54:WU00:FS00:0x15:- Files status OK
04:23:54:WU00:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
04:23:54:WU00:FS00:0x15:- Expanded 58812 -> 259314 (decompressed 440.9 percent)
04:23:54:WU00:FS00:0x15:Called DecompressByteArray: compressed_data_size=58812 data_size=259314, decompressed_data_size=259314 diff=0
04:23:54:WU00:FS00:0x15:- Digital signature verified
04:23:54:WU00:FS00:0x15:
04:23:54:WU00:FS00:0x15:Project: 8071 (Run 118, Clone 11, Gen 4)
04:23:54:WU00:FS00:0x15:
04:23:54:WU00:FS00:0x15:Assembly optimizations on if available.
04:23:54:WU00:FS00:0x15:Entering M.D.
04:23:55:WU00:FS00:0x15:Will resume from checkpoint file 00/wudata_01.ckp
04:23:55:WU00:FS00:0x15:Tpr hash 00/wudata_01.tpr:  4137678891 2023613714 3735999060 4019548950 179042560
04:23:55:WU00:FS00:0x15:GPU device id=0
04:23:55:WU00:FS00:0x15:Working on Glycine aRginine prOline Methionine Alanine Cystine Serine t= 118.00000
04:23:55:WU00:FS00:0x15:Client config unavailable.
04:23:56:WU00:FS00:0x15:Starting GUI Server
04:24:01:WU01:FS01:FahCore returned: INTERRUPTED (102 = 0x66)
04:24:01:WU01:FS01:Starting
04:24:01:WARNING:WU01:FS01:Changed SMP threads from 4 to 8 this can cause some work units to fail
04:24:01:WU01:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/ProgramData/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/Core_a4.fah/FahCore_a4.exe -dir 01 -suffix 01 -version 703 -lifeline 8388 -checkpoint 15 -np 8
04:24:01:WU01:FS01:Started FahCore on PID 4452
04:24:01:WU01:FS01:Core PID:4080
04:24:01:WU01:FS01:FahCore 0xa4 started
04:24:02:WU01:FS01:0xa4:
04:24:02:WU01:FS01:0xa4:*------------------------------*
04:24:02:WU01:FS01:0xa4:Folding@Home Gromacs GB Core
04:24:02:WU01:FS01:0xa4:Version 2.27 (Dec. 15, 2010)
04:24:02:WU01:FS01:0xa4:
04:24:02:WU01:FS01:0xa4:Preparing to commence simulation
04:24:02:WU01:FS01:0xa4:- Looking at optimizations...
04:24:02:WU01:FS01:0xa4:- Files status OK
04:24:02:WU01:FS01:0xa4:- Expanded 2054096 -> 5365960 (decompressed 261.2 percent)
04:24:02:WU01:FS01:0xa4:Called DecompressByteArray: compressed_data_size=2054096 data_size=5365960, decompressed_data_size=5365960 diff=0
04:24:02:WU01:FS01:0xa4:- Digital signature verified
04:24:02:WU01:FS01:0xa4:
04:24:02:WU01:FS01:0xa4:Project: 7808 (Run 2, Clone 180, Gen 57)
04:24:02:WU01:FS01:0xa4:
04:24:02:WU01:FS01:0xa4:Assembly optimizations on if available.
04:24:02:WU01:FS01:0xa4:Entering M.D.
04:24:03:141:127.0.0.1:New Web connection
04:24:08:WU01:FS01:0xa4:Using Gromacs checkpoints
04:24:08:WU01:FS01:0xa4:Mapping NT from 8 to 8 
04:24:08:WU01:FS01:0xa4:Resuming from checkpoint
04:24:08:WU01:FS01:0xa4:Verified 01/wudata_01.log
04:24:08:WU01:FS01:0xa4:Verified 01/wudata_01.trr
04:24:08:WU01:FS01:0xa4:Verified 01/wudata_01.xtc
04:24:08:WU01:FS01:0xa4:Verified 01/wudata_01.edr
04:24:08:WU01:FS01:0xa4:Completed 514750 out of 1500000 steps  (34%)
04:24:57:WARNING:WU00:FS00:Detected clock skew (1 mins 04 secs), adjusting time estimates
04:24:57:WU00:FS00:0x15:Resuming from checkpoint
04:24:57:WU00:FS00:0x15:fcCheckPointResume: retreived and current tpr file hash:
04:24:57:WU00:FS00:0x15:   0   4137678891   4137678891
04:24:57:WU00:FS00:0x15:   1   2023613714   2023613714
04:24:57:WU00:FS00:0x15:   2   3735999060   3735999060
04:24:57:WU00:FS00:0x15:   3   4019548950   4019548950
04:24:57:WU00:FS00:0x15:   4    179042560    179042560
04:24:57:WU00:FS00:0x15:fcCheckPointResume: file hashes same.
04:24:57:WU00:FS00:0x15:fcCheckPointResume: state restored.
04:24:57:WU00:FS00:0x15:fcCheckPointResume: name 00/wudata_01.log Verified 00/wudata_01.log
04:24:57:WU00:FS00:0x15:fcCheckPointResume: name 00/wudata_01.trr Verified 00/wudata_01.trr
04:24:57:WU00:FS00:0x15:fcCheckPointResume: name 00/wudata_01.xtc Verified 00/wudata_01.xtc
04:24:57:WU00:FS00:0x15:fcCheckPointResume: name 00/wudata_01.edr Verified 00/wudata_01.edr
04:24:57:WU00:FS00:0x15:fcCheckPointResume: state restored 2
04:24:57:WU00:FS00:0x15:Resumed from checkpoint
04:24:57:WU00:FS00:0x15:Setting checkpoint frequency: 500000
04:24:57:WU00:FS00:0x15:Completed   8500001 out of 50000000 steps (17%).
04:25:06:FS01:Paused
04:25:06:FS01:Shutting core down
04:25:09:FS01:Unpaused
04:25:12:WU01:FS01:FahCore returned: INTERRUPTED (102 = 0x66)
04:25:12:WU01:FS01:Starting
04:25:12:WU01:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/ProgramData/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/Core_a4.fah/FahCore_a4.exe -dir 01 -suffix 01 -version 703 -lifeline 8388 -checkpoint 15 -np 8
04:25:12:WU01:FS01:Started FahCore on PID 10176
04:25:12:WU01:FS01:Core PID:7836
04:25:12:WU01:FS01:FahCore 0xa4 started
04:25:12:WU01:FS01:0xa4:
04:25:12:WU01:FS01:0xa4:*------------------------------*
04:25:12:WU01:FS01:0xa4:Folding@Home Gromacs GB Core
04:25:12:WU01:FS01:0xa4:Version 2.27 (Dec. 15, 2010)
04:25:12:WU01:FS01:0xa4:
04:25:12:WU01:FS01:0xa4:Preparing to commence simulation
04:25:12:WU01:FS01:0xa4:- Looking at optimizations...
04:25:12:WU01:FS01:0xa4:- Files status OK
04:25:12:WU01:FS01:0xa4:- Expanded 2054096 -> 5365960 (decompressed 261.2 percent)
04:25:12:WU01:FS01:0xa4:Called DecompressByteArray: compressed_data_size=2054096 data_size=5365960, decompressed_data_size=5365960 diff=0
04:25:12:WU01:FS01:0xa4:- Digital signature verified
04:25:12:WU01:FS01:0xa4:
04:25:12:WU01:FS01:0xa4:Project: 7808 (Run 2, Clone 180, Gen 57)
04:25:12:WU01:FS01:0xa4:
04:25:12:WU01:FS01:0xa4:Assembly optimizations on if available.
04:25:12:WU01:FS01:0xa4:Entering M.D.
04:25:18:WU01:FS01:0xa4:Using Gromacs checkpoints
04:25:18:WU01:FS01:0xa4:Mapping NT from 8 to 8 
04:25:19:WU01:FS01:0xa4:Resuming from checkpoint
04:25:19:WU01:FS01:0xa4:Verified 01/wudata_01.log
04:25:19:WU01:FS01:0xa4:Verified 01/wudata_01.trr
04:25:19:WU01:FS01:0xa4:Verified 01/wudata_01.xtc
04:25:19:WU01:FS01:0xa4:Verified 01/wudata_01.edr
04:25:19:WU01:FS01:0xa4:Completed 514750 out of 1500000 steps  (34%)
04:27:30:WU00:FS00:0x15:Completed   9000000 out of 50000000 steps (18%).
04:30:03:WU00:FS00:0x15:Completed   9500000 out of 50000000 steps (19%).
04:32:36:WU00:FS00:0x15:Completed  10000000 out of 50000000 steps (20%).
04:35:09:WU00:FS00:0x15:Completed  10500000 out of 50000000 steps (21%).
04:37:42:WU00:FS00:0x15:Completed  11000000 out of 50000000 steps (22%).
04:40:16:WU00:FS00:0x15:Completed  11500000 out of 50000000 steps (23%).
04:42:48:WU00:FS00:0x15:Completed  12000000 out of 50000000 steps (24%).
04:43:58:WARNING:WU01:FS01:FahCore crashed with Windows unhandled exception code 0xUNKNOWN_ENUM, searching for this code online may provide more information
04:43:58:WARNING:WU01:FS01:FahCore returned: UNKNOWN_ENUM (1073807364 = 0x40010004)
04:43:58:WARNING:WU00:FS00:FahCore crashed with Windows unhandled exception code 0xUNKNOWN_ENUM, searching for this code online may provide more information
04:43:58:WARNING:WU00:FS00:FahCore returned: UNKNOWN_ENUM (1073807364 = 0x40010004)
04:43:59:WU01:FS01:Starting
04:43:59:WU01:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/ProgramData/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/Core_a4.fah/FahCore_a4.exe -dir 01 -suffix 01 -version 703 -lifeline 8388 -checkpoint 15 -np 8
04:43:59:WU01:FS01:Started FahCore on PID 8600
Top
bruce
Posts: 20824
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: 7808 (2,180,57) - ETEA 11.30 days!

Post by bruce »

There is an extremely wide varation in frame time.
{00:40:32:WU01:FS01:0xa4:Completed 427320 out of 1500000 steps (28%)}
> 01:22:35:WU01:FS01:0xa4:Completed 435000 out of 1500000 steps (29%)
> 07:31:49:WU01:FS01:0xa4:Completed 450000 out of 1500000 steps (30%)
> 10:37:33:WU01:FS01:0xa4:Completed 465000 out of 1500000 steps (31%)
> 16:44:20:WU01:FS01:0xa4:Completed 480000 out of 1500000 steps (32%)
> 20:52:58:WU01:FS01:0xa4:Completed 495000 out of 1500000 steps (33%)
> 02:33:06:WU01:FS01:0xa4:Completed 510000 out of 1500000 steps (34%)

{04:19:47:WU01:FS01:0xa4:Completed 514750 out of 1500000 steps (34%)}

{04:25:19:WU01:FS01:0xa4:Completed 514750 out of 1500000 steps (34%)}


Evaluating the duration of each of the full one percent intervals gives:

> 01:22:35
---------------6h9m
> 07:31:49
---------------3h6m
> 10:37:33
---------------6h7m
> 16:44:20
---------------4h9m
> 20:52:58
---------------5h41m
> 02:33:06
There's no possible way to establish an average. There's just too much variation. I strongly suspect that the FAHCore was competing with some other heavy application.
Top
P5-133XL
Posts: 2948
Joined: Sun Dec 02, 2007 4:36 am
Hardware configuration: Machine #1:

Intel Q9450; 2x2GB=8GB Ram; Gigabyte GA-X48-DS4 Motherboard; PC Power and Cooling Q750 PS; 2x GTX 460; Windows Server 2008 X64 (SP1).

Machine #2:

Intel Q6600; 2x2GB=4GB Ram; Gigabyte GA-X48-DS4 Motherboard; PC Power and Cooling Q750 PS; 2x GTX 460 video card; Windows 7 X64.

Machine 3:

Dell Dimension 8400, 3.2GHz P4 4x512GB Ram, Video card GTX 460, Windows 7 X32

I am currently folding just on the 5x GTX 460's for aprox. 70K PPD
Location: Salem. OR USA

Re: 7808 (2,180,57) - ETEA 11.30 days!

Post by P5-133XL »

When there are wide variations in frame times spread throughout a log, my immediate assumption is that there is an external process stealing CPU time from SMP-folding causing at least one SMP thread to pause for an extended time. The solution is to decrease the number of CPU cores that the SMP slot is using so it won't be paused by that external process.

Even a small amount of CPU being used will cause large delays to SMP folding. The folding cores are highly synchronized and run at a very low priority so that most anything takes priority causing a SMP thread to be suspended. Since all the threads are synchronized, the other folding threads go into a waiting loop for the one thread to become active again. As such even a small amount of non-folding CPU usage can have a very large effect of folding TPF's.

If you decrease the number of folding threads/cores used to allow for non-folding CPU use then the folding thread never gets suspended and TPF's will drastically improve.
Image
Top
telecentricity
Posts: 7
Joined: Fri Mar 01, 2013 4:40 am

Re: 7808 (2,180,57) - ETEA 11.30 days!

Post by telecentricity »

Since I changed from CPU:8 to CPU:6 it has been rock steady. I think the advice of Napoleon, Bruce and others was spot on. Thanks very much for the help.
Kind regards,
Gary
Top
bruce
Posts: 20824
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: 7808 (2,180,57) - ETEA 11.30 days!

Post by bruce »

bruce wrote:By changing the setting to 6, you've left two virtual cores available for other things that might be running on your system. Check performance monitor. You should see the FahCore using a consistent total of 6/8 = 75% of the CPU time. You should also see essentially 0% for other applications. If you're seeing a few percent consistently going to anything else that's running, that's probably the complete explanation of the problem. SMP can never perform well unless all of its threads are balanced (aka "symmetric").
Assuming FAH is using 75% of your logical cores, how much more that 75% total is being used and by what?
Top
Post Reply

Return to “Issues with a specific WU”