work units not sending. Also, idle not working properly

If you're new to FAH and need help getting started or you have very basic questions, start here.

Moderators: Site Moderators, FAHC Science Team

Post Reply
PantherX
Site Moderator
Posts: 6986
Joined: Wed Dec 23, 2009 9:33 am
Hardware configuration: V7.6.21 -> Multi-purpose 24/7
Windows 10 64-bit
CPU:2/3/4/6 -> Intel i7-6700K
GPU:1 -> Nvidia GTX 1080 Ti
§
Retired:
2x Nvidia GTX 1070
Nvidia GTX 675M
Nvidia GTX 660 Ti
Nvidia GTX 650 SC
Nvidia GTX 260 896 MB SOC
Nvidia 9600GT 1 GB OC
Nvidia 9500M GS
Nvidia 8800GTS 320 MB

Intel Core i7-860
Intel Core i7-3840QM
Intel i3-3240
Intel Core 2 Duo E8200
Intel Core 2 Duo E6550
Intel Core 2 Duo T8300
Intel Pentium E5500
Intel Pentium E5400
Location: Land Of The Long White Cloud
Contact:
Contact PantherX

Re: work units not sending. Also, idle not working properly

Post by PantherX »

bruce wrote:...Suppose your connection is so unreliable that 10 WUs accumulate. If you restart your client, I suspect that all 10 of them will attempt to share the connection by trying to upload concurrently. (I never have seen more that 2 WUs so I'm not sure what actually happens with 10. Perhaps only the first two or three share the bandwidth.) That's a good design if you have a connection that is reliable and fast. If you have a connection that's unreliable and slow, it's best to attempt to upload the WUs serially even though the total time spend uploading will increase. There may already be a ticket to enhance the client to upload serially.
Uploading 6 WUs @ 380 kbit/s was a nightmare for me. The enhancement to upload WUs serially is this one (https://fah.stanford.edu/projects/FAHClient/ticket/1038).
ETA:
Now ↞ Very Soon ↔ Soon ↔ Soon-ish ↔ Not Soon ↠ End Of Time

Welcome To The F@H Support Forum Ӂ Troubleshooting Bad WUs Ӂ Troubleshooting Server Connectivity Issues
Top
Volnaiskra
Posts: 57
Joined: Tue Nov 26, 2013 9:02 am

Re: work units not sending. Also, idle not working properly

Post by Volnaiskra »

7im wrote:Please post a list of what projects work, and which ones stack up. A list of the specific work units (PRCG numbers) also, if possible.
Not sure which bits to post, so I've just pasted a whole log. Hope you don't mind. So, this is the log from the past day or so. I've had many successful ACKs, and a few missed ACKs. The CPU ones all seem to work, but some of the GPU ones do not. Though the biggest difference I can see between those that work and those that don't are the amount of "upload" messages.

The ones that work look like this (3 examples):


06:46:11:WU01:FS01:Upload 47.73%
06:46:18:WU01:FS01:Upload complete
06:46:18:WU01:FS01:Server responded WORK_ACK (400)
06:46:18:WU01:FS01:Final credit estimate, 14093.00 points


23:57:21:WU00:FS00:Upload 76.70%
23:57:27:WU00:FS00:Upload complete
23:57:27:WU00:FS00:Server responded WORK_ACK (400)
23:57:27:WU00:FS00:Final credit estimate, 685.00 points


07:01:31:WU00:FS02:Upload 56.10%
07:01:38:WU00:FS02:Upload complete
07:01:38:WU00:FS02:Server responded WORK_ACK (400)
07:01:38:WU00:FS02:Final credit estimate, 14093.00 points

While the ones that don't work look like this (2 examples):

00:15:19:WU04:FS02:Upload 2.46%
00:15:25:WU04:FS02:Upload 4.61%
00:15:31:WU04:FS02:Upload 6.46%
00:15:37:WU04:FS02:Upload 8.61%
00:15:43:WU04:FS02:Upload 10.46%
00:15:49:WU04:FS02:Upload 12.61%
00:15:55:WU04:FS02:Upload 14.46%
00:16:01:WU04:FS02:Upload 16.61%
00:16:07:WU04:FS02:Upload 18.45%
00:16:13:WU04:FS02:Upload 20.30%
00:16:19:WU04:FS02:Upload 22.45%
00:16:25:WU04:FS02:Upload 24.30%
00:16:31:WU04:FS02:Upload 26.45%
00:16:37:WU04:FS02:Upload 28.30%
00:16:43:WU04:FS02:Upload 30.45%
00:16:49:WU04:FS02:Upload 32.30%
00:16:55:WU04:FS02:Upload 34.45%
00:17:01:WU04:FS02:Upload 36.29%
00:17:07:WU04:FS02:Upload 38.45%
00:17:13:WU04:FS02:Upload 40.29%
00:17:19:WU04:FS02:Upload 42.45%
00:17:25:WU04:FS02:Upload 44.29%
00:17:31:WU04:FS02:Upload 46.44%
00:17:37:WU04:FS02:Upload 48.29%
00:17:43:WU04:FS02:Upload 50.44%
00:17:49:WU04:FS02:Upload 52.29%
00:17:55:WU04:FS02:Upload 54.44%
00:18:01:WU04:FS02:Upload 56.29%
00:18:07:WU04:FS02:Upload 58.44%
00:18:13:WU04:FS02:Upload 60.29%
00:18:19:WU04:FS02:Upload 62.44%
00:18:25:WU04:FS02:Upload 64.28%
00:18:31:WU04:FS02:Upload 66.44%
00:18:37:WU04:FS02:Upload 68.28%
00:18:43:WU04:FS02:Upload 70.44%
00:18:49:WU04:FS02:Upload 72.28%
00:18:55:WU04:FS02:Upload 74.13%
00:19:01:WU04:FS02:Upload 76.28%
00:19:07:WU04:FS02:Upload 78.43%
00:19:13:WU04:FS02:Upload 80.28%
00:19:19:WU04:FS02:Upload 82.43%
00:19:25:WU04:FS02:Upload 84.28%
00:19:31:WU04:FS02:Upload 86.43%
00:19:37:WU04:FS02:Upload 88.28%
00:19:43:WU04:FS02:Upload 90.43%
00:19:49:WU04:FS02:Upload 92.27%
00:19:55:WU04:FS02:Upload 94.43%
00:20:01:WU04:FS02:Upload 96.27%
00:20:07:WU04:FS02:Upload 98.43%




23:30:06:WU02:FS02:Upload 2.46%
23:30:12:WU02:FS02:Upload 4.31%
23:30:21:WU02:FS02:Upload 6.15%
23:30:27:WU02:FS02:Upload 8.31%
23:30:33:WU02:FS02:Upload 10.15%
23:30:39:WU02:FS02:Upload 11.69%
23:30:45:WU02:FS02:Upload 13.54%
23:30:51:WU02:FS02:Upload 14.77%
23:30:57:WU02:FS02:Upload 16.00%
23:31:03:WU02:FS02:Upload 17.54%
23:31:09:WU02:FS02:Upload 18.77%
23:31:15:WU02:FS02:Upload 20.62%
23:31:21:WU02:FS02:Upload 22.16%
23:31:27:WU02:FS02:Upload 23.69%
23:31:33:WU02:FS02:Upload 25.54%
23:31:39:WU02:FS02:Upload 27.39%
23:31:45:WU02:FS02:Upload 29.23%
23:31:51:WU02:FS02:Upload 31.08%
23:31:57:WU02:FS02:Upload 33.23%
23:32:03:WU02:FS02:Upload 35.08%
23:32:09:WU02:FS02:Upload 36.93%
23:32:15:WU02:FS02:Upload 39.08%
23:32:21:WU02:FS02:Upload 40.93%
23:32:27:WU02:FS02:Upload 43.08%
23:32:33:WU02:FS02:Upload 44.93%
23:32:39:WU02:FS02:Upload 47.08%
23:32:45:WU02:FS02:Upload 48.93%
23:32:51:WU02:FS02:Upload 51.08%
23:32:57:WU02:FS02:Upload 52.93%
23:33:03:WU02:FS02:Upload 54.77%
23:33:09:WU02:FS02:Upload 56.93%
23:33:15:WU02:FS02:Upload 58.77%
23:33:21:WU02:FS02:Upload 60.93%
23:33:27:WU02:FS02:Upload 62.77%
23:33:33:WU02:FS02:Upload 64.93%
23:33:39:WU02:FS02:Upload 66.77%
23:33:45:WU02:FS02:Upload 68.93%
23:33:51:WU02:FS02:Upload 70.77%
23:33:57:WU02:FS02:Upload 72.93%
23:34:03:WU02:FS02:Upload 74.78%
23:34:09:WU02:FS02:Upload 76.93%
23:34:15:WU02:FS02:Upload 78.78%
23:34:21:WU02:FS02:Upload 80.93%
23:34:27:WU02:FS02:Upload 82.78%
23:34:33:WU02:FS02:Upload 84.62%
23:34:39:WU02:FS02:Upload 86.78%
23:34:45:WU02:FS02:Upload 88.31%
23:34:51:WU02:FS02:Upload 89.55%
23:34:57:WU02:FS02:Upload 90.78%
23:35:03:WU02:FS02:Upload 92.62%
23:35:09:WU02:FS02:Upload 94.78%
23:35:15:WU02:FS02:Upload 96.62%
23:35:21:WU02:FS02:Upload 98.78%

Does that suggest that the ones that are failing are much larger? Or they are from a different EXE process that uploads them slower? If they're larger, then I guess the size could be the problem (though 5 minutes to upload a WU hardly seems like it should cause problems), or perhaps larger ones have some other shared characteristic other than size - for example, maybe they're more complex and harder to verify on the other end, or they're all from a particular project?

Anyway, could someone please verify for me whether those two big WUs were actually received? The full log is split over the next few posts

Code: Select all

*********************** Log Started 2014-05-30T23:29:59Z ***********************
23:29:59:************************* Folding@home Client *************************
23:29:59:      Website: http://folding.stanford.edu/
23:29:59:    Copyright: (c) 2009-2014 Stanford University
23:29:59:       Author: Joseph Coffland <joseph@cauldrondevelopment.com>
23:29:59:         Args: 
23:29:59:       Config: C:/Users/Dave/AppData/Roaming/FAHClient/config.xml
23:29:59:******************************** Build ********************************
23:29:59:      Version: 7.4.4
23:29:59:         Date: Mar 4 2014
23:29:59:         Time: 20:26:54
23:29:59:      SVN Rev: 4130
23:29:59:       Branch: fah/trunk/client
23:29:59:     Compiler: Intel(R) C++ MSVC 1500 mode 1200
23:29:59:      Options: /TP /nologo /EHa /Qdiag-disable:4297,4103,1786,279 /Ox -arch:SSE
23:29:59:               /QaxSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 /Qopenmp /Qrestrict /MT /Qmkl
23:29:59:     Platform: win32 XP
23:29:59:         Bits: 32
23:29:59:         Mode: Release
23:29:59:******************************* System ********************************
23:29:59:          CPU: Intel(R) Core(TM) i7-4770K CPU @ 3.50GHz
23:29:59:       CPU ID: GenuineIntel Family 6 Model 60 Stepping 3
23:29:59:         CPUs: 8
23:29:59:       Memory: 15.93GiB
23:29:59:  Free Memory: 13.76GiB
23:29:59:      Threads: WINDOWS_THREADS
23:29:59:   OS Version: 6.1
23:29:59:  Has Battery: false
23:29:59:   On Battery: false
23:29:59:   UTC Offset: 10
23:29:59:          PID: 6808
23:29:59:          CWD: C:/Users/Dave/AppData/Roaming/FAHClient
23:29:59:           OS: Windows 7 Home Premium
23:29:59:      OS Arch: AMD64
23:29:59:         GPUs: 3
23:29:59:        GPU 0: NVIDIA:3 GK110 [GeForce GTX Titan]
23:29:59:        GPU 1: NVIDIA:3 GK110 [GeForce GTX Titan]
23:29:59:        GPU 2: NVIDIA:3 GK106 [GeForce GTX 650 Ti]
23:29:59:         CUDA: 3.5
23:29:59:  CUDA Driver: 6000
23:29:59:Win32 Service: false
23:29:59:***********************************************************************
23:29:59:<config>
23:29:59:  <!-- Network -->
23:29:59:  <proxy v=':8080'/>
23:29:59:
23:29:59:  <!-- Slot Control -->
23:29:59:  <power v='FULL'/>
23:29:59:
23:29:59:  <!-- User Information -->
23:29:59:  <passkey v='********************************'/>
23:29:59:  <team v='225663'/>
23:29:59:  <user v='Volnaiskra'/>
23:29:59:
23:29:59:  <!-- Work Unit Control -->
23:29:59:  <max-queue v='64'/>
23:29:59:
23:29:59:  <!-- Folding Slots -->
23:29:59:  <slot id='0' type='CPU'>
23:29:59:    <idle v='true'/>
23:29:59:  </slot>
23:29:59:  <slot id='1' type='GPU'>
23:29:59:    <idle v='true'/>
23:29:59:  </slot>
23:29:59:  <slot id='2' type='GPU'>
23:29:59:    <idle v='true'/>
23:29:59:  </slot>
23:29:59:  <slot id='3' type='GPU'>
23:29:59:    <idle v='true'/>
23:29:59:  </slot>
23:29:59:</config>
23:29:59:Trying to access database...
23:30:00:Successfully acquired database lock
23:30:00:Enabled folding slot 00: PAUSED cpu:5 (waiting for idle)
23:30:00:Enabled folding slot 01: PAUSED gpu:0:GK110 [GeForce GTX Titan] (waiting for idle)
23:30:00:Enabled folding slot 02: PAUSED gpu:1:GK110 [GeForce GTX Titan] (waiting for idle)
23:30:00:Enabled folding slot 03: PAUSED gpu:2:GK106 [GeForce GTX 650 Ti] (waiting for idle)
23:30:00:WU02:FS02:Sending unit results: id:02 state:SEND error:NO_ERROR project:9408 run:1627 clone:0 gen:13 core:0x17 unit:0x000000190a3b1e5c53432c6dbc80e132
23:30:00:WU02:FS02:Uploading 20.31MiB to 171.64.65.56
23:30:00:WU02:FS02:Connecting to 171.64.65.56:8080
23:30:06:WU02:FS02:Upload 2.46%
23:30:12:WU02:FS02:Upload 4.31%
23:30:21:WU02:FS02:Upload 6.15%
23:30:27:WU02:FS02:Upload 8.31%
23:30:33:WU02:FS02:Upload 10.15%
23:30:39:WU02:FS02:Upload 11.69%
23:30:45:WU02:FS02:Upload 13.54%
23:30:51:WU02:FS02:Upload 14.77%
23:30:57:WU02:FS02:Upload 16.00%
23:31:03:WU02:FS02:Upload 17.54%
23:31:09:WU02:FS02:Upload 18.77%
23:31:15:WU02:FS02:Upload 20.62%
23:31:21:WU02:FS02:Upload 22.16%
23:31:27:WU02:FS02:Upload 23.69%
23:31:33:WU02:FS02:Upload 25.54%
23:31:39:WU02:FS02:Upload 27.39%
23:31:45:WU02:FS02:Upload 29.23%
23:31:51:WU02:FS02:Upload 31.08%
23:31:57:WU02:FS02:Upload 33.23%
23:32:03:WU02:FS02:Upload 35.08%
23:32:09:WU02:FS02:Upload 36.93%
23:32:15:WU02:FS02:Upload 39.08%
23:32:21:WU02:FS02:Upload 40.93%
23:32:27:WU02:FS02:Upload 43.08%
23:32:33:WU02:FS02:Upload 44.93%
23:32:39:WU02:FS02:Upload 47.08%
23:32:45:WU02:FS02:Upload 48.93%
23:32:51:WU02:FS02:Upload 51.08%
23:32:57:WU02:FS02:Upload 52.93%
23:33:03:WU02:FS02:Upload 54.77%
23:33:09:WU02:FS02:Upload 56.93%
23:33:15:WU02:FS02:Upload 58.77%
23:33:21:WU02:FS02:Upload 60.93%
23:33:27:WU02:FS02:Upload 62.77%
23:33:33:WU02:FS02:Upload 64.93%
23:33:39:WU02:FS02:Upload 66.77%
23:33:45:WU02:FS02:Upload 68.93%
23:33:51:WU02:FS02:Upload 70.77%
23:33:57:WU02:FS02:Upload 72.93%
23:34:03:WU02:FS02:Upload 74.78%
23:34:09:WU02:FS02:Upload 76.93%
23:34:15:WU02:FS02:Upload 78.78%
23:34:21:WU02:FS02:Upload 80.93%
23:34:27:WU02:FS02:Upload 82.78%
23:34:33:WU02:FS02:Upload 84.62%
23:34:39:WU02:FS02:Upload 86.78%
23:34:45:WU02:FS02:Upload 88.31%
23:34:51:WU02:FS02:Upload 89.55%
23:34:57:WU02:FS02:Upload 90.78%
23:35:03:WU02:FS02:Upload 92.62%
23:35:09:WU02:FS02:Upload 94.78%
23:35:15:WU02:FS02:Upload 96.62%
23:35:21:WU02:FS02:Upload 98.78%
23:41:43:WU04:FS02:Starting
23:41:43:WU04:FS02:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Dave/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_17.fah/FahCore_17.exe -dir 04 -suffix 01 -version 704 -lifeline 6808 -checkpoint 15 -gpu 1 -gpu-vendor nvidia
23:41:43:WU04:FS02:Started FahCore on PID 8780
23:41:43:WU04:FS02:Core PID:6016
23:41:43:WU04:FS02:FahCore 0x17 started
23:41:43:WU00:FS00:Starting
23:41:43:WU00:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Dave/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/Core_a4.fah/FahCore_a4.exe -dir 00 -suffix 01 -version 704 -lifeline 6808 -checkpoint 15 -np 5
23:41:43:WU00:FS00:Started FahCore on PID 1724
23:41:43:WU00:FS00:Core PID:4600
23:41:43:WU00:FS00:FahCore 0xa4 started
23:41:44:WU04:FS02:0x17:*********************** Log Started 2014-05-30T23:41:43Z ***********************
23:41:44:WU04:FS02:0x17:Project: 9408 (Run 1623, Clone 0, Gen 24)
23:41:44:WU04:FS02:0x17:Unit: 0x000000250a3b1e5c53432c4f098b6279
23:41:44:WU04:FS02:0x17:CPU: 0x00000000000000000000000000000000
23:41:44:WU04:FS02:0x17:Machine: 2
23:41:44:WU04:FS02:0x17:Digital signatures verified
23:41:44:WU04:FS02:0x17:Folding@home GPU core17
23:41:44:WU04:FS02:0x17:Version 0.0.52
23:41:44:WU04:FS02:0x17:  Found a checkpoint file
23:41:44:WU00:FS00:0xa4:
23:41:44:WU00:FS00:0xa4:*------------------------------*
23:41:44:WU00:FS00:0xa4:Folding@Home Gromacs GB Core
23:41:44:WU00:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
23:41:44:WU00:FS00:0xa4:
23:41:44:WU00:FS00:0xa4:Preparing to commence simulation
23:41:44:WU00:FS00:0xa4:- Looking at optimizations...
23:41:44:WU00:FS00:0xa4:- Files status OK
23:41:44:WU00:FS00:0xa4:- Expanded 544641 -> 1307132 (decompressed 239.9 percent)
23:41:44:WU00:FS00:0xa4:Called DecompressByteArray: compressed_data_size=544641 data_size=1307132, decompressed_data_size=1307132 diff=0
23:41:44:WU00:FS00:0xa4:- Digital signature verified
23:41:44:WU00:FS00:0xa4:
23:41:44:WU00:FS00:0xa4:Project: 9500 (Run 1292, Clone 1, Gen 28)
23:41:44:WU00:FS00:0xa4:
23:41:44:WU00:FS00:0xa4:Assembly optimizations on if available.
23:41:44:WU00:FS00:0xa4:Entering M.D.
23:41:44:WU01:FS01:Connecting to 171.67.108.201:80
23:41:44:WU03:FS03:Connecting to 171.67.108.201:80
23:41:45:WU01:FS01:Assigned to work server 171.64.65.105
23:41:45:WU01:FS01:Requesting new work unit for slot 01: READY gpu:0:GK110 [GeForce GTX Titan] from 171.64.65.105
23:41:45:WU01:FS01:Connecting to 171.64.65.105:8080
23:41:45:WU03:FS03:Assigned to work server 171.64.65.105
23:41:45:WU03:FS03:Requesting new work unit for slot 03: READY gpu:2:GK106 [GeForce GTX 650 Ti] from 171.64.65.105
23:41:45:WU03:FS03:Connecting to 171.64.65.105:8080
23:41:46:WU01:FS01:Downloading 122.60KiB
23:41:46:WU03:FS03:Downloading 121.94KiB
23:41:47:WU03:FS03:Download complete
23:41:47:WU03:FS03:Received Unit: id:03 state:DOWNLOAD error:NO_ERROR project:7623 run:261 clone:0 gen:166 core:0x15 unit:0x000000dd664f2dd14fe4f9ed95b7e66f
23:41:47:WU03:FS03:Starting
23:41:47:WU03:FS03:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Dave/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 03 -suffix 01 -version 704 -lifeline 6808 -checkpoint 15 -gpu 2 -gpu-vendor nvidia
23:41:47:WU03:FS03:Started FahCore on PID 8036
23:41:47:WU01:FS01:Download complete
23:41:47:WU03:FS03:Core PID:5944
23:41:47:WU03:FS03:FahCore 0x15 started
23:41:47:WU01:FS01:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:7622 run:198 clone:0 gen:70 core:0x15 unit:0x0000006a664f2dd14edd57c7e7a3204b
23:41:47:WU01:FS01:Starting
23:41:47:WU01:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Dave/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 01 -suffix 01 -version 704 -lifeline 6808 -checkpoint 15 -gpu 0 -gpu-vendor nvidia
23:41:47:WU01:FS01:Started FahCore on PID 1372
23:41:47:WU01:FS01:Core PID:2980
23:41:47:WU01:FS01:FahCore 0x15 started
23:41:48:WU03:FS03:0x15:
23:41:48:WU03:FS03:0x15:*------------------------------*
23:41:48:WU03:FS03:0x15:Folding@Home GPU Core
23:41:48:WU03:FS03:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
23:41:48:WU03:FS03:0x15:Build host             AmoebaRemote
23:41:48:WU03:FS03:0x15:Board Type             NVIDIA/CUDA
23:41:48:WU03:FS03:0x15:Core                   15
23:41:48:WU03:FS03:0x15:GPU device info vendor=0 device=0 name=NA match=0 deviceId=2
23:41:48:WU03:FS03:0x15:
23:41:48:WU03:FS03:0x15:Window's signal control handler registered.
23:41:48:WU03:FS03:0x15:Preparing to commence simulation
23:41:48:WU03:FS03:0x15:- Looking at optimizations...
23:41:48:WU03:FS03:0x15:DeleteFrameFiles: successfully deleted file=03/wudata_01.ckp
23:41:48:WU03:FS03:0x15:- Created dyn
23:41:48:WU03:FS03:0x15:- Files status OK
23:41:48:WU03:FS03:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
23:41:48:WU03:FS03:0x15:- Expanded 124351 -> 501826 (decompressed 403.5 percent)
23:41:48:WU03:FS03:0x15:Called DecompressByteArray: compressed_data_size=124351 data_size=501826, decompressed_data_size=501826 diff=0
23:41:48:WU03:FS03:0x15:- Digital signature verified
23:41:48:WU03:FS03:0x15:
23:41:48:WU03:FS03:0x15:Project: 7623 (Run 261, Clone 0, Gen 166)
23:41:48:WU03:FS03:0x15:
23:41:48:WU03:FS03:0x15:Assembly optimizations on if available.
23:41:48:WU03:FS03:0x15:Entering M.D.
23:41:48:WU01:FS01:0x15:
23:41:48:WU01:FS01:0x15:*------------------------------*
23:41:48:WU01:FS01:0x15:Folding@Home GPU Core
23:41:48:WU01:FS01:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
23:41:48:WU01:FS01:0x15:Build host             AmoebaRemote
23:41:48:WU01:FS01:0x15:Board Type             NVIDIA/CUDA
23:41:48:WU01:FS01:0x15:Core                   15
23:41:48:WU01:FS01:0x15:
23:41:48:WU01:FS01:0x15:Window's signal control handler registered.
23:41:48:WU01:FS01:0x15:Preparing to commence simulation
23:41:48:WU01:FS01:0x15:- Looking at optimizations...
23:41:48:WU01:FS01:0x15:DeleteFrameFiles: successfully deleted file=01/wudata_01.ckp
23:41:48:WU01:FS01:0x15:- Created dyn
23:41:48:WU01:FS01:0x15:- Files status OK
23:41:48:WU01:FS01:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
23:41:48:WU01:FS01:0x15:- Expanded 125034 -> 501826 (decompressed 401.3 percent)
23:41:48:WU01:FS01:0x15:Called DecompressByteArray: compressed_data_size=125034 data_size=501826, decompressed_data_size=501826 diff=0
23:41:48:WU01:FS01:0x15:- Digital signature verified
23:41:48:WU01:FS01:0x15:
23:41:48:WU01:FS01:0x15:Project: 7622 (Run 198, Clone 0, Gen 70)
23:41:48:WU01:FS01:0x15:
23:41:48:WU01:FS01:0x15:Assembly optimizations on if available.
23:41:48:WU01:FS01:0x15:Entering M.D.
23:41:49:WU03:FS03:0x15:Tpr hash 03/wudata_01.tpr:  310465715 1833047723 3414748523 2877205478 1558103798
23:41:49:WU03:FS03:0x15:GPU device id=2
23:41:49:WU03:FS03:0x15:Working on Protein
23:41:49:WU03:FS03:0x15:Client config unavailable.
23:41:49:WU00:FS00:0xa4:Using Gromacs checkpoints
23:41:49:WU00:FS00:0xa4:Mapping NT from 5 to 5 
23:41:50:WU01:FS01:0x15:Tpr hash 01/wudata_01.tpr:  3336875573 1615538842 2324230187 4260796349 2057047944
23:41:50:WU01:FS01:0x15:GPU device id=0
23:41:50:WU01:FS01:0x15:Working on Protein
23:41:50:WU01:FS01:0x15:Client config unavailable.
23:41:50:WU03:FS03:0x15:Starting GUI Server
23:41:50:WU00:FS00:0xa4:Resuming from checkpoint
23:41:50:WU00:FS00:0xa4:Verified 00/wudata_01.log
23:41:50:WU00:FS00:0xa4:Verified 00/wudata_01.trr
23:41:50:WU00:FS00:0xa4:Verified 00/wudata_01.xtc
23:41:50:WU00:FS00:0xa4:Verified 00/wudata_01.edr
23:41:50:WU00:FS00:0xa4:Completed 197980 out of 250000 steps  (79%)
23:41:50:WU01:FS01:0x15:Starting GUI Server
23:42:24:WU00:FS00:0xa4:Completed 200000 out of 250000 steps  (80%)
23:42:51:WU03:FS03:0x15:Setting checkpoint frequency: 400000
23:42:51:WU03:FS03:0x15:Completed         3 out of 40000000 steps (0%).
23:42:52:WU01:FS01:0x15:Setting checkpoint frequency: 400000
23:42:52:WU01:FS01:0x15:Completed         3 out of 40000000 steps (0%).
23:43:07:WU00:FS00:0xa4:Completed 202500 out of 250000 steps  (81%)
23:43:12:WU04:FS02:0x17:Completed 4750000 out of 5000000 steps (95%)
23:43:12:WU04:FS02:0x17:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
23:43:50:WU00:FS00:0xa4:Completed 205000 out of 250000 steps  (82%)
23:44:34:WU00:FS00:0xa4:Completed 207500 out of 250000 steps  (83%)
23:45:17:WU00:FS00:0xa4:Completed 210000 out of 250000 steps  (84%)
23:46:00:WU00:FS00:0xa4:Completed 212500 out of 250000 steps  (85%)
23:46:43:WU01:FS01:0x15:Completed    400000 out of 40000000 steps (1%).
23:46:43:WU00:FS00:0xa4:Completed 215000 out of 250000 steps  (86%)
23:47:27:WU00:FS00:0xa4:Completed 217500 out of 250000 steps  (87%)
23:48:10:WU00:FS00:0xa4:Completed 220000 out of 250000 steps  (88%)
23:48:54:WU00:FS00:0xa4:Completed 222500 out of 250000 steps  (89%)
23:49:31:WU04:FS02:0x17:Completed 4800000 out of 5000000 steps (96%)
23:49:38:WU00:FS00:0xa4:Completed 225000 out of 250000 steps  (90%)
23:50:24:WU00:FS00:0xa4:Completed 227500 out of 250000 steps  (91%)
23:50:34:WU01:FS01:0x15:Completed    800000 out of 40000000 steps (2%).
23:51:08:WU00:FS00:0xa4:Completed 230000 out of 250000 steps  (92%)
23:51:52:WU00:FS00:0xa4:Completed 232500 out of 250000 steps  (93%)
23:52:37:WU00:FS00:0xa4:Completed 235000 out of 250000 steps  (94%)
23:53:20:WU00:FS00:0xa4:Completed 237500 out of 250000 steps  (95%)
23:54:04:WU00:FS00:0xa4:Completed 240000 out of 250000 steps  (96%)
23:54:25:WU01:FS01:0x15:Completed   1200000 out of 40000000 steps (3%).
23:54:35:WU03:FS03:0x15:Completed    400000 out of 40000000 steps (1%).
23:54:47:WU00:FS00:0xa4:Completed 242500 out of 250000 steps  (97%)
23:55:30:WU00:FS00:0xa4:Completed 245000 out of 250000 steps  (98%)
23:55:53:WU04:FS02:0x17:Completed 4850000 out of 5000000 steps (97%)
23:56:14:WU00:FS00:0xa4:Completed 247500 out of 250000 steps  (99%)
23:56:14:WU05:FS00:Connecting to 171.67.108.200:8080
23:56:15:WU05:FS00:Assigned to work server 171.67.108.60
23:56:15:WU05:FS00:Requesting new work unit for slot 00: RUNNING cpu:5 from 171.67.108.60
23:56:15:WU05:FS00:Connecting to 171.67.108.60:8080
23:56:16:WU05:FS00:Downloading 532.88KiB
23:56:19:WU05:FS00:Download complete
23:56:19:WU05:FS00:Received Unit: id:05 state:DOWNLOAD error:NO_ERROR project:9500 run:918 clone:0 gen:60 core:0xa4 unit:0x000000436652edcc53642cabb29c9b27
23:56:57:WU00:FS00:0xa4:Completed 250000 out of 250000 steps  (100%)
23:56:57:WU00:FS00:0xa4:DynamicWrapper: Finished Work Unit: sleep=10000
23:57:07:WU00:FS00:0xa4:
23:57:07:WU00:FS00:0xa4:Finished Work Unit:
23:57:07:WU00:FS00:0xa4:- Reading up to 771180 from "00/wudata_01.trr": Read 771180
23:57:07:WU00:FS00:0xa4:trr file hash check passed.
23:57:07:WU00:FS00:0xa4:- Reading up to 456860 from "00/wudata_01.xtc": Read 456860
23:57:07:WU00:FS00:0xa4:xtc file hash check passed.
23:57:07:WU00:FS00:0xa4:edr file hash check passed.
23:57:07:WU00:FS00:0xa4:logfile size: 23505
23:57:07:WU00:FS00:0xa4:Leaving Run
23:57:09:WU00:FS00:0xa4:- Writing 1256949 bytes of core data to disk...
23:57:09:WU00:FS00:0xa4:Done: 1256437 -> 1195714 (compressed to 95.1 percent)
23:57:09:WU00:FS00:0xa4:  ... Done.
23:57:09:WU00:FS00:0xa4:- Shutting down core
23:57:09:WU00:FS00:0xa4:
23:57:09:WU00:FS00:0xa4:Folding@home Core Shutdown: FINISHED_UNIT
23:57:09:WU00:FS00:FahCore returned: FINISHED_UNIT (100 = 0x64)
23:57:09:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:9500 run:1292 clone:1 gen:28 core:0xa4 unit:0x000000206652edcc53642d55cb0ba55d
23:57:09:WU00:FS00:Uploading 1.14MiB to 171.67.108.60
23:57:09:WU00:FS00:Connecting to 171.67.108.60:8080
23:57:09:WU05:FS00:Starting
23:57:09:WU05:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Dave/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/Core_a4.fah/FahCore_a4.exe -dir 05 -suffix 01 -version 704 -lifeline 6808 -checkpoint 15 -np 5
23:57:09:WU05:FS00:Started FahCore on PID 5004
23:57:09:WU05:FS00:Core PID:4196
23:57:09:WU05:FS00:FahCore 0xa4 started
23:57:10:WU05:FS00:0xa4:
23:57:10:WU05:FS00:0xa4:*------------------------------*
23:57:10:WU05:FS00:0xa4:Folding@Home Gromacs GB Core
23:57:10:WU05:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
23:57:10:WU05:FS00:0xa4:
23:57:10:WU05:FS00:0xa4:Preparing to commence simulation
23:57:10:WU05:FS00:0xa4:- Looking at optimizations...
23:57:10:WU05:FS00:0xa4:- Created dyn
23:57:10:WU05:FS00:0xa4:- Files status OK
23:57:10:WU05:FS00:0xa4:- Expanded 545157 -> 1307852 (decompressed 239.9 percent)
23:57:10:WU05:FS00:0xa4:Called DecompressByteArray: compressed_data_size=545157 data_size=1307852, decompressed_data_size=1307852 diff=0
23:57:10:WU05:FS00:0xa4:- Digital signature verified
23:57:10:WU05:FS00:0xa4:
23:57:10:WU05:FS00:0xa4:Project: 9500 (Run 918, Clone 0, Gen 60)
23:57:10:WU05:FS00:0xa4:
23:57:10:WU05:FS00:0xa4:Assembly optimizations on if available.
23:57:10:WU05:FS00:0xa4:Entering M.D.
23:57:15:WU00:FS00:Upload 38.35%
23:57:15:WU05:FS00:0xa4:Mapping NT from 5 to 5 
23:57:15:WU05:FS00:0xa4:Completed 0 out of 250000 steps  (0%)
23:57:21:WU00:FS00:Upload 76.70%
23:57:27:WU00:FS00:Upload complete
23:57:27:WU00:FS00:Server responded WORK_ACK (400)
23:57:27:WU00:FS00:Final credit estimate, 685.00 points
23:57:27:WU00:FS00:Cleaning up
23:57:59:WU05:FS00:0xa4:Completed 2500 out of 250000 steps  (1%)
23:58:16:WU01:FS01:0x15:Completed   1600000 out of 40000000 steps (4%).
23:58:43:WU05:FS00:0xa4:Completed 5000 out of 250000 steps  (2%)
23:59:26:WU05:FS00:0xa4:Completed 7500 out of 250000 steps  (3%)
00:00:10:WU05:FS00:0xa4:Completed 10000 out of 250000 steps  (4%)
00:00:54:WU05:FS00:0xa4:Completed 12500 out of 250000 steps  (5%)
00:01:38:WU05:FS00:0xa4:Completed 15000 out of 250000 steps  (6%)
00:02:07:WU01:FS01:0x15:Completed   2000000 out of 40000000 steps (5%).
00:02:14:WU04:FS02:0x17:Completed 4900000 out of 5000000 steps (98%)
00:02:22:WU05:FS00:0xa4:Completed 17500 out of 250000 steps  (7%)
00:03:05:WU05:FS00:0xa4:Completed 20000 out of 250000 steps  (8%)
00:03:49:WU05:FS00:0xa4:Completed 22500 out of 250000 steps  (9%)
00:04:33:WU05:FS00:0xa4:Completed 25000 out of 250000 steps  (10%)
00:05:17:WU05:FS00:0xa4:Completed 27500 out of 250000 steps  (11%)
00:05:58:WU01:FS01:0x15:Completed   2400000 out of 40000000 steps (6%).
00:06:01:WU05:FS00:0xa4:Completed 30000 out of 250000 steps  (12%)
00:06:44:WU05:FS00:0xa4:Completed 32500 out of 250000 steps  (13%)
00:07:28:WU05:FS00:0xa4:Completed 35000 out of 250000 steps  (14%)
00:08:12:WU05:FS00:0xa4:Completed 37500 out of 250000 steps  (15%)
00:08:35:WU04:FS02:0x17:Completed 4950000 out of 5000000 steps (99%)
00:08:36:WU00:FS02:Connecting to 171.67.108.201:80
00:08:37:WU00:FS02:Assigned to work server 171.64.65.105
00:08:37:WU00:FS02:Requesting new work unit for slot 02: RUNNING gpu:1:GK110 [GeForce GTX Titan] from 171.64.65.105
00:08:37:WU00:FS02:Connecting to 171.64.65.105:8080
00:08:38:WU00:FS02:Downloading 122.13KiB
00:08:40:WU00:FS02:Download complete
00:08:40:WU00:FS02:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:7623 run:512 clone:0 gen:160 core:0x15 unit:0x000000f8664f2dd14fe4fb5209f345e9
00:08:55:WU05:FS00:0xa4:Completed 40000 out of 250000 steps  (16%)
00:09:03:WU03:FS03:0x15:Completed    800000 out of 40000000 steps (2%).
00:09:39:WU05:FS00:0xa4:Completed 42500 out of 250000 steps  (17%)
00:09:48:WU01:FS01:0x15:Completed   2800000 out of 40000000 steps (7%).
00:10:22:WU05:FS00:0xa4:Completed 45000 out of 250000 steps  (18%)
00:11:06:WU05:FS00:0xa4:Completed 47500 out of 250000 steps  (19%)
00:11:50:WU05:FS00:0xa4:Completed 50000 out of 250000 steps  (20%)
00:12:33:WU05:FS00:0xa4:Completed 52500 out of 250000 steps  (21%)
00:13:17:WU05:FS00:0xa4:Completed 55000 out of 250000 steps  (22%)
00:13:39:WU01:FS01:0x15:Completed   3200000 out of 40000000 steps (8%).
00:14:01:WU05:FS00:0xa4:Completed 57500 out of 250000 steps  (23%)
00:14:44:WU05:FS00:0xa4:Completed 60000 out of 250000 steps  (24%)
00:14:56:WU04:FS02:0x17:Completed 5000000 out of 5000000 steps (100%)
00:15:07:WU04:FS02:0x17:Saving result file logfile_01.txt
00:15:07:WU04:FS02:0x17:Saving result file checkpointState.xml
00:15:08:WU04:FS02:0x17:Saving result file checkpt.crc
00:15:08:WU04:FS02:0x17:Saving result file log.txt
00:15:08:WU04:FS02:0x17:Saving result file positions.xtc
00:15:12:WU04:FS02:0x17:Folding@home Core Shutdown: FINISHED_UNIT
00:15:13:WU04:FS02:FahCore returned: FINISHED_UNIT (100 = 0x64)
00:15:13:WU04:FS02:Sending unit results: id:04 state:SEND error:NO_ERROR project:9408 run:1623 clone:0 gen:24 core:0x17 unit:0x000000250a3b1e5c53432c4f098b6279
00:15:13:WU04:FS02:Uploading 20.32MiB to 171.64.65.56
00:15:13:WU04:FS02:Connecting to 171.64.65.56:8080
00:15:13:WU00:FS02:Starting
00:15:13:WU00:FS02:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Dave/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 00 -suffix 01 -version 704 -lifeline 6808 -checkpoint 15 -gpu 1 -gpu-vendor nvidia
00:15:13:WU00:FS02:Started FahCore on PID 7544
00:15:13:WU00:FS02:Core PID:4680
00:15:13:WU00:FS02:FahCore 0x15 started
00:15:13:WU00:FS02:0x15:
00:15:13:WU00:FS02:0x15:*------------------------------*
00:15:13:WU00:FS02:0x15:Folding@Home GPU Core
00:15:13:WU00:FS02:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
00:15:13:WU00:FS02:0x15:Build host             AmoebaRemote
00:15:13:WU00:FS02:0x15:Board Type             NVIDIA/CUDA
00:15:13:WU00:FS02:0x15:Core                   15
00:15:13:WU00:FS02:0x15:GPU device info vendor=0 device=0 name=NA match=0 deviceId=1
00:15:13:WU00:FS02:0x15:
00:15:13:WU00:FS02:0x15:Window's signal control handler registered.
00:15:13:WU00:FS02:0x15:Preparing to commence simulation
00:15:13:WU00:FS02:0x15:- Looking at optimizations...
00:15:13:WU00:FS02:0x15:DeleteFrameFiles: successfully deleted file=00/wudata_01.ckp
00:15:13:WU00:FS02:0x15:- Created dyn
00:15:13:WU00:FS02:0x15:- Files status OK
00:15:13:WU00:FS02:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
00:15:13:WU00:FS02:0x15:- Expanded 124550 -> 501826 (decompressed 402.9 percent)
00:15:13:WU00:FS02:0x15:Called DecompressByteArray: compressed_data_size=124550 data_size=501826, decompressed_data_size=501826 diff=0
00:15:13:WU00:FS02:0x15:- Digital signature verified
00:15:13:WU00:FS02:0x15:
00:15:13:WU00:FS02:0x15:Project: 7623 (Run 512, Clone 0, Gen 160)
00:15:13:WU00:FS02:0x15:
00:15:13:WU00:FS02:0x15:Assembly optimizations on if available.
00:15:13:WU00:FS02:0x15:Entering M.D.
00:15:15:WU00:FS02:0x15:Tpr hash 00/wudata_01.tpr:  3454981011 1059706818 221401786 1919140222 3087881404
00:15:15:WU00:FS02:0x15:GPU device id=1
00:15:15:WU00:FS02:0x15:Working on Protein
00:15:15:WU00:FS02:0x15:Client config unavailable.
00:15:15:WU00:FS02:0x15:Starting GUI Server
00:15:19:WU04:FS02:Upload 2.46%
00:15:25:WU04:FS02:Upload 4.61%
00:15:28:WU05:FS00:0xa4:Completed 62500 out of 250000 steps  (25%)
00:15:31:WU04:FS02:Upload 6.46%
00:15:37:WU04:FS02:Upload 8.61%
00:15:43:WU04:FS02:Upload 10.46%
00:15:49:WU04:FS02:Upload 12.61%
00:15:55:WU04:FS02:Upload 14.46%
00:16:01:WU04:FS02:Upload 16.61%
00:16:07:WU04:FS02:Upload 18.45%
00:16:10:WU05:FS00:0xa4:Completed 65000 out of 250000 steps  (26%)
00:16:13:WU04:FS02:Upload 20.30%
00:16:17:WU00:FS02:0x15:Setting checkpoint frequency: 400000
00:16:17:WU00:FS02:0x15:Completed         3 out of 40000000 steps (0%).
00:16:19:WU04:FS02:Upload 22.45%
00:16:25:WU04:FS02:Upload 24.30%
00:16:31:WU04:FS02:Upload 26.45%
00:16:37:WU04:FS02:Upload 28.30%
00:16:43:WU04:FS02:Upload 30.45%
00:16:49:WU04:FS02:Upload 32.30%
00:16:53:WU05:FS00:0xa4:Completed 67500 out of 250000 steps  (27%)
00:16:55:WU04:FS02:Upload 34.45%
00:17:01:WU04:FS02:Upload 36.29%
00:17:07:WU04:FS02:Upload 38.45%
00:17:13:WU04:FS02:Upload 40.29%
00:17:19:WU04:FS02:Upload 42.45%
00:17:25:WU04:FS02:Upload 44.29%
00:17:30:WU01:FS01:0x15:Completed   3600000 out of 40000000 steps (9%).
00:17:31:WU04:FS02:Upload 46.44%
00:17:35:WU05:FS00:0xa4:Completed 70000 out of 250000 steps  (28%)
00:17:37:WU04:FS02:Upload 48.29%
00:17:43:WU04:FS02:Upload 50.44%
00:17:49:WU04:FS02:Upload 52.29%
00:17:55:WU04:FS02:Upload 54.44%
00:18:01:WU04:FS02:Upload 56.29%
00:18:07:WU04:FS02:Upload 58.44%
00:18:13:WU04:FS02:Upload 60.29%
00:18:18:WU05:FS00:0xa4:Completed 72500 out of 250000 steps  (29%)
00:18:19:WU04:FS02:Upload 62.44%
00:18:25:WU04:FS02:Upload 64.28%
00:18:31:WU04:FS02:Upload 66.44%
00:18:37:WU04:FS02:Upload 68.28%
00:18:43:WU04:FS02:Upload 70.44%
00:18:49:WU04:FS02:Upload 72.28%
00:18:55:WU04:FS02:Upload 74.13%
00:19:00:WU05:FS00:0xa4:Completed 75000 out of 250000 steps  (30%)
00:19:01:WU04:FS02:Upload 76.28%
00:19:07:WU04:FS02:Upload 78.43%
00:19:13:WU04:FS02:Upload 80.28%
00:19:19:WU04:FS02:Upload 82.43%
00:19:25:WU04:FS02:Upload 84.28%
00:19:31:WU04:FS02:Upload 86.43%
00:19:37:WU04:FS02:Upload 88.28%
00:19:43:WU05:FS00:0xa4:Completed 77500 out of 250000 steps  (31%)
00:19:43:WU04:FS02:Upload 90.43%
00:19:49:WU04:FS02:Upload 92.27%
00:19:55:WU04:FS02:Upload 94.43%
00:20:01:WU04:FS02:Upload 96.27%
00:20:07:WU00:FS02:0x15:Completed    400000 out of 40000000 steps (1%).
00:20:07:WU04:FS02:Upload 98.43%
Mod - Edited to increase font size to make it readable.
Last edited by Volnaiskra on Sat May 31, 2014 11:44 am, edited 8 times in total.
Top
Volnaiskra
Posts: 57
Joined: Tue Nov 26, 2013 9:02 am

Re: work units not sending. Also, idle not working properly

Post by Volnaiskra »

Code: Select all

00:20:25:WU05:FS00:0xa4:Completed 80000 out of 250000 steps  (32%)
00:21:08:WU05:FS00:0xa4:Completed 82500 out of 250000 steps  (33%)
00:21:36:WU01:FS01:0x15:Completed   4000000 out of 40000000 steps (10%).
00:21:50:WU05:FS00:0xa4:Completed 85000 out of 250000 steps  (34%)
00:22:33:WU05:FS00:0xa4:Completed 87500 out of 250000 steps  (35%)
00:23:15:WU05:FS00:0xa4:Completed 90000 out of 250000 steps  (36%)
00:23:50:WU03:FS03:0x15:Completed   1200000 out of 40000000 steps (3%).
00:23:57:WU00:FS02:0x15:Completed    800000 out of 40000000 steps (2%).
00:23:58:WU05:FS00:0xa4:Completed 92500 out of 250000 steps  (37%)
00:24:40:WU05:FS00:0xa4:Completed 95000 out of 250000 steps  (38%)
00:25:23:WU05:FS00:0xa4:Completed 97500 out of 250000 steps  (39%)
00:25:46:WU01:FS01:0x15:Completed   4400000 out of 40000000 steps (11%).
00:26:05:WU05:FS00:0xa4:Completed 100000 out of 250000 steps  (40%)
00:26:48:WU05:FS00:0xa4:Completed 102500 out of 250000 steps  (41%)
00:27:30:WU05:FS00:0xa4:Completed 105000 out of 250000 steps  (42%)
00:27:47:WU00:FS02:0x15:Completed   1200000 out of 40000000 steps (3%).
00:28:13:WU05:FS00:0xa4:Completed 107500 out of 250000 steps  (43%)
00:28:56:WU05:FS00:0xa4:Completed 110000 out of 250000 steps  (44%)
00:29:38:WU05:FS00:0xa4:Completed 112500 out of 250000 steps  (45%)
00:29:54:WU01:FS01:0x15:Completed   4800000 out of 40000000 steps (12%).
00:30:21:WU05:FS00:0xa4:Completed 115000 out of 250000 steps  (46%)
00:31:03:WU05:FS00:0xa4:Completed 117500 out of 250000 steps  (47%)
00:31:37:WU00:FS02:0x15:Completed   1600000 out of 40000000 steps (4%).
00:31:46:WU05:FS00:0xa4:Completed 120000 out of 250000 steps  (48%)
00:32:29:WU05:FS00:0xa4:Completed 122500 out of 250000 steps  (49%)
00:33:13:WU05:FS00:0xa4:Completed 125000 out of 250000 steps  (50%)
00:33:46:WU01:FS01:0x15:Completed   5200000 out of 40000000 steps (13%).
00:33:55:WU05:FS00:0xa4:Completed 127500 out of 250000 steps  (51%)
00:34:38:WU05:FS00:0xa4:Completed 130000 out of 250000 steps  (52%)
00:35:21:WU05:FS00:0xa4:Completed 132500 out of 250000 steps  (53%)
00:35:27:WU00:FS02:0x15:Completed   2000000 out of 40000000 steps (5%).
00:36:03:WU05:FS00:0xa4:Completed 135000 out of 250000 steps  (54%)
00:36:46:WU05:FS00:0xa4:Completed 137500 out of 250000 steps  (55%)
00:37:28:WU05:FS00:0xa4:Completed 140000 out of 250000 steps  (56%)
00:37:52:WU01:FS01:0x15:Completed   5600000 out of 40000000 steps (14%).
00:38:11:WU05:FS00:0xa4:Completed 142500 out of 250000 steps  (57%)
00:38:43:WU03:FS03:0x15:Completed   1600000 out of 40000000 steps (4%).
00:38:53:WU05:FS00:0xa4:Completed 145000 out of 250000 steps  (58%)
00:39:17:WU00:FS02:0x15:Completed   2400000 out of 40000000 steps (6%).
00:39:36:WU05:FS00:0xa4:Completed 147500 out of 250000 steps  (59%)
00:40:19:WU05:FS00:0xa4:Completed 150000 out of 250000 steps  (60%)
00:41:01:WU05:FS00:0xa4:Completed 152500 out of 250000 steps  (61%)
00:41:44:WU05:FS00:0xa4:Completed 155000 out of 250000 steps  (62%)
00:41:55:WU01:FS01:0x15:Completed   6000000 out of 40000000 steps (15%).
00:42:26:WU05:FS00:0xa4:Completed 157500 out of 250000 steps  (63%)
00:43:07:WU00:FS02:0x15:Completed   2800000 out of 40000000 steps (7%).
00:43:09:WU05:FS00:0xa4:Completed 160000 out of 250000 steps  (64%)
00:43:51:WU05:FS00:0xa4:Completed 162500 out of 250000 steps  (65%)
00:44:34:WU05:FS00:0xa4:Completed 165000 out of 250000 steps  (66%)
00:45:16:WU05:FS00:0xa4:Completed 167500 out of 250000 steps  (67%)
00:45:45:WU01:FS01:0x15:Completed   6400000 out of 40000000 steps (16%).
00:45:59:WU05:FS00:0xa4:Completed 170000 out of 250000 steps  (68%)
00:46:41:WU05:FS00:0xa4:Completed 172500 out of 250000 steps  (69%)
00:46:58:WU00:FS02:0x15:Completed   3200000 out of 40000000 steps (8%).
00:47:24:WU05:FS00:0xa4:Completed 175000 out of 250000 steps  (70%)
00:48:06:WU05:FS00:0xa4:Completed 177500 out of 250000 steps  (71%)
00:48:49:WU05:FS00:0xa4:Completed 180000 out of 250000 steps  (72%)
00:49:31:WU05:FS00:0xa4:Completed 182500 out of 250000 steps  (73%)
00:49:35:WU01:FS01:0x15:Completed   6800000 out of 40000000 steps (17%).
00:50:14:WU05:FS00:0xa4:Completed 185000 out of 250000 steps  (74%)
00:50:48:WU00:FS02:0x15:Completed   3600000 out of 40000000 steps (9%).
00:50:56:WU05:FS00:0xa4:Completed 187500 out of 250000 steps  (75%)
00:51:39:WU05:FS00:0xa4:Completed 190000 out of 250000 steps  (76%)
00:52:22:WU05:FS00:0xa4:Completed 192500 out of 250000 steps  (77%)
00:53:04:WU05:FS00:0xa4:Completed 195000 out of 250000 steps  (78%)
00:53:26:WU01:FS01:0x15:Completed   7200000 out of 40000000 steps (18%).
00:53:47:WU05:FS00:0xa4:Completed 197500 out of 250000 steps  (79%)
00:53:50:WU03:FS03:0x15:Completed   2000000 out of 40000000 steps (5%).
00:54:29:WU05:FS00:0xa4:Completed 200000 out of 250000 steps  (80%)
00:54:38:WU00:FS02:0x15:Completed   4000000 out of 40000000 steps (10%).
00:55:12:WU05:FS00:0xa4:Completed 202500 out of 250000 steps  (81%)
00:55:54:WU05:FS00:0xa4:Completed 205000 out of 250000 steps  (82%)
00:56:37:WU05:FS00:0xa4:Completed 207500 out of 250000 steps  (83%)
00:57:20:WU05:FS00:0xa4:Completed 210000 out of 250000 steps  (84%)
00:57:26:WU01:FS01:0x15:Completed   7600000 out of 40000000 steps (19%).
00:58:02:WU05:FS00:0xa4:Completed 212500 out of 250000 steps  (85%)
00:58:28:WU00:FS02:0x15:Completed   4400000 out of 40000000 steps (11%).
00:58:45:WU05:FS00:0xa4:Completed 215000 out of 250000 steps  (86%)
00:59:27:WU05:FS00:0xa4:Completed 217500 out of 250000 steps  (87%)
01:00:10:WU05:FS00:0xa4:Completed 220000 out of 250000 steps  (88%)
01:00:52:WU05:FS00:0xa4:Completed 222500 out of 250000 steps  (89%)
01:01:20:WU01:FS01:0x15:Completed   8000000 out of 40000000 steps (20%).
01:01:35:WU05:FS00:0xa4:Completed 225000 out of 250000 steps  (90%)
01:02:18:WU05:FS00:0xa4:Completed 227500 out of 250000 steps  (91%)
01:02:18:WU00:FS02:0x15:Completed   4800000 out of 40000000 steps (12%).
01:03:00:WU05:FS00:0xa4:Completed 230000 out of 250000 steps  (92%)
01:03:43:WU05:FS00:0xa4:Completed 232500 out of 250000 steps  (93%)
01:04:25:WU05:FS00:0xa4:Completed 235000 out of 250000 steps  (94%)
01:05:08:WU05:FS00:0xa4:Completed 237500 out of 250000 steps  (95%)
01:05:38:WU01:FS01:0x15:Completed   8400000 out of 40000000 steps (21%).
01:05:50:WU05:FS00:0xa4:Completed 240000 out of 250000 steps  (96%)
01:06:08:WU00:FS02:0x15:Completed   5200000 out of 40000000 steps (13%).
01:06:33:WU05:FS00:0xa4:Completed 242500 out of 250000 steps  (97%)
01:07:15:WU05:FS00:0xa4:Completed 245000 out of 250000 steps  (98%)
01:07:58:WU05:FS00:0xa4:Completed 247500 out of 250000 steps  (99%)
01:07:59:WU06:FS00:Connecting to 171.67.108.200:8080
01:08:00:WU06:FS00:Assigned to work server 128.143.231.202
01:08:00:WU06:FS00:Requesting new work unit for slot 00: RUNNING cpu:5 from 128.143.231.202
01:08:00:WU06:FS00:Connecting to 128.143.231.202:8080
01:08:02:WU06:FS00:Downloading 3.67MiB
01:08:08:WU06:FS00:Download 32.36%
01:08:14:WU06:FS00:Download 71.53%
01:08:18:WU06:FS00:Download complete
01:08:18:WU06:FS00:Received Unit: id:06 state:DOWNLOAD error:NO_ERROR project:8576 run:1 clone:0 gen:402 core:0xa3 unit:0x000016df0a3b1e595228885c98d86fdb
01:08:40:WU05:FS00:0xa4:Completed 250000 out of 250000 steps  (100%)
01:08:40:WU05:FS00:0xa4:DynamicWrapper: Finished Work Unit: sleep=10000
01:08:46:WU03:FS03:0x15:Completed   2400000 out of 40000000 steps (6%).
01:08:50:WU05:FS00:0xa4:
01:08:50:WU05:FS00:0xa4:Finished Work Unit:
01:08:50:WU05:FS00:0xa4:- Reading up to 771720 from "05/wudata_01.trr": Read 771720
01:08:50:WU05:FS00:0xa4:trr file hash check passed.
01:08:50:WU05:FS00:0xa4:- Reading up to 457812 from "05/wudata_01.xtc": Read 457812
01:08:50:WU05:FS00:0xa4:xtc file hash check passed.
01:08:50:WU05:FS00:0xa4:edr file hash check passed.
01:08:50:WU05:FS00:0xa4:logfile size: 22660
01:08:50:WU05:FS00:0xa4:Leaving Run
01:08:55:WU05:FS00:0xa4:- Writing 1257596 bytes of core data to disk...
01:08:55:WU05:FS00:0xa4:Done: 1257084 -> 1197115 (compressed to 95.2 percent)
01:08:55:WU05:FS00:0xa4:  ... Done.
01:08:55:WU05:FS00:0xa4:- Shutting down core
01:08:55:WU05:FS00:0xa4:
01:08:55:WU05:FS00:0xa4:Folding@home Core Shutdown: FINISHED_UNIT
01:08:55:WU05:FS00:FahCore returned: FINISHED_UNIT (100 = 0x64)
01:08:55:WU05:FS00:Sending unit results: id:05 state:SEND error:NO_ERROR project:9500 run:918 clone:0 gen:60 core:0xa4 unit:0x000000436652edcc53642cabb29c9b27
01:08:55:WU05:FS00:Uploading 1.14MiB to 171.67.108.60
01:08:55:WU05:FS00:Connecting to 171.67.108.60:8080
01:08:55:WU06:FS00:Starting
01:08:55:WU06:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Dave/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/Core_a3.fah/FahCore_a3.exe -dir 06 -suffix 01 -version 704 -lifeline 6808 -checkpoint 15 -np 5
01:08:55:WU06:FS00:Started FahCore on PID 9048
01:08:55:WU06:FS00:Core PID:8888
01:08:55:WU06:FS00:FahCore 0xa3 started
01:08:56:WU06:FS00:0xa3:
01:08:56:WU06:FS00:0xa3:*------------------------------*
01:08:56:WU06:FS00:0xa3:Folding@Home Gromacs SMP Core
01:08:56:WU06:FS00:0xa3:Version 2.27 (Dec. 15, 2010)
01:08:56:WU06:FS00:0xa3:
01:08:56:WU06:FS00:0xa3:Preparing to commence simulation
01:08:56:WU06:FS00:0xa3:- Looking at optimizations...
01:08:56:WU06:FS00:0xa3:- Created dyn
01:08:56:WU06:FS00:0xa3:- Files status OK
01:08:56:WU06:FS00:0xa3:- Expanded 3847704 -> 4384444 (decompressed 113.9 percent)
01:08:56:WU06:FS00:0xa3:Called DecompressByteArray: compressed_data_size=3847704 data_size=4384444, decompressed_data_size=4384444 diff=0
01:08:56:WU06:FS00:0xa3:- Digital signature verified
01:08:56:WU06:FS00:0xa3:
01:08:56:WU06:FS00:0xa3:Project: 8576 (Run 1, Clone 0, Gen 402)
01:08:56:WU06:FS00:0xa3:
01:08:56:WU06:FS00:0xa3:Assembly optimizations on if available.
01:08:56:WU06:FS00:0xa3:Entering M.D.
01:09:01:WU05:FS00:Upload 38.31%
01:09:02:WU06:FS00:0xa3:Mapping NT from 5 to 5 
01:09:02:WU06:FS00:0xa3:Completed 0 out of 500000 steps  (0%)
01:09:07:WU05:FS00:Upload 76.61%
01:09:13:WU05:FS00:Upload complete
01:09:13:WU05:FS00:Server responded WORK_ACK (400)
01:09:13:WU05:FS00:Final credit estimate, 775.00 points
01:09:13:WU05:FS00:Cleaning up
01:09:51:WU01:FS01:0x15:Completed   8800000 out of 40000000 steps (22%).
01:09:58:WU00:FS02:0x15:Completed   5600000 out of 40000000 steps (14%).
01:13:48:WU00:FS02:0x15:Completed   6000000 out of 40000000 steps (15%).
01:14:14:WU01:FS01:0x15:Completed   9200000 out of 40000000 steps (23%).
01:17:38:WU00:FS02:0x15:Completed   6400000 out of 40000000 steps (16%).
01:18:31:WU01:FS01:0x15:Completed   9600000 out of 40000000 steps (24%).
01:19:15:WU06:FS00:0xa3:Completed 5000 out of 500000 steps  (1%)
01:21:29:WU00:FS02:0x15:Completed   6800000 out of 40000000 steps (17%).
01:22:22:WU01:FS01:0x15:Completed  10000000 out of 40000000 steps (25%).
01:23:35:WU03:FS03:0x15:Completed   2800000 out of 40000000 steps (7%).
01:25:19:WU00:FS02:0x15:Completed   7200000 out of 40000000 steps (18%).
01:26:38:WU01:FS01:0x15:Completed  10400000 out of 40000000 steps (26%).
01:29:09:WU00:FS02:0x15:Completed   7600000 out of 40000000 steps (19%).
01:29:27:WU06:FS00:0xa3:Completed 10000 out of 500000 steps  (2%)
01:30:34:WU01:FS01:0x15:Completed  10800000 out of 40000000 steps (27%).
01:32:59:WU00:FS02:0x15:Completed   8000000 out of 40000000 steps (20%).
01:34:47:WU01:FS01:0x15:Completed  11200000 out of 40000000 steps (28%).
01:36:50:WU00:FS02:0x15:Completed   8400000 out of 40000000 steps (21%).
01:38:30:WU03:FS03:0x15:Completed   3200000 out of 40000000 steps (8%).
01:38:55:WU01:FS01:0x15:Completed  11600000 out of 40000000 steps (29%).
01:39:42:WU06:FS00:0xa3:Completed 15000 out of 500000 steps  (3%)
01:40:40:WU00:FS02:0x15:Completed   8800000 out of 40000000 steps (22%).
01:43:04:WU01:FS01:0x15:Completed  12000000 out of 40000000 steps (30%).
01:44:30:WU00:FS02:0x15:Completed   9200000 out of 40000000 steps (23%).
01:47:15:WU01:FS01:0x15:Completed  12400000 out of 40000000 steps (31%).
01:48:20:WU00:FS02:0x15:Completed   9600000 out of 40000000 steps (24%).
01:49:55:WU06:FS00:0xa3:Completed 20000 out of 500000 steps  (4%)
01:51:28:WU01:FS01:0x15:Completed  12800000 out of 40000000 steps (32%).
01:52:10:WU00:FS02:0x15:Completed  10000000 out of 40000000 steps (25%).
01:53:28:WU03:FS03:0x15:Completed   3600000 out of 40000000 steps (9%).
01:55:40:WU01:FS01:0x15:Completed  13200000 out of 40000000 steps (33%).
01:56:00:WU00:FS02:0x15:Completed  10400000 out of 40000000 steps (26%).
01:59:32:WU01:FS01:0x15:Completed  13600000 out of 40000000 steps (34%).
01:59:51:WU00:FS02:0x15:Completed  10800000 out of 40000000 steps (27%).
02:00:09:WU06:FS00:0xa3:Completed 25000 out of 500000 steps  (5%)
02:03:41:WU00:FS02:0x15:Completed  11200000 out of 40000000 steps (28%).
02:03:51:WU01:FS01:0x15:Completed  14000000 out of 40000000 steps (35%).
02:07:31:WU00:FS02:0x15:Completed  11600000 out of 40000000 steps (29%).
02:07:49:WU01:FS01:0x15:Completed  14400000 out of 40000000 steps (36%).
02:08:36:WU03:FS03:0x15:Completed   4000000 out of 40000000 steps (10%).
02:10:22:WU06:FS00:0xa3:Completed 30000 out of 500000 steps  (6%)
02:11:22:WU00:FS02:0x15:Completed  12000000 out of 40000000 steps (30%).
02:11:41:WU01:FS01:0x15:Completed  14800000 out of 40000000 steps (37%).
02:15:12:WU00:FS02:0x15:Completed  12400000 out of 40000000 steps (31%).
02:15:34:WU01:FS01:0x15:Completed  15200000 out of 40000000 steps (38%).
02:19:02:WU00:FS02:0x15:Completed  12800000 out of 40000000 steps (32%).
02:19:53:WU01:FS01:0x15:Completed  15600000 out of 40000000 steps (39%).
02:20:34:WU06:FS00:0xa3:Completed 35000 out of 500000 steps  (7%)
02:22:52:WU00:FS02:0x15:Completed  13200000 out of 40000000 steps (33%).
02:23:50:WU03:FS03:0x15:Completed   4400000 out of 40000000 steps (11%).
02:24:09:WU01:FS01:0x15:Completed  16000000 out of 40000000 steps (40%).
02:26:42:WU00:FS02:0x15:Completed  13600000 out of 40000000 steps (34%).
02:28:34:WU01:FS01:0x15:Completed  16400000 out of 40000000 steps (41%).
02:30:33:WU00:FS02:0x15:Completed  14000000 out of 40000000 steps (35%).
02:30:45:WU06:FS00:0xa3:Completed 40000 out of 500000 steps  (8%)
02:32:41:WU01:FS01:0x15:Completed  16800000 out of 40000000 steps (42%).
02:34:23:WU00:FS02:0x15:Completed  14400000 out of 40000000 steps (36%).
02:36:32:WU01:FS01:0x15:Completed  17200000 out of 40000000 steps (43%).
02:38:13:WU00:FS02:0x15:Completed  14800000 out of 40000000 steps (37%).
02:38:59:WU03:FS03:0x15:Completed   4800000 out of 40000000 steps (12%).
02:40:57:WU06:FS00:0xa3:Completed 45000 out of 500000 steps  (9%)
02:40:58:WU01:FS01:0x15:Completed  17600000 out of 40000000 steps (44%).
02:42:03:WU00:FS02:0x15:Completed  15200000 out of 40000000 steps (38%).
02:44:59:WU01:FS01:0x15:Completed  18000000 out of 40000000 steps (45%).
02:45:53:WU00:FS02:0x15:Completed  15600000 out of 40000000 steps (39%).
02:49:32:WU01:FS01:0x15:Completed  18400000 out of 40000000 steps (46%).
02:49:43:WU00:FS02:0x15:Completed  16000000 out of 40000000 steps (40%).
02:51:09:WU06:FS00:0xa3:Completed 50000 out of 500000 steps  (10%)
02:53:34:WU00:FS02:0x15:Completed  16400000 out of 40000000 steps (41%).
02:54:01:WU03:FS03:0x15:Completed   5200000 out of 40000000 steps (13%).
02:54:17:WU01:FS01:0x15:Completed  18800000 out of 40000000 steps (47%).
02:57:24:WU00:FS02:0x15:Completed  16800000 out of 40000000 steps (42%).
02:58:39:WU01:FS01:0x15:Completed  19200000 out of 40000000 steps (48%).
03:01:14:WU00:FS02:0x15:Completed  17200000 out of 40000000 steps (43%).
03:01:20:WU06:FS00:0xa3:Completed 55000 out of 500000 steps  (11%)
03:02:41:WU01:FS01:0x15:Completed  19600000 out of 40000000 steps (49%).
03:05:04:WU00:FS02:0x15:Completed  17600000 out of 40000000 steps (44%).
03:06:47:WU01:FS01:0x15:Completed  20000000 out of 40000000 steps (50%).
03:08:54:WU00:FS02:0x15:Completed  18000000 out of 40000000 steps (45%).
03:09:13:WU03:FS03:0x15:Completed   5600000 out of 40000000 steps (14%).
03:11:07:WU01:FS01:0x15:Completed  20400000 out of 40000000 steps (51%).
03:11:32:WU06:FS00:0xa3:Completed 60000 out of 500000 steps  (12%)
03:12:45:WU00:FS02:0x15:Completed  18400000 out of 40000000 steps (46%).
03:15:09:WU01:FS01:0x15:Completed  20800000 out of 40000000 steps (52%).
03:16:35:WU00:FS02:0x15:Completed  18800000 out of 40000000 steps (47%).
03:19:01:WU01:FS01:0x15:Completed  21200000 out of 40000000 steps (53%).
03:20:25:WU00:FS02:0x15:Completed  19200000 out of 40000000 steps (48%).
03:21:44:WU06:FS00:0xa3:Completed 65000 out of 500000 steps  (13%)
03:23:11:WU01:FS01:0x15:Completed  21600000 out of 40000000 steps (54%).
03:24:15:WU00:FS02:0x15:Completed  19600000 out of 40000000 steps (49%).
03:24:41:WU03:FS03:0x15:Completed   6000000 out of 40000000 steps (15%).
03:27:27:WU01:FS01:0x15:Completed  22000000 out of 40000000 steps (55%).
03:28:05:WU00:FS02:0x15:Completed  20000000 out of 40000000 steps (50%).
03:31:55:WU06:FS00:0xa3:Completed 70000 out of 500000 steps  (14%)
03:31:55:WU00:FS02:0x15:Completed  20400000 out of 40000000 steps (51%).
03:32:00:WU01:FS01:0x15:Completed  22400000 out of 40000000 steps (56%).
03:35:46:WU00:FS02:0x15:Completed  20800000 out of 40000000 steps (52%).
03:36:17:WU01:FS01:0x15:Completed  22800000 out of 40000000 steps (57%).
03:39:36:WU00:FS02:0x15:Completed  21200000 out of 40000000 steps (53%).
03:40:01:WU03:FS03:0x15:Completed   6400000 out of 40000000 steps (16%).
03:40:37:WU01:FS01:0x15:Completed  23200000 out of 40000000 steps (58%).
03:42:07:WU06:FS00:0xa3:Completed 75000 out of 500000 steps  (15%)
03:43:26:WU00:FS02:0x15:Completed  21600000 out of 40000000 steps (54%).
03:44:27:FS00:Shutting core down
03:44:27:FS01:Shutting core down
03:44:27:FS02:Shutting core down
03:44:27:FS03:Shutting core down
03:44:28:WU03:FS03:0x15:Client no longer detected. Shutting down core 
03:44:28:WU03:FS03:0x15:
03:44:28:WU03:FS03:0x15:Folding@home Core Shutdown: CLIENT_DIED
03:44:28:WU03:FS03:FahCore returned: INTERRUPTED (102 = 0x66)
03:44:28:WU01:FS01:0x15:Client no longer detected. Shutting down core 
03:44:28:WU01:FS01:0x15:
03:44:28:WU01:FS01:0x15:Folding@home Core Shutdown: CLIENT_DIED
03:44:29:WU01:FS01:FahCore returned: INTERRUPTED (102 = 0x66)
03:44:34:WU00:FS02:0x15:Client no longer detected. Shutting down core 
03:44:34:WU00:FS02:0x15:
03:44:34:WU00:FS02:0x15:Folding@home Core Shutdown: CLIENT_DIED
03:44:34:WU00:FS02:FahCore returned: INTERRUPTED (102 = 0x66)
03:44:36:WU06:FS00:0xa3:Client no longer detected. Shutting down core 
03:44:36:WU06:FS00:0xa3:
03:44:36:WU06:FS00:0xa3:Folding@home Core Shutdown: CLIENT_DIED
03:44:36:WU06:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
03:55:20:WU01:FS01:Starting
03:55:20:WU01:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Dave/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 01 -suffix 01 -version 704 -lifeline 6808 -checkpoint 15 -gpu 0 -gpu-vendor nvidia
03:55:20:WU01:FS01:Started FahCore on PID 8460
03:55:20:WU01:FS01:Core PID:7380
03:55:20:WU01:FS01:FahCore 0x15 started
03:55:20:WU03:FS03:Starting
03:55:20:WU03:FS03:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Dave/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 03 -suffix 01 -version 704 -lifeline 6808 -checkpoint 15 -gpu 2 -gpu-vendor nvidia
03:55:20:WU03:FS03:Started FahCore on PID 8964
03:55:20:WU03:FS03:Core PID:4240
03:55:20:WU03:FS03:FahCore 0x15 started
03:55:20:WU00:FS02:Starting
03:55:20:WU00:FS02:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Dave/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 00 -suffix 01 -version 704 -lifeline 6808 -checkpoint 15 -gpu 1 -gpu-vendor nvidia
03:55:20:WU00:FS02:Started FahCore on PID 7680
03:55:20:WU00:FS02:Core PID:7704
03:55:20:WU00:FS02:FahCore 0x15 started
03:55:20:WU06:FS00:Starting
03:55:20:WU06:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Dave/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/Core_a3.fah/FahCore_a3.exe -dir 06 -suffix 01 -version 704 -lifeline 6808 -checkpoint 15 -np 5
03:55:20:WU06:FS00:Started FahCore on PID 8372
03:55:20:WU06:FS00:Core PID:9080
03:55:20:WU06:FS00:FahCore 0xa3 started
03:55:21:WU01:FS01:0x15:
03:55:21:WU01:FS01:0x15:*------------------------------*
03:55:21:WU01:FS01:0x15:Folding@Home GPU Core
03:55:21:WU01:FS01:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
03:55:21:WU01:FS01:0x15:Build host             AmoebaRemote
03:55:21:WU01:FS01:0x15:Board Type             NVIDIA/CUDA
03:55:21:WU01:FS01:0x15:Core                   15
03:55:21:WU01:FS01:0x15:
03:55:21:WU01:FS01:0x15:Window's signal control handler registered.
03:55:21:WU01:FS01:0x15:Preparing to commence simulation
03:55:21:WU01:FS01:0x15:- Looking at optimizations...
03:55:21:WU01:FS01:0x15:- Files status OK
03:55:21:WU01:FS01:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
03:55:21:WU01:FS01:0x15:- Expanded 125034 -> 501826 (decompressed 401.3 percent)
03:55:21:WU01:FS01:0x15:Called DecompressByteArray: compressed_data_size=125034 data_size=501826, decompressed_data_size=501826 diff=0
03:55:21:WU01:FS01:0x15:- Digital signature verified
03:55:21:WU01:FS01:0x15:
03:55:21:WU01:FS01:0x15:Project: 7622 (Run 198, Clone 0, Gen 70)
03:55:21:WU01:FS01:0x15:
03:55:21:WU01:FS01:0x15:Assembly optimizations on if available.
03:55:21:WU01:FS01:0x15:Entering M.D.
03:55:21:WU03:FS03:0x15:
03:55:21:WU03:FS03:0x15:*------------------------------*
03:55:21:WU03:FS03:0x15:Folding@Home GPU Core
03:55:21:WU03:FS03:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
03:55:21:WU03:FS03:0x15:Build host             AmoebaRemote
03:55:21:WU03:FS03:0x15:Board Type             NVIDIA/CUDA
03:55:21:WU03:FS03:0x15:Core                   15
03:55:21:WU03:FS03:0x15:GPU device info vendor=0 device=0 name=NA match=0 deviceId=2
03:55:21:WU03:FS03:0x15:
03:55:21:WU03:FS03:0x15:Window's signal control handler registered.
03:55:21:WU03:FS03:0x15:Preparing to commence simulation
03:55:21:WU03:FS03:0x15:- Looking at optimizations...
03:55:21:WU03:FS03:0x15:- Files status OK
03:55:21:WU03:FS03:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
03:55:21:WU03:FS03:0x15:- Expanded 124351 -> 501826 (decompressed 403.5 percent)
03:55:21:WU03:FS03:0x15:Called DecompressByteArray: compressed_data_size=124351 data_size=501826, decompressed_data_size=501826 diff=0
03:55:21:WU03:FS03:0x15:- Digital signature verified
03:55:21:WU03:FS03:0x15:
03:55:21:WU03:FS03:0x15:Project: 7623 (Run 261, Clone 0, Gen 166)
03:55:21:WU03:FS03:0x15:
03:55:21:WU03:FS03:0x15:Assembly optimizations on if available.
03:55:21:WU03:FS03:0x15:Entering M.D.
03:55:21:WU00:FS02:0x15:
03:55:21:WU00:FS02:0x15:*------------------------------*
03:55:21:WU00:FS02:0x15:Folding@Home GPU Core
03:55:21:WU00:FS02:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
03:55:21:WU00:FS02:0x15:Build host             AmoebaRemote
03:55:21:WU00:FS02:0x15:Board Type             NVIDIA/CUDA
03:55:21:WU00:FS02:0x15:Core                   15
03:55:21:WU00:FS02:0x15:GPU device info vendor=0 device=0 name=NA match=0 deviceId=1
03:55:21:WU00:FS02:0x15:
03:55:21:WU00:FS02:0x15:Window's signal control handler registered.
03:55:21:WU00:FS02:0x15:Preparing to commence simulation
03:55:21:WU00:FS02:0x15:- Looking at optimizations...
03:55:21:WU00:FS02:0x15:- Files status OK
03:55:21:WU00:FS02:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
03:55:21:WU00:FS02:0x15:- Expanded 124550 -> 501826 (decompressed 402.9 percent)
03:55:21:WU00:FS02:0x15:Called DecompressByteArray: compressed_data_size=124550 data_size=501826, decompressed_data_size=501826 diff=0
03:55:21:WU00:FS02:0x15:- Digital signature verified
03:55:21:WU00:FS02:0x15:
03:55:21:WU00:FS02:0x15:Project: 7623 (Run 512, Clone 0, Gen 160)
03:55:21:WU00:FS02:0x15:
03:55:21:WU00:FS02:0x15:Assembly optimizations on if available.
03:55:21:WU00:FS02:0x15:Entering M.D.
03:55:21:WU06:FS00:0xa3:
03:55:21:WU06:FS00:0xa3:*------------------------------*
03:55:21:WU06:FS00:0xa3:Folding@Home Gromacs SMP Core
03:55:21:WU06:FS00:0xa3:Version 2.27 (Dec. 15, 2010)
03:55:21:WU06:FS00:0xa3:
03:55:21:WU06:FS00:0xa3:Preparing to commence simulation
03:55:21:WU06:FS00:0xa3:- Looking at optimizations...
03:55:21:WU06:FS00:0xa3:- Files status OK
03:55:21:WU06:FS00:0xa3:- Expanded 3847704 -> 4384444 (decompressed 113.9 percent)
03:55:21:WU06:FS00:0xa3:Called DecompressByteArray: compressed_data_size=3847704 data_size=4384444, decompressed_data_size=4384444 diff=0
03:55:21:WU06:FS00:0xa3:- Digital signature verified
03:55:21:WU06:FS00:0xa3:
03:55:21:WU06:FS00:0xa3:Project: 8576 (Run 1, Clone 0, Gen 402)
03:55:21:WU06:FS00:0xa3:
03:55:21:WU06:FS00:0xa3:Assembly optimizations on if available.
03:55:21:WU06:FS00:0xa3:Entering M.D.
03:55:23:WU01:FS01:0x15:Will resume from checkpoint file 01/wudata_01.ckp
03:55:23:WU01:FS01:0x15:Tpr hash 01/wudata_01.tpr:  3336875573 1615538842 2324230187 4260796349 2057047944
03:55:23:WU01:FS01:0x15:GPU device id=0
03:55:23:WU01:FS01:0x15:Working on Protein
03:55:23:WU01:FS01:0x15:Client config unavailable.
03:55:23:WU03:FS03:0x15:Will resume from checkpoint file 03/wudata_01.ckp
03:55:23:WU03:FS03:0x15:Tpr hash 03/wudata_01.tpr:  310465715 1833047723 3414748523 2877205478 1558103798
03:55:23:WU03:FS03:0x15:GPU device id=2
03:55:23:WU03:FS03:0x15:Working on Protein
03:55:23:WU03:FS03:0x15:Client config unavailable.
03:55:23:WU00:FS02:0x15:Will resume from checkpoint file 00/wudata_01.ckp
03:55:23:WU00:FS02:0x15:Tpr hash 00/wudata_01.tpr:  3454981011 1059706818 221401786 1919140222 3087881404
03:55:23:WU00:FS02:0x15:GPU device id=1
03:55:23:WU00:FS02:0x15:Working on Protein
03:55:23:WU00:FS02:0x15:Client config unavailable.
03:55:23:WU01:FS01:0x15:Starting GUI Server
03:55:23:WU03:FS03:0x15:Starting GUI Server
03:55:23:WU00:FS02:0x15:Starting GUI Server
03:55:27:WU06:FS00:0xa3:Using Gromacs checkpoints
03:55:27:WU06:FS00:0xa3:Mapping NT from 5 to 5 
03:55:27:WU06:FS00:0xa3:Resuming from checkpoint
03:55:27:WU06:FS00:0xa3:Verified 06/wudata_01.log
03:55:27:WU06:FS00:0xa3:Verified 06/wudata_01.trr
03:55:27:WU06:FS00:0xa3:Verified 06/wudata_01.edr
03:55:28:WU06:FS00:0xa3:Completed 73495 out of 500000 steps  (14%)
03:56:24:WU00:FS02:0x15:Resuming from checkpoint
03:56:24:WU00:FS02:0x15:fcCheckPointResume: retreived and current tpr file hash:
03:56:24:WU00:FS02:0x15:   0   3454981011   3454981011
03:56:24:WU00:FS02:0x15:   1   1059706818   1059706818
03:56:24:WU00:FS02:0x15:   2    221401786    221401786
03:56:24:WU00:FS02:0x15:   3   1919140222   1919140222
03:56:24:WU00:FS02:0x15:   4   3087881404   3087881404
03:56:24:WU00:FS02:0x15:fcCheckPointResume: file hashes same.
03:56:24:WU00:FS02:0x15:fcCheckPointResume: state restored.
03:56:24:WU00:FS02:0x15:fcCheckPointResume: name 00/wudata_01.log Verified 00/wudata_01.log
03:56:24:WU00:FS02:0x15:fcCheckPointResume: name 00/wudata_01.trr Verified 00/wudata_01.trr
03:56:24:WU00:FS02:0x15:fcCheckPointResume: name 00/wudata_01.xtc Verified 00/wudata_01.xtc
03:56:24:WU00:FS02:0x15:fcCheckPointResume: name 00/wudata_01.edr Verified 00/wudata_01.edr
03:56:24:WU00:FS02:0x15:fcCheckPointResume: state restored 2
03:56:24:WU00:FS02:0x15:Resumed from checkpoint
03:56:24:WU00:FS02:0x15:Setting checkpoint frequency: 400000
03:56:24:WU00:FS02:0x15:Completed  21600001 out of 40000000 steps (54%).
03:56:25:WU01:FS01:0x15:Resuming from checkpoint
03:56:25:WU01:FS01:0x15:fcCheckPointResume: retreived and current tpr file hash:
03:56:25:WU01:FS01:0x15:   0   3336875573   3336875573
03:56:25:WU01:FS01:0x15:   1   1615538842   1615538842
03:56:25:WU01:FS01:0x15:   2   2324230187   2324230187
03:56:25:WU01:FS01:0x15:   3   4260796349   4260796349
03:56:25:WU01:FS01:0x15:   4   2057047944   2057047944
03:56:25:WU01:FS01:0x15:fcCheckPointResume: file hashes same.
03:56:25:WU01:FS01:0x15:fcCheckPointResume: state restored.
03:56:25:WU01:FS01:0x15:fcCheckPointResume: name 01/wudata_01.log Verified 01/wudata_01.log
03:56:25:WU01:FS01:0x15:fcCheckPointResume: name 01/wudata_01.trr Verified 01/wudata_01.trr
03:56:25:WU01:FS01:0x15:fcCheckPointResume: name 01/wudata_01.xtc Verified 01/wudata_01.xtc
03:56:25:WU01:FS01:0x15:fcCheckPointResume: name 01/wudata_01.edr Verified 01/wudata_01.edr
03:56:25:WARNING:WU01:FS01:Detected clock skew (1 mins 05 secs), adjusting time estimates
03:56:25:WU01:FS01:0x15:fcCheckPointResume: state restored 2
03:56:25:WARNING:WU00:FS02:Detected clock skew (1 mins 05 secs), adjusting time estimates
03:56:25:WU01:FS01:0x15:Resumed from checkpoint
03:56:25:WU01:FS01:0x15:Setting checkpoint frequency: 400000
03:56:25:WU01:FS01:0x15:Completed  23200001 out of 40000000 steps (58%).
03:56:25:WU03:FS03:0x15:Resuming from checkpoint
03:56:25:WU03:FS03:0x15:fcCheckPointResume: retreived and current tpr file hash:
03:56:25:WU03:FS03:0x15:   0    310465715    310465715
03:56:25:WU03:FS03:0x15:   1   1833047723   1833047723
03:56:25:WU03:FS03:0x15:   2   3414748523   3414748523
03:56:25:WU03:FS03:0x15:   3   2877205478   2877205478
03:56:25:WU03:FS03:0x15:   4   1558103798   1558103798
03:56:25:WU03:FS03:0x15:fcCheckPointResume: file hashes same.
03:56:25:WU03:FS03:0x15:fcCheckPointResume: state restored.
03:56:25:WU03:FS03:0x15:fcCheckPointResume: name 03/wudata_01.log Verified 03/wudata_01.log
03:56:25:WU03:FS03:0x15:fcCheckPointResume: name 03/wudata_01.trr Verified 03/wudata_01.trr
03:56:25:WU03:FS03:0x15:fcCheckPointResume: name 03/wudata_01.xtc Verified 03/wudata_01.xtc
03:56:25:WU03:FS03:0x15:fcCheckPointResume: name 03/wudata_01.edr Verified 03/wudata_01.edr
03:56:25:WU03:FS03:0x15:fcCheckPointResume: state restored 2
03:56:25:WU03:FS03:0x15:Resumed from checkpoint
03:56:25:WU03:FS03:0x15:Setting checkpoint frequency: 400000
03:56:25:WU03:FS03:0x15:Completed   6400001 out of 40000000 steps (16%).
03:56:26:WARNING:WU03:FS03:Detected clock skew (1 mins 06 secs), adjusting time estimates
03:58:35:WU06:FS00:0xa3:Completed 75000 out of 500000 steps  (15%)
04:00:15:WU01:FS01:0x15:Completed  23600000 out of 40000000 steps (59%).
04:00:15:WU00:FS02:0x15:Completed  22000000 out of 40000000 steps (55%).
04:04:05:WU01:FS01:0x15:Completed  24000000 out of 40000000 steps (60%).
04:04:06:WU00:FS02:0x15:Completed  22400000 out of 40000000 steps (56%).
04:07:56:WU00:FS02:0x15:Completed  22800000 out of 40000000 steps (57%).
04:07:58:WU01:FS01:0x15:Completed  24400000 out of 40000000 steps (61%).
04:09:04:WU06:FS00:0xa3:Completed 80000 out of 500000 steps  (16%)
04:09:13:WU03:FS03:0x15:Completed   6800000 out of 40000000 steps (17%).
04:11:47:WU00:FS02:0x15:Completed  23200000 out of 40000000 steps (58%).
04:11:51:WU01:FS01:0x15:Completed  24800000 out of 40000000 steps (62%).
04:15:37:WU00:FS02:0x15:Completed  23600000 out of 40000000 steps (59%).
04:15:42:WU01:FS01:0x15:Completed  25200000 out of 40000000 steps (63%).
04:19:28:WU00:FS02:0x15:Completed  24000000 out of 40000000 steps (60%).
04:19:33:WU06:FS00:0xa3:Completed 85000 out of 500000 steps  (17%)
04:19:38:WU01:FS01:0x15:Completed  25600000 out of 40000000 steps (64%).
04:23:19:WU00:FS02:0x15:Completed  24400000 out of 40000000 steps (61%).
04:23:28:WU01:FS01:0x15:Completed  26000000 out of 40000000 steps (65%).
04:25:01:WU03:FS03:0x15:Completed   7200000 out of 40000000 steps (18%).
04:27:09:WU00:FS02:0x15:Completed  24800000 out of 40000000 steps (62%).
04:27:19:WU01:FS01:0x15:Completed  26400000 out of 40000000 steps (66%).
04:30:02:WU06:FS00:0xa3:Completed 90000 out of 500000 steps  (18%)
04:31:00:WU00:FS02:0x15:Completed  25200000 out of 40000000 steps (63%).
04:31:11:WU01:FS01:0x15:Completed  26800000 out of 40000000 steps (67%).
04:34:50:WU00:FS02:0x15:Completed  25600000 out of 40000000 steps (64%).
04:35:04:WU01:FS01:0x15:Completed  27200000 out of 40000000 steps (68%).
04:38:41:WU00:FS02:0x15:Completed  26000000 out of 40000000 steps (65%).
04:38:57:WU01:FS01:0x15:Completed  27600000 out of 40000000 steps (69%).
04:40:32:WU06:FS00:0xa3:Completed 95000 out of 500000 steps  (19%)
04:41:04:WU03:FS03:0x15:Completed   7600000 out of 40000000 steps (19%).
04:42:32:WU00:FS02:0x15:Completed  26400000 out of 40000000 steps (66%).
04:42:47:WU01:FS01:0x15:Completed  28000000 out of 40000000 steps (70%).
04:46:22:WU00:FS02:0x15:Completed  26800000 out of 40000000 steps (67%).
04:46:37:WU01:FS01:0x15:Completed  28400000 out of 40000000 steps (71%).
04:50:13:WU00:FS02:0x15:Completed  27200000 out of 40000000 steps (68%).
04:50:28:WU01:FS01:0x15:Completed  28800000 out of 40000000 steps (72%).
04:51:01:WU06:FS00:0xa3:Completed 100000 out of 500000 steps  (20%)
04:54:03:WU00:FS02:0x15:Completed  27600000 out of 40000000 steps (69%).
04:54:18:WU01:FS01:0x15:Completed  29200000 out of 40000000 steps (73%).
04:57:18:WU03:FS03:0x15:Completed   8000000 out of 40000000 steps (20%).
04:57:54:WU00:FS02:0x15:Completed  28000000 out of 40000000 steps (70%).
04:58:11:WU01:FS01:0x15:Completed  29600000 out of 40000000 steps (74%).
05:01:30:WU06:FS00:0xa3:Completed 105000 out of 500000 steps  (21%)
05:01:45:WU00:FS02:0x15:Completed  28400000 out of 40000000 steps (71%).
05:02:04:WU01:FS01:0x15:Completed  30000000 out of 40000000 steps (75%).
05:05:35:WU00:FS02:0x15:Completed  28800000 out of 40000000 steps (72%).
05:05:57:WU01:FS01:0x15:Completed  30400000 out of 40000000 steps (76%).
05:09:26:WU00:FS02:0x15:Completed  29200000 out of 40000000 steps (73%).
05:09:49:WU01:FS01:0x15:Completed  30800000 out of 40000000 steps (77%).
05:12:01:WU06:FS00:0xa3:Completed 110000 out of 500000 steps  (22%)
05:13:16:WU00:FS02:0x15:Completed  29600000 out of 40000000 steps (74%).
05:13:32:WU03:FS03:0x15:Completed   8400000 out of 40000000 steps (21%).
05:13:40:WU01:FS01:0x15:Completed  31200000 out of 40000000 steps (78%).
05:17:07:WU00:FS02:0x15:Completed  30000000 out of 40000000 steps (75%).
05:17:30:WU01:FS01:0x15:Completed  31600000 out of 40000000 steps (79%).
05:20:57:WU00:FS02:0x15:Completed  30400000 out of 40000000 steps (76%).
05:21:21:WU01:FS01:0x15:Completed  32000000 out of 40000000 steps (80%).
05:22:30:WU06:FS00:0xa3:Completed 115000 out of 500000 steps  (23%)
05:24:48:WU00:FS02:0x15:Completed  30800000 out of 40000000 steps (77%).
05:25:18:WU01:FS01:0x15:Completed  32400000 out of 40000000 steps (81%).
05:28:39:WU00:FS02:0x15:Completed  31200000 out of 40000000 steps (78%).
05:29:09:WU01:FS01:0x15:Completed  32800000 out of 40000000 steps (82%).
05:29:50:WU03:FS03:0x15:Completed   8800000 out of 40000000 steps (22%).
******************************* Date: 2014-05-31 *******************************
05:32:29:WU00:FS02:0x15:Completed  31600000 out of 40000000 steps (79%).
05:33:00:WU06:FS00:0xa3:Completed 120000 out of 500000 steps  (24%)
05:33:02:WU01:FS01:0x15:Completed  33200000 out of 40000000 steps (83%).
05:36:20:WU00:FS02:0x15:Completed  32000000 out of 40000000 steps (80%).
05:36:52:WU01:FS01:0x15:Completed  33600000 out of 40000000 steps (84%).
05:40:11:WU00:FS02:0x15:Completed  32400000 out of 40000000 steps (81%).
05:40:43:WU01:FS01:0x15:Completed  34000000 out of 40000000 steps (85%).
05:43:29:WU06:FS00:0xa3:Completed 125000 out of 500000 steps  (25%)
05:44:01:WU00:FS02:0x15:Completed  32800000 out of 40000000 steps (82%).
05:44:34:WU01:FS01:0x15:Completed  34400000 out of 40000000 steps (86%).
05:46:09:WU03:FS03:0x15:Completed   9200000 out of 40000000 steps (23%).
05:47:52:WU00:FS02:0x15:Completed  33200000 out of 40000000 steps (83%).
05:48:26:WU01:FS01:0x15:Completed  34800000 out of 40000000 steps (87%).
05:51:42:WU00:FS02:0x15:Completed  33600000 out of 40000000 steps (84%).
05:52:16:WU01:FS01:0x15:Completed  35200000 out of 40000000 steps (88%).
05:53:58:WU06:FS00:0xa3:Completed 130000 out of 500000 steps  (26%)
05:55:33:WU00:FS02:0x15:Completed  34000000 out of 40000000 steps (85%).
05:56:06:WU01:FS01:0x15:Completed  35600000 out of 40000000 steps (89%).
05:59:24:WU00:FS02:0x15:Completed  34400000 out of 40000000 steps (86%).
05:59:58:WU01:FS01:0x15:Completed  36000000 out of 40000000 steps (90%).
06:02:28:WU03:FS03:0x15:Completed   9600000 out of 40000000 steps (24%).
06:03:14:WU00:FS02:0x15:Completed  34800000 out of 40000000 steps (87%).
06:03:49:WU01:FS01:0x15:Completed  36400000 out of 40000000 steps (91%).
06:04:27:WU06:FS00:0xa3:Completed 135000 out of 500000 steps  (27%)
06:06:47:FS00:Shutting core down
06:06:47:FS01:Shutting core down
06:06:47:FS02:Shutting core down
06:06:47:FS03:Shutting core down
06:06:51:WU00:FS02:0x15:Client no longer detected. Shutting down core 
06:06:51:WU00:FS02:0x15:
06:06:51:WU00:FS02:0x15:Folding@home Core Shutdown: CLIENT_DIED
06:06:51:WU03:FS03:0x15:Client no longer detected. Shutting down core 
06:06:51:WU03:FS03:0x15:
06:06:51:WU03:FS03:0x15:Folding@home Core Shutdown: CLIENT_DIED
06:06:51:WU01:FS01:0x15:Client no longer detected. Shutting down core 
06:06:51:WU01:FS01:0x15:
06:06:51:WU01:FS01:0x15:Folding@home Core Shutdown: CLIENT_DIED
06:06:51:WU06:FS00:0xa3:Client no longer detected. Shutting down core 
06:06:51:WU06:FS00:0xa3:
06:06:51:WU06:FS00:0xa3:Folding@home Core Shutdown: CLIENT_DIED
06:06:52:WU01:FS01:FahCore returned: INTERRUPTED (102 = 0x66)
06:06:52:WU03:FS03:FahCore returned: INTERRUPTED (102 = 0x66)
06:06:52:WU00:FS02:FahCore returned: INTERRUPTED (102 = 0x66)
06:06:52:WU06:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
06:10:09:WU01:FS01:Starting
06:10:09:WU01:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Dave/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 01 -suffix 01 -version 704 -lifeline 6808 -checkpoint 15 -gpu 0 -gpu-vendor nvidia
06:10:09:WU01:FS01:Started FahCore on PID 8992
06:10:09:WU01:FS01:Core PID:3308
06:10:09:WU01:FS01:FahCore 0x15 started
06:10:09:WU03:FS03:Starting
06:10:09:WU03:FS03:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Dave/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 03 -suffix 01 -version 704 -lifeline 6808 -checkpoint 15 -gpu 2 -gpu-vendor nvidia
06:10:09:WU03:FS03:Started FahCore on PID 6332
06:10:09:WU03:FS03:Core PID:2860
06:10:09:WU03:FS03:FahCore 0x15 started
06:10:09:WU00:FS02:Starting
06:10:09:WU00:FS02:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Dave/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 00 -suffix 01 -version 704 -lifeline 6808 -checkpoint 15 -gpu 1 -gpu-vendor nvidia
06:10:09:WU00:FS02:Started FahCore on PID 3104
06:10:09:WU00:FS02:Core PID:6000
06:10:09:WU00:FS02:FahCore 0x15 started
06:10:09:WU06:FS00:Starting
06:10:09:WU06:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Dave/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/Core_a3.fah/FahCore_a3.exe -dir 06 -suffix 01 -version 704 -lifeline 6808 -checkpoint 15 -np 5
06:10:09:WU06:FS00:Started FahCore on PID 3612
06:10:09:WU06:FS00:Core PID:3960
06:10:09:WU06:FS00:FahCore 0xa3 started
06:10:10:WU01:FS01:0x15:
06:10:10:WU01:FS01:0x15:*------------------------------*
06:10:10:WU01:FS01:0x15:Folding@Home GPU Core
06:10:10:WU01:FS01:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
06:10:10:WU01:FS01:0x15:Build host             AmoebaRemote
06:10:10:WU01:FS01:0x15:Board Type             NVIDIA/CUDA
06:10:10:WU01:FS01:0x15:Core                   15
06:10:10:WU01:FS01:0x15:
06:10:10:WU01:FS01:0x15:Window's signal control handler registered.
06:10:10:WU01:FS01:0x15:Preparing to commence simulation
06:10:10:WU01:FS01:0x15:- Looking at optimizations...
06:10:10:WU01:FS01:0x15:- Files status OK
06:10:10:WU01:FS01:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
06:10:10:WU01:FS01:0x15:- Expanded 125034 -> 501826 (decompressed 401.3 percent)
06:10:10:WU01:FS01:0x15:Called DecompressByteArray: compressed_data_size=125034 data_size=501826, decompressed_data_size=501826 diff=0
06:10:10:WU01:FS01:0x15:- Digital signature verified
06:10:10:WU01:FS01:0x15:
06:10:10:WU01:FS01:0x15:Project: 7622 (Run 198, Clone 0, Gen 70)
06:10:10:WU01:FS01:0x15:
06:10:10:WU01:FS01:0x15:Assembly optimizations on if available.
06:10:10:WU01:FS01:0x15:Entering M.D.
06:10:10:WU03:FS03:0x15:
06:10:10:WU03:FS03:0x15:*------------------------------*
06:10:10:WU03:FS03:0x15:Folding@Home GPU Core
06:10:10:WU03:FS03:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
06:10:10:WU03:FS03:0x15:Build host             AmoebaRemote
06:10:10:WU03:FS03:0x15:Board Type             NVIDIA/CUDA
06:10:10:WU03:FS03:0x15:Core                   15
06:10:10:WU03:FS03:0x15:GPU device info vendor=0 device=0 name=NA match=0 deviceId=2
06:10:10:WU03:FS03:0x15:
06:10:10:WU03:FS03:0x15:Window's signal control handler registered.
06:10:10:WU03:FS03:0x15:Preparing to commence simulation
06:10:10:WU03:FS03:0x15:- Looking at optimizations...
06:10:10:WU03:FS03:0x15:- Files status OK
06:10:10:WU03:FS03:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
06:10:10:WU03:FS03:0x15:- Expanded 124351 -> 501826 (decompressed 403.5 percent)
06:10:10:WU03:FS03:0x15:Called DecompressByteArray: compressed_data_size=124351 data_size=501826, decompressed_data_size=501826 diff=0
06:10:10:WU03:FS03:0x15:- Digital signature verified
06:10:10:WU03:FS03:0x15:
06:10:10:WU03:FS03:0x15:Project: 7623 (Run 261, Clone 0, Gen 166)
06:10:10:WU03:FS03:0x15:
06:10:10:WU03:FS03:0x15:Assembly optimizations on if available.
06:10:10:WU03:FS03:0x15:Entering M.D.
06:10:10:WU00:FS02:0x15:
06:10:10:WU00:FS02:0x15:*------------------------------*
06:10:10:WU00:FS02:0x15:Folding@Home GPU Core
06:10:10:WU00:FS02:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
06:10:10:WU00:FS02:0x15:Build host             AmoebaRemote
06:10:10:WU00:FS02:0x15:Board Type             NVIDIA/CUDA
06:10:10:WU00:FS02:0x15:Core                   15
06:10:10:WU00:FS02:0x15:GPU device info vendor=0 device=0 name=NA match=0 deviceId=1
06:10:10:WU00:FS02:0x15:
06:10:10:WU00:FS02:0x15:Window's signal control handler registered.
06:10:10:WU00:FS02:0x15:Preparing to commence simulation
06:10:10:WU00:FS02:0x15:- Looking at optimizations...
06:10:10:WU00:FS02:0x15:- Files status OK
06:10:10:WU00:FS02:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
06:10:10:WU00:FS02:0x15:- Expanded 124550 -> 501826 (decompressed 402.9 percent)
06:10:10:WU00:FS02:0x15:Called DecompressByteArray: compressed_data_size=124550 data_size=501826, decompressed_data_size=501826 diff=0
06:10:10:WU00:FS02:0x15:- Digital signature verified
06:10:10:WU00:FS02:0x15:
06:10:10:WU00:FS02:0x15:Project: 7623 (Run 512, Clone 0, Gen 160)
06:10:10:WU00:FS02:0x15:
06:10:10:WU00:FS02:0x15:Assembly optimizations on if available.
06:10:10:WU00:FS02:0x15:Entering M.D.
06:10:10:WU06:FS00:0xa3:
06:10:10:WU06:FS00:0xa3:*------------------------------*
06:10:10:WU06:FS00:0xa3:Folding@Home Gromacs SMP Core
06:10:10:WU06:FS00:0xa3:Version 2.27 (Dec. 15, 2010)
06:10:10:WU06:FS00:0xa3:
06:10:10:WU06:FS00:0xa3:Preparing to commence simulation
06:10:10:WU06:FS00:0xa3:- Looking at optimizations...
06:10:10:WU06:FS00:0xa3:- Files status OK
06:10:10:WU06:FS00:0xa3:- Expanded 3847704 -> 4384444 (decompressed 113.9 percent)
06:10:10:WU06:FS00:0xa3:Called DecompressByteArray: compressed_data_size=3847704 data_size=4384444, decompressed_data_size=4384444 diff=0
06:10:10:WU06:FS00:0xa3:- Digital signature verified
06:10:10:WU06:FS00:0xa3:
06:10:10:WU06:FS00:0xa3:Project: 8576 (Run 1, Clone 0, Gen 402)
06:10:10:WU06:FS00:0xa3:
06:10:10:WU06:FS00:0xa3:Assembly optimizations on if available.
06:10:10:WU06:FS00:0xa3:Entering M.D.
06:10:11:WU01:FS01:0x15:Will resume from checkpoint file 01/wudata_01.ckp
06:10:11:WU01:FS01:0x15:Tpr hash 01/wudata_01.tpr:  3336875573 1615538842 2324230187 4260796349 2057047944
06:10:11:WU01:FS01:0x15:GPU device id=0
06:10:11:WU01:FS01:0x15:Working on Protein
06:10:11:WU01:FS01:0x15:Client config unavailable.
06:10:11:WU03:FS03:0x15:Will resume from checkpoint file 03/wudata_01.ckp
06:10:11:WU03:FS03:0x15:Tpr hash 03/wudata_01.tpr:  310465715 1833047723 3414748523 2877205478 1558103798
06:10:11:WU03:FS03:0x15:GPU device id=2
06:10:11:WU03:FS03:0x15:Working on Protein
06:10:11:WU03:FS03:0x15:Client config unavailable.
06:10:11:WU00:FS02:0x15:Will resume from checkpoint file 00/wudata_01.ckp
06:10:11:WU00:FS02:0x15:Tpr hash 00/wudata_01.tpr:  3454981011 1059706818 221401786 1919140222 3087881404
06:10:11:WU00:FS02:0x15:GPU device id=1
06:10:11:WU00:FS02:0x15:Working on Protein
06:10:11:WU00:FS02:0x15:Client config unavailable.
06:10:12:WU01:FS01:0x15:Starting GUI Server
06:10:12:WU03:FS03:0x15:Starting GUI Server
06:10:12:WU00:FS02:0x15:Starting GUI Server
06:10:16:WU06:FS00:0xa3:Using Gromacs checkpoints
06:10:16:WU06:FS00:0xa3:Mapping NT from 5 to 5 
06:10:16:WU06:FS00:0xa3:Resuming from checkpoint
06:10:16:WU06:FS00:0xa3:Verified 06/wudata_01.log
06:10:16:WU06:FS00:0xa3:Verified 06/wudata_01.trr
06:10:16:WU06:FS00:0xa3:Verified 06/wudata_01.edr
06:10:16:WU06:FS00:0xa3:Completed 130715 out of 500000 steps  (26%)
06:10:34:Saving configuration to config.xml
06:10:34:<config>
06:10:34:  <!-- Network -->
06:10:34:  <proxy v=':8080'/>
06:10:34:
06:10:34:  <!-- Slot Control -->
06:10:34:  <power v='FULL'/>
06:10:34:
06:10:34:  <!-- User Information -->
06:10:34:  <passkey v='********************************'/>
06:10:34:  <team v='225663'/>
06:10:34:  <user v='Volnaiskra'/>
06:10:34:
06:10:34:  <!-- Work Unit Control -->
06:10:34:  <max-queue v='64'/>
06:10:34:
06:10:34:  <!-- Folding Slots -->
06:10:34:  <slot id='0' type='CPU'/>
06:10:34:  <slot id='1' type='GPU'/>
06:10:34:  <slot id='2' type='GPU'/>
06:10:34:  <slot id='3' type='GPU'/>
06:10:34:</config>
06:11:13:WARNING:WU03:FS03:Detected clock skew (1 mins 04 secs), adjusting time estimates
06:11:13:WU03:FS03:0x15:Resuming from checkpoint
06:11:13:WU03:FS03:0x15:fcCheckPointResume: retreived and current tpr file hash:
06:11:13:WU03:FS03:0x15:   0    310465715    310465715
06:11:13:WU03:FS03:0x15:   1   1833047723   1833047723
06:11:13:WU03:FS03:0x15:   2   3414748523   3414748523
06:11:13:WU03:FS03:0x15:   3   2877205478   2877205478
06:11:13:WU03:FS03:0x15:   4   1558103798   1558103798
06:11:13:WU03:FS03:0x15:fcCheckPointResume: file hashes same.
06:11:13:WU03:FS03:0x15:fcCheckPointResume: state restored.
06:11:13:WU03:FS03:0x15:fcCheckPointResume: name 03/wudata_01.log Verified 03/wudata_01.log
06:11:13:WU03:FS03:0x15:fcCheckPointResume: name 03/wudata_01.trr Verified 03/wudata_01.trr
06:11:13:WU03:FS03:0x15:fcCheckPointResume: name 03/wudata_01.xtc Verified 03/wudata_01.xtc
06:11:13:WU03:FS03:0x15:fcCheckPointResume: name 03/wudata_01.edr Verified 03/wudata_01.edr
06:11:13:WU03:FS03:0x15:fcCheckPointResume: state restored 2
06:11:13:WU03:FS03:0x15:Resumed from checkpoint
06:11:13:WU03:FS03:0x15:Setting checkpoint frequency: 400000
06:11:13:WU03:FS03:0x15:Completed   9600001 out of 40000000 steps (24%).
06:11:14:WU01:FS01:0x15:Resuming from checkpoint
06:11:14:WU01:FS01:0x15:fcCheckPointResume: retreived and current tpr file hash:
06:11:14:WU01:FS01:0x15:   0   3336875573   3336875573
06:11:14:WU01:FS01:0x15:   1   1615538842   1615538842
06:11:14:WU01:FS01:0x15:   2   2324230187   2324230187
06:11:14:WU01:FS01:0x15:   3   4260796349   4260796349
06:11:14:WU01:FS01:0x15:   4   2057047944   2057047944
06:11:14:WU01:FS01:0x15:fcCheckPointResume: file hashes same.
06:11:14:WU01:FS01:0x15:fcCheckPointResume: state restored.
06:11:14:WU01:FS01:0x15:fcCheckPointResume: name 01/wudata_01.log Verified 01/wudata_01.log
06:11:14:WU01:FS01:0x15:fcCheckPointResume: name 01/wudata_01.trr Verified 01/wudata_01.trr
06:11:14:WU01:FS01:0x15:fcCheckPointResume: name 01/wudata_01.xtc Verified 01/wudata_01.xtc
06:11:14:WARNING:WU01:FS01:Detected clock skew (1 mins 05 secs), adjusting time estimates
06:11:14:WU01:FS01:0x15:fcCheckPointResume: name 01/wudata_01.edr Verified 01/wudata_01.edr
06:11:14:WARNING:WU00:FS02:Detected clock skew (1 mins 05 secs), adjusting time estimates
06:11:14:WU01:FS01:0x15:fcCheckPointResume: state restored 2
06:11:14:WU01:FS01:0x15:Resumed from checkpoint
06:11:14:WU01:FS01:0x15:Setting checkpoint frequency: 400000
06:11:14:WU01:FS01:0x15:Completed  36400001 out of 40000000 steps (91%).
06:11:14:WU00:FS02:0x15:Resuming from checkpoint
06:11:14:WU00:FS02:0x15:fcCheckPointResume: retreived and current tpr file hash:
06:11:14:WU00:FS02:0x15:   0   3454981011   3454981011
06:11:14:WU00:FS02:0x15:   1   1059706818   1059706818
06:11:14:WU00:FS02:0x15:   2    221401786    221401786
06:11:14:WU00:FS02:0x15:   3   1919140222   1919140222
06:11:14:WU00:FS02:0x15:   4   3087881404   3087881404
06:11:14:WU00:FS02:0x15:fcCheckPointResume: file hashes same.
06:11:14:WU00:FS02:0x15:fcCheckPointResume: state restored.
06:11:14:WU00:FS02:0x15:fcCheckPointResume: name 00/wudata_01.log Verified 00/wudata_01.log
06:11:14:WU00:FS02:0x15:fcCheckPointResume: name 00/wudata_01.trr Verified 00/wudata_01.trr
06:11:14:WU00:FS02:0x15:fcCheckPointResume: name 00/wudata_01.xtc Verified 00/wudata_01.xtc
06:11:14:WU00:FS02:0x15:fcCheckPointResume: name 00/wudata_01.edr Verified 00/wudata_01.edr
06:11:14:WU00:FS02:0x15:fcCheckPointResume: state restored 2
Top
Volnaiskra
Posts: 57
Joined: Tue Nov 26, 2013 9:02 am

Re: work units not sending. Also, idle not working properly

Post by Volnaiskra »

Code: Select all

06:11:14:WU00:FS02:0x15:Resumed from checkpoint
06:11:14:WU00:FS02:0x15:Setting checkpoint frequency: 400000
06:11:14:WU00:FS02:0x15:Completed  34800001 out of 40000000 steps (87%).
06:15:04:WU01:FS01:0x15:Completed  36800000 out of 40000000 steps (92%).
06:15:04:WU00:FS02:0x15:Completed  35200000 out of 40000000 steps (88%).
06:18:55:WU01:FS01:0x15:Completed  37200000 out of 40000000 steps (93%).
06:18:56:WU00:FS02:0x15:Completed  35600000 out of 40000000 steps (89%).
06:19:12:WU06:FS00:0xa3:Completed 135000 out of 500000 steps  (27%)
06:22:45:WU01:FS01:0x15:Completed  37600000 out of 40000000 steps (94%).
06:22:47:WU00:FS02:0x15:Completed  36000000 out of 40000000 steps (90%).
06:25:14:WU03:FS03:0x15:Completed  10000000 out of 40000000 steps (25%).
06:26:35:WU01:FS01:0x15:Completed  38000000 out of 40000000 steps (95%).
06:26:37:WU00:FS02:0x15:Completed  36400000 out of 40000000 steps (91%).
06:29:40:WU06:FS00:0xa3:Completed 140000 out of 500000 steps  (28%)
06:30:26:WU01:FS01:0x15:Completed  38400000 out of 40000000 steps (96%).
06:30:28:WU00:FS02:0x15:Completed  36800000 out of 40000000 steps (92%).
06:34:17:WU01:FS01:0x15:Completed  38800000 out of 40000000 steps (97%).
06:34:18:WU00:FS02:0x15:Completed  37200000 out of 40000000 steps (93%).
06:38:08:WU01:FS01:0x15:Completed  39200000 out of 40000000 steps (98%).
06:38:09:WU00:FS02:0x15:Completed  37600000 out of 40000000 steps (94%).
06:40:09:WU06:FS00:0xa3:Completed 145000 out of 500000 steps  (29%)
06:41:20:WU03:FS03:0x15:Completed  10400000 out of 40000000 steps (26%).
06:41:59:WU00:FS02:0x15:Completed  38000000 out of 40000000 steps (95%).
06:42:00:WU01:FS01:0x15:Completed  39600000 out of 40000000 steps (99%).
06:42:01:WU05:FS01:Connecting to 171.67.108.201:80
06:42:02:WU05:FS01:Assigned to work server 171.64.65.105
06:42:02:WU05:FS01:Requesting new work unit for slot 01: RUNNING gpu:0:GK110 [GeForce GTX Titan] from 171.64.65.105
06:42:02:WU05:FS01:Connecting to 171.64.65.105:8080
06:42:03:WU05:FS01:Downloading 121.64KiB
06:42:05:WU05:FS01:Download complete
06:42:05:WU05:FS01:Received Unit: id:05 state:DOWNLOAD error:NO_ERROR project:7620 run:370 clone:0 gen:234 core:0x15 unit:0x00000116664f2dd14e42f584391bf76e
06:45:50:WU00:FS02:0x15:Completed  38400000 out of 40000000 steps (96%).
06:45:53:WU01:FS01:0x15:Completed  40000000 out of 40000000 steps (100%).
06:45:53:WU01:FS01:0x15:Finished fah_main status=0
06:45:53:WU01:FS01:0x15:Successful run
06:45:53:WU01:FS01:0x15:DynamicWrapper: Finished Work Unit: sleep=10000
06:46:03:WU01:FS01:0x15:Reserved 766400 bytes for xtc file; Cosm status=0
06:46:03:WU01:FS01:0x15:Allocated 766400 bytes for xtc file
06:46:03:WU01:FS01:0x15:- Reading up to 766400 from "01/wudata_01.xtc": Read 766400
06:46:03:WU01:FS01:0x15:Read 766400 bytes from xtc file; available packet space=785664064
06:46:03:WU01:FS01:0x15:xtc file hash check passed.
06:46:03:WU01:FS01:0x15:Reserved 47688 47688 785664064 bytes for arc file=<01/wudata_01.trr> Cosm status=0
06:46:03:WU01:FS01:0x15:Allocated 47688 bytes for arc file
06:46:03:WU01:FS01:0x15:- Reading up to 47688 from "01/wudata_01.trr": Read 47688
06:46:03:WU01:FS01:0x15:Read 47688 bytes from arc file; available packet space=785616376
06:46:03:WU01:FS01:0x15:trr file hash check passed.
06:46:03:WU01:FS01:0x15:Allocated 544 bytes for edr file
06:46:03:WU01:FS01:0x15:Read bedfile
06:46:03:WU01:FS01:0x15:edr file hash check passed.
06:46:03:WU01:FS01:0x15:Allocated 37157 bytes for logfile
06:46:03:WU01:FS01:0x15:Read logfile
06:46:03:WU01:FS01:0x15:GuardedRun: success in DynamicWrapper
06:46:03:WU01:FS01:0x15:GuardedRun: done
06:46:03:WU01:FS01:0x15:Run: GuardedRun completed.
06:46:05:WU01:FS01:0x15:+ Opened results file
06:46:05:WU01:FS01:0x15:- Writing 852301 bytes of core data to disk...
06:46:05:WU01:FS01:0x15:Done: 851789 -> 823397 (compressed to 96.6 percent)
06:46:05:WU01:FS01:0x15:  ... Done.
06:46:05:WU01:FS01:0x15:DeleteFrameFiles: successfully deleted file=01/wudata_01.ckp
06:46:05:WU01:FS01:0x15:Shutting down core 
06:46:05:WU01:FS01:0x15:
06:46:05:WU01:FS01:0x15:Folding@home Core Shutdown: FINISHED_UNIT
06:46:05:WU01:FS01:FahCore returned: FINISHED_UNIT (100 = 0x64)
06:46:05:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:7622 run:198 clone:0 gen:70 core:0x15 unit:0x0000006a664f2dd14edd57c7e7a3204b
06:46:05:WU01:FS01:Uploading 804.60KiB to 171.64.65.105
06:46:05:WU05:FS01:Starting
06:46:05:WU01:FS01:Connecting to 171.64.65.105:8080
06:46:05:WU05:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Dave/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 05 -suffix 01 -version 704 -lifeline 6808 -checkpoint 15 -gpu 0 -gpu-vendor nvidia
06:46:05:WU05:FS01:Started FahCore on PID 6016
06:46:05:WU05:FS01:Core PID:5208
06:46:05:WU05:FS01:FahCore 0x15 started
06:46:06:WU05:FS01:0x15:
06:46:06:WU05:FS01:0x15:*------------------------------*
06:46:06:WU05:FS01:0x15:Folding@Home GPU Core
06:46:06:WU05:FS01:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
06:46:06:WU05:FS01:0x15:Build host             AmoebaRemote
06:46:06:WU05:FS01:0x15:Board Type             NVIDIA/CUDA
06:46:06:WU05:FS01:0x15:Core                   15
06:46:06:WU05:FS01:0x15:
06:46:06:WU05:FS01:0x15:Window's signal control handler registered.
06:46:06:WU05:FS01:0x15:Preparing to commence simulation
06:46:06:WU05:FS01:0x15:- Looking at optimizations...
06:46:06:WU05:FS01:0x15:DeleteFrameFiles: successfully deleted file=05/wudata_01.ckp
06:46:06:WU05:FS01:0x15:- Created dyn
06:46:06:WU05:FS01:0x15:- Files status OK
06:46:06:WU05:FS01:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
06:46:06:WU05:FS01:0x15:- Expanded 124046 -> 501826 (decompressed 404.5 percent)
06:46:06:WU05:FS01:0x15:Called DecompressByteArray: compressed_data_size=124046 data_size=501826, decompressed_data_size=501826 diff=0
06:46:06:WU05:FS01:0x15:- Digital signature verified
06:46:06:WU05:FS01:0x15:
06:46:06:WU05:FS01:0x15:Project: 7620 (Run 370, Clone 0, Gen 234)
06:46:06:WU05:FS01:0x15:
06:46:06:WU05:FS01:0x15:Assembly optimizations on if available.
06:46:06:WU05:FS01:0x15:Entering M.D.
06:46:08:WU05:FS01:0x15:Tpr hash 05/wudata_01.tpr:  2370305137 1104168512 3108158518 3895084807 2366664470
06:46:08:WU05:FS01:0x15:GPU device id=0
06:46:08:WU05:FS01:0x15:Working on Protein
06:46:08:WU05:FS01:0x15:Client config unavailable.
06:46:08:WU05:FS01:0x15:Starting GUI Server
06:46:11:WU01:FS01:Upload 47.73%
06:46:18:WU01:FS01:Upload complete
06:46:18:WU01:FS01:Server responded WORK_ACK (400)
06:46:18:WU01:FS01:Final credit estimate, 14093.00 points
06:46:18:WU01:FS01:Cleaning up
06:47:09:WU05:FS01:0x15:Setting checkpoint frequency: 400000
06:47:09:WU05:FS01:0x15:Completed         3 out of 40000000 steps (0%).
06:49:40:WU00:FS02:0x15:Completed  38800000 out of 40000000 steps (97%).
06:50:37:WU06:FS00:0xa3:Completed 150000 out of 500000 steps  (30%)
06:51:00:WU05:FS01:0x15:Completed    400000 out of 40000000 steps (1%).
06:53:31:WU00:FS02:0x15:Completed  39200000 out of 40000000 steps (98%).
06:54:54:WU05:FS01:0x15:Completed    800000 out of 40000000 steps (2%).
06:57:12:WU03:FS03:0x15:Completed  10800000 out of 40000000 steps (27%).
06:57:21:WU00:FS02:0x15:Completed  39600000 out of 40000000 steps (99%).
06:57:22:WU01:FS02:Connecting to 171.67.108.201:80
06:57:23:WU01:FS02:Assigned to work server 171.64.65.105
06:57:23:WU01:FS02:Requesting new work unit for slot 02: RUNNING gpu:1:GK110 [GeForce GTX Titan] from 171.64.65.105
06:57:23:WU01:FS02:Connecting to 171.64.65.105:8080
06:57:24:WU01:FS02:Downloading 121.82KiB
06:57:25:WU01:FS02:Download complete
06:57:26:WU01:FS02:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:7620 run:239 clone:0 gen:191 core:0x15 unit:0x000000ef664f2dd14e42f4e006bd4907
06:58:44:WU05:FS01:0x15:Completed   1200000 out of 40000000 steps (3%).
07:01:06:WU06:FS00:0xa3:Completed 155000 out of 500000 steps  (31%)
07:01:12:WU00:FS02:0x15:Completed  40000000 out of 40000000 steps (100%).
07:01:12:WU00:FS02:0x15:Finished fah_main status=0
07:01:12:WU00:FS02:0x15:Successful run
07:01:12:WU00:FS02:0x15:DynamicWrapper: Finished Work Unit: sleep=10000
07:01:22:WU00:FS02:0x15:Reserved 760176 bytes for xtc file; Cosm status=0
07:01:22:WU00:FS02:0x15:Allocated 760176 bytes for xtc file
07:01:22:WU00:FS02:0x15:- Reading up to 760176 from "00/wudata_01.xtc": Read 760176
07:01:22:WU00:FS02:0x15:Read 760176 bytes from xtc file; available packet space=785670288
07:01:22:WU00:FS02:0x15:xtc file hash check passed.
07:01:22:WU00:FS02:0x15:Reserved 47688 47688 785670288 bytes for arc file=<00/wudata_01.trr> Cosm status=0
07:01:22:WU00:FS02:0x15:Allocated 47688 bytes for arc file
07:01:22:WU00:FS02:0x15:- Reading up to 47688 from "00/wudata_01.trr": Read 47688
07:01:22:WU00:FS02:0x15:Read 47688 bytes from arc file; available packet space=785622600
07:01:22:WU00:FS02:0x15:trr file hash check passed.
07:01:22:WU00:FS02:0x15:Allocated 544 bytes for edr file
07:01:22:WU00:FS02:0x15:Read bedfile
07:01:22:WU00:FS02:0x15:edr file hash check passed.
07:01:22:WU00:FS02:0x15:Allocated 37157 bytes for logfile
07:01:22:WU00:FS02:0x15:Read logfile
07:01:22:WU00:FS02:0x15:GuardedRun: success in DynamicWrapper
07:01:22:WU00:FS02:0x15:GuardedRun: done
07:01:22:WU00:FS02:0x15:Run: GuardedRun completed.
07:01:25:WU00:FS02:0x15:+ Opened results file
07:01:25:WU00:FS02:0x15:- Writing 846077 bytes of core data to disk...
07:01:25:WU00:FS02:0x15:Done: 845565 -> 817168 (compressed to 96.6 percent)
07:01:25:WU00:FS02:0x15:  ... Done.
07:01:25:WU00:FS02:0x15:DeleteFrameFiles: successfully deleted file=00/wudata_01.ckp
07:01:25:WU00:FS02:0x15:Shutting down core 
07:01:25:WU00:FS02:0x15:
07:01:25:WU00:FS02:0x15:Folding@home Core Shutdown: FINISHED_UNIT
07:01:25:WU00:FS02:FahCore returned: FINISHED_UNIT (100 = 0x64)
07:01:25:WU00:FS02:Sending unit results: id:00 state:SEND error:NO_ERROR project:7623 run:512 clone:0 gen:160 core:0x15 unit:0x000000f8664f2dd14fe4fb5209f345e9
07:01:25:WU00:FS02:Uploading 798.52KiB to 171.64.65.105
07:01:25:WU00:FS02:Connecting to 171.64.65.105:8080
07:01:25:WU01:FS02:Starting
07:01:25:WU01:FS02:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Dave/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 01 -suffix 01 -version 704 -lifeline 6808 -checkpoint 15 -gpu 1 -gpu-vendor nvidia
07:01:25:WU01:FS02:Started FahCore on PID 4064
07:01:25:WU01:FS02:Core PID:5948
07:01:25:WU01:FS02:FahCore 0x15 started
07:01:26:WU01:FS02:0x15:
07:01:26:WU01:FS02:0x15:*------------------------------*
07:01:26:WU01:FS02:0x15:Folding@Home GPU Core
07:01:26:WU01:FS02:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
07:01:26:WU01:FS02:0x15:Build host             AmoebaRemote
07:01:26:WU01:FS02:0x15:Board Type             NVIDIA/CUDA
07:01:26:WU01:FS02:0x15:Core                   15
07:01:26:WU01:FS02:0x15:GPU device info vendor=0 device=0 name=NA match=0 deviceId=1
07:01:26:WU01:FS02:0x15:
07:01:26:WU01:FS02:0x15:Window's signal control handler registered.
07:01:26:WU01:FS02:0x15:Preparing to commence simulation
07:01:26:WU01:FS02:0x15:- Looking at optimizations...
07:01:26:WU01:FS02:0x15:DeleteFrameFiles: successfully deleted file=01/wudata_01.ckp
07:01:26:WU01:FS02:0x15:- Created dyn
07:01:26:WU01:FS02:0x15:- Files status OK
07:01:26:WU01:FS02:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
07:01:26:WU01:FS02:0x15:- Expanded 124230 -> 501826 (decompressed 403.9 percent)
07:01:26:WU01:FS02:0x15:Called DecompressByteArray: compressed_data_size=124230 data_size=501826, decompressed_data_size=501826 diff=0
07:01:26:WU01:FS02:0x15:- Digital signature verified
07:01:26:WU01:FS02:0x15:
07:01:26:WU01:FS02:0x15:Project: 7620 (Run 239, Clone 0, Gen 191)
07:01:26:WU01:FS02:0x15:
07:01:26:WU01:FS02:0x15:Assembly optimizations on if available.
07:01:26:WU01:FS02:0x15:Entering M.D.
07:01:27:WU01:FS02:0x15:Tpr hash 01/wudata_01.tpr:  3891773270 2367481901 213262063 416695753 4089469650
07:01:27:WU01:FS02:0x15:GPU device id=1
07:01:27:WU01:FS02:0x15:Working on Protein
07:01:27:WU01:FS02:0x15:Client config unavailable.
07:01:28:WU01:FS02:0x15:Starting GUI Server
07:01:31:WU00:FS02:Upload 56.10%
07:01:38:WU00:FS02:Upload complete
07:01:38:WU00:FS02:Server responded WORK_ACK (400)
07:01:38:WU00:FS02:Final credit estimate, 14093.00 points
07:01:38:WU00:FS02:Cleaning up
Top
P5-133XL
Posts: 2948
Joined: Sun Dec 02, 2007 4:36 am
Hardware configuration: Machine #1:

Intel Q9450; 2x2GB=8GB Ram; Gigabyte GA-X48-DS4 Motherboard; PC Power and Cooling Q750 PS; 2x GTX 460; Windows Server 2008 X64 (SP1).

Machine #2:

Intel Q6600; 2x2GB=4GB Ram; Gigabyte GA-X48-DS4 Motherboard; PC Power and Cooling Q750 PS; 2x GTX 460 video card; Windows 7 X64.

Machine 3:

Dell Dimension 8400, 3.2GHz P4 4x512GB Ram, Video card GTX 460, Windows 7 X32

I am currently folding just on the 5x GTX 460's for aprox. 70K PPD
Location: Salem. OR USA

Re: work units not sending. Also, idle not working properly

Post by P5-133XL »

Everything in those logs was received...

Hi Volnaiskra (team 225663),
Your WU (P9408 R1627 C0 G13) was added to the stats database on 2014-05-30 04:04:43 for 60619 points of credit.

Hi Volnaiskra (team 225663),
Your WU (P9500 R1292 C1 G28) was added to the stats database on 2014-05-30 17:09:19 for 754.073 points of credit.

Hi Volnaiskra (team 225663),
Your WU (P9408 R1623 C0 G24) was added to the stats database on 2014-05-30 18:04:12 for 55991 points of credit.

Hi Volnaiskra (team 225663),
Your WU (P9500 R918 C0 G60) was added to the stats database on 2014-05-30 19:09:40 for 852.786 points of credit.

Hi Volnaiskra (team 225663),
Your WU (P7622 R198 C0 G70) was added to the stats database on 2014-05-31 00:18:07 for 14093 points of credit.

Hi Volnaiskra (team 225663),
Your WU (P7623 R512 C0 G160) was added to the stats database on 2014-05-31 00:18:07 for 14093 points of credit.
Image
Top
Rel25917
Posts: 303
Joined: Wed Aug 15, 2012 2:31 am

Re: work units not sending. Also, idle not working properly

Post by Rel25917 »

Volnaiskra wrote:
Does that suggest that the ones that are failing are much larger? Or they are from a different EXE process that uploads them slower? If they're larger, then I guess the size could be the problem (though 5 minutes to upload a WU hardly seems like it should cause problems), or perhaps larger ones have some other shared characteristic other than size - for example, maybe they're more complex and harder to verify on the other end, or they're all from a particular project?
They are larger, you can see the size of the unit being sent when it starts the upload.

00:15:13:WU04:FS02:Sending unit results: id:04 state:SEND error:NO_ERROR project:9408 run:1623 clone:0 gen:24 core:0x17 unit:0x000000250a3b1e5c53432c4f098b6279
00:15:13:WU04:FS02:Uploading 20.32MiB to 171.64.65.56

01:08:55:WU05:FS00:Sending unit results: id:05 state:SEND error:NO_ERROR project:9500 run:918 clone:0 gen:60 core:0xa4 unit:0x000000436652edcc53642cabb29c9b27
01:08:55:WU05:FS00:Uploading 1.14MiB to 171.67.108.60

All the uploading and downloading is done by fahclient.
Top
Volnaiskra
Posts: 57
Joined: Tue Nov 26, 2013 9:02 am

Re: work units not sending. Also, idle not working properly

Post by Volnaiskra »

Thanks P5-133XL

For those big 20MB ones, am I right in assuming that 5 minutes per upload is reasonable? And that it wouldn't indicate that a timeout has occured?
Top
P5-133XL
Posts: 2948
Joined: Sun Dec 02, 2007 4:36 am
Hardware configuration: Machine #1:

Intel Q9450; 2x2GB=8GB Ram; Gigabyte GA-X48-DS4 Motherboard; PC Power and Cooling Q750 PS; 2x GTX 460; Windows Server 2008 X64 (SP1).

Machine #2:

Intel Q6600; 2x2GB=4GB Ram; Gigabyte GA-X48-DS4 Motherboard; PC Power and Cooling Q750 PS; 2x GTX 460 video card; Windows 7 X64.

Machine 3:

Dell Dimension 8400, 3.2GHz P4 4x512GB Ram, Video card GTX 460, Windows 7 X32

I am currently folding just on the 5x GTX 460's for aprox. 70K PPD
Location: Salem. OR USA

Re: work units not sending. Also, idle not working properly

Post by P5-133XL »

If I remember correctly the default server upload time-out is 15 minutes.

There are connection extra client options that you can play with in the client but there is no warranty given. It would be much better if you could figure out and correct your connection problem, than playing with these variables. Playing with these variables can result in a non-functioning client! They are not commonly fiddled with and they are not well documented so I can't predict what is going to happen. If you come here with problems relating to these, few are going to be able to be able to help and everyone is going to assume that they are at default values unless you explicitly tell them. The most likely reaction that someone will have, if you do tell them that they were changed is that they will request that you put them back to the default state. i.e. you are on your own...

connection-timeout (default value = 60)
max-connect-time (default value = 900)
min-connect-time (default value = 300)

I'm assuming the values are in seconds but I do not know.
Image
Top
PantherX
Site Moderator
Posts: 6986
Joined: Wed Dec 23, 2009 9:33 am
Hardware configuration: V7.6.21 -> Multi-purpose 24/7
Windows 10 64-bit
CPU:2/3/4/6 -> Intel i7-6700K
GPU:1 -> Nvidia GTX 1080 Ti
§
Retired:
2x Nvidia GTX 1070
Nvidia GTX 675M
Nvidia GTX 660 Ti
Nvidia GTX 650 SC
Nvidia GTX 260 896 MB SOC
Nvidia 9600GT 1 GB OC
Nvidia 9500M GS
Nvidia 8800GTS 320 MB

Intel Core i7-860
Intel Core i7-3840QM
Intel i3-3240
Intel Core 2 Duo E8200
Intel Core 2 Duo E6550
Intel Core 2 Duo T8300
Intel Pentium E5500
Intel Pentium E5400
Location: Land Of The Long White Cloud
Contact:
Contact PantherX

Re: work units not sending. Also, idle not working properly

Post by PantherX »

Here is a brief explanation of the settings that P5-133XL has mentioned:

Code: Select all

  connection-timeout <integer=60>
    The maximum amount of time, in seconds, a connection can be idle before
    being dropped.

  max-connect-time <integer=900>
    The maximum amount of time, in seconds, a client can be connected to the
    server.

  min-connect-time <integer=300>
    The minimum amount of time, in seconds, a client must be connected to the
    server before it can be dropped in favor or a new connection.
Additional entries can be viewed in this post (viewtopic.php?p=261088&f=61#p261088).
ETA:
Now ↞ Very Soon ↔ Soon ↔ Soon-ish ↔ Not Soon ↠ End Of Time

Welcome To The F@H Support Forum Ӂ Troubleshooting Bad WUs Ӂ Troubleshooting Server Connectivity Issues
Top
Volnaiskra
Posts: 57
Joined: Tue Nov 26, 2013 9:02 am

Re: work units not sending. Also, idle not working properly

Post by Volnaiskra »

P5-133XL wrote:It would be much better if you could figure out and correct your connection problem, than playing with these variables.
This begs the question: What connection problem?

-We can see that FAH can connect fine, with most of my WUs receiving ACKs.
-The large 20mb WUs take only 5 minutes to upload, which is well under the 15min limit.
-The logs show a steady progression in the uploads from 0% to 99-100%, without breaks or aborts.
-We can see that even those 'failed' uploads are actually successful, since I'm getting the points.
-Every other program on my system has no problems connecting, downloading, uploading, or streaming.

Surely the evidence points to "bad connection" not being the problem. Either the problem is something else, or we're looking at the wrong evidence. Yet again, this problem seems to defy diagnosis or understanding.
PantherX wrote:

Code: Select all

  min-connect-time <integer=300>
    The minimum amount of time, in seconds, a client must be connected to the
    server before it can be dropped in favor or a new connection.
The fact that this limit is 5 minutes, and both the above 'failed' uploads took about 5 minutes each, piques my interest, even though it's probably just a coincidence. Think it's worth a shot to increase this?
Top
7im
Posts: 10179
Joined: Thu Nov 29, 2007 4:30 pm
Hardware configuration: Intel i7-4770K @ 4.5 GHz, 16 GB DDR3-2133 Corsair Vengence (black/red), EVGA GTX 760 @ 1200 MHz, on an Asus Maximus VI Hero MB (black/red), in a blacked out Antec P280 Tower, with a Xigmatek Night Hawk (black) HSF, Seasonic 760w Platinum (black case, sleeves, wires), 4 SilenX 120mm Case fans with silicon fan gaskets and silicon mounts (all black), a 512GB Samsung SSD (black), and a 2TB Black Western Digital HD (silver/black).
Location: Arizona
Contact:
Contact 7im

Re: work units not sending. Also, idle not working properly

Post by 7im »

But why is the server trying to forcibly drop the connection unless the connection is bad?

Yes, I would raise it slightly if for nothing better than to get a better diagnostic message.
How to provide enough information to get helpful support
Tell me and I forget. Teach me and I remember. Involve me and I learn.
Top
PantherX
Site Moderator
Posts: 6986
Joined: Wed Dec 23, 2009 9:33 am
Hardware configuration: V7.6.21 -> Multi-purpose 24/7
Windows 10 64-bit
CPU:2/3/4/6 -> Intel i7-6700K
GPU:1 -> Nvidia GTX 1080 Ti
§
Retired:
2x Nvidia GTX 1070
Nvidia GTX 675M
Nvidia GTX 660 Ti
Nvidia GTX 650 SC
Nvidia GTX 260 896 MB SOC
Nvidia 9600GT 1 GB OC
Nvidia 9500M GS
Nvidia 8800GTS 320 MB

Intel Core i7-860
Intel Core i7-3840QM
Intel i3-3240
Intel Core 2 Duo E8200
Intel Core 2 Duo E6550
Intel Core 2 Duo T8300
Intel Pentium E5500
Intel Pentium E5400
Location: Land Of The Long White Cloud
Contact:
Contact PantherX

Re: work units not sending. Also, idle not working properly

Post by PantherX »

With the default settings, I very rarely encounter this issue (not getting ACK). I haven't changed them and here is the log showing the uploading of the WU. It took about 17 minutes @ ~200 kibt/s and was successfully sent:

Code: Select all

17:27:47:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:6098 run:8 clone:50 gen:470 core:0xa3 unit:0x000003270a3b1e594e91dfa2c5ab7e89
...
17:32:54:WARNING:WU00:FS00:Exception: Failed to send results to work server: Transfer failed
17:32:54:WU00:FS00:Trying to send results to collection server
17:32:54:WU00:FS00:Uploading 10.78MiB to 128.143.199.97
17:32:54:WU00:FS00:Connecting to 128.143.199.97:8080
17:33:39:WU00:FS00:Upload 0.58%
17:34:09:WU00:FS00:Upload 1.16%
17:34:38:WU00:FS00:Upload 1.74%
17:35:03:WU00:FS00:Upload 2.32%
17:35:21:WU00:FS00:Upload 2.90%
17:35:28:WU00:FS00:Upload 3.48%
17:35:57:WU00:FS00:Upload 4.06%
17:37:12:WU00:FS00:Upload 4.64%
17:38:17:WU00:FS00:Upload 5.22%
17:38:28:WU00:FS00:Upload 5.80%
17:38:42:WU00:FS00:Upload 6.96%
17:38:57:WU00:FS00:Upload 7.54%
17:39:19:WU00:FS00:Upload 8.11%
17:39:25:WU00:FS00:Upload 8.69%
17:39:44:WU00:FS00:Upload 9.27%
17:39:52:WU00:FS00:Upload 10.43%
17:39:59:WU00:FS00:Upload 11.59%
17:40:05:WU00:FS00:Upload 12.17%
17:40:12:WU00:FS00:Upload 12.75%
17:40:19:WU00:FS00:Upload 13.33%
17:40:27:WU00:FS00:Upload 14.49%
17:40:35:WU00:FS00:Upload 15.65%
17:40:42:WU00:FS00:Upload 16.81%
17:40:50:WU00:FS00:Upload 18.55%
17:40:56:WU00:FS00:Upload 19.71%
17:41:04:WU00:FS00:Upload 20.87%
17:41:10:WU00:FS00:Upload 22.03%
17:41:18:WU00:FS00:Upload 23.77%
17:41:25:WU00:FS00:Upload 24.92%
17:41:32:WU00:FS00:Upload 26.08%
17:41:38:WU00:FS00:Upload 27.24%
17:41:44:WU00:FS00:Upload 28.40%
17:41:55:WU00:FS00:Upload 29.56%
17:42:04:WU00:FS00:Upload 30.72%
17:42:10:WU00:FS00:Upload 31.88%
17:42:18:WU00:FS00:Upload 33.62%
17:42:24:WU00:FS00:Upload 34.78%
17:42:32:WU00:FS00:Upload 36.52%
17:42:39:WU00:FS00:Upload 37.68%
17:42:46:WU00:FS00:Upload 38.84%
17:42:55:WU00:FS00:Upload 40.57%
17:43:01:WU00:FS00:Upload 41.73%
17:43:10:WU00:FS00:Upload 42.89%
17:43:18:WU00:FS00:Upload 44.63%
17:43:26:WU00:FS00:Upload 45.79%
17:43:33:WU00:FS00:Upload 46.95%
17:43:40:WU00:FS00:Upload 48.11%
17:43:47:WU00:FS00:Upload 49.27%
17:43:56:WU00:FS00:Upload 51.01%
17:44:04:WU00:FS00:Upload 52.17%
17:44:10:WU00:FS00:Upload 53.33%
17:44:18:WU00:FS00:Upload 55.07%
17:44:28:WU00:FS00:Upload 56.80%
17:44:34:WU00:FS00:Upload 57.96%
17:44:40:WU00:FS00:Upload 59.12%
17:44:48:WU00:FS00:Upload 60.86%
17:44:57:WU00:FS00:Upload 62.02%
17:45:08:WU00:FS00:Upload 63.18%
17:45:18:WU00:FS00:Upload 64.34%
17:45:25:WU00:FS00:Upload 64.92%
17:45:31:WU00:FS00:Upload 65.50%
17:45:40:WU00:FS00:Upload 66.66%
17:45:47:WU00:FS00:Upload 67.24%
17:45:54:WU00:FS00:Upload 67.82%
17:46:01:WU00:FS00:Upload 68.40%
17:46:09:WU00:FS00:Upload 69.56%
17:46:15:WU00:FS00:Upload 70.14%
17:46:26:WU00:FS00:Upload 71.30%
17:46:40:WU00:FS00:Upload 71.88%
17:46:46:WU00:FS00:Upload 72.45%
17:46:53:WU00:FS00:Upload 73.61%
17:46:59:WU00:FS00:Upload 74.77%
17:47:06:WU00:FS00:Upload 75.93%
17:47:12:WU00:FS00:Upload 77.09%
17:47:19:WU00:FS00:Upload 78.83%
17:47:27:WU00:FS00:Upload 80.57%
17:47:36:WU00:FS00:Upload 81.73%
17:47:44:WU00:FS00:Upload 83.47%
17:47:51:WU00:FS00:Upload 85.21%
17:47:59:WU00:FS00:Upload 86.95%
17:48:05:WU00:FS00:Upload 88.11%
17:48:12:WU00:FS00:Upload 89.26%
17:48:20:WU00:FS00:Upload 89.84%
17:48:27:WU00:FS00:Upload 91.58%
17:48:33:WU00:FS00:Upload 92.74%
17:48:41:WU00:FS00:Upload 94.48%
17:48:49:WU00:FS00:Upload 96.22%
17:48:56:WU00:FS00:Upload 97.96%
17:49:04:WU00:FS00:Upload 99.70%
17:49:07:WU00:FS00:Upload complete
17:49:08:WU00:FS00:Server responded WORK_ACK (400)
17:49:08:WU00:FS00:Final credit estimate, 8918.00 points
17:49:08:WU00:FS00:Cleaning up
ETA:
Now ↞ Very Soon ↔ Soon ↔ Soon-ish ↔ Not Soon ↠ End Of Time

Welcome To The F@H Support Forum Ӂ Troubleshooting Bad WUs Ӂ Troubleshooting Server Connectivity Issues
Top
Volnaiskra
Posts: 57
Joined: Tue Nov 26, 2013 9:02 am

Re: work units not sending. Also, idle not working properly

Post by Volnaiskra »

ok, well changing max-queue to 64 didn't do anything. Once the WUs reached exactly 10 (7 complete, 3 incomplete), FAH froze again, just like always. There were 10 work folders, from "00" to "09". I wonder if FAH has trouble correctly assigning new WU numbers above 09.

Changing min-connect-time didn't seem to do anything either. I restarted FAH, and it retried 2 of the uploads. It took about 10 minutes to do both of them, but still no ACK.
Top
davidcoton
Posts: 1094
Joined: Wed Nov 05, 2008 3:19 pm
Location: Cambridge, UK

Re: work units not sending. Also, idle not working properly

Post by davidcoton »

Let's think again about where the ACK packets are going AWOL. It is, presumably but not absolutely certainly, within your network -- if not it is likely there would be far more reports here. So, have you tried rebooting your router/modem? (Sorry if that's already been mentioned, but the thread is rather long now.) I ask because I recently had two systems showing errors that looked like the GPU servers were out of WUs. But no-one else was reporting a problem. I changed client-type (which can get round server problems) -- no change. I rebooted the PCs -- no change. In desperation I rebooted the router, paused/unpaused the slots to force a retry, and immediately got new WUs. It looks as though the router firewall started blocking packets from some servers, but FAHControl still came up with errors as if it had received replies. Possibly there is an underlying bug in FAHControl or FAHClient that uses an old or a default packet when the request times out -- I don't have good enough evidence to ask for a ticket to be raised. Code examination might show the problem, but the programmer has more important things to worry about. And, once known and remembered, the workaround is simple enough that a code change (with its risk of further consequent problems) is probably not justified.

I don't know if there are any simple ways of tracing internet packets at OS or application level on an end-user PC -- I suspect not, which might end further investigation :(

David
Image
Top
Volnaiskra
Posts: 57
Joined: Tue Nov 26, 2013 9:02 am

Re: work units not sending. Also, idle not working properly

Post by Volnaiskra »

The modem/router may well be the catalyst, because these problems started around the same time I bought this new router. But I've gone through various settings with direction from people on this thread, and nothing unusual came up, and no changes to settings made any difference. I've tried rebooting the modem quite a few times, and I upgraded the firmware a while back too. Nothing changed.

I'll double check this over the next few weeks, but it seems like FAH freezes when there are *exactly* 10 work folders (ie 10 concurrent WUs - some of them finished, some still folding) present. I'd like to hear if someone has a theory as to why it would happen at 10 each time.
Top
Post Reply

Return to “New Donors start here”