Folding Forum

Posted: **Mon Jun 08, 2009 3:33 am**

Most times when my client is running 2653 projects, it hangs at core shutdown.
I have to close the client and restart. This does not happen on every 2653, just most. If I catch it in time I can start another job right away, if not it will hang for hours until I wake or get home from work and see that the job completed, but needs a good kick to start another one. This happens about every 3rd or 4th project.

This is the job before and the one after where it hung just from the previous log.

Code: Select all


--- Opening Log file [June 4 03:30:55 UTC] 


# Windows SMP Console Edition #################################################
###############################################################################

                       Folding@Home Client Version 6.23 Beta R1

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: C:\Program Files (x86)\Folding@Home Windows SMP Client V1.01
Executable: C:\Program Files (x86)\Folding@Home Windows SMP Client V1.01\FAH86.exe
Arguments: -smp -verbosity 9 

[03:30:55] - Ask before connecting: No
[03:30:55] - User name: Randman (Team 162247)
[03:30:55] - User ID: 417D5718532F2222
[03:30:55] - Machine ID: 1
[03:30:55] 
[03:30:56] Loaded queue successfully.
[03:30:56] 
.
.
.====>removed to save space on where other 2653s completed and moved to the next.<========
.
.

[11:04:34] 
[11:04:34] Finished Work Unit:
[11:04:34] - Reading up to 3715440 from "work/wudata_09.arc": Read 3715440
[11:04:34] - Reading up to 1767228 from "work/wudata_09.xtc": Read 1767228
[11:04:34] goefile size: 0
[11:04:34] logfile size: 30777
[11:04:34] Leaving Run
[11:04:38] - Writing 5517845 bytes of core data to disk...
[11:04:38]   ... Done.
[11:04:38] - Failed to delete work/wudata_09.sas
[11:04:38] - Failed to delete work/wudata_09.goe
[11:04:38] Warning:  check for stray files
[11:04:38] - Shutting down core
[11:06:38] 
[11:06:38] Folding@home Core Shutdown: FINISHED_UNIT
[11:06:38] 
[11:06:38] Folding@home Core Shutdown: FINISHED_UNIT
[11:06:42] CoreStatus = 64 (100)
[11:06:42] Unit 9 finished with 79 percent of time to deadline remaining.
[11:06:42] Updated performance fraction: 0.854733
[11:06:42] Sending work to server
[11:06:42] Project: 2653 (Run 19, Clone 126, Gen 111)


[11:06:42] + Attempting to send results [June 5 11:06:42 UTC]
[11:06:42] - Reading file work/wuresults_09.dat from core
[11:06:42]   (Read 5517845 bytes from disk)
[11:06:42] Connecting to http://171.64.65.64:8080/
[11:08:00] Posted data.
[11:08:00] Initial: 0000; - Uploaded at ~68 kB/s
[11:08:01] - Averaged speed for that direction ~96 kB/s
[11:08:01] + Results successfully sent
[11:08:01] Thank you for your contribution to Folding@Home.
[11:08:01] + Number of Units Completed: 15

[11:08:06] - Warning: Could not delete all work unit files (9): Core returned invalid code
[11:08:06] Trying to send all finished work units
[11:08:06] + No unsent completed units remaining.
[11:08:06] - Preparing to get new work unit...
[11:08:06] + Attempting to get work packet
[11:08:06] - Will indicate memory of 1024 MB
[11:08:06] - Detect CPU. Vendor: GenuineIntel, Family: 6, Model: 10, Stepping: 4
[11:08:06] - Connecting to assignment server
[11:08:06] Connecting to http://assign.stanford.edu:8080/
[11:08:06] Posted data.
[11:08:06] Initial: 40AB; - Successful: assigned to (171.64.65.64).
[11:08:06] + News From Folding@Home: Welcome to Folding@Home
[11:08:06] Loaded queue successfully.
[11:08:06] Connecting to http://171.64.65.64:8080/
[11:08:10] Posted data.
[11:08:10] Initial: 0000; - Receiving payload (expected size: 2433915)
[11:08:12] - Downloaded at ~1188 kB/s
[11:08:12] - Averaged speed for that direction ~908 kB/s
[11:08:12] + Received work.
[11:08:12] Trying to send all finished work units
[11:08:12] + No unsent completed units remaining.
[11:08:12] + Closed connections
[11:08:12] 
[11:08:12] + Processing work unit
[11:08:12] Work type a1 not eligible for variable processors
[11:08:12] Core required: FahCore_a1.exe
[11:08:12] Core found.
[11:08:12] Using generic mpiexec calls
[11:08:12] Working on queue slot 00 [June 5 11:08:12 UTC]
[11:08:12] + Working ...
[11:08:12] - Calling 'mpiexec -np 4 -channel auto -host 127.0.0.1 FahCore_a1.exe -dir work/ -suffix 00 -checkpoint 8 -verbose -lifeline 4348 -version 623'

[11:08:12] 
[11:08:12] *------------------------------*
[11:08:12] Folding@Home Gromacs SMP Core
[11:08:12] Version 1.74 (March 10, 2007)
[11:08:12] 
[11:08:12] Preparing to commence simulation
[11:08:12] - Ensuring status. Please wait.
[11:08:13] - Starting from initial work packet
[11:08:13] 
[11:08:13] Project: 2653 (Run 6, Clone 155, Gen 111)
[11:08:13] 
[11:08:13] Assembly optimizations on if available.
[11:08:13] Entering M.D.
[11:08:30] percent)
[11:08:30] - Starting from initial work packet
[11:08:30] 
[11:08:30] Project: 2653 (Run 6, Clone 155, Gen 111)
[11:08:30] 
[11:08:31] Entering M.D.
[11:08:37] Rejecting checkpoint
[11:08:38] SE boost OK.
[11:08:38] ein in POPCExtra SSE boost OK.
[11:08:38] 
[11:08:38] Extra SSE boost OK.
[11:08:38] Writing local files
[11:08:38] Completed 0 out of 500000 steps  (0 percent)
[11:15:18] Writing local files
[11:15:18] Completed 5000 out of 500000 steps  (1 percent)
[11:22:06] Writing local files
[11:22:06] Completed 10000 out of 500000 steps  (2 percent)
[11:28:56] Writing local files
[11:28:56] Completed 15000 out of 500000 steps  (3 percent)
[11:35:42] Writing local files
[11:35:42] Completed 20000 out of 500000 steps  (4 percent)
[11:42:27] Writing local files
[11:42:27] Completed 25000 out of 500000 steps  (5 percent)
[11:43:21] - Autosending finished units... [June 5 11:43:21 UTC]
.
.====>removed to save space <========
.
[22:08:06] Completed 490000 out of 500000 steps  (98 percent)
[22:14:49] Writing local files
[22:14:49] Completed 495000 out of 500000 steps  (99 percent)
[22:21:37] Writing local files
[22:21:37] Completed 500000 out of 500000 steps  (100 percent)
[22:21:37] Writing final coordinates.
[22:21:37] Past main M.D. loop
[22:21:37] Will end MPI now
[22:22:37] 
[22:22:37] Finished Work Unit:
[22:22:37] - Reading up to 3721776 from "work/wudata_00.arc": Read 3721776
[22:22:37] - Reading up to 1775216 from "work/wudata_00.xtc": Read 1775216
[22:22:37] goefile size: 0
[22:22:37] logfile size: 17312
[22:22:37] Leaving Run
[22:22:41] - Writing 5518704 bytes of core data to disk...
[22:22:41]   ... Done.
[22:22:41] - Failed to delete work/wudata_00.sas
[22:22:41] - Failed to delete work/wudata_00.goe
[22:22:41] Warning:  check for stray files
[22:22:41] - Shutting down core
[22:24:41] 
[22:24:41] Folding@home Core Shutdown: FINISHED_UNIT
[22:24:41] 
[22:24:41] Folding@home Core Shutdown: FINISHED_UNIT
[22:24:45] CoreStatus = 64 (100)
[22:24:45] Unit 0 finished with 88 percent of time to deadline remaining.
[22:24:45] Updated performance fraction: 0.860295
[22:24:45] Sending work to server
[22:24:45] Project: 2653 (Run 6, Clone 155, Gen 111)


[22:24:45] + Attempting to send results [June 5 22:24:45 UTC]
[22:24:45] - Reading file work/wuresults_00.dat from core
[22:24:45]   (Read 5518704 bytes from disk)
[22:24:45] Connecting to http://171.64.65.64:8080/
[22:26:45] Posted data.
[22:26:45] Initial: 0000; - Uploaded at ~44 kB/s
[22:26:46] - Averaged speed for that direction ~86 kB/s
[22:26:46] + Results successfully sent
[22:26:46] Thank you for your contribution to Folding@Home.
[22:26:46] + Number of Units Completed: 16

[22:26:50] - Warning: Could not delete all work unit files (0): Core returned invalid code
[22:26:50] Trying to send all finished work units
[22:26:50] + No unsent completed units remaining.
[22:26:50] - Preparing to get new work unit...
[22:26:50] + Attempting to get work packet
[22:26:50] - Will indicate memory of 1024 MB
[22:26:50] - Connecting to assignment server
[22:26:50] Connecting to http://assign.stanford.edu:8080/
[22:26:51] Posted data.
[22:26:51] Initial: 40AB; - Successful: assigned to (171.64.65.64).
[22:26:51] + News From Folding@Home: Welcome to Folding@Home
[22:26:51] Loaded queue successfully.
[22:26:51] Connecting to http://171.64.65.64:8080/
[22:26:54] Posted data.
[22:26:54] Initial: 0000; - Receiving payload (expected size: 2431663)
[22:26:59] - Downloaded at ~474 kB/s
[22:26:59] - Averaged speed for that direction ~822 kB/s
[22:26:59] + Received work.
[22:26:59] Trying to send all finished work units
[22:26:59] + No unsent completed units remaining.
[22:26:59] + Closed connections
[22:26:59] 
[22:26:59] + Processing work unit
[22:26:59] Work type a1 not eligible for variable processors
[22:26:59] Core required: FahCore_a1.exe
[22:26:59] Core found.
[22:26:59] Using generic mpiexec calls
[22:26:59] Working on queue slot 01 [June 5 22:26:59 UTC]
[22:26:59] + Working ...
[22:26:59] - Calling 'mpiexec -np 4 -channel auto -host 127.0.0.1 FahCore_a1.exe -dir work/ -suffix 01 -checkpoint 8 -verbose -lifeline 4348 -version 623'

[22:27:00] 
[22:27:00] *------------------------------*
[22:27:00] Folding@Home Gromacs SMP Core
[22:27:00] Version 1.74 (March 10, 2007)
[22:27:00] 
[22:27:00] Preparing to commence simulation
[22:27:00] - Ensuring status. Please wait.
[22:27:01] - Starting from initial work packet
[22:27:01] 
[22:27:01] Project: 2653 (Run 34, Clone 54, Gen 111)
[22:27:01] 
[22:27:01] Assembly optimizations on if available.
[22:27:01] Entering M.D.
[22:27:18] percent)
[22:27:18] - Starting from initial work packet
[22:27:18] 
[22:27:18] Project: 2653 (Run 34, Clone 54, Gen 111)
[22:27:18] 
[22:27:19] Entering M.D.
[22:27:25] Rejecting checkpoint
[22:27:26] SE boost OK.
[22:27:26] ein in POPCExtra SSE boost OK.
[22:27:26] 
[22:27:26] Extra SSE boost OK.
[22:27:26] Writing local files
[22:27:26] Completed 0 out of 500000 steps  (0 percent)
[22:34:15] Writing local files
[22:34:15] Completed 5000 out of 500000 steps  (1 percent)
[22:40:59] Writing local files
[22:40:59] Completed 10000 out of 500000 steps  (2 percent)
[22:47:38] Writing local files
[22:47:38] Completed 15000 out of 500000 steps  (3 percent)
[22:54:19] Writing local files
[22:54:19] Completed 20000 out of 500000 steps  (4 percent)
[23:01:06] Writing local files
[23:01:06] Completed 25000 out of 500000 steps  (5 percent)
[23:07:45] Writing local files
[23:07:45] Completed 30000 out of 500000 steps  (6 percent)
[23:14:28] Writing local files
[23:14:28] Completed 35000 out of 500000 steps  (7 percent)
[23:21:22] Writing local files
[23:21:22] Completed 40000 out of 500000 steps  (8 percent)
[23:28:12] Writing local files
[23:28:12] Completed 45000 out of 500000 steps  (9 percent)
[23:35:03] Writing local files
[23:35:04] Completed 50000 out of 500000 steps  (10 percent)
[23:41:59] Writing local files
[23:42:00] Completed 55000 out of 500000 steps  (11 percent)
[23:43:19] - Autosending finished units... [June 5 23:43:19 UTC]
[23:43:19] Trying to send all finished work units
[23:43:19] + No unsent completed units remaining.
[23:43:19] - Autosend completed
.
.====>removed to save space <========
.
[09:38:15] Writing local files
[09:38:15] Completed 485000 out of 500000 steps  (97 percent)
[09:45:17] Writing local files
[09:45:17] Completed 490000 out of 500000 steps  (98 percent)
[09:52:18] Writing local files
[09:52:18] Completed 495000 out of 500000 steps  (99 percent)
[09:59:18] Writing local files
[09:59:18] Completed 500000 out of 500000 steps  (100 percent)
[09:59:19] Writing final coordinates.
[09:59:19] Past main M.D. loop
[09:59:19] Will end MPI now
[10:00:19] 
[10:00:19] Finished Work Unit:
[10:00:19] - Reading up to 3724992 from "work/wudata_01.arc": Read 3724992
[10:00:19] - Reading up to 1779636 from "work/wudata_01.xtc": Read 1779636
[10:00:19] goefile size: 0
[10:00:19] logfile size: 17312
[10:00:19] Leaving Run
[10:00:23] - Writing 5526340 bytes of core data to disk...
[10:00:23]   ... Done.
[10:00:23] - Failed to delete work/wudata_01.sas
[10:00:23] - Failed to delete work/wudata_01.goe
[10:00:23] Warning:  check for stray files
[10:00:23] - Shutting down core
[10:02:23] 
[10:02:23] Folding@home Core Shutdown: FINISHED_UNIT
[10:02:23] 
[10:02:23] Folding@home Core Shutdown: FINISHED_UNIT
[11:43:16] - Autosending finished units... [June 6 11:43:16 UTC]
[11:43:16] Trying to send all finished work units
[11:43:16] + No unsent completed units remaining.
[11:43:16] - Autosend completed
[12:21:49] Killing all core threads
[12:21:49] Killing 3 cores
[12:21:49] Killing core 0
[12:21:49] Killing core 1
[12:21:49] Killing core 2

Folding@Home Client Shutdown at user request.
[12:21:49] ***** Got a SIGTERM signal (2)
[12:21:49] Killing all core threads
[12:21:49] Killing 3 cores
[12:21:49] Killing core 0
[12:21:49] Killing core 1
[12:21:49] Killing core 2

Folding@Home Client Shutdown.
[/list]



[color=#0000FF]This is the job before and the one after where it hung just now.[/color]
[list]


--- Opening Log file [June 6 12:21:57 UTC] 


# Windows SMP Console Edition #################################################
###############################################################################

                       Folding@Home Client Version 6.23 Beta R1

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: C:\Program Files (x86)\Folding@Home Windows SMP Client V1.01
Executable: C:\Program Files (x86)\Folding@Home Windows SMP Client V1.01\FAH86.exe
Arguments: -smp -verbosity 9 

[12:21:57] - Ask before connecting: No
[12:21:57] - User name: Randman (Team 162247)
[12:21:57] - User ID: 417D5718532F2222
[12:21:57] - Machine ID: 1
[12:21:57] 
[12:21:58] Loaded queue successfully.
[color=#FF0000].
.
.
.====>removed to save space on where other 2653s completed and moved to the next.<========
.
.[/color]

--- Opening Log file [June 6 12:21:57 UTC] 

[03:56:44] + Attempting to send results [June 7 03:56:44 UTC]
[03:56:44] - Reading file work/wuresults_02.dat from core
[03:56:44]   (Read 5511440 bytes from disk)
[03:56:44] Connecting to http://171.64.65.64:8080/
[03:57:27] Posted data.
[03:57:27] Initial: 0000; - Uploaded at ~119 kB/s
[03:57:29] - Averaged speed for that direction ~85 kB/s
[03:57:29] + Results successfully sent
[03:57:29] Thank you for your contribution to Folding@Home.
[03:57:49] - Warning: Could not delete all work unit files (2): Core returned invalid code
[03:57:49] Trying to send all finished work units
[03:57:49] + No unsent completed units remaining.
[03:57:49] - Preparing to get new work unit...
[03:57:49] + Attempting to get work packet
[03:57:49] - Will indicate memory of 1024 MB
[03:57:49] - Detect CPU. Vendor: GenuineIntel, Family: 6, Model: 10, Stepping: 4
[03:57:49] - Connecting to assignment server
[03:57:49] Connecting to http://assign.stanford.edu:8080/
[03:57:50] Posted data.
[03:57:50] Initial: 40AB; - Successful: assigned to (171.64.65.64).
[03:57:50] + News From Folding@Home: Welcome to Folding@Home
[03:57:50] Loaded queue successfully.
[03:57:50] Connecting to http://171.64.65.64:8080/
[03:57:53] Posted data.
[03:57:53] Initial: 0000; - Receiving payload (expected size: 2437391)
[03:57:59] - Downloaded at ~396 kB/s
[03:57:59] - Averaged speed for that direction ~681 kB/s
[03:57:59] + Received work.
[03:57:59] Trying to send all finished work units
[03:57:59] + No unsent completed units remaining.
[03:57:59] + Closed connections
[03:58:04] 
[03:58:04] + Processing work unit
[03:58:04] Work type a1 not eligible for variable processors
[03:58:04] Core required: FahCore_a1.exe
[03:58:04] Core found.
[03:58:04] Using generic mpiexec calls
[03:58:04] Working on queue slot 03 [June 7 03:58:04 UTC]
[03:58:04] + Working ...
[03:58:04] - Calling 'mpiexec -np 4 -channel auto -host 127.0.0.1 FahCore_a1.exe -dir work/ -suffix 03 -checkpoint 8 -verbose -lifeline 9560 -version 623'

[03:58:04] 
[03:58:04] *------------------------------*
[03:58:04] Folding@Home Gromacs SMP Core
[03:58:04] Version 1.74 (March 10, 2007)
[03:58:04] 
[03:58:04] Preparing to commence simulation
[03:58:04] - Ensuring status. Please wait.
[03:58:21] - Looking at optimizations...
[03:58:21] - Working with standard loops on this execution.
[03:58:21] - Previous termination of core was improper.
[03:58:21] - Going to use standard loops.
[03:58:21] - Files status OK
[03:58:22] - Expanded 2436879 -> 1292344- Starting from initial work packet
[03:58:22] 
[03:58:22] Project: 2653 (Run 10, Clone 125, Gen 112)
[03:58:22] 
[03:58:23]  (Run 10, Clone 125, Gen 112)
[03:58:23] 
[03:58:24] Entering M.D.
[03:58:30] Rejecting checkpoint
[03:58:44] SE boost OK.
[03:58:44] ein in POPCExtra SSE boost OK.
[03:58:44] 
[03:58:44] Extra SSE boost OK.
[03:58:44] Writing local files
[03:58:44] Completed 0 out of 500000 steps  (0 percent)
[04:05:30] Writing local files
[04:05:30] Completed 5000 out of 500000 steps  (1 percent)
[04:12:28] Writing local files
[04:12:28] Completed 10000 out of 500000 steps  (2 percent)
[04:19:15] Writing local files
[04:19:15] Completed 15000 out of 500000 steps  (3 percent)
[04:25:58] Writing local files
[04:25:58] Completed 20000 out of 500000 steps  (4 percent)
[04:32:45] Writing local files
[04:32:45] Completed 25000 out of 500000 steps  (5 percent)
[04:39:33] Writing local files
.
.====>removed to save space <========
.
[14:46:29] Writing local files
[14:46:29] Completed 475000 out of 500000 steps  (95 percent)
[14:53:23] Writing local files
[14:53:23] Completed 480000 out of 500000 steps  (96 percent)
[15:00:17] Writing local files
[15:00:18] Completed 485000 out of 500000 steps  (97 percent)
[15:07:08] Writing local files
[15:07:08] Completed 490000 out of 500000 steps  (98 percent)
[15:13:53] Writing local files
[15:13:53] Completed 495000 out of 500000 steps  (99 percent)
[15:20:51] Writing local files
[15:20:51] Completed 500000 out of 500000 steps  (100 percent)
[15:20:51] Writing final coordinates.
[15:20:52] Past main M.D. loop
[15:20:52] Will end MPI now
[15:21:52] 
[15:21:52] Finished Work Unit:
[15:21:52] - Reading up to 3729120 from "work/wudata_03.arc": Read 3729120
[15:21:52] - Reading up to 1788076 from "work/wudata_03.xtc": Read 1788076
[15:21:52] goefile size: 0
[15:21:52] logfile size: 17313
[15:21:52] Leaving Run
[15:21:54] - Writing 5538909 bytes of core data to disk...
[15:21:54]   ... Done.
[15:21:54] - Failed to delete work/wudata_03.sas
[15:21:54] - Failed to delete work/wudata_03.goe
[15:21:54] Warning:  check for stray files
[15:21:54] - Shutting down core
[15:23:54] 
[15:23:54] Folding@home Core Shutdown: FINISHED_UNIT
[15:23:54] 
[15:23:54] Folding@home Core Shutdown: FINISHED_UNIT
[15:23:58] CoreStatus = 64 (100)
[15:23:58] Unit 3 finished with 88 percent of time to deadline remaining.
[15:23:58] Updated performance fraction: 0.864417
[15:23:58] Sending work to server
[15:23:58] Project: 2653 (Run 10, Clone 125, Gen 112)


[15:23:58] + Attempting to send results [June 7 15:23:58 UTC]
[15:23:58] - Reading file work/wuresults_03.dat from core
[15:23:58]   (Read 5538909 bytes from disk)
[15:23:58] Connecting to http://171.64.65.64:8080/
[15:25:05] Posted data.
[15:25:05] Initial: 0000; - Uploaded at ~78 kB/s
[15:25:07] - Averaged speed for that direction ~84 kB/s
[15:25:07] + Results successfully sent
[15:25:07] Thank you for your contribution to Folding@Home.
[15:25:07] + Number of Units Completed: 17

[15:25:11] - Warning: Could not delete all work unit files (3): Core returned invalid code
[15:25:11] Trying to send all finished work units
[15:25:11] + No unsent completed units remaining.
[15:25:11] - Preparing to get new work unit...
[15:25:11] + Attempting to get work packet
[15:25:11] - Will indicate memory of 1024 MB
[15:25:11] - Connecting to assignment server
[15:25:11] Connecting to http://assign.stanford.edu:8080/
[15:25:12] Posted data.
[15:25:12] Initial: 40AB; - Successful: assigned to (171.64.65.64).
[15:25:12] + News From Folding@Home: Welcome to Folding@Home
[15:25:12] Loaded queue successfully.
[15:25:12] Connecting to http://171.64.65.64:8080/
[15:25:15] Posted data.
[15:25:15] Initial: 0000; - Receiving payload (expected size: 2442530)
[15:25:20] - Downloaded at ~477 kB/s
[15:25:20] - Averaged speed for that direction ~640 kB/s
[15:25:20] + Received work.
[15:25:20] Trying to send all finished work units
[15:25:20] + No unsent completed units remaining.
[15:25:20] + Closed connections
[15:25:20] 
[15:25:20] + Processing work unit
[15:25:20] Work type a1 not eligible for variable processors
[15:25:20] Core required: FahCore_a1.exe
[15:25:20] Core found.
[15:25:20] Using generic mpiexec calls
[15:25:20] Working on queue slot 04 [June 7 15:25:20 UTC]
[15:25:20] + Working ...
[15:25:20] - Calling 'mpiexec -np 4 -channel auto -host 127.0.0.1 FahCore_a1.exe -dir work/ -suffix 04 -checkpoint 8 -verbose -lifeline 9560 -version 623'

[15:25:20] 
[15:25:20] *------------------------------*
[15:25:20] Folding@Home Gromacs SMP Core
[15:25:20] Version 1.74 (March 10, 2007)
[15:25:20] 
[15:25:20] Preparing to commence simulation
[15:25:20] - Ensuring status. Please wait.
[15:25:21] - Starting from initial work packet
[15:25:21] 
[15:25:21] Project: 2653 (Run 5, Clone 105, Gen 112)
[15:25:21] 
[15:25:21] Assembly optimizations on if available.
[15:25:21] Entering M.D.
[15:25:38] percent)
[15:25:38] - Starting from initial work packet
[15:25:38] 
[15:25:38] Project: 2653 (Run 5, Clone 105, Gen 112)
[15:25:38] 
[15:25:41] Entering M.D.
[15:25:47] Rejecting checkpoint
[15:25:48] SE boost OK.
[15:25:48] ein in POPCExtra SSE boost OK.
[15:25:48] 
[15:25:48] Extra SSE boost OK.
[15:25:48] Writing local files
[15:25:48] Completed 0 out of 500000 steps  (0 percent)
[15:32:50] Writing local files
[15:32:50] Completed 5000 out of 500000 steps  (1 percent)
[15:39:45] Writing local files
[15:39:45] Completed 10000 out of 500000 steps  (2 percent)
[15:46:47] Writing local files
[15:46:47] Completed 15000 out of 500000 steps  (3 percent)
[15:53:53] Writing local files
[15:53:53] Completed 20000 out of 500000 steps  (4 percent)
[15:54:20] - Autosending finished units... [June 7 15:54:20 UTC]
[15:54:20] Trying to send all finished work units
[15:54:20] + No unsent completed units remaining.
[15:54:20] - Autosend completed
[16:00:59] Writing local files
[16:00:59] Completed 25000 out of 500000 steps  (5 percent)
.
.====>removed to save space <========
.
[02:32:09] Writing local files
[02:32:09] Completed 475000 out of 500000 steps  (95 percent)
[02:39:05] Writing local files
[02:39:05] Completed 480000 out of 500000 steps  (96 percent)
[02:46:00] Writing local files
[02:46:00] Completed 485000 out of 500000 steps  (97 percent)
[02:52:55] Writing local files
[02:52:55] Completed 490000 out of 500000 steps  (98 percent)
[02:59:52] Writing local files
[02:59:52] Completed 495000 out of 500000 steps  (99 percent)
[03:06:37] Writing local files
[03:06:37] Completed 500000 out of 500000 steps  (100 percent)
[03:06:37] Writing final coordinates.
[03:06:37] Past main M.D. loop
[03:06:37] Will end MPI now
[03:07:37] 
[03:07:37] Finished Work Unit:
[03:07:37] - Reading up to 3726768 from "work/wudata_04.arc": Read 3726768
[03:07:37] - Reading up to 1780968 from "work/wudata_04.xtc": Read 1780968
[03:07:37] goefile size: 0
[03:07:37] logfile size: 17312
[03:07:37] Leaving Run
[03:07:38] - Writing 5529448 bytes of core data to disk...
[03:07:38]   ... Done.
[03:07:38] - Failed to delete work/wudata_04.sas
[03:07:38] - Failed to delete work/wudata_04.goe
[03:07:38] Warning:  check for stray files
[03:07:38] - Shutting down core
[03:07:38] 
[03:07:38] Folding@home Core Shutdown: FINISHED_UNIT
[03:07:38] 
[03:07:38] Folding@home Core Shutdown: FINISHED_UNIT

edited to put log in code tags & shorten log

Posted: **Mon Jun 08, 2009 4:03 am**

The cause of this hang is not known but one theory is that it's associated with network issues interacting with MPI. Suggestions include
(A) Adding a Microsoft Loopback Adapter. (a software driver, not new hardware)
(B) Switching your machine from "automatic" to a fixed IP address.

Which version of Windows are you running?

The next time this happens, go to Task manager and look for active tasks called FahCore_a1.exe. Then see what happens if you manually kill them. Let us know how many were running and what happened next. This may not help, but at least we'll have a bit more information about the problem.

Posted: **Mon Jun 08, 2009 6:44 pm**

bruce wrote:The cause of this hang is not known but one theory is that it's associated with network issues interacting with MPI. Suggestions include
(A) Adding a Microsoft Loopback Adapter. (a software driver, not new hardware)
(B) Switching your machine from "automatic" to a fixed IP address.

Which version of Windows are you running?

The next time this happens, go to Task manager and look for active tasks called FahCore_a1.exe. Then see what happens if you manually kill them. Let us know how many were running and what happened next. This may not help, but at least we'll have a bit more information about the problem.

I captured images of that, but was too tired to figure out how to post a JPG here. Anyway, all 4 were running though CPU was back down to 1-3% versus 51-53%. Memory was still allocated to each job. The FahCore_a1.exe tasks would have sat there for hours until I restarted the client. After the restart it ran through the complete job as if it were reading the logfile in about 10secs. Can cut and paste the current log file when I get home tonight from the restart until current time.

Also, in the past I have tried to terminate each of the 4 via end-process manually and this has not helped the client get unjammed either.

Hope that helps some.
Randy

Posted: **Fri Jun 12, 2009 12:13 am**

Here it is again... I started this y'day after I could fire my computer back up. It ran one job fine, passed to a second just peachy, then crapped out on the third all 2653ssss... It has sat here for 2 hours waiting to start. All 4 FahCore_a1.exe*32 are running with memory tied against each one. The processes are not maxing out the CPUs, like before the machine has gone from 51-53% to 1-3% usage. I am cutting and pasting from the client window, if you require the whole log(s). I will be more than glad to post those too.

Code: Select all

[11:52:31] + Attempting to send results [June 11 11:52:31 UTC]
[11:52:31] - Reading file work/wuresults_07.dat from core
[11:52:31]   (Read 5538426 bytes from disk)
[11:52:31] Connecting to http://171.64.65.64:8080/
[11:53:14] Posted data.
[11:53:14] Initial: 0000; - Uploaded at ~120 kB/s
[11:53:16] - Averaged speed for that direction ~81 kB/s
[11:53:16] + Results successfully sent
[11:53:16] Thank you for your contribution to Folding@Home.
[11:53:16] + Number of Units Completed: 19

[11:53:20] - Warning: Could not delete all work unit files (7): Core returned in
valid code
[11:53:20] Trying to send all finished work units
[11:53:20] + No unsent completed units remaining.
[11:53:20] - Preparing to get new work unit...
[11:53:20] + Attempting to get work packet
[11:53:20] - Will indicate memory of 1024 MB
[11:53:20] - Connecting to assignment server
[11:53:20] Connecting to http://assign.stanford.edu:8080/
[11:53:21] Posted data.
[11:53:21] Initial: 40AB; - Successful: assigned to (171.64.65.64).
[11:53:21] + News From Folding@Home: Welcome to Folding@Home
[11:53:21] Loaded queue successfully.
[11:53:21] Connecting to http://171.64.65.64:8080/
[11:53:24] Posted data.
[11:53:24] Initial: 0000; - Receiving payload (expected size: 2434314)
[11:53:26] - Downloaded at ~1188 kB/s
[11:53:26] - Averaged speed for that direction ~725 kB/s
[11:53:26] + Received work.
[11:53:26] Trying to send all finished work units
[11:53:26] + No unsent completed units remaining.
[11:53:26] + Closed connections
[11:53:26]
[11:53:26] + Processing work unit
[11:53:26] Work type a1 not eligible for variable processors
[11:53:26] Core required: FahCore_a1.exe
[11:53:26] Core found.
[11:53:26] Using generic mpiexec calls
[11:53:26] Working on queue slot 08 [June 11 11:53:26 UTC]
[11:53:26] + Working ...
[11:53:26] - Calling 'mpiexec -np 4 -channel auto -host 127.0.0.1 FahCore_a1.exe
 -dir work/ -suffix 08 -checkpoint 8 -verbose -lifeline 5016 -version 623'

[11:53:27]
[11:53:27] *------------------------------*
[11:53:27] Folding@Home Gromacs SMP Core
[11:53:27] Version 1.74 (March 10, 2007)
[11:53:27]
[11:53:27] Preparing to commence simulation
[11:53:27] - Ensuring status. Please wait.
[11:53:28] - Starting from initial work packet
[11:53:28]
[11:53:28] Project: 2653 (Run 4, Clone 146, Gen 113)
[11:53:28]
[11:53:29] Assembly optimizations on if available.
[11:53:29] Entering M.D.
[11:53:45] ailed to - Starting from initial work packet
[11:53:45]
[11:53:45] Project: 2653 (Run 4, Clone 146, Gen 113)
[11:53:45]
[11:53:45] ect: 2653 (Run
[11:53:45] Project: 2653 (Run 4, Clone 146, Gen 113)
[11:53:45]
[11:53:46] Entering M.D.
[11:53:52] l files
[11:53:52]  Protein in POPC
[11:53:52] Writing local files
[11:53:53]
[11:53:53] xtra SSE boost OK.
[11:53:53] f 500000 steps  (0 percent)
[12:01:06] Writing local files
[12:01:06] Completed 5000 out of 500000 steps  (1 percent)
[12:08:16] Writing local files
[12:08:16] Completed 10000 out of 500000 steps  (2 percent)
[12:15:26] Writing local files
[12:15:26] Completed 15000 out of 500000 steps  (3 percent)
[12:22:39] Writing local files
[12:22:39] Completed 20000 out of 500000 steps  (4 percent)
[12:29:52] Writing local files
[12:29:52] Completed 25000 out of 500000 steps  (5 percent)
[12:37:10] Writing local files
[12:37:10] Completed 30000 out of 500000 steps  (6 percent)
[12:44:16] Writing local files
[12:44:16] Completed 35000 out of 500000 steps  (7 percent)
[12:51:37] Writing local files
[12:51:37] Completed 40000 out of 500000 steps  (8 percent)
[12:58:49] Writing local files
[12:58:49] Completed 45000 out of 500000 steps  (9 percent)
[13:06:00] Writing local files
[13:06:00] Completed 50000 out of 500000 steps  (10 percent)
[13:13:06] Writing local files
[13:13:06] Completed 55000 out of 500000 steps  (11 percent)
[13:20:26] Writing local files
[13:20:26] Completed 60000 out of 500000 steps  (12 percent)
[13:27:53] Writing local files
[13:27:53] Completed 65000 out of 500000 steps  (13 percent)
[13:35:00] Writing local files
[13:35:00] Completed 70000 out of 500000 steps  (14 percent)
[13:42:08] Writing local files
[13:42:09] Completed 75000 out of 500000 steps  (15 percent)
[13:49:24] Writing local files
[13:49:24] Completed 80000 out of 500000 steps  (16 percent)
[13:56:31] Writing local files
[13:56:31] Completed 85000 out of 500000 steps  (17 percent)
[14:03:37] Writing local files
[14:03:37] Completed 90000 out of 500000 steps  (18 percent)
[14:10:44] Writing local files
[14:10:44] Completed 95000 out of 500000 steps  (19 percent)
[14:17:58] Writing local files
[14:17:58] Completed 100000 out of 500000 steps  (20 percent)
[14:25:11] Writing local files
[14:25:11] Completed 105000 out of 500000 steps  (21 percent)
[14:32:22] Writing local files
[14:32:22] Completed 110000 out of 500000 steps  (22 percent)
[14:39:41] Writing local files
[14:39:41] Completed 115000 out of 500000 steps  (23 percent)
[14:46:58] Writing local files
[14:46:58] Completed 120000 out of 500000 steps  (24 percent)
[14:54:10] Writing local files
[14:54:10] Completed 125000 out of 500000 steps  (25 percent)
[15:01:29] Writing local files
[15:01:29] Completed 130000 out of 500000 steps  (26 percent)
[15:08:44] Writing local files
[15:08:44] Completed 135000 out of 500000 steps  (27 percent)
[15:15:59] Writing local files
[15:15:59] Completed 140000 out of 500000 steps  (28 percent)
[15:23:20] Writing local files
[15:23:20] Completed 145000 out of 500000 steps  (29 percent)
[15:30:45] Writing local files
[15:30:45] Completed 150000 out of 500000 steps  (30 percent)
[15:38:17] Writing local files
[15:38:17] Completed 155000 out of 500000 steps  (31 percent)
[15:45:36] Writing local files
[15:45:36] Completed 160000 out of 500000 steps  (32 percent)
[15:52:51] Writing local files
[15:52:51] Completed 165000 out of 500000 steps  (33 percent)
[16:00:17] Writing local files
[16:00:18] Completed 170000 out of 500000 steps  (34 percent)
[16:07:41] Writing local files
[16:07:41] Completed 175000 out of 500000 steps  (35 percent)
[16:14:53] Writing local files
[16:14:53] Completed 180000 out of 500000 steps  (36 percent)
[16:22:13] Writing local files
[16:22:13] Completed 185000 out of 500000 steps  (37 percent)
[16:29:34] Writing local files
[16:29:34] Completed 190000 out of 500000 steps  (38 percent)
[16:36:49] Writing local files
[16:36:49] Completed 195000 out of 500000 steps  (39 percent)
[16:44:03] Writing local files
[16:44:03] Completed 200000 out of 500000 steps  (40 percent)
[16:51:18] Writing local files
[16:51:18] Completed 205000 out of 500000 steps  (41 percent)
[16:58:34] Writing local files
[16:58:34] Completed 210000 out of 500000 steps  (42 percent)
[17:05:50] Writing local files
[17:05:50] Completed 215000 out of 500000 steps  (43 percent)
[17:13:10] Writing local files
[17:13:10] Completed 220000 out of 500000 steps  (44 percent)
[17:20:28] Writing local files
[17:20:28] Completed 225000 out of 500000 steps  (45 percent)
[17:27:44] Writing local files
[17:27:44] Completed 230000 out of 500000 steps  (46 percent)
[17:35:01] Writing local files
[17:35:01] Completed 235000 out of 500000 steps  (47 percent)
[17:42:17] Writing local files
[17:42:17] Completed 240000 out of 500000 steps  (48 percent)
[17:45:15] - Autosending finished units... [June 11 17:45:15 UTC]
[17:45:15] Trying to send all finished work units
[17:45:15] + No unsent completed units remaining.
[17:45:15] - Autosend completed
[17:49:43] Writing local files
[17:49:43] Completed 245000 out of 500000 steps  (49 percent)
[17:57:05] Writing local files
[17:57:05] Completed 250000 out of 500000 steps  (50 percent)
[18:04:17] Writing local files
[18:04:17] Completed 255000 out of 500000 steps  (51 percent)
[18:11:44] Writing local files
[18:11:44] Completed 260000 out of 500000 steps  (52 percent)
[18:19:07] Writing local files
[18:19:07] Completed 265000 out of 500000 steps  (53 percent)
[18:26:30] Writing local files
[18:26:30] Completed 270000 out of 500000 steps  (54 percent)
[18:33:50] Writing local files
[18:33:50] Completed 275000 out of 500000 steps  (55 percent)
[18:41:16] Writing local files
[18:41:16] Completed 280000 out of 500000 steps  (56 percent)
[18:48:29] Writing local files
[18:48:29] Completed 285000 out of 500000 steps  (57 percent)
[18:55:50] Writing local files
[18:55:50] Completed 290000 out of 500000 steps  (58 percent)
[19:03:01] Writing local files
[19:03:01] Completed 295000 out of 500000 steps  (59 percent)
[19:10:19] Writing local files
[19:10:20] Completed 300000 out of 500000 steps  (60 percent)
[19:17:29] Writing local files
[19:17:29] Completed 305000 out of 500000 steps  (61 percent)
[19:24:34] Writing local files
[19:24:34] Completed 310000 out of 500000 steps  (62 percent)
[19:31:49] Writing local files
[19:31:49] Completed 315000 out of 500000 steps  (63 percent)
[19:38:55] Writing local files
[19:38:56] Completed 320000 out of 500000 steps  (64 percent)
[19:46:03] Writing local files
[19:46:03] Completed 325000 out of 500000 steps  (65 percent)
[19:53:16] Writing local files
[19:53:16] Completed 330000 out of 500000 steps  (66 percent)
[20:00:32] Writing local files
[20:00:32] Completed 335000 out of 500000 steps  (67 percent)
[20:07:39] Writing local files
[20:07:40] Completed 340000 out of 500000 steps  (68 percent)
[20:15:05] Writing local files
[20:15:05] Completed 345000 out of 500000 steps  (69 percent)
[20:22:18] Writing local files
[20:22:18] Completed 350000 out of 500000 steps  (70 percent)
[20:29:44] Writing local files
[20:29:44] Completed 355000 out of 500000 steps  (71 percent)
[20:37:09] Writing local files
[20:37:09] Completed 360000 out of 500000 steps  (72 percent)
[20:44:32] Writing local files
[20:44:32] Completed 365000 out of 500000 steps  (73 percent)
[20:52:02] Writing local files
[20:52:02] Completed 370000 out of 500000 steps  (74 percent)
[20:59:24] Writing local files
[20:59:24] Completed 375000 out of 500000 steps  (75 percent)
[21:06:39] Writing local files
[21:06:39] Completed 380000 out of 500000 steps  (76 percent)
[21:13:50] Writing local files
[21:13:50] Completed 385000 out of 500000 steps  (77 percent)
[21:21:12] Writing local files
[21:21:12] Completed 390000 out of 500000 steps  (78 percent)
[21:28:26] Writing local files
[21:28:26] Completed 395000 out of 500000 steps  (79 percent)
[21:35:47] Writing local files
[21:35:47] Completed 400000 out of 500000 steps  (80 percent)
[21:43:08] Writing local files
[21:43:08] Completed 405000 out of 500000 steps  (81 percent)
[21:50:31] Writing local files
[21:50:31] Completed 410000 out of 500000 steps  (82 percent)
[21:57:52] Writing local files
[21:57:52] Completed 415000 out of 500000 steps  (83 percent)
[22:05:06] Writing local files
[22:05:06] Completed 420000 out of 500000 steps  (84 percent)
[22:12:13] Writing local files
[22:12:13] Completed 425000 out of 500000 steps  (85 percent)
[22:19:34] Writing local files
[22:19:34] Completed 430000 out of 500000 steps  (86 percent)
[22:26:57] Writing local files
[22:26:57] Completed 435000 out of 500000 steps  (87 percent)
[22:34:00] Writing local files
[22:34:01] Completed 440000 out of 500000 steps  (88 percent)
[22:41:19] Writing local files
[22:41:19] Completed 445000 out of 500000 steps  (89 percent)
[22:48:30] Writing local files
[22:48:30] Completed 450000 out of 500000 steps  (90 percent)
[22:55:53] Writing local files
[22:55:53] Completed 455000 out of 500000 steps  (91 percent)
[23:03:07] Writing local files
[23:03:07] Completed 460000 out of 500000 steps  (92 percent)
[23:10:21] Writing local files
[23:10:21] Completed 465000 out of 500000 steps  (93 percent)
[23:17:35] Writing local files
[23:17:35] Completed 470000 out of 500000 steps  (94 percent)
[23:24:50] Writing local files
[23:24:51] Completed 475000 out of 500000 steps  (95 percent)
[23:32:00] Writing local files
[23:32:00] Completed 480000 out of 500000 steps  (96 percent)
[23:39:08] Writing local files
[23:39:08] Completed 485000 out of 500000 steps  (97 percent)
[23:45:15] - Autosending finished units... [June 11 23:45:15 UTC]
[23:45:15] Trying to send all finished work units
[23:45:15] + No unsent completed units remaining.
[23:45:15] - Autosend completed
[23:46:14] Writing local files
[23:46:15] Completed 490000 out of 500000 steps  (98 percent)
[23:53:21] Writing local files
[23:53:21] Completed 495000 out of 500000 steps  (99 percent)
[00:00:34] Writing local files
[00:00:34] Completed 500000 out of 500000 steps  (100 percent)
[00:00:34] Writing final coordinates.
[00:00:34] Past main M.D. loop
[00:00:34] Will end MPI now
[00:01:34]
[00:01:34] Finished Work Unit:
[00:01:34] - Reading up to 3722208 from "work/wudata_08.arc": Read 3722208
[00:01:34] - Reading up to 1777452 from "work/wudata_08.xtc": Read 1777452
[00:01:34] goefile size: 0
[00:01:34] logfile size: 17700
[00:01:34] Leaving Run
[00:01:36] - Writing 5521760 bytes of core data to disk...
[00:01:36]   ... Done.
[00:01:36] - Failed to delete work/wudata_08.sas
[00:01:36] - Failed to delete work/wudata_08.goe
[00:01:36] Warning:  check for stray files
[00:01:36] - Shutting down core
[00:03:36]
[00:03:36] Folding@home Core Shutdown: FINISHED_UNIT
[00:03:36]
[00:03:36] Folding@home Core Shutdown: FINISHED_UNIT

Posted: **Fri Jun 12, 2009 12:20 am**

Restarted the client and none of the processes would cancel like normal. Had to "End Process" each in task manager before the client would start again.

Posted: **Fri Jun 12, 2009 12:31 am**

Sorry, noticed you asked what OS... Vista 64.

Posted: **Tue Jun 16, 2009 3:31 am**

Here is another series of 2653s running for a while then hangs

Code: Select all


--- Opening Log file [June 14 15:27:20 UTC] 


# Windows SMP Console Edition #################################################
###############################################################################

                       Folding@Home Client Version 6.23 Beta R1

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: C:\Program Files (x86)\Folding@Home Windows SMP Client V1.01
Executable: C:\Program Files (x86)\Folding@Home Windows SMP Client V1.01\FAH86.exe
Arguments: -smp -verbosity 9 

[15:27:20] - Ask before connecting: No
[15:27:20] - User name: Randman (Team 162247)
[15:27:20] - User ID: 417D5718532F2222
[15:27:20] - Machine ID: 1
[15:27:20] 
[15:27:21] Loaded queue successfully.
[15:27:21] 
[15:27:21] - Autosending finished units... [June 14 15:27:21 UTC]
[15:27:21] + Processing work unit
[15:27:21] Trying to send all finished work units
[15:27:21] Work type a1 not eligible for variable processors
[15:27:21] + No unsent completed units remaining.
[15:27:21] Core required: FahCore_a1.exe
[15:27:21] - Autosend completed
[15:27:21] Core found.
[15:27:21] Using generic mpiexec calls
[15:27:21] Working on queue slot 03 [June 14 15:27:21 UTC]
[15:27:21] + Working ...
[15:27:21] - Calling 'mpiexec -np 4 -channel auto -host 127.0.0.1 FahCore_a1.exe -dir work/ -suffix 03 -checkpoint 8 -verbose -lifeline 6008 -version 623'

[15:27:21] 
[15:27:21] *------------------------------*
[15:27:21] Folding@Home Gromacs SMP Core
[15:27:21] Version 1.74 (March 10, 2007)
[15:27:21] 
[15:27:21] Preparing to commence simulation
[15:27:21] - Ensuring status. Please wait.
[15:27:21] - Looking at optimizations...
[15:27:21] - Working with standard loops on this execution.
[15:27:21] - Previous termination of core was improper.
[15:27:21] - Going to use sta- Expanded 2437749 -> 12890749 - Expanded 2437749 -> 1289074- Starting from initiaProtein: Protein in POPC
[15:27:21] Writing local files
[15:27:21] Extra SSE boost OK.
[15:27:21] Past main M.D. loop
[15:27:21] 
[15:27:21] Finished Work Unit:
[15:27:21] Leaving Run
[15:27:21] - Shutting down core
[15:27:21] 
[15:27:21] Folding@home Core Shutdown: FINISHED_UNIT
[15:27:21] Finalizing output
[15:27:21]  down core
[15:27:21] 
[15:27:21] Folding@h- Shutting down core
[15:27:21] 
[15:27:21] Folding@home Core Shutdown: FINISHED_UNIT
[15:27:21] Finalizing output
[15:27:21] steps  (1 percent)
[15:27:21] Writing local files
[15:27:21] Completed 10000 out of 500000 steps  (2 percent)
[15:27:21] Writing local files
[15:27:21] Completed 15000 out of 500000 steps  (3 percent)
[15:27:21] Writing local files
[15:27:21] Completed 20000 out of 500000 steps  (4 percent)
[15:27:21] Writing local files
[15:27:21] Completed 25000 out of 500000 steps  (5 percent)
[15:27:21] Writing local files
[15:27:21] Completed 30000 out of 500000 steps  (6 percent)
[15:27:21] Writing local files
[15:27:21] Completed 35000 out of 500000 steps  (7 percent)
[15:27:21] Writing local files
[15:27:21] Completed 40000 out of 500000 steps  (8 percent)
[15:27:21] Writing local files
[15:27:21] Completed 45000 out of 500000 steps  (9 percent)
[15:27:21] Writing local files
[15:27:21] Completed 50000 out of 500000 steps  (10 percent)
[15:27:21] Writing local files
[15:27:21] Completed 55000 out of 500000 steps  (11 percent)
[15:27:21] Writing local files
[15:27:21] Completed 60000 out of 500000 steps  (12 percent)
[15:27:21] Writing local files
[15:27:21] Completed 65000 out of 500000 steps  (13 percent)
[15:27:21] Writing local files
[15:27:21] Completed 70000 out of 500000 steps  (14 percent)
[15:27:21] Writing local files
[15:27:21] Completed 75000 out of 500000 steps  (15 percent)
[15:27:21] Writing local files
[15:27:21] Completed 80000 out of 500000 steps  (16 percent)
[15:27:21] Writing local files
[15:27:21] Completed 85000 out of 500000 steps  (17 percent)
[15:27:21] Writing local files
[15:27:21] Completed 90000 out of 500000 steps  (18 percent)
[15:27:21] Writing local files
[15:27:21] Completed 95000 out of 500000 steps  (19 percent)
[15:27:21] Writing local files
[15:27:21] Completed 100000 out of 500000 steps  (20 percent)
[15:27:21] Writing local files
[15:27:21] Completed 105000 out of 500000 steps  (21 percent)
[15:27:21] Writing local files
[15:27:21] Completed 110000 out of 500000 steps  (22 percent)
[15:27:21] Writing local files
[15:27:21] Completed 115000 out of 500000 steps  (23 percent)
[15:27:21] Writing local files
[15:27:21] Completed 120000 out of 500000 steps  (24 percent)
[15:27:21] Writing local files
[15:27:21] Completed 125000 out of 500000 steps  (25 percent)
[15:27:21] Writing local files
[15:27:21] Completed 130000 out of 500000 steps  (26 percent)
[15:27:21] Writing local files
[15:27:21] Completed 135000 out of 500000 steps  (27 percent)
[15:27:21] Writing local files
[15:27:21] Completed 140000 out of 500000 steps  (28 percent)
[15:27:21] Writing local files
[15:27:21] Completed 145000 out of 500000 steps  (29 percent)
[15:27:21] Writing local files
[15:27:21] Completed 150000 out of 500000 steps  (30 percent)
[15:27:21] Writing local files
[15:27:21] Completed 155000 out of 500000 steps  (31 percent)
[15:27:21] Writing local files
[15:27:21] Completed 160000 out of 500000 steps  (32 percent)
[15:27:21] Writing local files
[15:27:21] Completed 165000 out of 500000 steps  (33 percent)
[15:27:21] Writing local files
[15:27:21] Completed 170000 out of 500000 steps  (34 percent)
[15:27:21] Writing local files
[15:27:21] Completed 175000 out of 500000 steps  (35 percent)
[15:27:21] Writing local files
[15:27:21] Completed 180000 out of 500000 steps  (36 percent)
[15:27:21] Writing local files
[15:27:21] Completed 185000 out of 500000 steps  (37 percent)
[15:27:21] Writing local files
[15:27:21] Completed 190000 out of 500000 steps  (38 percent)
[15:27:21] Writing local files
[15:27:21] Completed 195000 out of 500000 steps  (39 percent)
[15:27:21] Writing local files
[15:27:21] Completed 200000 out of 500000 steps  (40 percent)
[15:27:21] Writing local files
[15:27:21] Completed 205000 out of 500000 steps  (41 percent)
[15:27:21] Writing local files
[15:27:21] Completed 210000 out of 500000 steps  (42 percent)
[15:27:21] Writing local files
[15:27:21] Completed 215000 out of 500000 steps  (43 percent)
[15:27:21] Writing local files
[15:27:21] Completed 220000 out of 500000 steps  (44 percent)
[15:27:21] Writing local files
[15:27:21] Completed 225000 out of 500000 steps  (45 percent)
[15:27:21] Writing local files
[15:27:21] Completed 230000 out of 500000 steps  (46 percent)
[15:27:21] Writing local files
[15:27:21] Completed 235000 out of 500000 steps  (47 percent)
[15:27:21] Writing local files
[15:27:21] Completed 240000 out of 500000 steps  (48 percent)
[15:27:21] Writing local files
[15:27:21] Completed 245000 out of 500000 steps  (49 percent)
[15:27:21] Writing CoreStatus = 1 (1)
[15:29:42] Sending work to server
[15:29:42] Project: 2653 (Run 3, Clone 196, Gen 114)


[15:29:42] + Attempting to send results [June 14 15:29:42 UTC]
[15:29:42] - Reading file work/wuresults_03.dat from core
[15:29:42]   (Read 5523996 bytes from disk)
[15:29:42] Connecting to http://171.64.65.64:8080/
[15:30:26] Posted data.
[15:30:26] Initial: 0000; - Uploaded at ~117 kB/s
[15:30:28] - Averaged speed for that direction ~105 kB/s
[15:30:28] + Results successfully sent
[15:30:28] Thank you for your contribution to Folding@Home.
[15:30:48] - Warning: Could not delete all work unit files (3): Core returned invalid code
[15:30:48] Trying to send all finished work units
[15:30:48] + No unsent completed units remaining.
[15:30:48] - Preparing to get new work unit...
[15:30:48] + Attempting to get work packet
[15:30:48] - Will indicate memory of 1024 MB
[15:30:48] - Detect CPU. Vendor: GenuineIntel, Family: 6, Model: 10, Stepping: 4
[15:30:48] - Connecting to assignment server
[15:30:48] Connecting to http://assign.stanford.edu:8080/
[15:30:48] Posted data.
[15:30:48] Initial: 40AB; - Successful: assigned to (171.64.65.64).
[15:30:48] + News From Folding@Home: Welcome to Folding@Home
[15:30:49] Loaded queue successfully.
[15:30:49] Connecting to http://171.64.65.64:8080/
[15:30:52] Posted data.
[15:30:52] Initial: 0000; - Receiving payload (expected size: 2439923)
[15:30:54] - Downloaded at ~1191 kB/s
[15:30:54] - Averaged speed for that direction ~885 kB/s
[15:30:54] + Received work.
[15:30:54] Trying to send all finished work units
[15:30:54] + No unsent completed units remaining.
[15:30:54] + Closed connections
[15:30:59] 
[15:30:59] + Processing work unit
[15:30:59] Work type a1 not eligible for variable processors
[15:30:59] Core required: FahCore_a1.exe
[15:30:59] Core found.
[15:30:59] Using generic mpiexec calls
[15:30:59] Working on queue slot 04 [June 14 15:30:59 UTC]
[15:30:59] + Working ...
[15:30:59] - Calling 'mpiexec -np 4 -channel auto -host 127.0.0.1 FahCore_a1.exe -dir work/ -suffix 04 -checkpoint 8 -verbose -lifeline 6008 -version 623'

[15:31:00] 
[15:31:00] *------------------------------*
[15:31:00] Folding@Home Gromacs SMP Core
[15:31:00] Version 1.74 (March 10, 2007)
[15:31:00] 
[15:31:00] Preparing to commence simulation
[15:31:00] - Ensuring status. Please wait.
[15:31:17] - Looking at optimizations...
[15:31:17] - Working with standard loops on this execution.
[15:31:17] - Previous termination of core was improper.
[15:31:17] - Going to use standard loops.
[15:31:17] - Files status OK
[15:31:18] - Expanded 2439411 -> 12901785 (decompressed 528.8 percent)
[15:31:18] cket
[15:31:18] 
[15:31:18] Project: 2653 (Run 7, Clone 75, Gen 114)
[15:31:18] 
[15:31:18] 53 (Run 7, Clone 75, Gen 114)
[15:31:18] 
[15:31:19] Entering M.D.
[15:31:25] Rejecting checkpoint
[15:31:25] SE boost OK.
[15:31:25] ein in POPCExtra SSE boost OK.
[15:31:25] 
[15:31:26] Extra SSE boost OK.
[15:31:26] Writing local files
[15:31:26] Completed 0 out of 500000 steps  (0 percent)
[15:37:41] ***** Windows shutdown
[15:37:41] Killing all core threads
[15:37:41] Killing 3 cores
[15:37:41] Killing core 0
[15:37:41] Killing core 1
[15:37:41] Killing core 2

Folding@Home Client Shutdown at user request.
[15:37:41] ***** Got a SIGTERM signal (2)
[15:37:41] Killing all core threads
[15:37:41] Killing 3 cores
[15:37:41] Killing core 0
[15:37:41] Killing core 1
[15:37:41] Killing core 2

Folding@Home Client Shutdown.


--- Opening Log file [June 14 15:39:28 UTC] 


# Windows SMP Console Edition #################################################
###############################################################################

                       Folding@Home Client Version 6.23 Beta R1

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: C:\Program Files (x86)\Folding@Home Windows SMP Client V1.01
Executable: C:\Program Files (x86)\Folding@Home Windows SMP Client V1.01\FAH86.exe
Arguments: -smp -verbosity 9 

[15:39:28] - Ask before connecting: No
[15:39:28] - User name: Randman (Team 162247)
[15:39:28] - User ID: 417D5718532F2222
[15:39:28] - Machine ID: 1
[15:39:28] 
[15:39:29] Loaded queue successfully.
[15:39:29] 
[15:39:29] - Autosending finished units... [June 14 15:39:29 UTC]
[15:39:29] + Processing work unit
[15:39:29] Trying to send all finished work units
[15:39:29] Work type a1 not eligible for variable processors
[15:39:29] + No unsent completed units remaining.
[15:39:29] Core required: FahCore_a1.exe
[15:39:29] - Autosend completed
[15:39:29] Core found.
[15:39:29] Using generic mpiexec calls
[15:39:29] Working on queue slot 04 [June 14 15:39:29 UTC]
[15:39:29] + Working ...
[15:39:29] - Calling 'mpiexec -np 4 -channel auto -host 127.0.0.1 FahCore_a1.exe -dir work/ -suffix 04 -checkpoint 8 -verbose -lifeline 4872 -version 623'

[15:39:29] 
[15:39:29] *------------------------------*
[15:39:29] Folding@Home Gromacs SMP Core
[15:39:29] Version 1.74 (March 10, 2007)
[15:39:29] 
[15:39:29] Preparing to commence simulation
[15:39:29] - Ensuring status. Please wait.
[15:39:46] - Looking at optimizations...
[15:39:46] - Working with standard loops on this execution.
[15:39:46] - Previous termination of core was improper.
[15:39:46] - Going to use standard loops.
[15:39:46] - Files status OK
[15:39:47] - Expanded 2439411 -> 12901785 (decompressed 528.8 percent)
[15:39:50] 
[15:39:50] Project: 2653 (Run 7, Clone 75, Gen 114)
[15:39:50] 
[15:39:51] Entering M.D.
[15:39:58] Calling FAH init
[15:39:59] in POPC
[15:39:59] Writing local files
[15:39:59] Extra SSE boost OK.
[15:39:59] eckpoint
[15:39:59] Protein: Protein in POPC
[15:39:59] Writing local files
[15:39:59] Extra SSE boost OK.
[15:39:59] Writing local files
[15:39:59] Completed 0 out of 500000 steps  (0 percent)
[15:47:09] Writing local files
[15:47:10] Completed 5000 out of 500000 steps  (1 percent)
[15:54:09] Writing local files
[15:54:09] Completed 10000 out of 500000 steps  (2 percent)
[16:01:17] Writing local files
[16:01:17] Completed 15000 out of 500000 steps  (3 percent)
[16:08:23] Writing local files
[16:08:23] Completed 20000 out of 500000 steps  (4 percent)
[16:15:24] Writing local files
[16:15:24] Completed 25000 out of 500000 steps  (5 percent)
[16:22:26] Writing local files
[16:22:26] Completed 30000 out of 500000 steps  (6 percent)
[16:29:37] Writing local files
[16:29:37] Completed 35000 out of 500000 steps  (7 percent)
[16:36:51] Writing local files
[16:36:51] Completed 40000 out of 500000 steps  (8 percent)
[16:44:04] Writing local files
[16:44:04] Completed 45000 out of 500000 steps  (9 percent)
[16:51:25] Writing local files
[16:51:25] Completed 50000 out of 500000 steps  (10 percent)
[16:58:41] Writing local files
[16:58:41] Completed 55000 out of 500000 steps  (11 percent)
[17:05:54] Writing local files
[17:05:54] Completed 60000 out of 500000 steps  (12 percent)
[17:13:07] Writing local files
[17:13:07] Completed 65000 out of 500000 steps  (13 percent)
[17:20:23] Writing local files
[17:20:23] Completed 70000 out of 500000 steps  (14 percent)
[17:27:32] Writing local files
[17:27:32] Completed 75000 out of 500000 steps  (15 percent)
[17:34:45] Writing local files
[17:34:45] Completed 80000 out of 500000 steps  (16 percent)
[17:42:02] Writing local files
[17:42:02] Completed 85000 out of 500000 steps  (17 percent)
[17:49:16] Writing local files
[17:49:16] Completed 90000 out of 500000 steps  (18 percent)
[17:56:31] Writing local files
[17:56:31] Completed 95000 out of 500000 steps  (19 percent)
[18:03:48] Writing local files
[18:03:48] Completed 100000 out of 500000 steps  (20 percent)
[18:11:05] Writing local files
[18:11:05] Completed 105000 out of 500000 steps  (21 percent)
[18:18:25] Writing local files
[18:18:25] Completed 110000 out of 500000 steps  (22 percent)
[18:25:44] Writing local files
[18:25:45] Completed 115000 out of 500000 steps  (23 percent)
[18:33:04] Writing local files
[18:33:04] Completed 120000 out of 500000 steps  (24 percent)
[18:40:25] Writing local files
[18:40:25] Completed 125000 out of 500000 steps  (25 percent)
[18:47:40] Writing local files
[18:47:40] Completed 130000 out of 500000 steps  (26 percent)
[18:54:53] Writing local files
[18:54:54] Completed 135000 out of 500000 steps  (27 percent)
[19:02:10] Writing local files
[19:02:10] Completed 140000 out of 500000 steps  (28 percent)
[19:09:29] Writing local files
[19:09:29] Completed 145000 out of 500000 steps  (29 percent)
[19:16:47] Writing local files
[19:16:47] Completed 150000 out of 500000 steps  (30 percent)
[19:24:01] Writing local files
[19:24:01] Completed 155000 out of 500000 steps  (31 percent)
[19:31:15] Writing local files
[19:31:15] Completed 160000 out of 500000 steps  (32 percent)
[19:38:37] Writing local files
[19:38:37] Completed 165000 out of 500000 steps  (33 percent)
[19:45:54] Writing local files
[19:45:54] Completed 170000 out of 500000 steps  (34 percent)
[19:53:16] Writing local files
[19:53:16] Completed 175000 out of 500000 steps  (35 percent)
[20:00:36] Writing local files
[20:00:36] Completed 180000 out of 500000 steps  (36 percent)
[20:07:51] Writing local files
[20:07:51] Completed 185000 out of 500000 steps  (37 percent)
[20:15:07] Writing local files
[20:15:07] Completed 190000 out of 500000 steps  (38 percent)
[20:22:22] Writing local files
[20:22:22] Completed 195000 out of 500000 steps  (39 percent)
[20:29:33] Writing local files
[20:29:33] Completed 200000 out of 500000 steps  (40 percent)
[20:36:56] Writing local files
[20:36:56] Completed 205000 out of 500000 steps  (41 percent)
[20:44:14] Writing local files
[20:44:14] Completed 210000 out of 500000 steps  (42 percent)
[20:51:33] Writing local files
[20:51:33] Completed 215000 out of 500000 steps  (43 percent)
[20:58:53] Writing local files
[20:58:53] Completed 220000 out of 500000 steps  (44 percent)
[21:06:11] Writing local files
[21:06:11] Completed 225000 out of 500000 steps  (45 percent)
[21:13:31] Writing local files
[21:13:31] Completed 230000 out of 500000 steps  (46 percent)
[21:20:51] Writing local files
[21:20:51] Completed 235000 out of 500000 steps  (47 percent)
[21:28:16] Writing local files
[21:28:16] Completed 240000 out of 500000 steps  (48 percent)
[21:35:32] Writing local files
[21:35:32] Completed 245000 out of 500000 steps  (49 percent)
[21:39:28] - Autosending finished units... [June 14 21:39:28 UTC]
[21:39:28] Trying to send all finished work units
[21:39:28] + No unsent completed units remaining.
[21:39:28] - Autosend completed
[21:42:58] Writing local files
[21:42:58] Completed 250000 out of 500000 steps  (50 percent)
[21:50:16] Writing local files
[21:50:17] Completed 255000 out of 500000 steps  (51 percent)
[21:57:34] Writing local files
[21:57:34] Completed 260000 out of 500000 steps  (52 percent)
[22:04:46] Writing local files
[22:04:46] Completed 265000 out of 500000 steps  (53 percent)
[22:11:43] Writing local files
[22:11:43] Completed 270000 out of 500000 steps  (54 percent)
[22:18:53] Writing local files
[22:18:53] Completed 275000 out of 500000 steps  (55 percent)
[22:26:13] Writing local files
[22:26:13] Completed 280000 out of 500000 steps  (56 percent)
[22:33:36] Writing local files
[22:33:36] Completed 285000 out of 500000 steps  (57 percent)
[22:40:57] Writing local files
[22:40:57] Completed 290000 out of 500000 steps  (58 percent)
[22:48:16] Writing local files
[22:48:16] Completed 295000 out of 500000 steps  (59 percent)
[22:55:41] Writing local files
[22:55:41] Completed 300000 out of 500000 steps  (60 percent)
[23:03:05] Writing local files
[23:03:05] Completed 305000 out of 500000 steps  (61 percent)
[23:10:27] Writing local files
[23:10:27] Completed 310000 out of 500000 steps  (62 percent)
[23:17:49] Writing local files
[23:17:49] Completed 315000 out of 500000 steps  (63 percent)
[23:25:10] Writing local files
[23:25:10] Completed 320000 out of 500000 steps  (64 percent)
[23:32:32] Writing local files
[23:32:32] Completed 325000 out of 500000 steps  (65 percent)
[23:39:54] Writing local files
[23:39:54] Completed 330000 out of 500000 steps  (66 percent)
[23:47:09] Writing local files
[23:47:09] Completed 335000 out of 500000 steps  (67 percent)
[23:54:16] Writing local files
[23:54:16] Completed 340000 out of 500000 steps  (68 percent)
[00:01:31] Writing local files
[00:01:31] Completed 345000 out of 500000 steps  (69 percent)
[00:08:49] Writing local files
[00:08:49] Completed 350000 out of 500000 steps  (70 percent)
[00:16:13] Writing local files
[00:16:13] Completed 355000 out of 500000 steps  (71 percent)
[00:23:40] Writing local files
[00:23:40] Completed 360000 out of 500000 steps  (72 percent)
[00:31:05] Writing local files
[00:31:06] Completed 365000 out of 500000 steps  (73 percent)
[00:38:27] Writing local files
[00:38:27] Completed 370000 out of 500000 steps  (74 percent)
[00:45:36] Writing local files
[00:45:37] Completed 375000 out of 500000 steps  (75 percent)
[00:52:43] Writing local files
[00:52:43] Completed 380000 out of 500000 steps  (76 percent)
[01:00:03] Writing local files
[01:00:03] Completed 385000 out of 500000 steps  (77 percent)
[01:07:29] Writing local files
[01:07:29] Completed 390000 out of 500000 steps  (78 percent)
[01:14:52] Writing local files
[01:14:52] Completed 395000 out of 500000 steps  (79 percent)
[01:22:10] Writing local files
[01:22:10] Completed 400000 out of 500000 steps  (80 percent)
[01:29:35] Writing local files
[01:29:35] Completed 405000 out of 500000 steps  (81 percent)
[01:36:56] Writing local files
[01:36:56] Completed 410000 out of 500000 steps  (82 percent)
[01:44:13] Writing local files
[01:44:13] Completed 415000 out of 500000 steps  (83 percent)
[01:51:33] Writing local files
[01:51:33] Completed 420000 out of 500000 steps  (84 percent)
[01:58:50] Writing local files
[01:58:50] Completed 425000 out of 500000 steps  (85 percent)
[02:06:09] Writing local files
[02:06:09] Completed 430000 out of 500000 steps  (86 percent)
[02:13:28] Writing local files
[02:13:28] Completed 435000 out of 500000 steps  (87 percent)
[02:20:47] Writing local files
[02:20:47] Completed 440000 out of 500000 steps  (88 percent)
[02:28:08] Writing local files
[02:28:08] Completed 445000 out of 500000 steps  (89 percent)
[02:35:24] Writing local files
[02:35:24] Completed 450000 out of 500000 steps  (90 percent)
[02:42:43] Writing local files
[02:42:44] Completed 455000 out of 500000 steps  (91 percent)
[02:49:44] Writing local files
[02:49:44] Completed 460000 out of 500000 steps  (92 percent)
[02:56:56] Writing local files
[02:56:56] Completed 465000 out of 500000 steps  (93 percent)
[03:04:06] Writing local files
[03:04:06] Completed 470000 out of 500000 steps  (94 percent)
[03:11:21] Writing local files
[03:11:21] Completed 475000 out of 500000 steps  (95 percent)
[03:18:38] Writing local files
[03:18:38] Completed 480000 out of 500000 steps  (96 percent)
[03:25:54] Writing local files
[03:25:54] Completed 485000 out of 500000 steps  (97 percent)
[03:33:15] Writing local files
[03:33:15] Completed 490000 out of 500000 steps  (98 percent)
[03:39:26] - Autosending finished units... [June 15 03:39:26 UTC]
[03:39:26] Trying to send all finished work units
[03:39:26] + No unsent completed units remaining.
[03:39:26] - Autosend completed
[03:40:33] Writing local files
[03:40:33] Completed 495000 out of 500000 steps  (99 percent)
[03:47:54] Writing local files
[03:47:54] Completed 500000 out of 500000 steps  (100 percent)
[03:47:54] Writing final coordinates.
[03:47:54] Past main M.D. loop
[03:47:54] Will end MPI now
[03:48:54] 
[03:48:54] Finished Work Unit:
[03:48:54] - Reading up to 3724272 from "work/wudata_04.arc": Read 3724272
[03:48:54] - Reading up to 1942684 from "work/wudata_04.xtc": Read 1942684
[03:48:54] goefile size: 0
[03:48:54] logfile size: 25604
[03:48:54] Leaving Run
[03:48:55] - Writing 5698208 bytes of core data to disk...
[03:48:55]   ... Done.
[03:48:55] - Failed to delete work/wudata_04.sas
[03:48:55] - Failed to delete work/wudata_04.goe
[03:48:55] Warning:  check for stray files
[03:48:55] - Shutting down core
[03:50:55] 
[03:50:55] Folding@home Core Shutdown: FINISHED_UNIT
[03:50:55] 
[03:50:55] Folding@home Core Shutdown: FINISHED_UNIT
[03:50:57] CoreStatus = 64 (100)
[03:50:57] Unit 4 finished with 87 percent of time to deadline remaining.
[03:50:57] Updated performance fraction: 0.869556
[03:50:57] Sending work to server
[03:50:57] Project: 2653 (Run 7, Clone 75, Gen 114)


[03:50:57] + Attempting to send results [June 15 03:50:57 UTC]
[03:50:57] - Reading file work/wuresults_04.dat from core
[03:50:57]   (Read 5698208 bytes from disk)
[03:50:57] Connecting to http://171.64.65.64:8080/
[03:51:41] Posted data.
[03:51:41] Initial: 0000; - Uploaded at ~123 kB/s
[03:51:42] - Averaged speed for that direction ~108 kB/s
[03:51:42] + Results successfully sent
[03:51:42] Thank you for your contribution to Folding@Home.
[03:51:42] + Number of Units Completed: 22

[03:51:47] - Warning: Could not delete all work unit files (4): Core returned invalid code
[03:51:47] Trying to send all finished work units
[03:51:47] + No unsent completed units remaining.
[03:51:47] - Preparing to get new work unit...
[03:51:47] + Attempting to get work packet
[03:51:47] - Will indicate memory of 1024 MB
[03:51:47] - Detect CPU. Vendor: GenuineIntel, Family: 6, Model: 10, Stepping: 4
[03:51:47] - Connecting to assignment server
[03:51:47] Connecting to http://assign.stanford.edu:8080/
[03:51:47] Posted data.
[03:51:47] Initial: 40AB; - Successful: assigned to (171.64.65.64).
[03:51:47] + News From Folding@Home: Welcome to Folding@Home
[03:51:47] Loaded queue successfully.
[03:51:47] Connecting to http://171.64.65.64:8080/
[03:51:50] Posted data.
[03:51:50] Initial: 0000; - Receiving payload (expected size: 2442805)
[03:51:55] - Downloaded at ~477 kB/s
[03:51:55] - Averaged speed for that direction ~803 kB/s
[03:51:55] + Received work.
[03:51:56] Trying to send all finished work units
[03:51:56] + No unsent completed units remaining.
[03:51:56] + Closed connections
[03:51:56] 
[03:51:56] + Processing work unit
[03:51:56] Work type a1 not eligible for variable processors
[03:51:56] Core required: FahCore_a1.exe
[03:51:56] Core found.
[03:51:56] Using generic mpiexec calls
[03:51:56] Working on queue slot 05 [June 15 03:51:56 UTC]
[03:51:56] + Working ...
[03:51:56] - Calling 'mpiexec -np 4 -channel auto -host 127.0.0.1 FahCore_a1.exe -dir work/ -suffix 05 -checkpoint 8 -verbose -lifeline 4872 -version 623'

[03:51:56] 
[03:51:56] *------------------------------*
[03:51:56] Folding@Home Gromacs SMP Core
[03:51:56] Version 1.74 (March 10, 2007)
[03:51:56] 
[03:51:56] Preparing to commence simulation
[03:51:56] - Ensuring status. Please wait.
[03:51:57] - Starting from initial work packet
[03:51:57] 
[03:51:57] Project: 2653 (Run 13, Clone 89, Gen 114)
[03:51:57] 
[03:51:57] Assembly optimizations on if available.
[03:51:57] Entering M.D.
[03:52:14] l work pa- Starting from initial work packet
[03:52:14] 
[03:52:14] Project: 2653 (Run 13, Clone 89, Gen 114)
[03:52:14] 
[03:52:15] Entering M.D.
[03:52:20] Rejecting checkpoint
[03:52:21] SE boost OK.
[03:52:21] ein in POPCExtra SSE boost OK.
[03:52:21] 
[03:52:22] Extra SSE boost OK.
[03:52:22] Writing local files
[03:52:22] Completed 0 out of 500000 steps  (0 percent)
[03:59:47] Writing local files
[03:59:47] Completed 5000 out of 500000 steps  (1 percent)
[04:07:18] Writing local files
[04:07:18] Completed 10000 out of 500000 steps  (2 percent)
[04:14:46] Writing local files
[04:14:46] Completed 15000 out of 500000 steps  (3 percent)
[04:22:06] Writing local files
[04:22:06] Completed 20000 out of 500000 steps  (4 percent)
[04:29:21] Writing local files
[04:29:21] Completed 25000 out of 500000 steps  (5 percent)
[04:36:06] Writing local files
[04:36:06] Completed 30000 out of 500000 steps  (6 percent)
[04:43:11] Writing local files
[04:43:11] Completed 35000 out of 500000 steps  (7 percent)
[04:50:08] Writing local files
[04:50:08] Completed 40000 out of 500000 steps  (8 percent)
[04:57:16] Writing local files
[04:57:16] Completed 45000 out of 500000 steps  (9 percent)
[05:04:17] Writing local files
[05:04:17] Completed 50000 out of 500000 steps  (10 percent)
[05:11:14] Writing local files
[05:11:14] Completed 55000 out of 500000 steps  (11 percent)
[05:18:12] Writing local files
[05:18:12] Completed 60000 out of 500000 steps  (12 percent)
[05:25:12] Writing local files
[05:25:12] Completed 65000 out of 500000 steps  (13 percent)
[05:32:16] Writing local files
[05:32:16] Completed 70000 out of 500000 steps  (14 percent)
[05:39:14] Writing local files
[05:39:14] Completed 75000 out of 500000 steps  (15 percent)
[05:46:17] Writing local files
[05:46:17] Completed 80000 out of 500000 steps  (16 percent)
[05:53:15] Writing local files
[05:53:15] Completed 85000 out of 500000 steps  (17 percent)
[06:00:17] Writing local files
[06:00:17] Completed 90000 out of 500000 steps  (18 percent)
[06:07:14] Writing local files
[06:07:14] Completed 95000 out of 500000 steps  (19 percent)
[06:14:13] Writing local files
[06:14:13] Completed 100000 out of 500000 steps  (20 percent)
[06:21:08] Writing local files
[06:21:08] Completed 105000 out of 500000 steps  (21 percent)
[06:27:59] Writing local files
[06:27:59] Completed 110000 out of 500000 steps  (22 percent)
[06:34:41] Writing local files
[06:34:41] Completed 115000 out of 500000 steps  (23 percent)
[06:41:35] Writing local files
[06:41:35] Completed 120000 out of 500000 steps  (24 percent)
[06:48:29] Writing local files
[06:48:29] Completed 125000 out of 500000 steps  (25 percent)
[06:55:27] Writing local files
[06:55:27] Completed 130000 out of 500000 steps  (26 percent)
[07:02:18] Writing local files
[07:02:18] Completed 135000 out of 500000 steps  (27 percent)
[07:09:12] Writing local files
[07:09:12] Completed 140000 out of 500000 steps  (28 percent)
[07:16:04] Writing local files
[07:16:04] Completed 145000 out of 500000 steps  (29 percent)
[07:23:05] Writing local files
[07:23:06] Completed 150000 out of 500000 steps  (30 percent)
[07:29:55] Writing local files
[07:29:55] Completed 155000 out of 500000 steps  (31 percent)
[07:36:46] Writing local files
[07:36:46] Completed 160000 out of 500000 steps  (32 percent)
[07:43:49] Writing local files
[07:43:50] Completed 165000 out of 500000 steps  (33 percent)
[07:50:46] Writing local files
[07:50:46] Completed 170000 out of 500000 steps  (34 percent)
[07:57:41] Writing local files
[07:57:41] Completed 175000 out of 500000 steps  (35 percent)
[08:04:37] Writing local files
[08:04:38] Completed 180000 out of 500000 steps  (36 percent)
[08:11:30] Writing local files
[08:11:30] Completed 185000 out of 500000 steps  (37 percent)
[08:18:33] Writing local files
[08:18:33] Completed 190000 out of 500000 steps  (38 percent)
[08:25:21] Writing local files
[08:25:21] Completed 195000 out of 500000 steps  (39 percent)
[08:32:19] Writing local files
[08:32:19] Completed 200000 out of 500000 steps  (40 percent)
[08:39:20] Writing local files
[08:39:20] Completed 205000 out of 500000 steps  (41 percent)
[08:46:12] Writing local files
[08:46:12] Completed 210000 out of 500000 steps  (42 percent)
[08:53:09] Writing local files
[08:53:09] Completed 215000 out of 500000 steps  (43 percent)
[09:00:12] Writing local files
[09:00:12] Completed 220000 out of 500000 steps  (44 percent)
[09:07:14] Writing local files
[09:07:14] Completed 225000 out of 500000 steps  (45 percent)
[09:14:09] Writing local files
[09:14:09] Completed 230000 out of 500000 steps  (46 percent)
[09:20:55] Writing local files
[09:20:55] Completed 235000 out of 500000 steps  (47 percent)
[09:27:46] Writing local files
[09:27:46] Completed 240000 out of 500000 steps  (48 percent)
[09:34:37] Writing local files
[09:34:37] Completed 245000 out of 500000 steps  (49 percent)
[09:39:26] - Autosending finished units... [June 15 09:39:26 UTC]
[09:39:26] Trying to send all finished work units
[09:39:26] + No unsent completed units remaining.
[09:39:26] - Autosend completed
[09:41:28] Writing local files
[09:41:28] Completed 250000 out of 500000 steps  (50 percent)
[09:48:10] Writing local files
[09:48:10] Completed 255000 out of 500000 steps  (51 percent)
[09:55:11] Writing local files
[09:55:11] Completed 260000 out of 500000 steps  (52 percent)
[10:02:08] Writing local files
[10:02:08] Completed 265000 out of 500000 steps  (53 percent)
[10:09:10] Writing local files
[10:09:10] Completed 270000 out of 500000 steps  (54 percent)
[10:16:12] Writing local files
[10:16:13] Completed 275000 out of 500000 steps  (55 percent)
[10:23:13] Writing local files
[10:23:13] Completed 280000 out of 500000 steps  (56 percent)
[10:30:07] Writing local files
[10:30:07] Completed 285000 out of 500000 steps  (57 percent)
[10:36:59] Writing local files
[10:36:59] Completed 290000 out of 500000 steps  (58 percent)
[10:44:01] Writing local files
[10:44:01] Completed 295000 out of 500000 steps  (59 percent)
[10:50:56] Writing local files
[10:50:57] Completed 300000 out of 500000 steps  (60 percent)
[10:57:50] Writing local files
[10:57:51] Completed 305000 out of 500000 steps  (61 percent)
[11:04:42] Writing local files
[11:04:42] Completed 310000 out of 500000 steps  (62 percent)
[11:11:31] Writing local files
[11:11:31] Completed 315000 out of 500000 steps  (63 percent)
[11:18:29] Writing local files
[11:18:29] Completed 320000 out of 500000 steps  (64 percent)
[11:25:21] Writing local files
[11:25:21] Completed 325000 out of 500000 steps  (65 percent)
[11:32:14] Writing local files
[11:32:14] Completed 330000 out of 500000 steps  (66 percent)
[11:39:11] Writing local files
[11:39:11] Completed 335000 out of 500000 steps  (67 percent)
[11:46:13] Writing local files
[11:46:13] Completed 340000 out of 500000 steps  (68 percent)
[11:53:10] Writing local files
[11:53:10] Completed 345000 out of 500000 steps  (69 percent)
[12:00:09] Writing local files
[12:00:10] Completed 350000 out of 500000 steps  (70 percent)
[12:07:13] Writing local files
[12:07:13] Completed 355000 out of 500000 steps  (71 percent)
[12:14:11] Writing local files
[12:14:11] Completed 360000 out of 500000 steps  (72 percent)
[12:21:11] Writing local files
[12:21:12] Completed 365000 out of 500000 steps  (73 percent)
[12:28:16] Writing local files
[12:28:16] Completed 370000 out of 500000 steps  (74 percent)
[12:35:28] Writing local files
[12:35:28] Completed 375000 out of 500000 steps  (75 percent)
[12:42:35] Writing local files
[12:42:35] Completed 380000 out of 500000 steps  (76 percent)
[12:49:35] Writing local files
[12:49:36] Completed 385000 out of 500000 steps  (77 percent)
[12:56:39] Writing local files
[12:56:39] Completed 390000 out of 500000 steps  (78 percent)
[13:03:35] Writing local files
[13:03:35] Completed 395000 out of 500000 steps  (79 percent)
[13:10:35] Writing local files
[13:10:35] Completed 400000 out of 500000 steps  (80 percent)
[13:17:31] Writing local files
[13:17:31] Completed 405000 out of 500000 steps  (81 percent)
[13:24:36] Writing local files
[13:24:36] Completed 410000 out of 500000 steps  (82 percent)
[13:31:37] Writing local files
[13:31:37] Completed 415000 out of 500000 steps  (83 percent)
[13:38:38] Writing local files
[13:38:38] Completed 420000 out of 500000 steps  (84 percent)
[13:45:52] Writing local files
[13:45:52] Completed 425000 out of 500000 steps  (85 percent)
[13:53:05] Writing local files
[13:53:05] Completed 430000 out of 500000 steps  (86 percent)
[14:00:06] Writing local files
[14:00:06] Completed 435000 out of 500000 steps  (87 percent)
[14:07:09] Writing local files
[14:07:09] Completed 440000 out of 500000 steps  (88 percent)
[14:14:13] Writing local files
[14:14:13] Completed 445000 out of 500000 steps  (89 percent)
[14:21:20] Writing local files
[14:21:21] Completed 450000 out of 500000 steps  (90 percent)
[14:28:24] Writing local files
[14:28:24] Completed 455000 out of 500000 steps  (91 percent)
[14:35:30] Writing local files
[14:35:30] Completed 460000 out of 500000 steps  (92 percent)
[14:42:34] Writing local files
[14:42:34] Completed 465000 out of 500000 steps  (93 percent)
[14:49:33] Writing local files
[14:49:33] Completed 470000 out of 500000 steps  (94 percent)
[14:56:38] Writing local files
[14:56:38] Completed 475000 out of 500000 steps  (95 percent)
[15:03:41] Writing local files
[15:03:41] Completed 480000 out of 500000 steps  (96 percent)
[15:10:42] Writing local files
[15:10:42] Completed 485000 out of 500000 steps  (97 percent)
[15:17:42] Writing local files
[15:17:42] Completed 490000 out of 500000 steps  (98 percent)
[15:24:46] Writing local files
[15:24:46] Completed 495000 out of 500000 steps  (99 percent)
[15:31:49] Writing local files
[15:31:49] Completed 500000 out of 500000 steps  (100 percent)
[15:31:49] Writing final coordinates.
[15:31:49] Past main M.D. loop
[15:31:49] Will end MPI now
[15:32:49] 
[15:32:49] Finished Work Unit:
[15:32:49] - Reading up to 3721632 from "work/wudata_05.arc": Read 3721632
[15:32:49] - Reading up to 1776124 from "work/wudata_05.xtc": Read 1776124
[15:32:49] goefile size: 0
[15:32:49] logfile size: 17316
[15:32:49] Leaving Run
[15:32:50] - Writing 5519472 bytes of core data to disk...
[15:32:50]   ... Done.
[15:32:50] - Failed to delete work/wudata_05.sas
[15:32:50] - Failed to delete work/wudata_05.goe
[15:32:50] Warning:  check for stray files
[15:32:50] - Shutting down core
[15:34:50] 
[15:34:50] Folding@home Core Shutdown: FINISHED_UNIT
[15:34:50] 
[15:34:50] Folding@home Core Shutdown: FINISHED_UNIT
[15:34:56] CoreStatus = 64 (100)
[15:34:56] Unit 5 finished with 88 percent of time to deadline remaining.
[15:34:56] Updated performance fraction: 0.871234
[15:34:56] Sending work to server
[15:34:56] Project: 2653 (Run 13, Clone 89, Gen 114)


[15:34:56] + Attempting to send results [June 15 15:34:56 UTC]
[15:34:56] - Reading file work/wuresults_05.dat from core
[15:34:56]   (Read 5519472 bytes from disk)
[15:34:56] Connecting to http://171.64.65.64:8080/
[15:35:38] Posted data.
[15:35:38] Initial: 0000; - Uploaded at ~122 kB/s
[15:35:40] - Averaged speed for that direction ~111 kB/s
[15:35:40] + Results successfully sent
[15:35:40] Thank you for your contribution to Folding@Home.
[15:35:40] + Number of Units Completed: 23

[15:35:44] - Warning: Could not delete all work unit files (5): Core returned invalid code
[15:35:44] Trying to send all finished work units
[15:35:44] + No unsent completed units remaining.
[15:35:44] - Preparing to get new work unit...
[15:35:44] + Attempting to get work packet
[15:35:44] - Will indicate memory of 1024 MB
[15:35:44] - Connecting to assignment server
[15:35:44] Connecting to http://assign.stanford.edu:8080/
[15:35:44] Posted data.
[15:35:44] Initial: 40AB; - Successful: assigned to (171.64.65.64).
[15:35:44] + News From Folding@Home: Welcome to Folding@Home
[15:35:45] Loaded queue successfully.
[15:35:45] Connecting to http://171.64.65.64:8080/
[15:35:48] Posted data.
[15:35:48] Initial: 0000; - Receiving payload (expected size: 2433775)
[15:35:53] - Downloaded at ~475 kB/s
[15:35:53] - Averaged speed for that direction ~737 kB/s
[15:35:53] + Received work.
[15:35:53] Trying to send all finished work units
[15:35:53] + No unsent completed units remaining.
[15:35:53] + Closed connections
[15:35:53] 
[15:35:53] + Processing work unit
[15:35:53] Work type a1 not eligible for variable processors
[15:35:53] Core required: FahCore_a1.exe
[15:35:53] Core found.
[15:35:53] Using generic mpiexec calls
[15:35:53] Working on queue slot 06 [June 15 15:35:53 UTC]
[15:35:53] + Working ...
[15:35:53] - Calling 'mpiexec -np 4 -channel auto -host 127.0.0.1 FahCore_a1.exe -dir work/ -suffix 06 -checkpoint 8 -verbose -lifeline 4872 -version 623'

[15:35:53] 
[15:35:53] *------------------------------*
[15:35:53] Folding@Home Gromacs SMP Core
[15:35:53] Version 1.74 (March 10, 2007)
[15:35:53] 
[15:35:53] Preparing to commence simulation
[15:35:53] - Ensuring status. Please wait.
[15:35:54] - Starting from initial work packet
[15:35:54] 
[15:35:54] Project: 2653 (Run 10, Clone 101, Gen 114)
[15:35:54] 
[15:35:54] Assembly optimizations on if available.
[15:35:54] Entering M.D.
[15:36:11]  work pa- Starting from initial work packet
[15:36:11] 01, Gen 114)
[15:36:11] 
[15:36:11]  (Run 10, Clone 101, Gen 114)
[15:36:11] 
[15:36:12] Entering M.D.
[15:36:18] Rejecting checkpoint
[15:36:18] SE boost OK.
[15:36:18] ein in POPCExtra SSE boost OK.
[15:36:18] 
[15:36:19] Extra SSE boost OK.
[15:36:19] Writing local files
[15:36:19] Completed 0 out of 500000 steps  (0 percent)
[15:39:26] - Autosending finished units... [June 15 15:39:26 UTC]
[15:39:26] Trying to send all finished work units
[15:39:26] + No unsent completed units remaining.
[15:39:26] - Autosend completed
[15:43:17] Writing local files
[15:43:17] Completed 5000 out of 500000 steps  (1 percent)
[15:50:20] Writing local files
[15:50:20] Completed 10000 out of 500000 steps  (2 percent)
[15:57:21] Writing local files
[15:57:21] Completed 15000 out of 500000 steps  (3 percent)
[16:04:19] Writing local files
[16:04:19] Completed 20000 out of 500000 steps  (4 percent)
[16:11:20] Writing local files
[16:11:20] Completed 25000 out of 500000 steps  (5 percent)
[16:18:14] Writing local files
[16:18:14] Completed 30000 out of 500000 steps  (6 percent)
[16:25:05] Writing local files
[16:25:05] Completed 35000 out of 500000 steps  (7 percent)
[16:32:01] Writing local files
[16:32:01] Completed 40000 out of 500000 steps  (8 percent)
[16:38:54] Writing local files
[16:38:54] Completed 45000 out of 500000 steps  (9 percent)
[16:45:52] Writing local files
[16:45:52] Completed 50000 out of 500000 steps  (10 percent)
[16:52:41] Writing local files
[16:52:42] Completed 55000 out of 500000 steps  (11 percent)
[16:59:42] Writing local files
[16:59:42] Completed 60000 out of 500000 steps  (12 percent)
[17:06:42] Writing local files
[17:06:42] Completed 65000 out of 500000 steps  (13 percent)
[17:13:35] Writing local files
[17:13:35] Completed 70000 out of 500000 steps  (14 percent)
[17:20:33] Writing local files
[17:20:33] Completed 75000 out of 500000 steps  (15 percent)
[17:27:41] Writing local files
[17:27:41] Completed 80000 out of 500000 steps  (16 percent)
[17:34:39] Writing local files
[17:34:39] Completed 85000 out of 500000 steps  (17 percent)
[17:41:34] Writing local files
[17:41:34] Completed 90000 out of 500000 steps  (18 percent)
[17:48:33] Writing local files
[17:48:33] Completed 95000 out of 500000 steps  (19 percent)
[17:55:25] Writing local files
[17:55:25] Completed 100000 out of 500000 steps  (20 percent)
[18:02:20] Writing local files
[18:02:20] Completed 105000 out of 500000 steps  (21 percent)
[18:09:19] Writing local files
[18:09:19] Completed 110000 out of 500000 steps  (22 percent)
[18:16:07] Writing local files
[18:16:07] Completed 115000 out of 500000 steps  (23 percent)
[18:22:53] Writing local files
[18:22:53] Completed 120000 out of 500000 steps  (24 percent)
[18:29:45] Writing local files
[18:29:45] Completed 125000 out of 500000 steps  (25 percent)
[18:36:39] Writing local files
[18:36:39] Completed 130000 out of 500000 steps  (26 percent)
[18:43:33] Writing local files
[18:43:34] Completed 135000 out of 500000 steps  (27 percent)
[18:50:32] Writing local files
[18:50:32] Completed 140000 out of 500000 steps  (28 percent)
[18:57:25] Writing local files
[18:57:25] Completed 145000 out of 500000 steps  (29 percent)
[19:04:27] Writing local files
[19:04:27] Completed 150000 out of 500000 steps  (30 percent)
[19:11:26] Writing local files
[19:11:26] Completed 155000 out of 500000 steps  (31 percent)
[19:18:27] Writing local files
[19:18:27] Completed 160000 out of 500000 steps  (32 percent)
[19:25:24] Writing local files
[19:25:24] Completed 165000 out of 500000 steps  (33 percent)
[19:32:18] Writing local files
[19:32:18] Completed 170000 out of 500000 steps  (34 percent)
[19:39:21] Writing local files
[19:39:21] Completed 175000 out of 500000 steps  (35 percent)
[19:46:19] Writing local files
[19:46:19] Completed 180000 out of 500000 steps  (36 percent)
[19:53:12] Writing local files
[19:53:13] Completed 185000 out of 500000 steps  (37 percent)
[20:00:08] Writing local files
[20:00:09] Completed 190000 out of 500000 steps  (38 percent)
[20:06:56] Writing local files
[20:06:56] Completed 195000 out of 500000 steps  (39 percent)
[20:13:43] Writing local files
[20:13:43] Completed 200000 out of 500000 steps  (40 percent)
[20:20:39] Writing local files
[20:20:39] Completed 205000 out of 500000 steps  (41 percent)
[20:27:35] Writing local files
[20:27:35] Completed 210000 out of 500000 steps  (42 percent)
[20:34:32] Writing local files
[20:34:32] Completed 215000 out of 500000 steps  (43 percent)
[20:41:32] Writing local files
[20:41:32] Completed 220000 out of 500000 steps  (44 percent)
[20:48:27] Writing local files
[20:48:27] Completed 225000 out of 500000 steps  (45 percent)
[20:55:22] Writing local files
[20:55:23] Completed 230000 out of 500000 steps  (46 percent)
[21:02:12] Writing local files
[21:02:12] Completed 235000 out of 500000 steps  (47 percent)
[21:09:08] Writing local files
[21:09:09] Completed 240000 out of 500000 steps  (48 percent)
[21:16:00] Writing local files
[21:16:00] Completed 245000 out of 500000 steps  (49 percent)
[21:22:57] Writing local files
[21:22:57] Completed 250000 out of 500000 steps  (50 percent)
[21:29:55] Writing local files
[21:29:55] Completed 255000 out of 500000 steps  (51 percent)
[21:36:44] Writing local files
[21:36:44] Completed 260000 out of 500000 steps  (52 percent)
[21:39:26] - Autosending finished units... [June 15 21:39:26 UTC]
[21:39:26] Trying to send all finished work units
[21:39:26] + No unsent completed units remaining.
[21:39:26] - Autosend completed
[21:43:34] Writing local files
[21:43:35] Completed 265000 out of 500000 steps  (53 percent)
[21:50:26] Writing local files
[21:50:26] Completed 270000 out of 500000 steps  (54 percent)
[21:57:30] Writing local files
[21:57:30] Completed 275000 out of 500000 steps  (55 percent)
[22:04:22] Writing local files
[22:04:22] Completed 280000 out of 500000 steps  (56 percent)
[22:11:17] Writing local files
[22:11:17] Completed 285000 out of 500000 steps  (57 percent)
[22:18:15] Writing local files
[22:18:15] Completed 290000 out of 500000 steps  (58 percent)
[22:25:06] Writing local files
[22:25:06] Completed 295000 out of 500000 steps  (59 percent)
[22:32:00] Writing local files
[22:32:00] Completed 300000 out of 500000 steps  (60 percent)
[22:39:03] Writing local files
[22:39:03] Completed 305000 out of 500000 steps  (61 percent)
[22:46:04] Writing local files
[22:46:04] Completed 310000 out of 500000 steps  (62 percent)
[22:52:57] Writing local files
[22:52:57] Completed 315000 out of 500000 steps  (63 percent)
[22:59:59] Writing local files
[22:59:59] Completed 320000 out of 500000 steps  (64 percent)
[23:07:03] Writing local files
[23:07:03] Completed 325000 out of 500000 steps  (65 percent)
[23:14:04] Writing local files
[23:14:04] Completed 330000 out of 500000 steps  (66 percent)
[23:21:05] Writing local files
[23:21:05] Completed 335000 out of 500000 steps  (67 percent)
[23:28:02] Writing local files
[23:28:02] Completed 340000 out of 500000 steps  (68 percent)
[23:35:00] Writing local files
[23:35:00] Completed 345000 out of 500000 steps  (69 percent)
[23:41:55] Writing local files
[23:41:55] Completed 350000 out of 500000 steps  (70 percent)
[23:48:51] Writing local files
[23:48:51] Completed 355000 out of 500000 steps  (71 percent)
[23:55:51] Writing local files
[23:55:51] Completed 360000 out of 500000 steps  (72 percent)
[00:02:48] Writing local files
[00:02:48] Completed 365000 out of 500000 steps  (73 percent)
[00:09:50] Writing local files
[00:09:50] Completed 370000 out of 500000 steps  (74 percent)
[00:16:46] Writing local files
[00:16:46] Completed 375000 out of 500000 steps  (75 percent)
[00:23:48] Writing local files
[00:23:48] Completed 380000 out of 500000 steps  (76 percent)
[00:30:44] Writing local files
[00:30:45] Completed 385000 out of 500000 steps  (77 percent)
[00:37:45] Writing local files
[00:37:45] Completed 390000 out of 500000 steps  (78 percent)
[00:44:40] Writing local files
[00:44:40] Completed 395000 out of 500000 steps  (79 percent)
[00:51:36] Writing local files
[00:51:36] Completed 400000 out of 500000 steps  (80 percent)
[00:58:46] Writing local files
[00:58:46] Completed 405000 out of 500000 steps  (81 percent)
[01:05:43] Writing local files
[01:05:43] Completed 410000 out of 500000 steps  (82 percent)
[01:13:04] Writing local files
[01:13:04] Completed 415000 out of 500000 steps  (83 percent)
[01:20:18] Writing local files
[01:20:18] Completed 420000 out of 500000 steps  (84 percent)
[01:27:37] Writing local files
[01:27:37] Completed 425000 out of 500000 steps  (85 percent)
[01:34:47] Writing local files
[01:34:47] Completed 430000 out of 500000 steps  (86 percent)
[01:42:03] Writing local files
[01:42:03] Completed 435000 out of 500000 steps  (87 percent)
[01:49:20] Writing local files
[01:49:20] Completed 440000 out of 500000 steps  (88 percent)
[01:56:38] Writing local files
[01:56:38] Completed 445000 out of 500000 steps  (89 percent)
[02:03:58] Writing local files
[02:03:58] Completed 450000 out of 500000 steps  (90 percent)
[02:11:15] Writing local files
[02:11:15] Completed 455000 out of 500000 steps  (91 percent)
[02:18:33] Writing local files
[02:18:33] Completed 460000 out of 500000 steps  (92 percent)
[02:25:47] Writing local files
[02:25:47] Completed 465000 out of 500000 steps  (93 percent)
[02:33:06] Writing local files
[02:33:06] Completed 470000 out of 500000 steps  (94 percent)
[02:40:29] Writing local files
[02:40:29] Completed 475000 out of 500000 steps  (95 percent)
[02:47:48] Writing local files
[02:47:48] Completed 480000 out of 500000 steps  (96 percent)
[02:55:08] Writing local files
[02:55:08] Completed 485000 out of 500000 steps  (97 percent)
[03:02:26] Writing local files
[03:02:26] Completed 490000 out of 500000 steps  (98 percent)
[03:09:44] Writing local files
[03:09:44] Completed 495000 out of 500000 steps  (99 percent)
[03:17:01] Writing local files
[03:17:01] Completed 500000 out of 500000 steps  (100 percent)
[03:17:01] Writing final coordinates.
[03:17:01] Past main M.D. loop
[03:17:01] Will end MPI now
[03:18:02] 
[03:18:02] Finished Work Unit:
[03:18:02] - Reading up to 3729120 from "work/wudata_06.arc": Read 3729120
[03:18:02] - Reading up to 1786388 from "work/wudata_06.xtc": Read 1786388
[03:18:02] goefile size: 0
[03:18:02] logfile size: 17312
[03:18:02] Leaving Run
[03:18:02] - Writing 5537220 bytes of core data to disk...
[03:18:03]   ... Done.
[03:18:03] - Failed to delete work/wudata_06.sas
[03:18:03] - Failed to delete work/wudata_06.goe
[03:18:03] Warning:  check for stray files
[03:18:03] - Shutting down core
[03:20:03] 
[03:20:03] Folding@home Core Shutdown: FINISHED_UNIT
[03:20:03] 
[03:20:03] Folding@home Core Shutdown: FINISHED_UNIT
[03:27:24] Killing all core threads
[03:27:24] Killing 3 cores
[03:27:24] Killing core 0
[03:27:24] Killing core 1
[03:27:24] Killing core 2

Folding@Home Client Shutdown at user request.
[03:27:24] ***** Got a SIGTERM signal (2)
[03:27:24] Killing all core threads
[03:27:24] Killing 3 cores
[03:27:24] Killing core 0
[03:27:24] Killing core 1
[03:27:24] Killing core 2

Folding@Home Client Shutdown.

Folding Forum

Project 2653 Hangs after completion

Project 2653 Hangs after completion

Re: Project 2653 Hangs after completion

Re: Project 2653 Hangs after completion

Re: Project 2653 Hangs after completion

Re: Project 2653 Hangs after completion

Re: Project 2653 Hangs after completion

Re: Project 2653 Hangs after completion