Page 1 of 1

Project: 2665 (Run 2, Clone 903, Gen 42) multiple failures

Posted: Wed Sep 10, 2008 3:30 pm
by rjmiller
I have been assigned this work unit multiple times and they all fail in almost exactly the same place. I've tried just deleting to local files and rerunning, but it just gives me the same assignment. How do I get a different WU?

Code: Select all

# Windows SMP Console Edition #################################################
###############################################################################

                       Folding@Home Client Version 6.22 SMP Beta2

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: C:\FAH
Service: C:\FAH\fah6
Arguments: -svcstart -d C:\FAH -smp -deino -verbosity 9 

Launched as a service.
Entered C:\FAH to do work.

[09:54:11] - Ask before connecting: No
[09:54:11] - User name: rjmiller (Team 0)
[09:54:11] - User ID: 7730FE68050BB238
[09:54:11] - Machine ID: 1
[09:54:11] 
[09:54:11] Work directory not found. Creating...
[09:54:11] Could not open work queue, generating new queue...
[09:54:11] - Preparing to get new work unit...
[09:54:11] + Attempting to get work packet
[09:54:11] - Will indicate memory of 1024 MB
[09:54:11] - Detect CPU. Vendor: GenuineIntel, Family: 6, Model: 15, Stepping: 10
[09:54:11] - Connecting to assignment server
[09:54:11] Connecting to http://assign.stanford.edu:8080/
[09:54:11] - Autosending finished units... [September 10 09:54:11 UTC]
[09:54:11] Trying to send all finished work units
[09:54:11] + No unsent completed units remaining.
[09:54:11] - Autosend completed
[09:54:11] Posted data.
[09:54:11] Initial: 40AB; - Successful: assigned to (171.64.65.64).
[09:54:11] + News From Folding@Home: Welcome to Folding@Home
[09:54:12] Loaded queue successfully.
[09:54:12] Connecting to http://171.64.65.64:8080/
[09:54:17] Posted data.
[09:54:17] Initial: 0000; - Receiving payload (expected size: 4816737)
[09:54:40] - Downloaded at ~204 kB/s
[09:54:40] - Averaged speed for that direction ~204 kB/s
[09:54:40] + Received work.
[09:54:40] + Closed connections
[09:54:40] 
[09:54:40] + Processing work unit
[09:54:40] Work type a1 not eligible for variable processors
[09:54:40] Core required: FahCore_a1.exe
[09:54:40] Core found.
[09:54:40] Working on queue slot 01 [September 10 09:54:40 UTC]
[09:54:40] + Working ...
[09:54:40] - Calling 'mpiexec -np 4 -channel shm -env MPICH_USE_SMP_OPTIMIZATIONS 1 -host 127.0.0.1 FahCore_a1.exe -dir work/ -suffix 01 -checkpoint 15 -service -verbose -lifeline 2920 -version 622'

[09:54:43] 
[09:54:43] *------------------------------*
[09:54:43] Folding@Home Gromacs SMP Core
[09:54:43] Version 1.76 (February 23, 2008)
[09:54:43] 
[09:54:43] Preparing to commence simulation
[09:54:43] - Ensuring status. Please wait.
[09:54:51] - Starting from initial work packet
[09:54:52] 
[09:54:52] Project: 2665 (Run 2, Clone 903, Gen 42)
[09:54:52] 
[09:54:52] Assembly optimizations on if available.
[09:54:52] Entering M.D.
[09:55:16] al work packet
[09:55:16] 
[09:55:16] Project: 2665 (Run 2, Clone 903, Gen 42)
[09:55:16] 
[09:55:17] 65 (Run 2, Clone 903, Gen 42)
[09:55:17] 
[09:55:18] Entering M.D.
[09:55:30] Rejecting checkpoint
[09:55:32] cosylations
[09:55:32] Writing local files
[09:55:32] 
[09:55:32] Writing local files
[09:55:44] Extra SSE boost OK.
[09:55:45] Writing local files
[09:55:45] Completed 0 out of 250000 steps  (0 percent)
[10:10:46] Timered checkpoint triggered.
[10:25:46] Timered checkpoint triggered.
[10:27:16] Writing local files
[10:27:17] Completed 2500 out of 250000 steps  (1 percent)
[10:42:16] Timered checkpoint triggered.
[10:57:16] Timered checkpoint triggered.
[10:57:23] Writing local files
[10:57:23] Completed 5000 out of 250000 steps  (2 percent)
[11:12:24] Timered checkpoint triggered.
[11:27:25] Timered checkpoint triggered.
[11:27:31] Writing local files
[11:27:32] Completed 7500 out of 250000 steps  (3 percent)
[11:42:33] Timered checkpoint triggered.
[11:57:35] Timered checkpoint triggered.
[11:58:11] Writing local files
[11:58:11] Completed 10000 out of 250000 steps  (4 percent)
[12:13:12] Timered checkpoint triggered.
[12:28:13] Timered checkpoint triggered.
[12:28:20] Writing local files
[12:28:21] Completed 12500 out of 250000 steps  (5 percent)
[12:43:22] Timered checkpoint triggered.
[12:55:39] Gromacs cannot continue further.
[12:55:39] Going to send back what have done.
[12:55:39] logfile size: 19589
[12:55:39] - Writing 20125 bytes of core data to disk...
[12:55:39]   ... Done.
[12:55:40] - Failed to delete work/wudata_01.sas
[12:55:40] - Failed to delete work/wudata_01.goe
[12:55:40] Warning:  check for stray files
[12:57:40] 
[12:57:40] Folding@home Core Shutdown: EARLY_UNIT_END
[12:57:40] 
[12:57:40] Folding@home Core Shutdown: EARLY_UNIT_END
[12:57:45] CoreStatus = 63 (99)
[12:57:45] + Error starting Folding@Home core.
Every run looks the same. I have tried to let it run for at least 5 times and it just keeps failing. Any ideas?

Re: Project: 2665 (Run 2, Clone 903, Gen 42) multiple failures

Posted: Wed Sep 10, 2008 3:53 pm
by rjmiller
I figured out how to use qfix, but then when I restarted the client again it gave me the same WU again. So I'm not sure how to tell my machine, don't try this WU again. If anyone knows how to do this I would appreciate it.

Re: Project: 2665 (Run 2, Clone 903, Gen 42) multiple failures

Posted: Wed Sep 10, 2008 5:17 pm
by toTOW
This is apparently another bad WU ... :(

If you were able to send your partial results (I don't see them in the DB yet), you can delete both /work folder and queue.dat file. It should give you another WU.

Re: Project: 2665 (Run 2, Clone 903, Gen 42) multiple failures

Posted: Thu Sep 11, 2008 12:08 pm
by MDCRL
toTOW:

I have been having the same issues... Project: 2665 (Run 0, Clone 931, Gen 38)
I am however able to get the offending WU to 80% each time before it EUE's.
It has been chewing on the same WU since 09-02-08 unfortunately... I didn't realize until this morning what was actually been going on

Here are the logs in case it can help figure out what is going on....

Code: Select all

--- Opening Log file [September 2 03:02:47 UTC] 


# Windows SMP Console Edition #################################################
###############################################################################

                       Folding@Home Client Version 6.22 SMP Beta2r3

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: C:\Program Files\Folding@Home Windows SMP Client V1.01
Executable: C:\Program Files\Folding@Home Windows SMP Client V1.01\Folding@home-Win32-x86.exe
Arguments: -smp -verbosity 9 

[03:02:47] - Ask before connecting: No
[03:02:47] - User name: MDCRL (Team 35275)
[03:02:47] - User ID: 3FF6AE732BBDC17A
[03:02:47] - Machine ID: 1
[03:02:47] 
[03:02:48] Loaded queue successfully.
[03:02:48] 
[03:02:48] - Autosending finished units... [September 2 03:02:48 UTC]
[03:02:48] + Processing work unit
[03:02:48] Trying to send all finished work units
[03:02:48] Work type a1 not eligible for variable processors
[03:02:48] + No unsent completed units remaining.
[03:02:48] Core required: FahCore_a1.exe
[03:02:48] - Autosend completed
[03:02:48] Core found.
[03:02:48] Using generic mpiexec calls
[03:02:48] Working on queue slot 07 [September 2 03:02:48 UTC]
[03:02:48] + Working ...
[03:02:48] - Calling 'mpiexec -np 4 -channel auto -host 127.0.0.1 FahCore_a1.exe -dir work/ -suffix 07 -checkpoint 30 -verbose -lifeline 3768 -version 622'

[03:02:48] 
[03:02:48] *------------------------------*
[03:02:48] Folding@Home Gromacs SMP Core
[03:02:48] Version 1.74 (March 10, 2007)
[03:02:48] 
[03:02:48] Preparing to commence simulation
[03:02:48] - Ensuring status. Please wait.
[03:03:05] - Looking at optimizations...
[03:03:05] - Working with standard loops on this execution.
[03:03:05] - Previous termination of core was improper.
[03:03:05] - Going to use standard loops.
[03:03:05] - Files status OK
[03:05:05] 
[03:05:05] Folding@home Core Shutdown: MISSING_WORK_FILES
[03:05:05] Finalizing output
[03:05:08] CoreStatus = 1 (1)
[03:05:08] Client-core communications error: ERROR 0x1
[03:05:08] Deleting current work unit & continuing...
[03:05:08] Using generic mpiexec calls
[03:07:28] - Warning: Could not delete all work unit files (7): Core returned invalid code
[03:07:28] Trying to send all finished work units
[03:07:28] + No unsent completed units remaining.
[03:07:28] - Preparing to get new work unit...
[03:07:28] + Attempting to get work packet
[03:07:28] - Will indicate memory of 2046 MB
[03:07:28] - Detect CPU. Vendor: GenuineIntel, Family: 6, Model: 15, Stepping: 11
[03:07:28] - Connecting to assignment server
[03:07:28] Connecting to http://assign.stanford.edu:8080/
[03:07:28] Posted data.
[03:07:28] Initial: 40AB; - Successful: assigned to (171.64.65.64).
[03:07:28] + News From Folding@Home: Welcome to Folding@Home
[03:07:28] Loaded queue successfully.
[03:07:28] Connecting to http://171.64.65.64:8080/
[03:07:34] Posted data.
[03:07:34] Initial: 0000; - Receiving payload (expected size: 4739053)
[03:07:42] - Downloaded at ~578 kB/s
[03:07:42] - Averaged speed for that direction ~547 kB/s
[03:07:42] + Received work.
[03:07:42] + Closed connections
[03:07:47] 
[03:07:47] + Processing work unit
[03:07:47] Work type a1 not eligible for variable processors
[03:07:47] Core required: FahCore_a1.exe
[03:07:47] Core found.
[03:07:47] Using generic mpiexec calls
[03:07:47] Working on queue slot 08 [September 2 03:07:47 UTC]
[03:07:47] + Working ...
[03:07:47] - Calling 'mpiexec -np 4 -channel auto -host 127.0.0.1 FahCore_a1.exe -dir work/ -suffix 08 -checkpoint 30 -verbose -lifeline 3768 -version 622'

[03:07:48] 
[03:07:48] *------------------------------*
[03:07:48] Folding@Home Gromacs SMP Core
[03:07:48] Version 1.74 (March 10, 2007)
[03:07:48] 
[03:07:48] Preparing to commence simulation
[03:07:48] - Ensuring status. Please wait.
[03:08:05] - Looking at optimizations...
[03:08:05] - Working with standard loops on this execution.
[03:08:05] - Previous termination of core was improper.
[03:08:05] - Files status OK
[03:08:30] (decompressed 515.4 percent)
[03:08:30] - Starting from initial work packet
[03:08:30] 
[03:08:30] Project: 2665 (Run 0, Clone 931, Gen 38)
[03:08:30] 
[03:08:32] 65 (Run 0, Clone 931, Gen 38)
[03:08:32] 
[03:08:34] Entering M.D.
[03:08:40] Rejecting checkpoint
[03:08:42] Protein: HGG in water
[03:08:42] Writing local files
[03:08:54] Extra SSE boost OK.
[03:08:55] Writing local files
[03:08:55] Completed 0 out of 250000 steps  (0 percent)
[03:29:16] Writing local files
[03:29:17] Completed 2500 out of 250000 steps  (1 percent)
[03:49:37] Writing local files
[03:49:38] Completed 5000 out of 250000 steps  (2 percent)
[04:09:58] Writing local files
[04:09:58] Completed 7500 out of 250000 steps  (3 percent)
[04:30:16] Writing local files
[04:30:16] Completed 10000 out of 250000 steps  (4 percent)
[04:50:36] Writing local files
[04:50:37] Completed 12500 out of 250000 steps  (5 percent)
[05:10:58] Writing local files
[05:10:58] Completed 15000 out of 250000 steps  (6 percent)
[05:31:19] Writing local files
[05:31:20] Completed 17500 out of 250000 steps  (7 percent)
[05:51:40] Writing local files
[05:51:40] Completed 20000 out of 250000 steps  (8 percent)
[06:12:00] Writing local files
[06:12:00] Completed 22500 out of 250000 steps  (9 percent)
[06:32:21] Writing local files
[06:32:21] Completed 25000 out of 250000 steps  (10 percent)
[06:52:41] Writing local files
-------------------------------------------------------------------------------------
[02:47:55] Writing local files
[02:47:55] Completed 175000 out of 250000 steps  (70 percent)
[03:01:02] - Autosending finished units... [September 3 03:01:02 UTC]
[03:01:02] Trying to send all finished work units
[03:01:02] + No unsent completed units remaining.
[03:01:02] - Autosend completed
[03:08:10] Writing local files
[03:08:11] Completed 177500 out of 250000 steps  (71 percent)
[03:28:25] Writing local files
[03:28:25] Completed 180000 out of 250000 steps  (72 percent)
[03:48:39] Writing local files
[03:48:40] Completed 182500 out of 250000 steps  (73 percent)
[04:08:43] Writing local files
[04:08:43] Completed 185000 out of 250000 steps  (74 percent)
[04:28:54] Writing local files
[04:28:54] Completed 187500 out of 250000 steps  (75 percent)
[04:49:05] Writing local files
[04:49:05] Completed 190000 out of 250000 steps  (76 percent)
[05:09:16] Writing local files
[05:09:16] Completed 192500 out of 250000 steps  (77 percent)
[05:29:26] Writing local files
[05:29:27] Completed 195000 out of 250000 steps  (78 percent)
[05:49:38] Writing local files
[05:49:39] Completed 197500 out of 250000 steps  (79 percent)
[06:09:51] Writing local files
[06:09:51] Completed 200000 out of 250000 steps  (80 percent)
[06:16:44] Warning:  long 1-4 interactions
[06:16:45] Quit 101 - NaN detected: (ener[20])
[06:16:45] 
[06:16:45] Simulation instability has been encountered. The run has entered a
[06:16:45]   state from which no further progress can be made.
[06:16:45] This may be the correct result of the simulation, however if you
[06:16:45]   often see other project units terminating early like this
[06:16:45]   too, you may wish to check the stability of your computer (issues
[06:16:45]   such as high temperature, overclocking, etc.).
[06:16:45] Going to send back what have done.
[06:16:45] logfile size: 158723
[06:16:45] - Writing 159273 bytes of core data to disk...
[06:16:45]   ... Done.
[06:16:45] - Failed to delete work/wudata_08.arc
[06:16:46] No C.P. to delete.
[06:16:46] Warning:  check for stray files
[06:18:46] 
[06:18:46] Folding@home Core Shutdown: EARLY_UNIT_END
[06:18:46] 
[06:18:46] Folding@home Core Shutdown: EARLY_UNIT_END
[06:18:48] CoreStatus = 7B (123)
[06:18:48] Client-core communications error: ERROR 0x7b
[06:18:48] Deleting current work unit & continuing...
[06:18:48] Using generic mpiexec calls
[06:20:52] - Warning: Could not delete all work unit files (8): Core returned invalid code
[06:20:52] Trying to send all finished work units
[06:20:52] + No unsent completed units remaining.
[06:20:52] - Preparing to get new work unit...
[06:20:52] + Attempting to get work packet
[06:20:52] - Will indicate memory of 2046 MB
[06:20:52] - Connecting to assignment server
[06:20:52] Connecting to http://assign.stanford.edu:8080/
[06:20:53] Posted data.
[06:20:53] Initial: 40AB; - Successful: assigned to (171.64.65.64).
[06:20:53] + News From Folding@Home: Welcome to Folding@Home
[06:20:53] Loaded queue successfully.
[06:20:53] Connecting to http://171.64.65.64:8080/
[06:20:59] Posted data.
[06:20:59] Initial: 0000; - Receiving payload (expected size: 4739053)
[06:21:08] - Downloaded at ~514 kB/s
[06:21:08] - Averaged speed for that direction ~540 kB/s
[06:21:08] + Received work.
[06:21:08] + Closed connections
[06:21:13] 
[06:21:13] + Processing work unit
[06:21:13] Work type a1 not eligible for variable processors
[06:21:13] Core required: FahCore_a1.exe
[06:21:13] Core found.
[06:21:13] Using generic mpiexec calls
[06:21:13] Working on queue slot 09 [September 3 06:21:13 UTC]
[06:21:13] + Working ...
[06:21:13] - Calling 'mpiexec -np 4 -channel auto -host 127.0.0.1 FahCore_a1.exe -dir work/ -suffix 09 -checkpoint 30 -verbose -lifeline 3768 -version 622'

[06:21:13] 
[06:21:13] *------------------------------*
[06:21:13] Folding@Home Gromacs SMP Core
[06:21:13] Version 1.74 (March 10, 2007)
[06:21:13] 
[06:21:13] Preparing to commence simulation
[06:21:13] - Ensuring status. Please wait.
[06:21:20] - Starting from initial work packet
[06:21:20] 
[06:21:20] Project: 2665 (Run 0, Clone 931, Gen 38)
[06:21:20] 
[06:21:20] Assembly optimizations on if available.
[06:21:20] Entering M.D.
[06:21:51]  percent)
[06:21:51] - Starting from initial work packet
[06:21:51] 
[06:21:51] Project: 2665 (Run 0, Clone 931, Gen 38)
[06:21:51] 
[06:21:53] Entering M.D.
[06:21:59] Rejecting checkpoint
[06:22:01] Protein: HGG in water
[06:22:01] Writing local files
[06:22:13] Extra SSE boost OK.
[06:22:14] Writing local files
[06:22:14] Completed 0 out of 250000 steps  (0 percent)
[06:42:25] Writing local files
[06:42:26] Completed 2500 out of 250000 steps  (1 percent)
[07:02:39] Writing local files
[07:02:40] Completed 5000 out of 250000 steps  (2 percent)
[07:22:53] Writing local files
[07:22:53] Completed 7500 out of 250000 steps  (3 percent)
[07:43:04] Writing local files
[07:43:04] Completed 10000 out of 250000 steps  (4 percent)
[08:03:17] Writing local files
[08:03:17] Completed 12500 out of 250000 steps  (5 percent)
[08:23:31] Writing local files
[08:23:31] Completed 15000 out of 250000 steps  (6 percent)
[08:43:45] Writing local files
[08:43:46] Completed 17500 out of 250000 steps  (7 percent)
[09:01:02] - Autosending finished units... [September 3 09:01:02 UTC]
[09:01:02] Trying to send all finished work units
[09:01:02] + No unsent completed units remaining.
[09:01:02] - Autosend completed
[09:04:01] Writing local files
[09:04:01] Completed 20000 out of 250000 steps  (8 percent)
[09:24:15] Writing local files
[09:24:15] Completed 22500 out of 250000 steps  (9 percent)
-------------------------------------------------------------------------------------
[05:57:19] Writing local files
[05:57:20] Completed 175000 out of 250000 steps  (70 percent)
[06:17:30] Writing local files
[06:17:31] Completed 177500 out of 250000 steps  (71 percent)
[06:37:41] Writing local files
[06:37:41] Completed 180000 out of 250000 steps  (72 percent)
[06:57:51] Writing local files
[06:57:51] Completed 182500 out of 250000 steps  (73 percent)
[07:17:58] Writing local files
[07:17:58] Completed 185000 out of 250000 steps  (74 percent)
[07:38:07] Writing local files
[07:38:07] Completed 187500 out of 250000 steps  (75 percent)
[07:58:16] Writing local files
[07:58:17] Completed 190000 out of 250000 steps  (76 percent)
[08:18:25] Writing local files
[08:18:25] Completed 192500 out of 250000 steps  (77 percent)
[08:38:32] Writing local files
[08:38:32] Completed 195000 out of 250000 steps  (78 percent)
[08:58:42] Writing local files
[08:58:42] Completed 197500 out of 250000 steps  (79 percent)
[09:01:02] - Autosending finished units... [September 4 09:01:02 UTC]
[09:01:02] Trying to send all finished work units
[09:01:02] + No unsent completed units remaining.
[09:01:02] - Autosend completed
[09:18:52] Writing local files
[09:18:53] Completed 200000 out of 250000 steps  (80 percent)
[09:25:46] Warning:  long 1-4 interactions
[09:25:48] Quit 101 - NaN detected: (ener[20])
[09:25:48] 
[09:25:48] Simulation instability has been encountered. The run has entered a
[09:25:48]   state from which no further progress can be made.
[09:25:48] This may be the correct result of the simulation, however if you
[09:25:48]   often see other project units terminating early like this
[09:25:48]   too, you may wish to check the stability of your computer (issues
[09:25:48]   such as high temperature, overclocking, etc.).
[09:25:48] Going to send back what have done.
[09:25:48] logfile size: 158722
[09:25:48] - Writing 159272 bytes of core data to disk...
[09:25:48]   ... Done.
[09:25:48] - Failed to delete work/wudata_09.arc
[09:25:48] Warning:  check for stray files
[09:27:48] 
[09:27:48] Folding@home Core Shutdown: EARLY_UNIT_END
[09:27:48] 
[09:27:48] Folding@home Core Shutdown: EARLY_UNIT_END
[09:27:51] CoreStatus = 7B (123)
[09:27:51] Client-core communications error: ERROR 0x7b
[09:27:51] - Attempting to download new core...
[09:27:51] + Downloading new core: FahCore_a1.exe
[09:27:51] Downloading core (/~pande/Win32/x86/Core_a1.fah from www.stanford.edu)
[09:27:52] Initial: AFDE; + 10240 bytes downloaded
[09:27:52] Initial: AD21; + 20480 bytes downloaded
[09:27:52] Initial: CC38; + 30720 bytes downloaded
[09:27:52] Initial: 8501; + 40960 bytes downloaded
[09:27:52] Initial: F56A; + 51200 bytes downloaded
[09:27:52] Initial: ABAE; + 61440 bytes downloaded
[09:27:52] Initial: B6B0; + 71680 bytes downloaded
[09:27:52] Initial: 783A; + 81920 bytes downloaded
[09:27:52] Initial: B2A6; + 92160 bytes downloaded
[09:27:52] Initial: 1409; + 102400 bytes downloaded
[09:27:52] Initial: BBF0; + 112640 bytes downloaded
[09:27:52] Initial: 1861; + 122880 bytes downloaded
[09:27:52] Initial: 5950; + 133120 bytes downloaded
[09:27:52] Initial: 1081; + 143360 bytes downloaded
[09:27:52] Initial: 26BC; + 153600 bytes downloaded
[09:27:52] Initial: FE4A; + 163840 bytes downloaded
[09:27:52] Initial: C1C3; + 174080 bytes downloaded
[09:27:53] Initial: 9B49; + 184320 bytes downloaded
[09:27:53] Initial: 9EE5; + 194560 bytes downloaded
[09:27:53] Initial: D79D; + 204800 bytes downloaded
[09:27:53] Initial: 7801; + 215040 bytes downloaded
[09:27:53] Initial: 8B51; + 225280 bytes downloaded
[09:27:53] Initial: E26E; + 235520 bytes downloaded
[09:27:53] Initial: EDB0; + 245760 bytes downloaded
[09:27:53] Initial: 0919; + 256000 bytes downloaded
[09:27:53] Initial: CDDE; + 266240 bytes downloaded
[09:27:53] Initial: 7A7E; + 276480 bytes downloaded
[09:27:53] Initial: 034E; + 286720 bytes downloaded
[09:27:53] Initial: 88D0; + 296960 bytes downloaded
[09:27:53] Initial: D66D; + 307200 bytes downloaded
[09:27:53] Initial: 6A52; + 317440 bytes downloaded
[09:27:53] Initial: B478; + 327680 bytes downloaded
[09:27:53] Initial: CF8A; + 337920 bytes downloaded
[09:27:53] Initial: 8407; + 348160 bytes downloaded
[09:27:53] Initial: 2246; + 358400 bytes downloaded
[09:27:53] Initial: 1C69; + 368640 bytes downloaded
[09:27:53] Initial: 1287; + 378880 bytes downloaded
[09:27:53] Initial: 19B3; + 389120 bytes downloaded
[09:27:53] Initial: 1AD1; + 399360 bytes downloaded
[09:27:53] Initial: 5791; + 409600 bytes downloaded
[09:27:53] Initial: 76C5; + 419840 bytes downloaded
[09:27:53] Initial: 9B77; + 430080 bytes downloaded
[09:27:53] Initial: E82F; + 440320 bytes downloaded
[09:27:53] Initial: D0D3; + 450560 bytes downloaded
[09:27:53] Initial: 0F5E; + 460800 bytes downloaded
[09:27:53] Initial: D743; + 471040 bytes downloaded
[09:27:53] Initial: 0B7C; + 481280 bytes downloaded
[09:27:53] Initial: FAFD; + 491520 bytes downloaded
[09:27:53] Initial: 0E14; + 501760 bytes downloaded
[09:27:53] Initial: 4048; + 512000 bytes downloaded
[09:27:53] Initial: 21A5; + 522240 bytes downloaded
[09:27:53] Initial: C1A5; + 532480 bytes downloaded
[09:27:53] Initial: F716; + 542720 bytes downloaded
[09:27:53] Initial: DD98; + 552960 bytes downloaded
[09:27:53] Initial: 9F7B; + 563200 bytes downloaded
[09:27:53] Initial: 1CC0; + 573440 bytes downloaded
[09:27:53] Initial: 4D37; + 583680 bytes downloaded
[09:27:53] Initial: 222A; + 593920 bytes downloaded
[09:27:53] Initial: 8E33; + 604160 bytes downloaded
[09:27:53] Initial: D3C9; + 614400 bytes downloaded
[09:27:53] Initial: 9821; + 624640 bytes downloaded
[09:27:53] Initial: 236E; + 634880 bytes downloaded
[09:27:53] Initial: 1A7A; + 645120 bytes downloaded
[09:27:53] Initial: 6D64; + 655360 bytes downloaded
[09:27:53] Initial: 4ADC; + 665600 bytes downloaded
[09:27:53] Initial: 3854; + 675840 bytes downloaded
[09:27:53] Initial: CB5C; + 686080 bytes downloaded
[09:27:53] Initial: 2A88; + 696320 bytes downloaded
[09:27:53] Initial: 1199; + 706560 bytes downloaded
[09:27:53] Initial: 0512; + 716800 bytes downloaded
[09:27:53] Initial: 316E; + 727040 bytes downloaded
[09:27:53] Initial: D89D; + 737280 bytes downloaded
[09:27:53] Initial: E6A3; + 747520 bytes downloaded
[09:27:53] Initial: B488; + 757760 bytes downloaded
[09:27:53] Initial: BAFD; + 768000 bytes downloaded
[09:27:53] Initial: 34A0; + 778240 bytes downloaded
[09:27:53] Initial: DD6C; + 788480 bytes downloaded
[09:27:53] Initial: D2E9; + 789667 bytes downloaded
[09:27:53] Verifying core Core_a1.fah...
[09:27:53] Signature is VALID
[09:27:53] 
[09:27:53] Trying to unzip core FahCore_a1.exe
[09:27:54] Decompressed FahCore_a1.exe (2035712 bytes) successfully
[09:27:59] + Core successfully engaged
[09:27:59] Deleting current work unit & continuing...
[09:27:59] Using generic mpiexec calls
[09:30:03] - Warning: Could not delete all work unit files (9): Core returned invalid code
[09:30:03] Trying to send all finished work units
[09:30:03] + No unsent completed units remaining.
[09:30:03] - Preparing to get new work unit...
[09:30:03] + Attempting to get work packet
[09:30:03] - Will indicate memory of 2046 MB
[09:30:03] - Connecting to assignment server
[09:30:03] Connecting to http://assign.stanford.edu:8080/
[09:30:03] Posted data.
[09:30:03] Initial: 40AB; - Successful: assigned to (171.64.65.64).
[09:30:03] + News From Folding@Home: Welcome to Folding@Home
[09:30:03] Loaded queue successfully.
[09:30:03] Connecting to http://171.64.65.64:8080/
[09:30:09] Posted data.
[09:30:09] Initial: 0000; - Receiving payload (expected size: 4739053)
[09:30:18] - Downloaded at ~514 kB/s
[09:30:18] - Averaged speed for that direction ~535 kB/s
[09:30:18] + Received work.
[09:30:18] + Closed connections
[09:30:23] 
[09:30:23] + Processing work unit
[09:30:23] Work type a1 not eligible for variable processors
[09:30:23] Core required: FahCore_a1.exe
[09:30:23] Core found.
[09:30:23] Using generic mpiexec calls
[09:30:23] Working on queue slot 00 [September 4 09:30:23 UTC]
[09:30:23] + Working ...
[09:30:23] - Calling 'mpiexec -np 4 -channel auto -host 127.0.0.1 FahCore_a1.exe -dir work/ -suffix 00 -checkpoint 30 -verbose -lifeline 3768 -version 622'

[09:30:23] 
[09:30:23] *------------------------------*
[09:30:23] Folding@Home Gromacs SMP Core
[09:30:23] Version 1.74 (March 10, 2007)
[09:30:23] 
[09:30:23] Preparing to commence simulation
[09:30:23] - Ensuring status. Please wait.
[09:30:29] - Starting from initial work packet
[09:30:29] 
[09:30:29] Project: 2665 (Run 0, Clone 931, Gen 38)
[09:30:29] 
[09:30:30] Assembly optimizations on if available.
[09:30:30] Entering M.D.
[09:31:00]  percent)
[09:31:00] cket
[09:31:00] 
[09:31:00] Project: 2665 (Run 0, Clone 931, Gen 38)
[09:31:00] 
[09:31:01] 65 (Run 0, Clone 931, Gen 38)
[09:31:01] 
[09:31:02] Entering M.D.
[09:31:08] Rejecting checkpoint
[09:31:10] Protein: HGG in water
[09:31:10] Writing local files
[09:31:22] Extra SSE boost OK.
[09:31:23] Writing local files
[09:31:23] Completed 0 out of 250000 steps  (0 percent)
[09:51:38] Writing local files
[09:51:38] Completed 2500 out of 250000 steps  (1 percent)
[10:11:58] Writing local files
[10:11:59] Completed 5000 out of 250000 steps  (2 percent)
[10:32:18] Writing local files
[10:32:18] Completed 7500 out of 250000 steps  (3 percent)
[10:52:35] Writing local files
[10:52:35] Completed 10000 out of 250000 steps  (4 percent)
[11:12:56] Writing local files
[11:12:56] Completed 12500 out of 250000 steps  (5 percent)
[11:33:18] Writing local files
[11:33:18] Completed 15000 out of 250000 steps  (6 percent)
[11:53:40] Writing local files
[11:53:40] Completed 17500 out of 250000 steps  (7 percent)
[12:14:02] Writing local files
[12:14:02] Completed 20000 out of 250000 steps  (8 percent)
[12:34:24] Writing local files
[12:34:24] Completed 22500 out of 250000 steps  (9 percent)
[12:54:44] Writing local files
[12:54:44] Completed 25000 out of 250000 steps  (10 percent)
----------------------------------------------------------------------------------------------
[09:01:02] - Autosending finished units... [September 5 09:01:02 UTC]
[09:01:02] Trying to send all finished work units
[09:01:02] + No unsent completed units remaining.
[09:01:02] - Autosend completed
[09:13:53] Writing local files
[09:13:54] Completed 175000 out of 250000 steps  (70 percent)
[09:34:20] Writing local files
[09:34:20] Completed 177500 out of 250000 steps  (71 percent)
[09:54:44] Writing local files
[09:54:44] Completed 180000 out of 250000 steps  (72 percent)
[10:15:02] Writing local files
[10:15:02] Completed 182500 out of 250000 steps  (73 percent)
[10:35:17] Writing local files
[10:35:17] Completed 185000 out of 250000 steps  (74 percent)
[10:55:32] Writing local files
[10:55:33] Completed 187500 out of 250000 steps  (75 percent)
[11:15:49] Writing local files
[11:15:50] Completed 190000 out of 250000 steps  (76 percent)
[11:36:06] Writing local files
[11:36:07] Completed 192500 out of 250000 steps  (77 percent)
[11:56:24] Writing local files
[11:56:24] Completed 195000 out of 250000 steps  (78 percent)
[12:16:42] Writing local files
[12:16:42] Completed 197500 out of 250000 steps  (79 percent)
[12:37:01] Writing local files
[12:37:02] Completed 200000 out of 250000 steps  (80 percent)
[12:43:58] Warning:  long 1-4 interactions
[12:43:59] Quit 101 - NaN detected: (ener[20])
[12:43:59] 
[12:43:59] Simulation instability has been encountered. The run has entered a
[12:43:59]   state from which no further progress can be made.
[12:43:59] This may be the correct result of the simulation, however if you
[12:43:59]   often see other project units terminating early like this
[12:43:59]   too, you may wish to check the stability of your computer (issues
[12:43:59]   such as high temperature, overclocking, etc.).
[12:43:59] Going to send back what have done.
[12:43:59] logfile size: 158723
[12:43:59] - Writing 159273 bytes of core data to disk...
[12:43:59]   ... Done.
[12:43:59] - Failed to delete work/wudata_00.arc
[12:44:00] No C.P. to delete.
[12:44:01] Warning:  check for stray files
[12:44:01] 
[12:44:01] Folding@home Core Shutdown: EARLY_UNIT_END
[12:44:01] 
[12:44:01] Folding@home Core Shutdown: EARLY_UNIT_END
[12:44:05] CoreStatus = 7B (123)
[12:44:05] Client-core communications error: ERROR 0x7b
[12:44:05] Deleting current work unit & continuing...
[12:44:05] Using generic mpiexec calls
[12:46:09] - Warning: Could not delete all work unit files (0): Core returned invalid code
[12:46:09] Trying to send all finished work units
[12:46:09] + No unsent completed units remaining.
[12:46:09] - Preparing to get new work unit...
[12:46:09] + Attempting to get work packet
[12:46:09] - Will indicate memory of 2046 MB
[12:46:09] - Connecting to assignment server
[12:46:09] Connecting to http://assign.stanford.edu:8080/
[12:46:09] Posted data.
[12:46:09] Initial: 40AB; - Successful: assigned to (171.64.65.64).
[12:46:09] + News From Folding@Home: Welcome to Folding@Home
[12:46:10] Loaded queue successfully.
[12:46:10] Connecting to http://171.64.65.64:8080/
[12:46:15] Posted data.
[12:46:15] Initial: 0000; - Receiving payload (expected size: 4739053)
[12:46:24] - Downloaded at ~514 kB/s
[12:46:24] - Averaged speed for that direction ~531 kB/s
[12:46:24] + Received work.
[12:46:24] + Closed connections
[12:46:29] 
[12:46:29] + Processing work unit
[12:46:29] Work type a1 not eligible for variable processors
[12:46:29] Core required: FahCore_a1.exe
[12:46:29] Core found.
[12:46:29] Using generic mpiexec calls
[12:46:29] Working on queue slot 01 [September 5 12:46:29 UTC]
[12:46:29] + Working ...
[12:46:29] - Calling 'mpiexec -np 4 -channel auto -host 127.0.0.1 FahCore_a1.exe -dir work/ -suffix 01 -checkpoint 30 -verbose -lifeline 3768 -version 622'

[12:46:29] 
[12:46:29] *------------------------------*
[12:46:29] Folding@Home Gromacs SMP Core
[12:46:29] Version 1.74 (March 10, 2007)
[12:46:29] 
[12:46:29] Preparing to commence simulation
[12:46:29] - Ensuring status. Please wait.
[12:46:36] - Starting from initial work packet
[12:46:36] 
[12:46:36] Project: 2665 (Run 0, Clone 931, Gen 38)
[12:46:36] 
[12:46:36] Assembly optimizations on if available.
[12:46:36] Entering M.D.
[12:47:07]  percent)
[12:47:07] - Starting from initial work packet
[12:47:07] 
[12:47:07] Project: 2665 (Run 0, Clone 931, Gen 38)
[12:47:07] 
[12:47:08] Entering M.D.
[12:47:14] Rejecting checkpoint
[12:47:16] Protein: HGG in water
[12:47:17] Writing local files
[12:47:28] Extra SSE boost OK.
[12:47:29] Writing local files
[12:47:29] Completed 0 out of 250000 steps  (0 percent)
[13:07:48] Writing local files
[13:07:48] Completed 2500 out of 250000 steps  (1 percent)
[13:28:07] Writing local files
[13:28:07] Completed 5000 out of 250000 steps  (2 percent)
[13:48:26] Writing local files
[13:48:26] Completed 7500 out of 250000 steps  (3 percent)
[14:08:44] Writing local files
[14:08:44] Completed 10000 out of 250000 steps  (4 percent)
[14:29:03] Writing local files
[14:29:03] Completed 12500 out of 250000 steps  (5 percent)
[14:49:22] Writing local files
[14:49:22] Completed 15000 out of 250000 steps  (6 percent)
[15:01:02] - Autosending finished units... [September 5 15:01:02 UTC]
[15:01:02] Trying to send all finished work units
[15:01:02] + No unsent completed units remaining.
[15:01:02] - Autosend completed
[15:09:40] Writing local files
[15:09:41] Completed 17500 out of 250000 steps  (7 percent)
[15:30:00] Writing local files
[15:30:00] Completed 20000 out of 250000 steps  (8 percent)
[15:50:19] Writing local files
[15:50:19] Completed 22500 out of 250000 steps  (9 percent)
[16:10:37] Writing local files
[16:10:38] Completed 25000 out of 250000 steps  (10 percent)
------------------------------------------------------------------------------------------
[12:31:09] Writing local files
[12:31:09] Completed 175000 out of 250000 steps  (70 percent)
[12:51:25] Writing local files
[12:51:25] Completed 177500 out of 250000 steps  (71 percent)
[13:11:39] Writing local files
[13:11:39] Completed 180000 out of 250000 steps  (72 percent)
[13:31:54] Writing local files
[13:31:54] Completed 182500 out of 250000 steps  (73 percent)
[13:52:05] Writing local files
[13:52:06] Completed 185000 out of 250000 steps  (74 percent)
[14:12:18] Writing local files
[14:12:18] Completed 187500 out of 250000 steps  (75 percent)
[14:32:32] Writing local files
[14:32:32] Completed 190000 out of 250000 steps  (76 percent)
[14:52:45] Writing local files
[14:52:45] Completed 192500 out of 250000 steps  (77 percent)
[15:01:02] - Autosending finished units... [September 6 15:01:02 UTC]
[15:01:02] Trying to send all finished work units
[15:01:02] + No unsent completed units remaining.
[15:01:02] - Autosend completed
[15:12:58] Writing local files
[15:12:58] Completed 195000 out of 250000 steps  (78 percent)
[15:33:12] Writing local files
[15:33:12] Completed 197500 out of 250000 steps  (79 percent)
[15:53:26] Writing local files
[15:53:27] Completed 200000 out of 250000 steps  (80 percent)
[16:00:20] Warning:  long 1-4 interactions
[16:00:22] Quit 101 - NaN detected: (ener[20])
[16:00:22] 
[16:00:22] Simulation instability has been encountered. The run has entered a
[16:00:22]   state from which no further progress can be made.
[16:00:22] This may be the correct result of the simulation, however if you
[16:00:22]   often see other project units terminating early like this
[16:00:22]   too, you may wish to check the stability of your computer (issues
[16:00:22]   such as high temperature, overclocking, etc.).
[16:00:22] Going to send back what have done.
[16:00:22] logfile size: 158723
[16:00:22] - Writing 159273 bytes of core data to disk...
[16:00:22]   ... Done.
[16:02:22] 
[16:02:22] Folding@home Core Shutdown: EARLY_UNIT_END
[16:02:22] 
[16:02:22] Folding@home Core Shutdown: EARLY_UNIT_END
[16:02:25] CoreStatus = 7B (123)
[16:02:25] Client-core communications error: ERROR 0x7b
[16:02:25] Deleting current work unit & continuing...
[16:02:25] Using generic mpiexec calls
[16:04:29] - Warning: Could not delete all work unit files (1): Core returned invalid code
[16:04:29] Trying to send all finished work units
[16:04:29] + No unsent completed units remaining.
[16:04:29] - Preparing to get new work unit...
[16:04:29] + Attempting to get work packet
[16:04:29] - Will indicate memory of 2046 MB
[16:04:29] - Connecting to assignment server
[16:04:29] Connecting to http://assign.stanford.edu:8080/
[16:04:30] Posted data.
[16:04:30] Initial: 40AB; - Successful: assigned to (171.64.65.64).
[16:04:30] + News From Folding@Home: Welcome to Folding@Home
[16:04:30] Loaded queue successfully.
[16:04:30] Connecting to http://171.64.65.64:8080/
[16:04:37] Posted data.
[16:04:37] Initial: 0000; - Receiving payload (expected size: 4739053)
[16:04:47] - Downloaded at ~462 kB/s
[16:04:47] - Averaged speed for that direction ~517 kB/s
[16:04:47] + Received work.
[16:04:47] + Closed connections
[16:04:52] 
[16:04:52] + Processing work unit
[16:04:52] Work type a1 not eligible for variable processors
[16:04:52] Core required: FahCore_a1.exe
[16:04:52] Core found.
[16:04:52] Using generic mpiexec calls
[16:04:52] Working on queue slot 02 [September 6 16:04:52 UTC]
[16:04:52] + Working ...
[16:04:52] - Calling 'mpiexec -np 4 -channel auto -host 127.0.0.1 FahCore_a1.exe -dir work/ -suffix 02 -checkpoint 30 -verbose -lifeline 3768 -version 622'

[16:04:52] 
[16:04:52] *------------------------------*
[16:04:52] Folding@Home Gromacs SMP Core
[16:04:52] Version 1.74 (March 10, 2007)
[16:04:52] 
[16:04:52] Preparing to commence simulation
[16:04:52] - Ensuring status. Please wait.
[16:04:59] - Starting from initial work packet
[16:04:59] 
[16:04:59] Project: 2665 (Run 0, Clone 931, Gen 38)
[16:04:59] 
[16:04:59] Assembly optimizations on if available.
[16:04:59] Entering M.D.
[16:05:30]  percent)
[16:05:30] cket
[16:05:30] 
[16:05:30] Project: 2665 (Run 0, Clone 931, Gen 38)
[16:05:30] 
[16:05:30] 65 (Run 0, Clone 931, Gen 38)
[16:05:30] 
[16:05:31] Entering M.D.
[16:05:37] Rejecting checkpoint
[16:05:39] Protein: HGG in water
[16:05:40] Writing local files
[16:05:52] Extra SSE boost OK.
[16:05:52] Writing local files
[16:05:52] Completed 0 out of 250000 steps  (0 percent)
[16:26:11] Writing local files
[16:26:12] Completed 2500 out of 250000 steps  (1 percent)
[16:46:30] Writing local files
[16:46:30] Completed 5000 out of 250000 steps  (2 percent)
[17:06:46] Writing local files
[17:06:47] Completed 7500 out of 250000 steps  (3 percent)
[17:27:02] Writing local files
[17:27:02] Completed 10000 out of 250000 steps  (4 percent)
[17:47:20] Writing local files
[17:47:21] Completed 12500 out of 250000 steps  (5 percent)
[18:07:38] Writing local files
[18:07:38] Completed 15000 out of 250000 steps  (6 percent)
[18:27:56] Writing local files
[18:27:56] Completed 17500 out of 250000 steps  (7 percent)
[18:48:16] Writing local files
[18:48:16] Completed 20000 out of 250000 steps  (8 percent)
[19:08:34] Writing local files
[19:08:34] Completed 22500 out of 250000 steps  (9 percent)
[19:28:51] Writing local files
[19:28:52] Completed 25000 out of 250000 steps  (10 percent)
[19:49:11] Writing local files
---------------------------------------------------------------------------------
[15:35:31] Writing local files
[15:35:31] Completed 175000 out of 250000 steps  (70 percent)
[15:55:36] Writing local files
[15:55:36] Completed 177500 out of 250000 steps  (71 percent)
[16:15:41] Writing local files
[16:15:41] Completed 180000 out of 250000 steps  (72 percent)
[16:35:45] Writing local files
[16:35:45] Completed 182500 out of 250000 steps  (73 percent)
[16:55:47] Writing local files
[16:55:47] Completed 185000 out of 250000 steps  (74 percent)
[17:15:44] Writing local files
[17:15:44] Completed 187500 out of 250000 steps  (75 percent)
[17:35:47] Writing local files
[17:35:47] Completed 190000 out of 250000 steps  (76 percent)
[17:55:50] Writing local files
[17:55:51] Completed 192500 out of 250000 steps  (77 percent)
[18:15:52] Writing local files
[18:15:53] Completed 195000 out of 250000 steps  (78 percent)
[18:35:57] Writing local files
[18:35:58] Completed 197500 out of 250000 steps  (79 percent)
[18:56:03] Writing local files
[18:56:03] Completed 200000 out of 250000 steps  (80 percent)
[19:02:54] Warning:  long 1-4 interactions
[19:02:55] Quit 101 - NaN detected: (ener[20])
[19:02:55] 
[19:02:55] Simulation instability has been encountered. The run has entered a
[19:02:55]   state from which no further progress can be made.
[19:02:55] This may be the correct result of the simulation, however if you
[19:02:55]   often see other project units terminating early like this
[19:02:55]   too, you may wish to check the stability of your computer (issues
[19:02:55]   such as high temperature, overclocking, etc.).
[19:02:55] Going to send back what have done.
[19:02:55] logfile size: 158722
[19:02:55] - Writing 159272 bytes of core data to disk...
[19:02:55]   ... Done.
[19:02:55] - Failed to delete work/wudata_02.arc
[19:02:55] No C.P. to delete.
[19:02:55] Warning:  check for stray files
[19:02:55] 
[19:02:55] Folding@home Core Shutdown: EARLY_UNIT_END
[19:02:55] 
[19:02:55] Folding@home Core Shutdown: EARLY_UNIT_END
[19:03:00] CoreStatus = 7B (123)
[19:03:00] Client-core communications error: ERROR 0x7b
[19:03:00] - Attempting to download new core...
[19:03:00] + Downloading new core: FahCore_a1.exe
[19:03:00] Downloading core (/~pande/Win32/x86/Core_a1.fah from www.stanford.edu)
[19:03:02] Initial: AFDE; + 10240 bytes downloaded
[19:03:02] Initial: AD21; + 20480 bytes downloaded
[19:03:02] Initial: CC38; + 30720 bytes downloaded
[19:03:02] Initial: 8501; + 40960 bytes downloaded
[19:03:02] Initial: F56A; + 51200 bytes downloaded
[19:03:02] Initial: ABAE; + 61440 bytes downloaded
[19:03:02] Initial: B6B0; + 71680 bytes downloaded
[19:03:02] Initial: 783A; + 81920 bytes downloaded
[19:03:02] Initial: B2A6; + 92160 bytes downloaded
[19:03:02] Initial: 1409; + 102400 bytes downloaded
[19:03:02] Initial: BBF0; + 112640 bytes downloaded
[19:03:02] Initial: 1861; + 122880 bytes downloaded
[19:03:03] Initial: 5950; + 133120 bytes downloaded
[19:03:03] Initial: 1081; + 143360 bytes downloaded
[19:03:03] Initial: 26BC; + 153600 bytes downloaded
[19:03:03] Initial: FE4A; + 163840 bytes downloaded
[19:03:03] Initial: C1C3; + 174080 bytes downloaded
[19:03:03] Initial: 9B49; + 184320 bytes downloaded
[19:03:03] Initial: 9EE5; + 194560 bytes downloaded
[19:03:03] Initial: D79D; + 204800 bytes downloaded
[19:03:03] Initial: 7801; + 215040 bytes downloaded
[19:03:03] Initial: 8B51; + 225280 bytes downloaded
[19:03:03] Initial: E26E; + 235520 bytes downloaded
[19:03:03] Initial: EDB0; + 245760 bytes downloaded
[19:03:03] Initial: 0919; + 256000 bytes downloaded
[19:03:03] Initial: CDDE; + 266240 bytes downloaded
[19:03:03] Initial: 7A7E; + 276480 bytes downloaded
[19:03:03] Initial: 034E; + 286720 bytes downloaded
[19:03:03] Initial: 88D0; + 296960 bytes downloaded
[19:03:03] Initial: D66D; + 307200 bytes downloaded
[19:03:03] Initial: 6A52; + 317440 bytes downloaded
[19:03:03] Initial: B478; + 327680 bytes downloaded
[19:03:03] Initial: CF8A; + 337920 bytes downloaded
[19:03:03] Initial: 8407; + 348160 bytes downloaded
[19:03:03] Initial: 2246; + 358400 bytes downloaded
[19:03:03] Initial: 1C69; + 368640 bytes downloaded
[19:03:03] Initial: 1287; + 378880 bytes downloaded
[19:03:03] Initial: 19B3; + 389120 bytes downloaded
[19:03:03] Initial: 1AD1; + 399360 bytes downloaded
[19:03:03] Initial: 5791; + 409600 bytes downloaded
[19:03:03] Initial: 76C5; + 419840 bytes downloaded
[19:03:03] Initial: 9B77; + 430080 bytes downloaded
[19:03:03] Initial: E82F; + 440320 bytes downloaded
[19:03:03] Initial: D0D3; + 450560 bytes downloaded
[19:03:03] Initial: 0F5E; + 460800 bytes downloaded
[19:03:03] Initial: D743; + 471040 bytes downloaded
[19:03:03] Initial: 0B7C; + 481280 bytes downloaded
[19:03:03] Initial: FAFD; + 491520 bytes downloaded
[19:03:03] Initial: 0E14; + 501760 bytes downloaded
[19:03:03] Initial: 4048; + 512000 bytes downloaded
[19:03:03] Initial: 21A5; + 522240 bytes downloaded
[19:03:03] Initial: C1A5; + 532480 bytes downloaded
[19:03:03] Initial: F716; + 542720 bytes downloaded
[19:03:03] Initial: DD98; + 552960 bytes downloaded
[19:03:03] Initial: 9F7B; + 563200 bytes downloaded
[19:03:03] Initial: 1CC0; + 573440 bytes downloaded
[19:03:03] Initial: 4D37; + 583680 bytes downloaded
[19:03:03] Initial: 222A; + 593920 bytes downloaded
[19:03:03] Initial: 8E33; + 604160 bytes downloaded
[19:03:03] Initial: D3C9; + 614400 bytes downloaded
[19:03:03] Initial: 9821; + 624640 bytes downloaded
[19:03:03] Initial: 236E; + 634880 bytes downloaded
[19:03:03] Initial: 1A7A; + 645120 bytes downloaded
[19:03:03] Initial: 6D64; + 655360 bytes downloaded
[19:03:03] Initial: 4ADC; + 665600 bytes downloaded
[19:03:03] Initial: 3854; + 675840 bytes downloaded
[19:03:03] Initial: CB5C; + 686080 bytes downloaded
[19:03:03] Initial: 2A88; + 696320 bytes downloaded
[19:03:03] Initial: 1199; + 706560 bytes downloaded
[19:03:03] Initial: 0512; + 716800 bytes downloaded
[19:03:03] Initial: 316E; + 727040 bytes downloaded
[19:03:03] Initial: D89D; + 737280 bytes downloaded
[19:03:03] Initial: E6A3; + 747520 bytes downloaded
[19:03:04] Initial: B488; + 757760 bytes downloaded
[19:03:04] Initial: BAFD; + 768000 bytes downloaded
[19:03:04] Initial: 34A0; + 778240 bytes downloaded
[19:03:04] Initial: DD6C; + 788480 bytes downloaded
[19:03:04] Initial: D2E9; + 789667 bytes downloaded
[19:03:04] Verifying core Core_a1.fah...
[19:03:04] Signature is VALID
[19:03:04] 
[19:03:04] Trying to unzip core FahCore_a1.exe
[19:03:04] Decompressed FahCore_a1.exe (2035712 bytes) successfully
[19:03:09] + Core successfully engaged
[19:03:09] Deleting current work unit & continuing...
[19:03:09] Using generic mpiexec calls
[19:05:13] - Warning: Could not delete all work unit files (2): Core returned invalid code
[19:05:13] Trying to send all finished work units
[19:05:13] + No unsent completed units remaining.
[19:05:13] - Preparing to get new work unit...
[19:05:13] + Attempting to get work packet
[19:05:13] - Will indicate memory of 2046 MB
[19:05:13] - Connecting to assignment server
[19:05:13] Connecting to http://assign.stanford.edu:8080/
[19:05:13] Posted data.
[19:05:13] Initial: 40AB; - Successful: assigned to (171.64.65.64).
[19:05:13] + News From Folding@Home: Welcome to Folding@Home
[19:05:13] Loaded queue successfully.
[19:05:13] Connecting to http://171.64.65.64:8080/
[19:05:19] Posted data.
[19:05:19] Initial: 0000; - Receiving payload (expected size: 4739053)
[19:05:35] - Downloaded at ~289 kB/s
[19:05:35] - Averaged speed for that direction ~471 kB/s
[19:05:35] + Received work.
[19:05:35] + Closed connections
[19:05:40] 
[19:05:40] + Processing work unit
[19:05:40] Work type a1 not eligible for variable processors
[19:05:40] Core required: FahCore_a1.exe
[19:05:40] Core found.
[19:05:40] Using generic mpiexec calls
[19:05:40] Working on queue slot 03 [September 7 19:05:40 UTC]
[19:05:40] + Working ...
[19:05:40] - Calling 'mpiexec -np 4 -channel auto -host 127.0.0.1 FahCore_a1.exe -dir work/ -suffix 03 -checkpoint 30 -verbose -lifeline 3768 -version 622'

[19:05:40] 
[19:05:40] *------------------------------*
[19:05:40] Folding@Home Gromacs SMP Core
[19:05:40] Version 1.74 (March 10, 2007)
[19:05:40] 
[19:05:40] Preparing to commence simulation
[19:05:40] - Ensuring status. Please wait.
[19:05:46] - Starting from initial work packet
[19:05:47] 
[19:05:47] Project: 2665 (Run 0, Clone 931, Gen 38)
[19:05:47] 
[19:05:47] Assembly optimizations on if available.
[19:05:47] Entering M.D.
[19:06:17]  percent)
[19:06:17] cket
[19:06:17] 
[19:06:17] Project: 2665 (Run 0, Clone 931, Gen 38)
[19:06:17] 
[19:06:18] 65 (Run 0, Clone 931, Gen 38)
[19:06:18] 
[19:06:19] Entering M.D.
[19:06:25] Rejecting checkpoint
[19:06:27] Protein: HGG in water
[19:06:27] Writing local files
[19:06:39] Extra SSE boost OK.
[19:06:39] Writing local files
[19:06:40] Completed 0 out of 250000 steps  (0 percent)
[19:26:39] Writing local files
[19:26:39] Completed 2500 out of 250000 steps  (1 percent)
[19:46:39] Writing local files
[19:46:40] Completed 5000 out of 250000 steps  (2 percent)
[20:06:39] Writing local files
[20:06:40] Completed 7500 out of 250000 steps  (3 percent)
[20:26:38] Writing local files
[20:26:39] Completed 10000 out of 250000 steps  (4 percent)
[20:46:38] Writing local files
[20:46:38] Completed 12500 out of 250000 steps  (5 percent)
[21:01:02] - Autosending finished units... [September 7 21:01:02 UTC]
[21:01:02] Trying to send all finished work units
[21:01:02] + No unsent completed units remaining.
[21:01:02] - Autosend completed
[21:06:40] Writing local files
[21:06:40] Completed 15000 out of 250000 steps  (6 percent)
[21:26:42] Writing local files
[21:26:42] Completed 17500 out of 250000 steps  (7 percent)
[21:46:42] Writing local files
[21:46:43] Completed 20000 out of 250000 steps  (8 percent)
[22:06:44] Writing local files
[22:06:44] Completed 22500 out of 250000 steps  (9 percent)
[22:26:44] Writing local files
[22:26:44] Completed 25000 out of 250000 steps  (10 percent)
[22:46:44] Writing local files
---------------------------------------------------------------------------------------
[02:06:38] Completed 52500 out of 250000 steps  (21 percent)
[02:26:37] Writing local files
[02:26:38] Completed 55000 out of 250000 steps  (22 percent)
[02:46:18] Writing local files
[02:46:18] Completed 57500 out of 250000 steps  (23 percent)
[02:57:04] Killing all core threads
[02:57:04] Killing 4 cores
[02:57:04] Killing core 0
[02:57:04] Killing core 1
[02:57:04] Killing core 2
[02:57:04] Killing core 3

Folding@Home Client Shutdown at user request.
[02:57:04] ***** Got a SIGTERM signal (2)
[02:57:04] Killing all core threads
[02:57:04] Killing 4 cores
[02:57:04] Killing core 0
[02:57:04] Killing core 1
[02:57:04] Killing core 2
[02:57:04] Killing core 3

Folding@Home Client Shutdown.

Code: Select all

--- Opening Log file [September 10 05:56:15 UTC] 


# Windows CPU Console Edition #################################################
###############################################################################

                       Folding@Home Client Version 6.22 SMP Beta2r3

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: C:\Program Files\Folding@Home Windows SMP Client V1.01
Executable: C:\Program Files\Folding@Home Windows SMP Client V1.01\Folding@home-Win32-x86.exe
Arguments: -configonly 

[05:56:15] Configuring Folding@Home...


[05:57:02] - Ask before connecting: No
[05:57:02] - User name: MDCRL (Team 35275)
[05:57:02] - User ID: 3FF6AE732BBDC17A
[05:57:02] - Machine ID: 1
[05:57:02] 
[05:57:02] -configonly flag given, so exiting.


--- Opening Log file [September 10 05:57:11 UTC] 


# Windows SMP Console Edition #################################################
###############################################################################

                       Folding@Home Client Version 6.22 SMP Beta2r3

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: C:\Program Files\Folding@Home Windows SMP Client V1.01
Executable: C:\Program Files\Folding@Home Windows SMP Client V1.01\Folding@home-Win32-x86.exe
Arguments: -smp -verbosity 9 

[05:57:11] - Ask before connecting: No
[05:57:11] - User name: MDCRL (Team 35275)
[05:57:11] - User ID: 3FF6AE732BBDC17A
[05:57:11] - Machine ID: 1
[05:57:11] 
[05:57:11] Work directory not found. Creating...
[05:57:11] Could not open work queue, generating new queue...
[05:57:11] - Preparing to get new work unit...
[05:57:11] - Autosending finished units... [September 10 05:57:11 UTC]
[05:57:11] Trying to send all finished work units
[05:57:11] + Attempting to get work packet
[05:57:11] + No unsent completed units remaining.
[05:57:11] - Will indicate memory of 2046 MB
[05:57:11] - Autosend completed
[05:57:11] - Detect CPU. Vendor: GenuineIntel, Family: 6, Model: 15, Stepping: 11
[05:57:11] - Connecting to assignment server
[05:57:11] Connecting to http://assign.stanford.edu:8080/
[05:57:11] Posted data.
[05:57:11] Initial: 40AB; - Successful: assigned to (171.64.65.64).
[05:57:11] + News From Folding@Home: Welcome to Folding@Home
[05:57:12] Loaded queue successfully.
[05:57:12] Connecting to http://171.64.65.64:8080/
[05:57:17] Posted data.
[05:57:17] Initial: 0000; - Receiving payload (expected size: 4739053)
[05:57:26] - Downloaded at ~514 kB/s
[05:57:26] - Averaged speed for that direction ~514 kB/s
[05:57:26] + Received work.
[05:57:26] + Closed connections
[05:57:26] 
[05:57:26] + Processing work unit
[05:57:26] Work type a1 not eligible for variable processors
[05:57:26] Core required: FahCore_a1.exe
[05:57:26] Core not found.
[05:57:26] - Core is not present or corrupted.
[05:57:26] - Attempting to download new core...
[05:57:26] + Downloading new core: FahCore_a1.exe
[05:57:26] Downloading core (/~pande/Win32/x86/Core_a1.fah from www.stanford.edu)
[05:57:27] Initial: AFDE; + 10240 bytes downloaded
[05:57:27] Initial: AD21; + 20480 bytes downloaded
[05:57:27] Initial: CC38; + 30720 bytes downloaded
[05:57:27] Initial: 8501; + 40960 bytes downloaded
[05:57:27] Initial: F56A; + 51200 bytes downloaded
[05:57:27] Initial: ABAE; + 61440 bytes downloaded
[05:57:27] Initial: B6B0; + 71680 bytes downloaded
[05:57:27] Initial: 783A; + 81920 bytes downloaded
[05:57:27] Initial: B2A6; + 92160 bytes downloaded
[05:57:27] Initial: 1409; + 102400 bytes downloaded
[05:57:27] Initial: BBF0; + 112640 bytes downloaded
[05:57:27] Initial: 1861; + 122880 bytes downloaded
[05:57:27] Initial: 5950; + 133120 bytes downloaded
[05:57:27] Initial: 1081; + 143360 bytes downloaded
[05:57:27] Initial: 26BC; + 153600 bytes downloaded
[05:57:27] Initial: FE4A; + 163840 bytes downloaded
[05:57:27] Initial: C1C3; + 174080 bytes downloaded
[05:57:27] Initial: 9B49; + 184320 bytes downloaded
[05:57:27] Initial: 9EE5; + 194560 bytes downloaded
[05:57:27] Initial: D79D; + 204800 bytes downloaded
[05:57:27] Initial: 7801; + 215040 bytes downloaded
[05:57:27] Initial: 8B51; + 225280 bytes downloaded
[05:57:27] Initial: E26E; + 235520 bytes downloaded
[05:57:28] Initial: EDB0; + 245760 bytes downloaded
[05:57:28] Initial: 0919; + 256000 bytes downloaded
[05:57:28] Initial: CDDE; + 266240 bytes downloaded
[05:57:28] Initial: 7A7E; + 276480 bytes downloaded
[05:57:28] Initial: 034E; + 286720 bytes downloaded
[05:57:28] Initial: 88D0; + 296960 bytes downloaded
[05:57:28] Initial: D66D; + 307200 bytes downloaded
[05:57:28] Initial: 6A52; + 317440 bytes downloaded
[05:57:28] Initial: B478; + 327680 bytes downloaded
[05:57:28] Initial: CF8A; + 337920 bytes downloaded
[05:57:28] Initial: 8407; + 348160 bytes downloaded
[05:57:28] Initial: 2246; + 358400 bytes downloaded
[05:57:28] Initial: 1C69; + 368640 bytes downloaded
[05:57:28] Initial: 1287; + 378880 bytes downloaded
[05:57:28] Initial: 19B3; + 389120 bytes downloaded
[05:57:28] Initial: 1AD1; + 399360 bytes downloaded
[05:57:28] Initial: 5791; + 409600 bytes downloaded
[05:57:28] Initial: 76C5; + 419840 bytes downloaded
[05:57:28] Initial: 9B77; + 430080 bytes downloaded
[05:57:28] Initial: E82F; + 440320 bytes downloaded
[05:57:28] Initial: D0D3; + 450560 bytes downloaded
[05:57:28] Initial: 0F5E; + 460800 bytes downloaded
[05:57:28] Initial: D743; + 471040 bytes downloaded
[05:57:28] Initial: 0B7C; + 481280 bytes downloaded
[05:57:28] Initial: FAFD; + 491520 bytes downloaded
[05:57:28] Initial: 0E14; + 501760 bytes downloaded
[05:57:28] Initial: 4048; + 512000 bytes downloaded
[05:57:28] Initial: 21A5; + 522240 bytes downloaded
[05:57:28] Initial: C1A5; + 532480 bytes downloaded
[05:57:28] Initial: F716; + 542720 bytes downloaded
[05:57:28] Initial: DD98; + 552960 bytes downloaded
[05:57:28] Initial: 9F7B; + 563200 bytes downloaded
[05:57:28] Initial: 1CC0; + 573440 bytes downloaded
[05:57:28] Initial: 4D37; + 583680 bytes downloaded
[05:57:28] Initial: 222A; + 593920 bytes downloaded
[05:57:28] Initial: 8E33; + 604160 bytes downloaded
[05:57:28] Initial: D3C9; + 614400 bytes downloaded
[05:57:28] Initial: 9821; + 624640 bytes downloaded
[05:57:28] Initial: 236E; + 634880 bytes downloaded
[05:57:28] Initial: 1A7A; + 645120 bytes downloaded
[05:57:28] Initial: 6D64; + 655360 bytes downloaded
[05:57:28] Initial: 4ADC; + 665600 bytes downloaded
[05:57:28] Initial: 3854; + 675840 bytes downloaded
[05:57:28] Initial: CB5C; + 686080 bytes downloaded
[05:57:28] Initial: 2A88; + 696320 bytes downloaded
[05:57:28] Initial: 1199; + 706560 bytes downloaded
[05:57:28] Initial: 0512; + 716800 bytes downloaded
[05:57:28] Initial: 316E; + 727040 bytes downloaded
[05:57:28] Initial: D89D; + 737280 bytes downloaded
[05:57:28] Initial: E6A3; + 747520 bytes downloaded
[05:57:28] Initial: B488; + 757760 bytes downloaded
[05:57:28] Initial: BAFD; + 768000 bytes downloaded
[05:57:28] Initial: 34A0; + 778240 bytes downloaded
[05:57:28] Initial: DD6C; + 788480 bytes downloaded
[05:57:28] Initial: D2E9; + 789667 bytes downloaded
[05:57:28] Verifying core Core_a1.fah...
[05:57:28] Signature is VALID
[05:57:28] 
[05:57:28] Trying to unzip core FahCore_a1.exe
[05:57:29] Decompressed FahCore_a1.exe (2035712 bytes) successfully
[05:57:34] + Core successfully engaged
[05:57:39] 
[05:57:39] + Processing work unit
[05:57:39] Work type a1 not eligible for variable processors
[05:57:39] Core required: FahCore_a1.exe
[05:57:39] Core found.
[05:57:39] Using generic mpiexec calls
[05:57:39] Working on queue slot 01 [September 10 05:57:39 UTC]
[05:57:39] + Working ...
[05:57:39] - Calling 'mpiexec -np 4 -channel auto -host 127.0.0.1 FahCore_a1.exe -dir work/ -suffix 01 -checkpoint 30 -verbose -lifeline 2332 -version 622'

[05:57:39] 
[05:57:39] *------------------------------*
[05:57:39] Folding@Home Gromacs SMP Core
[05:57:39] Version 1.74 (March 10, 2007)
[05:57:39] 
[05:57:39] Preparing to commence simulation
[05:57:39] - Ensuring status. Please wait- Created dyn
[05:57:39] - Files status OK
[05:57:45] - Expanded 4738541 -> 24426905 (decompressed 515.4 percent)
[05:57:46] - Starting from initial work packet
[05:57:46] 
[05:57:46] Project: 2665 (Run 0, Clone 931, Gen 38)
[05:57:46] 
[05:57:46] Assembly optimizations on if available.
[05:57:46] Entering M.D.
[05:58:17] al work pa- Starting from initial work packet
[05:58:17] 
[05:58:17] Project: 2665 (Run 0, Clone 931, Gen 38)
[05:58:17] 
[05:58:18] Entering M.D.
[05:58:24] Rejecting checkpoint
[05:58:26] Protein: HGG in water
[05:58:26] Writing local files
[05:58:39] Extra SSE boost OK.
[05:58:39] Writing local files
[05:58:40] Completed 0 out of 250000 steps  (0 percent)
[06:18:54] Writing local files
[06:18:55] Completed 2500 out of 250000 steps  (1 percent)
[06:39:09] Writing local files
[06:39:10] Completed 5000 out of 250000 steps  (2 percent)
[06:59:24] Writing local files
[06:59:24] Completed 7500 out of 250000 steps  (3 percent)
[07:19:36] Writing local files
[07:19:36] Completed 10000 out of 250000 steps  (4 percent)
[07:39:49] Writing local files
[07:39:50] Completed 12500 out of 250000 steps  (5 percent)
[08:00:02] Writing local files
[08:00:02] Completed 15000 out of 250000 steps  (6 percent)
[08:20:17] Writing local files
[08:20:17] Completed 17500 out of 250000 steps  (7 percent)
[08:40:32] Writing local files
[08:40:33] Completed 20000 out of 250000 steps  (8 percent)
[09:00:47] Writing local files
[09:00:47] Completed 22500 out of 250000 steps  (9 percent)
[09:21:01] Writing local files
[09:21:01] Completed 25000 out of 250000 steps  (10 percent)
[09:41:14] Writing local files
-----------------------------------------------------------------------------------
[05:31:51] Completed 175000 out of 250000 steps  (70 percent)
[05:51:58] Writing local files
[05:51:58] Completed 177500 out of 250000 steps  (71 percent)
[05:57:11] - Autosending finished units... [September 11 05:57:11 UTC]
[05:57:11] Trying to send all finished work units
[05:57:11] + No unsent completed units remaining.
[05:57:11] - Autosend completed
[06:12:07] Writing local files
[06:12:07] Completed 180000 out of 250000 steps  (72 percent)
[06:32:15] Writing local files
[06:32:15] Completed 182500 out of 250000 steps  (73 percent)
[06:52:20] Writing local files
[06:52:21] Completed 185000 out of 250000 steps  (74 percent)
[07:12:29] Writing local files
[07:12:29] Completed 187500 out of 250000 steps  (75 percent)
[07:32:35] Writing local files
[07:32:35] Completed 190000 out of 250000 steps  (76 percent)
[07:52:42] Writing local files
[07:52:42] Completed 192500 out of 250000 steps  (77 percent)
[08:12:49] Writing local files
[08:12:49] Completed 195000 out of 250000 steps  (78 percent)
[08:32:57] Writing local files
[08:32:57] Completed 197500 out of 250000 steps  (79 percent)
[08:53:05] Writing local files
[08:53:05] Completed 200000 out of 250000 steps  (80 percent)
[08:59:57] Warning:  long 1-4 interactions
[08:59:58] Quit 101 - NaN detected: (ener[20])
[08:59:58] 
[08:59:58] Simulation instability has been encountered. The run has entered a
[08:59:58]   state from which no further progress can be made.
[08:59:58] This may be the correct result of the simulation, however if you
[08:59:58]   often see other project units terminating early like this
[08:59:58]   too, you may wish to check the stability of your computer (issues
[08:59:58]   such as high temperature, overclocking, etc.).
[08:59:58] Going to send back what have done.
[08:59:58] logfile size: 158722
[08:59:58] - Writing 159272 bytes of core data to disk...
[08:59:58]   ... Done.
[08:59:58] - Failed to delete work/wudata_01.arc
[08:59:59] Warning:  check for stray files
[09:01:59] 
[09:01:59] Folding@home Core Shutdown: EARLY_UNIT_END
[09:01:59] 
[09:01:59] Folding@home Core Shutdown: EARLY_UNIT_END
[09:02:03] CoreStatus = 7B (123)
[09:02:03] Client-core communications error: ERROR 0x7b
[09:02:03] Deleting current work unit & continuing...
[09:02:03] Using generic mpiexec calls
[09:04:07] - Warning: Could not delete all work unit files (1): Core returned invalid code
[09:04:07] Trying to send all finished work units
[09:04:07] + No unsent completed units remaining.
[09:04:07] - Preparing to get new work unit...
[09:04:07] + Attempting to get work packet
[09:04:07] - Will indicate memory of 2046 MB
[09:04:07] - Connecting to assignment server
[09:04:07] Connecting to http://assign.stanford.edu:8080/
[09:04:07] Posted data.
[09:04:07] Initial: 40AB; - Successful: assigned to (171.64.65.64).
[09:04:07] + News From Folding@Home: Welcome to Folding@Home
[09:04:07] Loaded queue successfully.
[09:04:07] Connecting to http://171.64.65.64:8080/
[09:04:13] Posted data.
[09:04:13] Initial: 0000; - Receiving payload (expected size: 4739053)
[09:04:21] - Downloaded at ~578 kB/s
[09:04:21] - Averaged speed for that direction ~546 kB/s
[09:04:21] + Received work.
[09:04:21] + Closed connections
[09:04:26] 
[09:04:26] + Processing work unit
[09:04:26] Work type a1 not eligible for variable processors
[09:04:26] Core required: FahCore_a1.exe
[09:04:26] Core found.
[09:04:26] Using generic mpiexec calls
[09:04:26] Working on queue slot 02 [September 11 09:04:26 UTC]
[09:04:26] + Working ...
[09:04:26] - Calling 'mpiexec -np 4 -channel auto -host 127.0.0.1 FahCore_a1.exe -dir work/ -suffix 02 -checkpoint 30 -verbose -lifeline 2332 -version 622'

[09:04:26] 
[09:04:26] *------------------------------*
[09:04:26] Folding@Home Gromacs SMP Core
[09:04:26] Version 1.74 (March 10, 2007)
[09:04:26] 
[09:04:26] Preparing to commence simulation
[09:04:26] - Ensuring status. Please wait.
[09:04:33] - Starting from initial work packet
[09:04:33] 
[09:04:33] Project: 2665 (Run 0, Clone 931, Gen 38)
[09:04:33] 
[09:04:33] Assembly optimizations on if available.
[09:04:33] Entering M.D.
[09:05:04] al work packet
[09:05:04] 
[09:05:04] Project: 2665 (Run 0, Clone 931, Gen 38)
[09:05:04] 
[09:05:04] 65 (Run 0, Clone 931, Gen 38)
[09:05:04] 
[09:05:06] Entering M.D.
[09:05:12] Rejecting checkpoint
[09:05:13] 
[09:05:13] Writing local files
[09:05:14] 
[09:05:14] Writing local files
[09:05:25] Extra SSE boost OK.
[09:05:25] Writing local files
[09:05:26] Completed 0 out of 250000 steps  (0 percent)
[09:25:29] Writing local files
[09:25:29] Completed 2500 out of 250000 steps  (1 percent)
[09:45:33] Writing local files
[09:45:33] Completed 5000 out of 250000 steps  (2 percent)
[10:05:36] Writing local files
[10:05:36] Completed 7500 out of 250000 steps  (3 percent)
[10:25:36] Writing local files
[10:25:36] Completed 10000 out of 250000 steps  (4 percent)
[10:45:39] Writing local files
[10:45:40] Completed 12500 out of 250000 steps  (5 percent)
[11:05:43] Writing local files
[11:05:43] Completed 15000 out of 250000 steps  (6 percent)
[11:25:47] Writing local files
[11:25:48] Completed 17500 out of 250000 steps  (7 percent)
Tried all of the deleting and restatrting tricks, replaced .exe w/ new, added correct flags to .exe
- and still no completion - just same WU again.
Sorry I didn't catch it sooner......

Reinstalled client & Did Not p/u new WU - same: (Run 0, Clone 931, Gen 38)

Any other suggestions - that didn't work?

Thanks.....

Re: Project: 2665 (Run 2, Clone 903, Gen 42) multiple failures

Posted: Thu Sep 11, 2008 12:58 pm
by toTOW
Did you try qfix to tell the server that you can't finish this WU :?:

Re: Project: 2665 (Run 2, Clone 903, Gen 42) multiple failures

Posted: Thu Sep 11, 2008 9:42 pm
by bruce
If qfix was successful, you should be able to upload the results which will tell the server not to reassign it. Did you capture the output from the first time qfix was run? If so, post it here.

Re: Project: 2665 (Run 2, Clone 903, Gen 42) multiple failures

Posted: Fri Sep 12, 2008 3:31 am
by The_Namek
What exactly is qfix?

Re: Project: 2665 (Run 2, Clone 903, Gen 42) multiple failures

Posted: Fri Sep 12, 2008 6:33 am
by bruce
Qfix can be found in the "Tools List" in our 3rd party forum. When your client has ceertain errors, it can help correct a few of them.

A "How to" can be found here: viewtopic.php?f=8&t=191

Re: Project: 2665 (Run 2, Clone 903, Gen 42) multiple failures

Posted: Fri Sep 12, 2008 1:24 pm
by MDCRL
I reinstalled to a new location and it p/u new WU 1st try... runnnin 2665 (R-2 C-13 G-49) so far up to 70% w/out problems

haven't tried qfix yet - not worried about points - just want to get it runnin right

Do those EUE/bad WU results actually have some benefit to the project
- should I be trying to salvage what is left when they implode

Thanks 4 the assistance

Re: Project: 2665 (Run 2, Clone 903, Gen 42) multiple failures

Posted: Sat Sep 13, 2008 1:17 am
by MDCRL
Here is the latest prob after finishing the most recent WU it died again...Client-core communications error: ERROR 0x7b

Code: Select all

--- Opening Log file [September 11 12:52:20 UTC] 

# Windows SMP Console Edition #################################################
###############################################################################
                       Folding@Home Client Version 6.22 SMP Beta2r3

                          http://folding.stanford.edu
###############################################################################
###############################################################################
Launch directory: C:\Documents and Settings\RL\My Documents\SMP Client
Executable: C:\Documents and Settings\RL\My Documents\SMP Client\Folding@home-Win32-x86.exe
Arguments: -smp -verbosity 9 

[12:52:20] - Ask before connecting: No
[12:52:20] - User name: MDCRL (Team 35275)
[12:52:20] - User ID: 3FF6AE732BBDC17A
[12:52:20] - Machine ID: 1
[12:52:20] 
[12:52:20] Work directory not found. Creating...
[12:52:20] Could not open work queue, generating new queue...
[12:52:20] - Preparing to get new work unit...
[12:52:20] - Autosending finished units... [September 11 12:52:20 UTC]
[12:52:20] + Attempting to get work packet
[12:52:20] Trying to send all finished work units
[12:52:20] + No unsent completed units remaining.
[12:52:20] - Will indicate memory of 2046 MB
[12:52:20] - Autosend completed[12:52:20] - Detect CPU.
 Vendor: GenuineIntel, Family: 6, Model: 15, Stepping: 11
[12:52:20] - Connecting to assignment server
[12:52:20] Connecting to http://assign.stanford.edu:8080/
[12:52:20] Posted data.
[12:52:20] Initial: 40AB; - Successful: assigned to (171.64.65.64).
[12:52:20] + News From Folding@Home: Welcome to Folding@Home
[12:52:20] Loaded queue successfully.
[12:52:20] Connecting to http://171.64.65.64:8080/
[12:52:21] Posted data.
[12:52:21] Initial: 0000; - Error: Bad packet type from server, expected work assignment
[12:52:21] - Attempt #1  to get work failed, and no other work to do.
Waiting before retry.
[12:52:38] + Attempting to get work packet
[12:52:38] - Will indicate memory of 2046 MB
[12:52:38] - Connecting to assignment server
[12:52:38] Connecting to http://assign.stanford.edu:8080/
[12:52:38] Posted data.
[12:52:38] Initial: 40AB; - Successful: assigned to (171.64.65.64).
[12:52:38] + News From Folding@Home: Welcome to Folding@Home
[12:52:38] Loaded queue successfully.
[12:52:39] Connecting to http://171.64.65.64:8080/
[12:52:44] Posted data.
[12:52:44] Initial: 0000; - Receiving payload (expected size: 4837927)
[12:52:53] - Downloaded at ~524 kB/s
[12:52:53] - Averaged speed for that direction ~524 kB/s
[12:52:53] + Received work.
[12:52:53] + Closed connections
[12:52:53] 
[12:52:53] + Processing work unit
[12:52:53] Work type a1 not eligible for variable processors
[12:52:53] Core required: FahCore_a1.exe
[12:52:53] Core not found.
[12:52:53] - Core is not present or corrupted.
[12:52:53] - Attempting to download new core...
[12:52:53] + Downloading new core: FahCore_a1.exe
[12:52:53] Downloading core (/~pande/Win32/x86/Core_a1.fah from www.stanford.edu)
[12:52:54] Initial: AFDE; + 10240 bytes downloaded
[12:52:54] Initial: AD21; + 20480 bytes downloaded
[12:52:54] Initial: CC38; + 30720 bytes downloaded
[12:52:54] Initial: 8501; + 40960 bytes downloaded
[12:52:55] Initial: F56A; + 51200 bytes downloaded
[12:52:55] Initial: ABAE; + 61440 bytes downloaded
[12:52:55] Initial: B6B0; + 71680 bytes downloaded
[12:52:55] Initial: 783A; + 81920 bytes downloaded
[12:52:55] Initial: B2A6; + 92160 bytes downloaded
[12:52:55] Initial: 1409; + 102400 bytes downloaded
[12:52:55] Initial: BBF0; + 112640 bytes downloaded
[12:52:55] Initial: 1861; + 122880 bytes downloaded
[12:52:55] Initial: 5950; + 133120 bytes downloaded
[12:52:55] Initial: 1081; + 143360 bytes downloaded
[12:52:55] Initial: 26BC; + 153600 bytes downloaded
[12:52:55] Initial: FE4A; + 163840 bytes downloaded
[12:52:55] Initial: C1C3; + 174080 bytes downloaded
[12:52:55] Initial: 9B49; + 184320 bytes downloaded
[12:52:55] Initial: 9EE5; + 194560 bytes downloaded
[12:52:55] Initial: D79D; + 204800 bytes downloaded
[12:52:55] Initial: 7801; + 215040 bytes downloaded
[12:52:55] Initial: 8B51; + 225280 bytes downloaded
[12:52:55] Initial: E26E; + 235520 bytes downloaded
[12:52:55] Initial: EDB0; + 245760 bytes downloaded
[12:52:55] Initial: 0919; + 256000 bytes downloaded
[12:52:55] Initial: CDDE; + 266240 bytes downloaded
[12:52:55] Initial: 7A7E; + 276480 bytes downloaded
[12:52:55] Initial: 034E; + 286720 bytes downloaded
[12:52:55] Initial: 88D0; + 296960 bytes downloaded
[12:52:55] Initial: D66D; + 307200 bytes downloaded
[12:52:55] Initial: 6A52; + 317440 bytes downloaded
[12:52:55] Initial: B478; + 327680 bytes downloaded
[12:52:55] Initial: CF8A; + 337920 bytes downloaded
[12:52:55] Initial: 8407; + 348160 bytes downloaded
[12:52:55] Initial: 2246; + 358400 bytes downloaded
[12:52:55] Initial: 1C69; + 368640 bytes downloaded
[12:52:55] Initial: 1287; + 378880 bytes downloaded
[12:52:55] Initial: 19B3; + 389120 bytes downloaded
[12:52:55] Initial: 1AD1; + 399360 bytes downloaded
[12:52:55] Initial: 5791; + 409600 bytes downloaded
[12:52:55] Initial: 76C5; + 419840 bytes downloaded
[12:52:55] Initial: 9B77; + 430080 bytes downloaded
[12:52:55] Initial: E82F; + 440320 bytes downloaded
[12:52:55] Initial: D0D3; + 450560 bytes downloaded
[12:52:55] Initial: 0F5E; + 460800 bytes downloaded
[12:52:55] Initial: D743; + 471040 bytes downloaded
[12:52:55] Initial: 0B7C; + 481280 bytes downloaded
[12:52:55] Initial: FAFD; + 491520 bytes downloaded
[12:52:55] Initial: 0E14; + 501760 bytes downloaded
[12:52:55] Initial: 4048; + 512000 bytes downloaded
[12:52:55] Initial: 21A5; + 522240 bytes downloaded
[12:52:55] Initial: C1A5; + 532480 bytes downloaded
[12:52:55] Initial: F716; + 542720 bytes downloaded
[12:52:55] Initial: DD98; + 552960 bytes downloaded
[12:52:55] Initial: 9F7B; + 563200 bytes downloaded
[12:52:56] Initial: 1CC0; + 573440 bytes downloaded
[12:52:56] Initial: 4D37; + 583680 bytes downloaded
[12:52:56] Initial: 222A; + 593920 bytes downloaded
[12:52:56] Initial: 8E33; + 604160 bytes downloaded
[12:52:56] Initial: D3C9; + 614400 bytes downloaded
[12:52:56] Initial: 9821; + 624640 bytes downloaded
[12:52:56] Initial: 236E; + 634880 bytes downloaded
[12:52:56] Initial: 1A7A; + 645120 bytes downloaded
[12:52:56] Initial: 6D64; + 655360 bytes downloaded
[12:52:56] Initial: 4ADC; + 665600 bytes downloaded
[12:52:56] Initial: 3854; + 675840 bytes downloaded
[12:52:56] Initial: CB5C; + 686080 bytes downloaded
[12:52:56] Initial: 2A88; + 696320 bytes downloaded
[12:52:56] Initial: 1199; + 706560 bytes downloaded
[12:52:56] Initial: 0512; + 716800 bytes downloaded
[12:52:56] Initial: 316E; + 727040 bytes downloaded
[12:52:56] Initial: D89D; + 737280 bytes downloaded
[12:52:56] Initial: E6A3; + 747520 bytes downloaded
[12:52:56] Initial: B488; + 757760 bytes downloaded
[12:52:56] Initial: BAFD; + 768000 bytes downloaded
[12:52:56] Initial: 34A0; + 778240 bytes downloaded
[12:52:56] Initial: DD6C; + 788480 bytes downloaded
[12:52:56] Initial: D2E9; + 789667 bytes downloaded
[12:52:56] Verifying core Core_a1.fah...
[12:52:56] Signature is VALID
[12:52:56] 
[12:52:56] Trying to unzip core FahCore_a1.exe
[12:52:56] Decompressed FahCore_a1.exe (2035712 bytes) successfully
[12:53:01] + Core successfully engaged
[12:53:06] 
[12:53:06] + Processing work unit
[12:53:06] Work type a1 not eligible for variable processors
[12:53:06] Core required: FahCore_a1.exe
[12:53:06] Core found.
[12:53:06] Using generic mpiexec calls
[12:53:06] Working on queue slot 01 [September 11 12:53:06 UTC]
[12:53:06] + Working ...
[12:53:06] - Calling 'mpiexec -np 4 -channel auto -host 127.0.0.1 FahCore_a1.exe -dir work/ -suffix 01 -checkpoint 30 -verbose -lifeline 2136 -version 622'

[12:53:06] 
[12:53:06] *------------------------------*
[12:53:06] Folding@Home Gromacs SMP Core
[12:53:06] Version 1.74 (March 10, 2007)
[12:53:06] 
[12:53:06] Preparing to commence simulation
[12:53:06] - Ensuring status. Please wait.
[12:53:21] - Starting from initial work packet
[12:53:22] 
[12:53:22] Project: 2665 (Run 2, Clone 313, Gen 49)
[12:53:22] 
[12:53:23] Assembly optimizations on if available.
[12:53:23] Entering M.D.
[12:53:38]  percent)
[12:53:39] cket
[12:53:39] 
[12:53:39] Project: 2665 (Run 2, Clone 313, Gen 49)
[12:53:39] 
[12:53:39] 65 (Run 2, Clone 313, Gen 49)
[12:53:39] 
[12:53:40] Entering M.D.
[12:53:46] Rejecting checkpoint
[12:53:48] Protein: HGG with glycosylations
[12:53:48] Writing local files
[12:54:01] Extra SSE boost OK.
[12:54:02] Writing local files
[12:54:02] Completed 0 out of 250000 steps  (0 percent)
[13:15:14] Writing local files
[13:15:14] Completed 2500 out of 250000 steps  (1 percent)
[13:36:25] Writing local files
[13:36:26] Completed 5000 out of 250000 steps  (2 percent)
[13:57:37] Writing local files
[13:57:37] Completed 7500 out of 250000 steps  (3 percent)
[14:18:47] Writing local files
[14:18:47] Completed 10000 out of 250000 steps  (4 percent)
[14:40:01] Writing local files
[14:40:02] Completed 12500 out of 250000 steps  (5 percent)
------------------------------------------------------------------------------------------------------------
[22:24:40] Completed 237500 out of 250000 steps  (95 percent)
[22:45:46] Writing local files
[22:45:46] Completed 240000 out of 250000 steps  (96 percent)
[23:06:51] Writing local files
[23:06:51] Completed 242500 out of 250000 steps  (97 percent)
[23:27:56] Writing local files
[23:27:56] Completed 245000 out of 250000 steps  (98 percent)
[23:49:03] Writing local files
[23:49:04] Completed 247500 out of 250000 steps  (99 percent)
[00:10:10] Writing local files
[00:10:10] Completed 250000 out of 250000 steps  (100 percent)
[00:10:10] Writing final coordinates.
[00:10:12] Past main M.D. loop
[00:10:12] Will end MPI now
[00:11:12] 
[00:11:12] Finished Work Unit:
[00:11:12] - Reading up to 21421872 from "work/wudata_01.arc": Read 21421872
[00:11:12] - Reading up to 595872 from "work/wudata_01.xtc": Read 595872
[00:11:12] goefile size: 0
[00:11:12] logfile size: 203295
[00:11:12] Leaving Run
[00:11:12] - Writing 22227411 bytes of core data to disk...
[00:11:13]   ... Done.
[00:11:13] - Failed to delete work/wudata_01.sas
[00:11:13] - Failed to delete work/wudata_01.goe
[00:11:13] Warning:  check for stray files
[00:11:13] - Shutting down core
[00:11:13] 
[00:11:13] Folding@home Core Shutdown: FINISHED_UNIT
[00:11:13] 
[00:11:13] Folding@home Core Shutdown: FINISHED_UNIT
[00:13:20] CoreStatus = 7B (123)
[00:13:20] Client-core communications error: ERROR 0x7b
[00:13:20] Deleting current work unit & continuing...
[00:13:20] Using generic mpiexec calls
[00:15:24] - Warning: Could not delete all work unit files (1): Core returned invalid code
[00:15:24] Trying to send all finished work units
[00:15:24] + No unsent completed units remaining.

Code: Select all

[00:15:24] - Preparing to get new work unit...
[00:15:24] + Attempting to get work packet
[00:15:24] - Will indicate memory of 2046 MB
[00:15:24] - Connecting to assignment server
[00:15:24] Connecting to http://assign.stanford.edu:8080/
[00:15:25] Posted data.
[00:15:25] Initial: 40AB; - Successful: assigned to (171.64.65.64).
[00:15:25] + News From Folding@Home: Welcome to Folding@Home
[00:15:25] Loaded queue successfully.
[00:15:25] Connecting to http://171.64.65.64:8080/
[00:15:26] Posted data.
[00:15:26] Initial: 0000; + Could not connect to Work Server
[00:15:26] - Attempt #1  to get work failed, and no other work to do.
Waiting before retry.
[00:15:42] + Attempting to get work packet
[00:15:42] - Will indicate memory of 2046 MB
[00:15:42] - Connecting to assignment server
[00:15:42] Connecting to http://assign.stanford.edu:8080/
[00:15:43] Posted data.
[00:15:43] Initial: 40AB; - Successful: assigned to (171.64.65.64).
[00:15:43] + News From Folding@Home: Welcome to Folding@Home
[00:15:43] Loaded queue successfully.
[00:15:43] Connecting to http://171.64.65.64:8080/
[00:15:48] Posted data.
[00:15:48] Initial: 0000; - Receiving payload (expected size: 4655125)
[00:15:57] - Downloaded at ~505 kB/s
[00:15:57] - Averaged speed for that direction ~515 kB/s
[00:15:57] + Received work.
[00:15:57] + Closed connections
[00:16:02] 
[00:16:02] + Processing work unit
[00:16:02] Work type a1 not eligible for variable processors
[00:16:02] Core required: FahCore_a1.exe
[00:16:02] Core found.
[00:16:02] Using generic mpiexec calls
[00:16:02] Working on queue slot 02 [September 13 00:16:02 UTC]
[00:16:02] + Working ...
[00:16:02] - Calling 'mpiexec -np 4 -channel auto -host 127.0.0.1 FahCore_a1.exe -dir work/ -suffix 02 -checkpoint 30 -verbose -lifeline 2136 -version 622'

[00:16:02] 
[00:16:02] *------------------------------*
[00:16:02] Folding@Home Gromacs SMP Core
[00:16:02] Version 1.74 (March 10, 2007)
[00:16:02] 
[00:16:02] Preparing to commence simulation
[00:16:02] - Ensuring status. Please wait.
[00:16:09] - Starting from initial work packet
[00:16:09] 
[00:16:09] Project: 2665 (Run 1, Clone 505, Gen 50)
[00:16:09] 
[00:16:09] Assembly optimizations on if available.
[00:16:09] Entering M.D.
[00:16:39]  percent)
[00:16:39] - Starting from initial work packet
[00:16:39] 
[00:16:39] Project: 2665 (Run 1, Clone 505, Gen 50)
[00:16:39] 
[00:16:41] Entering M.D.
[00:16:47] Rejecting checkpoint
[00:16:49] Protein: IBX in water
[00:16:49] Writing local files
[00:17:00] Extra SSE boost OK.
[00:17:00] Writing local files
[00:17:00] Completed 0 out of 250000 steps  (0 percent)
[00:32:17] Killing all core threads
[00:32:17] Killing 4 cores
[00:32:17] Killing core 0
[00:32:17] Killing core 1
[00:32:17] Killing core 2
[00:32:17] Killing core 3

Folding@Home Client Shutdown at user request.

OK... I think I used qfix correctly w/ toTOW's instructions...here is what resulted....

Code: Select all

--- Opening Log file [September 13 01:02:49 UTC] 


# Windows SMP Console Edition #################################################
###############################################################################

                       Folding@Home Client Version 6.22 SMP Beta2r3

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: C:\Documents and Settings\RL\My Documents\SMP Client
Executable: C:\Documents and Settings\RL\My Documents\SMP Client\Folding@home-Win32-x86.exe
Arguments: -smp -verbosity 9 -send all 

[01:02:49] - Ask before connecting: No
[01:02:49] - User name: MDCRL (Team 35275)
[01:02:49] - User ID: 3FF6AE732BBDC17A
[01:02:49] - Machine ID: 1
[01:02:49] 
[01:02:49] Loaded queue successfully.
[01:02:49] Attempting to return result(s) to server...
[01:02:49] Trying to send all finished work units
[01:02:49] + No unsent completed units remaining.
[01:02:49] ***** Got a SIGTERM signal (2)
[01:02:49] Killing all core threads
[01:02:49] Killing 4 cores
[01:02:49] Killing core 0
[01:02:49] Killing core 1
[01:02:49] Killing core 2
[01:02:49] Killing core 3

Folding@Home Client Shutdown.


--- Opening Log file [September 13 01:03:50 UTC] 


# Windows SMP Console Edition #################################################
###############################################################################

                       Folding@Home Client Version 6.22 SMP Beta2r3

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: C:\Documents and Settings\RL\My Documents\SMP Client
Executable: C:\Documents and Settings\RL\My Documents\SMP Client\Folding@home-Win32-x86.exe
Arguments: -smp -verbosity 9 

[01:03:50] - Ask before connecting: No
[01:03:50] - User name: MDCRL (Team 35275)
[01:03:50] - User ID: 3FF6AE732BBDC17A
[01:03:50] - Machine ID: 1
[01:03:50] 
[01:03:50] Loaded queue successfully.
[01:03:50] 
[01:03:50] - Autosending finished units... [September 13 01:03:50 UTC]
[01:03:50] + Processing work unit
[01:03:50] Trying to send all finished work units
[01:03:50] Work type a1 not eligible for variable processors
[01:03:50] + No unsent completed units remaining.
[01:03:50] Core required: FahCore_a1.exe
[01:03:50] - Autosend completed
[01:03:50] Core found.
[01:03:50] Using generic mpiexec calls
[01:03:50] Working on queue slot 02 [September 13 01:03:50 UTC]
[01:03:50] + Working ...
[01:03:50] - Calling 'mpiexec -np 4 -channel auto -host 127.0.0.1 FahCore_a1.exe -dir work/ -suffix 02 -checkpoint 30 -verbose -lifeline 3436 -version 622'

[01:03:50] 
[01:03:50] *------------------------------*
[01:03:50] Folding@Home Gromacs SMP Core
[01:03:50] Version 1.74 (March 10, 2007)
[01:03:50] 
[01:03:50] Preparing to commence simulation
[01:03:50] - Ensuring status. Please wait.
[01:04:07] - Looking at optimizations...
[01:04:07] - Working with standard loops on this execution.
[01:04:07] - Previous termination of core was improper.
[01:04:07] - Going to use standard loops.
[01:04:07] - Files status OK
[01:04:31] - Expanded 4654613 -> 24111057 (decompressed 518.0 percent)
[01:04:31] 
[01:04:31] Project: 2665 (Run 1, Clone 505, Gen 50)
[01:04:31] 
[01:04:34] Entering M.D.
[01:04:42] Calling FAH init
[01:04:43] Read topology
[01:04:44] ocal files
[01:04:44] rom checkpoint)
[01:04:44] Read checkpoint
[01:04:44] Protein: IBX in water
[01:04:44] Writing local files
[01:04:55] Extra SSE boost OK.
[01:04:55] Writing local files
[01:04:55] Completed 0 out of 250000 steps  (0 percent)
it looks like it ran - then closed the box... I then restarted the client w/ regular -smp -verbosity shortcut and it seems to be running again...

Can some1 please let me know if I did this correctly and did indeed have the partial results sent

Thanks.....

Re: Project: 2665 (Run 2, Clone 903, Gen 42) multiple failures

Posted: Sat Sep 13, 2008 8:29 am
by MstrBlstr
This says No.
+ No unsent completed units remaining.
And this is probably why.
[00:13:20] Client-core communications error: ERROR 0x7b
[00:13:20] Deleting current work unit & continuing...
Without having the qfix output listed though, it is hard to tell. If the client actually deleted the workunit files, qfix would not have had anything to fix.

Re: Project: 2665 (Run 2, Clone 903, Gen 42) multiple failures

Posted: Sat Sep 13, 2008 7:34 pm
by MDCRL
That's what I figured - Thanks 4 checkin

Re: Project: 2665 (Run 2, Clone 903, Gen 42) multiple failures

Posted: Sat Sep 13, 2008 7:53 pm
by ChelseaOilman
MDCRL wrote:Can some1 please let me know if I did this correctly and did indeed have the partial results sent
Hi MDCRL (team 35275),
Your WU (P2665 R2 C313 G49) was added to the stats database on 2008-09-12 18:43:46 for 0 points of credit.

Re: Project: 2665 (Run 2, Clone 903, Gen 42) multiple failures

Posted: Sun Sep 14, 2008 6:22 pm
by MDCRL
Chelsea: OK - cool, at least it got added to the project.... Thanks

After all the hassle w/ that set of WU's.. the latest 2665 WU (Run 1, Clone 505, Gen 50) seems to be going well- & tough too!
It even survived a fubar by myself - went to move some wires in this machine, and accidentally DC'd the HDD power cable :oops:

But once I caught it (Several hrs later) -Had a "I will now commit suicide" type of mssg in the cmd prompt window -reconnected, rebooted - and it started right back up where it left off and completed w/ successful send :D

Code: Select all

--- Opening Log file [September 13 01:03:50 UTC] 


# Windows SMP Console Edition #################################################
###############################################################################

                       Folding@Home Client Version 6.22 SMP Beta2r3

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: C:\Documents and Settings\RL\My Documents\SMP Client
Executable: C:\Documents and Settings\RL\My Documents\SMP Client\Folding@home-Win32-x86.exe
Arguments: -smp -verbosity 9 

[01:03:50] - Ask before connecting: No
[01:03:50] - User name: MDCRL (Team 35275)
[01:03:50] - User ID: 3FF6AE732BBDC17A
[01:03:50] - Machine ID: 1
[01:03:50] 
[01:03:50] Loaded queue successfully.
[01:03:50] 
[01:03:50] - Autosending finished units... [September 13 01:03:50 UTC]
[01:03:50] + Processing work unit
[01:03:50] Trying to send all finished work units
[01:03:50] Work type a1 not eligible for variable processors
[01:03:50] + No unsent completed units remaining.
[01:03:50] Core required: FahCore_a1.exe
[01:03:50] - Autosend completed
[01:03:50] Core found.
[01:03:50] Using generic mpiexec calls
[01:03:50] Working on queue slot 02 [September 13 01:03:50 UTC]
[01:03:50] + Working ...
[01:03:50] - Calling 'mpiexec -np 4 -channel auto -host 127.0.0.1 FahCore_a1.exe -dir work/ -suffix 02 -checkpoint 30 -verbose -lifeline 3436 -version 622'

[01:03:50] 
[01:03:50] *------------------------------*
[01:03:50] Folding@Home Gromacs SMP Core
[01:03:50] Version 1.74 (March 10, 2007)
[01:03:50] 
[01:03:50] Preparing to commence simulation
[01:03:50] - Ensuring status. Please wait.
[01:04:07] - Looking at optimizations...
[01:04:07] - Working with standard loops on this execution.
[01:04:07] - Previous termination of core was improper.
[01:04:07] - Going to use standard loops.
[01:04:07] - Files status OK
[01:04:31] - Expanded 4654613 -> 24111057 (decompressed 518.0 percent)
[01:04:31] 
[01:04:31] Project: 2665 (Run 1, Clone 505, Gen 50)
[01:04:31] 
[01:04:34] Entering M.D.
[01:04:42] Calling FAH init
[01:04:43] Read topology
[01:04:44] ocal files
[01:04:44] rom checkpoint)
[01:04:44] Read checkpoint
[01:04:44] Protein: IBX in water
[01:04:44] Writing local files
[01:04:55] Extra SSE boost OK.
[01:04:55] Writing local files
[01:04:55] Completed 0 out of 250000 steps  (0 percent)
[01:24:10] Writing local files
[01:24:10] Completed 2500 out of 250000 steps  (1 percent)
[01:43:18] Writing local files
[01:43:19] Completed 5000 out of 250000 steps  (2 percent)
[02:02:26] Writing local files
[02:02:26] Completed 7500 out of 250000 steps  (3 percent)
[02:21:32] Writing local files
[02:21:32] Completed 10000 out of 250000 steps  (4 percent)
[02:40:36] Writing local files
[02:40:37] Completed 12500 out of 250000 steps  (5 percent)
-------------------------------------------------------------------------------------------------------------
[21:44:24] Completed 162500 out of 250000 steps  (65 percent)
[22:03:27] Writing local files
[22:03:27] Completed 165000 out of 250000 steps  (66 percent)
[22:22:31] Writing local files
[22:22:31] Completed 167500 out of 250000 steps  (67 percent)
[22:41:34] Writing local files
[22:41:34] Completed 170000 out of 250000 steps  (68 percent)


-----------------------D'OH !!!!!!!!! Last recorded Log entry-------------------------------------
     


                               -------> RESTARTED <------------



--- Opening Log file [September 14 04:55:48 UTC] 

# Windows SMP Console Edition #################################################
###############################################################################

                       Folding@Home Client Version 6.22 SMP Beta2r3

                          http://folding.stanford.edu
###############################################################################
###############################################################################
Launch directory: C:\Documents and Settings\RL\My Documents\SMP Client
Executable: C:\Documents and Settings\RL\My Documents\SMP Client\Folding@home-Win32-x86.exe
Arguments: -smp -verbosity 9 

[04:55:48] - Ask before connecting: No
[04:55:48] - User name: MDCRL (Team 35275)
[04:55:48] - User ID: 3FF6AE732BBDC17A
[04:55:48] - Machine ID: 1
[04:55:48] 
[04:55:49] Loaded queue successfully.
[04:55:49] 
[04:55:49] - Autosending finished units... [September 14 04:55:49 UTC]
[04:55:49] + Processing work unit
[04:55:49] Trying to send all finished work units
[04:55:49] Work type a1 not eligible for variable processors
[04:55:49] + No unsent completed units remaining.
[04:55:49] Core required: FahCore_a1.exe
[04:55:49] - Autosend completed
[04:55:49] Core found.
[04:55:49] Using generic mpiexec calls
[04:55:49] Working on queue slot 02 [September 14 04:55:49 UTC]
[04:55:49] + Working ...
[04:55:49] - Calling 'mpiexec -np 4 -channel auto -host 127.0.0.1 FahCore_a1.exe -dir work/ -suffix 02 -checkpoint 30 -verbose -lifeline 3492 -version 622'

[04:55:49] 
[04:55:49] *------------------------------*
[04:55:49] Folding@Home Gromacs SMP Core
[04:55:49] Version 1.74 (March 10, 2007)
[04:55:49] 
[04:55:49] Preparing to commence simulation
[04:55:49] - Ensuring status. Please wait.
[04:56:06] - Looking at optimizations...
[04:56:06] - Working with standard loops on this execution.
[04:56:06] Examination of work files indicates 8 consecutive improper terminations of core.
[04:56:31] - Expanded 4654613 -> 24111057 (decompressed 518.0 percent)
[04:56:31] 
[04:56:31] Project: 2665 (Run 1, Clone 505, Gen 50)
[04:56:31] 
[04:56:34] Entering M.D.
[04:56:41] Calling FAH init
[04:56:43] Read topology
[04:56:43] (Starting from checkpoint)
[04:56:43]  out of 250000 steps  (68 percent)
[04:56:43] er
[04:56:43] Writing local files
[04:56:43] Completed 170000 out of 250000 steps  (68 percent)
[04:56:53] Extra SSE boost OK.
[05:16:14] Writing local files
[05:16:15] Completed 172500 out of 250000 steps  (69 percent)
[05:35:36] Writing local files
[05:35:36] Completed 175000 out of 250000 steps  (70 percent)
-----------------------------------------------------------------------------------------------
[13:38:00] Writing local files
[13:38:01] Completed 237500 out of 250000 steps  (95 percent)
[13:57:17] Writing local files
[13:57:18] Completed 240000 out of 250000 steps  (96 percent)
[14:16:37] Writing local files
[14:16:37] Completed 242500 out of 250000 steps  (97 percent)
[14:35:57] Writing local files
[14:35:57] Completed 245000 out of 250000 steps  (98 percent)
[14:55:15] Writing local files
[14:55:16] Completed 247500 out of 250000 steps  (99 percent)
[15:14:33] Writing local files
[15:14:34] Completed 250000 out of 250000 steps  (100 percent)
[15:14:34] Writing final coordinates.
[15:14:35] Past main M.D. loop
[15:14:35] Will end MPI now
[15:15:35] 
[15:15:35] Finished Work Unit:
[15:15:35] - Reading up to 21193200 from "work/wudata_02.arc": Read 21193200
[15:15:35] - Reading up to 617164 from "work/wudata_02.xtc": Read 617164
[15:15:35] goefile size: 0
[15:15:35] logfile size: 231383
[15:15:35] Leaving Run
[15:15:35] - Writing 22048967 bytes of core data to disk...
[15:15:36]   ... Done.
[15:15:36] - Failed to delete work/wudata_02.sas
[15:15:36] - Failed to delete work/wudata_02.goe
[15:15:36] Warning:  check for stray files
[15:15:36] - Shutting down core
[15:17:36] 
[15:17:36] Folding@home Core Shutdown: FINISHED_UNIT
[15:17:36] 
[15:17:36] Folding@home Core Shutdown: FINISHED_UNIT
[15:17:41] CoreStatus = 64 (100)
[15:17:41] Unit 2 finished with 73 percent of time to deadline remaining.
[15:17:41] Updated performance fraction: 0.728966
[15:17:41] Sending work to server
[15:17:41] Project: 2665 (Run 1, Clone 505, Gen 50)


[15:17:41] + Attempting to send results [September 14 15:17:41 UTC]
[15:17:41] - Reading file work/wuresults_02.dat from core
[15:17:41]   (Read 22048967 bytes from disk)
[15:17:41] Connecting to http://171.64.65.64:8080/
[15:19:51] Posted data.
[15:19:51] Initial: 0000; - Uploaded at ~160 kB/s
[15:19:55] - Averaged speed for that direction ~149 kB/s
[15:19:55] + Results successfully sent
[15:19:55] Thank you for your contribution to Folding@Home.
[15:19:55] + Number of Units Completed: 2