Page 1 of 3

Various Project: 601x problems

Posted: Fri Mar 26, 2010 1:11 pm
by Aardvark
Following is a brief listing of the WUs that have failed for me. There are several more but the Log file for them has been superseded so I could not obtain specifics. Perhaps someone can see some pattern in what is going on. I presently have an a3core WU that seems to be folding normally. I will be returning it tonight (Friday 3/26/2010).

Project Run Clone Gen Comment
-------- ------ ------- ----- --------------
6012 0 420 82 Did <1%. EUE rtnd to Stanford

6024 0 37 75 Did <2%. Could not return

6025 0 154 95 Did <7%. Could not return

6012 2 266 13 Did <1%. Rtnd 13099 bytes to Stanford



Intermixed with the above WUs, the following WUs completed successfully and were normally returned to Stanford.

Project Run Clone Gen Comment
-------- ------ ------- ----- --------------
6025 0 97 80 Normal

6012 1 177 10 Normal

Re: Various Project: 601x problems

Posted: Sun Mar 28, 2010 6:32 am
by bruce
Hi Aardvark (team 48057),
Your WU (P6012 R0 C420 G82) was added to the stats database on 2010-03-25 07:04:37 for 4.36 points of credit.
It was redistributed and finished successfully by two people.

Project: 6024, Run 0, Clone 37, Gen 75 was successfully completed by someone else.
Project: 6025, Run 0, Clone 154, Gen 95 was successfully completed by someone else.

Your WU (P6012 R2 C266 G13) was added to the stats database on 2010-03-23 05:04:55 for 8 points of credit.
It was successfully completed by someone else.

Something is wrong with your hardware.

Re: Various Project: 601x problems

Posted: Sun Mar 28, 2010 12:56 pm
by Gred
I agree with Aardvark that problem not in specific WU, but in MacOS Core A3 v 2.17 (Mar 7 2010). I completely removed overclocking from my mac, and have a lot of WU fails. Today failed WU:
Project: 6012 (Run 0, Clone 275, Gen 87) Did < 2%, Folding@home Core Shutdown: UNSTABLE_MACHINE CoreStatus = 7A (122)
Project: 6012 (Run 0, Clone 282, Gen 91) Did < 2%, Folding@home Core Shutdown: EARLY_UNIT_END CoreStatus = 72 (114)
Project: 6015 (Run 1, Clone 163, Gen 62) Did < 1%, Folding@home Core Shutdown: EARLY_UNIT_END CoreStatus = 72 (114)
Project: 6015 (Run 1, Clone 172, Gen 46) Did < 1%, Folding@home Core Shutdown: UNSTABLE_MACHINE CoreStatus = 7A (122)
Project: 6012 (Run 2, Clone 54, Gen 34) Did < 3%, CoreStatus = 0 (0)

Bruce, was WUs which fails on our mac's returned by windows client or by mac client?

Re: Various Project: 601x problems

Posted: Sun Mar 28, 2010 3:47 pm
by kasson
Can you list your mac hardware and OS version number?

Re: Various Project: 601x problems

Posted: Sun Mar 28, 2010 3:53 pm
by Gred
I have mac 10.6.2 installed on PC with Core 2 Quad 9400 CPU and 4 * 2 GB DDR2 memory.

Re: Various Project: 601x problems

Posted: Sun Mar 28, 2010 5:01 pm
by ChelseaOilman
Gred wrote:mac 10.6.2 installed on PC
Are we talking Hackintosh?

Re: Various Project: 601x problems

Posted: Sun Mar 28, 2010 5:12 pm
by Gred
ChelseaOilman wrote:
Gred wrote:mac 10.6.2 installed on PC
Are we talking Hackintosh?
Yes

Re: Various Project: 601x problems

Posted: Sun Mar 28, 2010 5:29 pm
by kasson
mac-on-pc is hard for us to debug. Is anyone having these issues on native mac installs?

Re: Various Project: 601x problems

Posted: Sun Mar 28, 2010 6:35 pm
by Aardvark
I am encountering the problems on a 1.83Ghz Mac Mini that is running OSX10.6.2 . I have had it for some time, but it has never been opened. I consider it "stock".

I have to take some exception to bruce's observation that the problem is in the hardware. At this point, I would agree that the problem rests with (or some interaction between) hardware, system software & client software. The "nasty" get through our net if we start with too large a mesh...:-)

Re: Various Project: 601x problems

Posted: Sun Mar 28, 2010 6:40 pm
by Aardvark
The following is always included in the FAHlog.txt file when one of these failures occurs:

Code: Select all

[[12:48:45] Completed 0 out of 500000 steps  (0%)

-------------------------------------------------------
Program mdrun, VERSION 4.0.99-dev-20100305
Source code file: /Users/kasson/a3_devnew/gromacs/src/mdlib/pme.c, line: 563

Fatal error:
3 particles communicated to PME node 0 are more than a cell length out of the domain decomposition cell of their charge group in dimension x
For more information and tips for trouble shooting please check the GROMACS website at
http://www.gromacs.org/Documentation/Errors
-------------------------------------------------------

Thanx for Using GROMACS - Have a Nice Day

[13:08:52] mdrun returned 255
[13:08:52] Going to send back what have done -- stepsTotalG=500000
[13:08:52] Work fraction=0.0093 steps=500000.
[13:08:56] logfile size=12500 infoLength=12500 edr=0 trr=25
[13:08:56] logfile size: 12500 info=12500 bed=0 hdr=25
[13:08:56] - Writing 13038 bytes of core data to disk...
[13:08:56]   ... Done.
[13:08:56] 
[13:08:56] Folding@home Core Shutdown: EARLY_UNIT_END
[13:08:57] CoreStatus = 72 (114)
[13:08:57] Sending work to server
[13:08:57] Project: 6012 (Run 0, Clone 420, Gen 82)
I believe the answer is indicated by the reference to specific source code at the time of failure.

Re: Various Project: 601x problems

Posted: Sun Mar 28, 2010 8:31 pm
by kasson
Interesting--the log snippet is indeed helpful. Thanks.

Re: Various Project: 601x problems

Posted: Tue Mar 30, 2010 1:48 am
by AlanH
I have reported similar problems on my system. Since they were affecting a variety of WUs, I didn't report them here, but in the v2,17 core thread.

I've stopped folding, as I'm seeing three or four aborts at 10% or less for every successful WU I fold, and I figure it can't be doing your scheduling processes any good to get all those partial returns.

Here are my logs for the last day or so:

Code: Select all


# Mac OS X SMP Console Edition ################################################
###############################################################################

                       Folding@Home Client Version 6.29r1

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: /Users/alan/Library/FAH
Executable: ./fah6
Arguments: -smp -local -verbosity 9 -advmethods 

[11:27:47] - Ask before connecting: No
[11:27:47] - User name: AlanH (Team 47958)
[11:27:47] - User ID: 63DC229F081DE390
[11:27:47] - Machine ID: 1
[11:27:47] 
[11:27:47] Loaded queue successfully.
[11:27:47] 
[11:27:47] - Autosending finished units... [March 29 11:27:47 UTC]
[11:27:47] + Processing work unit
[11:27:47] Trying to send all finished work units
[11:27:47] + No unsent completed units remaining.
[11:27:47] - Autosend completed
[11:27:47] Core required: FahCore_a3.exe
[11:27:47] Core found.
[11:27:47] Working on queue slot 06 [March 29 11:27:47 UTC]
[11:27:47] + Working ...
[11:27:47] - Calling './FahCore_a3.exe -dir work/ -nice 19 -suffix 06 -np 4 -checkpoint 15 -verbose -lifeline 571 -version 629'

[11:27:47] 
[11:27:47] *------------------------------*
[11:27:47] Folding@Home Gromacs SMP Core
[11:27:47] Version 2.17 (Mar 7 2010)
[11:27:47] 
[11:27:47] Preparing to commence simulation
[11:27:47] - Ensuring status. Please wait.
[11:27:57] - Looking at optimizations...
[11:27:57] - Working with standard loops on this execution.
[11:27:57] - Previous termination of core was improper.
[11:27:57] - Files status OK
[11:27:57] - Expanded 1768989 -> 1971489 (decompressed 111.4 percent)
[11:27:57] Called DecompressByteArray: compressed_data_size=1768989 data_size=1971489, decompressed_data_size=1971489 diff=0
[11:27:57] - Digital signature verified
[11:27:57] 
[11:27:57] Project: 6020 (Run 0, Clone 189, Gen 76)
[11:27:57] 
[11:27:57] Entering M.D.
[11:28:03] Using Gromacs checkpoints
[11:28:04] Resuming from checkpoint
[11:28:04] Verified work/wudata_06.log
[11:28:05] Verified work/wudata_06.trr
[11:28:05] Verified work/wudata_06.edr
[11:28:05] Completed 336996 out of 500000 steps  (67%)
[11:34:41] Completed 340000 out of 500000 steps  (68%)

....
....
....
[16:16:05] Completed 495000 out of 500000 steps  (99%)
[16:25:23] Completed 500000 out of 500000 steps  (100%)
[16:25:24] DynamicWrapper: Finished Work Unit: sleep=10000
[16:25:34] 
[16:25:34] Finished Work Unit:
[16:25:34] - Reading up to 20302920 from "work/wudata_06.trr": Read 20302920
[16:25:34] trr file hash check passed.
[16:25:35] edr file hash check passed.
[16:25:35] logfile size: 59116
[16:25:35] Leaving Run
[16:25:39] - Writing 20396320 bytes of core data to disk...
[16:25:40]   ... Done.
[16:25:42] - Shutting down core
[16:25:42] 
[16:25:42] Folding@home Core Shutdown: FINISHED_UNIT
[16:25:43] CoreStatus = 64 (100)
[16:25:43] Unit 6 finished with 87 percent of time to deadline remaining.
[16:25:43] Updated performance fraction: 0.860129
[16:25:43] Sending work to server
[16:25:43] Project: 6020 (Run 0, Clone 189, Gen 76)


[16:25:43] + Attempting to send results [March 29 16:25:43 UTC]
[16:25:43] - Reading file work/wuresults_06.dat from core
[16:25:43]   (Read 20396320 bytes from disk)
[16:25:43] Connecting to http://171.64.65.54:8080/
[16:32:57] Posted data.
[16:32:58] Initial: 0000; - Uploaded at ~45 kB/s
[16:32:58] - Averaged speed for that direction ~24 kB/s
[16:32:58] + Results successfully sent
[16:32:58] Thank you for your contribution to Folding@Home.
[16:32:58] + Number of Units Completed: 51

[16:33:00] Trying to send all finished work units
[16:33:00] + No unsent completed units remaining.
[16:33:00] - Preparing to get new work unit...
[16:33:00] Cleaning up work directory
[16:33:00] + Attempting to get work packet
[16:33:00] Passkey found
[16:33:00] - Will indicate memory of 4096 MB
[16:33:00] - Detect CPU. Vendor: GenuineIntel, Family: 6, Model: 15, Stepping: 6
[16:33:00] - Connecting to assignment server
[16:33:00] Connecting to http://assign.stanford.edu:8080/
[16:33:01] Posted data.
[16:33:01] Initial: ED82; - Successful: assigned to (130.237.232.140).
[16:33:01] + News From Folding@Home: Welcome to Folding@Home
[16:33:01] Loaded queue successfully.
[16:33:01] Sent data
[16:33:01] Connecting to http://130.237.232.140:8080/
[16:33:02] Posted data.
[16:33:02] Initial: 0000; - Receiving payload (expected size: 1799281)
[16:33:22] - Downloaded at ~87 kB/s
[16:33:22] - Averaged speed for that direction ~130 kB/s
[16:33:22] + Received work.
[16:33:22] Trying to send all finished work units
[16:33:22] + No unsent completed units remaining.
[16:33:22] + Closed connections
[16:33:22] 
[16:33:22] + Processing work unit
[16:33:22] Core required: FahCore_a3.exe
[16:33:22] Core found.
[16:33:22] Working on queue slot 07 [March 29 16:33:22 UTC]
[16:33:22] + Working ...
[16:33:22] - Calling './FahCore_a3.exe -dir work/ -nice 19 -suffix 07 -np 4 -checkpoint 15 -verbose -lifeline 571 -version 629'

[16:33:23] 
[16:33:23] *------------------------------*
[16:33:23] Folding@Home Gromacs SMP Core
[16:33:23] Version 2.17 (Mar 7 2010)
[16:33:23] 
[16:33:23] Preparing to commence simulation
[16:33:23] - Looking at optimizations...
[16:33:23] - Created dyn
[16:33:23] - Files status OK
[16:33:23] - Expanded 1798769 -> 2392545 (decompressed 133.0 percent)
[16:33:23] Called DecompressByteArray: compressed_data_size=1798769 data_size=2392545, decompressed_data_size=2392545 diff=0
[16:33:23] - Digital signature verified
[16:33:23] 
[16:33:23] Project: 6015 (Run 0, Clone 135, Gen 97)
[16:33:23] 
[16:33:23] Assembly optimizations on if available.
[16:33:23] Entering M.D.
[16:33:30] Completed 0 out of 500000 steps  (0%)
[16:42:06] Completed 5000 out of 500000 steps  (1%)
[16:50:38] Completed 10000 out of 500000 steps  (2%)
[16:59:07] Completed 15000 out of 500000 steps  (3%)
[17:07:42] Completed 20000 out of 500000 steps  (4%)
[17:16:06] Completed 25000 out of 500000 steps  (5%)
[17:25:14] Completed 30000 out of 500000 steps  (6%)
[17:26:51] mdrun returned 255
[17:26:51] Going to send back what have done -- stepsTotalG=500000
[17:26:51] Work fraction=0.0619 steps=500000.
[17:26:55] logfile size=14677 infoLength=14677 edr=0 trr=25
[17:26:55] logfile size: 14677 info=14677 bed=0 hdr=25
[17:26:55] - Writing 15215 bytes of core data to disk...
[17:26:55]   ... Done.
[17:26:56] 
[17:26:56] Folding@home Core Shutdown: UNSTABLE_MACHINE
[17:26:56] CoreStatus = 7A (122)
[17:26:56] Sending work to server
[17:26:56] Project: 6015 (Run 0, Clone 135, Gen 97)


[17:26:56] + Attempting to send results [March 29 17:26:56 UTC]
[17:26:56] - Reading file work/wuresults_07.dat from core
[17:26:56]   (Read 15215 bytes from disk)
[17:26:56] Connecting to http://130.237.232.140:8080/
[17:26:56] Posted data.
[17:26:56] Initial: 0000; - Uploaded at ~15 kB/s
[17:26:57] - Averaged speed for that direction ~22 kB/s
[17:26:57] + Results successfully sent
[17:26:57] Thank you for your contribution to Folding@Home.
[17:26:57] Trying to send all finished work units
[17:26:57] + No unsent completed units remaining.
[17:26:57] - Preparing to get new work unit...
[17:26:57] Cleaning up work directory
[17:26:57] + Attempting to get work packet
[17:26:57] Passkey found
[17:26:57] - Will indicate memory of 4096 MB
[17:26:57] - Connecting to assignment server
[17:26:57] Connecting to http://assign.stanford.edu:8080/
[17:26:58] Posted data.
[17:26:58] Initial: ED82; - Successful: assigned to (130.237.232.140).
[17:26:58] + News From Folding@Home: Welcome to Folding@Home
[17:26:58] Loaded queue successfully.
[17:26:58] Sent data
[17:26:58] Connecting to http://130.237.232.140:8080/
[17:26:59] Posted data.
[17:26:59] Initial: 0000; - Receiving payload (expected size: 1799811)
[17:27:10] - Downloaded at ~159 kB/s
[17:27:10] - Averaged speed for that direction ~136 kB/s
[17:27:10] + Received work.
[17:27:10] Trying to send all finished work units
[17:27:10] + No unsent completed units remaining.
[17:27:10] + Closed connections
[17:27:15] 
[17:27:15] + Processing work unit
[17:27:15] Core required: FahCore_a3.exe
[17:27:15] Core found.
[17:27:15] Working on queue slot 08 [March 29 17:27:15 UTC]
[17:27:15] + Working ...
[17:27:15] - Calling './FahCore_a3.exe -dir work/ -nice 19 -suffix 08 -np 4 -checkpoint 15 -verbose -lifeline 571 -version 629'

[17:27:15] 
[17:27:15] *------------------------------*
[17:27:15] Folding@Home Gromacs SMP Core
[17:27:15] Version 2.17 (Mar 7 2010)
[17:27:15] 
[17:27:15] Preparing to commence simulation
[17:27:15] - Looking at optimizations...
[17:27:15] - Created dyn
[17:27:15] - Files status OK
[17:27:16] - Expanded 1799299 -> 2396877 (decompressed 133.2 percent)
[17:27:16] Called DecompressByteArray: compressed_data_size=1799299 data_size=2396877, decompressed_data_size=2396877 diff=0
[17:27:16] - Digital signature verified
[17:27:16] 
[17:27:16] Project: 6014 (Run 1, Clone 32, Gen 62)
[17:27:16] 
[17:27:16] Assembly optimizations on if available.
[17:27:16] Entering M.D.
[17:27:22] Completed 0 out of 500000 steps  (0%)
[17:27:48] - Autosending finished units... [March 29 17:27:48 UTC]
[17:27:48] Trying to send all finished work units
[17:27:48] + No unsent completed units remaining.
[17:27:48] - Autosend completed
[17:35:55] Completed 5000 out of 500000 steps  (1%)
[17:44:21] Completed 10000 out of 500000 steps  (2%)
[17:52:50] Completed 15000 out of 500000 steps  (3%)
[17:59:10] mdrun returned 255
[17:59:10] Going to send back what have done -- stepsTotalG=500000
[17:59:10] Work fraction=0.0375 steps=500000.
[17:59:14] logfile size=13249 infoLength=13249 edr=0 trr=25
[17:59:14] logfile size: 13249 info=13249 bed=0 hdr=25
[17:59:14] - Writing 13787 bytes of core data to disk...
[17:59:14]   ... Done.
[17:59:14] 
[17:59:14] Folding@home Core Shutdown: EARLY_UNIT_END
[17:59:14] CoreStatus = 72 (114)
[17:59:14] Sending work to server
[17:59:14] Project: 6014 (Run 1, Clone 32, Gen 62)


[17:59:14] + Attempting to send results [March 29 17:59:14 UTC]
[17:59:14] - Reading file work/wuresults_08.dat from core
[17:59:14]   (Read 13787 bytes from disk)
[17:59:14] Connecting to http://130.237.232.140:8080/
[17:59:15] Posted data.
[17:59:15] Initial: 0000; - Uploaded at ~7 kB/s
[17:59:16] - Averaged speed for that direction ~19 kB/s
[17:59:16] + Results successfully sent
[17:59:16] Thank you for your contribution to Folding@Home.
[17:59:16] Trying to send all finished work units
[17:59:16] + No unsent completed units remaining.
[17:59:16] - Preparing to get new work unit...
[17:59:16] Cleaning up work directory
[17:59:16] + Attempting to get work packet
[17:59:16] Passkey found
[17:59:16] - Will indicate memory of 4096 MB
[17:59:16] - Connecting to assignment server
[17:59:16] Connecting to http://assign.stanford.edu:8080/
[17:59:17] Posted data.
[17:59:17] Initial: ED82; - Successful: assigned to (130.237.232.140).
[17:59:17] + News From Folding@Home: Welcome to Folding@Home
[17:59:17] Loaded queue successfully.
[17:59:17] Sent data
[17:59:17] Connecting to http://130.237.232.140:8080/
[17:59:18] Posted data.
[17:59:18] Initial: 0000; - Receiving payload (expected size: 1797319)
[17:59:38] - Downloaded at ~87 kB/s
[17:59:38] - Averaged speed for that direction ~126 kB/s
[17:59:38] + Received work.
[17:59:38] Trying to send all finished work units
[17:59:38] + No unsent completed units remaining.
[17:59:38] + Closed connections
[17:59:43] 
[17:59:43] + Processing work unit
[17:59:43] Core required: FahCore_a3.exe
[17:59:43] Core found.
[17:59:43] Working on queue slot 09 [March 29 17:59:43 UTC]
[17:59:43] + Working ...
[17:59:43] - Calling './FahCore_a3.exe -dir work/ -nice 19 -suffix 09 -np 4 -checkpoint 15 -verbose -lifeline 571 -version 629'

[17:59:43] 
[17:59:43] *------------------------------*
[17:59:43] Folding@Home Gromacs SMP Core
[17:59:43] Version 2.17 (Mar 7 2010)
[17:59:43] 
[17:59:43] Preparing to commence simulation
[17:59:43] - Looking at optimizations...
[17:59:43] - Created dyn
[17:59:43] - Files status OK
[17:59:43] - Expanded 1796807 -> 2078149 (decompressed 115.6 percent)
[17:59:43] Called DecompressByteArray: compressed_data_size=1796807 data_size=2078149, decompressed_data_size=2078149 diff=0
[17:59:43] - Digital signature verified
[17:59:43] 
[17:59:43] Project: 6012 (Run 2, Clone 394, Gen 10)
[17:59:43] 
[17:59:43] Assembly optimizations on if available.
[17:59:43] Entering M.D.
[17:59:50] Completed 0 out of 500000 steps  (0%)
[18:04:40] mdrun returned 255
[18:04:40] Going to send back what have done -- stepsTotalG=500000
[18:04:40] Work fraction=0.0057 steps=500000.
[18:04:44] logfile size=12374 infoLength=12374 edr=0 trr=25
[18:04:44] logfile size: 12374 info=12374 bed=0 hdr=25
[18:04:44] - Writing 12912 bytes of core data to disk...
[18:04:44]   ... Done.
[18:04:45] 
[18:04:45] Folding@home Core Shutdown: EARLY_UNIT_END
[18:04:45] CoreStatus = 72 (114)
[18:04:45] Sending work to server
[18:04:45] Project: 6012 (Run 2, Clone 394, Gen 10)


[18:04:45] + Attempting to send results [March 29 18:04:45 UTC]
[18:04:45] - Reading file work/wuresults_09.dat from core
[18:04:45]   (Read 12912 bytes from disk)
[18:04:45] Connecting to http://130.237.232.140:8080/
[18:04:46] Posted data.
[18:04:46] Initial: 0000; - Uploaded at ~6 kB/s
[18:04:47] - Averaged speed for that direction ~16 kB/s
[18:04:47] + Results successfully sent
[18:04:47] Thank you for your contribution to Folding@Home.
[18:04:47] Trying to send all finished work units
[18:04:47] + No unsent completed units remaining.
[18:04:47] - Preparing to get new work unit...
[18:04:47] Cleaning up work directory
[18:04:47] + Attempting to get work packet
[18:04:47] Passkey found
[18:04:47] - Will indicate memory of 4096 MB
[18:04:47] - Connecting to assignment server
[18:04:47] Connecting to http://assign.stanford.edu:8080/
[18:04:48] Posted data.
[18:04:48] Initial: ED82; - Successful: assigned to (130.237.232.140).
[18:04:48] + News From Folding@Home: Welcome to Folding@Home
[18:04:48] Loaded queue successfully.
[18:04:48] Sent data
[18:04:48] Connecting to http://130.237.232.140:8080/
[18:04:49] Posted data.
[18:04:49] Initial: 0000; - Receiving payload (expected size: 1798732)
[18:05:08] - Downloaded at ~92 kB/s
[18:05:08] - Averaged speed for that direction ~119 kB/s
[18:05:08] + Received work.
[18:05:09] Trying to send all finished work units
[18:05:09] + No unsent completed units remaining.
[18:05:09] + Closed connections
[18:05:14] 
[18:05:14] + Processing work unit
[18:05:14] Core required: FahCore_a3.exe
[18:05:14] Core found.
[18:05:14] Working on queue slot 00 [March 29 18:05:14 UTC]
[18:05:14] + Working ...
[18:05:14] - Calling './FahCore_a3.exe -dir work/ -nice 19 -suffix 00 -np 4 -checkpoint 15 -verbose -lifeline 571 -version 629'

[18:05:14] 
[18:05:14] *------------------------------*
[18:05:14] Folding@Home Gromacs SMP Core
[18:05:14] Version 2.17 (Mar 7 2010)
[18:05:14] 
[18:05:14] Preparing to commence simulation
[18:05:14] - Looking at optimizations...
[18:05:14] - Created dyn
[18:05:14] - Files status OK
[18:05:14] - Expanded 1798220 -> 2392545 (decompressed 133.0 percent)
[18:05:14] Called DecompressByteArray: compressed_data_size=1798220 data_size=2392545, decompressed_data_size=2392545 diff=0
[18:05:14] - Digital signature verified
[18:05:14] 
[18:05:14] Project: 6015 (Run 1, Clone 73, Gen 62)
[18:05:14] 
[18:05:14] Assembly optimizations on if available.
[18:05:14] Entering M.D.
[18:05:21] Completed 0 out of 500000 steps  (0%)
[18:05:52] mdrun returned 255
[18:05:52] Going to send back what have done -- stepsTotalG=500000
[18:05:52] Work fraction=0.0006 steps=500000.
[18:05:56] logfile size=11858 infoLength=11858 edr=0 trr=25
[18:05:56] logfile size: 11858 info=11858 bed=0 hdr=25
[18:05:56] - Writing 12396 bytes of core data to disk...
[18:05:56]   ... Done.
[18:05:56] 
[18:05:56] Folding@home Core Shutdown: EARLY_UNIT_END
[18:05:57] CoreStatus = 72 (114)
[18:05:57] Sending work to server
[18:05:57] Project: 6015 (Run 1, Clone 73, Gen 62)


[18:05:57] + Attempting to send results [March 29 18:05:57 UTC]
[18:05:57] - Reading file work/wuresults_00.dat from core
[18:05:57]   (Read 12396 bytes from disk)
[18:05:57] Connecting to http://130.237.232.140:8080/
[18:05:58] Posted data.
[18:05:58] Initial: 0000; - Uploaded at ~13 kB/s
[18:05:58] - Averaged speed for that direction ~16 kB/s
[18:05:58] + Results successfully sent
[18:05:58] Thank you for your contribution to Folding@Home.
[18:05:58] Trying to send all finished work units
[18:05:58] + No unsent completed units remaining.
[18:05:58] - Preparing to get new work unit...
[18:05:58] Cleaning up work directory
[18:05:58] + Attempting to get work packet
[18:05:58] Passkey found
[18:05:58] - Will indicate memory of 4096 MB
[18:05:58] - Connecting to assignment server
[18:05:58] Connecting to http://assign.stanford.edu:8080/
[18:05:59] Posted data.
[18:05:59] Initial: ED82; - Successful: assigned to (130.237.232.140).
[18:05:59] + News From Folding@Home: Welcome to Folding@Home
[18:05:59] Loaded queue successfully.
[18:05:59] Sent data
[18:05:59] Connecting to http://130.237.232.140:8080/
[18:06:00] Posted data.
[18:06:00] Initial: 0000; - Receiving payload (expected size: 1797868)
[18:06:21] - Downloaded at ~83 kB/s
[18:06:21] - Averaged speed for that direction ~112 kB/s
[18:06:21] + Received work.
[18:06:21] Trying to send all finished work units
[18:06:21] + No unsent completed units remaining.
[18:06:21] + Closed connections
[18:06:26] 
[18:06:26] + Processing work unit
[18:06:26] Core required: FahCore_a3.exe
[18:06:26] Core found.
[18:06:26] Working on queue slot 01 [March 29 18:06:26 UTC]
[18:06:26] + Working ...
[18:06:26] - Calling './FahCore_a3.exe -dir work/ -nice 19 -suffix 01 -np 4 -checkpoint 15 -verbose -lifeline 571 -version 629'

[18:06:26] 
[18:06:26] *------------------------------*
[18:06:26] Folding@Home Gromacs SMP Core
[18:06:26] Version 2.17 (Mar 7 2010)
[18:06:26] 
[18:06:26] Preparing to commence simulation
[18:06:26] - Looking at optimizations...
[18:06:26] - Created dyn
[18:06:26] - Files status OK
[18:06:27] - Expanded 1797356 -> 2078149 (decompressed 115.6 percent)
[18:06:27] Called DecompressByteArray: compressed_data_size=1797356 data_size=2078149, decompressed_data_size=2078149 diff=0
[18:06:27] - Digital signature verified
[18:06:27] 
[18:06:27] Project: 6012 (Run 1, Clone 168, Gen 27)
[18:06:27] 
[18:06:27] Assembly optimizations on if available.
[18:06:27] Entering M.D.
[18:06:33] Completed 0 out of 500000 steps  (0%)
[18:14:58] Completed 5000 out of 500000 steps  (1%)
[18:24:31] Completed 10000 out of 500000 steps  (2%)
[18:34:12] Completed 15000 out of 500000 steps  (3%)
[18:34:28] mdrun returned 255
[18:34:28] Going to send back what have done -- stepsTotalG=500000
[18:34:28] Work fraction=0.0303 steps=500000.
[18:34:30] CoreStatus = 0 (0)
[18:34:30] Sending work to server
[18:34:30] Project: 6012 (Run 1, Clone 168, Gen 27)
[18:34:30] - Error: Could not get length of results file work/wuresults_01.dat
[18:34:30] - Error: Could not read unit 01 file. Removing from queue.
[18:34:30] Trying to send all finished work units
[18:34:30] + No unsent completed units remaining.
[18:34:30] - Preparing to get new work unit...
[18:34:30] Cleaning up work directory
[18:34:31] + Attempting to get work packet
[18:34:31] Passkey found
[18:34:31] - Will indicate memory of 4096 MB
[18:34:31] - Connecting to assignment server
[18:34:31] Connecting to http://assign.stanford.edu:8080/
[18:34:32] Posted data.
[18:34:32] Initial: ED82; - Successful: assigned to (130.237.232.140).
[18:34:32] + News From Folding@Home: Welcome to Folding@Home
[18:34:32] Loaded queue successfully.
[18:34:32] Sent data
[18:34:32] Connecting to http://130.237.232.140:8080/
[18:34:33] Posted data.
[18:34:33] Initial: 0000; - Receiving payload (expected size: 1797868)
[18:34:53] - Downloaded at ~87 kB/s
[18:34:53] - Averaged speed for that direction ~107 kB/s
[18:34:53] + Received work.
[18:34:53] Trying to send all finished work units
[18:34:53] + No unsent completed units remaining.
[18:34:53] + Closed connections
[18:34:58] 
[18:34:58] + Processing work unit
[18:34:58] Core required: FahCore_a3.exe
[18:34:58] Core found.
[18:34:58] Working on queue slot 02 [March 29 18:34:58 UTC]
[18:34:58] + Working ...
[18:34:58] - Calling './FahCore_a3.exe -dir work/ -nice 19 -suffix 02 -np 4 -checkpoint 15 -verbose -lifeline 571 -version 629'

[18:34:58] 
[18:34:58] *------------------------------*
[18:34:58] Folding@Home Gromacs SMP Core
[18:34:58] Version 2.17 (Mar 7 2010)
[18:34:58] 
[18:34:58] Preparing to commence simulation
[18:34:58] - Ensuring status. Please wait.
[18:35:07] - Looking at optimizations...
[18:35:07] - Working with standard loops on this execution.
[18:35:07] - Created dyn
[18:35:07] - Files status OK
[18:35:08] - Expanded 1797356 -> 2078149 (decompressed 115.6 percent)
[18:35:08] Called DecompressByteArray: compressed_data_size=1797356 data_size=2078149, decompressed_data_size=2078149 diff=0
[18:35:08] - Digital signature verified
[18:35:08] 
[18:35:08] Project: 6012 (Run 1, Clone 168, Gen 27)
[18:35:08] 
[18:35:08] Entering M.D.
[18:35:14] Completed 0 out of 500000 steps  (0%)
[18:43:52] Completed 5000 out of 500000 steps  (1%)
[18:52:21] Completed 10000 out of 500000 steps  (2%)
[19:00:45] Completed 15000 out of 500000 steps  (3%)
[19:00:58] mdrun returned 255
[19:00:58] Going to send back what have done -- stepsTotalG=500000
[19:00:58] Work fraction=0.0303 steps=500000.
[19:00:58] CoreStatus = 0 (0)
[19:00:58] Sending work to server
[19:00:58] Project: 6012 (Run 1, Clone 168, Gen 27)
[19:00:58] - Error: Could not get length of results file work/wuresults_02.dat
[19:00:58] - Error: Could not read unit 02 file. Removing from queue.
[19:00:58] Trying to send all finished work units
[19:00:58] + No unsent completed units remaining.
[19:00:58] - Preparing to get new work unit...
[19:00:58] Cleaning up work directory
[19:00:59] + Attempting to get work packet
[19:00:59] Passkey found
[19:00:59] - Will indicate memory of 4096 MB
[19:00:59] - Connecting to assignment server
[19:00:59] Connecting to http://assign.stanford.edu:8080/
[19:01:00] Posted data.
[19:01:00] Initial: ED82; - Successful: assigned to (130.237.232.140).
[19:01:00] + News From Folding@Home: Welcome to Folding@Home
[19:01:00] Loaded queue successfully.
[19:01:00] Sent data
[19:01:00] Connecting to http://130.237.232.140:8080/
[19:01:01] Posted data.
[19:01:01] Initial: 0000; - Receiving payload (expected size: 1797868)
[19:01:20] - Downloaded at ~92 kB/s
[19:01:20] - Averaged speed for that direction ~104 kB/s
[19:01:20] + Received work.
[19:01:20] Trying to send all finished work units
[19:01:20] + No unsent completed units remaining.
[19:01:20] + Closed connections
[19:01:25] 
[19:01:25] + Processing work unit
[19:01:25] Core required: FahCore_a3.exe
[19:01:25] Core found.
[19:01:25] Working on queue slot 03 [March 29 19:01:25 UTC]
[19:01:25] + Working ...
[19:01:25] - Calling './FahCore_a3.exe -dir work/ -nice 19 -suffix 03 -np 4 -checkpoint 15 -verbose -lifeline 571 -version 629'

[19:01:25] 
[19:01:25] *------------------------------*
[19:01:25] Folding@Home Gromacs SMP Core
[19:01:25] Version 2.17 (Mar 7 2010)
[19:01:25] 
[19:01:25] Preparing to commence simulation
[19:01:25] - Ensuring status. Please wait.
[19:01:34] - Looking at optimizations...
[19:01:34] - Working with standard loops on this execution.
[19:01:34] - Created dyn
[19:01:34] - Files status OK
[19:01:35] - Expanded 1797356 -> 2078149 (decompressed 115.6 percent)
[19:01:35] Called DecompressByteArray: compressed_data_size=1797356 data_size=2078149, decompressed_data_size=2078149 diff=0
[19:01:35] - Digital signature verified
[19:01:35] 
[19:01:35] Project: 6012 (Run 1, Clone 168, Gen 27)
[19:01:35] 
[19:01:35] Entering M.D.
[19:01:41] Completed 0 out of 500000 steps  (0%)
[19:02:59] mdrun returned 255
[19:02:59] Going to send back what have done -- stepsTotalG=500000
[19:02:59] Work fraction=0.0013 steps=500000.
[19:03:03] logfile size=11829 infoLength=11829 edr=0 trr=25
[19:03:03] logfile size: 11829 info=11829 bed=0 hdr=25
[19:03:03] - Writing 12367 bytes of core data to disk...
[19:03:03]   ... Done.
[19:03:03] 
[19:03:03] Folding@home Core Shutdown: EARLY_UNIT_END
[19:03:04] CoreStatus = 72 (114)
[19:03:04] Sending work to server
[19:03:04] Project: 6012 (Run 1, Clone 168, Gen 27)


[19:03:04] + Attempting to send results [March 29 19:03:04 UTC]
[19:03:04] - Reading file work/wuresults_03.dat from core
[19:03:04]   (Read 12367 bytes from disk)
[19:03:04] Connecting to http://130.237.232.140:8080/
[19:03:05] Posted data.
[19:03:05] Initial: 0000; - Uploaded at ~13 kB/s
[19:03:05] - Averaged speed for that direction ~15 kB/s
[19:03:05] + Results successfully sent
[19:03:05] Thank you for your contribution to Folding@Home.
[19:03:06] Trying to send all finished work units
[19:03:06] + No unsent completed units remaining.
[19:03:06] - Preparing to get new work unit...
[19:03:06] Cleaning up work directory
[19:03:06] + Attempting to get work packet
[19:03:06] Passkey found
[19:03:06] - Will indicate memory of 4096 MB
[19:03:06] - Connecting to assignment server
[19:03:06] Connecting to http://assign.stanford.edu:8080/
[19:03:07] Posted data.
[19:03:07] Initial: ED82; - Successful: assigned to (130.237.232.140).
[19:03:07] + News From Folding@Home: Welcome to Folding@Home
[19:03:07] Loaded queue successfully.
[19:03:07] Sent data
[19:03:07] Connecting to http://130.237.232.140:8080/
[19:03:08] Posted data.
[19:03:08] Initial: 0000; - Receiving payload (expected size: 1799122)
[19:03:27] - Downloaded at ~92 kB/s
[19:03:27] - Averaged speed for that direction ~102 kB/s
[19:03:27] + Received work.
[19:03:27] Trying to send all finished work units
[19:03:27] + No unsent completed units remaining.
[19:03:27] + Closed connections
[19:03:32] 
[19:03:32] + Processing work unit
[19:03:32] Core required: FahCore_a3.exe
[19:03:32] Core found.
[19:03:32] Working on queue slot 04 [March 29 19:03:32 UTC]
[19:03:32] + Working ...
[19:03:32] - Calling './FahCore_a3.exe -dir work/ -nice 19 -suffix 04 -np 4 -checkpoint 15 -verbose -lifeline 571 -version 629'

[19:03:32] 
[19:03:32] *------------------------------*
[19:03:32] Folding@Home Gromacs SMP Core
[19:03:32] Version 2.17 (Mar 7 2010)
[19:03:32] 
[19:03:32] Preparing to commence simulation
[19:03:32] - Looking at optimizations...
[19:03:32] - Created dyn
[19:03:32] - Files status OK
[19:03:32] - Expanded 1798610 -> 2392545 (decompressed 133.0 percent)
[19:03:32] Called DecompressByteArray: compressed_data_size=1798610 data_size=2392545, decompressed_data_size=2392545 diff=0
[19:03:32] - Digital signature verified
[19:03:32] 
[19:03:32] Project: 6015 (Run 0, Clone 87, Gen 102)
[19:03:32] 
[19:03:32] Assembly optimizations on if available.
[19:03:32] Entering M.D.
[19:03:39] Completed 0 out of 500000 steps  (0%)
[19:13:16] Completed 5000 out of 500000 steps  (1%)
[19:22:49] Completed 10000 out of 500000 steps  (2%)
[19:31:27] Completed 15000 out of 500000 steps  (3%)
[19:39:51] Completed 20000 out of 500000 steps  (4%)
[19:48:02] Completed 25000 out of 500000 steps  (5%)
[19:49:55] mdrun returned 255
[19:49:55] Going to send back what have done -- stepsTotalG=500000
[19:49:55] Work fraction=0.0523 steps=500000.
[19:49:55] CoreStatus = 0 (0)
[19:49:55] Sending work to server
[19:49:55] Project: 6015 (Run 0, Clone 87, Gen 102)
[19:49:55] - Error: Could not get length of results file work/wuresults_04.dat
[19:49:55] - Error: Could not read unit 04 file. Removing from queue.
[19:49:55] Trying to send all finished work units
[19:49:55] + No unsent completed units remaining.
[19:49:55] - Preparing to get new work unit...
[19:49:55] Cleaning up work directory
[19:49:55] + Attempting to get work packet
[19:49:55] Passkey found
[19:49:55] - Will indicate memory of 4096 MB
[19:49:55] - Connecting to assignment server
[19:49:55] Connecting to http://assign.stanford.edu:8080/
[19:49:57] Posted data.
[19:49:57] Initial: ED82; - Successful: assigned to (130.237.232.140).
[19:49:57] + News From Folding@Home: Welcome to Folding@Home
[19:49:57] Loaded queue successfully.
[19:49:57] Sent data
[19:49:57] Connecting to http://130.237.232.140:8080/
[19:49:58] Posted data.
[19:49:58] Initial: 0000; - Receiving payload (expected size: 1799122)
[19:50:18] - Downloaded at ~87 kB/s
[19:50:18] - Averaged speed for that direction ~99 kB/s
[19:50:18] + Received work.
[19:50:18] Trying to send all finished work units
[19:50:18] + No unsent completed units remaining.
[19:50:18] + Closed connections
[19:50:23] 
[19:50:23] + Processing work unit
[19:50:23] Core required: FahCore_a3.exe
[19:50:23] Core found.
[19:50:23] Working on queue slot 05 [March 29 19:50:23 UTC]
[19:50:23] + Working ...
[19:50:23] - Calling './FahCore_a3.exe -dir work/ -nice 19 -suffix 05 -np 4 -checkpoint 15 -verbose -lifeline 571 -version 629'

[19:50:24] 
[19:50:24] *------------------------------*
[19:50:24] Folding@Home Gromacs SMP Core
[19:50:24] Version 2.17 (Mar 7 2010)
[19:50:24] 
[19:50:24] Preparing to commence simulation
[19:50:24] - Ensuring status. Please wait.
[19:50:33] - Looking at optimizations...
[19:50:33] - Working with standard loops on this execution.
[19:50:33] - Created dyn
[19:50:33] - Files status OK
[19:50:34] - Expanded 1798610 -> 2392545 (decompressed 133.0 percent)
[19:50:34] Called DecompressByteArray: compressed_data_size=1798610 data_size=2392545, decompressed_data_size=2392545 diff=0
[19:50:34] - Digital signature verified
[19:50:34] 
[19:50:34] Project: 6015 (Run 0, Clone 87, Gen 102)
[19:50:34] 
[19:50:34] Entering M.D.
[19:50:40] Completed 0 out of 500000 steps  (0%)
[19:57:23] mdrun returned 255
[19:57:23] Going to send back what have done -- stepsTotalG=500000
[19:57:23] Work fraction=0.0082 steps=500000.
[19:57:27] logfile size=11858 infoLength=11858 edr=0 trr=25
[19:57:27] logfile size: 11858 info=11858 bed=0 hdr=25
[19:57:27] - Writing 12396 bytes of core data to disk...
[19:57:27]   ... Done.
[19:57:27] 
[19:57:27] Folding@home Core Shutdown: EARLY_UNIT_END
[19:57:28] CoreStatus = 72 (114)
[19:57:28] Sending work to server
[19:57:28] Project: 6015 (Run 0, Clone 87, Gen 102)


[19:57:28] + Attempting to send results [March 29 19:57:28 UTC]
[19:57:28] - Reading file work/wuresults_05.dat from core
[19:57:28]   (Read 12396 bytes from disk)
[19:57:28] Connecting to http://130.237.232.140:8080/
[19:57:29] Posted data.
[19:57:29] Initial: 0000; - Uploaded at ~13 kB/s
[19:57:29] - Averaged speed for that direction ~15 kB/s
[19:57:29] + Results successfully sent
[19:57:29] Thank you for your contribution to Folding@Home.
[19:57:29] Trying to send all finished work units
[19:57:29] + No unsent completed units remaining.
[19:57:29] - Preparing to get new work unit...
[19:57:29] Cleaning up work directory
[19:57:29] + Attempting to get work packet
[19:57:29] Passkey found
[19:57:29] - Will indicate memory of 4096 MB
[19:57:29] - Connecting to assignment server
[19:57:29] Connecting to http://assign.stanford.edu:8080/
[19:57:30] Posted data.
[19:57:30] Initial: ED82; - Successful: assigned to (130.237.232.140).
[19:57:30] + News From Folding@Home: Welcome to Folding@Home
[19:57:31] Loaded queue successfully.
[19:57:31] Sent data
[19:57:31] Connecting to http://130.237.232.140:8080/
[19:57:32] Posted data.
[19:57:32] Initial: 0000; - Receiving payload (expected size: 1798838)
[19:57:51] - Downloaded at ~92 kB/s
[19:57:51] - Averaged speed for that direction ~97 kB/s
[19:57:51] + Received work.
[19:57:51] Trying to send all finished work units
[19:57:51] + No unsent completed units remaining.
[19:57:51] + Closed connections
[19:57:56] 
[19:57:56] + Processing work unit
[19:57:56] Core required: FahCore_a3.exe
[19:57:56] Core found.
[19:57:56] Working on queue slot 06 [March 29 19:57:56 UTC]
[19:57:56] + Working ...
[19:57:56] - Calling './FahCore_a3.exe -dir work/ -nice 19 -suffix 06 -np 4 -checkpoint 15 -verbose -lifeline 571 -version 629'

[19:57:56] 
[19:57:56] *------------------------------*
[19:57:56] Folding@Home Gromacs SMP Core
[19:57:56] Version 2.17 (Mar 7 2010)
[19:57:56] 
[19:57:56] Preparing to commence simulation
[19:57:56] - Looking at optimizations...
[19:57:56] - Created dyn
[19:57:56] - Files status OK
[19:57:57] - Expanded 1798326 -> 2392545 (decompressed 133.0 percent)
[19:57:57] Called DecompressByteArray: compressed_data_size=1798326 data_size=2392545, decompressed_data_size=2392545 diff=0
[19:57:57] - Digital signature verified
[19:57:57] 
[19:57:57] Project: 6015 (Run 0, Clone 41, Gen 106)
[19:57:57] 
[19:57:57] Assembly optimizations on if available.
[19:57:57] Entering M.D.
[19:58:03] Completed 0 out of 500000 steps  (0%)
[20:01:34] mdrun returned 255
[20:01:34] Going to send back what have done -- stepsTotalG=500000
[20:01:34] Work fraction=0.0043 steps=500000.
[20:01:35] CoreStatus = 0 (0)
[20:01:35] Sending work to server
[20:01:35] Project: 6015 (Run 0, Clone 41, Gen 106)
[20:01:35] - Error: Could not get length of results file work/wuresults_06.dat
[20:01:35] - Error: Could not read unit 06 file. Removing from queue.
[20:01:35] Trying to send all finished work units
[20:01:35] + No unsent completed units remaining.
[20:01:35] - Preparing to get new work unit...
[20:01:35] Cleaning up work directory
[20:01:35] + Attempting to get work packet
[20:01:35] Passkey found
[20:01:35] - Will indicate memory of 4096 MB
[20:01:35] - Connecting to assignment server
[20:01:35] Connecting to http://assign.stanford.edu:8080/
[20:01:36] Posted data.
[20:01:36] Initial: ED82; - Successful: assigned to (130.237.232.140).
[20:01:36] + News From Folding@Home: Welcome to Folding@Home
[20:01:36] Loaded queue successfully.
[20:01:36] Sent data
[20:01:36] Connecting to http://130.237.232.140:8080/
[20:01:37] Posted data.
[20:01:37] Initial: 0000; - Receiving payload (expected size: 1798838)
[20:01:57] - Downloaded at ~87 kB/s
[20:01:57] - Averaged speed for that direction ~95 kB/s
[20:01:57] + Received work.
[20:01:57] Trying to send all finished work units
[20:01:57] + No unsent completed units remaining.
[20:01:57] + Closed connections
[20:02:02] 
[20:02:02] + Processing work unit
[20:02:02] Core required: FahCore_a3.exe
[20:02:02] Core found.
[20:02:02] Working on queue slot 07 [March 29 20:02:02 UTC]
[20:02:02] + Working ...
[20:02:02] - Calling './FahCore_a3.exe -dir work/ -nice 19 -suffix 07 -np 4 -checkpoint 15 -verbose -lifeline 571 -version 629'

[20:02:02] 
[20:02:02] *------------------------------*
[20:02:02] Folding@Home Gromacs SMP Core
[20:02:02] Version 2.17 (Mar 7 2010)
[20:02:02] 
[20:02:02] Preparing to commence simulation
[20:02:02] - Ensuring status. Please wait.
[20:02:12] - Looking at optimizations...
[20:02:12] - Working with standard loops on this execution.
[20:02:12] - Created dyn
[20:02:12] - Files status OK
[20:02:12] - Expanded 1798326 -> 2392545 (decompressed 133.0 percent)
[20:02:12] Called DecompressByteArray: compressed_data_size=1798326 data_size=2392545, decompressed_data_size=2392545 diff=0
[20:02:12] - Digital signature verified
[20:02:12] 
[20:02:12] Project: 6015 (Run 0, Clone 41, Gen 106)
[20:02:12] 
[20:02:12] Entering M.D.
[20:02:19] Completed 0 out of 500000 steps  (0%)
[20:04:50] mdrun returned 255
[20:04:50] Going to send back what have done -- stepsTotalG=500000
[20:04:50] Work fraction=0.0030 steps=500000.
[20:04:54] logfile size=11857 infoLength=11857 edr=0 trr=25
[20:04:54] logfile size: 11857 info=11857 bed=0 hdr=25
[20:04:54] - Writing 12395 bytes of core data to disk...
[20:04:54]   ... Done.
[20:04:54] 
[20:04:54] Folding@home Core Shutdown: UNSTABLE_MACHINE
[20:04:55] CoreStatus = 7A (122)
[20:04:55] Sending work to server
[20:04:55] Project: 6015 (Run 0, Clone 41, Gen 106)


[20:04:55] + Attempting to send results [March 29 20:04:55 UTC]
[20:04:55] - Reading file work/wuresults_07.dat from core
[20:04:55]   (Read 12395 bytes from disk)
[20:04:55] Connecting to http://130.237.232.140:8080/
[20:04:56] Posted data.
[20:04:56] Initial: 0000; - Uploaded at ~13 kB/s
[20:04:56] - Averaged speed for that direction ~14 kB/s
[20:04:56] + Results successfully sent
[20:04:56] Thank you for your contribution to Folding@Home.
[20:04:56] Trying to send all finished work units
[20:04:56] + No unsent completed units remaining.
[20:04:56] - Preparing to get new work unit...
[20:04:56] Cleaning up work directory
[20:04:56] + Attempting to get work packet
[20:04:56] Passkey found
[20:04:56] - Will indicate memory of 4096 MB
[20:04:56] - Connecting to assignment server
[20:04:56] Connecting to http://assign.stanford.edu:8080/
[20:04:57] Posted data.
[20:04:57] Initial: ED82; - Successful: assigned to (130.237.232.140).
[20:04:57] + News From Folding@Home: Welcome to Folding@Home
[20:04:57] Loaded queue successfully.
[20:04:57] Sent data
[20:04:57] Connecting to http://130.237.232.140:8080/
[20:04:58] Posted data.
[20:04:58] Initial: 0000; - Receiving payload (expected size: 1796986)
[20:05:16] - Downloaded at ~97 kB/s
[20:05:16] - Averaged speed for that direction ~96 kB/s
[20:05:16] + Received work.
[20:05:16] Trying to send all finished work units
[20:05:16] + No unsent completed units remaining.
[20:05:16] + Closed connections
[20:05:21] 
[20:05:21] + Processing work unit
[20:05:21] Core required: FahCore_a3.exe
[20:05:21] Core found.
[20:05:21] Working on queue slot 08 [March 29 20:05:21 UTC]
[20:05:21] + Working ...
[20:05:21] - Calling './FahCore_a3.exe -dir work/ -nice 19 -suffix 08 -np 4 -checkpoint 15 -verbose -lifeline 571 -version 629'

[20:05:22] 
[20:05:22] *------------------------------*
[20:05:22] Folding@Home Gromacs SMP Core
[20:05:22] Version 2.17 (Mar 7 2010)
[20:05:22] 
[20:05:22] Preparing to commence simulation
[20:05:22] - Looking at optimizations...
[20:05:22] - Created dyn
[20:05:22] - Files status OK
[20:05:22] - Expanded 1796474 -> 2078149 (decompressed 115.6 percent)
[20:05:22] Called DecompressByteArray: compressed_data_size=1796474 data_size=2078149, decompressed_data_size=2078149 diff=0
[20:05:22] - Digital signature verified
[20:05:22] 
[20:05:22] Project: 6012 (Run 2, Clone 361, Gen 12)
[20:05:22] 
[20:05:22] Assembly optimizations on if available.
[20:05:22] Entering M.D.
[20:05:29] Completed 0 out of 500000 steps  (0%)
[20:13:44] Completed 5000 out of 500000 steps  (1%)
[20:22:26] Completed 10000 out of 500000 steps  (2%)
[20:31:20] Completed 15000 out of 500000 steps  (3%)
[20:40:04] Completed 20000 out of 500000 steps  (4%)
[20:48:34] Completed 25000 out of 500000 steps  (5%)
[20:57:44] Completed 30000 out of 500000 steps  (6%)
[21:06:41] Completed 35000 out of 500000 steps  (7%)
[21:08:07] mdrun returned 255
[21:08:07] Going to send back what have done -- stepsTotalG=500000
[21:08:07] Work fraction=0.0716 steps=500000.
[21:08:11] logfile size=14703 infoLength=14703 edr=0 trr=25
[21:08:11] logfile size: 14703 info=14703 bed=0 hdr=25
[21:08:11] - Writing 15241 bytes of core data to disk...
[21:08:11]   ... Done.
[21:08:11] 
[21:08:11] Folding@home Core Shutdown: UNSTABLE_MACHINE
[21:08:11] CoreStatus = 7A (122)
[21:08:11] Sending work to server
[21:08:11] Project: 6012 (Run 2, Clone 361, Gen 12)


[21:08:11] + Attempting to send results [March 29 21:08:11 UTC]
[21:08:11] - Reading file work/wuresults_08.dat from core
[21:08:11]   (Read 15241 bytes from disk)
[21:08:11] Connecting to http://130.237.232.140:8080/
[21:08:12] Posted data.
[21:08:12] Initial: 0000; - Uploaded at ~7 kB/s
[21:08:13] - Averaged speed for that direction ~13 kB/s
[21:08:13] + Results successfully sent
[21:08:13] Thank you for your contribution to Folding@Home.
[21:08:13] Trying to send all finished work units
[21:08:13] + No unsent completed units remaining.
[21:08:13] - Preparing to get new work unit...
[21:08:13] Cleaning up work directory
[21:08:13] + Attempting to get work packet
[21:08:13] Passkey found
[21:08:13] - Will indicate memory of 4096 MB
[21:08:13] - Connecting to assignment server
[21:08:13] Connecting to http://assign.stanford.edu:8080/
[21:08:14] Posted data.
[21:08:14] Initial: ED82; - Successful: assigned to (130.237.232.140).
[21:08:14] + News From Folding@Home: Welcome to Folding@Home
[21:08:14] Loaded queue successfully.
[21:08:14] Sent data
[21:08:14] Connecting to http://130.237.232.140:8080/
[21:08:15] Posted data.
[21:08:15] Initial: 0000; - Receiving payload (expected size: 1798133)
[21:08:20] ***** Got an Activate signal (2)
[21:08:20] Killing all core threads

Folding@Home Client Shutdown.


--- Opening Log file [March 29 21:08:25 UTC] 


# Mac OS X SMP Console Edition ################################################
###############################################################################

                       Folding@Home Client Version 6.29r1

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: /Users/alan/Library/FAH
Executable: ./fah6
Arguments: -smp -local -verbosity 9 -advmethods 

[21:08:25] - Ask before connecting: No
[21:08:25] - User name: AlanH (Team 47958)
[21:08:25] - User ID: 63DC229F081DE390
[21:08:25] - Machine ID: 1
[21:08:25] 
[21:08:25] Loaded queue successfully.
[21:08:25] - Preparing to get new work unit...
[21:08:25] Cleaning up work directory
[21:08:25] - Autosending finished units... [21:08:25]
[21:08:25] Trying to send all finished work units
[21:08:25] + No unsent completed units remaining.
[21:08:25] - Autosend completed
[21:08:25] + Attempting to get work packet
[21:08:25] Passkey found
[21:08:25] - Will indicate memory of 4096 MB
[21:08:25] - Detect CPU. Vendor: GenuineIntel, Family: 6, Model: 15, Stepping: 6
[21:08:25] - Connecting to assignment server
[21:08:25] Connecting to http://assign.stanford.edu:8080/
[21:08:26] Posted data.
[21:08:26] Initial: ED82; - Successful: assigned to (130.237.232.140).
[21:08:26] + News From Folding@Home: Welcome to Folding@Home
[21:08:26] Loaded queue successfully.
[21:08:26] Sent data
[21:08:26] Connecting to http://130.237.232.140:8080/
[21:08:27] Posted data.
[21:08:27] Initial: 0000; - Receiving payload (expected size: 1797558)
[21:08:43] - Downloaded at ~109 kB/s
[21:08:43] - Averaged speed for that direction ~98 kB/s
[21:08:43] + Received work.
[21:08:43] + Closed connections
[21:08:43] 
[21:08:43] + Processing work unit
[21:08:43] Core required: FahCore_a3.exe
[21:08:43] Core found.
[21:08:43] Working on queue slot 09 [March 29 21:08:43 UTC]
[21:08:43] + Working ...
[21:08:43] - Calling './FahCore_a3.exe -dir work/ -nice 19 -suffix 09 -np 4 -checkpoint 15 -verbose -lifeline 2421 -version 629'

[21:08:43] 
[21:08:43] *------------------------------*
[21:08:43] Folding@Home Gromacs SMP Core
[21:08:43] Version 2.17 (Mar 7 2010)
[21:08:43] 
[21:08:43] Preparing to commence simulation
[21:08:43] - Looking at optimizations...
[21:08:43] - Created dyn
[21:08:43] - Files status OK
[21:08:43] - Expanded 1797046 -> 2078149 (decompressed 115.6 percent)
[21:08:43] Called DecompressByteArray: compressed_data_size=1797046 data_size=2078149, decompressed_data_size=2078149 diff=0
[21:08:43] - Digital signature verified
[21:08:43] 
[21:08:43] Project: 6012 (Run 2, Clone 17, Gen 40)
[21:08:43] 
[21:08:43] Assembly optimizations on if available.
[21:08:43] Entering M.D.
[21:08:50] Completed 0 out of 500000 steps  (0%)
[21:11:46] mdrun returned 255
[21:11:46] Going to send back what have done -- stepsTotalG=500000
[21:11:46] Work fraction=0.0034 steps=500000.
[21:11:50] logfile size=11826 infoLength=11826 edr=0 trr=25
[21:11:50] logfile size: 11826 info=11826 bed=0 hdr=25
[21:11:50] - Writing 12364 bytes of core data to disk...
[21:11:50]   ... Done.
[21:11:50] 
[21:11:50] Folding@home Core Shutdown: EARLY_UNIT_END
[21:11:50] CoreStatus = 72 (114)
[21:11:50] Sending work to server
[21:11:50] Project: 6012 (Run 2, Clone 17, Gen 40)


[21:11:50] + Attempting to send results [March 29 21:11:50 UTC]
[21:11:50] - Reading file work/wuresults_09.dat from core
[21:11:50]   (Read 12364 bytes from disk)
[21:11:50] Connecting to http://130.237.232.140:8080/
[21:11:51] Posted data.
[21:11:51] Initial: 0000; - Uploaded at ~13 kB/s
[21:11:51] - Averaged speed for that direction ~13 kB/s
[21:11:51] + Results successfully sent
[21:11:51] Thank you for your contribution to Folding@Home.
[21:11:51] Trying to send all finished work units
[21:11:51] + No unsent completed units remaining.
[21:11:51] - Preparing to get new work unit...
[21:11:51] Cleaning up work directory
[21:11:51] + Attempting to get work packet
[21:11:51] Passkey found
[21:11:51] - Will indicate memory of 4096 MB
[21:11:51] - Connecting to assignment server
[21:11:51] Connecting to http://assign.stanford.edu:8080/
[21:11:52] Posted data.
[21:11:52] Initial: ED82; - Successful: assigned to (130.237.232.140).
[21:11:52] + News From Folding@Home: Welcome to Folding@Home
[21:11:52] Loaded queue successfully.
[21:11:52] Sent data
[21:11:52] Connecting to http://130.237.232.140:8080/
[21:11:53] Posted data.
[21:11:53] Initial: 0000; - Receiving payload (expected size: 1797659)
[21:12:07] - Downloaded at ~125 kB/s
[21:12:07] - Averaged speed for that direction ~104 kB/s
[21:12:07] + Received work.
[21:12:07] Trying to send all finished work units
[21:12:07] + No unsent completed units remaining.
[21:12:07] + Closed connections
[21:12:12] 
[21:12:12] + Processing work unit
[21:12:12] Core required: FahCore_a3.exe
[21:12:12] Core found.
[21:12:12] Working on queue slot 00 [March 29 21:12:12 UTC]
[21:12:12] + Working ...
[21:12:12] - Calling './FahCore_a3.exe -dir work/ -nice 19 -suffix 00 -np 4 -checkpoint 15 -verbose -lifeline 2421 -version 629'

[21:12:12] 
[21:12:12] *------------------------------*
[21:12:12] Folding@Home Gromacs SMP Core
[21:12:12] Version 2.17 (Mar 7 2010)
[21:12:12] 
[21:12:12] Preparing to commence simulation
[21:12:12] - Looking at optimizations...
[21:12:12] - Created dyn
[21:12:12] - Files status OK
[21:12:12] - Expanded 1797147 -> 2078149 (decompressed 115.6 percent)
[21:12:12] Called DecompressByteArray: compressed_data_size=1797147 data_size=2078149, decompressed_data_size=2078149 diff=0
[21:12:12] - Digital signature verified
[21:12:12] 
[21:12:12] Project: 6012 (Run 0, Clone 493, Gen 90)
[21:12:12] 
[21:12:12] Assembly optimizations on if available.
[21:12:12] Entering M.D.
[21:12:19] Completed 0 out of 500000 steps  (0%)
[21:20:54] Completed 5000 out of 500000 steps  (1%)
[21:29:40] Completed 10000 out of 500000 steps  (2%)
[21:39:42] Completed 15000 out of 500000 steps  (3%)
[21:50:52] Completed 20000 out of 500000 steps  (4%)
[22:01:36] Completed 25000 out of 500000 steps  (5%)
[22:09:18] mdrun returned 255
[22:09:18] Going to send back what have done -- stepsTotalG=500000
[22:09:18] Work fraction=0.0589 steps=500000.
[22:09:22] logfile size=13969 infoLength=13969 edr=0 trr=25
[22:09:22] logfile size: 13969 info=13969 bed=0 hdr=25
[22:09:22] - Writing 14507 bytes of core data to disk...
[22:09:22]   ... Done.
[22:09:23] 
[22:09:23] Folding@home Core Shutdown: EARLY_UNIT_END
[22:09:23] CoreStatus = 72 (114)
[22:09:23] Sending work to server
[22:09:23] Project: 6012 (Run 0, Clone 493, Gen 90)


[22:09:23] + Attempting to send results [March 29 22:09:23 UTC]
[22:09:23] - Reading file work/wuresults_00.dat from core
[22:09:23]   (Read 14507 bytes from disk)
[22:09:23] Connecting to http://130.237.232.140:8080/
[22:09:24] Posted data.
[22:09:24] Initial: 0000; - Uploaded at ~15 kB/s
[22:09:24] - Averaged speed for that direction ~13 kB/s
[22:09:24] + Results successfully sent
[22:09:24] Thank you for your contribution to Folding@Home.
[22:09:24] Trying to send all finished work units
[22:09:24] + No unsent completed units remaining.
[22:09:24] - Preparing to get new work unit...
[22:09:24] Cleaning up work directory
[22:09:25] + Attempting to get work packet
[22:09:25] Passkey found
[22:09:25] - Will indicate memory of 4096 MB
[22:09:25] - Connecting to assignment server
[22:09:25] Connecting to http://assign.stanford.edu:8080/
[22:09:25] Posted data.
[22:09:25] Initial: ED82; - Successful: assigned to (130.237.232.140).
[22:09:25] + News From Folding@Home: Welcome to Folding@Home
[22:09:26] Loaded queue successfully.
[22:09:26] Sent data
[22:09:26] Connecting to http://130.237.232.140:8080/
[22:09:27] Posted data.
[22:09:27] Initial: 0000; - Receiving payload (expected size: 1799456)
[22:09:38] - Downloaded at ~159 kB/s
[22:09:38] - Averaged speed for that direction ~115 kB/s
[22:09:38] + Received work.
[22:09:38] Trying to send all finished work units
[22:09:38] + No unsent completed units remaining.
[22:09:38] + Closed connections
[22:09:43] 
[22:09:43] + Processing work unit
[22:09:43] Core required: FahCore_a3.exe
[22:09:43] Core found.
[22:09:43] Working on queue slot 01 [March 29 22:09:43 UTC]
[22:09:43] + Working ...
[22:09:43] - Calling './FahCore_a3.exe -dir work/ -nice 19 -suffix 01 -np 4 -checkpoint 15 -verbose -lifeline 2421 -version 629'

[22:09:43] 
[22:09:43] *------------------------------*
[22:09:43] Folding@Home Gromacs SMP Core
[22:09:43] Version 2.17 (Mar 7 2010)
[22:09:43] 
[22:09:43] Preparing to commence simulation
[22:09:43] - Looking at optimizations...
[22:09:43] - Created dyn
[22:09:43] - Files status OK
[22:09:44] - Expanded 1798944 -> 2396877 (decompressed 133.2 percent)
[22:09:44] Called DecompressByteArray: compressed_data_size=1798944 data_size=2396877, decompressed_data_size=2396877 diff=0
[22:09:44] - Digital signature verified
[22:09:44] 
[22:09:44] Project: 6014 (Run 1, Clone 222, Gen 32)
[22:09:44] 
[22:09:44] Assembly optimizations on if available.
[22:09:44] Entering M.D.
[22:09:50] Completed 0 out of 500000 steps  (0%)
[22:14:51] mdrun returned 255
[22:14:51] Going to send back what have done -- stepsTotalG=500000
[22:14:51] Work fraction=0.0060 steps=500000.
[22:14:55] logfile size=11854 infoLength=11854 edr=0 trr=25
[22:14:55] logfile size: 11854 info=11854 bed=0 hdr=25
[22:14:55] - Writing 12392 bytes of core data to disk...
[22:14:55]   ... Done.
[22:14:55] 
[22:14:55] Folding@home Core Shutdown: UNSTABLE_MACHINE
[22:14:56] CoreStatus = 7A (122)
[22:14:56] Sending work to server
[22:14:56] Project: 6014 (Run 1, Clone 222, Gen 32)


[22:14:56] + Attempting to send results [March 29 22:14:56 UTC]
[22:14:56] - Reading file work/wuresults_01.dat from core
[22:14:56]   (Read 12392 bytes from disk)
[22:14:56] Connecting to http://130.237.232.140:8080/
[22:14:56] Posted data.
[22:14:56] Initial: 0000; - Uploaded at ~13 kB/s
[22:14:57] - Averaged speed for that direction ~13 kB/s
[22:14:57] + Results successfully sent
[22:14:57] Thank you for your contribution to Folding@Home.
[22:14:57] Trying to send all finished work units
[22:14:57] + No unsent completed units remaining.
[22:14:57] - Preparing to get new work unit...
[22:14:57] Cleaning up work directory
[22:14:57] + Attempting to get work packet
[22:14:57] Passkey found
[22:14:57] - Will indicate memory of 4096 MB
[22:14:57] - Connecting to assignment server
[22:14:57] Connecting to http://assign.stanford.edu:8080/
[22:14:58] Posted data.
[22:14:58] Initial: 40AB; - Successful: assigned to (171.64.65.54).
[22:14:58] + News From Folding@Home: Welcome to Folding@Home
[22:14:58] Loaded queue successfully.
[22:14:58] Sent data
[22:14:58] Connecting to http://171.64.65.54:8080/
[22:14:59] Posted data.
[22:14:59] Initial: 0000; - Receiving payload (expected size: 1780287)
[22:15:28] - Downloaded at ~59 kB/s
[22:15:28] - Averaged speed for that direction ~104 kB/s
[22:15:28] + Received work.
[22:15:28] Trying to send all finished work units
[22:15:28] + No unsent completed units remaining.
[22:15:28] + Closed connections
[22:15:33] 
[22:15:33] + Processing work unit
[22:15:33] Core required: FahCore_a3.exe
[22:15:33] Core found.
[22:15:33] Working on queue slot 02 [March 29 22:15:33 UTC]
[22:15:33] + Working ...
[22:15:33] - Calling './FahCore_a3.exe -dir work/ -nice 19 -suffix 02 -np 4 -checkpoint 15 -verbose -lifeline 2421 -version 629'

[22:15:33] 
[22:15:33] *------------------------------*
[22:15:33] Folding@Home Gromacs SMP Core
[22:15:33] Version 2.17 (Mar 7 2010)
[22:15:33] 
[22:15:33] Preparing to commence simulation
[22:15:33] - Looking at optimizations...
[22:15:33] - Created dyn
[22:15:33] - Files status OK
[22:15:34] - Expanded 1779775 -> 2059833 (decompressed 115.7 percent)
[22:15:34] Called DecompressByteArray: compressed_data_size=1779775 data_size=2059833, decompressed_data_size=2059833 diff=0
[22:15:34] - Digital signature verified
[22:15:34] 
[22:15:34] Project: 6024 (Run 0, Clone 5, Gen 91)
[22:15:34] 
[22:15:34] Assembly optimizations on if available.
[22:15:34] Entering M.D.
[22:15:40] Completed 0 out of 500000 steps  (0%)
[22:24:11] Completed 5000 out of 500000 steps  (1%)
[22:32:22] Completed 10000 out of 500000 steps  (2%)
[22:40:35] Completed 15000 out of 500000 steps  (3%)
[22:49:30] Completed 20000 out of 500000 steps  (4%)
[22:49:32] mdrun returned 255
[22:49:32] Going to send back what have done -- stepsTotalG=500000
[22:49:32] Work fraction=0.0400 steps=500000.
[22:49:33] CoreStatus = 0 (0)
[22:49:33] Sending work to server
[22:49:33] Project: 6024 (Run 0, Clone 5, Gen 91)
[22:49:33] - Error: Could not get length of results file work/wuresults_02.dat
[22:49:33] - Error: Could not read unit 02 file. Removing from queue.
[22:49:33] Trying to send all finished work units
[22:49:33] + No unsent completed units remaining.
[22:49:33] - Preparing to get new work unit...
[22:49:33] Cleaning up work directory
[22:49:33] + Attempting to get work packet
[22:49:33] Passkey found
[22:49:33] - Will indicate memory of 4096 MB
[22:49:33] - Connecting to assignment server
[22:49:33] Connecting to http://assign.stanford.edu:8080/
[22:49:35] Posted data.
[22:49:35] Initial: 40AB; - Successful: assigned to (171.64.65.54).
[22:49:35] + News From Folding@Home: Welcome to Folding@Home
[22:49:35] Loaded queue successfully.
[22:49:35] Sent data
[22:49:35] Connecting to http://171.64.65.54:8080/
[22:49:36] Posted data.
[22:49:36] Initial: 0000; - Receiving payload (expected size: 1780287)
[22:49:44] ***** Got an Activate signal (2)
[22:49:44] Killing all core threads

Folding@Home Client Shutdown.
 
I can also provide crash logs during the same period. Here's one example:

Code: Select all

Process:         FahCore_a3.exe [2621]
Path:            /Users/alan/Library/FAH/FahCore_a3.exe
Identifier:      FahCore_a3.exe
Version:         ??? (???)
Code Type:       X86 (Native)
Parent Process:  fah6 [2421]

Date/Time:       2010-03-29 23:49:32.568 +0100
OS Version:      Mac OS X 10.6.2 (10C540)
Report Version:  6

Exception Type:  EXC_BAD_ACCESS (SIGBUS)
Exception Codes: KERN_PROTECTION_FAILURE at 0x000000000000017a
Crashed Thread:  1

Thread 0:  Dispatch queue: com.apple.main-thread
0   libSystem.B.dylib             	0x991fb806 __semwait_signal + 10
1   libSystem.B.dylib             	0x99227441 nanosleep$UNIX2003 + 188
2   FahCore_a3.exe                	0x0054c88d 0x1000 + 5552269
3   FahCore_a3.exe                	0x00001a36 0x1000 + 2614

Thread 1 Crashed:
0   FahCore_a3.exe                	0x00100c27 0x1000 + 1047591

Thread 2:
0   libSystem.B.dylib             	0x991fb806 __semwait_signal + 10
1   libSystem.B.dylib             	0x99227441 nanosleep$UNIX2003 + 188
2   FahCore_a3.exe                	0x0054c88d 0x1000 + 5552269

Thread 3:
0   libSystem.B.dylib             	0x991fb806 __semwait_signal + 10
1   libSystem.B.dylib             	0x99227441 nanosleep$UNIX2003 + 188
2   FahCore_a3.exe                	0x0054c88d 0x1000 + 5552269

Thread 4:
0   libSystem.B.dylib             	0x991fb806 __semwait_signal + 10
1   libSystem.B.dylib             	0x99227441 nanosleep$UNIX2003 + 188
2   FahCore_a3.exe                	0x0054c88d 0x1000 + 5552269

Thread 5:
0   libSystem.B.dylib             	0x991cd9b2 swtch_pri + 10
1   libSystem.B.dylib             	0x992446dd sched_yield + 18
2   FahCore_a3.exe                	0x002424b8 0x1000 + 2364600
3   FahCore_a3.exe                	0x0023fccf 0x1000 + 2354383
4   FahCore_a3.exe                	0x00241402 0x1000 + 2360322
5   FahCore_a3.exe                	0x000ae8a0 0x1000 + 710816
6   FahCore_a3.exe                	0x0006bd47 0x1000 + 437575
7   FahCore_a3.exe                	0x000c4c0d 0x1000 + 801805
8   FahCore_a3.exe                	0x000265ff 0x1000 + 153087
9   FahCore_a3.exe                	0x0001770e 0x1000 + 91918
10  FahCore_a3.exe                	0x00018686 0x1000 + 95878
11  FahCore_a3.exe                	0x002435b5 0x1000 + 2368949
12  libSystem.B.dylib             	0x991fafbd _pthread_start + 345
13  libSystem.B.dylib             	0x991fae42 thread_start + 34

Thread 6:
0   libSystem.B.dylib             	0x991cd9b2 swtch_pri + 10
1   libSystem.B.dylib             	0x992446dd sched_yield + 18
2   FahCore_a3.exe                	0x002424b8 0x1000 + 2364600
3   FahCore_a3.exe                	0x0023fccf 0x1000 + 2354383
4   FahCore_a3.exe                	0x00241402 0x1000 + 2360322
5   FahCore_a3.exe                	0x000ae8a0 0x1000 + 710816
6   FahCore_a3.exe                	0x0006bd47 0x1000 + 437575
7   FahCore_a3.exe                	0x000c4c0d 0x1000 + 801805
8   FahCore_a3.exe                	0x000265ff 0x1000 + 153087
9   FahCore_a3.exe                	0x0001770e 0x1000 + 91918
10  FahCore_a3.exe                	0x00018686 0x1000 + 95878
11  FahCore_a3.exe                	0x002435b5 0x1000 + 2368949
12  libSystem.B.dylib             	0x991fafbd _pthread_start + 345
13  libSystem.B.dylib             	0x991fae42 thread_start + 34

Thread 7:
0   libSystem.B.dylib             	0x991d534a szone_free_definite_size + 3423
1   libSystem.B.dylib             	0x991d4388 free + 244
2   FahCore_a3.exe                	0x004f0207 0x1000 + 5173767
3   FahCore_a3.exe                	0x004fba7e 0x1000 + 5220990
4   FahCore_a3.exe                	0x0051a8fe 0x1000 + 5347582
5   libSystem.B.dylib             	0x991fafbd _pthread_start + 345
6   libSystem.B.dylib             	0x991fae42 thread_start + 34

Thread 1 crashed with X86 Thread State (32-bit):
  eax: 0x00000000  ebx: 0x001004a1  ecx: 0x00000000  edx: 0x00000002
  edi: 0x00014bca  esi: 0x00000002  ebp: 0xb00faa58  esp: 0xb00fa7f0
   ss: 0x0000001f  efl: 0x00210246  eip: 0x00100c27   cs: 0x00000017
   ds: 0x0000001f   es: 0x0000001f   fs: 0x0000001f   gs: 0x00000037
  cr2: 0x0000017a

Binary Images:
    0x1000 -   0x5baff3 +FahCore_a3.exe ??? (???) <8B85FF3E-D215-CF2A-88C7-95707DAEC888> /Users/alan/Library/FAH/FahCore_a3.exe
0x8fe00000 - 0x8fe4162b  dyld 132.1 (???) <211AF0DD-42D9-79C8-BB6A-1F4BEEF4B4AB> /usr/lib/dyld
0x91543000 - 0x915adfe7  libstdc++.6.dylib ??? (???) <411D87F4-B7E1-44EB-F201-F8B4F9227213> /usr/lib/libstdc++.6.dylib
0x93f4a000 - 0x93f57ff7  libbz2.1.0.dylib ??? (???) <495732E1-2AC4-44FC-E633-4CBCC503B924> /usr/lib/libbz2.1.0.dylib
0x93f91000 - 0x93f94fe7  libmathCommon.A.dylib ??? (???) <1622A54F-1A98-2CBE-B6A4-2122981A500E> /usr/lib/system/libmathCommon.A.dylib
0x991cd000 - 0x99371feb  libSystem.B.dylib ??? (???) <D45B91B2-2B4C-AAC0-8096-1FC48B7E9672> /usr/lib/libSystem.B.dylib
0xffff0000 - 0xffff1fff  libSystem.B.dylib ??? (???) <D45B91B2-2B4C-AAC0-8096-1FC48B7E9672> /usr/lib/libSystem.B.dylib

 

Project: 6012 (Run 0, Clone 241, Gen 90)

Posted: Tue Mar 30, 2010 4:20 pm
by Aardvark
Subject WU failed at <3%. Another in the continuing Saga.

Work file was not written adequately to allow return of remains to Stanford.

Is there any "action plan" on the part of PG to address this festering sore???

Log file follows:

Code: Select all

[14:21:52] *------------------------------*
[14:21:52] Folding@Home Gromacs SMP Core
[14:21:52] Version 2.17 (Mar 7 2010)
[14:21:52] 
[14:21:52] Preparing to commence simulation
[14:21:52] - Looking at optimizations...
[14:21:52] - Created dyn
[14:21:52] - Files status OK
[14:21:53] - Expanded 1796652 -> 2078149 (decompressed 115.6 percent)
[14:21:53] Called DecompressByteArray: compressed_data_size=1796652 data_size=2078149, decompressed_data_size=2078149 diff=0
[14:21:53] - Digital signature verified
[14:21:53] 
[14:21:53] Project: 6012 (Run 0, Clone 241, Gen 90)
[14:21:53] 
[14:21:53] Assembly optimizations on if available.
[14:21:53] Entering M.D.
Starting 2 threads
NNODES=2, MYRANK=0, HOSTNAME=thread #0
NNODES=2, MYRANK=1, HOSTNAME=thread #1
Reading file work/wudata_00.tpr, VERSION 4.0.99_development_20090605 (single precision)
Note: tpx file_version 68, software version 70
Making 1D domain decomposition 2 x 1 x 1
starting mdrun 'Protein in POPC'
45500004 steps,  91000.0 ps (continuing from step 45000004,  90000.0 ps).
[14:21:59] Completed 0 out of 500000 steps  (0%)
[14:44:00] Completed 5000 out of 500000 steps  (1%)
[15:05:36] Completed 10000 out of 500000 steps  (2%)

-------------------------------------------------------
Program mdrun, VERSION 4.0.99-dev-20100305
Source code file: /Users/kasson/a3_devnew/gromacs/src/mdlib/pme.c, line: 563

Fatal error:
2 particles communicated to PME node 1 are more than a cell length out of the domain decomposition cell of their charge group in dimension x
For more information and tips for trouble shooting please check the GROMACS website at
http://www.gromacs.org/Documentation/Errors
-------------------------------------------------------

Thanx for Using GROMACS - Have a Nice Day

[15:14:52] mdrun returned 255
[15:14:52] Going to send back what have done -- stepsTotalG=500000
[15:14:52] Work fraction=0.0243 steps=500000.
[15:14:53] CoreStatus = 0 (0)
[15:14:53] Sending work to server
[15:14:53] Project: 6012 (Run 0, Clone 241, Gen 90)
[15:14:53] - Error: Could not get length of results file work/wuresults_00.dat
[15:14:53] - Error: Could not read unit 00 file. Removing from queue.
[15:14:53] Trying to send all finished work units
[15:14:53] + No unsent completed units remaining.
[15:14:53] - Preparing to get new work unit...
[15:14:54] > Press "c" to connect to the server to download unit


Re: Various Project: 601x problems

Posted: Tue Mar 30, 2010 4:36 pm
by bruce
As a matter of fact, the main developer is working on the problem. Apparently you didn't see his response to your previous report.
viewtopic.php?f=19&t=13980&p=137199#p137199

Two topics on the same subject merged.

Re: Various Project: 601x problems

Posted: Tue Mar 30, 2010 4:47 pm
by Aardvark
@bruce;
I had noted that remark by Kasson. I had not translated it into an "action plan". Sorry about that...

Is it recommended that those of us that are experiencing the problem put Folding on HOLD? If there is some benefit being gained by adding to the compost heap of failed WUs then I suppose we should continue to "fold and report".