Project 2675 R3, C169 G78 atom can not be settled

[weedacres]

I've restarted and reloaded this wu several times with the same result. This is running on an XP 32 VMWare machine, Ubuntu on a Q6600.
I've tried clearing the work folder and queue.dat and get the same wu again. The following shows up on the monitor, NOT in the log.

Code: Select all

--- Opening Log file [June 17 20:22:01 UTC] 


# Linux SMP Console Edition ###################################################
###############################################################################

                       Folding@Home Client Version 6.24beta

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: /home/felix1/folding/FAH
Executable: ./fah6_alt
Arguments: -smp -verbosity 9 

[20:22:01] - Ask before connecting: No
[20:22:01] - User name: Felix_Pasqualli (Team 52523)
[20:22:01] - User ID: 2B6555C85D5937B1
[20:22:01] - Machine ID: 5
[20:22:01] 
[20:22:01] Work directory not found. Creating...
[20:22:01] Could not open work queue, generating new queue...
[20:22:01] - Preparing to get new work unit...
[20:22:01] + Attempting to get work packet
[20:22:01] - Will indicate memory of 752 MB
[20:22:01] - Connecting to assignment server
[20:22:01] Connecting to http://assign.stanford.edu:8080/
[20:22:01] - Autosending finished units... [June 17 20:22:01 UTC]
[20:22:01] Trying to send all finished work units
[20:22:01] + No unsent completed units remaining.
[20:22:01] - Autosend completed
[20:22:01] Posted data.
[20:22:01] Initial: 40AB; - Successful: assigned to (171.64.65.56).
[20:22:01] + News From Folding@Home: Welcome to Folding@Home
[20:22:01] Loaded queue successfully.
[20:22:01] Connecting to http://171.64.65.56:8080/
[20:22:07] Posted data.
[20:22:07] Initial: 0000; - Receiving payload (expected size: 4845004)
[20:22:19] - Downloaded at ~394 kB/s
[20:22:19] - Averaged speed for that direction ~394 kB/s
[20:22:19] + Received work.
[20:22:19] + Closed connections
[20:22:19] 
[20:22:19] + Processing work unit
[20:22:19] At least 4 processors must be requested.Core required: FahCore_a2.exe
[20:22:19] Core not found.
[20:22:19] - Core is not present or corrupted.
[20:22:19] - Attempting to download new core...
[20:22:19] + Downloading new core: FahCore_a2.exe
[20:22:19] Downloading core (/~pande/Linux/AMD64/Core_a2.fah from www.stanford.edu)
[20:22:19] Initial: AFDE; + 10240 bytes downloaded
[20:22:19] Initial: 5BF9; + 20480 bytes downloaded
[20:22:19] Initial: 4F5A; + 30720 bytes downloaded
[20:22:19] Initial: 0685; + 40960 bytes downloaded
[20:22:19] Initial: E287; + 51200 bytes downloaded
[20:22:19] Initial: 8CA4; + 61440 bytes downloaded
[20:22:19] Initial: 6946; + 71680 bytes downloaded
[20:22:19] Initial: B005; + 81920 bytes downloaded
[20:22:19] Initial: 4800; + 92160 bytes downloaded
[20:22:19] Initial: 596A; + 102400 bytes downloaded
[20:22:19] Initial: 3F3D; + 112640 bytes downloaded
[20:22:19] Initial: A6A6; + 122880 bytes downloaded
[20:22:19] Initial: 375D; + 133120 bytes downloaded
[20:22:19] Initial: 36B2; + 143360 bytes downloaded
[20:22:19] Initial: 4709; + 153600 bytes downloaded
[20:22:19] Initial: 18BE; + 163840 bytes downloaded
[20:22:19] Initial: 16AB; + 174080 bytes downloaded
[20:22:19] Initial: 4C49; + 184320 bytes downloaded
[20:22:19] Initial: 0EFA; + 194560 bytes downloaded
[20:22:19] Initial: 2F30; + 204800 bytes downloaded
[20:22:19] Initial: 0295; + 215040 bytes downloaded
[20:22:19] Initial: E05C; + 225280 bytes downloaded
[20:22:19] Initial: 5364; + 235520 bytes downloaded
[20:22:19] Initial: 1AD5; + 245760 bytes downloaded
[20:22:19] Initial: A21D; + 256000 bytes downloaded
[20:22:19] Initial: EFDF; + 266240 bytes downloaded
[20:22:19] Initial: 4EDF; + 276480 bytes downloaded
[20:22:19] Initial: 09C7; + 286720 bytes downloaded
[20:22:19] Initial: 0858; + 296960 bytes downloaded
[20:22:19] Initial: C447; + 307200 bytes downloaded
[20:22:20] Initial: 95C6; + 317440 bytes downloaded
[20:22:20] Initial: C575; + 327680 bytes downloaded
[20:22:20] Initial: 4D67; + 337920 bytes downloaded
[20:22:20] Initial: FB03; + 348160 bytes downloaded
[20:22:20] Initial: BAE5; + 358400 bytes downloaded
[20:22:20] Initial: A50D; + 368640 bytes downloaded
[20:22:20] Initial: 4B34; + 378880 bytes downloaded
[20:22:20] Initial: 222F; + 389120 bytes downloaded
[20:22:20] Initial: 87AC; + 399360 bytes downloaded
[20:22:20] Initial: 0072; + 409600 bytes downloaded
[20:22:20] Initial: E916; + 419840 bytes downloaded
[20:22:20] Initial: 4C6E; + 430080 bytes downloaded
[20:22:20] Initial: FE2E; + 440320 bytes downloaded
[20:22:20] Initial: 2BE6; + 450560 bytes downloaded
[20:22:20] Initial: D16C; + 460800 bytes downloaded
[20:22:20] Initial: C89F; + 471040 bytes downloaded
[20:22:20] Initial: B4AC; + 481280 bytes downloaded
[20:22:20] Initial: 7E87; + 491520 bytes downloaded
[20:22:20] Initial: 4620; + 501760 bytes downloaded
[20:22:20] Initial: F5A5; + 512000 bytes downloaded
[20:22:20] Initial: 3644; + 522240 bytes downloaded
[20:22:20] Initial: E57A; + 532480 bytes downloaded
[20:22:20] Initial: 76B5; + 542720 bytes downloaded
[20:22:20] Initial: 2A3A; + 552960 bytes downloaded
[20:22:20] Initial: 48E6; + 563200 bytes downloaded
[20:22:20] Initial: A378; + 573440 bytes downloaded
[20:22:20] Initial: 9E7C; + 583680 bytes downloaded
[20:22:20] Initial: 0AD5; + 593920 bytes downloaded
[20:22:20] Initial: E4AD; + 604160 bytes downloaded
[20:22:20] Initial: E212; + 614400 bytes downloaded
[20:22:20] Initial: D75B; + 624640 bytes downloaded
[20:22:20] Initial: 5122; + 634880 bytes downloaded
[20:22:20] Initial: 4667; + 645120 bytes downloaded
[20:22:20] Initial: 074D; + 655360 bytes downloaded
[20:22:20] Initial: 6631; + 665600 bytes downloaded
[20:22:20] Initial: 2DC2; + 675840 bytes downloaded
[20:22:20] Initial: 293F; + 686080 bytes downloaded
[20:22:20] Initial: 231E; + 696320 bytes downloaded
[20:22:20] Initial: 5393; + 706560 bytes downloaded
[20:22:20] Initial: 5EB3; + 716800 bytes downloaded
[20:22:20] Initial: 3A78; + 727040 bytes downloaded
[20:22:20] Initial: 7C1F; + 737280 bytes downloaded
[20:22:20] Initial: DECE; + 747520 bytes downloaded
[20:22:20] Initial: 8919; + 757760 bytes downloaded
[20:22:20] Initial: D696; + 768000 bytes downloaded
[20:22:20] Initial: 8E9F; + 778240 bytes downloaded
[20:22:20] Initial: 1934; + 788480 bytes downloaded
[20:22:20] Initial: F6A9; + 798720 bytes downloaded
[20:22:20] Initial: 7F60; + 808960 bytes downloaded
[20:22:20] Initial: 77AB; + 819200 bytes downloaded
[20:22:20] Initial: AFDB; + 829440 bytes downloaded
[20:22:21] Initial: 0008; + 839680 bytes downloaded
[20:22:21] Initial: 96BE; + 849920 bytes downloaded
[20:22:21] Initial: F003; + 860160 bytes downloaded
[20:22:21] Initial: 6D01; + 870400 bytes downloaded
[20:22:21] Initial: 3CCB; + 880640 bytes downloaded
[20:22:21] Initial: 9350; + 890880 bytes downloaded
[20:22:21] Initial: 5223; + 901120 bytes downloaded
[20:22:21] Initial: 2749; + 911360 bytes downloaded
[20:22:21] Initial: 7879; + 921600 bytes downloaded
[20:22:21] Initial: D1F3; + 931840 bytes downloaded
[20:22:21] Initial: 2CB4; + 942080 bytes downloaded
[20:22:21] Initial: 2DCA; + 952320 bytes downloaded
[20:22:21] Initial: F7AD; + 962560 bytes downloaded
[20:22:21] Initial: BFB0; + 972800 bytes downloaded
[20:22:21] Initial: 2409; + 983040 bytes downloaded
[20:22:21] Initial: 26B8; + 993280 bytes downloaded
[20:22:21] Initial: 5206; + 1003520 bytes downloaded
[20:22:21] Initial: 1F32; + 1013760 bytes downloaded
[20:22:21] Initial: A8B2; + 1024000 bytes downloaded
[20:22:21] Initial: 9787; + 1034240 bytes downloaded
[20:22:21] Initial: F171; + 1044480 bytes downloaded
[20:22:21] Initial: 9D74; + 1054720 bytes downloaded
[20:22:21] Initial: 61D6; + 1064960 bytes downloaded
[20:22:21] Initial: D983; + 1075200 bytes downloaded
[20:22:21] Initial: 4D85; + 1085440 bytes downloaded
[20:22:21] Initial: 7A18; + 1095680 bytes downloaded
[20:22:21] Initial: 1219; + 1105920 bytes downloaded
[20:22:21] Initial: 8C12; + 1116160 bytes downloaded
[20:22:21] Initial: 0747; + 1126400 bytes downloaded
[20:22:21] Initial: DDC4; + 1136640 bytes downloaded
[20:22:21] Initial: 402C; + 1146880 bytes downloaded
[20:22:21] Initial: 6799; + 1157120 bytes downloaded
[20:22:21] Initial: DFCB; + 1167360 bytes downloaded
[20:22:21] Initial: BCB8; + 1177600 bytes downloaded
[20:22:21] Initial: 5209; + 1187840 bytes downloaded
[20:22:21] Initial: 96B9; + 1198080 bytes downloaded
[20:22:21] Initial: 05A2; + 1208320 bytes downloaded
[20:22:21] Initial: 119E; + 1218560 bytes downloaded
[20:22:21] Initial: 2C54; + 1228800 bytes downloaded
[20:22:21] Initial: C227; + 1239040 bytes downloaded
[20:22:21] Initial: E460; + 1249280 bytes downloaded
[20:22:21] Initial: 8D63; + 1259520 bytes downloaded
[20:22:21] Initial: 0049; + 1269760 bytes downloaded
[20:22:21] Initial: E919; + 1280000 bytes downloaded
[20:22:21] Initial: DB4D; + 1290240 bytes downloaded
[20:22:21] Initial: 539E; + 1300480 bytes downloaded
[20:22:21] Initial: 10E3; + 1310720 bytes downloaded
[20:22:22] Initial: FD0D; + 1320960 bytes downloaded
[20:22:22] Initial: FC57; + 1331200 bytes downloaded
[20:22:22] Initial: A967; + 1341440 bytes downloaded
[20:22:22] Initial: 6EBD; + 1351680 bytes downloaded
[20:22:22] Initial: EFD8; + 1361920 bytes downloaded
[20:22:22] Initial: A0F8; + 1372160 bytes downloaded
[20:22:22] Initial: 8605; + 1382400 bytes downloaded
[20:22:22] Initial: 2CEC; + 1392640 bytes downloaded
[20:22:22] Initial: 9154; + 1402880 bytes downloaded
[20:22:22] Initial: B1BD; + 1413120 bytes downloaded
[20:22:22] Initial: 0CAC; + 1423360 bytes downloaded
[20:22:22] Initial: 009F; + 1433600 bytes downloaded
[20:22:22] Initial: B735; + 1443840 bytes downloaded
[20:22:22] Initial: 3D44; + 1454080 bytes downloaded
[20:22:22] Initial: 3B64; + 1464320 bytes downloaded
[20:22:22] Initial: 57BE; + 1474560 bytes downloaded
[20:22:22] Initial: 57F3; + 1484800 bytes downloaded
[20:22:22] Initial: CDAB; + 1495040 bytes downloaded
[20:22:22] Initial: 2E3C; + 1505280 bytes downloaded
[20:22:22] Initial: 6950; + 1515520 bytes downloaded
[20:22:22] Initial: 32B0; + 1525760 bytes downloaded
[20:22:22] Initial: F5CC; + 1536000 bytes downloaded
[20:22:22] Initial: 7931; + 1546240 bytes downloaded
[20:22:22] Initial: 8961; + 1556480 bytes downloaded
[20:22:22] Initial: 6934; + 1566720 bytes downloaded
[20:22:22] Initial: 5597; + 1576960 bytes downloaded
[20:22:22] Initial: E7D3; + 1587200 bytes downloaded
[20:22:22] Initial: F5EE; + 1597440 bytes downloaded
[20:22:22] Initial: 5AF2; + 1607680 bytes downloaded
[20:22:22] Initial: 6960; + 1617920 bytes downloaded
[20:22:22] Initial: 210B; + 1628160 bytes downloaded
[20:22:22] Initial: 80A1; + 1638400 bytes downloaded
[20:22:22] Initial: DB07; + 1648640 bytes downloaded
[20:22:22] Initial: 841A; + 1658880 bytes downloaded
[20:22:22] Initial: D3CA; + 1669120 bytes downloaded
[20:22:22] Initial: 6DE7; + 1679360 bytes downloaded
[20:22:22] Initial: 8A1F; + 1689600 bytes downloaded
[20:22:22] Initial: 22E5; + 1699840 bytes downloaded
[20:22:22] Initial: 30C1; + 1710080 bytes downloaded
[20:22:22] Initial: A4C1; + 1720320 bytes downloaded
[20:22:22] Initial: AFB1; + 1730560 bytes downloaded
[20:22:22] Initial: A3C7; + 1740800 bytes downloaded
[20:22:22] Initial: E565; + 1751040 bytes downloaded
[20:22:23] Initial: C262; + 1761280 bytes downloaded
[20:22:23] Initial: 09F5; + 1770268 bytes downloaded
[20:22:23] Verifying core Core_a2.fah...
[20:22:23] Signature is VALID
[20:22:23] 
[20:22:23] Trying to unzip core FahCore_a2.exe
[20:22:23] Decompressed FahCore_a2.exe (4341288 bytes) successfully
[20:22:23] + Core successfully engaged
[20:22:28] 
[20:22:28] + Processing work unit
[20:22:28] At least 4 processors must be requested.Core required: FahCore_a2.exe
[20:22:28] Core found.
[20:22:28] Working on queue slot 01 [June 17 20:22:28 UTC]
[20:22:28] + Working ...
[20:22:28] - Calling './mpiexec -np 4 -host 127.0.0.1 ./FahCore_a2.exe -dir work/ -suffix 01 -priority 96 -checkpoint 15 -verbose -lifeline 5715 -version 624'

[20:22:29] 
[20:22:29] *------------------------------*
[20:22:29] Folding@Home Gromacs SMP Core
[20:22:29] Version 2.07 (Sun Apr 19 14:51:09 PDT 2009)
[20:22:29] 
[20:22:29] Preparing to commence simulation
[20:22:29] - Looking at optimizations...
[20:22:29] - Working with standard loops on this execution.
[20:22:29] - Files status OK
[20:22:31] 2 percent)
[20:22:33] 4844492 -> 23994061 (decompressed 495.2 percent)
[20:22:33] 4844492 data_size=23994061, decompressed_data_size=23994061 diff=0
[20:22:33] - Digital signature verified
[20:22:33] 
[20:22:33] Project: 2675 (Run 3, Clone 169, Gen 78)
[20:22:33] 
[20:22:33] Assembly optimizations on if available.
[20:22:33] Entering M.D.
[20:22:33] ing M.D.
[20:22:43] one 169, Gen 78)
[20:22:43] 
[20:22:43] Entering M.D.
NNODES=4, MYRANK=0, HOSTNAME=felix1-desktop
NNODES=4, MYRANK=1, HOSTNAME=felix1-desktop
NNODES=4, MYRANK=2, HOSTNAME=felix1-desktop
NODEID=2 argc=20
NNODES=4, MYRANK=3, HOSTNAME=felix1-desktop
NODEID=3 argc=20
NODEID=0 argc=20
NODEID=1 argc=20
                         :-)  G  R  O  M  A  C  S  (-:

                   Groningen Machine for Chemical Simulation

                 :-)  VERSION 4.0.99_development_20090307  (-:


      Written by David van der Spoel, Erik Lindahl, Berk Hess, and others.
       Copyright (c) 1991-2000, University of Groningen, The Netherlands.
             Copyright (c) 2001-2008, The GROMACS development team,
            check out http://www.gromacs.org for more information.


                                :-)  mdrun  (-:

Reading file work/wudata_01.tpr, VERSION 3.3.99_development_20070618 (single precision)
Note: tpx file_version 48, software version 64

NOTE: The tpr file used for this simulation is in an old format, for less memory usage and possibly more performance create a new tpr file with an up to date version of grompp

Making 1D domain decomposition 1 x 1 x 4
starting mdrun '22878 system in water'
19750002 steps,  39500.0 ps (continuing from step 19000022,  38000.0 ps).
[20:22:58]  (0%)

t = 38000.046 ps: Water molecule starting at atom 59665 can not be settled.
Check for bad contacts and/or reduce the timestep.

t = 38000.048 ps: Water molecule starting at atom 59665 can not be settled.
Check for bad contacts and/or reduce the timestep.

t = 38000.048 ps: Water molecule starting at atom 120505 can not be settled.
Check for bad contacts and/or reduce the timestep.

t = 38000.050 ps: Water molecule starting at atom 75892 can not be settled.
Check for bad contacts and/or reduce the timestep.

t = 38000.050 ps: Water molecule starting at atom 55189 can not be settled.
Check for bad contacts and/or reduce the timestep.
[cli_2]: aborting job:
Fatal error in MPI_Waitall: Error message texts are not available
[23:31:16] ***** Got an Activate signal (2)
[23:31:16] Killing all core threads

It hangs after the last "Check for bad contact" message.

Here's the log data:

Code: Select all

# Linux SMP Console Edition ###################################################
###############################################################################

                       Folding@Home Client Version 6.24beta

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: /home/felix1/folding/FAH
Executable: ./fah6_alt
Arguments: -smp -verbosity 9 

[20:22:01] - Ask before connecting: No
[20:22:01] - User name: Felix_Pasqualli (Team 52523)
[20:22:01] - User ID: 2B6555C85D5937B1
[20:22:01] - Machine ID: 5
[20:22:01] 
[20:22:01] Work directory not found. Creating...
[20:22:01] Could not open work queue, generating new queue...
[20:22:01] - Preparing to get new work unit...
[20:22:01] + Attempting to get work packet
[20:22:01] - Will indicate memory of 752 MB
[20:22:01] - Connecting to assignment server
[20:22:01] Connecting to http://assign.stanford.edu:8080/
[20:22:01] - Autosending finished units... [June 17 20:22:01 UTC]
[20:22:01] Trying to send all finished work units
[20:22:01] + No unsent completed units remaining.
[20:22:01] - Autosend completed
[20:22:01] Posted data.
[20:22:01] Initial: 40AB; - Successful: assigned to (171.64.65.56).
[20:22:01] + News From Folding@Home: Welcome to Folding@Home
[20:22:01] Loaded queue successfully.
[20:22:01] Connecting to http://171.64.65.56:8080/
[20:22:07] Posted data.
[20:22:07] Initial: 0000; - Receiving payload (expected size: 4845004)
[20:22:19] - Downloaded at ~394 kB/s
[20:22:19] - Averaged speed for that direction ~394 kB/s
[20:22:19] + Received work.
[20:22:19] + Closed connections
[20:22:19] 
[20:22:19] + Processing work unit
[20:22:19] At least 4 processors must be requested.Core required: FahCore_a2.exe
[20:22:19] Core not found.
[20:22:19] - Core is not present or corrupted.
[20:22:19] - Attempting to download new core...
[20:22:19] + Downloading new core: FahCore_a2.exe
[20:22:19] Downloading core (/~pande/Linux/AMD64/Core_a2.fah from www.stanford.edu)
[20:22:19] Initial: AFDE; + 10240 bytes downloaded
[20:22:19] Initial: 5BF9; + 20480 bytes downloaded
[20:22:19] Initial: 4F5A; + 30720 bytes downloaded
[20:22:19] Initial: 0685; + 40960 bytes downloaded
[20:22:19] Initial: E287; + 51200 bytes downloaded
[20:22:19] Initial: 8CA4; + 61440 bytes downloaded
[20:22:19] Initial: 6946; + 71680 bytes downloaded
[20:22:19] Initial: B005; + 81920 bytes downloaded
[20:22:19] Initial: 4800; + 92160 bytes downloaded
[20:22:19] Initial: 596A; + 102400 bytes downloaded
[20:22:19] Initial: 3F3D; + 112640 bytes downloaded
[20:22:19] Initial: A6A6; + 122880 bytes downloaded
[20:22:19] Initial: 375D; + 133120 bytes downloaded
[20:22:19] Initial: 36B2; + 143360 bytes downloaded
[20:22:19] Initial: 4709; + 153600 bytes downloaded
[20:22:19] Initial: 18BE; + 163840 bytes downloaded
[20:22:19] Initial: 16AB; + 174080 bytes downloaded
[20:22:19] Initial: 4C49; + 184320 bytes downloaded
[20:22:19] Initial: 0EFA; + 194560 bytes downloaded
[20:22:19] Initial: 2F30; + 204800 bytes downloaded
[20:22:19] Initial: 0295; + 215040 bytes downloaded
[20:22:19] Initial: E05C; + 225280 bytes downloaded
[20:22:19] Initial: 5364; + 235520 bytes downloaded
[20:22:19] Initial: 1AD5; + 245760 bytes downloaded
[20:22:19] Initial: A21D; + 256000 bytes downloaded
[20:22:19] Initial: EFDF; + 266240 bytes downloaded
[20:22:19] Initial: 4EDF; + 276480 bytes downloaded
[20:22:19] Initial: 09C7; + 286720 bytes downloaded
[20:22:19] Initial: 0858; + 296960 bytes downloaded
[20:22:19] Initial: C447; + 307200 bytes downloaded
[20:22:20] Initial: 95C6; + 317440 bytes downloaded
[20:22:20] Initial: C575; + 327680 bytes downloaded
[20:22:20] Initial: 4D67; + 337920 bytes downloaded
[20:22:20] Initial: FB03; + 348160 bytes downloaded
[20:22:20] Initial: BAE5; + 358400 bytes downloaded
[20:22:20] Initial: A50D; + 368640 bytes downloaded
[20:22:20] Initial: 4B34; + 378880 bytes downloaded
[20:22:20] Initial: 222F; + 389120 bytes downloaded
[20:22:20] Initial: 87AC; + 399360 bytes downloaded
[20:22:20] Initial: 0072; + 409600 bytes downloaded
[20:22:20] Initial: E916; + 419840 bytes downloaded
[20:22:20] Initial: 4C6E; + 430080 bytes downloaded
[20:22:20] Initial: FE2E; + 440320 bytes downloaded
[20:22:20] Initial: 2BE6; + 450560 bytes downloaded
[20:22:20] Initial: D16C; + 460800 bytes downloaded
[20:22:20] Initial: C89F; + 471040 bytes downloaded
[20:22:20] Initial: B4AC; + 481280 bytes downloaded
[20:22:20] Initial: 7E87; + 491520 bytes downloaded
[20:22:20] Initial: 4620; + 501760 bytes downloaded
[20:22:20] Initial: F5A5; + 512000 bytes downloaded
[20:22:20] Initial: 3644; + 522240 bytes downloaded
[20:22:20] Initial: E57A; + 532480 bytes downloaded
[20:22:20] Initial: 76B5; + 542720 bytes downloaded
[20:22:20] Initial: 2A3A; + 552960 bytes downloaded
[20:22:20] Initial: 48E6; + 563200 bytes downloaded
[20:22:20] Initial: A378; + 573440 bytes downloaded
[20:22:20] Initial: 9E7C; + 583680 bytes downloaded
[20:22:20] Initial: 0AD5; + 593920 bytes downloaded
[20:22:20] Initial: E4AD; + 604160 bytes downloaded
[20:22:20] Initial: E212; + 614400 bytes downloaded
[20:22:20] Initial: D75B; + 624640 bytes downloaded
[20:22:20] Initial: 5122; + 634880 bytes downloaded
[20:22:20] Initial: 4667; + 645120 bytes downloaded
[20:22:20] Initial: 074D; + 655360 bytes downloaded
[20:22:20] Initial: 6631; + 665600 bytes downloaded
[20:22:20] Initial: 2DC2; + 675840 bytes downloaded
[20:22:20] Initial: 293F; + 686080 bytes downloaded
[20:22:20] Initial: 231E; + 696320 bytes downloaded
[20:22:20] Initial: 5393; + 706560 bytes downloaded
[20:22:20] Initial: 5EB3; + 716800 bytes downloaded
[20:22:20] Initial: 3A78; + 727040 bytes downloaded
[20:22:20] Initial: 7C1F; + 737280 bytes downloaded
[20:22:20] Initial: DECE; + 747520 bytes downloaded
[20:22:20] Initial: 8919; + 757760 bytes downloaded
[20:22:20] Initial: D696; + 768000 bytes downloaded
[20:22:20] Initial: 8E9F; + 778240 bytes downloaded
[20:22:20] Initial: 1934; + 788480 bytes downloaded
[20:22:20] Initial: F6A9; + 798720 bytes downloaded
[20:22:20] Initial: 7F60; + 808960 bytes downloaded
[20:22:20] Initial: 77AB; + 819200 bytes downloaded
[20:22:20] Initial: AFDB; + 829440 bytes downloaded
[20:22:21] Initial: 0008; + 839680 bytes downloaded
[20:22:21] Initial: 96BE; + 849920 bytes downloaded
[20:22:21] Initial: F003; + 860160 bytes downloaded
[20:22:21] Initial: 6D01; + 870400 bytes downloaded
[20:22:21] Initial: 3CCB; + 880640 bytes downloaded
[20:22:21] Initial: 9350; + 890880 bytes downloaded
[20:22:21] Initial: 5223; + 901120 bytes downloaded
[20:22:21] Initial: 2749; + 911360 bytes downloaded
[20:22:21] Initial: 7879; + 921600 bytes downloaded
[20:22:21] Initial: D1F3; + 931840 bytes downloaded
[20:22:21] Initial: 2CB4; + 942080 bytes downloaded
[20:22:21] Initial: 2DCA; + 952320 bytes downloaded
[20:22:21] Initial: F7AD; + 962560 bytes downloaded
[20:22:21] Initial: BFB0; + 972800 bytes downloaded
[20:22:21] Initial: 2409; + 983040 bytes downloaded
[20:22:21] Initial: 26B8; + 993280 bytes downloaded
[20:22:21] Initial: 5206; + 1003520 bytes downloaded
[20:22:21] Initial: 1F32; + 1013760 bytes downloaded
[20:22:21] Initial: A8B2; + 1024000 bytes downloaded
[20:22:21] Initial: 9787; + 1034240 bytes downloaded
[20:22:21] Initial: F171; + 1044480 bytes downloaded
[20:22:21] Initial: 9D74; + 1054720 bytes downloaded
[20:22:21] Initial: 61D6; + 1064960 bytes downloaded
[20:22:21] Initial: D983; + 1075200 bytes downloaded
[20:22:21] Initial: 4D85; + 1085440 bytes downloaded
[20:22:21] Initial: 7A18; + 1095680 bytes downloaded
[20:22:21] Initial: 1219; + 1105920 bytes downloaded
[20:22:21] Initial: 8C12; + 1116160 bytes downloaded
[20:22:21] Initial: 0747; + 1126400 bytes downloaded
[20:22:21] Initial: DDC4; + 1136640 bytes downloaded
[20:22:21] Initial: 402C; + 1146880 bytes downloaded
[20:22:21] Initial: 6799; + 1157120 bytes downloaded
[20:22:21] Initial: DFCB; + 1167360 bytes downloaded
[20:22:21] Initial: BCB8; + 1177600 bytes downloaded
[20:22:21] Initial: 5209; + 1187840 bytes downloaded
[20:22:21] Initial: 96B9; + 1198080 bytes downloaded
[20:22:21] Initial: 05A2; + 1208320 bytes downloaded
[20:22:21] Initial: 119E; + 1218560 bytes downloaded
[20:22:21] Initial: 2C54; + 1228800 bytes downloaded
[20:22:21] Initial: C227; + 1239040 bytes downloaded
[20:22:21] Initial: E460; + 1249280 bytes downloaded
[20:22:21] Initial: 8D63; + 1259520 bytes downloaded
[20:22:21] Initial: 0049; + 1269760 bytes downloaded
[20:22:21] Initial: E919; + 1280000 bytes downloaded
[20:22:21] Initial: DB4D; + 1290240 bytes downloaded
[20:22:21] Initial: 539E; + 1300480 bytes downloaded
[20:22:21] Initial: 10E3; + 1310720 bytes downloaded
[20:22:22] Initial: FD0D; + 1320960 bytes downloaded
[20:22:22] Initial: FC57; + 1331200 bytes downloaded
[20:22:22] Initial: A967; + 1341440 bytes downloaded
[20:22:22] Initial: 6EBD; + 1351680 bytes downloaded
[20:22:22] Initial: EFD8; + 1361920 bytes downloaded
[20:22:22] Initial: A0F8; + 1372160 bytes downloaded
[20:22:22] Initial: 8605; + 1382400 bytes downloaded
[20:22:22] Initial: 2CEC; + 1392640 bytes downloaded
[20:22:22] Initial: 9154; + 1402880 bytes downloaded
[20:22:22] Initial: B1BD; + 1413120 bytes downloaded
[20:22:22] Initial: 0CAC; + 1423360 bytes downloaded
[20:22:22] Initial: 009F; + 1433600 bytes downloaded
[20:22:22] Initial: B735; + 1443840 bytes downloaded
[20:22:22] Initial: 3D44; + 1454080 bytes downloaded
[20:22:22] Initial: 3B64; + 1464320 bytes downloaded
[20:22:22] Initial: 57BE; + 1474560 bytes downloaded
[20:22:22] Initial: 57F3; + 1484800 bytes downloaded
[20:22:22] Initial: CDAB; + 1495040 bytes downloaded
[20:22:22] Initial: 2E3C; + 1505280 bytes downloaded
[20:22:22] Initial: 6950; + 1515520 bytes downloaded
[20:22:22] Initial: 32B0; + 1525760 bytes downloaded
[20:22:22] Initial: F5CC; + 1536000 bytes downloaded
[20:22:22] Initial: 7931; + 1546240 bytes downloaded
[20:22:22] Initial: 8961; + 1556480 bytes downloaded
[20:22:22] Initial: 6934; + 1566720 bytes downloaded
[20:22:22] Initial: 5597; + 1576960 bytes downloaded
[20:22:22] Initial: E7D3; + 1587200 bytes downloaded
[20:22:22] Initial: F5EE; + 1597440 bytes downloaded
[20:22:22] Initial: 5AF2; + 1607680 bytes downloaded
[20:22:22] Initial: 6960; + 1617920 bytes downloaded
[20:22:22] Initial: 210B; + 1628160 bytes downloaded
[20:22:22] Initial: 80A1; + 1638400 bytes downloaded
[20:22:22] Initial: DB07; + 1648640 bytes downloaded
[20:22:22] Initial: 841A; + 1658880 bytes downloaded
[20:22:22] Initial: D3CA; + 1669120 bytes downloaded
[20:22:22] Initial: 6DE7; + 1679360 bytes downloaded
[20:22:22] Initial: 8A1F; + 1689600 bytes downloaded
[20:22:22] Initial: 22E5; + 1699840 bytes downloaded
[20:22:22] Initial: 30C1; + 1710080 bytes downloaded
[20:22:22] Initial: A4C1; + 1720320 bytes downloaded
[20:22:22] Initial: AFB1; + 1730560 bytes downloaded
[20:22:22] Initial: A3C7; + 1740800 bytes downloaded
[20:22:22] Initial: E565; + 1751040 bytes downloaded
[20:22:23] Initial: C262; + 1761280 bytes downloaded
[20:22:23] Initial: 09F5; + 1770268 bytes downloaded
[20:22:23] Verifying core Core_a2.fah...
[20:22:23] Signature is VALID
[20:22:23] 
[20:22:23] Trying to unzip core FahCore_a2.exe
[20:22:23] Decompressed FahCore_a2.exe (4341288 bytes) successfully
[20:22:23] + Core successfully engaged
[20:22:28] 
[20:22:28] + Processing work unit
[20:22:28] At least 4 processors must be requested.Core required: FahCore_a2.exe
[20:22:28] Core found.
[20:22:28] Working on queue slot 01 [June 17 20:22:28 UTC]
[20:22:28] + Working ...
[20:22:28] - Calling './mpiexec -np 4 -host 127.0.0.1 ./FahCore_a2.exe -dir work/ -suffix 01 -priority 96 -checkpoint 15 -verbose -lifeline 5715 -version 624'

[20:22:29] 
[20:22:29] *------------------------------*
[20:22:29] Folding@Home Gromacs SMP Core
[20:22:29] Version 2.07 (Sun Apr 19 14:51:09 PDT 2009)
[20:22:29] 
[20:22:29] Preparing to commence simulation
[20:22:29] - Looking at optimizations...
[20:22:29] - Working with standard loops on this execution.
[20:22:29] - Files status OK
[20:22:31] 2 percent)
[20:22:33] 4844492 -> 23994061 (decompressed 495.2 percent)
[20:22:33] 4844492 data_size=23994061, decompressed_data_size=23994061 diff=0
[20:22:33] - Digital signature verified
[20:22:33] 
[20:22:33] Project: 2675 (Run 3, Clone 169, Gen 78)
[20:22:33] 
[20:22:33] Assembly optimizations on if available.
[20:22:33] Entering M.D.
[20:22:33] ing M.D.
[20:22:43] one 169, Gen 78)
[20:22:43] 
[20:22:43] Entering M.D.
[20:22:58]  (0%)
[23:31:16] ***** Got an Activate signal (2)
[23:31:16] Killing all core threads

I don't know if this is a bad work unit but I can't get past it as it keep reloading the same one.

[weedacres]

I tried deleting the work folder and queue.dat again and here are the results. Note the segmentation fault.

Code: Select all

felix1@felix1-desktop:~/folding/FAH$ ./fah6_alt

Note: Please read the license agreement (fah6_alt -license). Further 
use of this software requires that you have read and accepted this agreement.

2 cores detected


--- Opening Log file [June 18 04:12:05 UTC] 


# Linux SMP Console Edition ###################################################
###############################################################################

                       Folding@Home Client Version 6.24beta

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: /home/felix1/folding/FAH
Executable: ./fah6_alt
Arguments: -smp -verbosity 9 

[04:12:05] - Ask before connecting: No
[04:12:05] - User name: Felix_Pasqualli (Team 52523)
[04:12:05] - User ID: 2B6555C85D5937B1
[04:12:05] - Machine ID: 5
[04:12:05] 
[04:12:05] Work directory not found. Creating...
[04:12:05] Could not open work queue, generating new queue...
[04:12:05] - Preparing to get new work unit...
[04:12:05] + Attempting to get work packet
[04:12:05] - Will indicate memory of 752 MB
[04:12:05] - Connecting to assignment server
[04:12:05] Connecting to http://assign.stanford.edu:8080/
[04:12:05] - Autosending finished units... [June 18 04:12:05 UTC]
[04:12:05] Trying to send all finished work units
[04:12:05] + No unsent completed units remaining.
[04:12:05] - Autosend completed
[04:12:06] Posted data.
[04:12:06] Initial: 40AB; - Successful: assigned to (171.64.65.56).
[04:12:06] + News From Folding@Home: Welcome to Folding@Home
[04:12:06] Loaded queue successfully.
[04:12:06] Connecting to http://171.64.65.56:8080/
[04:12:11] Posted data.
[04:12:11] Initial: 0000; - Receiving payload (expected size: 4845004)
[04:12:22] - Downloaded at ~430 kB/s
[04:12:22] - Averaged speed for that direction ~430 kB/s
[04:12:22] + Received work.
[04:12:22] + Closed connections
[04:12:22] 
[04:12:22] + Processing work unit
[04:12:22] At least 4 processors must be requested.Core required: FahCore_a2.exe
[04:12:22] Core found.
[04:12:22] Working on queue slot 01 [June 18 04:12:22 UTC]
[04:12:22] + Working ...
[04:12:22] - Calling './mpiexec -np 4 -host 127.0.0.1 ./FahCore_a2.exe -dir work/ -suffix 01 -priority 96 -checkpoint 15 -verbose -lifeline 5986 -version 624'

[04:12:23] 
[04:12:23] *------------------------------*
[04:12:23] Folding@Home Gromacs SMP Core
[04:12:23] Version 2.07 (Sun Apr 19 14:51:09 PDT 2009)
[04:12:23] 
[04:12:23] Preparing to commence simulation
[04:12:23] - Ensuring status. Please wait.
[04:12:24] Called DecompressByteArray: compressed_data_size=4844492 data_size=23994061, decompressed_data_size=23994061 diff=0
[04:12:25] - Digital signature verified
[04:12:25] 
[04:12:25] Project: 2675 (Run 3, Clone 169, Gen 78)
[04:12:25] 
[04:12:25] Assembly optimizations on if available.
[04:12:25] Entering M.D.
[04:12:35] Run 3, Clone 169, Gen 78)
[04:12:35] 
[04:12:36] Entering M.D.
NNODES=4, MYRANK=2, HOSTNAME=felix1-desktop
NNODES=4, MYRANK=3, HOSTNAME=felix1-desktop
NNODES=4, MYRANK=0, HOSTNAME=felix1-desktop
NNODES=4, MYRANK=1, HOSTNAME=felix1-desktop
NODEID=2 argc=20
NODEID=3 argc=20
NODEID=0 argc=20
                         :-)  G  R  O  M  A  C  S  (-:

                   Groningen Machine for Chemical Simulation

                 :-)  VERSION 4.0.99_development_20090307  (-:


      Written by David van der Spoel, Erik Lindahl, Berk Hess, and others.
       Copyright (c) 1991-2000, University of Groningen, The Netherlands.
             Copyright (c) 2001-2008, The GROMACS development team,
            check out http://www.gromacs.org for more information.


                                :-)  mdrun  (-:

NODEID=1 argc=20
Reading file work/wudata_01.tpr, VERSION 3.3.99_development_20070618 (single precision)
Note: tpx file_version 48, software version 64

NOTE: The tpr file used for this simulation is in an old format, for less memory usage and possibly more performance create a new tpr file with an up to date version of grompp

Making 1D domain decomposition 1 x 1 x 4
starting mdrun '22878 system in water'
19750002 steps,  39500.0 ps (continuing from step 19000022,  38000.0 ps).

t = 38000.046 ps: Water molecule starting at atom 59665 can not be settled.
Check for bad contacts and/or reduce the timestep.

t = 38000.048 ps: Water molecule starting at atom 120505 can not be settled.
Check for bad contacts and/or reduce the timestep.

t = 38000.048 ps: Water molecule starting at atom 59665 can not be settled.
Check for bad contacts and/or reduce the timestep.

t = 38000.050 ps: Water molecule starting at atom 55189 can not be settled.
Check for bad contacts and/or reduce the timestep.

t = 38000.050 ps: Water molecule starting at atom 75892 can not be settled.
Check for bad contacts and/or reduce the timestep.
[cli_2]: aborting job:
Fatal error in MPI_Waitall: Error message texts are not available
[0]0:Return code = 0, signaled with Segmentation fault
[0]1:Return code = 0, signaled with Quit
[0]2:Return code = 1
[0]3:Return code = 0, signaled with Segmentation fault
[04:12:55] CoreStatus = 1 (1)
[04:12:55] Sending work to server
[04:12:55] Project: 2675 (Run 3, Clone 169, Gen 78)
[04:12:55] - Error: Could not get length of results file work/wuresults_01.dat
[04:12:55] - Error: Could not read unit 01 file. Removing from queue.
[04:12:55] Trying to send all finished work units
[04:12:55] + No unsent completed units remaining.
[04:12:55] - Preparing to get new work unit...
[04:12:55] + Attempting to get work packet
[04:12:55] - Will indicate memory of 752 MB
[04:12:55] - Connecting to assignment server
[04:12:55] Connecting to http://assign.stanford.edu:8080/
[04:12:55] Posted data.
[04:12:55] Initial: 40AB; - Successful: assigned to (171.64.65.56).
[04:12:55] + News From Folding@Home: Welcome to Folding@Home
[04:12:55] Loaded queue successfully.
[04:12:55] Connecting to http://171.64.65.56:8080/
[04:13:01] Posted data.
[04:13:01] Initial: 0000; - Receiving payload (expected size: 4845004)
[04:13:12] - Downloaded at ~430 kB/s
[04:13:12] - Averaged speed for that direction ~430 kB/s
[04:13:12] + Received work.
[04:13:12] Trying to send all finished work units
[04:13:12] + No unsent completed units remaining.
[04:13:12] + Closed connections
[04:13:17] 
[04:13:17] + Processing work unit
[04:13:17] At least 4 processors must be requested.Core required: FahCore_a2.exe
[04:13:17] Core found.
[04:13:17] Working on queue slot 02 [June 18 04:13:17 UTC]
[04:13:17] + Working ...
[04:13:17] - Calling './mpiexec -np 4 -host 127.0.0.1 ./FahCore_a2.exe -dir work/ -suffix 02 -priority 96 -checkpoint 15 -verbose -lifeline 5986 -version 624'

[04:13:18] 
[04:13:18] *------------------------------*
[04:13:18] Folding@Home Gromacs SMP Core
[04:13:18] Version 2.07 (Sun Apr 19 14:51:09 PDT 2009)
[04:13:18] 
[04:13:18] Preparing to commence simulation
[04:13:18] - Ensuring status. Please wait.
[04:13:27] - Looking at optimizations...
[04:13:27] - Working with standard loops on this execution.
[04:13:27] - Files status OK
[04:13:29] - Expanded 4844492 -> 23994061 (decompressed 495.2 percent)
[04:13:29] Called DecompressByteArray: compressed_data_size=4844492 data_size=23994061, decompressed_data_size=23994061 diff=0
[04:13:30] - Digital signature verified
[04:13:30] 
[04:13:30] Project: 2675 (Run 3, Clone 169, Gen 78)
[04:13:30] 
[04:13:30] Entering M.D.
NNODES=4, MYRANK=0, HOSTNAME=felix1-desktop
NNODES=4, MYRANK=1, HOSTNAME=felix1-desktop
NNODES=4, MYRANK=3, HOSTNAME=felix1-desktop
NNODES=4, MYRANK=2, HOSTNAME=felix1-desktop
NODEID=0 argc=20
                         :-)  G  R  O  M  A  C  S  (-:

                   Groningen Machine for Chemical Simulation

                 :-)  VERSION 4.0.99_development_20090307  (-:


      Written by David van der Spoel, Erik Lindahl, Berk Hess, and others.
       Copyright (c) 1991-2000, University of Groningen, The Netherlands.
             Copyright (c) 2001-2008, The GROMACS development team,
            check out http://www.gromacs.org for more information.


                                :-)  mdrun  (-:

Reading file work/wudata_02.tpr, VERSION 3.3.99_development_20070618 (single precision)
NODEID=1 argc=20
NODEID=3 argc=20
NODEID=2 argc=20
Note: tpx file_version 48, software version 64

NOTE: The tpr file used for this simulation is in an old format, for less memory usage and possibly more performance create a new tpr file with an up to date version of grompp

Making 1D domain decomposition 1 x 1 x 4
starting mdrun '22878 system in water'
19750002 steps,  39500.0 ps (continuing from step 19000022,  38000.0 ps).
[04:13:41] Completed 0 out of 749980 steps  (0%)

t = 38000.046 ps: Water molecule starting at atom 59665 can not be settled.
Check for bad contacts and/or reduce the timestep.

t = 38000.048 ps: Water molecule starting at atom 120505 can not be settled.
Check for bad contacts and/or reduce the timestep.

t = 38000.048 ps: Water molecule starting at atom 59665 can not be settled.
Check for bad contacts and/or reduce the timestep.

t = 38000.050 ps: Water molecule starting at atom 75892 can not be settled.
Check for bad contacts and/or reduce the timestep.

t = 38000.050 ps: Water molecule starting at atom 55189 can not be settled.
Check for bad contacts and/or reduce the timestep.
[cli_2]: aborting job:
Fatal error in MPI_Sendrecv: Error message texts are not available

For what it's worth, I'm running another SMP on this machine and 2 gpu2 clients. This particular VM had just completed it's 211th wu prior to this failure.

[harlam357]

This WU will not process for me... subsequent attempts end in this same result... "errors" out immediately.

Code: Select all

[15:36:34] *------------------------------*
[15:36:34] Folding@Home Gromacs SMP Core
[15:36:34] Version 2.07 (Sun Apr 19 14:51:09 PDT 2009)
[15:36:34] 
[15:36:34] Preparing to commence simulation
[15:36:34] - Ensuring status. Please wait.
[15:36:35] Called DecompressByteArray: compressed_data_size=4844492 data_size=23994061, decompressed_data_size=23994061 diff=0
[15:36:35] - Digital signature verified
[15:36:35] 
[15:36:35] Project: 2675 (Run 3, Clone 169, Gen 78)
[15:36:35] 
[15:36:35] Assembly optimizations on if available.
[15:36:35] Entering M.D.
[15:36:45] Run 3, Clone 169, Gen 78)
[15:36:45] 
[15:36:45] Entering M.D.
NNODES=4, MYRANK=0, HOSTNAME=ubuntu804-vm
NNODES=4, MYRANK=2, HOSTNAME=ubuntu804-vm
NNODES=4, MYRANK=3, HOSTNAME=ubuntu804-vm
NODEID=2 argc=20
NODEID=3 argc=20
NNODES=4, MYRANK=1, HOSTNAME=ubuntu804-vm
NODEID=0 argc=20
                         :-)  G  R  O  M  A  C  S  (-:

                   NODEID=1 argc=20
Groningen Machine for Chemical Simulation

                 :-)  VERSION 4.0.99_development_20090307  (-:


      Written by David van der Spoel, Erik Lindahl, Berk Hess, and others.
       Copyright (c) 1991-2000, University of Groningen, The Netherlands.
             Copyright (c) 2001-2008, The GROMACS development team,
            check out http://www.gromacs.org for more information.


                                :-)  mdrun  (-:

Reading file work/wudata_01.tpr, VERSION 3.3.99_development_20070618 (single precision)
Note: tpx file_version 48, software version 64

NOTE: The tpr file used for this simulation is in an old format, for less memory usage and possibly more performance create a new tpr file with an up to date version of grompp

Making 1D domain decomposition 1 x 1 x 4
starting mdrun '22878 system in water'
19750002 steps,  39500.0 ps (continuing from step 19000022,  38000.0 ps).

t = 38000.046 ps: Water molecule starting at atom 59665 can not be settled.
Check for bad contacts and/or reduce the timestep.

t = 38000.048 ps: Water molecule starting at atom 120505 can not be settled.
Check for bad contacts and/or reduce the timestep.

t = 38000.048 ps: Water molecule starting at atom 59665 can not be settled.
Check for bad contacts and/or reduce the timestep.

t = 38000.050 ps: Water molecule starting at atom 55189 can not be settled.
Check for bad contacts and/or reduce the timestep.

t = 38000.050 ps: Water molecule starting at atom 75892 can not be settled.
Check for bad contacts and/or reduce the timestep.

Step 19000025, time 38000.1 (ps)  LINCS WARNING
relative constraint deviation after LINCS:
rms 8.423115, max 103.543610 (between atoms 4142 and 4143)
bonds that rotated more than 90 degrees:
 atom 1 atom 2  angle  previous, current, constraint length
   4347   4349   90.0    0.1095   4.0048      0.1090
   4347   4348   90.0    0.1090   0.6988      0.1090
   4347   4350   90.0    0.1090   0.6884      0.1090
   4693   4696  154.2    0.1088   6.5036      0.1090
   4693   4694   90.0    0.1090   0.5128      0.1090
   4693   4695   90.0    0.1090   0.5349      0.1090
   4697   4698   90.0    0.1090   0.4439      0.1090
   4700   4701   90.0    0.1090   0.4627      0.1090
   4700   4702  145.4    0.1081   8.0878      0.1090
   4700   4703   90.0    0.1090   0.5003      0.1090
   1585   1586  153.5    0.1004   4.1702      0.1010
  13773  13774  127.1    0.1091   7.0007      0.1090
  13853  13854   90.0    0.1010   8.4362      0.1010
  13857  13859   90.0    0.1091   4.0004      0.1090
  13857  13858   90.0    0.1090   0.7122      0.1090
  13691  13692   90.0    0.1086   1.8831      0.1090
  13693  13695  132.1    0.1084   8.1343      0.1090
  13693  13694   90.0    0.1090   0.2976      0.1090
  19194  19196   90.0    0.1080   3.5455      0.1090
  19197  19199   90.0    0.1087   5.0150      0.1090
   9120   9121   90.0    0.1078   6.5892      0.1080
   9176   9177   90.0    0.1090   0.7189      0.1090
   9176   9178   90.0    0.1090   0.7225      0.1090
   9176   9179   90.0    0.1077   3.6715      0.1090
  17582  17583   90.0    0.1077  10.4789      0.1080
  11622  11623  153.0    0.1072   8.8168      0.1090
   9407   9408  159.5    0.1095   4.5874      0.1090
   9407   9409   90.0    0.1085   8.3940      0.1090
   9407   9410   90.0    0.1090   0.5066      0.1090
   4318   4319  149.6    0.1090   6.5835      0.1090
   1196   1197  109.8    0.1066   6.1778      0.1080
  13775  13777  129.4    0.1078   7.4697      0.1090
  19019  19020   90.0    0.1096  10.1183      0.1090
   8580   8583  136.5    0.1084   7.3366      0.1090
   8580   8581   90.0    0.1090   0.3588      0.1090
   8580   8582   90.0    0.1090   0.3614      0.1090
   9074   9075   90.0    0.1075   9.8231      0.1080
   9076   9077  147.5    0.1067   9.0002      0.1080
   9896   9898   99.7    0.1004   6.7606      0.1010
   9896   9897   90.0    0.1010   0.4830      0.1010
  10386  10387   90.0    0.1090   0.1184      0.1090
  10386  10388  141.9    0.1090  10.0088      0.1090
  10386  10389   90.0    0.1090   0.1185      0.1090
  11578  11579   90.0    0.1090   0.6388      0.1090
   9422   9423   90.0    0.1094  10.1750      0.1090
   1148   1149   90.0    0.1088  10.3940      0.1090
   1148   1150  142.4    0.1092   7.7451      0.1090
   1154   1155   90.0    0.1008  10.4108      0.1010
   1598   1599  131.0    0.1106   4.8349      0.1090
   1604   1605   90.0    0.1090   0.2667      0.1090
   1604   1606  110.2    0.1080   9.4906      0.1090
   1604   1607   90.0    0.1090   0.2730      0.1090
   3667   3668  132.4    0.1093   6.0652      0.1090
   3667   3669   90.0    0.1090   0.6315      0.1090
  19048  19049   90.0    0.1102   4.9079      0.1090
  19060  19061   90.0    0.1027   6.3378      0.1010
  19082  19083   90.0    0.1086   3.7275      0.1090
  19095  19096   90.0    0.1002   4.5009      0.1010
   9049   9051   90.0    0.1100   7.2417      0.1090
   9927   9928   90.0    0.1095   7.0606      0.1090
   9927   9929   90.0    0.1090   0.5519      0.1090
   9927   9930   90.0    0.1090   0.5591      0.1090
   9482   9483   90.0    0.1090   0.5276      0.1090
   9482   9484  114.5    0.1098   6.1730      0.1090
   9485   9486   90.0    0.1086   3.7994      0.1090
   9489   9490   90.0    0.1011   4.5711      0.1010
   9511   9512  101.1    0.1019   8.7553      0.1010
   9515   9517   90.0    0.1090   0.4150      0.1090
   9518   9519  149.3    0.0958   8.2181      0.0960
   1736   1737  128.4    0.1102   6.6816      0.1090
   4046   4047  107.0    0.1067   6.5963      0.1090
   4046   4049   90.0    0.1090   0.4971      0.1090
   4046   4048   90.0    0.1090   9.7169      0.1090
   3663   3664   90.0    0.1096   6.5557      0.1090
   3663   3665   90.0    0.1090   0.5912      0.1090
   3663   3666   90.0    0.1090   0.5903      0.1090
   3200   3201  107.4    0.0971   7.1129      0.0960
   9049   9050   90.0    0.1089   9.5857      0.1090
  11155  11156   90.0    0.1108   3.8316      0.1090
  11188  11191   90.0    0.1090   4.2523      0.1090
  11188  11189   90.0    0.1090   0.6761      0.1090
  11188  11190   90.0    0.1090   0.7041      0.1090
  18206  18207   90.0    0.1098  10.5351      0.1090
   9923   9924   90.0    0.1094   6.6346      0.1090
   9923   9925   90.0    0.1090   0.5744      0.1090
   9923   9926   90.0    0.1090   0.5930      0.1090
   9933   9934  140.2    0.1003   5.2326      0.1010
   9618   9619  152.9    0.1097   8.0105      0.1090
   9618   9620   90.0    0.1090   0.2557      0.1090
   9618   9621   90.0    0.1090   0.2601      0.1090
   1781   1783   90.0    0.1096   3.3303      0.1090
   2700   2701  133.4    0.1091   5.7020      0.1090
   3689   3690  112.8    0.1092   5.0262      0.1090
   3689   3691   90.0    0.1090   0.6160      0.1090
   3689   3692   90.0    0.1090   0.6201      0.1090
   3685   3686   90.0    0.1090   0.7364      0.1090
   3685   3687   90.0    0.1090   0.7112      0.1090
  11139  11141  129.0    0.1097   5.0820      0.1090
  11139  11140   90.0    0.1090   0.5508      0.1090
   9946   9947  126.9    0.1083   7.6577      0.1080
   9989   9990   90.0    0.1090   0.1651      0.1090
   9989   9991  128.7    0.1098   9.8163      0.1090
   9989   9992   90.0    0.1079  10.2386      0.1090
  10621  10622   90.0    0.1090   0.6904      0.1090
  10621  10623   90.0    0.1082   4.4299      0.1090
  10621  10624   90.0    0.1090   0.6764      0.1090
   2588   2589   90.0    0.1095   6.4348      0.1080
  20826  20828  122.3    0.1081   6.1282      0.1090
  20829  20830   90.0    0.1090   0.4427      0.1090
  20829  20831  140.2    0.1092   5.7888      0.1090
  20561  20562   90.0    0.1009   8.7975      0.1010
  20575  20576  123.5    0.1091   6.7690      0.1090
  20575  20577   90.0    0.1090   0.4539      0.1090
  20876  20877   90.0    0.1093  10.3669      0.1090
  20588  20589   90.0    0.1071  10.5422      0.1080
  15659  15660  138.8    0.1087   8.0495      0.1090
  15428  15430   90.0    0.1090   0.6731      0.1090
  15435  15436   90.0    0.1009   4.7361      0.1010
  15449  15450  126.0    0.1020   9.5995      0.1010
  20966  20968   90.0    0.1087   4.9811      0.1090
  20199  20200  143.3    0.1079   7.4455      0.1090
  20138  20139  163.5    0.1022   7.7384      0.1010
    303    305  153.3    0.1090   8.2944      0.1090
    303    304   90.0    0.1090   0.2333      0.1090
    306    307  152.8    0.1090   7.4808      0.1090
  20129  20131  113.6    0.1098   6.4684      0.1090
   5040   5041   90.0    0.1093   5.7699      0.1090
    179    180  129.2    0.1090   4.1484      0.1090
    239    240  107.9    0.1089   6.6758      0.1090
    249    250  134.1    0.1012   4.6606      0.1010
    259    260  146.1    0.0967   5.3406      0.0960
    276    278  147.3    0.1096   7.6051      0.1090
    276    277   90.0    0.1090   0.4440      0.1090
   6597   6598  103.4    0.1084   5.6185      0.1090
   6597   6599   90.0    0.1090   0.6203      0.1090
   6597   6600   90.0    0.1090   0.6101      0.1090
  21782  21783  172.9    0.1089   4.8126      0.1090
  19571  19572   90.0    0.1008   6.6696      0.1010
    159    160  119.1    0.1007   4.8327      0.1010
    241    242  123.5    0.1096  10.2124      0.1090
    241    243   90.0    0.1090   0.1332      0.1090
   4908   4911   90.0    0.1079   3.4328      0.1090
   4908   4909   90.0    0.1090   0.7394      0.1090
   4908   4910   90.0    0.1090   0.6806      0.1090
  21751  21752   90.0    0.0995   7.1195      0.1010
   4885   4886   90.0    0.1086   9.8441      0.1090
   4887   4889   90.0    0.1090   0.7662      0.1090
   6422   6423  161.1    0.1083   5.5390      0.1090
   6422   6424   90.0    0.1090   0.5612      0.1090
   6422   6425   90.0    0.1090   0.5963      0.1090
  21682  21683  125.7    0.1088   9.4025      0.1090
  21682  21684   90.0    0.1090   0.1983      0.1090
  16396  16398   90.0    0.1090   0.6165      0.1090
   4802   4803   90.0    0.1098   9.0848      0.1090
   4802   4804   90.0    0.1086   6.9961      0.1090
   4802   4805   90.0    0.1090   0.5063      0.1090
   6381   6382   90.0    0.1071   8.3651      0.1080
  16393  16394   90.0    0.1090   0.6008      0.1090
  13592  13593  131.7    0.1084   5.8920      0.1080
  13594  13595   90.0    0.1083   4.5561      0.1080
  21562  21564   90.0    0.1070   5.4320      0.1090
  21565  21567   90.0    0.1090   0.6582      0.1090
  21568  21569  107.9    0.1021   6.4582      0.1010
  21568  21570   90.0    0.1010   0.4656      0.1010
  21568  21571   90.0    0.1010   0.4751      0.1010
   4586   4587  104.4    0.1074   5.6598      0.1080
  13669  13670   90.0    0.1010   0.4549      0.1010
  13669  13671   90.0    0.1000   6.5914      0.1010
   9286   9287  150.2    0.1007   6.4895      0.1010
   9286   9288   90.0    0.1010   0.3111      0.1010
   9304   9305   90.0    0.1088  10.2155      0.1090
   9304   9306   90.0    0.1083  10.3726      0.1090
   4477   4478   90.0    0.1090   0.6122      0.1090
   4477   4479   90.0    0.1093   6.0885      0.1090
   4480   4481   90.0    0.1092   7.3648      0.1090
   4480   4482   90.0    0.1090   0.5385      0.1090
   4480   4483  152.4    0.1074   8.9618      0.1090
   6007   6009  130.4    0.1091   9.9879      0.1090
   6007   6010   90.0    0.1090   0.1360      0.1090
  13926  13928   90.0    0.1090   0.7798      0.1090
  17904  17905   90.0    0.1010   0.4129      0.1010
  17526  17528   90.0    0.1010   7.0019      0.1010
  11641  11642  147.8    0.1007   7.1049      0.1010
  11645  11646   90.0    0.1095   4.8718      0.1090
  11645  11647   90.0    0.1090   0.6948      0.1090
  11654  11656   90.0    0.1090   0.7261      0.1090
  11657  11658  117.4    0.1004   6.2371      0.1010
  11657  11659   90.0    0.1010   0.4342      0.1010
  11657  11660   90.0    0.1010   0.4386      0.1010
[15:37:07] ***** Got an Activate signal (2)
[15:37:07] Killing all core threads

[susato]

Thanks harlam357 - we will await future reports on this WU and will check it in the mods WU database when that service is restored.

[Phantom]

I get the same problems with this WU. Keeps stopping all by itself after I restart it...

Code: Select all

--- Opening Log file [July 28 03:17:39 UTC] 


# Mac OS X SMP Console Edition ################################################
###############################################################################

                       Folding@Home Client Version 6.24beta

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: /Users/mini8/Library/InCrease/cpu2
Executable: /Users/mini8/Library/InCrease/cpu2/fah6
Arguments: -local -advmethods -forceasm -verbosity 9 -smp 

[03:17:39] - Ask before connecting: No
[03:17:39] - User name: Phantom (Team 1971)
[03:17:39] - User ID: xxxxxxxxxxxxxxxxxxxxx
[03:17:39] - Machine ID: 2
[03:17:39] 
[03:17:39] Loaded queue successfully.
[03:17:39] 
[03:17:39] - Autosending finished units... [03:17:39]
[03:17:39] + Processing work unit
[03:17:39] Trying to send all finished work units
[03:17:39] At least 4 processors must be requested.[03:17:39] + No unsent completed units remaining.
Core required: FahCore_a2.exe
[03:17:39] - Autosend completed
[03:17:39] Core found.
[03:17:39] - Using generic ./mpiexec
[03:17:39] Working on queue slot 02 [July 28 03:17:39 UTC]
[03:17:39] + Working ...
[03:17:39] - Calling './mpiexec -np 4 -host 127.0.0.1 ./FahCore_a2.exe -dir work/ -suffix 02 -checkpoint 30 -forceasm -verbose -lifeline 6031 -version 624'

[03:17:40] 
[03:17:40] *------------------------------*
[03:17:40] Folding@Home Gromacs SMP Core
[03:17:40] Version 2.07 (Sun Apr 19 14:29:51 PDT 2009)
[03:17:40] 
[03:17:40] Preparing to commence simulation
[03:17:40] - Ensuring status. Please wait.
[03:17:40] y forced on.
[03:17:40] - Not checking prior termination.
[03:17:42] - Expanded 4844492 -> 23994061 (decompressed 495.2 percent)
[03:17:42] Called DecompressByteArray: compressed_data_size=4844492 data_size=23994061, decompressed_data_size=23994061 diff=0
[03:17:42] - Digital signature verified
[03:17:42] 
[03:17:42] Project: 2675 (Run 3, Clone 169, Gen 78)
[03:17:42] 
[03:17:42] Assembly optimizations on if available.
[03:17:42] Entering M.D.
[03:17:51]  on if available.
[03:17:51] Entering M.D.
[03:18:01] Completed 0 out of 749980 steps  (0%)
[03:18:01] 
[03:18:01] Folding@home Core Shutdown: INTERRUPTED
[03:18:05] CoreStatus = 66 (102)
[03:18:05] + Shutdown requested by user. Exiting.***** Got a SIGTERM signal (15)
[03:18:05] Killing all core threads

[vladh4x0r]

Same thing, same WU failing the same way here. I kept restarting it and that did not help. Segfaulting on exit. I'll get another one...

[bruce]

WU reported.

[BrokenWolf]

I just received (well 2.6 hours ago) this WU on one of my linux SMP folders.

Folding@home core Shutdown:INTERRUPTED
CoreStatus = 66(102)

Errors I can see on the screen (system is at work and I am on my home system here): Water Molecule starting @ atom 90424 can not be settled.
Same as above but atom 59665 as well.

Broken

edit:Cleared out the work folder, rm'ed queue.dat and machinedependent.dat and was lucky enough to download an 2669 A2 core WU.

[BrokenWolf]

Ya all ain't gonna believe what one of my systems was assigned about 4 hours ago. Yep...you guessed it. p2675, R3/C169/G78. Can we _please_ do something about this R/C/G?

Thanks,

BrokenWolf

Code: Select all

                        :-)  G  R  O  M  A  C  S  (-:

                   Groningen Machine for Chemical Simulation

                 :-)  VERSION 4.0.99_development_20090425  (-:
NODEID=1 argc=22


      Written by David van der Spoel, Erik Lindahl, Berk Hess, and others.
       Copyright (c) 1991-2000, University of Groningen, The Netherlands.
             Copyright (c) 2001-2008, The GROMACS development team,
            check out http://www.gromacs.org for more information.


                                :-)  mdrun  (-:

NODEID=3 argc=22
Reading file work/wudata_04.tpr, VERSION 3.3.99_development_20070618 (single precision)
Note: tpx file_version 48, software version 65

NOTE: The tpr file used for this simulation is in an old format, for less memory usage and possibly more performance create a new tpr file with an up to date version of grompp

Making 1D domain decomposition 1 x 1 x 4
starting mdrun '22878 system in water'
19750002 steps,  39500.0 ps (continuing from step 19000022,  38000.0 ps).
[12:12:04] Completed 0 out of 749980 steps  (0%)

t = 38000.046 ps: Water molecule starting at atom 59665 can not be settled.
Check for bad contacts and/or reduce the timestep.

t = 38000.048 ps: Water molecule starting at atom 59665 can not be settled.
Check for bad contacts and/or reduce the timestep.

t = 38000.048 ps: Water molecule starting at atom 120505 can not be settled.
Check for bad contacts and/or reduce the timestep.

t = 38000.050 ps: Water molecule starting at atom 90424 can not be settled.
Check for bad contacts and/or reduce the timestep.

t = 38000.050 ps: Water molecule starting at atom 59665 can not be settled.
Check for bad contacts and/or reduce the timestep.
[12:12:06]
[12:12:06] Folding@home Core Shutdown: INTERRUPTED
[cli_0]: aborting job:
application called MPI_Abort(MPI_COMM_WORLD, 102) - process 0
[cli_1]: aborting job:
Fatal error in MPI_Sendrecv: Error message texts are not available
[cli_2]: aborting job:
Fatal error in MPI_Sendrecv: Error message texts are not available
[0]0:Return code = 102
[0]1:Return code = 1
[0]2:Return code = 1
[0]3:Return code = 0, signaled with Segmentation fault
[12:12:10] CoreStatus = 66 (102)
[12:12:10] + Shutdown requested by user. Exiting.***** Got a SIGTERM signal (15)[12:12:10] Killing all core threads

[kasson]

WU terminated.