Page 1 of 1

Project: 3866 (Run 3811, Clone 0, Gen 0)

Posted: Wed Jun 22, 2011 8:36 pm
by DrSpalding
Hi,

This new A6 WU project seems kind of slow, as in looking hung and making no progress. We let a Linux v6 client continue for a day w/o any progress at 87% completion after doing about 2h:30 per frame. It is about half the PPD production as usual. In any case, we finally decided to kill it and try a new one. Here is a snippet of the log files.

Code: Select all

--- Opening Log file [April 11 15:22:18] 


# Linux Console Edition #######################################################
###############################################################################

                       Folding@Home Client Version 6.02

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: /var/www/html/folding
Executable: ./fah6


[15:22:18] - Ask before connecting: No
[15:22:18] - User name: DrSpalding (Team 48083)
[15:22:18] - User ID not found locally
[15:22:18] + Requesting User ID from server
[15:23:51] - Machine ID: 1
[15:23:51] 
[15:23:51] Could not open work queue, generating new queue...
[15:23:51] - Preparing to get new work unit...
[15:23:51] + Attempting to get work packet
[15:23:51] - Connecting to assignment server
[15:27:00] - Couldn't send HTTP request to server
[15:27:00] + Could not connect to Assignment Server
[15:27:01] - Successful: assigned to (171.67.108.33).
[15:27:01] + News From Folding@Home: Welcome to Folding@Home
[15:27:01] Loaded queue successfully.
[15:27:08] + Closed connections
[15:27:08] 
[15:27:08] + Processing work unit
[15:27:08] Core required: FahCore_78.exe
[15:27:08] Core not found.
[15:27:08] - Core is not present or corrupted.
[15:27:08] - Attempting to download new core...
[15:27:08] + Downloading new core: FahCore_78.exe
[15:27:25] + 1134407 bytes downloaded
[15:27:25] Verifying core Core_78.fah...
[15:27:26] Signature is VALID
[15:27:26] 
[15:27:26] Trying to unzip core FahCore_78.exe
[15:27:26] Decompressed FahCore_78.exe (3435296 bytes) successfully
[15:27:26] + Core successfully engaged
[15:27:32] 
[15:27:32] + Processing work unit
[15:27:32] Core required: FahCore_78.exe
[15:27:32] Core found.
[15:27:32] Working on Unit 01 [April 11 15:27:32]
[15:27:32] + Working ...
[15:27:32] 
[15:27:32] *------------------------------*
[15:27:32] Folding@Home Gromacs Core
[15:27:32] Version 1.90 (March 8, 2006)
[15:27:32] 
[15:27:32] Preparing to commence simulation
[15:27:32] - Looking at optimizations...
[15:27:32] - Created dyn
[15:27:32] - Files status OK
[15:27:32] - Expanded 375366 -> 1809212 (decompressed 481.9 percent)
[15:27:32] - Starting from initial work packet
[15:27:32] 
[15:27:32] Project: 6886 (Run 253, Clone 8, Gen 106)
[15:27:32] 
[15:27:32] Assembly optimizations on if available.
[15:27:32] Entering M.D.
[15:27:38] Protein: 634 Abeta42_37dPro
[15:27:38] 
[15:27:38] Writing local files
[15:27:47] Extra SSE boost OK.
[15:27:47] Writing local files
[15:27:47] Completed 0 out of 250000 steps  (0%)
[15:42:53] Writing local files
[15:42:53] Completed 2500 out of 250000 steps  (1%)
[15:57:02] Writing local files
[15:57:02] Completed 5000 out of 250000 steps  (2%)
[16:09:38] Writing local files
[16:09:38] Completed 7500 out of 250000 steps  (3%)
[16:22:21] Writing local files
[16:22:21] Completed 10000 out of 250000 steps  (4%)
[16:37:25] Writing local files
[16:37:25] Completed 12500 out of 250000 steps  (5%)
[16:52:28] Writing local files
[16:52:28] Completed 15000 out of 250000 steps  (6%)
[17:07:30] Writing local files

...

[15:58:35] Completed 245000 out of 250000 steps  (98%)
[16:13:37] Writing local files
[16:13:37] Completed 247500 out of 250000 steps  (99%)
[16:28:53] Writing local files
[16:28:53] Completed 250000 out of 250000 steps  (100%)
[16:28:53] Writing final coordinates.
[16:28:53] Past main M.D. loop
[16:29:53] 
[16:29:53] Finished Work Unit:
[16:29:53] - Reading up to 293616 from "work/wudata_01.arc": Read 293616
[16:29:53] - Reading up to 261268 from "work/wudata_01.xtc": Read 261268
[16:29:53] goefile size: 0
[16:29:53] logfile size: 21566
[16:29:53] Leaving Run
[16:29:53] - Writing 582358 bytes of core data to disk...
[16:29:54] Done: 581846 -> 561187 (compressed to 96.4 percent)
[16:29:54]   ... Done.
[16:29:54] - Shutting down core
[16:29:54] 
[16:29:54] Folding@home Core Shutdown: FINISHED_UNIT
[16:29:55] CoreStatus = 64 (100)
[16:29:55] Sending work to server


[16:29:55] + Attempting to send results
[16:30:15] + Results successfully sent
[16:30:15] Thank you for your contribution to Folding@Home.
[16:30:15] + Number of Units Completed: 301

...

[23:48:37] - Preparing to get new work unit...
[23:48:37] + Attempting to get work packet
[23:48:37] - Connecting to assignment server
[23:48:38] - Successful: assigned to (128.143.48.226).
[23:48:38] + News From Folding@Home: Welcome to Folding@Home
[23:48:39] Loaded queue successfully.
[23:49:02] + Closed connections
[23:49:02] 
[23:49:02] + Processing work unit
[23:49:02] Core required: FahCore_a6.exe
[23:49:02] Core not found.
[23:49:02] - Core is not present or corrupted.
[23:49:02] - Attempting to download new core...
[23:49:02] + Downloading new core: FahCore_a6.exe
[23:49:41] + 2334692 bytes downloaded
[23:49:41] Verifying core Core_a6.fah...
[23:49:41] Signature is VALID
[23:49:41] 
[23:49:41] Trying to unzip core FahCore_a6.exe
[23:49:43] Decompressed FahCore_a6.exe (6074460 bytes) successfully
[23:49:43] + Core successfully engaged
[23:49:48] 
[23:49:48] + Processing work unit
[23:49:48] Core required: FahCore_a6.exe
[23:49:48] Core found.
[23:49:48] Working on Unit 09 [June 10 23:49:48]
[23:49:48] + Working ...
[23:49:48] 
[23:49:48] *------------------------------*
[23:49:48] Folding@Home Gromacs Core
[23:49:48] Version 2.28 (Wed Mar 23 13:51:17 PDT 2011)
[23:49:48] 
[23:49:48] Preparing to commence simulation
[23:49:48] - Looking at optimizations...
[23:49:48] - Created dyn
[23:49:48] - Files status OK
[23:49:49] - Expanded 1010615 -> 2407352 (decompressed 238.2 percent)
[23:49:49] Called DecompressByteArray: compressed_data_size=1010615 data_size=2407352, decompressed_data_size=2407352 diff=0
[23:49:49] - Digital signature verified
[23:49:49] 
[23:49:49] Project: 3866 (Run 3811, Clone 0, Gen 0)
[23:49:49] 
[23:49:49] Assembly optimizations on if available.
[23:49:49] Entering M.D.
[23:49:55] Mapping NT from 1 to 1 
[23:50:04] Completed 0 out of 250000 steps  (0%)
[02:18:39] Completed 2500 out of 250000 steps  (1%)
[04:47:45] Completed 5000 out of 250000 steps  (2%)
[07:16:49] Completed 7500 out of 250000 steps  (3%)
[09:46:09] Completed 10000 out of 250000 steps  (4%)
[12:16:08] Completed 12500 out of 250000 steps  (5%)
[14:46:28] Completed 15000 out of 250000 steps  (6%)
[17:16:50] Completed 17500 out of 250000 steps  (7%)
[19:46:21] Completed 20000 out of 250000 steps  (8%)
[22:17:33] Completed 22500 out of 250000 steps  (9%)
[00:49:18] Completed 25000 out of 250000 steps  (10%)
[03:09:45] Completed 27500 out of 250000 steps  (11%)
[05:41:34] Completed 30000 out of 250000 steps  (12%)
[08:14:08] Completed 32500 out of 250000 steps  (13%)
[10:47:41] Completed 35000 out of 250000 steps  (14%)
[13:25:51] Completed 37500 out of 250000 steps  (15%)
[16:00:52] Completed 40000 out of 250000 steps  (16%)
[18:34:54] Completed 42500 out of 250000 steps  (17%)
[21:06:45] Completed 45000 out of 250000 steps  (18%)
[23:39:18] Completed 47500 out of 250000 steps  (19%)
[02:12:40] Completed 50000 out of 250000 steps  (20%)
[04:45:32] Completed 52500 out of 250000 steps  (21%)
[07:21:14] Completed 55000 out of 250000 steps  (22%)
[09:55:12] Completed 57500 out of 250000 steps  (23%)
[12:31:08] Completed 60000 out of 250000 steps  (24%)
[15:04:33] Completed 62500 out of 250000 steps  (25%)
[17:38:41] Completed 65000 out of 250000 steps  (26%)
[20:16:58] Completed 67500 out of 250000 steps  (27%)
[22:50:09] Completed 70000 out of 250000 steps  (28%)
[01:26:01] Completed 72500 out of 250000 steps  (29%)
[03:58:59] Completed 75000 out of 250000 steps  (30%)
[06:32:04] Completed 77500 out of 250000 steps  (31%)
[09:10:06] Completed 80000 out of 250000 steps  (32%)
[11:47:58] Completed 82500 out of 250000 steps  (33%)
[14:23:06] Completed 85000 out of 250000 steps  (34%)
[16:56:12] Completed 87500 out of 250000 steps  (35%)
[19:29:34] Completed 90000 out of 250000 steps  (36%)
[22:04:38] Completed 92500 out of 250000 steps  (37%)
[00:38:11] Completed 95000 out of 250000 steps  (38%)
[03:16:10] Completed 97500 out of 250000 steps  (39%)
[05:51:31] Completed 100000 out of 250000 steps  (40%)
[08:24:56] Completed 102500 out of 250000 steps  (41%)
[10:59:39] Completed 105000 out of 250000 steps  (42%)
[13:38:28] Completed 107500 out of 250000 steps  (43%)
[16:15:57] Completed 110000 out of 250000 steps  (44%)
[18:49:23] Completed 112500 out of 250000 steps  (45%)
[21:23:24] Completed 115000 out of 250000 steps  (46%)
[23:59:00] Completed 117500 out of 250000 steps  (47%)
[02:35:27] Completed 120000 out of 250000 steps  (48%)
[05:12:39] Completed 122500 out of 250000 steps  (49%)
[07:48:24] Completed 125000 out of 250000 steps  (50%)
[10:22:02] Completed 127500 out of 250000 steps  (51%)
[12:57:05] Completed 130000 out of 250000 steps  (52%)
[03:12:47] Completed 132500 out of 250000 steps  (53%)
[05:51:56] Completed 135000 out of 250000 steps  (54%)
[08:28:19] Completed 137500 out of 250000 steps  (55%)
[11:04:05] Completed 140000 out of 250000 steps  (56%)
[13:39:39] Completed 142500 out of 250000 steps  (57%)
[16:16:51] Completed 145000 out of 250000 steps  (58%)
[18:56:26] Completed 147500 out of 250000 steps  (59%)
[21:32:40] Completed 150000 out of 250000 steps  (60%)
[01:00:36] Completed 152500 out of 250000 steps  (61%)
[03:36:44] Completed 155000 out of 250000 steps  (62%)
[06:13:27] Completed 157500 out of 250000 steps  (63%)
[08:50:54] Completed 160000 out of 250000 steps  (64%)
[11:28:50] Completed 162500 out of 250000 steps  (65%)
[14:06:12] Completed 165000 out of 250000 steps  (66%)
[16:42:50] Completed 167500 out of 250000 steps  (67%)
[19:18:29] Completed 170000 out of 250000 steps  (68%)
[21:57:15] Completed 172500 out of 250000 steps  (69%)
[00:31:41] Completed 175000 out of 250000 steps  (70%)
[03:08:45] Completed 177500 out of 250000 steps  (71%)
[05:44:39] Completed 180000 out of 250000 steps  (72%)
[08:23:43] Completed 182500 out of 250000 steps  (73%)
[10:59:59] Completed 185000 out of 250000 steps  (74%)
[13:42:32] Completed 187500 out of 250000 steps  (75%)
[16:21:59] Completed 190000 out of 250000 steps  (76%)
[18:57:43] Completed 192500 out of 250000 steps  (77%)
[21:34:04] Completed 195000 out of 250000 steps  (78%)
[00:12:25] Completed 197500 out of 250000 steps  (79%)
[02:47:01] Completed 200000 out of 250000 steps  (80%)
[05:23:31] Completed 202500 out of 250000 steps  (81%)
[08:02:55] Completed 205000 out of 250000 steps  (82%)
[10:38:36] Completed 207500 out of 250000 steps  (83%)
[13:15:32] Completed 210000 out of 250000 steps  (84%)
[15:53:33] Completed 212500 out of 250000 steps  (85%)
[18:29:15] Completed 215000 out of 250000 steps  (86%)
[21:05:04] Completed 217500 out of 250000 steps  (87%)

Folding@Home Client Shutdown.

--- Opening Log file [June 21 15:48:18] 


# Linux Console Edition #######################################################
###############################################################################

                       Folding@Home Client Version 6.02

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: /var/www/html/folding
Executable: ./fah6


[15:48:18] - Ask before connecting: No
[15:48:18] - User name: DrSpalding (Team 48083)
[15:48:18] - User ID: 6D0B77E112F08588
[15:48:18] - Machine ID: 1
[15:48:18] 
[15:48:18] Loaded queue successfully.
[15:48:18] 
[15:48:18] + Processing work unit
[15:48:18] Core required: FahCore_a6.exe
[15:48:18] Core found.
[15:48:18] Working on Unit 09 [June 21 15:48:18]
[15:48:18] + Working ...
[15:48:18] 
[15:48:18] *------------------------------*
[15:48:18] Folding@Home Gromacs Core
[15:48:18] Version 2.28 (Wed Mar 23 13:51:17 PDT 2011)
[15:48:18] 
[15:48:18] Preparing to commence simulation
[15:48:18] - Looking at optimizations...
[15:48:18] - Files status OK
[16:00:08] - Expanded 1010615 -> 2407352 (decompressed 238.2 percent)
[16:00:22] Called DecompressByteArray: compressed_data_size=1010615 data_size=2407352, decompressed_data_size=2407352 diff=0
[16:01:57] - Digital signature verified
[16:01:59] 
[16:01:59] Project: 3866 (Run 3811, Clone 0, Gen 0)
[16:01:59] 
[16:02:02] Assembly optimizations on if available.
[16:02:02] Entering M.D.
[16:02:08] Using Gromacs checkpoints
[16:07:33] Mapping NT from 1 to 1 
[16:48:35] Resuming from checkpoint
[16:50:21] Verified work/wudata_09.log
[16:50:55] Verified work/wudata_09.trr
[16:51:08] Verified work/wudata_09.xtc
[16:51:23] Verified work/wudata_09.edr
[19:06:41] Completed 219780 out of 250000 steps  (87%)

Folding@Home Client Shutdown.

--- Opening Log file [June 22 15:34:19] 


# Linux Console Edition #######################################################
###############################################################################

                       Folding@Home Client Version 6.02

                          http://folding.stanford.edu

###############################################################################
###############################################################################

Launch directory: /var/www/html/folding
Executable: ./fah6


[15:34:19] - Ask before connecting: No
[15:34:19] - User name: DrSpalding (Team 48083)
[15:34:19] - User ID: 6D0B77E112F08588
[15:34:19] - Machine ID: 1
[15:34:19] 
[15:34:19] Work directory not found. Creating...
[15:34:19] Could not open work queue, generating new queue...
[15:34:19] - Preparing to get new work unit...
[15:34:19] + Attempting to get work packet
[15:34:19] - Connecting to assignment server
[15:34:19] - Successful: assigned to (128.143.48.226).
[15:34:19] + News From Folding@Home: Welcome to Folding@Home
[15:34:20] Loaded queue successfully.
[15:34:46] + Closed connections
[15:34:46] 
[15:34:46] + Processing work unit
[15:34:46] Core required: FahCore_a6.exe
[15:34:46] Core found.
[15:34:46] Working on Unit 01 [June 22 15:34:46]
[15:34:46] + Working ...
[15:34:47] 
[15:34:47] *------------------------------*
[15:34:47] Folding@Home Gromacs Core
[15:34:47] Version 2.28 (Wed Mar 23 13:51:17 PDT 2011)
[15:34:47] 
[15:34:47] Preparing to commence simulation
[15:34:47] - Looking at optimizations...
[15:34:47] - Created dyn
[15:34:47] - Files status OK
[15:49:14] - Expanded 1489083 -> 2411692 (decompressed 161.9 percent)
[15:49:28] Called DecompressByteArray: compressed_data_size=1489083 data_size=2411692, decompressed_data_size=2411692 diff=0
[15:51:04] - Digital signature verified
[15:51:04]
[15:51:04] Project: 3867 (Run 3474, Clone 0, Gen 5)
[15:51:04]
[15:51:08] Assembly optimizations on if available.
[15:51:08] Entering M.D.
[15:53:28] Mapping NT from 1 to 1
[18:28:29] Completed 0 out of 250000 steps  (0%)
As a further note, Project: 3867 (Run 3474, Clone 0, Gen 5) is now running for two hours on its first frame w/o finishing it. How long would we expect the A6 core and WUs to run to completion on middling hardware? It's an Intel something or other running 2.4 GHz and Linux. Is the ~2h:30 TPF on the WU in the subject expected or should we investigate if something else is up with the machine?

Thanks

Re: Project: 3866 (Run 3811, Clone 0, Gen 0)

Posted: Thu Jun 23, 2011 1:59 am
by PantherX
Project: 3866 (Run 3811, Clone 0, Gen 0) was successfully completed by another donor:
Your WU (P3866 R3811 C0 G0) was added to the stats database on 2011-06-18 13:07:51 for 333 points of credit.
Project: 3867 (Run 3474, Clone 0, Gen 5) doesn't have any data in the WU database yet:
No data back from query
I have marked it for a follow up.

Re: Project: 3866 (Run 3811, Clone 0, Gen 0)

Posted: Thu Jun 23, 2011 2:02 am
by DrSpalding
Project: 3867 (Run 3474, Clone 0, Gen 5) is currently sitting at 7h:30 on the first frame and hasn't completed 1% yet. I'm going to ask to reboot the machine after checking the ps output on whether or not the core executable is actually consuming any CPU at all. I think that 7h:30 or more per frame is not what the doctor ordered here.

Edit:
Machine has the following CPU specs:

Code: Select all

processor	: 0
vendor_id	: GenuineIntel
cpu family	: 15
model		: 2
model name	: Intel(R) Celeron(R) CPU 2.40GHz
stepping	: 9
cpu MHz		: 2392.270
cache size	: 128 KB
fdiv_bug	: no
hlt_bug		: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 2
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe up cid xtpr
bogomips	: 4787.53
clflush size	: 64

Re: Project: 3866 (Run 3811, Clone 0, Gen 0)

Posted: Sat Jun 25, 2011 3:36 am
by mrshirts
Thanks for the report. 3867 should be the same as 3866, so that's strange that is behaving so poorly. It's actually at very low weight compared to the others, so I'll take it off until we understand better what is going on.

Re: Project: 3866 (Run 3811, Clone 0, Gen 0)

Posted: Mon Jul 04, 2011 2:32 am
by PantherX
Looks like you completed it:
Hi DrSpalding (team 48083),
Your WU (P3867 R3474 C0 G5) was added to the stats database on 2011-07-03 09:07:06 for 333 points of credit.