Page 1 of 1
Project: 7012 (Run 2, Clone 84, Gen 99) - High TPF
Posted: Sun Nov 04, 2012 12:50 am
by Fahrenheit451
And another high TPF WU (s.a. my other thread viewtopic.php?f=19&t=22853). Normally I need ~2:30min per frame for this project, now TPF is on 8:40min and my PPD dropped by factor 8. CPU utilization from A4 core is at 99%. Something went wrong with 70xx projects in the last days
Code: Select all
16:05:17:WU01:FS00:Starting
16:05:17:WU01:FS00:Running FahCore: D:\FAHClient_V7/FAHCoreWrapper.exe C:/ProgramData/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/Core_a4.fah/FahCore_a4.exe -dir 01 -suffix 01 -version 701 -lifeline 5860 -checkpoint 15 -np 8
16:05:17:WU01:FS00:Started FahCore on PID 4264
16:05:17:WU01:FS00:Core PID:4724
16:05:17:WU01:FS00:FahCore 0xa4 started
16:05:18:WU01:FS00:0xa4:
16:05:18:WU01:FS00:0xa4:*------------------------------*
16:05:18:WU01:FS00:0xa4:Folding@Home Gromacs GB Core
16:05:18:WU01:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
16:05:18:WU01:FS00:0xa4:
16:05:18:WU01:FS00:0xa4:Preparing to commence simulation
16:05:18:WU01:FS00:0xa4:- Looking at optimizations...
16:05:18:WU01:FS00:0xa4:- Created dyn
16:05:18:WU01:FS00:0xa4:- Files status OK
16:05:18:WU01:FS00:0xa4:- Expanded 41028 -> 211504 (decompressed 515.5 percent)
16:05:18:WU01:FS00:0xa4:Called DecompressByteArray: compressed_data_size=41028 data_size=211504, decompressed_data_size=211504 diff=0
16:05:18:WU01:FS00:0xa4:- Digital signature verified
16:05:18:WU01:FS00:0xa4:
16:05:18:WU01:FS00:0xa4:Project: 7012 (Run 2, Clone 84, Gen 99)
16:05:18:WU01:FS00:0xa4:
16:05:18:WU01:FS00:0xa4:Assembly optimizations on if available.
16:05:18:WU01:FS00:0xa4:Entering M.D.
16:05:23:WU02:FS00:Upload 21.38%
16:05:23:WU01:FS00:0xa4:Mapping NT from 8 to 8
16:05:23:WU01:FS00:0xa4:Completed 0 out of 10000000 steps (0%)
16:05:29:WU02:FS00:Upload 42.75%
16:05:35:WU02:FS00:Upload 64.13%
16:05:41:WU02:FS00:Upload 85.50%
16:05:46:WU02:FS00:Upload complete
16:05:46:WU02:FS00:Server responded WORK_ACK (400)
16:05:46:WU02:FS00:Final credit estimate, 2885.00 points
16:05:46:WU02:FS00:Cleaning up
16:14:08:WU01:FS00:0xa4:Completed 100000 out of 10000000 steps (1%)
16:23:02:WU01:FS00:0xa4:Completed 200000 out of 10000000 steps (2%)
16:31:46:WU01:FS00:0xa4:Completed 300000 out of 10000000 steps (3%)
16:40:31:WU01:FS00:0xa4:Completed 400000 out of 10000000 steps (4%)
16:48:42:WU01:FS00:0xa4:Completed 500000 out of 10000000 steps (5%)
16:56:52:WU01:FS00:0xa4:Completed 600000 out of 10000000 steps (6%)
17:05:38:WU01:FS00:0xa4:Completed 700000 out of 10000000 steps (7%)
17:14:48:WU01:FS00:0xa4:Completed 800000 out of 10000000 steps (8%)
17:23:58:WU01:FS00:0xa4:Completed 900000 out of 10000000 steps (9%)
17:32:23:WU01:FS00:0xa4:Completed 1000000 out of 10000000 steps (10%)
17:40:49:WU01:FS00:0xa4:Completed 1100000 out of 10000000 steps (11%)
17:49:59:WU01:FS00:0xa4:Completed 1200000 out of 10000000 steps (12%)
17:59:09:WU01:FS00:0xa4:Completed 1300000 out of 10000000 steps (13%)
18:08:16:WU01:FS00:0xa4:Completed 1400000 out of 10000000 steps (14%)
18:17:28:WU01:FS00:0xa4:Completed 1500000 out of 10000000 steps (15%)
18:26:40:WU01:FS00:0xa4:Completed 1600000 out of 10000000 steps (16%)
18:34:54:WU01:FS00:0xa4:Completed 1700000 out of 10000000 steps (17%)
18:43:35:WU01:FS00:0xa4:Completed 1800000 out of 10000000 steps (18%)
18:51:49:WU01:FS00:0xa4:Completed 1900000 out of 10000000 steps (19%)
18:59:56:WU01:FS00:0xa4:Completed 2000000 out of 10000000 steps (20%)
19:08:04:WU01:FS00:0xa4:Completed 2100000 out of 10000000 steps (21%)
19:16:11:WU01:FS00:0xa4:Completed 2200000 out of 10000000 steps (22%)
19:24:19:WU01:FS00:0xa4:Completed 2300000 out of 10000000 steps (23%)
19:33:00:WU01:FS00:0xa4:Completed 2400000 out of 10000000 steps (24%)
19:41:44:WU01:FS00:0xa4:Completed 2500000 out of 10000000 steps (25%)
19:50:05:WU01:FS00:0xa4:Completed 2600000 out of 10000000 steps (26%)
19:58:46:WU01:FS00:0xa4:Completed 2700000 out of 10000000 steps (27%)
20:07:49:WU01:FS00:0xa4:Completed 2800000 out of 10000000 steps (28%)
20:16:33:WU01:FS00:0xa4:Completed 2900000 out of 10000000 steps (29%)
20:25:17:WU01:FS00:0xa4:Completed 3000000 out of 10000000 steps (30%)
20:34:01:WU01:FS00:0xa4:Completed 3100000 out of 10000000 steps (31%)
20:42:44:WU01:FS00:0xa4:Completed 3200000 out of 10000000 steps (32%)
20:51:26:WU01:FS00:0xa4:Completed 3300000 out of 10000000 steps (33%)
20:59:49:WU01:FS00:0xa4:Completed 3400000 out of 10000000 steps (34%)
21:08:12:WU01:FS00:0xa4:Completed 3500000 out of 10000000 steps (35%)
21:16:22:WU01:FS00:0xa4:Completed 3600000 out of 10000000 steps (36%)
21:24:31:WU01:FS00:0xa4:Completed 3700000 out of 10000000 steps (37%)
21:33:24:WU01:FS00:0xa4:Completed 3800000 out of 10000000 steps (38%)
******************************** Date: 03/11/12 ********************************
21:41:58:WU01:FS00:0xa4:Completed 3900000 out of 10000000 steps (39%)
21:50:43:WU01:FS00:0xa4:Completed 4000000 out of 10000000 steps (40%)
21:58:51:WU01:FS00:0xa4:Completed 4100000 out of 10000000 steps (41%)
22:07:01:WU01:FS00:0xa4:Completed 4200000 out of 10000000 steps (42%)
22:15:47:WU01:FS00:0xa4:Completed 4300000 out of 10000000 steps (43%)
22:24:32:WU01:FS00:0xa4:Completed 4400000 out of 10000000 steps (44%)
22:32:55:WU01:FS00:0xa4:Completed 4500000 out of 10000000 steps (45%)
22:41:21:WU01:FS00:0xa4:Completed 4600000 out of 10000000 steps (46%)
22:49:34:WU01:FS00:0xa4:Completed 4700000 out of 10000000 steps (47%)
22:57:46:WU01:FS00:0xa4:Completed 4800000 out of 10000000 steps (48%)
23:05:56:WU01:FS00:0xa4:Completed 4900000 out of 10000000 steps (49%)
23:15:03:WU01:FS00:0xa4:Completed 5000000 out of 10000000 steps (50%)
23:24:12:WU01:FS00:0xa4:Completed 5100000 out of 10000000 steps (51%)
23:33:18:WU01:FS00:0xa4:Completed 5200000 out of 10000000 steps (52%)
23:41:44:WU01:FS00:0xa4:Completed 5300000 out of 10000000 steps (53%)
23:50:49:WU01:FS00:0xa4:Completed 5400000 out of 10000000 steps (54%)
23:59:15:WU01:FS00:0xa4:Completed 5500000 out of 10000000 steps (55%)
00:08:02:WU01:FS00:0xa4:Completed 5600000 out of 10000000 steps (56%)
00:16:49:WU01:FS00:0xa4:Completed 5700000 out of 10000000 steps (57%)
00:25:36:WU01:FS00:0xa4:Completed 5800000 out of 10000000 steps (58%)
Values from HFM:
Project ID: 7012
Core: GRO-A4
Credit: 600
Frames: 100
Name: FAH Box One Slot 00
Path: 192.168.2.106-36330
Number of Frames Observed: 247
Min. Time / Frame : 00:02:31 - 23.587,6 PPD
Avg. Time / Frame : 00:04:13 - 10.876,0 PPD
Cur. Time / Frame : 00:08:47 - 3.622,9 PPD
R3F. Time / Frame : 00:08:40 - 3.682,2 PPD
All Time / Frame : 00:08:37 - 3.708,2 PPD
Eff. Time / Frame : 00:08:44 - 3.648,1 PPD
Name: FAH Box One Slot 01
Path: 192.168.2.106-36330
Number of Frames Observed: 257
Min. Time / Frame : 00:02:11 - 29.190,5 PPD
Avg. Time / Frame : 00:02:24 - 25.328,2 PPD
Re: Project: 7012 (Run 2, Clone 84, Gen 99) - High TPF
Posted: Sun Nov 04, 2012 1:26 am
by bruce
Normally nobody really pays any attention to the number of steps because different projects do have different numbers. In the past, one significant sign of the problem I think we're seeing is that for a single Project, moving to a different Run/Clone/Gen will find a different number of steps. Can anybody determine if that's happening?
Re: Project: 7012 (Run 2, Clone 84, Gen 99) - High TPF
Posted: Sun Nov 04, 2012 2:06 am
by bollix47
The only 7012 I could find appears to have the same number of steps and afaik it ran normally:
Code: Select all
15:10:05:WU00:FS01:0xa4:Project: 7012 (Run 3, Clone 19, Gen 53)
15:10:05:WU00:FS01:0xa4:
15:10:05:WU00:FS01:0xa4:Assembly optimizations on if available.
15:10:05:WU00:FS01:0xa4:Entering M.D.
15:10:10:WU01:FS01:Upload 32.16%
15:10:11:WU00:FS01:0xa4:Mapping NT from 8 to 8
15:10:11:WU00:FS01:0xa4:Completed 0 out of 10000000 steps (0%)
15:10:16:WU01:FS01:Upload 71.47%
15:10:20:WU01:FS01:Upload complete
15:10:20:WU01:FS01:Server responded WORK_ACK (400)
15:10:20:WU01:FS01:Final credit estimate, 3575.00 points
15:10:20:WU01:FS01:Cleaning up
15:13:58:WU00:FS01:0xa4:Completed 100000 out of 10000000 steps (1%)
15:17:47:WU00:FS01:0xa4:Completed 200000 out of 10000000 steps (2%)
15:21:35:WU00:FS01:0xa4:Completed 300000 out of 10000000 steps (3%)
15:25:23:WU00:FS01:0xa4:Completed 400000 out of 10000000 steps (4%)
15:29:11:WU00:FS01:0xa4:Completed 500000 out of 10000000 steps (5%)
15:32:59:WU00:FS01:0xa4:Completed 600000 out of 10000000 steps (6%)
15:36:47:WU00:FS01:0xa4:Completed 700000 out of 10000000 steps (7%)
15:40:36:WU00:FS01:0xa4:Completed 800000 out of 10000000 steps (8%)
15:44:29:WU00:FS01:0xa4:Completed 900000 out of 10000000 steps (9%)
15:48:21:WU00:FS01:0xa4:Completed 1000000 out of 10000000 steps (10%)
15:52:14:WU00:FS01:0xa4:Completed 1100000 out of 10000000 steps (11%)
15:56:06:WU00:FS01:0xa4:Completed 1200000 out of 10000000 steps (12%)
15:59:59:WU00:FS01:0xa4:Completed 1300000 out of 10000000 steps (13%)
16:03:51:WU00:FS01:0xa4:Completed 1400000 out of 10000000 steps (14%)
16:07:45:WU00:FS01:0xa4:Completed 1500000 out of 10000000 steps (15%)
16:11:38:WU00:FS01:0xa4:Completed 1600000 out of 10000000 steps (16%)
16:15:26:WU00:FS01:0xa4:Completed 1700000 out of 10000000 steps (17%)
16:19:15:WU00:FS01:0xa4:Completed 1800000 out of 10000000 steps (18%)
16:23:02:WU00:FS01:0xa4:Completed 1900000 out of 10000000 steps (19%)
16:26:50:WU00:FS01:0xa4:Completed 2000000 out of 10000000 steps (20%)
16:30:38:WU00:FS01:0xa4:Completed 2100000 out of 10000000 steps (21%)
16:34:25:WU00:FS01:0xa4:Completed 2200000 out of 10000000 steps (22%)
16:38:13:WU00:FS01:0xa4:Completed 2300000 out of 10000000 steps (23%)
16:42:01:WU00:FS01:0xa4:Completed 2400000 out of 10000000 steps (24%)
16:45:50:WU00:FS01:0xa4:Completed 2500000 out of 10000000 steps (25%)
16:49:38:WU00:FS01:0xa4:Completed 2600000 out of 10000000 steps (26%)
16:53:12:WU00:FS01:0xa4:Completed 2700000 out of 10000000 steps (27%)
16:56:51:WU00:FS01:0xa4:Completed 2800000 out of 10000000 steps (28%)
17:00:37:WU00:FS01:0xa4:Completed 2900000 out of 10000000 steps (29%)
17:04:22:WU00:FS01:0xa4:Completed 3000000 out of 10000000 steps (30%)
17:08:07:WU00:FS01:0xa4:Completed 3100000 out of 10000000 steps (31%)
17:11:54:WU00:FS01:0xa4:Completed 3200000 out of 10000000 steps (32%)
17:15:43:WU00:FS01:0xa4:Completed 3300000 out of 10000000 steps (33%)
17:19:34:WU00:FS01:0xa4:Completed 3400000 out of 10000000 steps (34%)
17:23:23:WU00:FS01:0xa4:Completed 3500000 out of 10000000 steps (35%)
17:27:13:WU00:FS01:0xa4:Completed 3600000 out of 10000000 steps (36%)
17:31:03:WU00:FS01:0xa4:Completed 3700000 out of 10000000 steps (37%)
17:34:53:WU00:FS01:0xa4:Completed 3800000 out of 10000000 steps (38%)
17:38:43:WU00:FS01:0xa4:Completed 3900000 out of 10000000 steps (39%)
17:42:44:WU00:FS01:0xa4:Completed 4000000 out of 10000000 steps (40%)
17:46:29:WU00:FS01:0xa4:Completed 4100000 out of 10000000 steps (41%)
17:50:14:WU00:FS01:0xa4:Completed 4200000 out of 10000000 steps (42%)
17:54:03:WU00:FS01:0xa4:Completed 4300000 out of 10000000 steps (43%)
17:57:53:WU00:FS01:0xa4:Completed 4400000 out of 10000000 steps (44%)
18:01:43:WU00:FS01:0xa4:Completed 4500000 out of 10000000 steps (45%)
18:05:32:WU00:FS01:0xa4:Completed 4600000 out of 10000000 steps (46%)
18:09:22:WU00:FS01:0xa4:Completed 4700000 out of 10000000 steps (47%)
18:13:11:WU00:FS01:0xa4:Completed 4800000 out of 10000000 steps (48%)
18:17:01:WU00:FS01:0xa4:Completed 4900000 out of 10000000 steps (49%)
18:20:50:WU00:FS01:0xa4:Completed 5000000 out of 10000000 steps (50%)
18:24:40:WU00:FS01:0xa4:Completed 5100000 out of 10000000 steps (51%)
18:28:29:WU00:FS01:0xa4:Completed 5200000 out of 10000000 steps (52%)
18:32:19:WU00:FS01:0xa4:Completed 5300000 out of 10000000 steps (53%)
18:36:07:WU00:FS01:0xa4:Completed 5400000 out of 10000000 steps (54%)
18:39:57:WU00:FS01:0xa4:Completed 5500000 out of 10000000 steps (55%)
18:43:46:WU00:FS01:0xa4:Completed 5600000 out of 10000000 steps (56%)
18:47:36:WU00:FS01:0xa4:Completed 5700000 out of 10000000 steps (57%)
18:51:25:WU00:FS01:0xa4:Completed 5800000 out of 10000000 steps (58%)
18:55:16:WU00:FS01:0xa4:Completed 5900000 out of 10000000 steps (59%)
18:59:07:WU00:FS01:0xa4:Completed 6000000 out of 10000000 steps (60%)
19:02:59:WU00:FS01:0xa4:Completed 6100000 out of 10000000 steps (61%)
19:06:50:WU00:FS01:0xa4:Completed 6200000 out of 10000000 steps (62%)
19:10:40:WU00:FS01:0xa4:Completed 6300000 out of 10000000 steps (63%)
19:14:28:WU00:FS01:0xa4:Completed 6400000 out of 10000000 steps (64%)
19:18:17:WU00:FS01:0xa4:Completed 6500000 out of 10000000 steps (65%)
19:22:07:WU00:FS01:0xa4:Completed 6600000 out of 10000000 steps (66%)
19:25:51:WU00:FS01:0xa4:Completed 6700000 out of 10000000 steps (67%)
19:29:36:WU00:FS01:0xa4:Completed 6800000 out of 10000000 steps (68%)
19:33:21:WU00:FS01:0xa4:Completed 6900000 out of 10000000 steps (69%)
19:37:05:WU00:FS01:0xa4:Completed 7000000 out of 10000000 steps (70%)
19:40:50:WU00:FS01:0xa4:Completed 7100000 out of 10000000 steps (71%)
19:44:40:WU00:FS01:0xa4:Completed 7200000 out of 10000000 steps (72%)
19:48:29:WU00:FS01:0xa4:Completed 7300000 out of 10000000 steps (73%)
19:52:19:WU00:FS01:0xa4:Completed 7400000 out of 10000000 steps (74%)
19:56:08:WU00:FS01:0xa4:Completed 7500000 out of 10000000 steps (75%)
19:59:58:WU00:FS01:0xa4:Completed 7600000 out of 10000000 steps (76%)
20:03:42:WU00:FS01:0xa4:Completed 7700000 out of 10000000 steps (77%)
20:07:27:WU00:FS01:0xa4:Completed 7800000 out of 10000000 steps (78%)
20:11:12:WU00:FS01:0xa4:Completed 7900000 out of 10000000 steps (79%)
20:14:57:WU00:FS01:0xa4:Completed 8000000 out of 10000000 steps (80%)
20:18:47:WU00:FS01:0xa4:Completed 8100000 out of 10000000 steps (81%)
20:22:37:WU00:FS01:0xa4:Completed 8200000 out of 10000000 steps (82%)
20:26:26:WU00:FS01:0xa4:Completed 8300000 out of 10000000 steps (83%)
20:30:12:WU00:FS01:0xa4:Completed 8400000 out of 10000000 steps (84%)
20:33:56:WU00:FS01:0xa4:Completed 8500000 out of 10000000 steps (85%)
20:37:41:WU00:FS01:0xa4:Completed 8600000 out of 10000000 steps (86%)
20:41:26:WU00:FS01:0xa4:Completed 8700000 out of 10000000 steps (87%)
20:45:11:WU00:FS01:0xa4:Completed 8800000 out of 10000000 steps (88%)
20:48:56:WU00:FS01:0xa4:Completed 8900000 out of 10000000 steps (89%)
20:52:41:WU00:FS01:0xa4:Completed 9000000 out of 10000000 steps (90%)
20:56:26:WU00:FS01:0xa4:Completed 9100000 out of 10000000 steps (91%)
21:00:12:WU00:FS01:0xa4:Completed 9200000 out of 10000000 steps (92%)
21:03:36:WU00:FS01:0xa4:Completed 9300000 out of 10000000 steps (93%)
21:07:21:WU00:FS01:0xa4:Completed 9400000 out of 10000000 steps (94%)
21:11:07:WU00:FS01:0xa4:Completed 9500000 out of 10000000 steps (95%)
21:14:52:WU00:FS01:0xa4:Completed 9600000 out of 10000000 steps (96%)
21:18:37:WU00:FS01:0xa4:Completed 9700000 out of 10000000 steps (97%)
21:22:22:WU00:FS01:0xa4:Completed 9800000 out of 10000000 steps (98%)
21:26:07:WU00:FS01:0xa4:Completed 9900000 out of 10000000 steps (99%)
21:29:52:WU00:FS01:0xa4:Completed 10000000 out of 10000000 steps (100%)
21:29:52:WU00:FS01:0xa4:DynamicWrapper: Finished Work Unit: sleep=10000
21:29:53:WU01:FS01:Connecting to assign3.stanford.edu:8080
21:29:53:WU01:FS01:News: Welcome to Folding@Home
21:29:53:WU01:FS01:Assigned to work server 129.74.85.15
21:29:53:WU01:FS01:Requesting new work unit for slot 01: RUNNING smp:8 from 129.74.85.15
21:29:53:WU01:FS01:Connecting to 129.74.85.15:8080
21:29:54:WU01:FS01:Downloading 131.60KiB
21:29:55:WU01:FS01:Download complete
21:29:55:WU01:FS01:Received Unit: id:01 state:DOWNLOAD error:OK project:7029 run:0 clone:217 gen:5 core:0xa4 unit:0x000000070001329c50294c56fc4920c8
21:29:55:WU01:FS01:Downloading project 7029 description
21:29:55:WU01:FS01:Connecting to fah-web.stanford.edu:80
21:29:56:WU01:FS01:Project 7029 description downloaded successfully
21:30:02:WU00:FS01:0xa4:
21:30:02:WU00:FS01:0xa4:Finished Work Unit:
21:30:02:WU00:FS01:0xa4:- Reading up to 2179176 from "00/wudata_01.trr": Read 2179176
21:30:02:WU00:FS01:0xa4:trr file hash check passed.
21:30:02:WU00:FS01:0xa4:- Reading up to 226492 from "00/wudata_01.xtc": Read 226492
21:30:02:WU00:FS01:0xa4:xtc file hash check passed.
21:30:02:WU00:FS01:0xa4:edr file hash check passed.
21:30:02:WU00:FS01:0xa4:logfile size: 79152
21:30:02:WU00:FS01:0xa4:Leaving Run
21:30:04:WU00:FS01:0xa4:- Writing 2509292 bytes of core data to disk...
21:30:05:WU00:FS01:0xa4:Done: 2508780 -> 1688462 (compressed to 67.3 percent)
21:30:05:WU00:FS01:0xa4: ... Done.
21:30:05:WU00:FS01:0xa4:- Shutting down core
21:30:05:WU00:FS01:0xa4:
21:30:05:WU00:FS01:0xa4:Folding@home Core Shutdown: FINISHED_UNIT
21:30:05:WU00:FS01:FahCore returned: FINISHED_UNIT (100 = 0x64)
21:30:05:WU00:FS01:Sending unit results: id:00 state:SEND error:OK project:7012 run:3 clone:19 gen:53 core:0xa4 unit:0x000000780001329c4dfb9924c5733abf
21:30:05:WU00:FS01:Uploading 1.61MiB to 129.74.85.15
21:30:05:WU01:FS01:Starting
21:30:05:WU00:FS01:Connecting to 129.74.85.15:8080
21:30:05:WU01:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/bollix/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/Core_a4.fah/FahCore_a4.exe -dir 01 -suffix 01 -version 701 -lifeline 2704 -checkpoint 30 -np 8
21:30:05:WU01:FS01:Started FahCore on PID 2296
21:30:05:WU01:FS01:Core PID:3636
21:30:05:WU01:FS01:FahCore 0xa4 started
21:30:06:WU01:FS01:0xa4:
21:30:06:WU01:FS01:0xa4:*------------------------------*
21:30:06:WU01:FS01:0xa4:Folding@Home Gromacs GB Core
21:30:06:WU01:FS01:0xa4:Version 2.27 (Dec. 15, 2010)
21:30:06:WU01:FS01:0xa4:
21:30:06:WU01:FS01:0xa4:Preparing to commence simulation
21:30:06:WU01:FS01:0xa4:- Looking at optimizations...
21:30:06:WU01:FS01:0xa4:- Created dyn
21:30:06:WU01:FS01:0xa4:- Files status OK
21:30:06:WU01:FS01:0xa4:- Expanded 134243 -> 306440 (decompressed 228.2 percent)
21:30:06:WU01:FS01:0xa4:Called DecompressByteArray: compressed_data_size=134243 data_size=306440, decompressed_data_size=306440 diff=0
21:30:06:WU01:FS01:0xa4:- Digital signature verified
21:30:06:WU01:FS01:0xa4:
21:30:06:WU01:FS01:0xa4:Project: 7029 (Run 0, Clone 217, Gen 5)
21:30:06:WU01:FS01:0xa4:
21:30:06:WU01:FS01:0xa4:Assembly optimizations on if available.
21:30:06:WU01:FS01:0xa4:Entering M.D.
21:30:11:WU00:FS01:Upload 38.80%
21:30:12:WU01:FS01:0xa4:Mapping NT from 8 to 8
21:30:12:WU01:FS01:0xa4:Completed 0 out of 25000000 steps (0%)
21:30:17:WU00:FS01:Upload 81.48%
21:30:20:WU00:FS01:Upload complete
21:30:20:WU00:FS01:Server responded WORK_ACK (400)
21:30:20:WU00:FS01:Final credit estimate, 3352.00 points
Code: Select all
Project ID: 7012
Core: GRO-A4
Credit: 600
Frames: 100
Number of Frames Observed: 100
Avg. Time / Frame : 00:03:47 - 12,797 PPD
Days taken to complete WU: 0.29
System specs:
i7 920 @ 2.66Ghz
4 gig
GTX 460
Windows 7 Pro 64 bit
v7 - SMP:8 + GPU
Re: Project: 7012 (Run 2, Clone 84, Gen 99) - High TPF
Posted: Sun Nov 04, 2012 2:50 am
by bollix47
One other user ran the same WU as Fahrenheit451 and assuming it ran on one of the systems in his profile he had the same slowness:
Days taken to complete WU: 0.46
Hi xxxxxxx (team xxxxx),
Your WU (P7012 R2 C84 G99) was added to the stats database on 2012-11-03 11:08:01 for 2909.28 points of credit.
Another project is having similar problems:
viewtopic.php?p=227675#p227675
Could this slowdown be related to the same a4 bug?
Re: Project: 7012 (Run 2, Clone 84, Gen 99) - High TPF
Posted: Sun Nov 04, 2012 4:24 am
by P5-133XL
I think that there is a real problem with automatically assuming high TPF's are all caused by this bug. The issue is that there are legitimate causes for this behavior that are not related to the bug. It means that they do not go through the diagnostic process to see if it is the bug or some external application using up CPU time. Since diagnosing a problem takes time and effort while dumping is quick and easy there are going to be people dumping where they shouldn't be. I do not know how to distinguish between bug and not without going through the diagnostic process.
That being said, if the diagnostic process has determined a specific WU has the bug by elimination of the alternative causes, it is likely that the WU is outright bad and should be marked in the database so it does not get reissued.
Re: Project: 7012 (Run 2, Clone 84, Gen 99) - High TPF
Posted: Sun Nov 04, 2012 5:26 am
by bruce
Both Kasson and tjlane have seen WUs in their project which were completed after running with a high TPF and determined that the result was corrupt. They have examples demonstrating that there is a bug in one or more of the SMP cores and the core developer is aware of the issue but it has not yet been fixed.
It's not appropriate to assume that this bug is the ONLY reason someone might have a WU that has a high TPF. That's one reason why I asked if the step count changed. It could help them to understand if it's the same problem or a different one.
Re: Project: 7012 (Run 2, Clone 84, Gen 99) - High TPF
Posted: Sun Nov 04, 2012 8:27 am
by Fahrenheit451
Thank you to all for your replies. If I understand aright, we can only wait for a updated A4 core.
I hope Stanford can use the results of this WU and it was not for nothing.
@bollix47: My folding name is superduper4711. Fahrenheit451 is only my forum name.
Here is the rest of the log file for this WU. Perhaps it helps the developers.
Code: Select all
00:34:27:WU01:FS00:0xa4:Completed 5900000 out of 10000000 steps (59%)
00:42:52:WU01:FS00:0xa4:Completed 6000000 out of 10000000 steps (60%)
00:52:49:WU01:FS00:0xa4:Completed 6100000 out of 10000000 steps (61%)
01:01:46:WU01:FS00:0xa4:Completed 6200000 out of 10000000 steps (62%)
01:10:42:WU01:FS00:0xa4:Completed 6300000 out of 10000000 steps (63%)
01:20:36:WU01:FS00:0xa4:Completed 6400000 out of 10000000 steps (64%)
01:29:54:WU01:FS00:0xa4:Completed 6500000 out of 10000000 steps (65%)
01:39:11:WU01:FS00:0xa4:Completed 6600000 out of 10000000 steps (66%)
01:48:26:WU01:FS00:0xa4:Completed 6700000 out of 10000000 steps (67%)
01:57:22:WU01:FS00:0xa4:Completed 6800000 out of 10000000 steps (68%)
02:06:16:WU01:FS00:0xa4:Completed 6900000 out of 10000000 steps (69%)
02:15:11:WU01:FS00:0xa4:Completed 7000000 out of 10000000 steps (70%)
02:24:07:WU01:FS00:0xa4:Completed 7100000 out of 10000000 steps (71%)
02:32:24:WU01:FS00:0xa4:Completed 7200000 out of 10000000 steps (72%)
02:41:19:WU01:FS00:0xa4:Completed 7300000 out of 10000000 steps (73%)
02:50:34:WU01:FS00:0xa4:Completed 7400000 out of 10000000 steps (74%)
02:59:29:WU01:FS00:0xa4:Completed 7500000 out of 10000000 steps (75%)
03:08:23:WU01:FS00:0xa4:Completed 7600000 out of 10000000 steps (76%)
03:17:18:WU01:FS00:0xa4:Completed 7700000 out of 10000000 steps (77%)
03:26:14:WU01:FS00:0xa4:Completed 7800000 out of 10000000 steps (78%)
03:35:09:WU01:FS00:0xa4:Completed 7900000 out of 10000000 steps (79%)
******************************** Date: 04/11/12 ********************************
03:43:48:WU01:FS00:0xa4:Completed 8000000 out of 10000000 steps (80%)
03:53:04:WU01:FS00:0xa4:Completed 8100000 out of 10000000 steps (81%)
04:02:20:WU01:FS00:0xa4:Completed 8200000 out of 10000000 steps (82%)
04:11:36:WU01:FS00:0xa4:Completed 8300000 out of 10000000 steps (83%)
04:20:49:WU01:FS00:0xa4:Completed 8400000 out of 10000000 steps (84%)
04:29:41:WU01:FS00:0xa4:Completed 8500000 out of 10000000 steps (85%)
04:38:32:WU01:FS00:0xa4:Completed 8600000 out of 10000000 steps (86%)
04:47:25:WU01:FS00:0xa4:Completed 8700000 out of 10000000 steps (87%)
04:56:18:WU01:FS00:0xa4:Completed 8800000 out of 10000000 steps (88%)
05:05:11:WU01:FS00:0xa4:Completed 8900000 out of 10000000 steps (89%)
05:13:43:WU01:FS00:0xa4:Completed 9000000 out of 10000000 steps (90%)
05:22:35:WU01:FS00:0xa4:Completed 9100000 out of 10000000 steps (91%)
05:31:49:WU01:FS00:0xa4:Completed 9200000 out of 10000000 steps (92%)
05:40:43:WU01:FS00:0xa4:Completed 9300000 out of 10000000 steps (93%)
05:49:36:WU01:FS00:0xa4:Completed 9400000 out of 10000000 steps (94%)
05:58:30:WU01:FS00:0xa4:Completed 9500000 out of 10000000 steps (95%)
06:07:47:WU01:FS00:0xa4:Completed 9600000 out of 10000000 steps (96%)
06:16:41:WU01:FS00:0xa4:Completed 9700000 out of 10000000 steps (97%)
06:25:58:WU01:FS00:0xa4:Completed 9800000 out of 10000000 steps (98%)
06:34:32:WU01:FS00:0xa4:Completed 9900000 out of 10000000 steps (99%)
06:34:33:WU03:FS00:Connecting to assign3.stanford.edu:8080
06:34:34:WU03:FS00:News: Welcome to Folding@Home
06:34:34:WU03:FS00:Assigned to work server 129.74.85.15
06:34:34:WU03:FS00:Requesting new work unit for slot 00: RUNNING smp:8 from 129.74.85.15
06:34:34:WU03:FS00:Connecting to 129.74.85.15:8080
06:34:35:WU03:FS00:Downloading 54.10KiB
06:34:36:WU03:FS00:Download complete
06:34:36:WU03:FS00:Received Unit: id:03 state:DOWNLOAD error:OK project:7001 run:0 clone:153 gen:74 core:0xa4 unit:0x000000ae0001329c4dfb86daad9a0a7b
06:34:36:WU03:FS00:Downloading project 7001 description
06:34:36:WU03:FS00:Connecting to fah-web.stanford.edu:80
06:34:38:WU03:FS00:Project 7001 description downloaded successfully
06:42:55:WU01:FS00:0xa4:Completed 10000000 out of 10000000 steps (100%)
06:42:55:WU01:FS00:0xa4:DynamicWrapper: Finished Work Unit: sleep=10000
06:43:05:WU01:FS00:0xa4:
06:43:05:WU01:FS00:0xa4:Finished Work Unit:
06:43:05:WU01:FS00:0xa4:- Reading up to 2179176 from "01/wudata_01.trr": Read 2179176
06:43:05:WU01:FS00:0xa4:trr file hash check passed.
06:43:05:WU01:FS00:0xa4:- Reading up to 121604 from "01/wudata_01.xtc": Read 121604
06:43:05:WU01:FS00:0xa4:xtc file hash check passed.
06:43:05:WU01:FS00:0xa4:edr file hash check passed.
06:43:05:WU01:FS00:0xa4:logfile size: 81567
06:43:05:WU01:FS00:0xa4:Leaving Run
06:43:08:WU01:FS00:0xa4:- Writing 2406819 bytes of core data to disk...
06:43:09:WU01:FS00:0xa4:Done: 2406307 -> 12405 (compressed to 0.5 percent)
06:43:09:WU01:FS00:0xa4: ... Done.
06:43:09:WU01:FS00:0xa4:- Shutting down core
06:43:09:WU01:FS00:0xa4:
06:43:09:WU01:FS00:0xa4:Folding@home Core Shutdown: FINISHED_UNIT
06:43:09:WU01:FS00:FahCore returned: FINISHED_UNIT (100 = 0x64)
06:43:09:WU01:FS00:Sending unit results: id:01 state:SEND error:OK project:7012 run:2 clone:84 gen:99 core:0xa4 unit:0x0000015e0001329c4dfb98cb6af56a7e
06:43:09:WU01:FS00:Uploading 12.61KiB to 129.74.85.15
06:43:09:WU01:FS00:Connecting to 129.74.85.15:8080
06:43:09:WU03:FS00:Starting
06:43:09:WU03:FS00:Running FahCore: D:\FAHClient_V7/FAHCoreWrapper.exe C:/ProgramData/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/Core_a4.fah/FahCore_a4.exe -dir 03 -suffix 01 -version 701 -lifeline 5860 -checkpoint 15 -np 8
06:43:09:WU03:FS00:Started FahCore on PID 5940
06:43:09:WU03:FS00:Core PID:4508
06:43:09:WU03:FS00:FahCore 0xa4 started
06:43:10:WU03:FS00:0xa4:
06:43:10:WU03:FS00:0xa4:*------------------------------*
06:43:10:WU03:FS00:0xa4:Folding@Home Gromacs GB Core
06:43:10:WU03:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
06:43:10:WU03:FS00:0xa4:
06:43:10:WU03:FS00:0xa4:Preparing to commence simulation
06:43:10:WU03:FS00:0xa4:- Looking at optimizations...
06:43:10:WU03:FS00:0xa4:- Created dyn
06:43:10:WU03:FS00:0xa4:- Files status OK
06:43:10:WU03:FS00:0xa4:- Expanded 54886 -> 204784 (decompressed 373.1 percent)
06:43:10:WU03:FS00:0xa4:Called DecompressByteArray: compressed_data_size=54886 data_size=204784, decompressed_data_size=204784 diff=0
06:43:10:WU03:FS00:0xa4:- Digital signature verified
06:43:10:WU03:FS00:0xa4:
06:43:10:WU03:FS00:0xa4:Project: 7001 (Run 0, Clone 153, Gen 74)
06:43:10:WU03:FS00:0xa4:
06:43:10:WU03:FS00:0xa4:Assembly optimizations on if available.
06:43:10:WU03:FS00:0xa4:Entering M.D.
06:43:10:WU01:FS00:Upload complete
06:43:10:WU01:FS00:Server responded WORK_ACK (400)
06:43:10:WU01:FS00:Final credit estimate, 2203.00 points
06:43:10:WU01:FS00:Cleaning up
Re: Project: 7012 (Run 2, Clone 84, Gen 99) - High TPF
Posted: Sun Nov 04, 2012 9:40 am
by bollix47
@bollix47: My folding name is superduper4711. Fahrenheit451 is only my forum name.
Just letting you know that the other folder mentioned above was not you. Your entry is now in the database as well as a 3rd one that took 2.46 days to complete. I have no way of knowing that person's computer setup or whether they run 24/7.
According to the database your run took .63 days.
The reissuing of the WU might be another indication of whether or not the results were corrupt but that can't be assumed as has been explained above.
Re: Project: 7012 (Run 2, Clone 84, Gen 99) - High TPF
Posted: Sun Nov 04, 2012 6:09 pm
by chaosdsm
bruce wrote:Normally nobody really pays any attention to the number of steps because different projects do have different numbers. In the past, one significant sign of the problem I think we're seeing is that for a single Project, moving to a different Run/Clone/Gen will find a different number of steps. Can anybody determine if that's happening?
This is the same problem I've been experiencing
p7007 &
p7005 I've looked through my old logs & as best I can tell, the high TPF wu's are the same as the regular wu's except of course for the much higher than normal TPF. The only non-OS programs running besides F@H (GPU & SMP) when this problem occurs is monitoring software, HFM to track work unit data, & Core Temp to track maximum CPU temps. All unneccessary services are dissabled.
If it would be of any help, I can post complete log files of the two 7005's that completed normally on my PC plus the one that ran high TPF, as well as the two 7007's that completed normally plus the one that ran high TPF.
Re: Project: 7012 (Run 2, Clone 84, Gen 99) - High TPF
Posted: Sun Nov 04, 2012 6:52 pm
by chaosdsm
P5-133XL wrote:I think that there is a real problem with automatically assuming high TPF's are all caused by this bug. The issue is that there are legitimate causes for this behavior that are not related to the bug. It means that they do not go through the diagnostic process to see if it is the bug or some external application using up CPU time. Since diagnosing a problem takes time and effort while dumping is quick and easy there are going to be people dumping where they shouldn't be. I do not know how to distinguish between bug and not without going through the diagnostic process.
That being said, if the diagnostic process has determined a specific WU has the bug by elimination of the alternative causes, it is likely that the WU is outright bad and should be marked in the database so it does not get reissued.
Actually, diagnosis is pretty simple. Open Task Manager (if you don't know how to do that, you might want to reconsider whether folding is for you...), take a minute to set it up properly by:
> clicking the 'view' menu
> then clicking "Select columns..."
> select the "CPU Time" option from the pop-up window, & click 'okay'.
Task Manager is now setup properly, so click the process tab, be sure to click the "show processes from all users" option, then click the 'CPU Time' header to sort all processes by CPU Time. Anything other than your wu core taking a significant amount of time (more than a minute or two) should be investigated, i.e. Google search, if you don't already know what it is & why it's running.
example:
Nothing significant running to slow down my 7005 wu there, so it's probably the bug...
Also, the entire work unit will run at about the same speed (+/- a couple of seconds per frame) from start to finish, pretty much eliminating anything that's not taking up dozens of minutes of CPU time.