Project 10001, Run 533, Clone 1, Gen 8 - EUE - ERROR 0x0

djchandler · Post by **djchandler** » Fri Dec 25, 2009 7:18 am

10001 (Run 533, Clone 1, Gen 8)

This is new. First time for an EUE on my Mythbuntu 9.10 server. Got this same WU twice before getting re-assigned after 23 frames in both instances. In neither case was any other CPU intensive task being performed. Nearly 100% cpu usage allocated to Core B4.

This latest build (1747) of Core B4 not doing so well perhaps. Looks like maybe exactly the same thing happened twice. Any chance a different C++ compiler could be used, such as GNU C++ compiler (g++ 4.4.1) exclusively. Current Ubuntu distribution is up 4.4.1. ProtoMol Core B4 is shown as g++ 4.1, which may have bugs that have been fixed since. Compiling machine running out-of-date Linux kernel as well.

Code: Select all

[18:43:25] *********************** Log Started 24/Dec/2009 18:43:25 ***********************
[18:43:25] ************************** ProtoMol Folding@Home Core **************************
[18:43:25]   Version: 20
[18:43:25]      Type: 180
[18:43:25]      Core: ProtoMol
[18:43:25]   Website: http://folding.stanford.edu/
[18:43:25] Copyright: (c) 2009 Stanford University
[18:43:25]    Author: Joseph Coffland <joseph@cauldrondevelopment.com>
[18:43:25]      Args: -dir work/ -suffix 05 -checkpoint 15 -lifeline 2168 -version 602
[18:43:25] ************************************ Build *************************************
[18:43:25]      Date: Dec 19 2009
[18:43:25]      Time: 18:25:17
[18:43:25]  Revision: 1747
[18:43:25]  Compiler: Intel(R) C++ g++ 4.1 mode 1110
[18:43:25]   Options: -std=gnu++98 -mia32 -O3 -funroll-loops
[18:43:25]            -axSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 -restrict
[18:43:25]  Platform: Linux 2.4.27-3-386
[18:43:25]      Bits: 32
[18:43:25] ************************************ System ************************************
[18:43:25]        OS: Linux 2.6.31-16-generic i686
[18:43:25]       CPU: AMD Athlon(tm) 64 Processor 3400+
[18:43:25]    CPU ID: AuthenticAMD Family 15 Model 44 Stepping 0
[18:43:25]      CPUs: 1 Logical, 1 Physical
[18:43:25]    Memory: 1.97 GB
[18:43:25] ********************************************************************************
[18:43:25] Project: 10001 (Run 533, Clone 1, Gen 8)
[18:43:25] Reading tar file par_all27_prot_lipid.inp
[18:43:25] Reading tar file scpismQuartic.inp
[18:43:25] Reading tar file ww_exteq_nowater1.pdb
[18:43:25] Reading tar file ww_exteq_nowater1.psf
[18:43:25] Reading tar file checkpt
[18:43:25] Reading tar file ww_exteq_nowater1.189.pos
[18:43:25] Reading tar file ww_exteq_nowater1.189.vel
[18:43:25] Reading tar file protomol.conf
[18:43:25] Reading tar file core.xml
[18:43:25] Digital signatures verified
[18:43:25] Completed 0 out of 200000 steps (0%)
[18:47:23] Completed 2000 out of 200000 steps (1%)
[18:51:26] Completed 4000 out of 200000 steps (2%)
[18:55:30] Completed 6000 out of 200000 steps (3%)
[18:59:21] Completed 8000 out of 200000 steps (4%)
[19:03:38] Completed 10000 out of 200000 steps (5%)
[19:07:42] Completed 12000 out of 200000 steps (6%)
[19:11:36] Completed 14000 out of 200000 steps (7%)
[19:15:30] Completed 16000 out of 200000 steps (8%)
[19:19:36] Completed 18000 out of 200000 steps (9%)
[19:23:33] Completed 20000 out of 200000 steps (10%)
[19:27:30] Completed 22000 out of 200000 steps (11%)
[19:31:25] Completed 24000 out of 200000 steps (12%)
[19:35:18] Completed 26000 out of 200000 steps (13%)
[19:39:24] Completed 28000 out of 200000 steps (14%)
[19:43:42] Completed 30000 out of 200000 steps (15%)
[19:47:53] Completed 32000 out of 200000 steps (16%)
[19:51:55] Completed 34000 out of 200000 steps (17%)
[19:56:10] Completed 36000 out of 200000 steps (18%)
[20:00:37] Completed 38000 out of 200000 steps (19%)
[20:04:38] Completed 40000 out of 200000 steps (20%)
[20:08:31] Completed 42000 out of 200000 steps (21%)
[20:12:41] Completed 44000 out of 200000 steps (22%)
[20:16:55] Completed 46000 out of 200000 steps (23%)
[20:18:55] CoreStatus = 0 (0)
[20:18:55] Client-core communications error: ERROR 0x0
[20:18:55] Deleting current work unit & continuing...
<snip>...
[20:24:59] *********************** Log Started 24/Dec/2009 20:24:59 ***********************
[20:24:59] ************************** ProtoMol Folding@Home Core **************************
[20:24:59]   Version: 20
[20:24:59]      Type: 180
[20:24:59]      Core: ProtoMol
[20:24:59]   Website: http://folding.stanford.edu/
[20:24:59] Copyright: (c) 2009 Stanford University
[20:24:59]    Author: Joseph Coffland <joseph@cauldrondevelopment.com>
[20:24:59]      Args: -dir work/ -suffix 06 -checkpoint 15 -lifeline 2168 -version 602
[20:24:59] ************************************ Build *************************************
[20:24:59]      Date: Dec 19 2009
[20:24:59]      Time: 18:25:17
[20:24:59]  Revision: 1747
[20:24:59]  Compiler: Intel(R) C++ g++ 4.1 mode 1110
[20:24:59]   Options: -std=gnu++98 -mia32 -O3 -funroll-loops
[20:24:59]            -axSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 -restrict
[20:24:59]  Platform: Linux 2.4.27-3-386
[20:24:59]      Bits: 32
[20:24:59] ************************************ System ************************************
[20:24:59]        OS: Linux 2.6.31-16-generic i686
[20:24:59]       CPU: AMD Athlon(tm) 64 Processor 3400+
[20:24:59]    CPU ID: AuthenticAMD Family 15 Model 44 Stepping 0
[20:24:59]      CPUs: 1 Logical, 1 Physical
[20:24:59]    Memory: 1.97 GB
[20:24:59] ********************************************************************************
[20:24:59] Project: 10001 (Run 533, Clone 1, Gen 8)
[20:24:59] Reading tar file par_all27_prot_lipid.inp
[20:24:59] Reading tar file scpismQuartic.inp
[20:24:59] Reading tar file ww_exteq_nowater1.pdb
[20:24:59] Reading tar file ww_exteq_nowater1.psf
[20:24:59] Reading tar file checkpt
[20:24:59] Reading tar file ww_exteq_nowater1.189.pos
[20:24:59] Reading tar file ww_exteq_nowater1.189.vel
[20:24:59] Reading tar file protomol.conf
[20:24:59] Reading tar file core.xml
[20:24:59] Digital signatures verified
[20:25:00] Completed 0 out of 200000 steps (0%)
[20:28:57] Completed 2000 out of 200000 steps (1%)
[20:33:00] Completed 4000 out of 200000 steps (2%)
[20:37:04] Completed 6000 out of 200000 steps (3%)
[20:40:54] Completed 8000 out of 200000 steps (4%)
[20:45:12] Completed 10000 out of 200000 steps (5%)
[20:49:15] Completed 12000 out of 200000 steps (6%)
[20:53:09] Completed 14000 out of 200000 steps (7%)
[20:57:04] Completed 16000 out of 200000 steps (8%)
[21:01:09] Completed 18000 out of 200000 steps (9%)
[21:05:06] Completed 20000 out of 200000 steps (10%)
[21:09:02] Completed 22000 out of 200000 steps (11%)
[21:12:58] Completed 24000 out of 200000 steps (12%)
[21:16:50] Completed 26000 out of 200000 steps (13%)
[21:20:56] Completed 28000 out of 200000 steps (14%)
[21:25:14] Completed 30000 out of 200000 steps (15%)
[21:29:25] Completed 32000 out of 200000 steps (16%)
[21:33:26] Completed 34000 out of 200000 steps (17%)
[21:37:42] Completed 36000 out of 200000 steps (18%)
[21:42:09] Completed 38000 out of 200000 steps (19%)
[21:46:10] Completed 40000 out of 200000 steps (20%)
[21:50:02] Completed 42000 out of 200000 steps (21%)
[21:54:12] Completed 44000 out of 200000 steps (22%)
[21:58:26] Completed 46000 out of 200000 steps (23%)
[22:00:29] CoreStatus = 0 (0)
[22:00:29] Client-core communications error: ERROR 0x0
[22:00:29] Deleting current work unit & continuing...

Post by **toTOW** » Fri Dec 25, 2009 2:40 pm

Joe released v21 of the core ... it might help with this issue too.

djchandler · Post by **djchandler** » Sat Dec 26, 2009 2:00 am

New version (build 1748) halting at the same point, different error code (ERROR 0x8b) but still a client-core communication failure on this WU. Is it the WU itself?

How about it being a problem due to disparity in kernel versions? Maybe I need another library installed.

Code: Select all

[23:55:44] Decompressed FahCore_b4.exe (21549840 bytes) successfully
[23:55:44] + Core successfully engaged
[23:55:49] 
[23:55:49] + Processing work unit
[23:55:49] Core required: FahCore_b4.exe
[23:55:49] Core found.
[23:55:49] Working on Unit 08 [December 25 23:55:49]
[23:55:49] + Working ...
[23:55:49] *********************** Log Started 25/Dec/2009 23:55:49 ***********************
[23:55:49] ************************** ProtoMol Folding@Home Core **************************
[23:55:49]   Version: 21
[23:55:49]      Type: 180
[23:55:49]      Core: ProtoMol
[23:55:49]   Website: http://folding.stanford.edu/
[23:55:49] Copyright: (c) 2009 Stanford University
[23:55:49]    Author: Joseph Coffland <joseph@cauldrondevelopment.com>
[23:55:49]      Args: -dir work/ -suffix 08 -checkpoint 15 -lifeline 2168 -version 602
[23:55:49] ************************************ Build *************************************
[23:55:49]      Date: Dec 24 2009
[23:55:49]      Time: 06:37:42
[23:55:49]  Revision: 1748
[23:55:49]  Compiler: Intel(R) C++ g++ 4.1 mode 1110
[23:55:49]   Options: -std=gnu++98 -mia32 -O3 -funroll-loops
[23:55:49]            -axSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 -restrict
[23:55:49]  Platform: Linux 2.4.27-3-386
[23:55:49]      Bits: 32
[23:55:49] ************************************ System ************************************
[23:55:49]        OS: Linux 2.6.31-16-generic i686
[23:55:49]       CPU: AMD Athlon(tm) 64 Processor 3400+
[23:55:49]    CPU ID: AuthenticAMD Family 15 Model 44 Stepping 0
[23:55:49]      CPUs: 1 Logical, 1 Physical
[23:55:49]    Memory: 1.97 GB
[23:55:49] ********************************************************************************
[23:55:49] Project: 10001 (Run 533, Clone 1, Gen 8)
[23:55:49] Reading tar file par_all27_prot_lipid.inp
[23:55:49] Reading tar file scpismQuartic.inp
[23:55:49] Reading tar file ww_exteq_nowater1.pdb
[23:55:49] Reading tar file ww_exteq_nowater1.psf
[23:55:49] Reading tar file checkpt
[23:55:49] Reading tar file ww_exteq_nowater1.189.pos
[23:55:49] Reading tar file ww_exteq_nowater1.189.vel
[23:55:49] Reading tar file protomol.conf
[23:55:49] Reading tar file core.xml
[23:55:49] Digital signatures verified
[23:55:50] Completed 0 out of 200000 steps (0%)
[23:59:49] Completed 2000 out of 200000 steps (1%)
[00:03:54] Completed 4000 out of 200000 steps (2%)
[00:07:59] Completed 6000 out of 200000 steps (3%)
[00:11:52] Completed 8000 out of 200000 steps (4%)
[00:16:11] Completed 10000 out of 200000 steps (5%)
[00:20:16] Completed 12000 out of 200000 steps (6%)
[00:24:12] Completed 14000 out of 200000 steps (7%)
[00:28:08] Completed 16000 out of 200000 steps (8%)
[00:32:15] Completed 18000 out of 200000 steps (9%)
[00:36:13] Completed 20000 out of 200000 steps (10%)
[00:40:12] Completed 22000 out of 200000 steps (11%)
[00:44:08] Completed 24000 out of 200000 steps (12%)
[00:48:03] Completed 26000 out of 200000 steps (13%)
[00:52:10] Completed 28000 out of 200000 steps (14%)
[00:56:30] Completed 30000 out of 200000 steps (15%)
[01:00:42] Completed 32000 out of 200000 steps (16%)
[01:04:46] Completed 34000 out of 200000 steps (17%)
[01:09:04] Completed 36000 out of 200000 steps (18%)
[01:13:32] Completed 38000 out of 200000 steps (19%)
[01:17:35] Completed 40000 out of 200000 steps (20%)
[01:21:30] Completed 42000 out of 200000 steps (21%)
[01:25:42] Completed 44000 out of 200000 steps (22%)
[01:29:57] Completed 46000 out of 200000 steps (23%)
[01:32:04] CoreStatus = 8B (139)
[01:32:04] Client-core communications error: ERROR 0x8b
[01:32:04] Deleting current work unit & continuing...

djchandler · Post by **djchandler** » Sun Dec 27, 2009 2:30 am

Is sending me the same WU 10 times in a row resulting in EUE normal procedure? Thats 10 times in a row now with build 1748, not counting the two reported in the OP with build 1747. That's 12 x 94 minutes each to get 12 EUE results. Not to put too fine a point on it, thats almost 19 hours of a CPU running 100% for nil results. This last set of 10 was 17 hours, 30 minutes and 36 seconds including 4 separate downloads of build 1748 of FahCore_b4.exe. I see no signs of corruption on my end. Why all the repeated attempts? It's my understanding that 3 failed attempts gets a reassignment.

This is on my Mythbuntu 9.10 (32-bit) backend. Nothing else was running but F@H even though the backend was active. There were no recordings, EPG database updates or any post-recording processing going on during this time. This sytem is not giving me any other problems except for the frontend welcome routine hanging, which has not caused any cascading failures. Usually I don't run the frontend on this box. There is mysql and Apache server giving me intranet access to the frontend even if the frontend isn't locally resident in memory.

Can't five you the whole FAHlog excerpt in one post. Breaking it up into two additional posts to follow.

djchandler · Post by **djchandler** » Sun Dec 27, 2009 2:34 am

FAHlog part one covering 10 consecutive assignment period.

Code: Select all

[01:32:05] - Preparing to get new work unit...
[01:32:05] + Attempting to get work packet
[01:32:05] - Connecting to assignment server
[01:32:05] - Successful: assigned to (129.74.85.48).
[01:32:05] + News From Folding@Home: Welcome to Folding@Home
[01:32:06] Loaded queue successfully.
[01:32:07] + Closed connections
[01:32:12] 
[01:32:12] + Processing work unit
[01:32:12] Core required: FahCore_b4.exe
[01:32:12] Core found.
[01:32:12] Working on Unit 09 [December 26 01:32:12]
[01:32:12] + Working ...
[01:32:12] *********************** Log Started 26/Dec/2009 01:32:12 ***********************
[01:32:12] ************************** ProtoMol Folding@Home Core **************************
[01:32:12]   Version: 21
[01:32:12]      Type: 180
[01:32:12]      Core: ProtoMol
[01:32:12]   Website: http://folding.stanford.edu/
[01:32:12] Copyright: (c) 2009 Stanford University
[01:32:12]    Author: Joseph Coffland <joseph@cauldrondevelopment.com>
[01:32:12]      Args: -dir work/ -suffix 09 -checkpoint 15 -lifeline 2168 -version 602
[01:32:12] ************************************ Build *************************************
[01:32:12]      Date: Dec 24 2009
[01:32:12]      Time: 06:37:42
[01:32:12]  Revision: 1748
[01:32:12]  Compiler: Intel(R) C++ g++ 4.1 mode 1110
[01:32:12]   Options: -std=gnu++98 -mia32 -O3 -funroll-loops
[01:32:12]            -axSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 -restrict
[01:32:12]  Platform: Linux 2.4.27-3-386
[01:32:12]      Bits: 32
[01:32:12] ************************************ System ************************************
[01:32:12]        OS: Linux 2.6.31-16-generic i686
[01:32:12]       CPU: AMD Athlon(tm) 64 Processor 3400+
[01:32:12]    CPU ID: AuthenticAMD Family 15 Model 44 Stepping 0
[01:32:12]      CPUs: 1 Logical, 1 Physical
[01:32:12]    Memory: 1.97 GB
[01:32:12] ********************************************************************************
[01:32:12] Project: 10001 (Run 533, Clone 1, Gen 8)
[01:32:12] Reading tar file par_all27_prot_lipid.inp
[01:32:12] Reading tar file scpismQuartic.inp
[01:32:12] Reading tar file ww_exteq_nowater1.pdb
[01:32:12] Reading tar file ww_exteq_nowater1.psf
[01:32:12] Reading tar file checkpt
[01:32:12] Reading tar file ww_exteq_nowater1.189.pos
[01:32:12] Reading tar file ww_exteq_nowater1.189.vel
[01:32:12] Reading tar file protomol.conf
[01:32:12] Reading tar file core.xml
[01:32:12] Digital signatures verified
[01:32:12] Completed 0 out of 200000 steps (0%)
[01:36:11] Completed 2000 out of 200000 steps (1%)
[01:40:16] Completed 4000 out of 200000 steps (2%)
[01:44:22] Completed 6000 out of 200000 steps (3%)
[01:48:14] Completed 8000 out of 200000 steps (4%)
[01:52:33] Completed 10000 out of 200000 steps (5%)
[01:56:37] Completed 12000 out of 200000 steps (6%)
[02:00:33] Completed 14000 out of 200000 steps (7%)
[02:04:29] Completed 16000 out of 200000 steps (8%)
[02:08:36] Completed 18000 out of 200000 steps (9%)
[02:12:34] Completed 20000 out of 200000 steps (10%)
[02:16:33] Completed 22000 out of 200000 steps (11%)
[02:20:29] Completed 24000 out of 200000 steps (12%)
[02:24:24] Completed 26000 out of 200000 steps (13%)
[02:28:32] Completed 28000 out of 200000 steps (14%)
[02:32:59] Completed 30000 out of 200000 steps (15%)
[02:37:21] Completed 32000 out of 200000 steps (16%)
[02:41:32] Completed 34000 out of 200000 steps (17%)
[02:45:59] Completed 36000 out of 200000 steps (18%)
[02:50:36] Completed 38000 out of 200000 steps (19%)
[02:54:48] Completed 40000 out of 200000 steps (20%)
[02:58:50] Completed 42000 out of 200000 steps (21%)
[03:03:05] Completed 44000 out of 200000 steps (22%)
[03:07:20] Completed 46000 out of 200000 steps (23%)
[03:09:22] CoreStatus = 0 (0)
[03:09:22] Client-core communications error: ERROR 0x0
[03:09:22] Deleting current work unit & continuing...
[03:09:23] - Preparing to get new work unit...
[03:09:23] + Attempting to get work packet
[03:09:23] - Connecting to assignment server
[03:09:24] - Successful: assigned to (129.74.85.48).
[03:09:24] + News From Folding@Home: Welcome to Folding@Home
[03:09:24] Loaded queue successfully.
[03:09:25] + Closed connections
[03:09:30] 
[03:09:30] + Processing work unit
[03:09:30] Core required: FahCore_b4.exe
[03:09:30] Core found.
[03:09:30] Working on Unit 00 [December 26 03:09:30]
[03:09:30] + Working ...
[03:09:30] *********************** Log Started 26/Dec/2009 03:09:30 ***********************
[03:09:30] ************************** ProtoMol Folding@Home Core **************************
[03:09:30]   Version: 21
[03:09:30]      Type: 180
[03:09:30]      Core: ProtoMol
[03:09:30]   Website: http://folding.stanford.edu/
[03:09:30] Copyright: (c) 2009 Stanford University
[03:09:30]    Author: Joseph Coffland <joseph@cauldrondevelopment.com>
[03:09:30]      Args: -dir work/ -suffix 00 -checkpoint 15 -lifeline 2168 -version 602
[03:09:30] ************************************ Build *************************************
[03:09:30]      Date: Dec 24 2009
[03:09:30]      Time: 06:37:42
[03:09:30]  Revision: 1748
[03:09:30]  Compiler: Intel(R) C++ g++ 4.1 mode 1110
[03:09:30]   Options: -std=gnu++98 -mia32 -O3 -funroll-loops
[03:09:30]            -axSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 -restrict
[03:09:30]  Platform: Linux 2.4.27-3-386
[03:09:30]      Bits: 32
[03:09:30] ************************************ System ************************************
[03:09:30]        OS: Linux 2.6.31-16-generic i686
[03:09:30]       CPU: AMD Athlon(tm) 64 Processor 3400+
[03:09:30]    CPU ID: AuthenticAMD Family 15 Model 44 Stepping 0
[03:09:30]      CPUs: 1 Logical, 1 Physical
[03:09:30]    Memory: 1.97 GB
[03:09:30] ********************************************************************************
[03:09:30] Project: 10001 (Run 533, Clone 1, Gen 8)
[03:09:30] Reading tar file par_all27_prot_lipid.inp
[03:09:30] Reading tar file scpismQuartic.inp
[03:09:30] Reading tar file ww_exteq_nowater1.pdb
[03:09:30] Reading tar file ww_exteq_nowater1.psf
[03:09:30] Reading tar file checkpt
[03:09:30] Reading tar file ww_exteq_nowater1.189.pos
[03:09:30] Reading tar file ww_exteq_nowater1.189.vel
[03:09:30] Reading tar file protomol.conf
[03:09:30] Reading tar file core.xml
[03:09:30] Digital signatures verified
[03:09:30] Completed 0 out of 200000 steps (0%)
[03:13:31] Completed 2000 out of 200000 steps (1%)
[03:17:38] Completed 4000 out of 200000 steps (2%)
[03:21:44] Completed 6000 out of 200000 steps (3%)
[03:25:38] Completed 8000 out of 200000 steps (4%)
[03:29:58] Completed 10000 out of 200000 steps (5%)
[03:34:04] Completed 12000 out of 200000 steps (6%)
[03:38:02] Completed 14000 out of 200000 steps (7%)
[03:41:59] Completed 16000 out of 200000 steps (8%)
[03:46:07] Completed 18000 out of 200000 steps (9%)
[03:50:07] Completed 20000 out of 200000 steps (10%)
[03:54:07] Completed 22000 out of 200000 steps (11%)
[03:58:05] Completed 24000 out of 200000 steps (12%)
[04:02:00] Completed 26000 out of 200000 steps (13%)
[04:06:10] Completed 28000 out of 200000 steps (14%)
[04:10:30] Completed 30000 out of 200000 steps (15%)
[04:14:45] Completed 32000 out of 200000 steps (16%)
[04:18:49] Completed 34000 out of 200000 steps (17%)
[04:23:09] Completed 36000 out of 200000 steps (18%)
[04:27:40] Completed 38000 out of 200000 steps (19%)
[04:31:44] Completed 40000 out of 200000 steps (20%)
[04:35:39] Completed 42000 out of 200000 steps (21%)
[04:39:53] Completed 44000 out of 200000 steps (22%)
[04:44:09] Completed 46000 out of 200000 steps (23%)
[04:46:14] CoreStatus = 0 (0)
[04:46:14] Client-core communications error: ERROR 0x0
[04:46:14] - Attempting to download new core...
[04:46:14] + Downloading new core: FahCore_b4.exe
[04:46:15] + 10240 bytes downloaded
[04:46:15] + 20480 bytes downloaded
[04:46:15] + 30720 bytes downloaded
[04:46:15] + 40960 bytes downloaded
[04:46:15] + 51200 bytes downloaded
[04:46:15] + 61440 bytes downloaded
[04:46:15] + 71680 bytes downloaded
[04:46:15] + 81920 bytes downloaded
[04:46:15] + 92160 bytes downloaded
[04:46:15] + 102400 bytes downloaded
[04:46:15] + 112640 bytes downloaded
[04:46:15] + 122880 bytes downloaded
[04:46:15] + 133120 bytes downloaded
[04:46:15] + 143360 bytes downloaded
[04:46:15] + 153600 bytes downloaded
[04:46:15] + 163840 bytes downloaded
[04:46:15] + 174080 bytes downloaded
[04:46:15] + 184320 bytes downloaded
[04:46:16] + 194560 bytes downloaded
[04:46:16] + 204800 bytes downloaded
[04:46:16] + 215040 bytes downloaded
[04:46:16] + 225280 bytes downloaded
[04:46:16] + 235520 bytes downloaded
[04:46:16] + 245760 bytes downloaded
[04:46:16] + 256000 bytes downloaded
[04:46:16] + 266240 bytes downloaded
[04:46:16] + 276480 bytes downloaded
[04:46:16] + 286720 bytes downloaded
[04:46:16] + 296960 bytes downloaded
[04:46:16] + 307200 bytes downloaded
[04:46:16] + 317440 bytes downloaded
[04:46:16] + 327680 bytes downloaded
[04:46:16] + 337920 bytes downloaded
[04:46:16] + 348160 bytes downloaded
[04:46:16] + 358400 bytes downloaded
[04:46:16] + 368640 bytes downloaded
[04:46:16] + 378880 bytes downloaded
[04:46:16] + 389120 bytes downloaded
[04:46:16] + 399360 bytes downloaded
[04:46:16] + 409600 bytes downloaded
[04:46:16] + 419840 bytes downloaded
[04:46:16] + 430080 bytes downloaded
[04:46:16] + 440320 bytes downloaded
[04:46:16] + 450560 bytes downloaded
[04:46:16] + 460800 bytes downloaded
[04:46:16] + 471040 bytes downloaded
[04:46:16] + 481280 bytes downloaded
[04:46:16] + 491520 bytes downloaded
[04:46:16] + 501760 bytes downloaded
[04:46:16] + 512000 bytes downloaded
[04:46:16] + 522240 bytes downloaded
[04:46:16] + 532480 bytes downloaded
[04:46:16] + 542720 bytes downloaded
[04:46:16] + 552960 bytes downloaded
[04:46:16] + 563200 bytes downloaded
[04:46:16] + 573440 bytes downloaded
[04:46:16] + 583680 bytes downloaded
[04:46:16] + 593920 bytes downloaded
[04:46:16] + 604160 bytes downloaded
[04:46:16] + 614400 bytes downloaded
[04:46:16] + 624640 bytes downloaded
[04:46:16] + 634880 bytes downloaded
[04:46:16] + 645120 bytes downloaded
[04:46:16] + 655360 bytes downloaded
[04:46:16] + 665600 bytes downloaded
[04:46:16] + 675840 bytes downloaded
[04:46:16] + 686080 bytes downloaded
[04:46:16] + 696320 bytes downloaded
[04:46:16] + 706560 bytes downloaded
[04:46:16] + 716800 bytes downloaded
[04:46:16] + 727040 bytes downloaded
[04:46:16] + 737280 bytes downloaded
[04:46:16] + 747520 bytes downloaded
[04:46:16] + 757760 bytes downloaded
[04:46:16] + 768000 bytes downloaded
[04:46:16] + 778240 bytes downloaded
[04:46:16] + 788480 bytes downloaded
[04:46:16] + 798720 bytes downloaded
[04:46:16] + 808960 bytes downloaded
[04:46:16] + 819200 bytes downloaded
[04:46:16] + 829440 bytes downloaded
[04:46:17] + 839680 bytes downloaded
[04:46:17] + 849920 bytes downloaded
[04:46:17] + 860160 bytes downloaded
[04:46:17] + 870400 bytes downloaded
[04:46:17] + 880640 bytes downloaded
[04:46:17] + 890880 bytes downloaded
[04:46:17] + 901120 bytes downloaded
[04:46:17] + 911360 bytes downloaded
[04:46:17] + 921600 bytes downloaded
[04:46:17] + 931840 bytes downloaded
[04:46:17] + 942080 bytes downloaded
[04:46:17] + 952320 bytes downloaded
[04:46:17] + 962560 bytes downloaded
[04:46:17] + 972800 bytes downloaded
[04:46:17] + 983040 bytes downloaded
[04:46:17] + 993280 bytes downloaded
[04:46:17] + 1003520 bytes downloaded
[04:46:17] + 1013760 bytes downloaded
[04:46:17] + 1024000 bytes downloaded
[04:46:17] + 1034240 bytes downloaded
[04:46:17] + 1044480 bytes downloaded
[04:46:17] + 1054720 bytes downloaded
[04:46:17] + 1064960 bytes downloaded
[04:46:17] + 1075200 bytes downloaded
[04:46:17] + 1085440 bytes downloaded
[04:46:17] + 1095680 bytes downloaded
[04:46:17] + 1105920 bytes downloaded
[04:46:17] + 1116160 bytes downloaded
[04:46:17] + 1126400 bytes downloaded
[04:46:17] + 1136640 bytes downloaded
[04:46:17] + 1146880 bytes downloaded
[04:46:17] + 1157120 bytes downloaded
[04:46:17] + 1167360 bytes downloaded
[04:46:17] + 1177600 bytes downloaded
[04:46:17] + 1187840 bytes downloaded
[04:46:17] + 1198080 bytes downloaded
[04:46:17] + 1208320 bytes downloaded
[04:46:17] + 1218560 bytes downloaded
[04:46:17] + 1228800 bytes downloaded
[04:46:17] + 1239040 bytes downloaded
[04:46:17] + 1249280 bytes downloaded
[04:46:17] + 1259520 bytes downloaded
[04:46:17] + 1269760 bytes downloaded
[04:46:17] + 1280000 bytes downloaded
[04:46:17] + 1290240 bytes downloaded
[04:46:17] + 1300480 bytes downloaded
[04:46:17] + 1310720 bytes downloaded
[04:46:17] + 1320960 bytes downloaded
[04:46:17] + 1331200 bytes downloaded
[04:46:17] + 1341440 bytes downloaded
[04:46:17] + 1351680 bytes downloaded
[04:46:17] + 1361920 bytes downloaded
[04:46:17] + 1372160 bytes downloaded
[04:46:17] + 1382400 bytes downloaded
[04:46:17] + 1392640 bytes downloaded
[04:46:17] + 1402880 bytes downloaded
[04:46:17] + 1413120 bytes downloaded
[04:46:17] + 1423360 bytes downloaded
[04:46:17] + 1433600 bytes downloaded
[04:46:17] + 1443840 bytes downloaded
[04:46:17] + 1454080 bytes downloaded
[04:46:17] + 1464320 bytes downloaded
[04:46:17] + 1474560 bytes downloaded
[04:46:17] + 1484800 bytes downloaded
[04:46:17] + 1495040 bytes downloaded
[04:46:17] + 1505280 bytes downloaded
[04:46:17] + 1515520 bytes downloaded
[04:46:17] + 1525760 bytes downloaded
[04:46:17] + 1536000 bytes downloaded
[04:46:17] + 1546240 bytes downloaded
[04:46:18] + 1556480 bytes downloaded
[04:46:18] + 1566720 bytes downloaded
[04:46:18] + 1576960 bytes downloaded
[04:46:18] + 1587200 bytes downloaded
[04:46:18] + 1597440 bytes downloaded
[04:46:18] + 1607680 bytes downloaded
[04:46:18] + 1617920 bytes downloaded
[04:46:18] + 1628160 bytes downloaded
[04:46:18] + 1638400 bytes downloaded
[04:46:18] + 1648640 bytes downloaded
[04:46:18] + 1658880 bytes downloaded
[04:46:18] + 1669120 bytes downloaded
[04:46:18] + 1679360 bytes downloaded
[04:46:18] + 1689600 bytes downloaded
[04:46:18] + 1699840 bytes downloaded
[04:46:18] + 1710080 bytes downloaded
[04:46:18] + 1720320 bytes downloaded
[04:46:18] + 1730560 bytes downloaded
[04:46:18] + 1740800 bytes downloaded
[04:46:18] + 1751040 bytes downloaded
[04:46:18] + 1761280 bytes downloaded
[04:46:18] + 1771520 bytes downloaded
[04:46:18] + 1781760 bytes downloaded
[04:46:18] + 1792000 bytes downloaded
[04:46:18] + 1802240 bytes downloaded
[04:46:18] + 1812480 bytes downloaded
[04:46:18] + 1822720 bytes downloaded
[04:46:18] + 1832960 bytes downloaded
[04:46:18] + 1843200 bytes downloaded
[04:46:18] + 1853440 bytes downloaded
[04:46:18] + 1863680 bytes downloaded
[04:46:18] + 1873920 bytes downloaded
[04:46:18] + 1884160 bytes downloaded
[04:46:18] + 1894400 bytes downloaded
[04:46:18] + 1904640 bytes downloaded
[04:46:18] + 1914880 bytes downloaded
[04:46:18] + 1925120 bytes downloaded
[04:46:18] + 1935360 bytes downloaded
[04:46:18] + 1945600 bytes downloaded
[04:46:18] + 1955840 bytes downloaded
[04:46:18] + 1966080 bytes downloaded
[04:46:18] + 1976320 bytes downloaded
[04:46:18] + 1986560 bytes downloaded
[04:46:18] + 1996800 bytes downloaded
[04:46:18] + 2007040 bytes downloaded
[04:46:18] + 2017280 bytes downloaded
[04:46:18] + 2027520 bytes downloaded
[04:46:18] + 2037760 bytes downloaded
[04:46:18] + 2048000 bytes downloaded
[04:46:18] + 2058240 bytes downloaded
[04:46:18] + 2068480 bytes downloaded
[04:46:18] + 2078720 bytes downloaded
[04:46:18] + 2088960 bytes downloaded
[04:46:18] + 2099200 bytes downloaded
[04:46:18] + 2109440 bytes downloaded
[04:46:18] + 2119680 bytes downloaded
[04:46:18] + 2129920 bytes downloaded
[04:46:18] + 2140160 bytes downloaded
[04:46:18] + 2150400 bytes downloaded
[04:46:18] + 2160640 bytes downloaded
[04:46:18] + 2170880 bytes downloaded
[04:46:18] + 2181120 bytes downloaded
[04:46:18] + 2191360 bytes downloaded
[04:46:18] + 2201600 bytes downloaded
[04:46:18] + 2211840 bytes downloaded
[04:46:18] + 2222080 bytes downloaded
[04:46:18] + 2232320 bytes downloaded
[04:46:18] + 2242560 bytes downloaded
[04:46:18] + 2252800 bytes downloaded
[04:46:18] + 2263040 bytes downloaded
[04:46:19] + 2273280 bytes downloaded
[04:46:19] + 2283520 bytes downloaded
[04:46:19] + 2293760 bytes downloaded
[04:46:19] + 2304000 bytes downloaded
[04:46:19] + 2314240 bytes downloaded
[04:46:19] + 2324480 bytes downloaded
[04:46:19] + 2334720 bytes downloaded
[04:46:19] + 2344960 bytes downloaded
[04:46:19] + 2355200 bytes downloaded
[04:46:19] + 2365440 bytes downloaded
[04:46:19] + 2375680 bytes downloaded
[04:46:19] + 2385920 bytes downloaded
[04:46:19] + 2396160 bytes downloaded
[04:46:19] + 2406400 bytes downloaded
[04:46:19] + 2416640 bytes downloaded
[04:46:19] + 2426880 bytes downloaded
[04:46:19] + 2437120 bytes downloaded
[04:46:19] + 2447360 bytes downloaded
[04:46:19] + 2457600 bytes downloaded
[04:46:19] + 2467840 bytes downloaded
[04:46:19] + 2478080 bytes downloaded
[04:46:19] + 2488320 bytes downloaded
[04:46:19] + 2498560 bytes downloaded
[04:46:19] + 2508800 bytes downloaded
[04:46:19] + 2519040 bytes downloaded
[04:46:19] + 2529280 bytes downloaded
[04:46:19] + 2539520 bytes downloaded
[04:46:19] + 2549760 bytes downloaded
[04:46:19] + 2560000 bytes downloaded
[04:46:19] + 2570240 bytes downloaded
[04:46:19] + 2580480 bytes downloaded
[04:46:19] + 2590720 bytes downloaded
[04:46:19] + 2600960 bytes downloaded
[04:46:19] + 2611200 bytes downloaded
[04:46:19] + 2621440 bytes downloaded
[04:46:19] + 2631680 bytes downloaded
[04:46:19] + 2641920 bytes downloaded
[04:46:19] + 2652160 bytes downloaded
[04:46:19] + 2662400 bytes downloaded
[04:46:19] + 2672640 bytes downloaded
[04:46:19] + 2682880 bytes downloaded
[04:46:19] + 2693120 bytes downloaded
[04:46:19] + 2703360 bytes downloaded
[04:46:19] + 2713600 bytes downloaded
[04:46:19] + 2723840 bytes downloaded
[04:46:19] + 2734080 bytes downloaded
[04:46:19] + 2744320 bytes downloaded
[04:46:19] + 2754560 bytes downloaded
[04:46:19] + 2764800 bytes downloaded
[04:46:19] + 2775040 bytes downloaded
[04:46:19] + 2785280 bytes downloaded
[04:46:19] + 2795520 bytes downloaded
[04:46:19] + 2805760 bytes downloaded
[04:46:19] + 2816000 bytes downloaded
[04:46:19] + 2826240 bytes downloaded
[04:46:19] + 2836480 bytes downloaded
[04:46:19] + 2846720 bytes downloaded
[04:46:19] + 2856960 bytes downloaded
[04:46:19] + 2867200 bytes downloaded
[04:46:19] + 2877440 bytes downloaded
[04:46:19] + 2887680 bytes downloaded
[04:46:19] + 2897920 bytes downloaded
[04:46:19] + 2908160 bytes downloaded
[04:46:19] + 2918400 bytes downloaded
[04:46:19] + 2928640 bytes downloaded
[04:46:19] + 2938880 bytes downloaded
[04:46:19] + 2949120 bytes downloaded
[04:46:19] + 2959360 bytes downloaded
[04:46:19] + 2969600 bytes downloaded
[04:46:19] + 2979840 bytes downloaded
[04:46:20] + 2990080 bytes downloaded
[04:46:20] + 3000320 bytes downloaded
[04:46:20] + 3010560 bytes downloaded
[04:46:20] + 3020800 bytes downloaded
[04:46:20] + 3031040 bytes downloaded
[04:46:20] + 3041280 bytes downloaded
[04:46:20] + 3051520 bytes downloaded
[04:46:20] + 3061760 bytes downloaded
[04:46:20] + 3072000 bytes downloaded
[04:46:20] + 3082240 bytes downloaded
[04:46:20] + 3092480 bytes downloaded
[04:46:20] + 3102720 bytes downloaded
[04:46:20] + 3112960 bytes downloaded
[04:46:20] + 3123200 bytes downloaded
[04:46:20] + 3133440 bytes downloaded
[04:46:20] + 3143680 bytes downloaded
[04:46:20] + 3153920 bytes downloaded
[04:46:20] + 3164160 bytes downloaded
[04:46:20] + 3174400 bytes downloaded
[04:46:20] + 3184640 bytes downloaded
[04:46:20] + 3194880 bytes downloaded
[04:46:20] + 3205120 bytes downloaded
[04:46:20] + 3215360 bytes downloaded
[04:46:20] + 3225600 bytes downloaded
[04:46:20] + 3235840 bytes downloaded
[04:46:20] + 3246080 bytes downloaded
[04:46:20] + 3256320 bytes downloaded
[04:46:20] + 3266560 bytes downloaded
[04:46:20] + 3276800 bytes downloaded
[04:46:20] + 3287040 bytes downloaded
[04:46:20] + 3297280 bytes downloaded
[04:46:20] + 3307520 bytes downloaded
[04:46:20] + 3317760 bytes downloaded
[04:46:20] + 3328000 bytes downloaded
[04:46:20] + 3338240 bytes downloaded
[04:46:20] + 3348480 bytes downloaded
[04:46:20] + 3358720 bytes downloaded
[04:46:20] + 3368960 bytes downloaded
[04:46:20] + 3379200 bytes downloaded
[04:46:20] + 3389440 bytes downloaded
[04:46:20] + 3399680 bytes downloaded
[04:46:20] + 3409920 bytes downloaded
[04:46:20] + 3420160 bytes downloaded
[04:46:20] + 3430400 bytes downloaded
[04:46:20] + 3440640 bytes downloaded
[04:46:20] + 3450880 bytes downloaded
[04:46:20] + 3461120 bytes downloaded
[04:46:20] + 3471360 bytes downloaded
[04:46:20] + 3481600 bytes downloaded
[04:46:20] + 3491840 bytes downloaded
[04:46:20] + 3502080 bytes downloaded
[04:46:20] + 3512320 bytes downloaded
[04:46:20] + 3522560 bytes downloaded
[04:46:20] + 3532800 bytes downloaded
[04:46:20] + 3543040 bytes downloaded
[04:46:20] + 3553280 bytes downloaded
[04:46:20] + 3563520 bytes downloaded
[04:46:20] + 3573760 bytes downloaded
[04:46:20] + 3584000 bytes downloaded
[04:46:20] + 3594240 bytes downloaded
[04:46:20] + 3604480 bytes downloaded
[04:46:20] + 3614720 bytes downloaded
[04:46:20] + 3624960 bytes downloaded
[04:46:20] + 3635200 bytes downloaded
[04:46:20] + 3645440 bytes downloaded
[04:46:20] + 3655680 bytes downloaded
[04:46:20] + 3665920 bytes downloaded
[04:46:20] + 3676160 bytes downloaded
[04:46:20] + 3686400 bytes downloaded
[04:46:21] + 3696640 bytes downloaded
[04:46:21] + 3706880 bytes downloaded
[04:46:21] + 3717120 bytes downloaded
[04:46:21] + 3727360 bytes downloaded
[04:46:21] + 3737600 bytes downloaded
[04:46:21] + 3747840 bytes downloaded
[04:46:21] + 3758080 bytes downloaded
[04:46:21] + 3768320 bytes downloaded
[04:46:21] + 3778560 bytes downloaded
[04:46:21] + 3788800 bytes downloaded
[04:46:21] + 3799040 bytes downloaded
[04:46:21] + 3809280 bytes downloaded
[04:46:21] + 3819520 bytes downloaded
[04:46:21] + 3829760 bytes downloaded
[04:46:21] + 3840000 bytes downloaded
[04:46:21] + 3850240 bytes downloaded
[04:46:21] + 3860480 bytes downloaded
[04:46:21] + 3870720 bytes downloaded
[04:46:21] + 3880960 bytes downloaded
[04:46:21] + 3891200 bytes downloaded
[04:46:21] + 3901440 bytes downloaded
[04:46:21] + 3911680 bytes downloaded
[04:46:21] + 3921920 bytes downloaded
[04:46:21] + 3932160 bytes downloaded
[04:46:21] + 3942400 bytes downloaded
[04:46:21] + 3952640 bytes downloaded
[04:46:21] + 3962880 bytes downloaded
[04:46:21] + 3973120 bytes downloaded
[04:46:21] + 3983360 bytes downloaded
[04:46:21] + 3993600 bytes downloaded
[04:46:21] + 4003840 bytes downloaded
[04:46:21] + 4014080 bytes downloaded
[04:46:21] + 4024320 bytes downloaded
[04:46:21] + 4034560 bytes downloaded
[04:46:21] + 4044800 bytes downloaded
[04:46:21] + 4055040 bytes downloaded
[04:46:21] + 4065280 bytes downloaded
[04:46:21] + 4075520 bytes downloaded
[04:46:21] + 4085760 bytes downloaded
[04:46:21] + 4096000 bytes downloaded
[04:46:21] + 4106240 bytes downloaded
[04:46:21] + 4116480 bytes downloaded
[04:46:21] + 4126720 bytes downloaded
[04:46:21] + 4136960 bytes downloaded
[04:46:21] + 4147200 bytes downloaded
[04:46:21] + 4157440 bytes downloaded
[04:46:21] + 4167680 bytes downloaded
[04:46:21] + 4177920 bytes downloaded
[04:46:21] + 4188160 bytes downloaded
[04:46:21] + 4198400 bytes downloaded
[04:46:21] + 4208640 bytes downloaded
[04:46:21] + 4218880 bytes downloaded
[04:46:21] + 4229120 bytes downloaded
[04:46:21] + 4239360 bytes downloaded
[04:46:21] + 4249600 bytes downloaded
[04:46:21] + 4259840 bytes downloaded
[04:46:21] + 4270080 bytes downloaded
[04:46:21] + 4280320 bytes downloaded
[04:46:21] + 4290560 bytes downloaded
[04:46:21] + 4300800 bytes downloaded
[04:46:21] + 4311040 bytes downloaded
[04:46:21] + 4321280 bytes downloaded
[04:46:21] + 4331520 bytes downloaded
[04:46:21] + 4341760 bytes downloaded
[04:46:21] + 4352000 bytes downloaded
[04:46:21] + 4362240 bytes downloaded
[04:46:21] + 4372480 bytes downloaded
[04:46:21] + 4382720 bytes downloaded
[04:46:21] + 4392960 bytes downloaded
[04:46:21] + 4403200 bytes downloaded
[04:46:22] + 4413440 bytes downloaded
[04:46:22] + 4423680 bytes downloaded
[04:46:22] + 4433920 bytes downloaded
[04:46:22] + 4444160 bytes downloaded
[04:46:22] + 4454400 bytes downloaded
[04:46:22] + 4464640 bytes downloaded
[04:46:22] + 4474880 bytes downloaded
[04:46:22] + 4485120 bytes downloaded
[04:46:22] + 4495360 bytes downloaded
[04:46:22] + 4505600 bytes downloaded
[04:46:22] + 4515840 bytes downloaded
[04:46:22] + 4526080 bytes downloaded
[04:46:22] + 4536320 bytes downloaded
[04:46:22] + 4546560 bytes downloaded
[04:46:22] + 4556800 bytes downloaded
[04:46:22] + 4567040 bytes downloaded
[04:46:22] + 4577280 bytes downloaded
[04:46:22] + 4587520 bytes downloaded
[04:46:22] + 4597760 bytes downloaded
[04:46:22] + 4608000 bytes downloaded
[04:46:22] + 4618240 bytes downloaded
[04:46:22] + 4628480 bytes downloaded
[04:46:22] + 4638720 bytes downloaded
[04:46:22] + 4648960 bytes downloaded
[04:46:22] + 4659200 bytes downloaded
[04:46:22] + 4669440 bytes downloaded
[04:46:22] + 4679680 bytes downloaded
[04:46:22] + 4689920 bytes downloaded
[04:46:22] + 4700160 bytes downloaded
[04:46:22] + 4710400 bytes downloaded
[04:46:22] + 4720640 bytes downloaded
[04:46:22] + 4730880 bytes downloaded
[04:46:22] + 4741120 bytes downloaded
[04:46:22] + 4751360 bytes downloaded
[04:46:22] + 4761600 bytes downloaded
[04:46:22] + 4771840 bytes downloaded
[04:46:22] + 4782080 bytes downloaded
[04:46:22] + 4792320 bytes downloaded
[04:46:22] + 4802560 bytes downloaded
[04:46:22] + 4812800 bytes downloaded
[04:46:22] + 4823040 bytes downloaded
[04:46:22] + 4833280 bytes downloaded
[04:46:22] + 4843520 bytes downloaded
[04:46:22] + 4853760 bytes downloaded
[04:46:22] + 4864000 bytes downloaded
[04:46:22] + 4874240 bytes downloaded
[04:46:22] + 4884480 bytes downloaded
[04:46:22] + 4894720 bytes downloaded
[04:46:22] + 4904960 bytes downloaded
[04:46:22] + 4915200 bytes downloaded
[04:46:22] + 4925440 bytes downloaded
[04:46:22] + 4935680 bytes downloaded
[04:46:22] + 4945920 bytes downloaded
[04:46:22] + 4956160 bytes downloaded
[04:46:22] + 4966400 bytes downloaded
[04:46:22] + 4976640 bytes downloaded
[04:46:22] + 4986880 bytes downloaded
[04:46:22] + 4997120 bytes downloaded
[04:46:22] + 5007360 bytes downloaded
[04:46:22] + 5017600 bytes downloaded
[04:46:22] + 5027840 bytes downloaded
[04:46:22] + 5038080 bytes downloaded
[04:46:22] + 5048320 bytes downloaded
[04:46:22] + 5058560 bytes downloaded
[04:46:22] + 5068800 bytes downloaded
[04:46:22] + 5079040 bytes downloaded
[04:46:22] + 5089280 bytes downloaded
[04:46:22] + 5099520 bytes downloaded
[04:46:22] + 5109760 bytes downloaded
[04:46:22] + 5120000 bytes downloaded
[04:46:23] + 5130240 bytes downloaded
[04:46:23] + 5140480 bytes downloaded
[04:46:23] + 5150720 bytes downloaded
[04:46:23] + 5160960 bytes downloaded
[04:46:23] + 5171200 bytes downloaded
[04:46:23] + 5181440 bytes downloaded
[04:46:23] + 5191680 bytes downloaded
[04:46:23] + 5201920 bytes downloaded
[04:46:23] + 5212160 bytes downloaded
[04:46:23] + 5222400 bytes downloaded
[04:46:23] + 5232640 bytes downloaded
[04:46:23] + 5242880 bytes downloaded
[04:46:23] + 5253120 bytes downloaded
[04:46:23] + 5263360 bytes downloaded
[04:46:23] + 5273600 bytes downloaded
[04:46:23] + 5283840 bytes downloaded
[04:46:23] + 5294080 bytes downloaded
[04:46:23] + 5304320 bytes downloaded
[04:46:23] + 5314560 bytes downloaded
[04:46:23] + 5324800 bytes downloaded
[04:46:23] + 5335040 bytes downloaded
[04:46:23] + 5345280 bytes downloaded
[04:46:23] + 5355520 bytes downloaded
[04:46:23] + 5365760 bytes downloaded
[04:46:23] + 5376000 bytes downloaded
[04:46:23] + 5386240 bytes downloaded
[04:46:23] + 5396480 bytes downloaded
[04:46:23] + 5406720 bytes downloaded
[04:46:23] + 5416960 bytes downloaded
[04:46:23] + 5427200 bytes downloaded
[04:46:23] + 5437440 bytes downloaded
[04:46:23] + 5447680 bytes downloaded
[04:46:23] + 5457920 bytes downloaded
[04:46:23] + 5468160 bytes downloaded
[04:46:23] + 5478400 bytes downloaded
[04:46:23] + 5488640 bytes downloaded
[04:46:23] + 5498880 bytes downloaded
[04:46:23] + 5509120 bytes downloaded
[04:46:23] + 5519360 bytes downloaded
[04:46:23] + 5529600 bytes downloaded
[04:46:23] + 5539840 bytes downloaded
[04:46:23] + 5550080 bytes downloaded
[04:46:23] + 5560320 bytes downloaded
[04:46:23] + 5570560 bytes downloaded
[04:46:23] + 5580800 bytes downloaded
[04:46:23] + 5591040 bytes downloaded
[04:46:23] + 5601280 bytes downloaded
[04:46:23] + 5609500 bytes downloaded
[04:46:23] Verifying core Core_b4.fah...
[04:46:23] Signature is VALID
[04:46:23] 
[04:46:23] Trying to unzip core FahCore_b4.exe
[04:46:25] Decompressed FahCore_b4.exe (21549840 bytes) successfully
[04:46:25] + Core successfully engaged
[04:46:26] Deleting current work unit & continuing...
[04:46:26] - Preparing to get new work unit...
[04:46:26] + Attempting to get work packet
[04:46:26] - Connecting to assignment server
[04:46:26] - Successful: assigned to (129.74.85.48).
[04:46:26] + News From Folding@Home: Welcome to Folding@Home
[04:46:26] Loaded queue successfully.
[04:46:27] + Closed connections
[04:46:32] 
[04:46:32] + Processing work unit
[04:46:32] Core required: FahCore_b4.exe
[04:46:32] Core found.
[04:46:32] Working on Unit 01 [December 26 04:46:32]
[04:46:32] + Working ...
[04:46:32] *********************** Log Started 26/Dec/2009 04:46:32 ***********************
[04:46:32] ************************** ProtoMol Folding@Home Core **************************
[04:46:32]   Version: 21
[04:46:32]      Type: 180
[04:46:32]      Core: ProtoMol
[04:46:32]   Website: http://folding.stanford.edu/
[04:46:32] Copyright: (c) 2009 Stanford University
[04:46:32]    Author: Joseph Coffland <joseph@cauldrondevelopment.com>
[04:46:32]      Args: -dir work/ -suffix 01 -checkpoint 15 -lifeline 2168 -version 602
[04:46:32] ************************************ Build *************************************
[04:46:32]      Date: Dec 24 2009
[04:46:32]      Time: 06:37:42
[04:46:32]  Revision: 1748
[04:46:32]  Compiler: Intel(R) C++ g++ 4.1 mode 1110
[04:46:32]   Options: -std=gnu++98 -mia32 -O3 -funroll-loops
[04:46:32]            -axSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 -restrict
[04:46:32]  Platform: Linux 2.4.27-3-386
[04:46:32]      Bits: 32
[04:46:32] ************************************ System ************************************
[04:46:32]        OS: Linux 2.6.31-16-generic i686
[04:46:32]       CPU: AMD Athlon(tm) 64 Processor 3400+
[04:46:32]    CPU ID: AuthenticAMD Family 15 Model 44 Stepping 0
[04:46:32]      CPUs: 1 Logical, 1 Physical
[04:46:32]    Memory: 1.97 GB
[04:46:32] ********************************************************************************
[04:46:32] Project: 10001 (Run 533, Clone 1, Gen 8)
[04:46:32] Reading tar file par_all27_prot_lipid.inp
[04:46:32] Reading tar file scpismQuartic.inp
[04:46:32] Reading tar file ww_exteq_nowater1.pdb
[04:46:32] Reading tar file ww_exteq_nowater1.psf
[04:46:32] Reading tar file checkpt
[04:46:32] Reading tar file ww_exteq_nowater1.189.pos
[04:46:32] Reading tar file ww_exteq_nowater1.189.vel
[04:46:32] Reading tar file protomol.conf
[04:46:32] Reading tar file core.xml
[04:46:32] Digital signatures verified
[04:46:33] Completed 0 out of 200000 steps (0%)
[04:50:35] Completed 2000 out of 200000 steps (1%)
[04:54:45] Completed 4000 out of 200000 steps (2%)
[04:58:54] Completed 6000 out of 200000 steps (3%)
[05:02:57] Completed 8000 out of 200000 steps (4%)
[05:07:29] Completed 10000 out of 200000 steps (5%)
[05:11:46] Completed 12000 out of 200000 steps (6%)
[05:15:53] Completed 14000 out of 200000 steps (7%)
[05:20:00] Completed 16000 out of 200000 steps (8%)
[05:24:19] Completed 18000 out of 200000 steps (9%)
[05:28:29] Completed 20000 out of 200000 steps (10%)
[05:42:54] Completed 22000 out of 200000 steps (11%)
[05:46:55] Completed 24000 out of 200000 steps (12%)
[05:50:53] Completed 26000 out of 200000 steps (13%)
[05:55:05] Completed 28000 out of 200000 steps (14%)
[05:59:29] Completed 30000 out of 200000 steps (15%)
[06:03:46] Completed 32000 out of 200000 steps (16%)
[06:07:53] Completed 34000 out of 200000 steps (17%)
[06:12:15] Completed 36000 out of 200000 steps (18%)
[06:16:48] Completed 38000 out of 200000 steps (19%)
[06:20:55] Completed 40000 out of 200000 steps (20%)
[06:24:53] Completed 42000 out of 200000 steps (21%)
[06:29:10] Completed 44000 out of 200000 steps (22%)
[06:33:37] Completed 46000 out of 200000 steps (23%)
[06:35:45] CoreStatus = 0 (0)
[06:35:45] Client-core communications error: ERROR 0x0
[06:35:45] Deleting current work unit & continuing...
[06:35:47] - Preparing to get new work unit...
[06:35:47] + Attempting to get work packet
[06:35:47] - Connecting to assignment server
[06:35:47] - Successful: assigned to (129.74.85.48).
[06:35:47] + News From Folding@Home: Welcome to Folding@Home
[06:35:48] Loaded queue successfully.
[06:35:48] + Closed connections
[06:35:53] 
[06:35:53] + Processing work unit
[06:35:53] Core required: FahCore_b4.exe
[06:35:53] Core found.
[06:35:53] Working on Unit 02 [December 26 06:35:53]
[06:35:53] + Working ...

djchandler · Post by **djchandler** » Sun Dec 27, 2009 2:39 am

FAHlog part two covering 10 consecutive assignment period.

Code: Select all

[06:35:54] *********************** Log Started 26/Dec/2009 06:35:54 ***********************
[06:35:54] ************************** ProtoMol Folding@Home Core **************************
[06:35:54]   Version: 21
[06:35:54]      Type: 180
[06:35:54]      Core: ProtoMol
[06:35:54]   Website: http://folding.stanford.edu/
[06:35:54] Copyright: (c) 2009 Stanford University
[06:35:54]    Author: Joseph Coffland <joseph@cauldrondevelopment.com>
[06:35:54]      Args: -dir work/ -suffix 02 -checkpoint 15 -lifeline 2168 -version 602
[06:35:54] ************************************ Build *************************************
[06:35:54]      Date: Dec 24 2009
[06:35:54]      Time: 06:37:42
[06:35:54]  Revision: 1748
[06:35:54]  Compiler: Intel(R) C++ g++ 4.1 mode 1110
[06:35:54]   Options: -std=gnu++98 -mia32 -O3 -funroll-loops
[06:35:54]            -axSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 -restrict
[06:35:54]  Platform: Linux 2.4.27-3-386
[06:35:54]      Bits: 32
[06:35:54] ************************************ System ************************************
[06:35:54]        OS: Linux 2.6.31-16-generic i686
[06:35:54]       CPU: AMD Athlon(tm) 64 Processor 3400+
[06:35:54]    CPU ID: AuthenticAMD Family 15 Model 44 Stepping 0
[06:35:54]      CPUs: 1 Logical, 1 Physical
[06:35:54]    Memory: 1.97 GB
[06:35:54] ********************************************************************************
[06:35:54] Project: 10001 (Run 533, Clone 1, Gen 8)
[06:35:54] Reading tar file par_all27_prot_lipid.inp
[06:35:54] Reading tar file scpismQuartic.inp
[06:35:54] Reading tar file ww_exteq_nowater1.pdb
[06:35:54] Reading tar file ww_exteq_nowater1.psf
[06:35:54] Reading tar file checkpt
[06:35:54] Reading tar file ww_exteq_nowater1.189.pos
[06:35:54] Reading tar file ww_exteq_nowater1.189.vel
[06:35:54] Reading tar file protomol.conf
[06:35:54] Reading tar file core.xml
[06:35:54] Digital signatures verified
[06:35:54] Completed 0 out of 200000 steps (0%)
[06:40:00] Completed 2000 out of 200000 steps (1%)
[06:44:13] Completed 4000 out of 200000 steps (2%)
[06:48:26] Completed 6000 out of 200000 steps (3%)
[06:52:25] Completed 8000 out of 200000 steps (4%)
[06:56:53] Completed 10000 out of 200000 steps (5%)
[07:01:05] Completed 12000 out of 200000 steps (6%)
[07:16:05] Completed 14000 out of 200000 steps (7%)
[07:20:02] Completed 16000 out of 200000 steps (8%)
[07:24:09] Completed 18000 out of 200000 steps (9%)
[07:28:07] Completed 20000 out of 200000 steps (10%)
[07:32:05] Completed 22000 out of 200000 steps (11%)
[07:36:02] Completed 24000 out of 200000 steps (12%)
[07:39:56] Completed 26000 out of 200000 steps (13%)
[07:44:04] Completed 28000 out of 200000 steps (14%)
[07:48:23] Completed 30000 out of 200000 steps (15%)
[07:52:36] Completed 32000 out of 200000 steps (16%)
[07:56:39] Completed 34000 out of 200000 steps (17%)
[08:00:57] Completed 36000 out of 200000 steps (18%)
[08:05:25] Completed 38000 out of 200000 steps (19%)
[08:09:28] Completed 40000 out of 200000 steps (20%)
[08:13:22] Completed 42000 out of 200000 steps (21%)
[08:17:34] Completed 44000 out of 200000 steps (22%)
[08:21:49] Completed 46000 out of 200000 steps (23%)
[08:23:51] CoreStatus = 0 (0)
[08:23:51] Client-core communications error: ERROR 0x0
[08:23:51] Deleting current work unit & continuing...
[08:23:53] - Preparing to get new work unit...
[08:23:53] + Attempting to get work packet
[08:23:53] - Connecting to assignment server
[08:23:53] - Successful: assigned to (129.74.85.48).
[08:23:53] + News From Folding@Home: Welcome to Folding@Home
[08:23:53] Loaded queue successfully.
[08:23:55] + Closed connections
[08:24:00] 
[08:24:00] + Processing work unit
[08:24:00] Core required: FahCore_b4.exe
[08:24:00] Core found.
[08:24:00] Working on Unit 03 [December 26 08:24:00]
[08:24:00] + Working ...
[08:24:00] *********************** Log Started 26/Dec/2009 08:24:00 ***********************
[08:24:00] ************************** ProtoMol Folding@Home Core **************************
[08:24:00]   Version: 21
[08:24:00]      Type: 180
[08:24:00]      Core: ProtoMol
[08:24:00]   Website: http://folding.stanford.edu/
[08:24:00] Copyright: (c) 2009 Stanford University
[08:24:00]    Author: Joseph Coffland <joseph@cauldrondevelopment.com>
[08:24:00]      Args: -dir work/ -suffix 03 -checkpoint 15 -lifeline 2168 -version 602
[08:24:00] ************************************ Build *************************************
[08:24:00]      Date: Dec 24 2009
[08:24:00]      Time: 06:37:42
[08:24:00]  Revision: 1748
[08:24:00]  Compiler: Intel(R) C++ g++ 4.1 mode 1110
[08:24:00]   Options: -std=gnu++98 -mia32 -O3 -funroll-loops
[08:24:00]            -axSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 -restrict
[08:24:00]  Platform: Linux 2.4.27-3-386
[08:24:00]      Bits: 32
[08:24:00] ************************************ System ************************************
[08:24:00]        OS: Linux 2.6.31-16-generic i686
[08:24:00]       CPU: AMD Athlon(tm) 64 Processor 3400+
[08:24:00]    CPU ID: AuthenticAMD Family 15 Model 44 Stepping 0
[08:24:00]      CPUs: 1 Logical, 1 Physical
[08:24:00]    Memory: 1.97 GB
[08:24:00] ********************************************************************************
[08:24:00] Project: 10001 (Run 533, Clone 1, Gen 8)
[08:24:00] Reading tar file par_all27_prot_lipid.inp
[08:24:00] Reading tar file scpismQuartic.inp
[08:24:00] Reading tar file ww_exteq_nowater1.pdb
[08:24:00] Reading tar file ww_exteq_nowater1.psf
[08:24:00] Reading tar file checkpt
[08:24:00] Reading tar file ww_exteq_nowater1.189.pos
[08:24:00] Reading tar file ww_exteq_nowater1.189.vel
[08:24:00] Reading tar file protomol.conf
[08:24:00] Reading tar file core.xml
[08:24:00] Digital signatures verified
[08:24:01] Completed 0 out of 200000 steps (0%)
[08:27:59] Completed 2000 out of 200000 steps (1%)
[08:32:04] Completed 4000 out of 200000 steps (2%)
[08:36:09] Completed 6000 out of 200000 steps (3%)
[08:40:01] Completed 8000 out of 200000 steps (4%)
[08:44:20] Completed 10000 out of 200000 steps (5%)
[08:48:25] Completed 12000 out of 200000 steps (6%)
[08:52:20] Completed 14000 out of 200000 steps (7%)
[08:56:15] Completed 16000 out of 200000 steps (8%)
[09:00:22] Completed 18000 out of 200000 steps (9%)
[09:04:20] Completed 20000 out of 200000 steps (10%)
[09:08:19] Completed 22000 out of 200000 steps (11%)
[09:12:15] Completed 24000 out of 200000 steps (12%)
[09:16:09] Completed 26000 out of 200000 steps (13%)
[09:20:17] Completed 28000 out of 200000 steps (14%)
[09:24:36] Completed 30000 out of 200000 steps (15%)
[09:28:48] Completed 32000 out of 200000 steps (16%)
[09:32:51] Completed 34000 out of 200000 steps (17%)
[09:37:09] Completed 36000 out of 200000 steps (18%)
[09:41:37] Completed 38000 out of 200000 steps (19%)
[09:45:39] Completed 40000 out of 200000 steps (20%)
[09:49:33] Completed 42000 out of 200000 steps (21%)
[09:53:45] Completed 44000 out of 200000 steps (22%)
[09:58:00] Completed 46000 out of 200000 steps (23%)
[10:00:01] CoreStatus = 8B (139)
[10:00:01] Client-core communications error: ERROR 0x8b
[10:00:01] - Attempting to download new core...
[10:00:01] + Downloading new core: FahCore_b4.exe
[10:00:02] + 10240 bytes downloaded
[10:00:02] ...
[10:00:10] + 5609500 bytes downloaded
[10:00:10] Verifying core Core_b4.fah...
[10:00:10] Signature is VALID
[10:00:10] 
[10:00:10] Trying to unzip core FahCore_b4.exe
[10:00:11] Decompressed FahCore_b4.exe (21549840 bytes) successfully
[10:00:11] + Core successfully engaged
[10:00:12] Deleting current work unit & continuing...
[10:00:12] - Preparing to get new work unit...
[10:00:12] + Attempting to get work packet
[10:00:12] - Connecting to assignment server
[10:00:13] - Successful: assigned to (129.74.85.48).
[10:00:13] + News From Folding@Home: Welcome to Folding@Home
[10:00:13] Loaded queue successfully.
[10:00:14] + Closed connections
[10:00:19] 
[10:00:19] + Processing work unit
[10:00:19] Core required: FahCore_b4.exe
[10:00:19] Core found.
[10:00:19] Working on Unit 04 [December 26 10:00:19]
[10:00:19] + Working ...
[10:00:19] *********************** Log Started 26/Dec/2009 10:00:19 ***********************
[10:00:19] ************************** ProtoMol Folding@Home Core **************************
[10:00:19]   Version: 21
[10:00:19]      Type: 180
[10:00:19]      Core: ProtoMol
[10:00:19]   Website: http://folding.stanford.edu/
[10:00:19] Copyright: (c) 2009 Stanford University
[10:00:19]    Author: Joseph Coffland <joseph@cauldrondevelopment.com>
[10:00:19]      Args: -dir work/ -suffix 04 -checkpoint 15 -lifeline 2168 -version 602
[10:00:19] ************************************ Build *************************************
[10:00:19]      Date: Dec 24 2009
[10:00:19]      Time: 06:37:42
[10:00:19]  Revision: 1748
[10:00:19]  Compiler: Intel(R) C++ g++ 4.1 mode 1110
[10:00:19]   Options: -std=gnu++98 -mia32 -O3 -funroll-loops
[10:00:19]            -axSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 -restrict
[10:00:19]  Platform: Linux 2.4.27-3-386
[10:00:19]      Bits: 32
[10:00:19] ************************************ System ************************************
[10:00:19]        OS: Linux 2.6.31-16-generic i686
[10:00:19]       CPU: AMD Athlon(tm) 64 Processor 3400+
[10:00:19]    CPU ID: AuthenticAMD Family 15 Model 44 Stepping 0
[10:00:19]      CPUs: 1 Logical, 1 Physical
[10:00:19]    Memory: 1.97 GB
[10:00:19] ********************************************************************************
[10:00:19] Project: 10001 (Run 533, Clone 1, Gen 8)
[10:00:19] Reading tar file par_all27_prot_lipid.inp
[10:00:19] Reading tar file scpismQuartic.inp
[10:00:19] Reading tar file ww_exteq_nowater1.pdb
[10:00:19] Reading tar file ww_exteq_nowater1.psf
[10:00:19] Reading tar file checkpt
[10:00:19] Reading tar file ww_exteq_nowater1.189.pos
[10:00:19] Reading tar file ww_exteq_nowater1.189.vel
[10:00:19] Reading tar file protomol.conf
[10:00:19] Reading tar file core.xml
[10:00:19] Digital signatures verified
[10:00:20] Completed 0 out of 200000 steps (0%)
[10:04:18] Completed 2000 out of 200000 steps (1%)
[10:08:24] Completed 4000 out of 200000 steps (2%)
[10:12:29] Completed 6000 out of 200000 steps (3%)
[10:16:20] Completed 8000 out of 200000 steps (4%)
[10:20:40] Completed 10000 out of 200000 steps (5%)
[10:24:45] Completed 12000 out of 200000 steps (6%)
[10:28:40] Completed 14000 out of 200000 steps (7%)
[10:32:36] Completed 16000 out of 200000 steps (8%)
[10:36:43] Completed 18000 out of 200000 steps (9%)
[10:40:41] Completed 20000 out of 200000 steps (10%)
[10:44:39] Completed 22000 out of 200000 steps (11%)
[10:48:36] Completed 24000 out of 200000 steps (12%)
[10:52:30] Completed 26000 out of 200000 steps (13%)
[10:56:38] Completed 28000 out of 200000 steps (14%)
[11:00:56] Completed 30000 out of 200000 steps (15%)
[11:05:09] Completed 32000 out of 200000 steps (16%)
[11:09:12] Completed 34000 out of 200000 steps (17%)
[11:13:30] Completed 36000 out of 200000 steps (18%)
[11:18:01] Completed 38000 out of 200000 steps (19%)
[11:22:04] Completed 40000 out of 200000 steps (20%)
[11:25:58] Completed 42000 out of 200000 steps (21%)
[11:30:10] Completed 44000 out of 200000 steps (22%)
[11:34:25] Completed 46000 out of 200000 steps (23%)
[11:36:28] CoreStatus = 0 (0)
[11:36:28] Client-core communications error: ERROR 0x0
[11:36:28] Deleting current work unit & continuing...
[11:36:30] - Preparing to get new work unit...
[11:36:30] + Attempting to get work packet
[11:36:30] - Connecting to assignment server
[11:36:30] - Successful: assigned to (129.74.85.48).
[11:36:30] + News From Folding@Home: Welcome to Folding@Home
[11:36:30] Loaded queue successfully.
[11:36:31] + Closed connections
[11:36:36] 
[11:36:36] + Processing work unit
[11:36:36] Core required: FahCore_b4.exe
[11:36:36] Core found.
[11:36:36] Working on Unit 05 [December 26 11:36:36]
[11:36:36] + Working ...
[11:36:36] *********************** Log Started 26/Dec/2009 11:36:36 ***********************
[11:36:36] ************************** ProtoMol Folding@Home Core **************************
[11:36:36]   Version: 21
[11:36:36]      Type: 180
[11:36:36]      Core: ProtoMol
[11:36:36]   Website: http://folding.stanford.edu/
[11:36:36] Copyright: (c) 2009 Stanford University
[11:36:36]    Author: Joseph Coffland <joseph@cauldrondevelopment.com>
[11:36:36]      Args: -dir work/ -suffix 05 -checkpoint 15 -lifeline 2168 -version 602
[11:36:36] ************************************ Build *************************************
[11:36:36]      Date: Dec 24 2009
[11:36:36]      Time: 06:37:42
[11:36:36]  Revision: 1748
[11:36:36]  Compiler: Intel(R) C++ g++ 4.1 mode 1110
[11:36:36]   Options: -std=gnu++98 -mia32 -O3 -funroll-loops
[11:36:36]            -axSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 -restrict
[11:36:36]  Platform: Linux 2.4.27-3-386
[11:36:36]      Bits: 32
[11:36:36] ************************************ System ************************************
[11:36:36]        OS: Linux 2.6.31-16-generic i686
[11:36:36]       CPU: AMD Athlon(tm) 64 Processor 3400+
[11:36:36]    CPU ID: AuthenticAMD Family 15 Model 44 Stepping 0
[11:36:36]      CPUs: 1 Logical, 1 Physical
[11:36:36]    Memory: 1.97 GB
[11:36:36] ********************************************************************************
[11:36:36] Project: 10001 (Run 533, Clone 1, Gen 8)
[11:36:36] Reading tar file par_all27_prot_lipid.inp
[11:36:36] Reading tar file scpismQuartic.inp
[11:36:36] Reading tar file ww_exteq_nowater1.pdb
[11:36:36] Reading tar file ww_exteq_nowater1.psf
[11:36:36] Reading tar file checkpt
[11:36:36] Reading tar file ww_exteq_nowater1.189.pos
[11:36:36] Reading tar file ww_exteq_nowater1.189.vel
[11:36:36] Reading tar file protomol.conf
[11:36:36] Reading tar file core.xml
[11:36:36] Digital signatures verified
[11:36:37] Completed 0 out of 200000 steps (0%)
[11:40:35] Completed 2000 out of 200000 steps (1%)
[11:44:40] Completed 4000 out of 200000 steps (2%)
[11:48:46] Completed 6000 out of 200000 steps (3%)
[11:52:37] Completed 8000 out of 200000 steps (4%)
[11:56:56] Completed 10000 out of 200000 steps (5%)
[12:01:01] Completed 12000 out of 200000 steps (6%)
[12:04:56] Completed 14000 out of 200000 steps (7%)
[12:08:52] Completed 16000 out of 200000 steps (8%)
[12:12:58] Completed 18000 out of 200000 steps (9%)
[12:16:56] Completed 20000 out of 200000 steps (10%)
[12:20:55] Completed 22000 out of 200000 steps (11%)
[12:24:51] Completed 24000 out of 200000 steps (12%)
[12:28:46] Completed 26000 out of 200000 steps (13%)
[12:32:55] Completed 28000 out of 200000 steps (14%)
[12:37:38] Completed 30000 out of 200000 steps (15%)
[12:41:55] Completed 32000 out of 200000 steps (16%)
[12:45:58] Completed 34000 out of 200000 steps (17%)
[12:50:15] Completed 36000 out of 200000 steps (18%)
[12:54:43] Completed 38000 out of 200000 steps (19%)
[12:58:46] Completed 40000 out of 200000 steps (20%)
[13:02:39] Completed 42000 out of 200000 steps (21%)
[13:06:51] Completed 44000 out of 200000 steps (22%)
[13:11:06] Completed 46000 out of 200000 steps (23%)
[13:13:08] CoreStatus = 0 (0)
[13:13:08] Client-core communications error: ERROR 0x0
[13:13:08] Deleting current work unit & continuing...
[13:13:10] - Preparing to get new work unit...
[13:13:10] + Attempting to get work packet
[13:13:10] - Connecting to assignment server
[13:13:10] - Successful: assigned to (129.74.85.48).
[13:13:10] + News From Folding@Home: Welcome to Folding@Home
[13:13:10] Loaded queue successfully.
[13:13:11] + Closed connections
[13:13:16] 
[13:13:16] + Processing work unit
[13:13:16] Core required: FahCore_b4.exe
[13:13:16] Core found.
[13:13:16] Working on Unit 06 [December 26 13:13:16]
[13:13:16] + Working ...
[13:13:16] *********************** Log Started 26/Dec/2009 13:13:16 ***********************
[13:13:16] ************************** ProtoMol Folding@Home Core **************************
[13:13:16]   Version: 21
[13:13:16]      Type: 180
[13:13:16]      Core: ProtoMol
[13:13:16]   Website: http://folding.stanford.edu/
[13:13:16] Copyright: (c) 2009 Stanford University
[13:13:16]    Author: Joseph Coffland <joseph@cauldrondevelopment.com>
[13:13:16]      Args: -dir work/ -suffix 06 -checkpoint 15 -lifeline 2168 -version 602
[13:13:16] ************************************ Build *************************************
[13:13:16]      Date: Dec 24 2009
[13:13:16]      Time: 06:37:42
[13:13:16]  Revision: 1748
[13:13:16]  Compiler: Intel(R) C++ g++ 4.1 mode 1110
[13:13:16]   Options: -std=gnu++98 -mia32 -O3 -funroll-loops
[13:13:16]            -axSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 -restrict
[13:13:16]  Platform: Linux 2.4.27-3-386
[13:13:16]      Bits: 32
[13:13:16] ************************************ System ************************************
[13:13:16]        OS: Linux 2.6.31-16-generic i686
[13:13:16]       CPU: AMD Athlon(tm) 64 Processor 3400+
[13:13:16]    CPU ID: AuthenticAMD Family 15 Model 44 Stepping 0
[13:13:16]      CPUs: 1 Logical, 1 Physical
[13:13:16]    Memory: 1.97 GB
[13:13:16] ********************************************************************************
[13:13:16] Project: 10001 (Run 533, Clone 1, Gen 8)
[13:13:16] Reading tar file par_all27_prot_lipid.inp
[13:13:16] Reading tar file scpismQuartic.inp
[13:13:16] Reading tar file ww_exteq_nowater1.pdb
[13:13:16] Reading tar file ww_exteq_nowater1.psf
[13:13:16] Reading tar file checkpt
[13:13:16] Reading tar file ww_exteq_nowater1.189.pos
[13:13:16] Reading tar file ww_exteq_nowater1.189.vel
[13:13:16] Reading tar file protomol.conf
[13:13:16] Reading tar file core.xml
[13:13:16] Digital signatures verified
[13:13:16] Completed 0 out of 200000 steps (0%)
[13:17:16] Completed 2000 out of 200000 steps (1%)
[13:21:22] Completed 4000 out of 200000 steps (2%)
[13:25:28] Completed 6000 out of 200000 steps (3%)
[13:29:20] Completed 8000 out of 200000 steps (4%)
[13:33:40] Completed 10000 out of 200000 steps (5%)
[13:37:46] Completed 12000 out of 200000 steps (6%)
[13:41:43] Completed 14000 out of 200000 steps (7%)
[13:45:39] Completed 16000 out of 200000 steps (8%)
[13:49:46] Completed 18000 out of 200000 steps (9%)
[13:53:46] Completed 20000 out of 200000 steps (10%)
[13:57:45] Completed 22000 out of 200000 steps (11%)
[14:01:42] Completed 24000 out of 200000 steps (12%)
[14:05:36] Completed 26000 out of 200000 steps (13%)
[14:09:45] Completed 28000 out of 200000 steps (14%)
[14:14:05] Completed 30000 out of 200000 steps (15%)
[14:18:19] Completed 32000 out of 200000 steps (16%)
[14:22:23] Completed 34000 out of 200000 steps (17%)
[14:26:41] Completed 36000 out of 200000 steps (18%)
[14:31:10] Completed 38000 out of 200000 steps (19%)
[14:35:14] Completed 40000 out of 200000 steps (20%)
[14:39:09] Completed 42000 out of 200000 steps (21%)
[14:43:21] Completed 44000 out of 200000 steps (22%)
[14:47:37] Completed 46000 out of 200000 steps (23%)
[14:49:39] CoreStatus = 0 (0)
[14:49:39] Client-core communications error: ERROR 0x0
[14:49:39] - Attempting to download new core...
[14:49:39] + Downloading new core: FahCore_b4.exe
[14:49:39] + 10240 bytes downloaded
[14:49:39]...
[14:49:47] + 5609500 bytes downloaded
[14:49:47] Verifying core Core_b4.fah...
[14:49:48] Signature is VALID
[14:49:48] 
[14:49:48] Trying to unzip core FahCore_b4.exe
[14:49:49] Decompressed FahCore_b4.exe (21549840 bytes) successfully
[14:49:49] + Core successfully engaged
[14:49:50] Deleting current work unit & continuing...
[14:49:50] - Preparing to get new work unit...
[14:49:50] + Attempting to get work packet
[14:49:50] - Connecting to assignment server
[14:49:51] - Successful: assigned to (129.74.85.48).
[14:49:51] + News From Folding@Home: Welcome to Folding@Home
[14:49:51] Loaded queue successfully.
[14:49:51] + Closed connections
[14:49:56] 
[14:49:56] + Processing work unit
[14:49:56] Core required: FahCore_b4.exe
[14:49:56] Core found.
[14:49:56] Working on Unit 07 [December 26 14:49:56]
[14:49:56] + Working ...
[14:49:56] *********************** Log Started 26/Dec/2009 14:49:56 ***********************
[14:49:56] ************************** ProtoMol Folding@Home Core **************************
[14:49:56]   Version: 21
[14:49:56]      Type: 180
[14:49:56]      Core: ProtoMol
[14:49:56]   Website: http://folding.stanford.edu/
[14:49:56] Copyright: (c) 2009 Stanford University
[14:49:56]    Author: Joseph Coffland <joseph@cauldrondevelopment.com>
[14:49:56]      Args: -dir work/ -suffix 07 -checkpoint 15 -lifeline 2168 -version 602
[14:49:56] ************************************ Build *************************************
[14:49:56]      Date: Dec 24 2009
[14:49:56]      Time: 06:37:42
[14:49:56]  Revision: 1748
[14:49:56]  Compiler: Intel(R) C++ g++ 4.1 mode 1110
[14:49:56]   Options: -std=gnu++98 -mia32 -O3 -funroll-loops
[14:49:56]            -axSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 -restrict
[14:49:56]  Platform: Linux 2.4.27-3-386
[14:49:56]      Bits: 32
[14:49:56] ************************************ System ************************************
[14:49:56]        OS: Linux 2.6.31-16-generic i686
[14:49:56]       CPU: AMD Athlon(tm) 64 Processor 3400+
[14:49:56]    CPU ID: AuthenticAMD Family 15 Model 44 Stepping 0
[14:49:56]      CPUs: 1 Logical, 1 Physical
[14:49:56]    Memory: 1.97 GB
[14:49:56] ********************************************************************************
[14:49:56] Project: 10001 (Run 533, Clone 1, Gen 8)
[14:49:56] Reading tar file par_all27_prot_lipid.inp
[14:49:56] Reading tar file scpismQuartic.inp
[14:49:56] Reading tar file ww_exteq_nowater1.pdb
[14:49:56] Reading tar file ww_exteq_nowater1.psf
[14:49:56] Reading tar file checkpt
[14:49:56] Reading tar file ww_exteq_nowater1.189.pos
[14:49:56] Reading tar file ww_exteq_nowater1.189.vel
[14:49:56] Reading tar file protomol.conf
[14:49:56] Reading tar file core.xml
[14:49:56] Digital signatures verified
[14:49:57] Completed 0 out of 200000 steps (0%)
[14:53:56] Completed 2000 out of 200000 steps (1%)
[14:58:01] Completed 4000 out of 200000 steps (2%)
[15:02:06] Completed 6000 out of 200000 steps (3%)
[15:05:59] Completed 8000 out of 200000 steps (4%)
[15:10:18] Completed 10000 out of 200000 steps (5%)
[15:14:23] Completed 12000 out of 200000 steps (6%)
[15:18:19] Completed 14000 out of 200000 steps (7%)
[15:22:15] Completed 16000 out of 200000 steps (8%)
[15:26:23] Completed 18000 out of 200000 steps (9%)
[15:30:21] Completed 20000 out of 200000 steps (10%)
[15:34:19] Completed 22000 out of 200000 steps (11%)
[15:38:16] Completed 24000 out of 200000 steps (12%)
[15:42:10] Completed 26000 out of 200000 steps (13%)
[15:46:19] Completed 28000 out of 200000 steps (14%)
[15:50:38] Completed 30000 out of 200000 steps (15%)
[15:54:51] Completed 32000 out of 200000 steps (16%)
[15:58:54] Completed 34000 out of 200000 steps (17%)
[16:03:12] Completed 36000 out of 200000 steps (18%)
[16:07:41] Completed 38000 out of 200000 steps (19%)
[16:11:43] Completed 40000 out of 200000 steps (20%)
[16:15:38] Completed 42000 out of 200000 steps (21%)
[16:19:50] Completed 44000 out of 200000 steps (22%)
[16:24:06] Completed 46000 out of 200000 steps (23%)
[16:26:07] CoreStatus = 0 (0)
[16:26:07] Client-core communications error: ERROR 0x0
[16:26:07] Deleting current work unit & continuing...
[16:26:08] - Preparing to get new work unit...
[16:26:08] + Attempting to get work packet
[16:26:08] - Connecting to assignment server
[16:26:09] - Successful: assigned to (129.74.85.48).
[16:26:09] + News From Folding@Home: Welcome to Folding@Home
[16:26:09] Loaded queue successfully.
[17:26:09] + Could not connect to Work Server
[17:26:09] - Attempt #1  to get work failed, and no other work to do.
             Waiting before retry.

djchandler · Post by **djchandler** » Sun Dec 27, 2009 3:10 am

Went to the post on Dr. Pande's blog about this b4 core. I'm going to install MPIch2 on the box that's getting those WUs and see if that makes a difference. Is there any way for this to be available on your servers and downloaded with the b4 core? Probably not as root access may be necessary.

Anyway, I'm willing to give it a try. Shoot me this WU again if possible when the current one (p6313) finishes. please.

How about an error trapping routine in the b4 core to check if MPIch is installed? I'm reading that only the master node needs it installed, but I'm not sure how this is being implemented. Think this will help?

7im · Post by **7im** » Sun Dec 27, 2009 4:41 am

10 WUs in a row is a bug, IIRC. Previous clients/fahcores moved on to a new WU sooner than that. The b4 core doesn't handle some error trapping gracefully yet. It's a whole new fahcore type, so it has a few more kinks to work out yet.

Also, there may be an issue with the v6.02 client vs. the v6.2x client. Upgrade if you can.

Post by **bruce** » Sun Dec 27, 2009 5:28 am

If one of the wudata_0*.dat files that caused this problem is still somewhere on your system, send me a PM.

djchandler · Post by **djchandler** » Sun Dec 27, 2009 5:30 pm

7im wrote:Also, there may be an issue with the v6.02 client vs. the v6.2x client. Upgrade if you can.

This is a 32-bit OS. Some MythTV components/drivers still too buggy on 64-bit, so that's why I'm running 32-bit even though the hardware (Athlon 64 3400+, Venice core [sse3 implemented] and 2 gigs of RAM) will handle a 64-bit OS easily. So I'm stuck with 6.02 for now.

Maybe I should turn off the -adv flag, but I'm a glutton for punishment. I really don't mind losing a few points if we can get this de-bugged.

Any way to inform the client/core that MPIch is now installed locally?

djchandler · Post by **djchandler** » Sun Dec 27, 2009 5:33 pm

bruce wrote:If one of the wudata_0*.dat files that caused this problem is still somewhere on your system, send me a PM.

I'll check ASAP; have several projects on my plate at the moment though.

djchandler · Post by **djchandler** » Tue Dec 29, 2009 3:14 am

7im wrote:Also, there may be an issue with the v6.02 client vs. the v6.2x client. Upgrade if you can.

And maybe that it's the ability to run 64-bit that makes a difference. Thanks for giving me the idea to try a 64-bit client instead of the 32 bit. Seems the 64-bit kernel even running the 6.02 client cures some problems too. Running on almost identical hardware (same chipset and virtually identical CPUs), Core_B4 looks like it's doing better on the notfred running native 64-bit. It's ending early, but it's not getting the core-client communication error.

A big thank you to Bruce too for all the support via PM.

Even though this other WU EUE'd, here's what happened on the notfred. I'm betting that 7im is right--the Core_B4 requires 6.2x plus running 64-bit to complete correctly.

Code: Select all

[21:38:22] Completed 116000 out of 200000 steps (58%)
[21:41:14] Saving result file logfile_01.txt
[21:41:14] Saving result file checkpt
[21:41:14] Saving result file checkpt.crc
[21:41:14] Saving result file log.txt
[21:41:14] Saving result file protomol.conf
[21:41:14] Saving result file ww.dcd
[21:41:15] Saving result file ww_exteq_nowater1.239.pos
[21:41:15] Saving result file ww_exteq_nowater1.239.vel
[21:41:15] Folding@home Core Shutdown: FINISHED_UNIT
[21:41:16] CoreStatus = 64 (100)
[21:41:16] Sending work to server
[21:41:16] - Read packet limit of 540015616... Set to 524286976.
[21:41:16] + Attempting to send results
[21:41:32] + Results successfully sent
[21:41:32] Thank you for your contribution to Folding@Home.

djchandler · Post by **djchandler** » Tue Dec 29, 2009 8:29 am

7im is on the money. You need 6.2x client plus I think you need to be running 64-bit for the Core_B4. Notfred was using 6.02 and was spewing all sorts of garbage eventually.

I wiped my USB stick, installed a command line only copy of Debian 64 5.x, installed samba and set up lan access, then manually installed the 6.24 beta client. It certainly is faster too. It's lean and mean for an old CPU (Athlon 64 3200+ Venice core (90nm, SSE3, single core).

Folding Forum

Project 10001, Run 533, Clone 1, Gen 8 - EUE - ERROR 0x0

Project 10001, Run 533, Clone 1, Gen 8 - EUE - ERROR 0x0

Re: Project 10001, Run 533, Clone 1, Gen 8 - EUE - ERROR 0x0

Re: Project 10001, Run 533, Clone 1, Gen 8 - EUE - ERROR 0x0

Re: Project 10001, Run 533, Clone 1, Gen 8 - EUE - ERROR 0x0

Re: Project 10001, Run 533, Clone 1, Gen 8 - EUE - ERROR 0x0

Re: Project 10001, Run 533, Clone 1, Gen 8 - EUE - ERROR 0x0

Re: Project 10001, Run 533, Clone 1, Gen 8 - EUE - ERROR 0x0

Re: Project 10001, Run 533, Clone 1, Gen 8 - EUE - ERROR 0x0

Re: Project 10001, Run 533, Clone 1, Gen 8 - EUE - ERROR 0x0

Re: Project 10001, Run 533, Clone 1, Gen 8 - EUE - ERROR 0x0

Re: Project 10001, Run 533, Clone 1, Gen 8 - EUE - ERROR 0x0

Re: Project 10001, Run 533, Clone 1, Gen 8 - EUE - ERROR 0x0

Re: Project 10001, Run 533, Clone 1, Gen 8 - EUE - ERROR 0x0