Just about every time my -bigadv clients finish up a standard SMP WU, they try to get a new -bigadv from the above server. They are almost always failing with the FILE_IO_ERROR, and the latest was about 10:50 PST today:
Code: Select all
[18:51:50] Folding@home Core Shutdown: FINISHED_UNIT
[18:51:52] CoreStatus = 64 (100)
[18:51:52] Sending work to server
[18:51:52] Project: 7504 (Run 2, Clone 156, Gen 8)
[18:51:52] + Attempting to send results [November 14 18:51:52 UTC]
[18:52:32] + Results successfully sent
[18:52:32] Thank you for your contribution to Folding@Home.
[18:52:32] + Number of Units Completed: 371
[18:52:36] - Preparing to get new work unit...
[18:52:36] Cleaning up work directory
[18:52:36] + Attempting to get work packet
[18:52:36] Passkey found
[18:52:36] - Connecting to assignment server
[18:52:37] - Successful: assigned to (130.237.232.141).
[18:52:37] + News From Folding@Home: Welcome to Folding@Home
[18:52:37] Loaded queue successfully.
[18:52:38] + Closed connections
[18:52:38]
[18:52:38] + Processing work unit
[18:52:38] Core required: FahCore_a5.exe
[18:52:38] Core found.
[18:52:38] Working on queue slot 02 [November 14 18:52:38 UTC]
[18:52:38] + Working ...
[18:52:38]
[18:52:38] *------------------------------*
[18:52:38] Folding@Home Gromacs SMP Core
[18:52:38] Version 2.27 (Mar 12, 2010)
[18:52:38]
[18:52:38] Preparing to commence simulation
[18:52:38] - Looking at optimizations...
[18:52:38] - Created dyn
[18:52:38] - Files status OK
[18:52:38] Couldn't Decompress
[18:52:38] Called DecompressByteArray: compressed_data_size=0 data_size=0, decompressed_data_size=0 diff=0
[18:52:38] -Error: Couldn't update checksum variables
[18:52:38] Error: Could not open work file
[18:52:38]
[18:52:38] Folding@home Core Shutdown: FILE_IO_ERROR
[18:52:42] CoreStatus = 75 (117)
[18:52:42] Error opening or reading from a file.
[18:52:42] Deleting current work unit & continuing...
[18:52:46] - Preparing to get new work unit...
[18:52:46] Cleaning up work directory
[18:52:46] + Attempting to get work packet
[18:52:46] Passkey found
[18:52:46] - Connecting to assignment server
[18:52:46] - Successful: assigned to (130.237.232.141).
[18:52:46] + News From Folding@Home: Welcome to Folding@Home
[18:52:46] Loaded queue successfully.
[18:52:47] + Closed connections
[18:52:52]
[18:52:52] + Processing work unit
[18:52:52] Core required: FahCore_a5.exe
[18:52:52] Core found.
[18:52:52] Working on queue slot 03 [November 14 18:52:52 UTC]
[18:52:52] + Working ...
[18:52:52]
[18:52:52] *------------------------------*
[18:52:52] Folding@Home Gromacs SMP Core
[18:52:52] Version 2.27 (Mar 12, 2010)
[18:52:52]
[18:52:52] Preparing to commence simulation
[18:52:52] - Looking at optimizations...
[18:52:52] - Created dyn
[18:52:52] - Files status OK
[18:52:52] Couldn't Decompress
[18:52:52] Called DecompressByteArray: compressed_data_size=0 data_size=0, decompressed_data_size=0 diff=0
[18:52:52] -Error: Couldn't update checksum variables
[18:52:52] Error: Could not open work file
[18:52:52]
[18:52:52] Folding@home Core Shutdown: FILE_IO_ERROR
[18:52:56] CoreStatus = 75 (117)
[18:52:56] Error opening or reading from a file.
[18:52:56] Deleting current work unit & continuing...
[18:53:00] - Preparing to get new work unit...
[18:53:00] Cleaning up work directory
[18:53:00] + Attempting to get work packet
[18:53:00] Passkey found
[18:53:00] - Connecting to assignment server
[18:53:01] - Successful: assigned to (130.237.232.141).
[18:53:01] + News From Folding@Home: Welcome to Folding@Home
[18:53:01] Loaded queue successfully.
[18:53:01] + Closed connections
[18:53:06]
[18:53:06] + Processing work unit
[18:53:06] Core required: FahCore_a5.exe
[18:53:06] Core found.
[18:53:06] Working on queue slot 04 [November 14 18:53:06 UTC]
[18:53:06] + Working ...
[18:53:06]
[18:53:06] *------------------------------*
[18:53:06] Folding@Home Gromacs SMP Core
[18:53:06] Version 2.27 (Mar 12, 2010)
[18:53:06]
[18:53:06] Preparing to commence simulation
[18:53:06] - Looking at optimizations...
[18:53:06] - Created dyn
[18:53:06] - Files status OK
[18:53:06] Couldn't Decompress
[18:53:06] Called DecompressByteArray: compressed_data_size=0 data_size=0, decompressed_data_size=0 diff=0
[18:53:06] -Error: Couldn't update checksum variables
[18:53:06] Error: Could not open work file
[18:53:06]
[18:53:06] Folding@home Core Shutdown: FILE_IO_ERROR
[18:53:11] CoreStatus = 75 (117)
[18:53:11] Error opening or reading from a file.
[18:53:11] Deleting current work unit & continuing...
[18:53:15] - Preparing to get new work unit...
[18:53:15] Cleaning up work directory
[18:53:15] + Attempting to get work packet
[18:53:15] Passkey found
[18:53:15] - Connecting to assignment server
[18:53:16] - Successful: assigned to (130.237.232.141).
[18:53:16] + News From Folding@Home: Welcome to Folding@Home
[18:53:16] Loaded queue successfully.
[18:53:16] + Closed connections
[18:53:21]
[18:53:21] + Processing work unit
[18:53:21] Core required: FahCore_a5.exe
[18:53:21] Core found.
[18:53:21] Working on queue slot 05 [November 14 18:53:21 UTC]
[18:53:21] + Working ...
[18:53:21]
[18:53:21] *------------------------------*
[18:53:21] Folding@Home Gromacs SMP Core
[18:53:21] Version 2.27 (Mar 12, 2010)
[18:53:21]
[18:53:21] Preparing to commence simulation
[18:53:21] - Looking at optimizations...
[18:53:21] - Created dyn
[18:53:21] - Files status OK
[18:53:21] Couldn't Decompress
[18:53:21] Called DecompressByteArray: compressed_data_size=0 data_size=0, decompressed_data_size=0 diff=0
[18:53:21] -Error: Couldn't update checksum variables
[18:53:21] Error: Could not open work file
[18:53:21]
[18:53:21] Folding@home Core Shutdown: FILE_IO_ERROR
[18:53:26] CoreStatus = 75 (117)
[18:53:26] Error opening or reading from a file.
[18:53:26] Deleting current work unit & continuing...
[18:53:30] - Preparing to get new work unit...
[18:53:30] Cleaning up work directory
[18:53:30] + Attempting to get work packet
[18:53:30] Passkey found
[18:53:30] - Connecting to assignment server
[18:53:31] - Successful: assigned to (128.143.199.97).
[18:53:31] + News From Folding@Home: Welcome to Folding@Home
[18:53:31] Loaded queue successfully.
[18:53:36] + Closed connections
[18:53:41]
[18:53:41] + Processing work unit
[18:53:41] Core required: FahCore_a3.exe
[18:53:41] Core found.
[18:53:41] Working on queue slot 06 [November 14 18:53:41 UTC]
[18:53:41] + Working ...
[18:53:42]
[18:53:42] *------------------------------*
[18:53:42] Folding@Home Gromacs SMP Core
[18:53:42] Version 2.27 (Dec. 15, 2010)
[18:53:42]
[18:53:42] Preparing to commence simulation
[18:53:42] - Looking at optimizations...
[18:53:42] - Created dyn
[18:53:42] - Files status OK
[18:53:42] - Expanded 1765543 -> 2700832 (decompressed 152.9 percent)
[18:53:42] Called DecompressByteArray: compressed_data_size=1765543 data_size=2700832, decompressed_data_size=2700832 diff=0
[18:53:42] - Digital signature verified
[18:53:42]
[18:53:42] Project: 7504 (Run 4, Clone 70, Gen 19)
[18:53:42]
[18:53:42] Assembly optimizations on if available.
[18:53:42] Entering M.D.
[18:53:48] Mapping NT from 12 to 12
[18:53:48] Completed 0 out of 500000 steps (0%)
[18:57:13] Completed 5000 out of 500000 steps (1%)
[19:00:39] Completed 10000 out of 500000 steps (2%)
[19:04:01] Completed 15000 out of 500000 steps (3%)
[19:07:24] Completed 20000 out of 500000 steps (4%)
[19:10:53] Completed 25000 out of 500000 steps (5%)
[19:14:15] Completed 30000 out of 500000 steps (6%)
[19:17:38] Completed 35000 out of 500000 steps (7%)
[19:21:03] Completed 40000 out of 500000 steps (8%)
[19:24:34] Completed 45000 out of 500000 steps (9%)
Here is the QD output of my queue with this most recent spate, if that will give any more info. Index slots 2 through 5 are all deleted from the FILE_IO_ERROR:
Code: Select all
qd released 30 August 2011 (fr 086)
qd executed Mon Nov 14 11:21:58 Pacific Standard Time 2011 (Mon Nov 14 19:21:58 UTC 2011)
Queue version 6.00
Current index: 6
Index 7: finished 644.00 pts (141.271 pt/hr, 3363.45 ppd) 27.4 X min speed
bonus pts: 5803.27 (1262.877 pt/hr, 30309.05 ppd); bonus factor: 9.01; kfactor: 2.99
server: 128.143.199.97:8080; project: 7504
Folding: run 18, clone 168, generation 9; benchmark 0; misc: 500, 200, 12 (be)
issue: Sun Nov 13 04:41:53 2011; begin: Sun Nov 13 04:44:05 2011
end: Sun Nov 13 09:17:36 2011; due: Fri Nov 18 09:32:05 2011 (5 days)
preferred: Wed Nov 16 07:08:05 2011 (3 days)
core URL: http://www.stanford.edu/~pande/Win32/x86/Core_a3.fah
core number: 0xa3; core name: GRO-A3
CPU: 1,687 Pentium II/III; OS: 1,0 Windows
smp cores: 12; cores to use: 12
flops: 1064695772 (1064.695772 megaflops)
memory: 8192 MB
client type: 7 BigAdv
assignment info (be): Sun Nov 13 04:42:25 2011; 96C4F72F
CS: 130.237.165.141; P limit: 524286976
user: DrSpalding; team: 48083; ID: 4F89B31FE1BADD60; mach ID: 2
work/wudata_07.dat file size: 1762377; WU type: Folding@home
Index 8: finished 27.5 X min speed
server: 171.64.65.53:8080; project: 6099
Folding: run 5, clone 17, generation 9; benchmark 0; misc: 500, 200, 12 (be)
issue: Sun Nov 13 09:18:20 2011; begin: Sun Nov 13 09:18:24 2011
end: Sun Nov 13 20:29:34 2011; due: Sat Nov 26 04:30:24 2011 (13 days)
core URL: http://www.stanford.edu/~pande/Win32/x86/Core_a3.fah
core number: 0xa3; core name: GRO-A3
CPU: 1,687 Pentium II/III; OS: 1,0 Windows
smp cores: 12; cores to use: 12
flops: 1064704695 (1064.704695 megaflops)
memory: 8192 MB
client type: 7 BigAdv
assignment info (be): Sun Nov 13 09:16:45 2011; BD0BB137
CS: 130.237.165.141; P limit: 524286976
user: DrSpalding; team: 48083; ID: 4F89B31FE1BADD60; mach ID: 2
work/wudata_08.dat file size: 3814903; WU type: Folding@home
Index 9: finished 644.00 pts (141.954 pt/hr, 3378.57 ppd) 27.5 X min speed
bonus pts: 5816.29 (1271.398 pt/hr, 30513.56 ppd); bonus factor: 9.03; kfactor: 2.99
server: 128.143.199.97:8080; project: 7504
Folding: run 20, clone 3, generation 51; benchmark 0; misc: 500, 200, 12 (be)
issue: Sun Nov 13 20:28:10 2011; begin: Sun Nov 13 20:30:27 2011
end: Mon Nov 14 01:02:39 2011; due: Sat Nov 19 01:18:27 2011 (5 days)
preferred: Wed Nov 16 22:54:27 2011 (3 days)
core URL: http://www.stanford.edu/~pande/Win32/x86/Core_a3.fah
core number: 0xa3; core name: GRO-A3
CPU: 1,687 Pentium II/III; OS: 1,0 Windows
smp cores: 12; cores to use: 12
flops: 1064712217 (1064.712217 megaflops)
memory: 8192 MB
client type: 7 BigAdv
assignment info (be): Sun Nov 13 20:28:44 2011; 96C5D5E2
CS: 130.237.165.141; P limit: 524286976
user: DrSpalding; team: 48083; ID: 4F89B31FE1BADD60; mach ID: 2
work/wudata_09.dat file size: 1764699; WU type: Folding@home
Index 0: finished 644.00 pts (143.235 pt/hr, 3408.58 ppd) 27.8 X min speed
bonus pts: 5842.07 (1288.376 pt/hr, 30921.02 ppd); bonus factor: 9.07; kfactor: 2.99
server: 128.143.199.97:8080; project: 7504
Folding: run 20, clone 3, generation 52; benchmark 0; misc: 500, 200, 12 (be)
issue: Mon Nov 14 01:01:10 2011; begin: Mon Nov 14 01:03:28 2011
end: Mon Nov 14 05:33:14 2011; due: Sat Nov 19 05:51:28 2011 (5 days)
preferred: Thu Nov 17 03:27:28 2011 (3 days)
core URL: http://www.stanford.edu/~pande/Win32/x86/Core_a3.fah
core number: 0xa3; core name: GRO-A3
CPU: 1,687 Pentium II/III; OS: 1,0 Windows
smp cores: 12; cores to use: 12
flops: 1064718441 (1064.718441 megaflops)
memory: 8192 MB
client type: 7 BigAdv
assignment info (be): Mon Nov 14 01:01:45 2011; 96C515E7
CS: 130.237.165.141; P limit: 524286976
user: DrSpalding; team: 48083; ID: 4F89B31FE1BADD60; mach ID: 2
work/wudata_00.dat file size: 1761463; WU type: Folding@home
Index 1: finished 644.00 pts (121.592 pt/hr, 2896.79 ppd) 23.6 X min speed
bonus pts: 5385.66 (1009.391 pt/hr, 24225.38 ppd); bonus factor: 8.36; kfactor: 2.99
server: 128.143.199.97:8080; project: 7504
Folding: run 2, clone 156, generation 8; benchmark 0; misc: 500, 200, 12 (be)
issue: Mon Nov 14 05:31:44 2011; begin: Mon Nov 14 05:34:05 2011
end: Mon Nov 14 10:51:52 2011; due: Sat Nov 19 10:22:05 2011 (5 days)
preferred: Thu Nov 17 07:58:05 2011 (3 days)
core URL: http://www.stanford.edu/~pande/Win32/x86/Core_a3.fah
core number: 0xa3; core name: GRO-A3
CPU: 1,687 Pentium II/III; OS: 1,0 Windows
smp cores: 12; cores to use: 12
flops: 1064724511 (1064.724511 megaflops)
memory: 8192 MB
client type: 7 BigAdv
assignment info (be): Mon Nov 14 05:32:19 2011; 96C5547D
CS: 130.237.165.141; P limit: 524286976
user: DrSpalding; team: 48083; ID: 4F89B31FE1BADD60; mach ID: 2
work/wudata_01.dat file size: 1761454; WU type: Folding@home
Index 2: deleted 7164.00 pts
server: 130.237.232.141:8080; project: 6900
Folding: run 42, clone 19, generation 71; benchmark 0; misc: 500, 200, 12 (be)
issue: Mon Nov 14 10:52:33 2011; begin: Mon Nov 14 10:52:38 2011
end: ZERO; due: Sun Nov 20 10:52:38 2011 (6 days)
preferred: Fri Nov 18 10:52:38 2011 (4 days)
core URL: http://www.stanford.edu/~pande/Win32/x86/Core_a5.fah
core number: 0xa5; core name: GRO-A5
CPU: 1,687 Pentium II/III; OS: 1,0 Windows
smp cores: 12; cores to use: 12
flops: 1064707850 (1064.707850 megaflops)
memory: 8192 MB
client type: 7 BigAdv
assignment info (be): Mon Nov 14 10:50:57 2011; 94A0B0E3
CS: 130.237.165.141; P limit: 524286976
user: DrSpalding; team: 48083; ID: 4F89B31FE1BADD60; mach ID: 2
work/wudata_02.dat file size: 512; WU type: Folding@home
Index 3: deleted 7164.00 pts
server: 130.237.232.141:8080; project: 6900
Folding: run 42, clone 19, generation 71; benchmark 0; misc: 500, 200, 12 (be)
issue: Mon Nov 14 10:52:42 2011; begin: Mon Nov 14 10:52:47 2011
end: ZERO; due: Sun Nov 20 10:52:47 2011 (6 days)
preferred: Fri Nov 18 10:52:47 2011 (4 days)
core URL: http://www.stanford.edu/~pande/Win32/x86/Core_a5.fah
core number: 0xa5; core name: GRO-A5
CPU: 1,687 Pentium II/III; OS: 1,0 Windows
smp cores: 12; cores to use: 12
flops: 1064707850 (1064.707850 megaflops)
memory: 8192 MB
client type: 7 BigAdv
assignment info (be): Mon Nov 14 10:51:07 2011; 94A0B0E9
CS: 130.237.165.141; P limit: 524286976
user: DrSpalding; team: 48083; ID: 4F89B31FE1BADD60; mach ID: 2
work/wudata_03.dat file size: 512; WU type: Folding@home
Index 4: deleted 7164.00 pts
server: 130.237.232.141:8080; project: 6900
Folding: run 42, clone 19, generation 71; benchmark 0; misc: 500, 200, 12 (be)
issue: Mon Nov 14 10:52:57 2011; begin: Mon Nov 14 10:53:01 2011
end: ZERO; due: Sun Nov 20 10:53:01 2011 (6 days)
preferred: Fri Nov 18 10:53:01 2011 (4 days)
core URL: http://www.stanford.edu/~pande/Win32/x86/Core_a5.fah
core number: 0xa5; core name: GRO-A5
CPU: 1,687 Pentium II/III; OS: 1,0 Windows
smp cores: 12; cores to use: 12
flops: 1064707850 (1064.707850 megaflops)
memory: 8192 MB
client type: 7 BigAdv
assignment info (be): Mon Nov 14 10:51:21 2011; 94A0B0DB
CS: 130.237.165.141; P limit: 524286976
user: DrSpalding; team: 48083; ID: 4F89B31FE1BADD60; mach ID: 2
work/wudata_04.dat file size: 512; WU type: Folding@home
Index 5: deleted 7164.00 pts
server: 130.237.232.141:8080; project: 6900
Folding: run 42, clone 19, generation 71; benchmark 0; misc: 500, 200, 12 (be)
issue: Mon Nov 14 10:53:12 2011; begin: Mon Nov 14 10:53:16 2011
end: ZERO; due: Sun Nov 20 10:53:16 2011 (6 days)
preferred: Fri Nov 18 10:53:16 2011 (4 days)
core URL: http://www.stanford.edu/~pande/Win32/x86/Core_a5.fah
core number: 0xa5; core name: GRO-A5
CPU: 1,687 Pentium II/III; OS: 1,0 Windows
smp cores: 12; cores to use: 12
flops: 1064707850 (1064.707850 megaflops)
memory: 8192 MB
client type: 7 BigAdv
assignment info (be): Mon Nov 14 10:51:36 2011; 94A0B0CA
CS: 130.237.165.141; P limit: 524286976
user: DrSpalding; team: 48083; ID: 4F89B31FE1BADD60; mach ID: 2
work/wudata_05.dat file size: 512; WU type: Folding@home
Index 6: folding now 644.00 pts (112.612 pt/hr, 2684.24 ppd) 21.8 X min speed; 8% complete
bonus pts: 5184.31 (72.029 pt/hr, 21608.58 ppd); bonus factor: 8.05; kfactor: 2.99
server: 128.143.199.97:8080; project: 7504
Folding: run 4, clone 70, generation 19; benchmark 0; misc: 500, 200, 12 (be)
issue: Mon Nov 14 10:51:14 2011; begin: Mon Nov 14 10:53:36 2011
expect: Mon Nov 14 16:36:43 2011; due: Sat Nov 19 15:41:36 2011 (5 days)
preferred: Thu Nov 17 13:17:36 2011 (3 days)
core URL: http://www.stanford.edu/~pande/Win32/x86/Core_a3.fah
core number: 0xa3; core name: GRO-A3
CPU: 1,687 Pentium II/III; OS: 1,0 Windows
smp cores: 12; cores to use: 12
flops: 1064707850 (1064.707850 megaflops)
memory: 8192 MB
client type: 7 BigAdv
assignment info (be): Mon Nov 14 10:51:51 2011; 96C29F59
CS: 130.237.165.141; P limit: 524286976
user: DrSpalding; team: 48083; ID: 4F89B31FE1BADD60; mach ID: 2
work/wudata_06.dat file size: 1766055; WU type: Folding@home
Average download rate 345.633 KB/s (u=4); upload rate 254.864 KB/s (u=4)
Performance fraction 0.961533 (u=4)
Average pph: 135.822, ppd: 3259.74, ppw: 22818.1, ppy: 1190586
Average bonus pph: 938.059, ppd: 22513.42, ppw: 157593.9, ppy: 8222800
Average alternate pph: 129.849, ppd: 3116.37, ppw: 21814.6, ppy: 1138224
Average alternate bonus pph: 1130.395, ppd: 27129.49, ppw: 189906.4, ppy: 9908773
The clients eventually fall back to a standard SMP assignment/work server and immediately get a WU to work on and off we go. This is affecting all three of my -bigadv machines.