Page 1 of 2
Upload Lost: 129.74.85.15
Posted: Fri Feb 01, 2013 11:39 pm
by brityank
Found I had uploaded a WU last night, but it did not reflect in the stats pages:
Code: Select all
--- Opening Log file [January 31 19:35:50 UTC]
# Windows CPU Console Edition #################################################
###############################################################################
Folding@Home Client Version 6.23
http://folding.stanford.edu
###############################################################################
###############################################################################
Launch directory: C:\F@H2
Executable: C:\F@H2\F@H2.exe
Arguments: -local -verbosity 9
[19:35:50] - Ask before connecting: No
[19:35:50] - User name: brityank (Team 36120)
[19:35:50] - User ID: 3B5xxxxxxxxxxxxxxxxxxxxxxx
[19:35:50] - Machine ID: 2
[19:35:50]
[19:35:50] Loaded queue successfully.
[19:35:50]
[19:35:50] - Autosending finished units... [January 31 19:35:50 UTC]
[19:35:50] + Processing work unit
[19:35:50] Trying to send all finished work units
[19:35:50] Core required: FahCore_a4.exe
[19:35:50] + No unsent completed units remaining.
[19:35:50] - Autosend completed
[19:35:50] Core found.
[19:35:50] Working on queue slot 04 [January 31 19:35:50 UTC]
[19:35:50] + Working ...
[19:35:50] - Calling '.\FahCore_a4.exe -dir work/ -suffix 04 -checkpoint 15 -verbose -lifeline 4084 -version 623'
[19:35:50]
[19:35:50] *------------------------------*
[19:35:50] Folding@Home Gromacs GB Core
[19:35:50] Version 2.27 (Dec. 15, 2010)
[19:35:50]
[19:35:50] Preparing to commence simulation
[19:35:50] - Looking at optimizations...
[19:35:50] - Files status OK
[19:35:50] - Expanded 52479 -> 197152 (decompressed 375.6 percent)
[19:35:50] Called DecompressByteArray: compressed_data_size=52479 data_size=197152, decompressed_data_size=197152 diff=0
[19:35:50] - Digital signature verified
[19:35:50]
[19:35:50] Project: 7016 (Run 2, Clone 147, Gen 23)
[19:35:50]
[19:35:50] Assembly optimizations on if available.
[19:35:50] Entering M.D.
[19:35:56] Using Gromacs checkpoints
[19:35:56] Mapping NT from 1 to 1
[19:35:56] Resuming from checkpoint
[19:35:56] Verified work/wudata_04.log
[19:35:56] Verified work/wudata_04.trr
[19:35:56] Verified work/wudata_04.xtc
[19:35:56] Verified work/wudata_04.edr
[19:35:56] Completed 6700001 out of 10000000 steps (67%)
------------- snip -------------
[03:49:09] Completed 10000000 out of 10000000 steps (100%)
[03:49:09] DynamicWrapper: Finished Work Unit: sleep=10000
[03:49:19]
[03:49:19] Finished Work Unit:
[03:49:19] - Reading up to 2026464 from "work/wudata_04.trr": Read 2026464
[03:49:19] trr file hash check passed.
[03:49:19] - Reading up to 211036 from "work/wudata_04.xtc": Read 211036
[03:49:19] xtc file hash check passed.
[03:49:19] edr file hash check passed.
[03:49:19] logfile size: 81183
[03:49:19] Leaving Run
[03:49:22] - Writing 2343155 bytes of core data to disk...
[03:49:22] Done: 2342643 -> 1755692 (compressed to 74.9 percent)
[03:49:22] ... Done.
[03:49:23] - Shutting down core
[03:49:23]
[03:49:23] Folding@home Core Shutdown: FINISHED_UNIT
[03:49:25] CoreStatus = 64 (100)
[03:49:25] Unit 4 finished with 90 percent of time to deadline remaining.
[03:49:25] Updated performance fraction: 0.889798
[03:49:25] Sending work to server
[03:49:25] Project: 7016 (Run 2, Clone 147, Gen 23)
[03:49:25] + Attempting to send results [February 1 03:49:25 UTC]
[03:49:25] - Reading file work/wuresults_04.dat from core
[03:49:25] (Read 1756204 bytes from disk)
[03:49:25] Connecting to http://129.74.85.15:8080/
[03:49:56] Posted data.
[03:49:56] Initial: 0000; - Uploaded at ~55 kB/s
[03:49:56] - Averaged speed for that direction ~57 kB/s
[03:49:56] + Results successfully sent
[03:49:56] Thank you for your contribution to Folding@Home.
[03:49:56] + Number of Units Completed: 1085
[03:50:01] Trying to send all finished work units
[03:50:01] + No unsent completed units remaining.
[03:50:01] - Preparing to get new work unit...
[03:50:01] + Attempting to get work packet
[03:50:01] - Will indicate memory of 1536 MB
[03:50:01] - Detect CPU. Vendor: GenuineIntel, Family: 6, Model: 15, Stepping: 6
[03:50:01] - Connecting to assignment server
[03:50:01] Connecting to http://assign.stanford.edu:8080/
[03:50:02] Posted data.
[03:50:02] Initial: 4A81; - Successful: assigned to (129.74.85.15).
[03:50:02] + News From Folding@Home: Welcome to Folding@Home
[03:50:02] Loaded queue successfully.
[03:50:02] Connecting to http://129.74.85.15:8080/
[03:50:04] Posted data.
[03:50:04] Initial: 0000; - Receiving payload (expected size: 141618)
[03:50:05] - Downloaded at ~138 kB/s
[03:50:05] - Averaged speed for that direction ~139 kB/s
[03:50:05] + Received work.
[03:50:05] Trying to send all finished work units
[03:50:05] + No unsent completed units remaining.
[03:50:05] + Closed connections
Checking the Server shows it was in Reject for part of the evening, so my (possibly invalid) assumption is that it got hung in the system someplace. Can I get a check run please? Many thanks.
Re: Upload Lost: 129.74.85.15
Posted: Fri Feb 01, 2013 11:50 pm
by Joe_H
There is no result reported by the database for this WU. However, the stats server has been experiencing intermittent problems for the last 24 hours. Reports on that are in
this topic. It may still show up, the server may take bit to catch up.
Re: Upload Lost: 129.74.85.15
Posted: Sat Feb 02, 2013 8:59 am
by brityank
Many thanks, Joe.
I've experienced several missing points over the years where the upload has gone through clean, as this excerpt shows, but the network fails to deliver the points due. I believe the last time they found that one of the hard drives in the server was glitching, but it took several weeks for a resolution. Most times the catch up only takes a few hours after the server is tied back in. The longer missing ones need an audit of the server to recover them. The WU I'm running now (P7083) will run through 2/6 @~07:00, so I'll check back then.
Cheers.
Re: Upload Lost: 129.74.85.15
Posted: Sat Feb 02, 2013 9:17 pm
by PSI_Performance
notice the same thing this morning, had a ~9000 pointer upload this morning, and ~5hrs late it still isnt showing up on the stats.
But, im not going to jump the gun yet, when the stats were FUBAR'd the other day, it took a while for everything to catch up, so i assume this one is going to (hopefully) appear within the next few stats updates
Re: Upload Lost: 129.74.85.15
Posted: Sat Feb 02, 2013 10:35 pm
by art_l_j_PlanetAMD64
PSI_Performance wrote:notice the same thing this morning, had a ~9000 pointer upload this morning, and ~5hrs late it still isnt showing up on the stats.
But, im not going to jump the gun yet, when the stats were FUBAR'd the other day, it took a while for everything to catch up, so i assume this one is going to (hopefully) appear within the next few stats updates
I'm sure it will turn up before too long. I seem to recall the last time there was a big outage, that it was a day or two before the last of the backlog was cleared up. And the weekend is often the time when a lot of regularly-scheduled maintenance and backups/upgrades are done, so the catching-up process might take a bit longer, than if the same thing happened during the week.
Re: Upload Lost: 129.74.85.15
Posted: Sun Feb 03, 2013 4:02 am
by PSI_Performance
9hrs later and still didnt show up...?
Points from my other computers are still coming in steady as usual, but this one seems to have disappeared...
Code: Select all
*********************** Log Started 2013-02-02T18:40:40Z ***********************
18:40:40:************************* Folding@home Client *************************
18:40:40: Website: http://folding.stanford.edu/
18:40:40: Copyright: (c) 2009-2012 Stanford University
18:40:40: Author: Joseph Coffland <joseph@cauldrondevelopment.com>
18:40:40: Args: --lifeline 3888 --command-port=36330
18:40:40: Config: C:/Users/dwerrell/AppData/Roaming/FAHClient/config.xml
18:40:40:******************************** Build ********************************
18:40:40: Version: 7.2.9
18:40:40: Date: Oct 3 2012
18:40:40: Time: 18:05:48
18:40:40: SVN Rev: 3578
18:40:40: Branch: fah/trunk/client
18:40:40: Compiler: Intel(R) C++ MSVC 1500 mode 1200
18:40:40: Options: /TP /nologo /EHa /Qdiag-disable:4297,4103,1786,279 /Ox -arch:SSE
18:40:40: /QaxSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 /Qopenmp /Qrestrict /MT /Qmkl
18:40:40: Platform: win32 XP
18:40:40: Bits: 32
18:40:40: Mode: Release
18:40:40:******************************* System ********************************
18:40:40: CPU: Intel(R) Core(TM) i5-2520M CPU @ 2.50GHz
18:40:40: CPU ID: GenuineIntel Family 6 Model 42 Stepping 7
18:40:40: CPUs: 4
18:40:40: Memory: 3.88GiB
18:40:40: Free Memory: 2.41GiB
18:40:40: Threads: WINDOWS_THREADS
18:40:40: On Battery: false
18:40:40: UTC offset: -7
18:40:40: PID: 6740
18:40:40: CWD: C:/Users/dwerrell/AppData/Roaming/FAHClient
18:40:40: OS: Windows 7 Ultimate
18:40:40: OS Arch: AMD64
18:40:40: GPUs: 0
18:40:40: CUDA: Not detected
18:40:40:Win32 Service: false
18:40:40:***********************************************************************
18:40:40:<config>
18:40:40: <!-- Folding Slot Configuration -->
18:40:40: <gpu v='true'/>
18:40:40:
18:40:40: <!-- Network -->
18:40:40: <proxy v=':0'/>
18:40:40:
18:40:40: <!-- Slot Control -->
18:40:40: <pause-on-battery v='true'/>
18:40:40:
18:40:40: <!-- User Information -->
18:40:40: <passkey v='********************************'/>
18:40:40: <team v='11314'/>
18:40:40: <user v='PSI_Performance'/>
18:40:40:
18:40:40: <!-- Folding Slots -->
18:40:40: <slot id='0' type='SMP'/>
18:40:40:</config>
18:40:40:Trying to access database...
18:40:40:Successfully acquired database lock
18:40:40:Enabled folding slot 00: READY smp:4
18:40:40:WU01:FS00:Sending unit results: id:01 state:SEND error:NO_ERROR project:7031 run:0 clone:4 gen:13 core:0xa4 unit:0x000000180001329c4f394041446be505
18:40:40:WU01:FS00:Uploading 16.81MiB to 129.74.85.15
18:40:40:WU00:FS00:Starting
18:40:40:WU01:FS00:Connecting to 129.74.85.15:8080
18:40:40:WU00:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/dwerrell/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/Core_a4.fah/FahCore_a4.exe -dir 00 -suffix 01 -version 702 -lifeline 6740 -checkpoint 15 -np 4
18:40:40:WU00:FS00:Started FahCore on PID 4856
18:40:40:WU00:FS00:Core PID:2128
18:40:40:WU00:FS00:FahCore 0xa4 started
18:40:41:WU00:FS00:0xa4:
18:40:41:WU00:FS00:0xa4:*------------------------------*
18:40:41:WU00:FS00:0xa4:Folding@Home Gromacs GB Core
18:40:41:WU00:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
18:40:41:WU00:FS00:0xa4:
18:40:41:WU00:FS00:0xa4:Preparing to commence simulation
18:40:41:WU00:FS00:0xa4:- Ensuring status. Please wait.
18:40:43:Server connection id=1 on 0.0.0.0:36330 from 127.0.0.1
18:40:46:WU01:FS00:Upload 5.95%
18:40:50:WU00:FS00:0xa4:- Looking at optimizations...
18:40:50:WU00:FS00:0xa4:- Working with standard loops on this execution.
18:40:50:WU00:FS00:0xa4:- Previous termination of core was improper.
18:40:50:WU00:FS00:0xa4:- Files status OK
18:40:50:WU00:FS00:0xa4:- Expanded 524848 -> 1189472 (decompressed 226.6 percent)
18:40:50:WU00:FS00:0xa4:Called DecompressByteArray: compressed_data_size=524848 data_size=1189472, decompressed_data_size=1189472 diff=0
18:40:50:WU00:FS00:0xa4:- Digital signature verified
18:40:50:WU00:FS00:0xa4:
18:40:50:WU00:FS00:0xa4:Project: 8028 (Run 1276, Clone 1, Gen 33)
18:40:50:WU00:FS00:0xa4:
18:40:50:WU00:FS00:0xa4:Entering M.D.
18:40:52:WU01:FS00:Upload 13.76%
18:40:56:WU00:FS00:0xa4:Mapping NT from 4 to 4
18:40:56:WU00:FS00:0xa4:Completed 0 out of 500000 steps (0%)
18:40:58:WU01:FS00:Upload 23.06%
18:41:04:WU01:FS00:Upload 27.52%
18:41:10:WU01:FS00:Upload 35.70%
18:41:16:WU01:FS00:Upload 44.63%
18:41:22:WU01:FS00:Upload 53.55%
18:41:28:WU01:FS00:Upload 62.85%
18:41:34:WU01:FS00:Upload 71.78%
18:41:40:WU01:FS00:Upload 80.70%
18:41:46:WU01:FS00:Upload 88.14%
18:41:52:WU01:FS00:Upload 96.32%
18:41:58:WU01:FS00:Upload complete
18:41:58:WU01:FS00:Server responded WORK_ACK (400)
18:41:58:WU01:FS00:Final credit estimate, 9047.00 points
18:41:58:WU01:FS00:Cleaning up
18:43:59:WU00:FS00:0xa4:Completed 5000 out of 500000 steps (1%)
18:46:55:WU00:FS00:0xa4:Completed 10000 out of 500000 steps (2%)
18:49:53:WU00:FS00:0xa4:Completed 15000 out of 500000 steps (3%)
Re: Upload Lost: 129.74.85.15
Posted: Sun Feb 03, 2013 4:07 am
by art_l_j_PlanetAMD64
PSI_Performance wrote:9hrs later and still didnt show up...?
I don't know about the details this time, but for one big outage a couple of years ago, I think it took a few days for the last of the backlog to be processed. But all of the points did eventually get awarded.
Re: Upload Lost: 129.74.85.15
Posted: Sun Feb 03, 2013 4:10 am
by PSI_Performance
art_l_j_PlanetAMD64 wrote:PSI_Performance wrote:9hrs later and still didnt show up...?
I don't know about the details this time, but for one big outage a couple of years ago, I think it took a few days for the last of the backlog to be processed. But all of the points did eventually get awarded.
Im just curious why the points from my other computer have been coming in steady as usual since the server was back up and running, now all of a sudden the points from this laptop just arent showing up?
Re: Upload Lost: 129.74.85.15
Posted: Sun Feb 03, 2013 3:39 pm
by Joe_H
Hi PSI_Performance (team 11314),
Your WU (P7031 R0 C4 G13) was added to the stats database on 2013-02-03 02:13:47 for 9953.39 points of credit.
Your points showed up a few hours later in the database. As to why points from some systems show up and others don't right away, what I have seen in prior outages of the stats server is that it can depend on when something was uploaded. Or it can be related to which WS or CS the upload was done to and possibly when. All I can say for certain with this outage is that things are still catching up, I am seeing a bunch more points being credited to my record in the last 12-24 hours than were turned in during that period.
Re: Upload Lost: 129.74.85.15
Posted: Mon Feb 04, 2013 4:01 am
by bruce
art_l_j_PlanetAMD64 wrote:I'm sure it will turn up before too long. I seem to recall the last time there was a big outage, that it was a day or two before the last of the backlog was cleared up. And the weekend is often the time when a lot of regularly-scheduled maintenance and backups/upgrades are done, so the catching-up process might take a bit longer, than if the same thing happened during the week.
I've seen cases where it took a week or two before the backlog was reinstated. If the PG has to go through a cumbersome manual process...without any mistakes, they generally check and carefully recheck every step between the original entries in the server log to the final addition to the stats. The points can be a pleasant surprise long after you've given up hope.
Re: Upload Lost: 129.74.85.15
Posted: Tue Feb 05, 2013 12:43 am
by art_l_j_PlanetAMD64
bruce wrote:art_l_j_PlanetAMD64 wrote:I'm sure it will turn up before too long. I seem to recall the last time there was a big outage, that it was a day or two before the last of the backlog was cleared up. And the weekend is often the time when a lot of regularly-scheduled maintenance and backups/upgrades are done, so the catching-up process might take a bit longer, than if the same thing happened during the week.
I've seen cases where it took a week or two before the backlog was reinstated. If the PG has to go through a cumbersome manual process...without any mistakes, they generally check and carefully recheck every step between the original entries in the server log to the final addition to the stats. The points can be a pleasant surprise long after you've given up hope.
Wow, I sure didn't have to wait for my 'pleasant surprise';
today's report from Kakao Stats:
Code: Select all
Date 1:00 4:00 7:00 10:00 13:00 16:00 19:00 22:00 Total
2013-02-04 76,140 4,388 15,592 60,780 4,364 10,012 72,496 7,832 251,604
Re: Upload Lost: 129.74.85.15
Posted: Tue Feb 05, 2013 6:45 am
by brityank
art_l_j_PlanetAMD64 wrote:bruce wrote:art_l_j_PlanetAMD64 wrote:I'm sure it will turn up before too long. I seem to recall the last time there was a big outage, that it was a day or two before the last of the backlog was cleared up. And the weekend is often the time when a lot of regularly-scheduled maintenance and backups/upgrades are done, so the catching-up process might take a bit longer, than if the same thing happened during the week.
I've seen cases where it took a week or two before the backlog was reinstated. If the PG has to go through a cumbersome manual process...without any mistakes, they generally check and carefully recheck every step between the original entries in the server log to the final addition to the stats. The points can be a pleasant surprise long after you've given up hope.
Wow, I sure didn't have to wait for my 'pleasant surprise';
today's report from Kakao Stats:
Code: Select all
Date 1:00 4:00 7:00 10:00 13:00 16:00 19:00 22:00 Total
2013-02-04 76,140 4,388 15,592 60,780 4,364 10,012 72,496 7,832 251,604
Glad to see that some folks are getting their due points, guess there's hope for me yet. As I noted above, based on prior exposure to similar situations, doubt I'll see anything for several weeks, if ever. Plus by the time it gets credited, any bonus points will be gone as it will be past the final deadline date.
Re: Upload Lost: 129.74.85.15
Posted: Tue Feb 05, 2013 6:54 am
by Joe_H
The bonus will be based on when the WU was uploaded, not when the result is posted to the database. I will flag your initial post to recheck and see if the WU shows up.
Re: Upload Lost: 129.74.85.15
Posted: Tue Feb 05, 2013 1:48 pm
by brityank
Joe_H wrote:The bonus will be based on when the WU was uploaded, not when the result is posted to the database. I will flag your initial post to recheck and see if the WU shows up.
Many thanks, Joe. I may have been mistaken, but believe was told that, if the WU is sent back out for recalc to a second client only one client receives the bonus. Regardless, I will check with my next email, grateful to help the science. Many thanks for your and the other Moderators support.
Re: Upload Lost: 129.74.85.15
Posted: Tue Feb 05, 2013 2:05 pm
by 7im
That's not always true. Depends on why the WU was reassigned. If the WU expired (passed the deadlines) on the first donor, then obviously no bonus. However, I have seen duplicate WUs both receive full credit due to server error, and other issues.