Stats server is down, [and fixed] again

Moderators: Site Moderators, FAHC Science Team

Kent Irwin
Posts: 22
Joined: Sun Jan 26, 2014 1:44 pm

Stats server is down, [and fixed] again

Post by Kent Irwin »

Thread title says it all.
sortofageek
Site Admin
Posts: 3110
Joined: Fri Nov 30, 2007 8:06 pm
Location: Team Helix
Contact:

Re: Stats server is down, again

Post by sortofageek »

What are you seeing that tells you stats are down? I just looked around at my account, a few team members and the team itself. It looked ok to me.
Joe_H
Site Admin
Posts: 7927
Joined: Tue Apr 21, 2009 4:41 pm
Hardware configuration: Mac Pro 2.8 quad 12 GB smp4
MacBook Pro 2.9 i7 8 GB smp2
Location: W. MA

Re: Stats server is down, again

Post by Joe_H »

I have seen some slow connections to the fah-web.stanford.edu sites today, and looking at my team stats on EOC it appears an update may not have been done in time for the 3:00 load. But the 6:00 one is there on EOC. SO if there was a problem, it may have been just a temporary one.
Image

iMac 2.8 i7 12 GB smp8, Mac Pro 2.8 quad 12 GB smp6
MacBook Pro 2.9 i7 8 GB smp3
sco01
Posts: 79
Joined: Sun Mar 21, 2010 8:52 pm
Hardware configuration: Core i7 3930K(C2)@4,2 (1.30v)
Swiftech Apogee Drive II + MCR-H220 Radiator
Asus P9X79
4*4Gb SEC 1600@1866 1.5v
GeForce GTX780ti ref (nztx G10+zalman lq315)
Creative X-Fi Titanium Fatal1ty
Lite-On iHAS124-34
HDD- WD3000GLFS + 2*Hitaсhi HDS723030BLE640 + 2*HDT721010SLA360
case-Corsair Obsidian 700D
PSU - Seasonic Platinum 860W (SS-860XP2)

2) Xeon E5645@4Ghz (1,34v)
ASUS P6T
Thermalright Silver Arrow
Corsair TR3X6G1600C7 (3x2048Mb, 7-7-7-20 1,62v)
GeForce GTX780 ref (nztx G10+corsair H75)
Zotac GTX980 AMP! Edition (ZT-90204-10P)
Hitachi 320Gb HTS543232A7A384
case - Chieftec LBX-03B Dremel edition :-)
PSU - Zalman ZM-850HP+ (Enhance silver)

APC SUA1500RMI2U
APC SUA1500RMI2U

Re: Stats server is down, again [not]

Post by sco01 »

Internal Server Error
The server encountered an internal error or misconfiguration and was unable to complete your request.
Please contact the server administrator, root@localhost and inform them of the time the error occurred, and anything you might have done that may have caused the error.
More information about this error may be available in the server error log.

Apache/2.0.52 (CentOS) Server at fah-web.stanford.edu Port 80
billford
Posts: 1003
Joined: Thu May 02, 2013 8:46 pm
Hardware configuration: Full Time:

2x NVidia GTX 980
1x NVidia GTX 780 Ti
2x 3GHz Core i5 PC (Linux)

Retired:

3.2GHz Core i5 PC (Linux)
3.2GHz Core i5 iMac
2.8GHz Core i5 iMac
2.16GHz Core 2 Duo iMac
2GHz Core 2 Duo MacBook
1.6GHz Core 2 Duo Acer laptop
Location: Near Oxford, United Kingdom
Contact:

Re: Stats server is down, again [not]

Post by billford »

Same here.

As far as I can see it was OK until the update run at 01:00 (UTC), then something went pear-shaped.
Image
Kent Irwin
Posts: 22
Joined: Sun Jan 26, 2014 1:44 pm

Re: Stats server is down, again [not]

Post by Kent Irwin »

Kent Irwin wrote:Thread title says it all.
Stats server is down, again [yes]
It's down again. It was down for a short period when I originally posted. It is a chronic problem.
Skufer
Posts: 39
Joined: Fri Feb 14, 2014 10:45 am
Location: UK/London

Re: Stats server is down, again [not]

Post by Skufer »

I can confirm that I am also seeing an error 500 Internal Server Error on the stats page
toTOW
Site Moderator
Posts: 6349
Joined: Sun Dec 02, 2007 10:38 am
Location: Bordeaux, France
Contact:

Re: Stats server is down, again

Post by toTOW »

Yes it's down again, no data on 3 tast EOC updates :(
Image

Folding@Home beta tester since 2002. Folding Forum moderator since July 2008.
drougnor
Posts: 147
Joined: Tue Dec 29, 2009 2:21 am

Re: Stats server is down, again

Post by drougnor »

My stats collector hasn't seen an update since the last one at 9pm EDT.
Joe_H
Site Admin
Posts: 7927
Joined: Tue Apr 21, 2009 4:41 pm
Hardware configuration: Mac Pro 2.8 quad 12 GB smp4
MacBook Pro 2.9 i7 8 GB smp2
Location: W. MA

Re: Stats server is down, again

Post by Joe_H »

I have sent emails to the appropriate PG members. The problems appear to be only with fah-web.stanford.edu which accesses and reports the stats, as best as I can tell the database server itself is not affected.
Image

iMac 2.8 i7 12 GB smp8, Mac Pro 2.8 quad 12 GB smp6
MacBook Pro 2.9 i7 8 GB smp3
VijayPande
Pande Group Member
Posts: 2058
Joined: Fri Nov 30, 2007 6:25 am
Location: Stanford

Re: Stats server is down, again

Post by VijayPande »

Yes, the db is on a separate server and is isolated from these issues. This is related to the NaCl issue I posted about yesterday.

Short term, this should be fixed on Monday. Longer term, as mentioned in that post, we need to migrate out some services onto a new server to take care of this. We have those servers ready so it’s about migrating now.
Prof. Vijay Pande, PhD
Departments of Chemistry, Structural Biology, and Computer Science
Chair, Biophysics
Director, Folding@home Distributed Computing Project
Stanford University
drougnor
Posts: 147
Joined: Tue Dec 29, 2009 2:21 am

Re: Stats server is down, again

Post by drougnor »

*lame joke warning* That's what you get for putting table salt in your servers!

Lame joking aside, thanks for the info, sir. I know there's nothing we can do but wait, but we at EVGA had our overtake of the Default team estimated to be in the next 12-18 hours or so and are a little disappointed not to be able to track the 'official' time of overtake. However, it is what it is and we gladly continue to Fold for the cause!

d
beylie
Posts: 5
Joined: Mon May 12, 2014 3:26 pm

Re: Stats server is down, again

Post by beylie »

So this is why I've been seeing no stats for the last few days! Please tell me the work done isn't lost, but will show up when the stats servers are up again?
cxh
Pande Group Member
Posts: 126
Joined: Thu Jan 23, 2014 1:16 am
Location: Stanford, CA
Contact:

Re: Stats server is down, again

Post by cxh »

@beylie: No need to worry! Work is being returned and points are still being updated in our internal database. It's just the front-end that's not working at the moment.
beylie
Posts: 5
Joined: Mon May 12, 2014 3:26 pm

Re: Stats server is down, again

Post by beylie »

Excellent! Thank you!
Post Reply