Stats Server Down

Moderators: Site Moderators, FAHC Science Team

snapshot
Posts: 132
Joined: Thu Apr 09, 2009 7:25 pm
Location: Wiltshire, UK

Re: Stats Server Down

Post by snapshot »

Thanks. Will do.
rafwiewiora
Scientist
Posts: 165
Joined: Mon Aug 03, 2015 8:23 pm
Location: New York

Re: Stats Server Down

Post by rafwiewiora »

Re: spikes in points - choderalab is all good, I had 150 GPUs on 11730 in internal testing.
EOC_Jason
Posts: 10
Joined: Wed Feb 10, 2010 3:15 pm
Location: USA
Contact:

Re: Stats Server Down

Post by EOC_Jason »

Okay, I managed to finish (what I thought) was everything before the 3PM update so I let things run as scheduled. There looks to be a few names that are still duplicates due to some special character issues, but after the 6pm update I'm going to run a script that will find any users in the DB that aren't in the flat-file and remove them. It's probably going to be just a small handful of people (double digits at most), most aren't even active. There was one that had a lot of points and I fixed his account manually so he wouldn't lose all the historical data. Also I had a little oops in the Team 0 processing that wiped out & re-created some of those users, but I've fixed that bug. Can't do anything about that lost data...

I'm going to see what happens with the next update (in about 30 minutes) and tweak as necessary. But for the most part things should be back to normal.

I've got some other little code tweaks related to processing to do this weekend, but it shouldn't affect anything.
snapshot
Posts: 132
Joined: Thu Apr 09, 2009 7:25 pm
Location: Wiltshire, UK

Re: Stats Server Down

Post by snapshot »

rafwiewiora wrote:Re: spikes in points - choderalab is all good, I had 150 GPUs on 11730 in internal testing.
Thanks for the info. That must have been fun. :D
I can easily accept one team doing this but another one to the same extent plus the five lesser ones I've found in the same narrow window seems unlikely. It may just be that I'm paranoid but it still looks odd especially as Jason has finished his updates and nothing has changed.
DutchForce
Posts: 60
Joined: Sun Sep 08, 2013 12:43 pm
Location: Netherlands

Re: Stats Server Down

Post by DutchForce »

It looks like the Stats server is not updating.

I'm also getting an "Error failed to get project data" when I go to the Project Summary and no servers are showing up on the Server Status page.
Image
gordonbb
Posts: 511
Joined: Mon May 21, 2018 4:12 pm
Hardware configuration: Ubuntu 22.04.2 LTS; NVidia 525.60.11; 2 x 4070ti; 4070; 4060ti; 3x 3080; 3070ti; 3070
Location: Great White North

Re: Stats Server Down

Post by gordonbb »

Ditto,

@bruce @sortofageek @Macaholic @susato @Ivoshiee @toTOW @Joe_H @ PantherX
The stats servers reported about 1/2 the expected points for both myself and my team at the 15:00PST Update and now at the 18:00PST update is reporting 0 points for both.

No servers are showing in in the Server Stats but the individual stats are still showing values they are just not current.
Image
JimboPalmer
Posts: 2522
Joined: Mon Feb 16, 2009 4:12 am
Location: Greenwood MS USA

Re: Stats Server Down

Post by JimboPalmer »

Just before a long holiday weekend in the US, that server has it down pat.
Tsar of all the Rushers
I tried to remain childlike, all I achieved was childish.
A friend to those who want no friends
gordonbb
Posts: 511
Joined: Mon May 21, 2018 4:12 pm
Hardware configuration: Ubuntu 22.04.2 LTS; NVidia 525.60.11; 2 x 4070ti; 4070; 4060ti; 3x 3080; 3070ti; 3070
Location: Great White North

Re: Stats Server Down

Post by gordonbb »

JimboPalmer wrote:Just before a long holiday weekend in the US, that server has it down pat.
its failure is exhibiting different behaviour though. It is still producing the flat files but those flat files are not being updates from the collection servers.

I’m still being assigned work units and the completd jobs are being accepted so it’s all good.

I’ve worked in academic environments for over twenty years and understand that support is, of necessity, “best effort and running code”

Perhaps the stats server just needs and extended long weekend before mid terms to spend some time chillin’ with family :D
Image
drougnor
Posts: 147
Joined: Tue Dec 29, 2009 2:21 am

Re: Stats Server Down

Post by drougnor »

It looks like the stats 'froze' starting at the 1PM EST update for Users flat file and 2PM EST for the Teams flat file. The time/date stamp is updating, but the numbers for both the individual users and teams have both frozen.

d
bruce
Posts: 20824
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Stats Server Down

Post by bruce »

DutchForce wrote: I'm also getting an "Error failed to get project data" when I go to the Project Summary and no servers are showing up on the Server Status page.
Fixed.
goodyca
Posts: 187
Joined: Sun Dec 02, 2007 12:36 pm

Re: Stats Server Down

Post by goodyca »

The stats have stopped being updated. The last update occurred at 5:45 GMT today.
fluke76
Posts: 10
Joined: Wed Nov 22, 2017 10:33 am

Re: Stats Server Down

Post by fluke76 »

goodyca wrote:The stats have stopped being updated. The last update occurred at 5:45 GMT today.
Yep. I noticed that, too. I have at least 1 WU that was successfully uploaded, but not updated on Stat server. It is a US holiday so that might have something to do with it. If you have been folding for a while, you probably know that the Stat server thinks it gets a day off for a holiday.
gordonbb
Posts: 511
Joined: Mon May 21, 2018 4:12 pm
Hardware configuration: Ubuntu 22.04.2 LTS; NVidia 525.60.11; 2 x 4070ti; 4070; 4060ti; 3x 3080; 3070ti; 3070
Location: Great White North

Re: Stats Server Down

Post by gordonbb »

fluke76 wrote:
goodyca wrote:The stats have stopped being updated. The last update occurred at 5:45 GMT today.
Yep. I noticed that, too. I have at least 1 WU that was successfully uploaded, but not updated on Stat server. It is a US holiday so that might have something to do with it. If you have been folding for a while, you probably know that the Stat server thinks it gets a day off for a holiday.
LOL or the support folks (likely grad students) do
Image
Murford
Posts: 2
Joined: Fri Dec 07, 2018 8:41 am

Completed WUs not receiving credit

Post by Murford »

Mod note: Moved to existing topic on stats being down, added Code tags to your log - j

I have been running F@H for about a week without issue. I noticed yesterday that I received no credit, but when I checked the log there were completed WUs with estimated credit listed. I restarted my computer to see if it was an issue on my end, but it seems to still be happening. Restarting reset the log so I don't have the logs from yesterday, but I do have a log of a completed WU from today that it looks like I haven't been credited with.
Last edited by Murford on Tue Jan 22, 2019 7:12 pm, edited 1 time in total.
Joe_H
Site Admin
Posts: 7927
Joined: Tue Apr 21, 2009 4:41 pm
Hardware configuration: Mac Pro 2.8 quad 12 GB smp4
MacBook Pro 2.9 i7 8 GB smp2
Location: W. MA

Re: Stats Server Down

Post by Joe_H »

As I posted here - viewtopic.php?f=16&t=31338&p=304877#p304877 - I have notified the stats site maintainer.
Image

iMac 2.8 i7 12 GB smp8, Mac Pro 2.8 quad 12 GB smp6
MacBook Pro 2.9 i7 8 GB smp3
Post Reply