Stats not updating yet again
Moderators: Site Moderators, FAHC Science Team
Stats not updating yet again
It appears that at 10pm Eastern time last night, the stats flat files stopped updating. The dates are updating (Or else my system wouldn't be pulling and processing the new files) but the values within those files are stuck at the 10pm EDT update.
A quick look at EOC shows they're not processing new values either.
Thanks!
A quick look at EOC shows they're not processing new values either.
Thanks!
-
- Posts: 86
- Joined: Wed Jan 06, 2016 4:16 am
- Location: Northern Sweden
Re: Stats not updating yet again
Reported
Posting FAH's log:
How to provide enough info to get helpful support.
How to provide enough info to get helpful support.
-
- Posts: 1576
- Joined: Tue May 28, 2013 12:14 pm
- Location: Tokyo
Re: Stats not updating yet again
Ok, I know no one can fix it just now, but I want to put some stank on this!
We at the EVGA folding team have been folding under the name of one of our folder's who is currently battling with cancer - wrinvert. Our goal was to push him over 1 billion points and we are within 5-6 hours of doing so, if not less, and I just noticed that my stats software picked up a perfect duplicate flat file, which is usually the indication that the stats server has burped and needs to be restarted.
SO, if there is ANY way anyone can get ahold of the appropriate people and rattle some cages to get that server in line so we can actually track the 1 billion point crossover, Folding @ EVGA would be in your debt, hard core. Please, and thank you!
d
We at the EVGA folding team have been folding under the name of one of our folder's who is currently battling with cancer - wrinvert. Our goal was to push him over 1 billion points and we are within 5-6 hours of doing so, if not less, and I just noticed that my stats software picked up a perfect duplicate flat file, which is usually the indication that the stats server has burped and needs to be restarted.
SO, if there is ANY way anyone can get ahold of the appropriate people and rattle some cages to get that server in line so we can actually track the 1 billion point crossover, Folding @ EVGA would be in your debt, hard core. Please, and thank you!
d
Re: Stats not updating yet again
We've restarted the stats server. New reports should be generated within the hour. Thanks for bringing this to our attention!
Re: Stats not updating yet again
Now it works again. Tnx
Re: Stats not updating yet again
seems like the stats server is stuck again ...
-
- Site Admin
- Posts: 7927
- Joined: Tue Apr 21, 2009 4:41 pm
- Hardware configuration: Mac Pro 2.8 quad 12 GB smp4
MacBook Pro 2.9 i7 8 GB smp2 - Location: W. MA
Re: Stats not updating yet again
Stats database was updated for a WU I turned in less than an hour ago, what are you basing your statement on that the server is stuck? The official stats reflect the turn in of that WU as well.
If you are referring to EOC stats, they were down for most of the day for a database upgrade and just came back up about 2 hours ago.
If you are referring to EOC stats, they were down for most of the day for a database upgrade and just came back up about 2 hours ago.
iMac 2.8 i7 12 GB smp8, Mac Pro 2.8 quad 12 GB smp6
MacBook Pro 2.9 i7 8 GB smp3
-
- Posts: 177
- Joined: Tue Aug 26, 2014 9:48 pm
- Hardware configuration: 10 SMP folding slots on Intel Phi "Knights Landing" system, configured as 24 CPUs/slot
9 AMD GPU folding slots
31 Nvidia GPU folding slots
50 total folding slots
Average PPD/slot = 459,500 - Location: Dallas, TX
Re: Stats not updating yet again
Could someone please check the stats server? Date and time of last work unit submitted was recorded on 2016-07-11 20:05:39.
Is the stats server status listed on this page - and if not, should it? http://fah-web.stanford.edu/pybeta/serverstat.html
Is the stats server status listed on this page - and if not, should it? http://fah-web.stanford.edu/pybeta/serverstat.html
Hardware config viewtopic.php?f=66&t=17997&p=277235#p277235
-
- Site Admin
- Posts: 7927
- Joined: Tue Apr 21, 2009 4:41 pm
- Hardware configuration: Mac Pro 2.8 quad 12 GB smp4
MacBook Pro 2.9 i7 8 GB smp2 - Location: W. MA
Re: Stats not updating yet again
I will notify the stats team about the outage. It appears the scripts have stopped, but the the server is up.
As for being on the Server Status page, how would you add it to a report on WS stats? In addition, the scripts that report on the stats run on fah-web, so being able to see that page shows the server itself is up. Also, at the current time the Server Status page does not include the AS's.
As for being on the Server Status page, how would you add it to a report on WS stats? In addition, the scripts that report on the stats run on fah-web, so being able to see that page shows the server itself is up. Also, at the current time the Server Status page does not include the AS's.
iMac 2.8 i7 12 GB smp8, Mac Pro 2.8 quad 12 GB smp6
MacBook Pro 2.9 i7 8 GB smp3
-
- Posts: 177
- Joined: Tue Aug 26, 2014 9:48 pm
- Hardware configuration: 10 SMP folding slots on Intel Phi "Knights Landing" system, configured as 24 CPUs/slot
9 AMD GPU folding slots
31 Nvidia GPU folding slots
50 total folding slots
Average PPD/slot = 459,500 - Location: Dallas, TX
Re: Stats not updating yet again
Can't the Server Status web page pull JSON or XML status updates from any IP address it's configured to communicate with?
Maybe I'm oversimplifying, but just add the Stats Server as a record (row) in the database that generates the web page.
For example, the field headings would be:
Server IP: "IP v4 Address"
Name: "Statistics Server"
Who: "Name of Admin responsible for maintaining S.S."
Client: Add new category: "Stats"
Status: "Accept" means the S.S. is configured to collect statistics data, "Standby" means not accepting statistic collection data due to scheduled maintenance or other planned downtime.
Connect: "Accepting" means the S.S. is currently accepting statistics data from reporting servers, "Reject" means the server scripts crashed or some other non-normal status and statistics collection has stopped
Other stats can be collected such as the CPU load and Network load to help troubleshoot a Reject connect status. Every other field can be null. I may be assuming too much, but if the Stats server was in the same reporting and system management database as the WS servers, the admin responsible would be immediately alerted if any abnormal event occurs and could fix it before it has to be reported on the forum. Again, assuming the same would be true if AS's were added to the database. If no such automated alerting and monitoring system exists, a very good solution is Solarwinds. I'm sure you could get a big non-profit academic discount.
Maybe I'm oversimplifying, but just add the Stats Server as a record (row) in the database that generates the web page.
For example, the field headings would be:
Server IP: "IP v4 Address"
Name: "Statistics Server"
Who: "Name of Admin responsible for maintaining S.S."
Client: Add new category: "Stats"
Status: "Accept" means the S.S. is configured to collect statistics data, "Standby" means not accepting statistic collection data due to scheduled maintenance or other planned downtime.
Connect: "Accepting" means the S.S. is currently accepting statistics data from reporting servers, "Reject" means the server scripts crashed or some other non-normal status and statistics collection has stopped
Other stats can be collected such as the CPU load and Network load to help troubleshoot a Reject connect status. Every other field can be null. I may be assuming too much, but if the Stats server was in the same reporting and system management database as the WS servers, the admin responsible would be immediately alerted if any abnormal event occurs and could fix it before it has to be reported on the forum. Again, assuming the same would be true if AS's were added to the database. If no such automated alerting and monitoring system exists, a very good solution is Solarwinds. I'm sure you could get a big non-profit academic discount.
Hardware config viewtopic.php?f=66&t=17997&p=277235#p277235
-
- Posts: 10179
- Joined: Thu Nov 29, 2007 4:30 pm
- Hardware configuration: Intel i7-4770K @ 4.5 GHz, 16 GB DDR3-2133 Corsair Vengence (black/red), EVGA GTX 760 @ 1200 MHz, on an Asus Maximus VI Hero MB (black/red), in a blacked out Antec P280 Tower, with a Xigmatek Night Hawk (black) HSF, Seasonic 760w Platinum (black case, sleeves, wires), 4 SilenX 120mm Case fans with silicon fan gaskets and silicon mounts (all black), a 512GB Samsung SSD (black), and a 2TB Black Western Digital HD (silver/black).
- Location: Arizona
- Contact:
Re: Stats not updating yet again
From a PG member. No additional details available.
"IT was [...] to have migrated that server to a cloud service."
"IT was [...] to have migrated that server to a cloud service."
How to provide enough information to get helpful support
Tell me and I forget. Teach me and I remember. Involve me and I learn.
Tell me and I forget. Teach me and I remember. Involve me and I learn.
-
- Posts: 10179
- Joined: Thu Nov 29, 2007 4:30 pm
- Hardware configuration: Intel i7-4770K @ 4.5 GHz, 16 GB DDR3-2133 Corsair Vengence (black/red), EVGA GTX 760 @ 1200 MHz, on an Asus Maximus VI Hero MB (black/red), in a blacked out Antec P280 Tower, with a Xigmatek Night Hawk (black) HSF, Seasonic 760w Platinum (black case, sleeves, wires), 4 SilenX 120mm Case fans with silicon fan gaskets and silicon mounts (all black), a 512GB Samsung SSD (black), and a 2TB Black Western Digital HD (silver/black).
- Location: Arizona
- Contact:
Re: Stats not updating yet again
And stats are back up again.
Last updated: Wed Jul 13 10:00:13 PDT 2016
Last updated: Wed Jul 13 10:00:13 PDT 2016
How to provide enough information to get helpful support
Tell me and I forget. Teach me and I remember. Involve me and I learn.
Tell me and I forget. Teach me and I remember. Involve me and I learn.
-
- Site Admin
- Posts: 7927
- Joined: Tue Apr 21, 2009 4:41 pm
- Hardware configuration: Mac Pro 2.8 quad 12 GB smp4
MacBook Pro 2.9 i7 8 GB smp2 - Location: W. MA
Re: Stats not updating yet again
Actually they were back up by about noon on Tuesday PDT.7im wrote:And stats are back up again.
Last updated: Wed Jul 13 10:00:13 PDT 2016
iMac 2.8 i7 12 GB smp8, Mac Pro 2.8 quad 12 GB smp6
MacBook Pro 2.9 i7 8 GB smp3