STAT updates have stalled.

[cine.chris]

Stat feeds have stalled... output from my feed timestamp tracker log.
20210227-010001
Sat Feb 27 04:04:26 GMT 2021
TimeStamp FAIL: EXITING

[drougnor]

I can confirm that the flat files are no longer updating as of 11pm EST on 2/26 (04:04 GMT, 2/27).

d

[smoking2000]

It works in a browser, but not with scripts (Perl LWP):

Code: Select all

$ HEAD -USe https://apps.foldingathome.org/teamstats/team92.txt
HEAD https://apps.foldingathome.org/teamstats/team92.txt
User-Agent: lwp-request/6.36 libwww-perl/6.36

500 Server closed connection without sending any data back
Content-Type: text/plain
Client-Date: Sat, 27 Feb 2021 14:21:58 GMT
Client-Warning: Internal response

Code: Select all

$ GET -USe https://apps.foldingathome.org/teamstats/team92.txt 
GET https://apps.foldingathome.org/teamstats/team92.txt
User-Agent: lwp-request/6.36 libwww-perl/6.36

500 Can't connect to apps.foldingathome.org:443 (SSL connect attempt failed)
Content-Type: text/plain
Client-Date: Sat, 27 Feb 2021 14:22:32 GMT
Client-Warning: Internal response

Can't connect to apps.foldingathome.org:443 (SSL connect attempt failed)

SSL connect attempt failed at /usr/share/perl5/LWP/Protocol/http.pm line 50.

The GET request sometimes works, but most times it fails.

[cine.chris]

I was working on the tracker to follow the shifting updates.
Inspired to utilize the hourly updates, I wrote a fast updater.
Fast & phone friendly.. no ads, no graphics, no trackers.
Hopefully, they don't revert to the old 3hr cycle.

If people are interested, I'll upload the reduced active data file for download.

Page for my TechPowerUp friends

[drougnor]

Stats are updating again, just FYI

[Xavier Zepherious]

Is stat server down again
EOC is not reporting

[Neil-B]

Is stat server down again
EOC is not reporting

I'm seeing updates in EOC ... haven't checked they are correct yet.

[Xavier Zepherious]

it was very late like 5:38
i was checking at regular intervals every few mins

it might mean that EOC will update later now from last stat server freeze

[Neil-B]

There does seem to be a bit of varience in length of time to process - and possibly some minor(ish) differences between official stats and EOC that aren't readilly understandable ... I'll be giving it a day or so to stabilise as I guess there is a bit of catchup to resolve/balance out

[Joe_H]

EOC downloads a copy of the team and donor lists every 3 hours. For a while pre-COVID those files were generated every hour, so at worst of the creation fo the file took a bit longer than normal, the file EOC would get would only be an hour old.

Due to the loads on the database and front end server during the peaks of WU submission, the list files were taking longer than an hour to create. The frequency was reduced to every 4 hours. Jason from EOC contacted the person handling the stats server setup, and he agreed that every 3 hours would also work. As of the last I checked, that was still the frequency for creating those files.

Also during the peak period last year much of the stats API based checks were disabled. I have not looked to see which have been reenabled if any. I am assuming cine.chris is using the donor and team lists that can be downloaded from links on this page - https://foldingathome.org/statistics/, any others are not guaranteed to be always available. The page itself was not updated to reflect the change from every hour to every 3 hours last year, if they have returned to generating those files more frequently, that will show up in the timestamps.

Timestamp of the Team List I just downloaded is Sun Feb 28 16:54:58 GMT 2021, just under an hour old.

Edit: and just downloaded the team list again, its time stamp is about an hour later than the previous, so it appears they have gone back to hourly updates. No idea when the update starts.

[smoking2000]

The update frequency of the stats files not particularly important, clients should use the If-Modified-Since to only download the files if they were modified on the server. That can be executed every hour with the actual processing of the data kicking in after there was something to download.

The problem was 500 Internal Server Error responses for something that should be a flat file.

[Joe_H]

The problem was 500 Internal Server Error responses for something that should be a flat file.

Yes, that was fixed yesterday by the time of the EOC 3 PM CST update. My response was to the bits about EOC and stats updating frequency.

[Xavier Zepherious]

EOC updates now at 32 mins past the hour every 3 hrs
was something like 15m after before

so other than it's an annoyance that the time changed... it's fine

just sort make people go ??? every time the time drifts on it

begs the question is it EOC fault or the server

[Joe_H]

Just the number of entries EOC is needing to process from the flat files. The update still starts on the hour every 3 hours, it just is taking longer to process, especially since Jason increased the number of teams and users tracked.

[cine.chris]

Timestamps, since the restart, have drifted about 3 minutes, settling in at 52:25 initially, currently 55:09.
So, it appears, the drift of ~3.6min/day that I was seeing, is still occurring.
But, that is a different thread. The stall is fixed.