Daily stats update timed 15:00

[AJMSmith]

The appears to be a problem here ... with the file showing no update from the noon GMT update (actual time of file might be slightly earlier). By that I mean that the noon and 15:00 GMT update show no differences.

It's not just a one off ... it seems to happen almost every day.

[uncle fuzzy]

Which stats are you looking at? I use EOC, which updates every 3 hours. The official Stanford stats update every 2 hours. You'll have blank spots from this. I usually see a big zero at noon and midnight, Eastern Daylight Time. It gets caught up the next update.

[AJMSmith]

Which stats are you looking at? I use EOC, which updates every 3 hours. The official Stanford stats update every 2 hours. You'll have blank spots from this. I usually see a big zero at noon and midnight, Eastern Daylight Time. It gets caught up the next update.

The stats get updated every 2 hours and the 3rd party user file every 3. This means that there are 3 stat updates per 2 third party file updates. Now admittedly this means that the updates contain a alternating sequence of 2 and 4 hours work ... but you should never see a nil update ... and why only once a day? The problem only seems to happen with the 15:00 GMT update (08:00 PST)

BTW I use KaKao's stats. But it should make no difference as they should all be using the same update files.

[bollix47]

It appears that the 6am PST update at Stanford takes 2-3 hours(may only be 2 but that takes things into the next update) and as a result the flat files for the 3rd party stats are not being created in time for them to be picked up by kakao and o/c for their stats. That's creating the zeroes for everyone once a day at a time which depends on where you are in the world.

If there was another way of organizing that 6am update so that the stat files were produced earlier in the process then I'm sure Pande would have done so and we would not be seeing the zeroes, but without their input to this thread we may never know.

[AJMSmith]

It appears that the 6am PST update at Stanford takes 2-3 hours(may only be 2 but that takes things into the next update) and as a result the flat files for the 3rd party stats are not being created in time for them to be picked up by kakao and o/c for their stats. That's creating the zeroes for everyone once a day at a time which depends on where you are in the world.

If there was another way of organizing that 6am update so that the stat files were produced earlier in the process then I'm sure Pande would have done so and we would not be seeing the zeroes, but without their input to this thread we may never know.

So ... thought ... if they delayed the extra processing for the 6am update to 8am ... or brought it forward to 4am then we would have a smoother line of updates as both the 4am and 8am updates have a 4 hour window afterwards before then next 3rd party file is generated whereas the 6am update only has a 2 hour window.

Alternatively the 2 hourly updates could be done at 90 minute intervals instead with the third party file being produced as the last act of every other update.

[Archangelboy]

I am currently looking around for answers to this question. Can an Admin or somebody from Pande/Stanford please comment on the delay period between when an update on Stanford's servers is completed, and when it's captured (for lack of any more technical a term) by the stats update sites out there? It seems to me that generally a one or two-update delay is typical, excpet for the morning (U.S. time) update, which generally catches up with Stanford, and then a goosegg at the noon (EOC) update because the updates done between the 9am EOC update (7 am Stanford) and the noon update (10 am Stanford time) aren't being released for capture by Stanford.

Comments from Admin/Pande/somebody who knows what the actual process is and why it's delaying would be most enlightening.

[codysluder]

I don't think you'll get the answer you're looking for from and Admin/Pande.

The servers at Stanford start processing the stats on a regular schedule. One part of that update creates files that can be downloaded by those who run 3rd party stats sites. Each 3rd party stat site creates their own method to detect, download, and process those files and to avoid reprocessing the same data if Stanford's update happens to be delayed. The Stanford is not responsible for any of that latter process. They are only responsible for the updates on their own site.

Contact the admin of EOC or whatever other sites you're interested in.

[Nathan_P]

I use both sites and both of them read a zero point update once a day as does hardfolding.com. Surely this means that the stats are taking too long to process and thus the sites can't process them in time?? FWIW its the 15.00 UTC update that seems to go awry

[bruce]

Codysluder is correct. What did the administrator of EOC tell you?

Contact the admin of EOC or whatever other sites you're interested in.

[bollix47]

http://folding.extremeoverclocking.com/

[AJMSmith]

I don't think you'll get the answer you're looking for from and Admin/Pande.

The servers at Stanford start processing the stats on a regular schedule. One part of that update creates files that can be downloaded by those who run 3rd party stats sites. Each 3rd party stat site creates their own method to detect, download, and process those files and to avoid reprocessing the same data if Stanford's update happens to be delayed. The Stanford is not responsible for any of that latter process. They are only responsible for the updates on their own site.

It would be sensible if it WAS the case that the generation of the third party download WAS part of the the update. It's not (it used to be that the update was run once an hour and the files generated every three runs). Now the updates are every two hours but the files are generated every three with the files getting alternately 2 and 4 hours worth of updates.

I've already suggested 90 minute updates ... and then generate the file at the end of the update if the update started on the hour.

The one thing that puzzles me is why less a less frequent update schedule should reduce the time spent doing the updates. The same amount of work needs processing regardless ... and so it should take roughly the same to process the work.

[Archangelboy]

lol Jason at EOC (a great guy, if his e-correspondance is any indication) has nothing useful to say yet except, "Stanford updates their stats in real-time, and we here at EOC only update every 3 hours." This statement blew me away, as it indicates a vast misunderstanding of how the stats he polls from even function. Nobody not from Stafnord/Pande and not involved with the code behind what results are updated when, would know, hence my request for horses mouth input.

AFAIK, points are accrued in real time, but only 'updated' every two hours through Stanford. What makes me think about some level of readiness to be released is that fairly frequently (ok, in my observation always except the 6-8AM Stanford updates) the stats compilation sites out there are behind the most recent update, making me wonder what the delay is between when a point/WU level is reached at Stanford, and when EOC or Kakao can harvest that information from Stanford.

So when (again, between the 6 and 8 oclock Stanford updates, PST) the stats sites catch up, the release valve has been hit on the latest update, and so the 1500 UTC (Kakao) or Noon (EOC) update comes up a goose egg. Take a look at the pattern, Kakao does a better job of keeping track of a history of visible updates for days back. If you've got time, notice the 1200 UTC/0900 CST updates (KS and EOC, respectively). They equal Stanford's total. Then nothing at noon/1500 UTC. I just want to know what gives, out of curiosity. I don't really worry that the nooner is nil, I just wonder about the process.

[VijayPande]

I don't think you'll get the answer you're looking for from and Admin/Pande.

The servers at Stanford start processing the stats on a regular schedule. One part of that update creates files that can be downloaded by those who run 3rd party stats sites. Each 3rd party stat site creates their own method to detect, download, and process those files and to avoid reprocessing the same data if Stanford's update happens to be delayed. The Stanford is not responsible for any of that latter process. They are only responsible for the updates on their own site.

It would be sensible if it WAS the case that the generation of the third party download WAS part of the the update. It's not (it used to be that the update was run once an hour and the files generated every three runs). Now the updates are every two hours but the files are generated every three with the files getting alternately 2 and 4 hours worth of updates.

I've already suggested 90 minute updates ... and then generate the file at the end of the update if the update started on the hour.

The one thing that puzzles me is why less a less frequent update schedule should reduce the time spent doing the updates. The same amount of work needs processing regardless ... and so it should take roughly the same to process the work.

The main reason why the main stats update does not trigger the file for 3rd party updates is that the latter takes some time to build and we don't want that to delay the main stats update.

[Nathan_P]

Does that mean then that for some reason the 15.00 update takes longer to process than is normal??