Project Status Information?

[Fireball0236]

I'm quite a fan of numbers and information (so long as it's not actual math, and am quite curious ), and am wondering if there are any information pages about the status of the projects? The project summary page lists general information, and the server status lists the amount of WUs that can be handed out, though that number generally seems odd to me... Most servers I take a peek at the values for the WUs AVAIL, WUs to go and WUs WAIT columns are always the same... (or at least for the servers I check).

Anyway, is there any page that lists per project actually how many WUs are left? Or how many runs/gens and clones are planned? It would give me an idea of how long a project really runs for, and seeing a counter (of WUs left to process) go down gives the idea that progress is being made (more so then just seeing your personal WUs and points go up).

Then again, haven't found anything like that in my lots of browsing since I started folding, but figured it couldn't hurt to ask and wonder ^^.

---

Other than that, I hope you don't mind me adding a few tiny questions I have been wondering about in regards to WU.

From another topic:

Project: 10632 (Run 9, Clone 76, Gen 31) has been completed successfully for full credit by multiple other folders.

The same WU by multiple folders, isn't the only way a WU is reassigned if the original folder passed the preferred deadline? Would this mean multiple people couldn't/didn't completed the WU by the preferred deadline, and thus causing extra duplicate calculations? Or are there other reasons why a WU could be sent out more than once?

Secondly, as for a message site moderators sometimes post when it comes to WU problems:

Your WU (Pxxxx Rx Cx Gx) was added to the stats database on XXXX for YYY points of credit.

I don't suppose we can view our own personal log somewhere? It becomes hard to keep track of all WU when you have a dozen clients running .



Thanks,
~ Fireball0236

[PantherX]

In the Server Stats, if you see WUs RCV, it tells you how many WUs the Server is has received from folders, it updates every hour. Although not sure if you find this useful or not. The WUs are Serial meaning that it is X+1 hence are "infinite" unless it reaches a stage where it just can't fold, like temperature is out-of-range, etc. That is known as a bad WU as it can't be folded and is flagged by the Admin/Mod and that is the end of line for that "WU path" (unless the researcher can identify the problem and fix it) Usually the Projects run for a very long time and the research will check the results for the required data. If obtained, they may stop it or release a similar project to study the results further. Here is an explanation of PRCG:

So to summarize, when I'm setting up a project, I might do the following:

1. Pick 100 different unfolded or partially unfolded conformations of my protein of interest. These become my RUNs.
2. Then, I set up 100 different CLONEs for each RUN. (Well, I don't actually set them up myself, I just run a program. But I run it really well. And intelligently. And I look good doing it.) Each CLONE contains one WU at this point.
3. Then, I let the (100 RUNs) x (100 CLONEs) = 10,000 WUs loose on the world ("you guys").
4. Then, I go have lunch.
5. I come back weeks later to find WUs crunched and GENerations progressing -- each of the original 10,000 WUs was the beginning of one trajectory, so at the end, I have 10,000 trajectories of 50 or 100 or more ns.
6. Finally, I sift through the data and learn something new about protein folding!

Source

The only time a WU is assigned to another folder is when the original WU:
A) Gave an error. It can be UNSTABLE_MACHINE, EARLY_UNIT_ENDED, etc (IIRC, it is sent out twice for each error)
B) When the WU hasn't been returned before its preferred deadline

Only an Admin/Mod can view that information. Users can't view it even if they are the ones who have folded the WU. I usually check the Official Stats every hour and EOC Stats every 3 hours to keep track of my F@h Clients. Plus I am using HFM.NET which keeps a log off all the WUs the F@h Clients have done. Although, sometimes a few WUs may not end up being recorded but it covers >95% in my case.

[bruce]

Pantherx: That's a very good summary except for one fact. I believe the WU RCV are the WUs which have been receive but not yet credited, not is waiting to be received. I've never had it explained to me, but it makes sense since the Stats are updated hourly and that seems to correspond to the number being reset to zero.

Anyway, is there any page that lists per project actually how many WUs are left? Or how many runs/gens and clones are planned? It would give me an idea of how long a project really runs for, and seeing a counter (of WUs left to process) go down gives the idea that progress is being made (more so then just seeing your personal WUs and points go up).

In case you missed the conclusion that can be drawn from the information that PantherX quoted, it's impossible for any of us to know how many Gens are left on a specific project. The researcher may have some idea and assign a limit, but more likely he'll compare the progress that has been made with some goal that's not easily defined by a number of Gens. There are many scientific breakthroughs that will only be discovered if each trajectory is allowed to run for millions of Gens, and that's simply not possible, so they have to settle for learning what they can with a more practical number of Gens and report what has been learned up to that point.

There are also many scientific breakthroughs that have been and will continue to be discovered within the practical limits of today's computers. Suppose that everybody's computer was suddenly 100x faster than what we have today. (Moore's Law says that I'm talking about FAH about seven years in the future.) The researchers would probably aggregate maybe 10 or 20 of today's Gens into a new, larger Gen and they'd still run about as many new Gens as old Gens because of the same limits of practicality. They'd also study five to ten times as many proteins or they'd study proteins that are two to three times more complex (which means 5 to 10 times as difficult to compute).

Bottom line: There's no end in sight for FAH.

[PantherX]

Pantherx: That's a very good summary except for one fact. I believe the WU RCV are the WUs which have been receive but not yet credited, not is waiting to be received. I've never had it explained to me, but it makes sense since the Stats are updated hourly and that seems to correspond to the number being reset to zero...

Yikes, the part about recieved is correct and I have edited my post.

To test the part about the count being reset, what I did was before the 01:00 UTC Update of the F@h Stats, I check 171.64.65.56:
Before Stats Update, 85 WUS RCV
After Stats Update, 172 WUS RCV
This is the definition that I came across and it didn't mention anything about the counter being reset:

WUS RCVD shows how many WUs have been received since the last time the servers WUs were updated into the stats. This shows the relative number of WUs being received on the different servers (if all is fine) or which servers are not being inputted into the stats db if there is some problem.

It could be that the Server has some kind of problem so will keep an eye on it and see what happens

UPDATE -> So I followed-up with 02:00 UTC Update, here are the results for 171.64.65.56:
Before Stats Update, value fell from 172 to 108 WUS RCV
After Stats Update, value increased from 108 to 152 WUS RCV
I guess, that the calculation is more complex then initially perceived

[Fireball0236]

Thank you both for taking the time to explain!

It seems I had misinterpreted the meaning of Gen when I read that wiki page originally. I thought Gens were a subdivision of Clones (as Clones are to Runs). Of course, if they keep being generated, there is no count available.
I wasn't implying an end to FAH, though. Since if a project ends I'm sure there are a couple of new ones on the way.

Too bad we can't view our own log like that. I'll have to look into a program like HFM.NET then which can be used on both Windows and Linux, the Linux pc's being remote though (managing them through ssh isn't a problem)...


Thanks,
~ Fireball0236

[7im]

The Moderator log isn't that helpful anyway. It only shows the last WU completed by each of your clients. It is not a complete list.

And yes, the WU count is a moving target. Not helpful there either.

[Fireball0236]

One more small question, if I may.
Do WUs that end in ERROR 0x0's count towards the % of WUs that haven't been completed before the preferred deadline, in regards to getting bonus point for SMP WUs? Or are only SMP WUs used to calculate that %? (Not that I'm elegible for a bonus yet, as I haven't done 10 SMP WUs yet. That's something I'll be checking on this weekend, see if my laptop can fold SMP WUs on time).


~ Fireball0236

[PantherX]

If it errors and the F@h Client reports it to the Server and you are assigned a new WU, that WU is "ignored". If it does send some data that is scientifically valid, you may get partial points. However, for the Bonus, you need to complete 10 FahCore_a3 WUs before the Preferred Deadline and submit them to the Server

[codysluder]

Do WUs that end in ERROR 0x0's count towards the % of WUs that haven't been completed before the preferred deadline, in regards to getting bonus point for SMP WUs? Or are only SMP WUs used to calculate that %?

In the original definition of the bonus plan

For a Bonus, you must ... successfully return >=80% of assigned WU's,

I'm not sure anybody knows whether that counts ERROR returns or not. Does the word "successfully apply just to the return process or does it also include a requirement that FahCore calls the analysis compeleted with a FINISHED_UNIT CoreStatus = 64 (100) end? I think the Pande Group needs to clarify this point.

Successfully completed and uploaded: Good
WU expired and was not returned: Bad
WU had an error and the error report was returned: (Unknown)

[Fireball0236]

Thanks for the last replies. I did read them, but didn't know anything at the time to add to make this post more meaningful. Reusing this topic for a new question if I may, rather than making a new one.

I am, however, still curious to the exact answer to what codysluder said. A WU that was returned and given partial credit for, does that count as a "successful return" or not? (And what about the Linux client, where I've only gotten Error 0x0, but my client never said "Sending back what we have"?)

As for the new short question, any chance we could ever see our running % of successfully completed WUs on our profile page? With multiple clients it becomes hard(er) to keep track of the number of failed WUs (because of Error 0x0, or unforeseen events, like this weekend's server malfunction, and me losing 3-4 WUs that were waiting because the "Server reports problem with unit")...


~ Fireball0236

[codysluder]

After an EUE, the client tries to send back whatever it finds. After some EUEs, there is enough information to report the type of error, the particular WU, and the partial results. After other EUEs, there's not enough information to make such a report, such as when the checkpoint files are badly corrupted.

A feature that has been suggested several times is for the client to report PRCG, EUE-code, and User ID, even if there is no progress report. That would allow the servers to identify bad-WUs and save the Mods a lot of unnecessary work. That's not presently possible, but maybe it's in v7. (We can hope.) It could also identify unreliable hardware thought I expect that it would also need some big new database tables unless they're already doing something like that.

I don't know the answers to your questions but that has never stopped me from making a guess.

I'll bet that partial credit WUs are discarded and the WU is reissued.

They are calculating a percent completed for SMP using the passkey. As far as I know, they don't capture that information for GPU/CPU clients yet so it doesn't make sense to show that number on the website at this stage of development.