What 's up with the bigadv server(s) ?

noorman · Post by **noorman** » Tue May 31, 2011 7:54 pm

cfr. "no work to do" shit for at least two hours ...

report from a friend (who needed to hit the road) / so, I report it for him.

I see a few servers with 90+% CPU load !

.

Post by **kasson** » Tue May 31, 2011 7:59 pm

The CPU load is not relevant in this case (and also not the bigadv servers). There is currently less bigadv work than demand. If you are having difficulty obtaining work, we would suggest swapping to standard -smp for a while. Additional bigadv projects are currently in testing.

noorman · Post by **noorman** » Tue May 31, 2011 8:03 pm

thx for that swift reply; forwarded the message to him ...

CSM580 · Post by **CSM580** » Tue May 31, 2011 10:24 pm

Seems to me as though the simple law of supply and demand doesn't apply here. Nor do people matter who spend many thousands of dollars to support the cause. It is plain shortsightedness and ineffective planning on Stanford's part, plain and simple.

Post by **PantherX** » Tue May 31, 2011 10:40 pm

Welcome to the F@H Forum CSM580,

It seems that you aren't aware of the "full picture" so here are some bits that you may have missed:
1) Server Software being upgraded (Post)
2) Additional bigadv Projects entering Beta Testing soon (Thread)

I am sure that one you have the bigger picture, you can understand that this is a vital step to ensure that bigadv continues without any hiccups.

uncle fuzzy · Post by **uncle fuzzy** » Tue May 31, 2011 10:40 pm

...or people with less than ideal hardware taking so long to finish a WU that someone has to wait a bit for the next one.

7im · Post by **7im** » Tue May 31, 2011 11:02 pm

...Or so many people with less than ideal hardware taking so many WUs that someone has to wait a bit for the next one.

With such a high bonus in -bigadv, a lot more people are pulling the trigger on 2600s instead of the cheaper i7s.

noorman · Post by **noorman** » Tue May 31, 2011 11:18 pm

Systems (even farms) running 'idle' isn't a plus for the Project and not everyone has the time or inclination for altering switches in all of their Clients to enable their hardware to keep on 'Work'ing.
With today's energy prices, this should be taken in to account.
The Project should be thinking 'Green', trying to avoid unnecessary idling of anyone's hardware.
Since the switches are there to be used, many - with the hardware that is able to - chose to run 'bigadv' units and expect to have Work for their expensive setups.
.

7im · Post by **7im** » Tue May 31, 2011 11:31 pm

You know the "green" argument is very ironic, right? Which is greener? 10 Systems running at 100%, or 10 systems idling at 10%?

That ironic argument aside, I do agree with you. -bigadv should fall back to regular -smp automatically. No one should have to change any switches. But until that happens... there is a shared responsibility when one chooses to run high end hardware and software. Otherwise you should stop running SMP and run CPU work units.

The expense of the hardware statment is also ironic. One does expect Stanford to try to keep ALL systems churning WUs. And Stanford has good incentive to do so. No one wants to see any systems sitting idle. But also, if I had expensive hardware, I would also be doing my best to keep them churning WUs also. And that means if I have to change switches, then I do that myself, while waiting for Stanford to fix it. Again, it's a shared responsibility, otherwise you're the one letting those machines go to waste by not switching to -smp manually.

Remember, -bigadv is a trial program, not unlike software called beta. There will be some bumps in the road during the trial. And like the standard Stanford beta warning says, "DO NOT run a beta client if you or your machines cannot tolerate even the slightest instability or problems. Beta clients' and servers' performance may vary significantly from standard FAH clients during the development process, including but not limited to work unit shortages, server downtime for upgrades, short notice of client upgrades, and Points Per Day that differs a little or a lot from the developmental benchmark level."

I'm on your side, I'm just not sure that anything can be done about it right now. Stanford is in the middle of software updates all over the place.

k1wi · Post by **k1wi** » Tue May 31, 2011 11:38 pm

As with any project, be it a new building being built or rolling out a new type of bigadv project (-smp 12+), there can be some inconveniences involved with the development.

I'm confident that things will be up and running quickly, or else users will be able to adjust their clients in order to ensure they're folding to their 'potential'. After all, -bigadv is an 'advanced' flag, so users may have to "babysit" their machines from time to time. A couple of days of folding regular smp work units can't be too harmful can it? Especially if everyone else is doing regular -smp work units as well.

I know that any downtime affects PG just as much as it affects folders who have invested money into buying hardware to fold - after all, any downtime delays their research!

In the mean time, I think it's about time I did some maintenance on my bigadv machine

Post by **bruce** » Tue May 31, 2011 11:54 pm

CSM580 wrote:Seems to me as though the simple law of supply and demand doesn't apply here. Nor do people matter who spend many thousands of dollars to support the cause. It is plain shortsightedness and ineffective planning on Stanford's part, plain and simple.

Welcome to Foldingforum.org, CSM580.

You seem to think that Stanford's only purpose is to provide you with whatever you want. Sorry, but that's not the way it works. Stanford is running a series of research projects into how proteins fold. When a project has enough data, that project ends. When a new research project is started, new WUs are available. Those two types of events don't happen to suit your goals, they happen because of the needs of science. Stanford does not create unscientific work assignments just to keep your machine busy.

noorman wrote:Systems (even farms) running 'idle' isn't a plus for the Project and not everyone has the time or inclination for altering switches in all of their Clients to enable their hardware to keep on 'Work'ing.
With today's energy prices, this should be taken in to account.
The Project should be thinking 'Green', trying to avoid unnecessary idling of anyone's hardware.
Since the switches are there to be used, many - with the hardware that is able to - chose to run 'bigadv' units and expect to have Work for their expensive setups.

Shortages of -bigadv have happened before and will continue to happen from time to time. There's always a uproar until either new projects are released or everybody decides that no new projects are on the horizon.

Last time this happened, a few of suggestions were made to deal with the situation when a machine can't get a bigadv WU. As far as I know, none have been implemented, but if still doesn't hurt to discuss them and see what the Donors want to be done.

1) What do you think about randomly assigning standard SMP projects to people who have asked for -bigadv when there's a shortage of the preferred projects?
2) What do you think about dynamically reducing the bigadv bonus to reduce demand and make standard SMP more equitable?
3) What do you think about dynamically increasing the requirements for -bigadv (Give increasing priority to machines with higher numbers of core or larger RAM or machines that return WUs by increasingly wide margins ahead of deadlines)

the animal · Post by **the animal** » Wed Jun 01, 2011 2:16 am

7im wrote:-bigadv should fall back to regular -smp automatically. No one should have to change any switches. But until that happens...

Then something must have been changed in assigning WU because I did get regular SMP occasionally even as little as 4 days ago.

road-runner · Post by **road-runner** » Wed Jun 01, 2011 2:32 am

I see not much has changed around here

Do like I did, save your money dont buy any more hardware and when what you have is out of date (6 months from now) turn it off or sell it, find a new cheaper hobby...

huyetma · Post by **huyetma** » Wed Jun 01, 2011 2:36 am

RR, lol
I can feel your pain. I have 2 sr2 they all can not get bigadv.
Probably need to shutdown to save electric, while AS out of commission.

GreyWhiskers · Post by **GreyWhiskers** » Wed Jun 01, 2011 2:49 am

Good discussion. A couple of questions/comments:
1. re: kasson's announcement. I took that as saying the one server listed would be down. For the -bigadv WUs that I have processed, there are three or more different bigadv servers.
a. 130.237.232.141 for p6900
b. 130.237.232.237 for p6901
c. 171.67.108.22 for p2684/2685
d ??? for others

Are all of these servers to be out of service, or just 130.237.232.141, as noted in the announcement below??? That will say whether all of the bigadv folders need to change their flag NOW to get regular SMP WUs next, or whether the mix will change. e.g., give all of us 2684s to chew on for the next few weeks, if there are that many WUs queued up.

Re: Updates thread
by kasson » Tue May 31, 2011 9:10 am

bigadv server 130.237.232.141 is going into accept-only mode for a few days. We're getting ready to upgrade the server software there, and we can't have any outstanding jobs when we do that.

2. re: Bruce's post.

DISCLAIMER. I'm folding bigadvs on my new Sandy bridge i7 2600k system. This system folds a p6900/6901 bigadv, with only a moderate overclock, in 2.2 days vs the Stanford deadline of 4 days. The p2685s take a few hours longer, and the p2684s a lot longer, but still well within the established deadlines. I'm more than willing to abide by whatever decision is made to best further the science, and hope that objective value functions are used to make these decisions.

My view of FAH is that there is a finite but large set of things to be studied - driven by the capability and capacity of the researchers front end preparation and back end processing of results/issuing next-gen WUs. There may be some problems that become "solved", and we should celebrate just that (HINT: it would be very good for PL to let the donors know when one series of projects has come to an end - that the set of trajectories has taken a particular problem set to a logical conclusion.)

I think there are definitely things to be said for the options. I myself think that the PROJECT would be better served by implementing something like #3. I know there was a recent thread asking about the assignment server being able to make decisions based on the Performance Fraction (to use a v6 term) - to preference assignment to those best able to do it.

But, it is an interesting queuing problem. the PROJECT wants the best and fastest completion of its study trajectories.

(Disclaimer. I'm just pulling all of these numbers out of the air for talking purposes. I have no idea how many there really are. The conclusions will differ depending on the actual pouplations)
If there are 10,000 WUs, should they all be reserved for the, say, 100 users who can complete one of the 2684s in less than a day, thus taking ~ 100 wall-clock days to complete the 10,000 WUs, or should the, say, 500 users who can process one of the 2684s in 2.5 days (which still beats the deadline by 1.5 days) be also folding? There is a serialization of the trajectories to consider too - the results of one generation needed to seed the subsequent generations.

a. 100 fastest only complete 10,000 WUs in 100 days
b. 100 fastest plus 500 i7s complete 10,000 WUs in 33 days? (I apologize for what may be fuzzy math.)

What exactly, then, is Stanford's value proposition for the completion?

I threw out somewhat random numbers - but those who know what the real numbers can use an analysis something like this to help determine what the value function is.

Along the way, if we do tweak with the QRB and tweak with the deadlines (say 3 days vs 4 days), we want to be able to up front weed out the incapable folders. The idea of preferencing, or even filtering, the users who can complete with the best performance fraction is good - but we don't want to cut the pool of folders so low that the net result is delaying the science. That's why Stanford needs to articulate their REAL capacity to generate new projects, and process the incremental returns.

It's not rocket science to come up with preferencing formulas. That's a good classroom assignment for undergrads. But the system does need to articulate its value function. i.e., what's the value of finishing an arc of 10,000 bigadv WUs in 33 days vs 100 days for the assumptions I've made? What's the value function for the REAL population of folders we are seeing? The "folder in the street" like me see none of those stats, nor do I think I really want to. But, someone knows them.

Re: What 's up with the bigadv server(s) ?
by bruce » Tue May 31, 2011 4:54 pm

...

1) What do you think about randomly assigning standard SMP projects to people who have asked for -bigadv when there's a shortage of the preferred projects?
2) What do you think about dynamically reducing the bigadv bonus to reduce demand and make standard SMP more equitable?
3) What do you think about dynamically increasing the requirements for -bigadv (Give increasing priority to machines with higher numbers of core or larger RAM or machines that return WUs by increasingly wide margins ahead of deadlines)