PG announcement about Big Advanced Projects
Moderators: Site Moderators, FAHC Science Team
-
- Posts: 3
- Joined: Sat Jun 11, 2011 3:48 pm
- Hardware configuration: -2600k @ 4.7ghz - Windows BigAdv
-2600k @ 4.6ghz - Windows BigAdv
-2x E5649 SR-2 @ 3.46ghz - Gentoo 11 - BigAdv
-1x G34 dodeca @ 3.0 ghz - Ubuntu 10.10 - BigAdv
Re: Really
I have to believe that those complaining about this change were not around when bigadv kicked off or they wouldn't be saying this. These WU's are supposed to be cutting edge stuff, not things that a single $200 desktop cpu can do. The fact that you don't have a 16 core cpu is the whole point, they need to be able to write tough bigadv WU's without worrying that they will get stranded on some thread spoofed quad core.
Having said that, SMP k-factors and deadlines will need revision to make this work. You will either need a way to make SMP scale from slow dual cores up to SB-E chips or move the SMP time lines up as well to make it more competitive. If Stanford doesn't move SMP upward then the gulf between the multiproc guys and everyone else will only get larger, and since we know Trickle Down PPD is not going to work I can only assume you will be faced with an Occupy PG movement...
I am confident that this can be accomplished and definitely appreciate all the work Stanford has done with the Donor Advisory Board prior to making this change, this is handled much better than the previous points reduction was. Implement the SMP increase properly (and leave GPU alone) and I think the net result will be pretty amicable to most of the community while still achieving the intent of the change.
Having said that, SMP k-factors and deadlines will need revision to make this work. You will either need a way to make SMP scale from slow dual cores up to SB-E chips or move the SMP time lines up as well to make it more competitive. If Stanford doesn't move SMP upward then the gulf between the multiproc guys and everyone else will only get larger, and since we know Trickle Down PPD is not going to work I can only assume you will be faced with an Occupy PG movement...
I am confident that this can be accomplished and definitely appreciate all the work Stanford has done with the Donor Advisory Board prior to making this change, this is handled much better than the previous points reduction was. Implement the SMP increase properly (and leave GPU alone) and I think the net result will be pretty amicable to most of the community while still achieving the intent of the change.
Re: Really
I really feel for Stanford - every time they make an adjustment or make a change in order to improve their science based on their needs, these forums go ballistic...
I completely understand Pande Group's justification for this. My i7 920 won't be able to fold BigAdv, but it's still perfectly good for folding regular SMP work units. Furthermore, when they reduced the 'bonus' for folding BA they reduced the diss-incentive for folding SMP - folding on an 8 or 12 'core' machine is still going to earn you a lot of points on SMP.
I'm sure that Pande and the DAB - that's Donor Advisory Board (they represent our interests and opinions) - has done a lot of discussion about this and I honestly don't know that 'consulting with the wider community on a one-to-one basis would be of any use. All it would achieve is a lot different opinions and invoke Godwin's law.
In my humble opinion and analysis of what has developed over the past while is that there has been downward creep occuring in the BA field - BA should continue to be the very top percent of machines and over time that percent has grown and grown. It's a simple fact of computer evolution. As a result, SMP work production has declined and BA work units have come into short supply. In order to rectify the situation there needs to be an adjustment and a recalibration of the system - what good is it if 40% of SMP capable machines are running BA if BA only represents 10% of Pande's science? It's better for all if the recalibration brings those numbers back into line and the seriously fast machines start hitting BA after BA.
k1wi
I completely understand Pande Group's justification for this. My i7 920 won't be able to fold BigAdv, but it's still perfectly good for folding regular SMP work units. Furthermore, when they reduced the 'bonus' for folding BA they reduced the diss-incentive for folding SMP - folding on an 8 or 12 'core' machine is still going to earn you a lot of points on SMP.
I'm sure that Pande and the DAB - that's Donor Advisory Board (they represent our interests and opinions) - has done a lot of discussion about this and I honestly don't know that 'consulting with the wider community on a one-to-one basis would be of any use. All it would achieve is a lot different opinions and invoke Godwin's law.
In my humble opinion and analysis of what has developed over the past while is that there has been downward creep occuring in the BA field - BA should continue to be the very top percent of machines and over time that percent has grown and grown. It's a simple fact of computer evolution. As a result, SMP work production has declined and BA work units have come into short supply. In order to rectify the situation there needs to be an adjustment and a recalibration of the system - what good is it if 40% of SMP capable machines are running BA if BA only represents 10% of Pande's science? It's better for all if the recalibration brings those numbers back into line and the seriously fast machines start hitting BA after BA.
k1wi
-
- Site Moderator
- Posts: 2850
- Joined: Mon Jul 18, 2011 4:44 am
- Hardware configuration: OS: Windows 10, Kubuntu 19.04
CPU: i7-6700k
GPU: GTX 970, GTX 1080 TI
RAM: 24 GB DDR4 - Location: Western Washington
Re: Really
Hmm. That's an interesting idea. I think the problem though is the fluctuation of WUs on the same hardware, and WUs coming into F@h. Please read this post, Dr. Pande explains his reasoning and communication behind a sudden change that happened back in July: http://folding.typepad.com/news/2011/07 ... units.htmlXavier Zepherious wrote:what Vijay has to do... is slowly implement changes and phase out older 8 core 16 threaded servers as well as the high end enthusiast...do it gradually...slowly squeezing point system down
keep the bottom fixed
based on time to get it in for bonus(preferred).... not on cores
when people stop making deadlines - then they will either adapt/improve hardware or migrate the old to SMP
F@h is now the top computing platform on the planet and nothing unites people like a dedicated fight against a common enemy. This virus affects all of us. Lets end it together.
-
- Posts: 140
- Joined: Fri Jan 21, 2011 8:02 am
Re: Really
Jesse_V wrote:If you've read Dr. Pande's previous post http://folding.typepad.com/news/2011/11 ... d-dab.html it describes a Donor Advisory Board which is designed to improve communication and transparency. You'll notice that its goals are "What can PG do to help improve communication?" and "How to make PPD more consistent". Understand, it is probably very difficult for them to make a WUs have consistent PPD. Each simulation has different numbers of atoms, uses the same architecture in varying ways, and performs differently on different hardware. The Kfactor would help with this, but I believe we need to get a lot more data on this. The change is probably based on a large sample of long-term patterns donors take. It's likely many of the admins that monitor and reply in these topic were noting what they saw. It's been my experience that when donors propose a change by using data, logic, and neutral tones, their chances of making an impact are much higher. Let's discuss your ideas as such, and perhaps they'll be heard somehow. We'll see.Simba123 wrote:ok, fine, I understand to a degree that Stanford wants BA results in quicker and more people folding SMP.
With this change they really, really need to up the Kfactor or maybe include an 'additional' bonus if an SMP unit is returned with more than 95% completion time (my 2600k can do that, so I reckon most of the 9xx can too).
If they do that, they will keep the competition going, and maybe not lose too many folders.
This has been very poorly implemented to say the least.
The Advance notification of the time frame for the change is nice, the lack of notification for what is/was under discussion is NOT.
They (Pande and Stanford), need to remember that without us their work would grind to a halt, and they would all be out of a job.
They need to treat us with a LOT more respect,
There are millions of donors world-wide who collectively would have donated BILLIONS in hardware and electricity to support their project over the years.
Angering (I'll put it that way, saying what many people really think of this decision would get this post deleted) people off like this is not going to do them any good.
Adding an additional bonus would not be too difficult to add to the formula, and if they get on it and at least do something to alleviate the points nerf from not being able to fold BA
before the new requirements come into effect, they may keep a few more folders.
They are very much aware of how valuable we are. Dr. Pande has mentioned it numerous times on his blog, and its pretty much a well known fact. What would you propose that they do? How would they notify us? What fraction of those that would care or be impacted would respond? Would the decision from such a survey positively affect science? And I'm not in any way trying to downplay your importance, but aren't the bigadv people a very small fraction of F@h? It's my understanding that each scientist who launches their projects sets the k-factor, so Dr. Pande is not directly responsible there. Adding an additional bonus is very unlikely on these grounds alone, as you would have to update all the WUs and the servers, and I don't know about the WUs but I know server updates are very difficult and cause downtime and other issues, which negatively affects science. There's a lot of factors here that come into play for any change.
you have the Donar Advisory Board (DAB) ...
you are suppose to talk to them they get feedback from community and it gets sent back thru them to FAH
You make the discussions open transparent - with plenty of lead time
In this case you could have explained the new points system or have something worked out(with DAB and community) to explain to us BEFORE announcing the changes and BEFORE implementation
By including Us in the discussion from the get go you would alleviate concerns and probably get better ideas and solutions
At the least you would have better PR
Re: Really
Not sure where some of you were, but people at the [H] knew changes were coming over a week ago.
-
- Posts: 43
- Joined: Thu Aug 06, 2009 4:07 am
- Hardware configuration: Rig1: Asus Z8PE-D12X/Dual Xeon X5675 3.06 Ghz
Rig2: Asus Z8NA-D6/Dual Xeon E5620 2.4 Ghz
Rig3: Asus Z8NA-D6C/Dual Xeon X5670 2.93 Ghz
Rig4: Asus Z8NA-D6C/Dual Xeon E5649 2.53 Ghz
Re: Really
Uhhh... we knew it was coming too and have been discussing it. Now that it has been officially announced Today... We are discussing it here.kendrak wrote:Not sure where some of you were, but people at the [H] knew changes were coming over a week ago.
-
- Posts: 3
- Joined: Sat Jun 11, 2011 3:48 pm
- Hardware configuration: -2600k @ 4.7ghz - Windows BigAdv
-2600k @ 4.6ghz - Windows BigAdv
-2x E5649 SR-2 @ 3.46ghz - Gentoo 11 - BigAdv
-1x G34 dodeca @ 3.0 ghz - Ubuntu 10.10 - BigAdv
Re: Really
I think he was speaking specifically to those complaining that there had been no advance warning.Horvat wrote:Uhhh... we knew it was coming too and have been discussing it. Now that it has been officially announced Today... We are discussing it here.kendrak wrote:Not sure where some of you were, but people at the [H] knew changes were coming over a week ago.
If everyone can just adopt the mindset that in order to run bigadv you need dual processors or higher then there will be much less complaining when this changes again in the future. The fact people were able to run it on single processors of any speed was meant to be the exception, not the rule.
-
- Posts: 1024
- Joined: Sun Dec 02, 2007 12:43 pm
Re: PG announcement about Big Advanced Projects
How can you say you didn't get any warning?
Today you received a two-month advanced warning that the policy will change.
Today you received a two-month advanced warning that the policy will change.
-
- Posts: 43
- Joined: Thu Aug 06, 2009 4:07 am
- Hardware configuration: Rig1: Asus Z8PE-D12X/Dual Xeon X5675 3.06 Ghz
Rig2: Asus Z8NA-D6/Dual Xeon E5620 2.4 Ghz
Rig3: Asus Z8NA-D6C/Dual Xeon X5670 2.93 Ghz
Rig4: Asus Z8NA-D6C/Dual Xeon E5649 2.53 Ghz
Re: Really
That is a question I still cannot get answered, will a dual quad core setup with H/T meet the new reqwuirements?If everyone can just adopt the mindset that in order to run bigadv you need dual processors or higher then there will be much less complaining when this changes again in the future. The fact people were able to run it on single processors of any speed was meant to be the exception, not the rule.
-
- Site Moderator
- Posts: 2850
- Joined: Mon Jul 18, 2011 4:44 am
- Hardware configuration: OS: Windows 10, Kubuntu 19.04
CPU: i7-6700k
GPU: GTX 970, GTX 1080 TI
RAM: 24 GB DDR4 - Location: Western Washington
Re: Really
I think we are getting a bit mixed up here. The Pande Group offers a couple of different setups for multi-core processors. You can run just SMP, which makes some great scientific productions and needs a minimum of 2 powerful physical processing cores to complete the WUs on time, according to this page: http://folding.stanford.edu/English/FAQ-SMP Then there is bigadv, which as we all know will in two months require a minimum of 16 cores, and I believe it currently requires 8. I believe Horvat means "SMP" not "bigadv" but maybe I'm missing something.Horvat wrote:That is a question I still cannot get answered, will a dual quad core setup with H/T meet the new reqwuirements?If everyone can just adopt the mindset that in order to run bigadv you need dual processors or higher then there will be much less complaining when this changes again in the future. The fact people were able to run it on single processors of any speed was meant to be the exception, not the rule.
F@h is now the top computing platform on the planet and nothing unites people like a dedicated fight against a common enemy. This virus affects all of us. Lets end it together.
-
- Posts: 140
- Joined: Fri Jan 21, 2011 8:02 am
Re: PG announcement about Big Advanced Projects
the fact that a dual xeon(4c/8t) at stock is slower than a sandy-e 6 core overclocked doesn't make this a fair system either
it should be based on time to complete the WU...not on the hardware of your system
a old xeon system gets to run and complete the bigadv wu slower than an Overclocked sandy-e could but the sandy-e not allowed to do them
I thought this was about getting the jobs in quicker... not slower
What I was referring to in my previous post is that IF your are gonna make changes have the solutions in hand and posted for everyone to see (like exactly what the point system will be)
rather than taking your word your fixing it and then you mess it up again infuriating more people
the solution - new points system worked out ahead of time to alleviate concerns of people....all we got is we are looking at fixing it.
seems we were not notified of changes and our DAB person caught of guard
the whole intent of the system is for feedback both ways... listen not to just those on the board but to the users too
I can understand slowly moving older hardware to SMP... I agree with it
but do it fairly based on system that can turn the wu's in quick enough...not on the hardware they have
keeping old xeons servers - allowed to do them while not allowing new hardware that completes them sooner isn't fair either
doing based on time to completion is. that's the whole goal of the point system. Entice people to finish quicker
shorten preferred times and thus the bonuses... if it more advantages for older systems to move to straight SMP and earn more point there they will
unless you have the solution(points system changes) at hand don't change the system.
you currently aren't that's nice but until you have a proper solution don't change it...delay it then.. til you do
next time have it all worked out before springing bad news on us all....ie bad news- good news
like
bad news - bigadv times are made shorter so older machines like 2600k won't finish in time
good news - is we have a improve SMP points change and here it is...and post it
all you did today is bring bad news ...leave a sour taste in peoples mouth and maybe lost a quite a few good folders
it should be based on time to complete the WU...not on the hardware of your system
a old xeon system gets to run and complete the bigadv wu slower than an Overclocked sandy-e could but the sandy-e not allowed to do them
I thought this was about getting the jobs in quicker... not slower
What I was referring to in my previous post is that IF your are gonna make changes have the solutions in hand and posted for everyone to see (like exactly what the point system will be)
rather than taking your word your fixing it and then you mess it up again infuriating more people
the solution - new points system worked out ahead of time to alleviate concerns of people....all we got is we are looking at fixing it.
seems we were not notified of changes and our DAB person caught of guard
the whole intent of the system is for feedback both ways... listen not to just those on the board but to the users too
I can understand slowly moving older hardware to SMP... I agree with it
but do it fairly based on system that can turn the wu's in quick enough...not on the hardware they have
keeping old xeons servers - allowed to do them while not allowing new hardware that completes them sooner isn't fair either
doing based on time to completion is. that's the whole goal of the point system. Entice people to finish quicker
shorten preferred times and thus the bonuses... if it more advantages for older systems to move to straight SMP and earn more point there they will
unless you have the solution(points system changes) at hand don't change the system.
you currently aren't that's nice but until you have a proper solution don't change it...delay it then.. til you do
next time have it all worked out before springing bad news on us all....ie bad news- good news
like
bad news - bigadv times are made shorter so older machines like 2600k won't finish in time
good news - is we have a improve SMP points change and here it is...and post it
all you did today is bring bad news ...leave a sour taste in peoples mouth and maybe lost a quite a few good folders
-
- Posts: 43
- Joined: Thu Aug 06, 2009 4:07 am
- Hardware configuration: Rig1: Asus Z8PE-D12X/Dual Xeon X5675 3.06 Ghz
Rig2: Asus Z8NA-D6/Dual Xeon E5620 2.4 Ghz
Rig3: Asus Z8NA-D6C/Dual Xeon X5670 2.93 Ghz
Rig4: Asus Z8NA-D6C/Dual Xeon E5649 2.53 Ghz
Re: Really
No, I mean any -bigadv. Why is this being such an elusive answer? I want a plain, simple answer.. Yes or no. Anyone... please. To run BA, 16 physical cores or 16 logical cores.Jesse_V wrote:I think we are getting a bit mixed up here. The Pande Group offers a couple of different setups for multi-core processors. You can run just SMP, which makes some great scientific productions and needs a minimum of 2 powerful physical processing cores to complete the WUs on time, according to this page: http://folding.stanford.edu/English/FAQ-SMP Then there is bigadv, which as we all know will in two months require a minimum of 16 cores, and I believe it currently requires 8. I believe Horvat means "SMP" not "bigadv" but maybe I'm missing something.Horvat wrote:That is a question I still cannot get answered, will a dual quad core setup with H/T meet the new reqwuirements?If everyone can just adopt the mindset that in order to run bigadv you need dual processors or higher then there will be much less complaining when this changes again in the future. The fact people were able to run it on single processors of any speed was meant to be the exception, not the rule.
Re: PG announcement about Big Advanced Projects
As I said in the other thread you created - it's 11pm in California, I don't think Pande Group should be required to respond to your message at such a time in the evening.
-
- Site Moderator
- Posts: 2850
- Joined: Mon Jul 18, 2011 4:44 am
- Hardware configuration: OS: Windows 10, Kubuntu 19.04
CPU: i7-6700k
GPU: GTX 970, GTX 1080 TI
RAM: 24 GB DDR4 - Location: Western Washington
Re: Really
Clearly 16 physical cores does bigadv much faster than 16 logical cores. But I believe the servers see the number 16 either way, which may be why it isn't specified. In light of that, I'd say that 16 logical cores will be sufficient, but 16 physical cores is better.Horvat wrote: No, I mean any -bigadv. Why is this being such an elusive answer? I want a plain, simple answer.. Yes or no. Anyone... please. To run BA, 16 physical cores or 16 logical cores.
F@h is now the top computing platform on the planet and nothing unites people like a dedicated fight against a common enemy. This virus affects all of us. Lets end it together.
-
- Posts: 43
- Joined: Thu Aug 06, 2009 4:07 am
- Hardware configuration: Rig1: Asus Z8PE-D12X/Dual Xeon X5675 3.06 Ghz
Rig2: Asus Z8NA-D6/Dual Xeon E5620 2.4 Ghz
Rig3: Asus Z8NA-D6C/Dual Xeon X5670 2.93 Ghz
Rig4: Asus Z8NA-D6C/Dual Xeon E5649 2.53 Ghz
Re: PG announcement about Big Advanced Projects
I hope the Pande group doesn't come to an abrupt stop, you'll become momentarilly blind by the loss of light.k1wi wrote:As I said in the other thread you created - it's 11pm in California, I don't think Pande Group should be required to respond to your message at such a time in the evening.