Page 2 of 10
Re: Project 5801 issues.
Posted: Tue Oct 28, 2008 6:22 pm
by Teddy
Same here 5801 has paused every GPU client that I have for 24hrs... & these are all well cooled, stock running video cards..
What a mess Stanford...
Going to reassess my contribution to the Folding@Home project after this mess, Sorry Stanford!
Today there are going to be some very unhappy Forum members here @ Overclockers Australia.
Teddy
Re: Project 5801 issues.
Posted: Tue Oct 28, 2008 6:25 pm
by road-runner
Seems like 5801 would have at least been tested one time....
Re: Project 5801 issues.
Posted: Tue Oct 28, 2008 6:30 pm
by Flathead74
I can now be counted among those whose GPU folders are out of service.
Up until this load of WUs arrived I was running fairly well, even considering the slow server issue.
Now I'm completely down!
Thank you.
Code: Select all
[17:35:22] + Attempting to send results [October 28 17:35:22 UTC]
[17:35:22] - Reading file work/wuresults_03.dat from core
[17:35:22] (Read 1295056 bytes from disk)
[17:35:22] Connecting to http://171.67.108.25:8080/
[17:35:22] - Couldn't send HTTP request to server
[17:35:22] + Could not connect to Work Server (results)
[17:35:22] (171.67.108.25:8080)
[17:35:22] + Retrying using alternative port
[17:35:22] Connecting to http://171.67.108.25:80/
[17:35:22] - Couldn't send HTTP request to server
[17:35:22] + Could not connect to Work Server (results)
[17:35:22] (171.67.108.25:80)
[17:35:22] Could not transmit unit 03 to Collection server; keeping in queue.
[17:35:22] + Sent 0 of 1 completed units to the server
[17:35:22] + Closed connections
[17:35:22] Folding@Home GPU Core - Beta
[17:35:22] Version 1.13 (Mon Sep 29 12:58:49 PDT 2008)
[17:35:22]
[17:35:22] Compiler : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86
[17:35:22] Build host: amoeba
[17:35:22] Board Type: Nvidia
[17:35:22] Core :
[17:35:22] Preparing to commence simulation
[17:35:22] - Looking at optimizations...
[17:35:22] - Created dyn
[17:35:22] - Files status OK
[17:35:22] - Expanded 42904 -> 246265 (decompressed 573.9 percent)
[17:35:22] Called DecompressByteArray: compressed_data_size=42904 data_size=246265, decompressed_data_size=246265 diff=0
[17:35:22] - Digital signature verified
[17:35:22]
[17:35:22] Project: 5801 (Run 4, Clone 323, Gen 0)
[17:35:22]
[17:35:22] Assembly optimizations on if available.
[17:35:22] Entering M.D.
[17:35:29] Working on p5801_supervillin_e1
[17:35:29] Client config found, loading data.
[17:35:29] Starting GUI Server
[17:35:42] mdrun_gpu returned
[17:35:42] NANs detected on GPU
[17:35:42]
[17:35:42] Folding@home Core Shutdown: INTERRUPTED
[17:35:44] CoreStatus = 66 (102)
[17:35:44] + Shutdown requested by user. Exiting.***** Got a SIGTERM signal (2)
[17:35:44] Killing all core threads
Folding@Home Client Shutdown.
Re: Project 5801 issues.
Posted: Tue Oct 28, 2008 6:32 pm
by Cenzor
Same here....
My systems are water-cooled. Overclocked - Yes. But I have tried both stock settings and also reinstalled Cuda (that sometimes can stop working and cause the same problem).
Nothing helps. Project 5801 needs to be taken offline ASAP - especially since it is the only project we seem to get lately. No matter how much I retry with cleaning queues, work folders etc - that is the only project I get. Ive tried at least 15-20 times to avoid it now but nope... 5801 is glued to me
Change request for the future - if crap like this happens then the user, or rather - the contributor, should have the option to reject certain projects. We aren't dumb robots you know - well at least most aren't
Whoever released 5801 maybe should have tested it first?
Re: Project 5801 issues.
Posted: Tue Oct 28, 2008 6:33 pm
by toTOW
Teddy wrote:Same here 5801 has paused every GPU client that I have for 24hrs... & these are all well cooled, stock running video cards..
What a mess Stanford...
Going to reassess my contribution to the Folding@Home project after this mess, Sorry Stanford!
Today there are going to be some very unhappy Forum members here @ Overclockers Australia.
Teddy
If you can't afford this kind of issue, don't run experimental client like GPU ones ...
Re: Project 5801 issues.
Posted: Tue Oct 28, 2008 6:35 pm
by Dead Things
So far Project 5801 has crashed on every type of Nvidia card I have folding including...
- 9800GT
- 8800GT
- 8800GS
- 9600GSO
- 8800GTS 512
- 8600GT
- 9800GX2
It is indiscriminate in its suckage. Almost all of my nv cards are sitting idle right now.
Re: Project 5801 issues.
Posted: Tue Oct 28, 2008 6:35 pm
by road-runner
toTOW wrote:Teddy wrote:Same here 5801 has paused every GPU client that I have for 24hrs... & these are all well cooled, stock running video cards..
What a mess Stanford...
Going to reassess my contribution to the Folding@Home project after this mess, Sorry Stanford!
Today there are going to be some very unhappy Forum members here @ Overclockers Australia.
Teddy
If you can't afford this kind of issue, don't run experimental client like GPU ones ...
We can sure do that also...
Re: Project 5801 issues.
Posted: Tue Oct 28, 2008 6:39 pm
by Drugless
Arrrrrrrrrrrrrrrrrrrrrgggggggggggggggggggggggghhhhhhhhhh!
Has Stanford been hit by a twister or something?
Small sample:
[17:54:54] EUE limit exceeded. Pausing 24 hours.
[18:03:19] EUE limit exceeded. Pausing 24 hours.
[17:55:59] EUE limit exceeded. Pausing 24 hours.
[17:23:33] EUE limit exceeded. Pausing 24 hours.
Geez! I thought I was being lucky with so many people reporting EUE's with the Nvdia 1.15 Core in past week since I was only having 1-2 EUE's across 20 Clients a day.
NONE of my GPU's are overclocked and all except 2 are watercooled down to around 52C.
In last 1 hour I have exceeded EUE limit on 13 clients with this 5801 Project and I expect the same fate for the rest of my farm!
BBBRRRR! My farm room is getting so cold (13C right now!) I have turned off the aircon since the GPU's are just sitting freezing their butts off.
Also can't upload!
Code: Select all
[18:24:03] + Could not connect to Work Server (results)
[18:24:03] (171.64.65.106:80)
[18:24:03] - Error: Could not transmit unit 00 (completed October 28) to work server.
[18:24:03] - 20 failed uploads of this unit.
[18:24:03] - Read packet limit of 540015616... Set to 524286976.
Re: Project 5801 issues.
Posted: Tue Oct 28, 2008 6:41 pm
by Pette Broad
Same here. 2 (make that 3) machines now on a 24 hour idle cycle because of these units. No classic units available either so I may as well just turn them off
Pete
Re: Project 5801 issues.
Posted: Tue Oct 28, 2008 6:42 pm
by Cenzor
toTOW wrote:Teddy wrote:Same here 5801 has paused every GPU client that I have for 24hrs... & these are all well cooled, stock running video cards..
What a mess Stanford...
Going to reassess my contribution to the Folding@Home project after this mess, Sorry Stanford!
Today there are going to be some very unhappy Forum members here @ Overclockers Australia.
Teddy
If you can't afford this kind of issue, don't run experimental client like GPU ones ...
Which is wiser? To do public "experiments" with untested WU's and cause threads like these - AND lose a lot of computing power and 24-hour disabled machines, or, perhaps test the WU's on at least 2-3 different GPU's/CPU's before going public? We are not folding for our own sake - the loss is on Stanfords side, but sure the annoyance is on our, remember...
Re: Project 5801 issues.
Posted: Tue Oct 28, 2008 6:46 pm
by Curly99
Getting the same problems here.... not good
toTOW wrote:Teddy wrote:Same here 5801 has paused every GPU client that I have for 24hrs... & these are all well cooled, stock running video cards..
What a mess Stanford...
Going to reassess my contribution to the Folding@Home project after this mess, Sorry Stanford!
Today there are going to be some very unhappy Forum members here @ Overclockers Australia.
Teddy
If you can't afford this kind of issue, don't run experimental client like GPU ones ...
Not a very helpful reply there, are you saying if you don't like it tough...... probably shows what the mods think of us folders
Curly
Re: Project 5801 issues.
Posted: Tue Oct 28, 2008 6:55 pm
by outbakoz
6 GPU's sitting idle here, all done over 1000 WU each no issues UNTIL it started getting 5801's.
Shutting down till this gets resolved
Re: Project 5801 issues.
Posted: Tue Oct 28, 2008 7:00 pm
by MichaelO
toTOW wrote:Teddy wrote:Same here 5801 has paused every GPU client that I have for 24hrs... & these are all well cooled, stock running video cards..
What a mess Stanford...
Going to reassess my contribution to the Folding@Home project after this mess, Sorry Stanford!
Today there are going to be some very unhappy Forum members here @ Overclockers Australia.
Teddy
If you can't afford this kind of issue, don't run experimental client like GPU ones ...
ToTOW,
This is an incredibly unprofessional message and shows a complete lack of sense. This many complaints about a specific WU indicate that there really is a problem, but you obviously don't care. This completely turns off people who are not getting paid and are donating time and money to this project. You need to go find someplace else to show your backside off.
Re: Project 5801 issues.
Posted: Tue Oct 28, 2008 7:00 pm
by Teddy
Well I don't run my machines just to have them run idle.
But Yep typical response, I don't care anymore I used to, at the end of the day it hurts the Stanford community more than it hurts me.
Lots of our team members, well you know what I am going to say!
& these p5801's made it through beta testing, Yea right...
Teddy
Re: Project 5801 issues.
Posted: Tue Oct 28, 2008 7:04 pm
by mikeb12
hopefully they'll get this one fixed pretty soon.. I kinda wonder if this was a band aid for the server load issue and it back fired. I noticed a new address in my log with 5801
http://171.67.108.11/ and now it seems to be offline if I click the link. but it shows as online in server stats
http://fah-web.stanford.edu/logs/171.67.108.11.log.html
anyway, I just noticed that new address, maybe it's been there before and I just never saw it. usually I see
http://171.64.65.106/
but all these 5801's are coming from
http://171.67.108.11/
and yeas all mine are failing too.. immediate eue on 5801... then 24hr pause. I've cycled thru quite a few trying to figure it out before posting here. removed oc's. switched from 177.84 drivers to 178.24
8800gt, 9800gt, 9600gso
Vista 64 gpu 6.20r1 console clients