WARNING Do not upgrade to 375/376.xx drivers (for xx<48)

It seems that a lot of GPU problems revolve around specific versions of drivers. Though NVidia has their own support structure, you can often learn from information reported by others who fold.

Moderators: Site Moderators, FAHC Science Team

Post Reply
DutchForce
Posts: 60
Joined: Sun Sep 08, 2013 12:43 pm
Location: Netherlands

Re: WARNING ! Do not upgrade to 375.xx drivers.

Post by DutchForce »

The new NVidia driver 376.09 WHQL released today still has no fix for FaH. (maybe add 376.xx to this thread title).
See NVidia forum post of Manuel Guzman:
Official 376.09 Game Ready WHQL Display Driver Feedback Thread (Released 11/28/16)

villageidiot182 said:
Any Updates on Folding@Home?
-----------------

ManuelGuzman:
We still have not resolved this issue.
Image
OliverY1992
Posts: 3
Joined: Mon Nov 28, 2016 6:53 pm

Re: WARNING ! Do not upgrade to 375.xx drivers.

Post by OliverY1992 »

I can confirm the 376.09 drivers released today don't work, tested it and fails.

Will wait for the next set of drivers and test them XD

Though atm it seems like it might not be until next year when it's fixed with the current trend, I'm still baffled as to how they cocked them up for simply folding.
snapshot
Posts: 132
Joined: Thu Apr 09, 2009 7:25 pm
Location: Wiltshire, UK

Re: WARNING ! Do not upgrade to 375.xx drivers.

Post by snapshot »

DutchForce wrote:The new NVidia driver 376.09 WHQL released today still has no fix for FaH. (maybe add 376.xx to this thread title).
See NVidia forum post of Manuel Guzman:
Official 376.09 Game Ready WHQL Display Driver Feedback Thread (Released 11/28/16)

villageidiot182 said:
Any Updates on Folding@Home?
-----------------

ManuelGuzman:
We still have not resolved this issue.
Thank you. I shall stop building my test rig and tidy up the mess I've made.....
FWIW I've commented in that thread that I'm unhappy. I get a strong sense that NV is rushing drivers out without proper testing, hoping they'll fix some of the problems.
Curtis_S
Posts: 2
Joined: Sun Nov 27, 2016 8:35 pm

Re: WARNING ! Do not upgrade to 375/376.xx drivers.

Post by Curtis_S »

Yeah, this is strange. None of the other openCL or Cuda applications that I run have been affected. That said, FAH core 21 is built on a legacy openMM API, and some target architectures have been deprecated since then. I have noticed that some binaries compiled for architectures pre-6.0 generate errors on my Pascal GPU that are fixed by compiling with the most recent version of nvcc for the current architectures.
bruce
Posts: 20824
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: WARNING ! Do not upgrade to 375/376.xx drivers.

Post by bruce »

NV is asking for bug report numbers. If you filed one, please post.

If you didn't, then file one.
silveradocyn
Posts: 8
Joined: Thu Feb 14, 2008 6:08 pm
Location: Silverado Canyon, Southern California, USA

Re: WARNING ! Do not upgrade to 375/376.xx drivers.

Post by silveradocyn »

The progression of the issue went something like this for me:
Get new Windows 10 preview build - install
(Like normal the windows upgrade will break CUDA. )
Download latest build from NVidia.
Start install - Custom/Clean Install options
System will reboot.

After the reboot the installer normally starts up again. In the most recent Windows Preview builds there has been some conflict that the NVidia installer thinks the "Found New Hardware" wizard is running and has to wait to continue. With these two builds over the weekend, the installer does not seem to restart after the reboot. When I manually restart the installer the installer does not seem to realize that it was halfway through the driver install.

My Guess:
Microsoft has tightened up the security of driver installers again, and not told the OEMs. Microsoft expects to manage all of the driver installs, and has not recognized CUDA, and never installed CUDA correctly, and now may have blocked NVidia from performing the install correctly.

Are the other users having this issue also running with the pre-release version of windows?

- - Steve
Image
In the wild Canyons of OC.
SombraGuerrero
Posts: 117
Joined: Mon Mar 16, 2009 3:06 am

Re: WARNING ! Do not upgrade to 375/376.xx drivers.

Post by SombraGuerrero »

Nvidia's driver feedback page appears to be inaccessible due to outdated TLS certs and related access denials. I have posted to their dev forums about our situation. I did also mention to them about the client's implementation of OpenMM for the core and Gromacs in general, as I feel a dev may find that information relevant.

https://devtalk.nvidia.com/default/topi ... me-client/
massenbg
Posts: 2
Joined: Sun Sep 20, 2015 9:36 pm

Re: WARNING ! Do not upgrade to 375/376.xx drivers.

Post by massenbg »

Updated Windows 10 (64) to "Anniversary Update" (v1607) and noticed my 1070 was no longer folding. Dug a little on NVIDIA's site and downloaded several previous drivers. Went from 376.09 to 375.95 to 375.70 to 375.63 and finally 375.57 before F@H GPU folding resumed. Had that failed, my next version to attempt was 373.06. I see here that many users have had a problem with 375.57, but my 1070 is using that version successfully so far. Thank you to the originator of this thread. Helped me greatly. I don't game, I fold. And this 'break' was cramping my style!
** Obviously spoke too soon. Didn't get even 20% through and the folding broke. On to 373.06!
wuffy68
Posts: 168
Joined: Wed Jun 04, 2014 11:06 pm
Hardware configuration: 1x nVidia 1080Ti, 1x nVidia 1070, 1x nVidia 1060, 1x nVidia 750Ti, AMD Radeon R7 M460
Location: Roxborough, Colorado USA
Contact:

Re: WARNING ! Do not upgrade to 375/376.xx drivers.

Post by wuffy68 »

Same problem when I upgraded my Win10-64 from 373.06 to latest 376.xx driver. Both my 970 and 1060 GPU jobs started failing. I reverted back to 373.06 and they are now completing WU again. Something is seriously changed in the last month+ of nVidia driver builds. Has anyone seen anything from nVidia on the issue?
1x nVidia 1070, 1x nVidia 1060 3g,
1x nVidia 970, 2x nVidia 960,
1x nVidia 555, 1x AMD R7, 2x AMD 295,
6x i5 CPU-only rigs
jesanfafon
Posts: 9
Joined: Wed Sep 14, 2016 3:01 pm

Re: WARNING ! Do not upgrade to 375/376.xx drivers.

Post by jesanfafon »

silveradocyn wrote:Are the other users having this issue also running with the pre-release version of windows?
On the latest build from the "Fast" ring, I've had to resort to uninstalling the drivers from the device manager after the upgrade as otherwise an NVIDIA clean install fails.
bruce
Posts: 20824
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: WARNING ! Do not upgrade to 375/376.xx drivers.

Post by bruce »

wuffy68 wrote:Same problem when I upgraded my Win10-64 from 373.06 to latest 376.xx driver. Both my 970 and 1060 GPU jobs started failing. I reverted back to 373.06 and they are now completing WU again. Something is seriously changed in the last month+ of nVidia driver builds. Has anyone seen anything from nVidia on the issue?
Yes. In my earlier post, I gave URLs to several discussion topics at the NVidia forums. There are lots of people griping about these drivers, there area also responses from a NVidia staff person. Unfortunately most of the discussions are about games with almost nothing about FAH. We have only ourselves to blame for that.

Did you create a bug report there? If not, do so. Having just had a failure, you should be able to answer most of the questions ... and in either case, post the bug number here. I suspect that they're not going to work on the problem soon because while we gripe a lot, we don't fill out bug reports.
wuffy68
Posts: 168
Joined: Wed Jun 04, 2014 11:06 pm
Hardware configuration: 1x nVidia 1080Ti, 1x nVidia 1070, 1x nVidia 1060, 1x nVidia 750Ti, AMD Radeon R7 M460
Location: Roxborough, Colorado USA
Contact:

Re: WARNING ! Do not upgrade to 375/376.xx drivers.

Post by wuffy68 »

I saw your link bruce. Thanks. I was just surprised there was no response.

Mine happened while I was installing a new card in my rig, so I initially blamed the card. Then my dvi-d interface fried on my monitor (literally no longer works with any system) so no folding for a couple of days until I put a new monitor on and fixed the driver problem. I'll see if I have any Windows logs from the time of my failure, but I think my client logs are gone since I re-installed it to try fix a problem with the new card.

Agree - since there's little incentive (like gamers have - "I WANT TO PLAY MY GAME"), driver upgrades on folding rigs don't happen as often (which in this case is a good thing, otherwise our network probably wouldn't be at 104 petaFLOPs today.)
1x nVidia 1070, 1x nVidia 1060 3g,
1x nVidia 970, 2x nVidia 960,
1x nVidia 555, 1x AMD R7, 2x AMD 295,
6x i5 CPU-only rigs
antropofob
Posts: 57
Joined: Mon Aug 22, 2011 8:03 am

Re: WARNING ! Do not upgrade to 375/376.xx drivers.

Post by antropofob »

Has anyone tried the newest 375.20 Linux drivers? Do they suffer also?
Joe_H
Site Admin
Posts: 7937
Joined: Tue Apr 21, 2009 4:41 pm
Hardware configuration: Mac Pro 2.8 quad 12 GB smp4
MacBook Pro 2.9 i7 8 GB smp2
Location: W. MA

Re: WARNING ! Do not upgrade to 375/376.xx drivers.

Post by Joe_H »

Yes, it was reported a page or two back, and no they do not work.
Image

iMac 2.8 i7 12 GB smp8, Mac Pro 2.8 quad 12 GB smp6
MacBook Pro 2.9 i7 8 GB smp3
bruce
Posts: 20824
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: WARNING ! Do not upgrade to 375/376.xx drivers.

Post by bruce »

Come on, guys. Doesn't anybody care enough about this problem to submit a NV bug report?
Post Reply