Maxwell GPUs

A forum for discussing FAH-related hardware choices and info on actual products (not speculation).

Moderator: Site Moderators

Forum rules
Please read the forum rules before posting.
billford
Posts: 1003
Joined: Thu May 02, 2013 8:46 pm
Hardware configuration: Full Time:

2x NVidia GTX 980
1x NVidia GTX 780 Ti
2x 3GHz Core i5 PC (Linux)

Retired:

3.2GHz Core i5 PC (Linux)
3.2GHz Core i5 iMac
2.8GHz Core i5 iMac
2.16GHz Core 2 Duo iMac
2GHz Core 2 Duo MacBook
1.6GHz Core 2 Duo Acer laptop
Location: Near Oxford, United Kingdom
Contact:

Maxwell GPUs

Post by billford »

I've got two nominally identical systems using NVidia GTX 980's. I've always seen slight differences in the behaviour of the two cards, but with the advent of Core_21 that difference is becoming significant.

I believe (from occasional comments here) that there are two types of Maxwell cards- the cards were bought several months apart so it's quite possible I've got one of each.

But I don't know how to tell... could someone give me a clue what to look for please?

(If it involves software in some way, both are running under Linux)
Image
ChristianVirtual
Posts: 1576
Joined: Tue May 28, 2013 12:14 pm
Location: Tokyo

Re: Maxwell GPUs

Post by ChristianVirtual »

What differences in behavior ?

Are the system really the same, CPU, memory, storage ...
Some of the newer core are writing more frequent checkpoints which might impact runtime characteristics.

And of course differences within different PRCGs might hit you, too.
ImageImage
Please contribute your logs to http://ppd.fahmm.net
billford
Posts: 1003
Joined: Thu May 02, 2013 8:46 pm
Hardware configuration: Full Time:

2x NVidia GTX 980
1x NVidia GTX 780 Ti
2x 3GHz Core i5 PC (Linux)

Retired:

3.2GHz Core i5 PC (Linux)
3.2GHz Core i5 iMac
2.8GHz Core i5 iMac
2.16GHz Core 2 Duo iMac
2GHz Core 2 Duo MacBook
1.6GHz Core 2 Duo Acer laptop
Location: Near Oxford, United Kingdom
Contact:

Re: Maxwell GPUs

Post by billford »

ChristianVirtual wrote:What differences in behavior ?

Are the system really the same, CPU, memory, storage ...
Some of the newer core are writing more frequent checkpoints which might impact runtime characteristics.

And of course differences within different PRCGs might hit you, too.
There are slight differences in the motherboards (Gigabyte GA-H81M-DS2V @ 3.2GHz, GA-H81M-S2PV @ 3.0GHz), otherwise identical.

Cards are both from MSI- these ones. The newer card is on the 3.0GHz mobo.

OS (Mint 17.2), drivers (346.35) etc- identical.

The differences on Core_17 and 18 were slight but noticeable- mainly in the way the boost clock adjusted itself, to the extent that the average PPD (over several thousand WUs) was about 10k (in ~420k) higher on the (newer) card in the slower mobo.

On Core_21, the newer card seems quite happy (so far) with 125MHz of further overclocking whereas the older one struggles with no extra overclock. Particularly on the later P69xx series- the earlier ones that show up in psummary as "UNKNOWN_ENUM" were more or less OK. Though I'll admit I haven't had many of the new ones.

If I can find that the cards are of different generations then I'll just put it down to one of those things and play with the clocks accordingly... and that's all I'm asking- how do I find out?
Image
toTOW
Site Moderator
Posts: 6349
Joined: Sun Dec 02, 2007 10:38 am
Location: Bordeaux, France
Contact:

Re: Maxwell GPUs

Post by toTOW »

The new core 21 tends to push the hardware harder than we're used to. If your GPU quality are very different, one might be throttling more than the other (on Windows, GPU-Z gives the value which is called PerfCap reason).

Also, core 21 projects are often bigger (number of atoms) and the sanity checks take more time. Those sanity checks are performed on the CPU, so different CPUs might also be a reason that exacerbate the differences.

P.S : I have the same card as yours ;)
Image

Folding@Home beta tester since 2002. Folding Forum moderator since July 2008.
billford
Posts: 1003
Joined: Thu May 02, 2013 8:46 pm
Hardware configuration: Full Time:

2x NVidia GTX 980
1x NVidia GTX 780 Ti
2x 3GHz Core i5 PC (Linux)

Retired:

3.2GHz Core i5 PC (Linux)
3.2GHz Core i5 iMac
2.8GHz Core i5 iMac
2.16GHz Core 2 Duo iMac
2GHz Core 2 Duo MacBook
1.6GHz Core 2 Duo Acer laptop
Location: Near Oxford, United Kingdom
Contact:

Re: Maxwell GPUs

Post by billford »

toTOW wrote:The new core 21 tends to push the hardware harder than we're used to.
Yes, and Linux pushes it harder than Windows...
If your GPU quality are very different..
It's looking as if that may be the case, regardless of whether there are two versions of Maxwells or not :( .

I've put it back to full overclock and taken the adv flag off while I have a think. Can only be short-term of course, until Core_21s get released to full FAH.
P.S : I have the same card as yours ;)
Clearly a man of excellent judgement :wink:
Image
Joe_H
Site Admin
Posts: 7927
Joined: Tue Apr 21, 2009 4:41 pm
Hardware configuration: Mac Pro 2.8 quad 12 GB smp4
MacBook Pro 2.9 i7 8 GB smp2
Location: W. MA

Re: Maxwell GPUs

Post by Joe_H »

billford wrote: I believe (from occasional comments here) that there are two types of Maxwell cards- the cards were bought several months apart so it's quite possible I've got one of each.

But I don't know how to tell... could someone give me a clue what to look for please?
The difference in Maxwell generations is not applicable here. What the difference refers to is the the chips used in different models of nVidia GPU's. First generation Maxwell chips were the GM107 and 108 used in the 745, 750, 750ti and some 8nnM GPU's. The second generation Maxwells are the GM20n chips used on the GTX 900 series cards. So your two cards would have the same generation Maxwell GPU chips on them.

The difference you are seeing might be related to slight variances seen during production of any chip. The cards you have are already factory overclocked, so one might be closer to its limit for stable computations as opposed to video output which is the criteria MSI is using when binning GPU chips.
Image

iMac 2.8 i7 12 GB smp8, Mac Pro 2.8 quad 12 GB smp6
MacBook Pro 2.9 i7 8 GB smp3
billford
Posts: 1003
Joined: Thu May 02, 2013 8:46 pm
Hardware configuration: Full Time:

2x NVidia GTX 980
1x NVidia GTX 780 Ti
2x 3GHz Core i5 PC (Linux)

Retired:

3.2GHz Core i5 PC (Linux)
3.2GHz Core i5 iMac
2.8GHz Core i5 iMac
2.16GHz Core 2 Duo iMac
2GHz Core 2 Duo MacBook
1.6GHz Core 2 Duo Acer laptop
Location: Near Oxford, United Kingdom
Contact:

Re: Maxwell GPUs

Post by billford »

Joe_H wrote: The difference in Maxwell generations is not applicable here. What the difference refers to is the the chips used in different models of nVidia GPU's. First generation Maxwell chips were the GM107 and 108 used in the 745, 750, 750ti and some 8nnM GPU's. The second generation Maxwells are the GM20n chips used on the GTX 900 series cards. So your two cards would have the same generation Maxwell GPU chips on them.
That explains where I got the (wrong!) thought from, thank you :)
The difference you are seeing might be related to slight variances seen during production of any chip. The cards you have are already factory overclocked, so one might be closer to its limit for stable computations as opposed to video output which is the criteria MSI is using when binning GPU chips.
Seems likely :(

When I've got a bit more time to baby-sit it I'll clock it right back to NVidia base and see what happens. It'll be a shame, it'll cut the card's PPD by about 20% even on Core_17 and 18, which are working fine. Such is life.
Image
toTOW
Site Moderator
Posts: 6349
Joined: Sun Dec 02, 2007 10:38 am
Location: Bordeaux, France
Contact:

Re: Maxwell GPUs

Post by toTOW »

If you can run GPU Z on Windows (I don't know if a Linux tool can give such detailed informations), it's pretty easy to compare chip quality :

- chip with lower quality will run at higher voltage and lower boost clock
- as a consequence, chips with lower quality will run closer to their maximum TPD.

GPU-Z will also give you the PerfCap reason, which is the code of what is limiting your GPU.
Image

Folding@Home beta tester since 2002. Folding Forum moderator since July 2008.
billford
Posts: 1003
Joined: Thu May 02, 2013 8:46 pm
Hardware configuration: Full Time:

2x NVidia GTX 980
1x NVidia GTX 780 Ti
2x 3GHz Core i5 PC (Linux)

Retired:

3.2GHz Core i5 PC (Linux)
3.2GHz Core i5 iMac
2.8GHz Core i5 iMac
2.16GHz Core 2 Duo iMac
2GHz Core 2 Duo MacBook
1.6GHz Core 2 Duo Acer laptop
Location: Near Oxford, United Kingdom
Contact:

Re: Maxwell GPUs

Post by billford »

toTOW wrote:If you can run GPU Z on Windows (I don't know if a Linux tool can give such detailed informations)
Linux tools for GPUs are a bit thin on the ground, if anyone knows of one I'll be interested. Installing Windows is not an acceptable option :wink:
- chip with lower quality will run at higher voltage and lower boost clock
Thanks, that might be a clue- I can't monitor the voltage but I mentioned up-topic that the older card tends to run at lower boost than the new one. Looks like a marginal chip on that card.
Image
7im
Posts: 10179
Joined: Thu Nov 29, 2007 4:30 pm
Hardware configuration: Intel i7-4770K @ 4.5 GHz, 16 GB DDR3-2133 Corsair Vengence (black/red), EVGA GTX 760 @ 1200 MHz, on an Asus Maximus VI Hero MB (black/red), in a blacked out Antec P280 Tower, with a Xigmatek Night Hawk (black) HSF, Seasonic 760w Platinum (black case, sleeves, wires), 4 SilenX 120mm Case fans with silicon fan gaskets and silicon mounts (all black), a 512GB Samsung SSD (black), and a 2TB Black Western Digital HD (silver/black).
Location: Arizona
Contact:

Re: Maxwell GPUs

Post by 7im »

My impression was that Core_21 ran better with drivers 350.xx and higher.
How to provide enough information to get helpful support
Tell me and I forget. Teach me and I remember. Involve me and I learn.
billford
Posts: 1003
Joined: Thu May 02, 2013 8:46 pm
Hardware configuration: Full Time:

2x NVidia GTX 980
1x NVidia GTX 780 Ti
2x 3GHz Core i5 PC (Linux)

Retired:

3.2GHz Core i5 PC (Linux)
3.2GHz Core i5 iMac
2.8GHz Core i5 iMac
2.16GHz Core 2 Duo iMac
2GHz Core 2 Duo MacBook
1.6GHz Core 2 Duo Acer laptop
Location: Near Oxford, United Kingdom
Contact:

Re: Maxwell GPUs

Post by billford »

7im wrote:My impression was that Core_21 ran better with drivers 350.xx and higher.
You may well be right, but although they may alleviate the problem a little I doubt they'll cure it. And I put updating Linux GPU drivers firmly under the heading "If it ain't broke, don't fix it" :wink:

I've ended up re-installing the OS too often for my liking.

I hadn't expected the cards to be completely identical even with the same spec, but couldn't understand the drastic difference... it was Joe's observation that MSI sort them by video rather than computational performance that made that clearer to me.
Image
7im
Posts: 10179
Joined: Thu Nov 29, 2007 4:30 pm
Hardware configuration: Intel i7-4770K @ 4.5 GHz, 16 GB DDR3-2133 Corsair Vengence (black/red), EVGA GTX 760 @ 1200 MHz, on an Asus Maximus VI Hero MB (black/red), in a blacked out Antec P280 Tower, with a Xigmatek Night Hawk (black) HSF, Seasonic 760w Platinum (black case, sleeves, wires), 4 SilenX 120mm Case fans with silicon fan gaskets and silicon mounts (all black), a 512GB Samsung SSD (black), and a 2TB Black Western Digital HD (silver/black).
Location: Arizona
Contact:

Re: Maxwell GPUs

Post by 7im »

Correct, drivers will not change differences between cards. However, if newer drivers are better coded for computing, then it might bring both cards back within a similar performance envelope, and then run at about the same speed again.

Also make sure both GPUs are running the same version of Core_21. That's more likely an issue than a driver version.
How to provide enough information to get helpful support
Tell me and I forget. Teach me and I remember. Involve me and I learn.
bollix47
Posts: 2957
Joined: Sun Dec 02, 2007 5:04 am
Location: Canada

Re: Maxwell GPUs

Post by bollix47 »

Another point re later drivers is that nvidia-smi gives a bit more info than previous versions. The following is from 355.11:

Code: Select all

 nvidia-smi
Tue Oct  6 16:12:46 2015
+------------------------------------------------------+
| NVIDIA-SMI 355.11     Driver Version: 355.11         |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 980     Off  | 0000:01:00.0      On |                  N/A |
| 70%   55C    P2   142W / 185W |    239MiB /  4090MiB |     97%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID  Type  Process name                               Usage      |
|=============================================================================|
|    0      1062    G   /usr/bin/X                                      95MiB |
|    0      1750    G   compiz                                          31MiB |
|    0     19528    C   ...AMD64/NVIDIA/Fermi/Core_18.fah/FahCore_18    94MiB |
+-----------------------------------------------------------------------------+
billford
Posts: 1003
Joined: Thu May 02, 2013 8:46 pm
Hardware configuration: Full Time:

2x NVidia GTX 980
1x NVidia GTX 780 Ti
2x 3GHz Core i5 PC (Linux)

Retired:

3.2GHz Core i5 PC (Linux)
3.2GHz Core i5 iMac
2.8GHz Core i5 iMac
2.16GHz Core 2 Duo iMac
2GHz Core 2 Duo MacBook
1.6GHz Core 2 Duo Acer laptop
Location: Near Oxford, United Kingdom
Contact:

Re: Maxwell GPUs

Post by billford »

@ 7im - yes, both cards are using 0.0.11 which is the latest afaik.

@ bollix47 - I use the "NVIDIA X Server Settings" app to get the info in the top box, it also gives the PCIe bandwidth utilisation. But not the power, which might be useful.

Maybe I'll pluck up courage to update the drivers to see if they've added anything else useful. But not tonight, too close to bedtime in my time zone :wink:
Image
bruce
Posts: 20824
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Maxwell GPUs

Post by bruce »

billford wrote:I've got two nominally identical systems using NVidia GTX 980's. I've always seen slight differences in the behaviour of the two cards, but with the advent of Core_21 that difference is becoming significant.
As long as they're identical, you can switch them and resume work. That would eliminate the slight chance that one is positioned in a hotter part of it's case than the other.
Post Reply