Adding a 2nd Nvidia GPU - any head's up?

[kiore]

Kiore, I remember hearing back in the day, that the length of the bus to the CPU was critical to speed, even to the dimensions within the processor itself. I left the tech back a long time ago & haven't kept up with what's current. I have been thinking the GPU was essentially separate from the CPU, and the WU's done by the GPU weren't dependent on the CPU or the bus. From what I'm hearing here, that's not true, that there's a significant difference the bus makes.

I'm still confused about how moving a card via a riser, to a new location, and not having it vent to the outside, would be better than leaving it in place and venting directly outside.

OK take two identical double fan cards and lay them on top of each other, as these cards use 2 or 3 spaces the gap between them is thin and hot so even though the air is being moved by the GPU fans it is hot air to start with. In the days when I ran 4 x GPUs per board I ran them on test benches vertically to moderate for this. Right now in a dual card system I have a GTX 1080ti hybrid in the Top slot running at 60C all good, but the GTX1660ti twin fan below it even though a shorter card is operating in a hot environment and has to be power limited to keep it under 80C due to the heat generated by the GTX 1080ti above it. I wish I had never bought an air cooled RTX 2080ti and waited and paid premium for the hybrid, and yes I know I could buy a conversion kit but that very expensive in parts and effort.

[MeeLee]

Kiore, I remember hearing back in the day, that the length of the bus to the CPU was critical to speed, even to the dimensions within the processor itself. I left the tech back a long time ago & haven't kept up with what's current. I have been thinking the GPU was essentially separate from the CPU, and the WU's done by the GPU weren't dependent on the CPU or the bus. From what I'm hearing here, that's not true, that there's a significant difference the bus makes.

I'm still confused about how moving a card via a riser, to a new location, and not having it vent to the outside, would be better than leaving it in place and venting directly outside.

Only blower style GPUs vent directly outside.
Most GPUs use open coolers.
They vent inside the case.
It's mostly ok for sub 200Watt GPUs, if you're using one, or using 2 in a taller tower.
but for the fastest GPUs, heat gets trapped inside, and they will run very hot (hotter than you'd want them to run).

[KA1J]

Kiore & MeeLee: Well heck, I understand now. The only cards I have that are good for Folding vent out the back. Only the earlier cards without fans heated the inside. The RTX 2080 coming is a 11G-P4-2383-KR and as I wasn't interested in it for video, didn't study the back & now see there's no real outlet for air. I assumed if it had fans, they were to blow the hot air out of the case. Yeah, I see the point & why a hybrid would have been a better choice.

I had a Storm G5 Waterblock with an Iwaki water pump the last time I was water-cooling, that was like 15 years ago. Used to have a radiator & then realized if I just got surplus Tygon tubing off of Fleabay I could just make a long loop of the tubing along the basebord & put the Iwaki in a padded box to absorb any pump noise at the far end at the hairpin loop, & convection would cool it fine & by the time the liquid got back there was close to room temp & zero fan noise. Perfect for my recording studio at the time. When the next generation of processors came out I wasn't recording and never used water cooling after that.

This changes some things... I'll have to re-think how I'm going to do this. Maybe my best bet for this is to get a used motherboard & cheap ram, migrate an old SSD & PS with proper PCI-e plugs & run the other GPUs in it and let it run in the open without being in a case. That would mean 3 computers running at the same time & I didn't want to do that but it would solve the cooling issue. If it's true that Linux helps get the WUs done faster, I could try and muddle through that enough to get it to run on that motherboard in the open. My last attempt with Linux was maybe 2003 and it was a PITA to get my windows programs running on it. Maybe it's more user friendly now. Haven't looked.

Anyway, I ramble. Thanks for pointing out the issue.

[kiore]

Multi GPU gives multi problems, I am sure you will find a way to open it up there are lots of DIY options. Open case helps, risers also create open space, extra case fans can help but I find them the least useful at delivering on the promise of air movement. I have moved from overclocking GPU to power limiting which I find frustrating but gone are the days (2010) when I had 4 x GPUs GTX 260s overclocked on a test bed with a hairs breath between them and a pedestal fan blowing over them to keep them from melting.If we are to reminisce I needed two PSUs for that rig, well before the days of PSUs with 8 gpu cords.

[KA1J]

Kiore, Heh, reminiscing is always fun. One of the problems with proper air flow with a muffin fan is the lousy directionality of the air flow. I have a tube stereo amplifier in an open front & back cabinet. I wanted to blow air across the tubes to keep them cool and had some muffin fans. When I put my hand in front of the fan, there was hardly any air hitting my hand, it was all coming out maybe 30 degrees tangential to the body of the fan. I made a cardboard model to hold the fan and made a 4" manifold with the base of the muffin at the right location to blow across the tubes. It works perfectly.

Point being, a muffin is a lousy fan for cooling in a linear direction, it has to have a guide to direct the air, & then it works nicely. None of these computer cases have any such guide so adding muffins to blow onto your target area is useless. if anything, you'd want to feel the wind with your hand and angle the muffin so the air would move in the direction you want. None of these cases are able to provide that width to angle muffins on cards effectively. A squirrel cage blower would do the job perfectly but then you have the issue of a lot of noise. My Ham Radio amplifier has one and it's like a leaf blower behind the amp from the exiting air. I also wear Bose Noise cancelling headphones at the radio because of that blower noise. The best quality of a muffin is they take little power and they can be very quiet.

[PantherX]

...If it's true that Linux helps get the WUs done faster, I could try and muddle through that enough to get it to run on that motherboard in the open. My last attempt with Linux was maybe 2003 and it was a PITA to get my windows programs running on it. Maybe it's more user friendly now. Haven't looked...

Currently, the FahCores that fold on the CPU and GPU provides a boost in PPD in Linux. That may or may change in the future. However, if you have an Nvidia GPU, there are four steps to fold:
1) Install Ubuntu/Mint and fully update
2) Install Nvidia closed-source drivers (proprietary)
3) Install OpenCL package (sudo apt install ocl-icd-opencl-dev)
4) Install V7.6.13 and configure it

For AMD GPUs, it does get a bit tricky since the "correct" version of the driver is needed depending on the GPU model.

[HaloJones]

and yes I know I could buy a conversion kit but that very expensive in parts and effort.

conversion kits don't need to be that dear. Buy an NZXT G12 and a used compatible AIO like a Corsair H55.

[MeeLee]

I think Linux is going to remain to be higher in performance than Windows, not until they completely overhaul Windows.
Which they might with their ARM version.

Linux isn't made to run Windows programs, but older programs run fine on wine and wine32.
Once you get to Windows 7 programs (or beyond; the 64 bit kind), you better do a side-install of Windows, or run everything in a VM.

Power Capping a GPU is actually healthy.
It's the same as modern phones nowadays.
4 years ago, modern phones had big and little cores much like modern phones do today.
They found out that most of the programs ran on the big cores, causing higher phone heat, and lower battery life.
Modern phones still have big and little cores, but now they only boost on the big cores, and run most programs on the little cores.

Modern phones have much higher performance than their thermals allow.
And so is with GPUs.
Power capping them, makes them run at frequencies that are closer to what the GPU should have been running at.
If you ask me, I prefer more cores (like 4k cores) at lower frequencies (like 1.5 Ghz), than fewer cores (say 1.5k cores) at higher frequencies (say 4Ghz).
The former would run more efficient.

[KA1J]

PantherX, MeeLee & Halo, Thanks again for the thoughts & advice. I have the 2080Ti XC Ultra running alone in the computer upstairs. It's not terribly loud at the moment but the sides are off the case & it's as expected, a whole lot faster than the GTX 1080 FWC & the 2060 KO. I do have those two running together in the downstairs computer. I've used the EVGA Precision X1's scan to get the OC it provides with all three cards and saved those profiles. I'm especially happy that no errors have shown in the F@H log, other than the many instances of no WU or a lack of connecting to the remote host, I have no control over that. While I might get more stable OC manually, I'm satisfied these are reasonable overclocks and will not push temps or add stress to the cards.

I will get a riser and put the lowest performing card in along with the 2080ti, it won't generate much heat, I think the GTX 1050 has 500 some cores. If I get it away from the 2080 it should be fine.

I contacted EVGA about getting a hybrid kind of watercooler for the 2080 for my GPU. The one that fits is EVGA # 400-hy-1384-b1 but they are out of stock and I have the impression from the conversation they will not be back in stock. They are not available online. They also have no waterblock for this particular card available & suggested I go with a NZXT G12 but my attempts to find someone selling a compatible enclosed AIO to fit that card has fallen victim to the corona virus, with no available people to answer questions. I'll have to see how this is enclosure is running with the sides on & maybe the temps will not be so much as to cause throttling back.

Maybe to make room in the case I should look for an inexpensive AIO that fits the i7-8700K & cools at least as well as the air does, I have some repurposing I could do with the two large quiet Nocturna fans attached to the NH-D15 CPU cooler.

I have a few small SSD and will connect one of them to the computer & load Ubuntu/Mint into it and play with it for a bit. I don't know if it'll be easier to work than 17 years ago, but if I can make sense of it & it works better enough for PPD than windows 10, then I'll cobble an inexpensive dedicated computer from parts here just for folding with Linux & put the 2080 in it.

Thanks again.

[Mxyzptlk]

Just for a comparison on a pair of lower powered cards. I run a 2060 Super and 2060 in one machine. They are both twin fan EVGA’s with the same cooler. The lower card (2060) runs around 54 and the upper card (2060 Super) runs at around 62. I could probably get the upper card a bit cooler if I ran the fans at 100%, but I figure the temps aren’t all that bad overall. I’m also running them both at about 75% power without any major PPD sacrifice.

[HaloJones]

The G12 works with any Asetek manufactured AIO that has the round cold plate on the block/pump. If the base plate is hexagonal it won't fit.

I have one working with a Corsair H55 (120mm), another with a Corsair H110 (280mm).

This is the officially supported list:

NZXT
Kraken X62, X52, X42, X61, X41, X31, X60, X40
Corsair
H105, H110, H90, H75 (CW-9060015-WW only), H55, H50 (CW-906006-WW only)
Thermaltake
Water 3.0 Riing RGB 360, 280, 240, Red 280, 140,
Water 3.0 Ultimate, Extreme S, Extreme, Pro, Performer
Water 2.0 Extreme, Pro, Performer
Antec
KUHLER H2O 920V4, 620V4, 920, 620
Zalman
LQ-320, LQ-315, LQ-310

[MeeLee]

PantherX, MeeLee & Halo, Thanks again for the thoughts & advice. I have the 2080Ti XC Ultra running alone in the computer upstairs. It's not terribly loud at the moment but the sides are off the case & it's as expected, a whole lot faster than the GTX 1080 FWC & the 2060 KO. I do have those two running together in the downstairs computer. I've used the EVGA Precision X1's scan to get the OC it provides with all three cards and saved those profiles. I'm especially happy that no errors have shown in the F@H log, other than the many instances of no WU or a lack of connecting to the remote host, I have no control over that. While I might get more stable OC manually, I'm satisfied these are reasonable overclocks and will not push temps or add stress to the cards.

I will get a riser and put the lowest performing card in along with the 2080ti, it won't generate much heat, I think the GTX 1050 has 500 some cores. If I get it away from the 2080 it should be fine.

I contacted EVGA about getting a hybrid kind of watercooler for the 2080 for my GPU. The one that fits is EVGA # 400-hy-1384-b1 but they are out of stock and I have the impression from the conversation they will not be back in stock. They are not available online. They also have no waterblock for this particular card available & suggested I go with a NZXT G12 but my attempts to find someone selling a compatible enclosed AIO to fit that card has fallen victim to the corona virus, with no available people to answer questions. I'll have to see how this is enclosure is running with the sides on & maybe the temps will not be so much as to cause throttling back.

Maybe to make room in the case I should look for an inexpensive AIO that fits the i7-8700K & cools at least as well as the air does, I have some repurposing I could do with the two large quiet Nocturna fans attached to the NH-D15 CPU cooler.

I have a few small SSD and will connect one of them to the computer & load Ubuntu/Mint into it and play with it for a bit. I don't know if it'll be easier to work than 17 years ago, but if I can make sense of it & it works better enough for PPD than windows 10, then I'll cobble an inexpensive dedicated computer from parts here just for folding with Linux & put the 2080 in it.

Thanks again.

Due to the Corona Virus, a lot of tech hardware products are limited in supply. Even on Amazon.
Nothing comes in by plane anymore.
Add to that the trade tax (tariffs) our current govt have imposed on Chinese products, and you're going to see a rise in prices for the next few months (probably until the end of the year)!
Even products US made, having parts and resources from China, will increase in price.
The Kingpin isn't really recommended to fold on. Those water/air cooling units are good for games, where you get performance peaks that can be buffered by the cooling circuit;
But once you run a constant load on GPU or even CPU, AIO coolers or water cooling systems are pretty much the same as air cooling units, with the exception of allowing to use the hardware in a more constricted space.

[kiore]

I disagree with Meelee about AIO coolers in regard to Hybrid GPUs, I have owned many versions usually EVGA high end GPUs and find that they work brilliantly for folding. I seriously regret not waiting for a hybrid RTX 2080ti and getting the aircooled version. My GTX1080ti hybrid sits at <60c even though above a GTX 1660ti twin fan aircooled which I have to power limit to keep below 80c. A dedicated liquid loop probably best but the hybrids really keep things cooler not just the liquid bit but that you can move the heat exchange area to a different part of the case for example the above mentioned GTX1080ti radiator is on the top of the case a huge advantage and far away from the card and CPU.

[MeeLee]

I have 3x 2080Tis each running at below 60C. On Core 21 they would run in the lower 50's (depending on ambient sometimes in the higher 40s).

Like I said, unless you're working with a constricted space, air coolers work just as well as those water cooling systems you find on GPUs that are at least $500 overpriced compared to open air cooled GPUs.
Many websites (like Linus Tech Tips, Tom's hardware, and a few others) would agree that water cooling and decent air cooling are pretty much the same.
Think about it, a water cooling loop, is essentially also an aircooling circuit.
The radiator gets cooled by a fan, which is... air cooled...

[HaloJones]

The tech sites have tested AIO coolers against high-end air coolers for cpus not gpus

An aircooler on a cpu can be enormous, five or six heatpipes attached to giant radiators cooled by 140mm fans. That means they're not dissimilar in overall surface area to a 240mm or even 360mm radiator in an AIO.

But the largest aircoolers on gpus come nowhere near the surface area of a 240mm radiator let alone a 360mm. Your aircooled 280tis running <60C are nice but at what clock and what fan curve?

I have a TitanX hitting 270W and it's sub 50C and silent. Water-cooling allows for much larger surface areas for the fans to work on and are highly efficient at removing that air from the case.