Anyone Folding with a 5700XT on Ubuntu 18

[gordonbb]

I'm at home under Quarantine for the next two weeks so I'm taking advantage of this and I've ordered a PowerColor Red Dragon 5700XT and one of the Gigabyte tri-axial 5700XTs to give them a try.

Another goal is to update my Linux Folding HOWTO for my Folding Team.

I must admit that other than my trusty FirePro W4100 in my Win10 Daily Driver I have almost no experience with AMD Drivers and next to none under Linux but have built 5 or 6 systems using the Nvidia PPA Drivers.

I've seen a lot of posts about the 5700XT from folks running Windows but haven't see much on the Linux side.

So if anyone can provide some guidance on getting the drivers working it would be appreciated.

[MrFrizzy]

From everything that I had read so far (mind you, I don't fold on Linux but I do utilize it frequently so take this with a grain of salt) is that you need Ubuntu 18.04 LTS, install the official AMD drivers, install OpenCL separately, setup the F@H client, and you should be good to go.

Perhaps this guide could help you? It is listed for server, but a lot of it should work for desktop as well. viewtopic.php?f=106&t=33090&hilit=ubuntu

Here is another thread I found that could be useful: viewtopic.php?f=106&t=32346&hilit=ubuntu

[gordonbb]

Thanks for those links. Good to have soem information specific for Folding.

I've been doing some Googling and it appears that most of the info for the RX 500-series cards is also applicable to the Navi cards.

It does appear, however, that you need at least 18.04.3 to get the kernel support for Navi so good thing I upgraded all my boxes to 18.04.4 in the last couple of weeks.

The most interesting post I found at Ask Ubuntu discusses how OpenCL is under performing and what this person did to get it perform decently.

So even though I ordered the cards from a popular on-line eTailer late in the day and didn't expect them to ship until Monday I got tracking confirmation that they shipped this evening and are on their way and should be here Monday. I guess spending the extra $20Cdn on expedited shipping also might have put them at the top of the queue.

[toTOW]

Good luck with Linux and AMD drivers ...

[gordonbb]

Good luck with Linux and AMD drivers ...

Thanks - Like I said I need a project to keep me going but I fear some Kernel-Fu will be required to get this going.

I believe either you or @foldy mentioned in another thread that he had a version of FAHBench built against OpenMM_22. I'm thinking using FAHBench might be a better method to test with given AMD's less than stellar history with drivers these days.

[Rainmaker]

Good luck with Linux and AMD drivers ...

I found it quite easy OP, so you should be OK (Threadripper 3960X and Vega 56 on Ubuntu 18.04.4). FWIW you can install the latest proprietary AMDGPU-PRO driver on 19.10 and even 20.04 as well. You just need to temporarily edit /etc/os-release to say '18.04.4', and then back again after installing. Probably best to stick to 18.04 now you've got it though, as it is the 'official' supported route.

First install just the AMD OpenCL stuff we need, omitting the actual display driver (you don't need it, the open source one is great and works with vaapi in patched Chromium, which the pro one doesn't):

Code: Select all

amdgpu-pro-install --headless --opencl=pal

Boom. The AMD Pro stack and OpenCL is done. Easy.

Then add the unprivileged fahclient user to the video group:

Code: Select all

sudo usermod -a -G video fahclient

Then replace the systemd service file so that fahclient loads properly (it's bugged and it can't access the video group or opencl out of the box):

Code: Select all

sudo nano /etc/systemd/system/FAHClient.service

Code: Select all

[Unit]
Description=Folding@Home Client
After=network.target

[Service]
Type=simple
ExecStart=/usr/bin/FAHClient /etc/fahclient/config.xml --pid-file=/var/run/fahclient/fahclient.pid --daemon
PIDFile=/var/run/fahclient/fahclient.pid
User=fahclient
RuntimeDirectory=fahclient
WorkingDirectory=~

[Install]
WantedBy=multi-user.target

To be safe, download GPUs.txt and copy it to the correct location. My FaH install didn't fetch GPUs.txt and a GPU won't show up properly without it:

Code: Select all

wget https://fah-web.stanford.edu/file-releases/public/GPUs.txt
sudo mv GPUs.txt /var/lib/fahclient/

After a reboot you should be golden. Just launch FAHControl and add the GPU slot, which should be detected and work fine now. Without the above steps you may get the GPU recognised in the slot (or may not), but you'll get errors ranging from BAD_WORK_UNIT to no OpenCL to plain old no-gpu. I've been crunching GPU WU on Linux no problem since I followed the above (and used WattmanGTK to overclock and undervolt the card for maximum gains).

[gordonbb]

I found it quite easy OP, so you should be OK (Threadripper 3960X and Vega 56 on Ubuntu 18.04.4). ...

Thanks!

I’ll definitely give that a try, but I expect to have some issues with OpenCL.

Moved some hardware around today and freed up:

Fractal Meshify S
2 x 140mm Corsair AF front-intake
140mm Fractal Bottom Intake
140mm Noctua iPPC 3000 as rear exhaust
Corsair TX-750M
Gigabyte x370 Gaming K7
AMD 2700x with a Gammax 120mm AIO
8GB DDR4

The video cards arrived this morning in my City but won’t be delivered until Monday.

I’ve got a trusty HD5670 in the system now and I’m going to do a Fresh Install of Win 10 Home 64-bit and Ubuntu 18.04.4 Desktop tomorrow so I can dual-boot and compare Windows vs. Linux performance.

And, of course, the CPU is folding away tonight while it waits.

[gordonbb]

The hardware has arrived. It's folding under Windows for now so I can get a base-line of what to expect. About 1MPPD per card so far

[TxRedneck]

The hardware has arrived. It's folding under Windows for now so I can get a base-line of what to expect. About 1MPPD per card so far

1M PPD per card?! HORY SHEET! I need to update stat!

[MrFrizzy]

1M PPD per card?! HORY SHEET! I need to update stat!

Yup, the 5700 XT and Radeon 7 both regularly do over 1M PPD from the AMD side of things. I suspect the 5700 could as well, but probably needs a good overclock to achieve consistently.

From what I have seen on the Nvidia side, the RTX 2060 Super (maybe the non-super too) and up can all consistently do over 1M PPD. The 2080 Ti can actually exceed 3M PPD on some projects.

[TxRedneck]

1M PPD per card?! HORY SHEET! I need to update stat!

Yup, the 5700 XT and Radeon 7 both regularly do over 1M PPD from the AMD side of things. I suspect the 5700 could as well, but probably needs a good overclock to achieve consistently.

From what I have seen on the Nvidia side, the RTX 2060 Super (maybe the non-super too) and up can all consistently do over 1M PPD. The 2080 Ti can actually exceed 3M PPD on some projects.

I've been outta the fold since 2014 and as I recall, you had to have a small farm of hardware to pull down those kinda numbers. It's insane to see how far things have some.

Tx

[Shirty]

From what I have seen on the Nvidia side, the RTX 2060 Super (maybe the non-super too) and up can all consistently do over 1M PPD. The 2080 Ti can actually exceed 3M PPD on some projects.

Spot on, on Core22 jobs it's unusual to see the base 2060 lower than 1M, the one in my collection is sitting at around 1.3M at present. It's my least powerful card

[TxRedneck]

From what I have seen on the Nvidia side, the RTX 2060 Super (maybe the non-super too) and up can all consistently do over 1M PPD. The 2080 Ti can actually exceed 3M PPD on some projects.

Spot on, on Core22 jobs it's unusual to see the base 2060 lower than 1M, the one in my collection is sitting at around 1.3M at present. It's my least powerful card

You mind detailing what you're folding on to average what you do and if you've an estimate on your electricity costs?

TIA,
Tx

[gordonbb]

Its Alive!

Thanks to @rainmaker his instructions and the Preview version 20 drivers and the other fixes all worked like a charm. Definitely easier than the first time I tried to get NVidia cards working under Linux.

Both GPUs are folding away each with an estimate of a little over 1MPPD 10% into their 1st WU. So the same or slightly better that the average from the 6 or 7 WUs per card on Windows from my baseline there yesterday: 968,472PPD & 955,377PPD

Now to work on fan-control and getting the cards instrumented in Zabbix

[gordonbb]

Tried a mainline Kernel 5.5 to see if the OpenCL enhancements AMD put in place had any effect. Unfortunately though the preview driver would not compile against Kernel 5.5.14.

So the GPUs are working as well under Linux as under Windows producing a little under 1MPPD (975.31 & 977.14kPPD on a 24-hour average) but comparing the theoretical Single Precision performance using the values on TechPowerup.com

RTX 2060S 7.181 TFLOPS
RTX 2070S 9.062 TFLOPS
RX 5700XT 9.754 TFLOPS

Leads me to suspect there is still some performance left on the table with Navi.

My observed performance for these GPUs are:
RX 5700XT 976kPPD
RTX 2060S 1420kPPD
RTX 2070S 1910kPPD

It is a shame the new 20.04 LTS Ubuntu version due out in a few days will only support Kernel 5.4 as if the AMD changes submitted to Kernel 5.5 don't get back-ported to 5.4 then Navi's compute performance may continue to suffer.