HOWTO: How I got my AMD card folding on Linux
Posted: Wed Mar 25, 2020 6:47 pm
My setup is as follows:
I created this guide to share my findings and experience in setting up Folding@Home on this computer. I hope that it can provide some insight and help to others.
Installing the AMDGPU-PRO OpenCL implementation
I use the free software GPU driver stack. Unfortunately, I could not get Folding@Home to work with the free software OpenCL implementations from ROCm or Mesa. So I will not bother explaining how to set those up. If you have them, you may want to remove them in case they interfere.
What did work was something suggested by dschuermann on IRC (#radeon on freenode; great people there). This allows you to use the OpenCL implementation from the proprietary AMDGPU-PRO driver, without installing the whole driver.
Add the fahclient user to the "video" and "render" (if you have it) groups:
This is necessary because FAHClient must be able to access devices in /dev/dri, such as /dev/dri/card0 and /dev/dri/renderD128. Some distributions place the latter under the "render" group instead of the "video" group, so you may need both if your system is like that.
Some donors have been getting "OpenCL: Not detected: clGetDeviceIDs() returned -1" unless they run FAHClient as root, and adding the fahclient user to the video group doesn't fix it.
I observed this myself, and upon investigating, I discovered the problem. The official Folding@Home packages (at least the Debian-based ones) set up FAHClient to be launched by an init.d script, which launches FAHClient with "--run-as fahclient", so that it runs as the "fahclient" user.
The problem is that when FAHClient is launched with the --run-as option, it does not give itself the supplementary groups of the user passed into the --run-as option. Thus the FAHClient process will not have the permissions of the groups that the "fahclient" user is a member of. You can see this for yourself:
The lack of any group numbers after "Groups:" shows that the FAHClient process does not have any supplementary groups, even though it is running as the "fahclient" user. This is due to a bug in FAHClient.
For the developers: If I had to guess (I have not verified this), this is probably happening because FAHClient uses setuid() to drop privileges with the "--run-as" argument. This is not sufficient. You must call initgroups() before setuid() to give the process the supplementary groups of the target user.
TL;DR For the users: To work around this, I created a systemd unit to start FAHClient instead of the init.d script. This works by using the systemd unit's "User" option to change users instead of having FAHClient do it.
First, stop FAHClient ("sudo systemctl stop FAHClient").
Create /etc/systemd/system/FAHClient.service (The name matters; calling it this will automatically supersede the init.d script of the same name), with the following contents:
Then do the following:
Code: Select all
Radeon R9 290 (Hawaii) with free software drivers (amdgpu/mesa)
Intel Core i7-4790
Linux Mint 18.1 64-bit
Linux kernel v5.5.1
Installing the AMDGPU-PRO OpenCL implementation
I use the free software GPU driver stack. Unfortunately, I could not get Folding@Home to work with the free software OpenCL implementations from ROCm or Mesa. So I will not bother explaining how to set those up. If you have them, you may want to remove them in case they interfere.
What did work was something suggested by dschuermann on IRC (#radeon on freenode; great people there). This allows you to use the OpenCL implementation from the proprietary AMDGPU-PRO driver, without installing the whole driver.
- Get the AMDGPU-PRO driver package:
- Go to https://www.amd.com/en/support and select your graphics card.
- Scroll down to Ubuntu x86 64-bit and download the file. (currently: amdgpu-pro-19.50-967956-ubuntu-18.04.tar.xz)
- Extract it to an empty directory somewhere. Do not run the install script or anything else.
- Extract the OpenCL driver from the appropriate .deb file.
For Vega 10 and later products, this is the .deb file matching the pattern opencl-amdgpu-pro-icd_*_amd64.deb.
For products older than Vega 10, this is the .deb file matching the pattern opencl-orca-amdgpu-pro-icd_*_amd64.deb.- In a terminal in the directory with the .deb files, run the following:
Code: Select all
# Extract the contents of the .deb package to the current directory dpkg -x DEB_FILE . # replace DEB_FILE with the appropriate pattern from above # Replace the path in the .icd file with the full path to where we're going to put the library sed -i 's|^|/opt/amdgpu-pro/lib/x86_64-linux-gnu/|' etc/OpenCL/vendors/amdocl*.icd # Copy the etc and opt directories extracted from the package to the system root. sudo cp -r etc opt /
- In a terminal in the directory with the .deb files, run the following:
Add the fahclient user to the "video" and "render" (if you have it) groups:
Code: Select all
sudo usermod -aG video fahclient
sudo usermod -aG render fahclient # don't worry if you don't have a "render" group
Some donors have been getting "OpenCL: Not detected: clGetDeviceIDs() returned -1" unless they run FAHClient as root, and adding the fahclient user to the video group doesn't fix it.
I observed this myself, and upon investigating, I discovered the problem. The official Folding@Home packages (at least the Debian-based ones) set up FAHClient to be launched by an init.d script, which launches FAHClient with "--run-as fahclient", so that it runs as the "fahclient" user.
The problem is that when FAHClient is launched with the --run-as option, it does not give itself the supplementary groups of the user passed into the --run-as option. Thus the FAHClient process will not have the permissions of the groups that the "fahclient" user is a member of. You can see this for yourself:
Code: Select all
$ pidof FAHClient
26801 26796
$ sudo cat /proc/26796/status | grep 'Groups:'
Groups:
For the developers: If I had to guess (I have not verified this), this is probably happening because FAHClient uses setuid() to drop privileges with the "--run-as" argument. This is not sufficient. You must call initgroups() before setuid() to give the process the supplementary groups of the target user.
TL;DR For the users: To work around this, I created a systemd unit to start FAHClient instead of the init.d script. This works by using the systemd unit's "User" option to change users instead of having FAHClient do it.
First, stop FAHClient ("sudo systemctl stop FAHClient").
Create /etc/systemd/system/FAHClient.service (The name matters; calling it this will automatically supersede the init.d script of the same name), with the following contents:
Code: Select all
[Unit]
Description=Folding@Home Client
After=network.target
[Service]
Type=simple
ExecStart=/usr/bin/FAHClient /etc/fahclient/config.xml --pid-file=/var/run/fahclient/fahclient.pid --daemon
PIDFile=/var/run/fahclient/fahclient.pid
User=fahclient
RuntimeDirectory=fahclient
WorkingDirectory=~
[Install]
WantedBy=multi-user.target
Code: Select all
sudo systemctl daemon-reload
sudo systemctl start FAHClient.service
# And, if you want it to start automatically on boot...
sudo systemctl enable FAHClient.service
