FAH Core 24 Fails Xubuntu 18.04 extended

[58Enfield]

Most of my platforms are Intel Gen 4 & Nvidia....on Xubuntu 18.04 extended. All core 24 WUs die instantly. Spent a lot of effort last Fall & Winter trying to find something I liked more modern OS while avoiding V8. I like my Cli control of FAH processes and Nvidia wattages. Can't tolerate 200 watts. Didn't find any so extended 18.04. Now core 24 won't run....any work arounds?

******************************* System ********************************
04:30:12: CPU: Intel(R) Core(TM) i5-4690S CPU @ 3.20GHz
04:30:12: CPU ID: GenuineIntel Family 6 Model 60 Stepping 3
04:30:12: CPUs: 4
04:30:12: Memory: 31.20GiB
04:30:12: Free Memory: 27.68GiB
04:30:12: Threads: POSIX_THREADS
04:30:12: OS Version: 5.4
04:30:12: Has Battery: false
04:30:12: On Battery: false
04:30:12: UTC Offset: -7
04:30:12: PID: 2340
04:30:12: CWD: /home/h3rt/FAH13
04:30:12: OS: Linux 5.4.0-195-generic x86_64
04:30:12: OS Arch: AMD64
04:30:12: GPUs: 1
04:30:12: GPU 0: Bus:1 Slot:0 Func:0 NVIDIA:8 GA104 [GeForce RTX 3060 Ti]
04:30:12: CUDA Device 0: Platform:0 Device:0 Bus:1 Slot:0 Compute:8.6 Driver:12.2
04:30:12: OpenCL Device 0: Platform:0 Device:0 Bus:1 Slot:0 Compute:3.0 Driver:535.183

Xubuntu 18.04 (extended) Ubuntu Pro

***********************************************************************

This is on a 4th Gen Intel i7 4790K and RTX 4070 Super

(4 rapid failures)

Xubuntu 18.04 (extended) Ubuntu Pro Software & OS All The Same

Still doesn't work, at all, with openmm-core-24 aka Core_24.fah

I left another 4th Gen Intel RTX 3060 Ti combo running @ new house 95 miles NW of here would highly surprised if it isn't doing the same thing.

Over last 2 weeks tried stone stock, way underwatted 100 watts, stable underwatt for last 9 months.....nothing works for Core 24.

I'm in midst of moving so replies will be very slow. Thank You

[Gary480six]

I'm having the same issue as the OP.
Linux Mint 19.2 - Intel i5-2500K - GTX 1070
System is GPU only Folding.
Works fine with Core22 and Core23 work
Same four rapid Failures with Core24.

My solution is to shut off Folding on that system.

Because... after what happened after the release of Core23, I do not feel anyone will make any effort to fix this issue either.
For some history, Core23 was released and Windows 7 GPU Folding boxes started locking up.
Seemed like nobody ever looked into it - or offered a working solution. Or even offered to restrict Windows 7 systems to Core22 work.
I know people who gave up on Folding when that Core23 - Windows 7 issue started.

This new Core24 issue could be fixed by blocking Linux systems from getting Core24 work.

Can someone at least look into that?

[58Enfield]

Gary, are you using FAH7 or FAH8 to manage folding? And what is the newest kernel available to Mint 19.2?

I did go to github for openmm-core-24 hoping for a simple declarative statement such as "wont work on kernel XXXX or earlier" but no luck. I'm not a programmer or developer and frequently have to rely on those type of statements in documentation to give me a clue as to what's going wrong.

Based on nothing but experience (23 years of GAH & FAH) my guess would be a developer or team relied on a pull in the kernel or some compute function in our older cpu's that's not available and either did not document the change nor communicate it to us.

That's not an attack on FAH or developers, but it certainly is frustrating for we users who are after all are providing massive compute services to the project for free.

[bollix47]

FYI
Although I'm currently using Ubuntu 24.04 I updated to it from 22.04 and 18.04 and have not had a problem with core 24.
Folding v8 but that's a fairly recent update. Prior to v8 I was using v7.6.21.

Drivers:

NVIDIA-SMI 535.183.01 Driver Version: 535.183.01 CUDA Version: 12.2

The diver, "nvidia-driver-535 (proprietary)", was installed using Software & Updates > Additional Drivers found in the Show Apps (bottom left on ubuntu) menu

Another software that I make sure is installed that seems to help is ocl-icd-opencl-dev :

Code: Select all

sudo apt install ocl-icd-opencl-dev

One other option to explore might be to remove core 24 and let the client download it again in case something was not right with the first download.

[Gary480six]

Hi guys - thanks for the feedback.
Forgot to mention - it's running the last FAH7 version.
Yesterday I updated From Mint 19.2 to Mint 19.3.
I see no option from within 19.3 to update to Mint 20 or newer.
Today I deleted the Core24 Folder and let it download again.
Nothing has changed. Core 22 and Core 23 work starts and completes.
Core 24 just errors out.
Oh and it has the latest Kernel that 19.3 allows. I believe it was 5.4
(the system is off now)
Also cannot check the Nvidia drivers now - but it should be somewhere
in the 500 series. 512.xx 520.xx somewhere in there.
The last issue is that if I keep fooling with this and have 4 failed work units
in a row again and again... eventually I will lose my Bonus status.
Not worth it.
And, it appears that the GitHub 'fix' that allows Linux to run the Advanced Control
app has also reached end of life. (files no longer in the repository)
So for now I will give up on Linux Folding @ Home.
But I will keep an eye on this post - in case someone spots an easy fix.

[toTOW]

I think core 24 requires a newer glibc library that it only available with Ubuntu 20.04 or newer ...

[Nicolas_orleans]

Hello,

I have resumed folding recently, good to see familiar names like toTOW and bollix47 are still around.

After installing a v8.3.18 client on an Ubuntu 20.04 LTS machine (upgrade from a 18.04 old install) equipped with a GTX 980 Ti, I faced the same issue as 58Enfield and Gary480six, eg an immediate failure (FAILED_2 (1) and then WU dump) after WU downloading for my first 3-4 Core24 assignments. Core22 works flawlessly, not seen Core23 assignments yet.

I took me a day to solve it. Summary of my findings:
- Ubuntu 20.04 LTS (5.15 kernel & 470.256 Ubuntu Nvidia driver): FAILED_2 (1) and then WU dump
- Ubuntu 22.04 LTS (470.256 & logically 6.5 kernel since upgraded from 20.04 LTS): folds with OpenCL, no way to install a more recent driver and have it folding with CUDA, see next post.
- Ubuntu 24.04 LTS (6.8 kernel & 550.107 Ubuntu Nvidia driver): folds with CUDA

I will post the detailed steps in my next post in case it has some usefulness. But I would think Mint 22 & Xubuntu 24.04 with nvidia-driver-550 would do the trick.

Best regards

Nicolas

[Nicolas_orleans]

Hello,

Here are the steps taken and investigations carried out:

1/ Ubuntu 20.04 LTS: make OpenCL work just in case it would help the core to work on OpenCL.

My GPU was identified by the client as OpenCL = unsupported. Installed opencl-headers, ocl-icd-libopencl1 and nvidia-cuda-toolkit. One of them installed ocl-icd-opencl-dev mentioned by bollix47. The GPU became identified as OpenCL = supported, Compute = 3.0. But the Core24 still FAILED_2 (1) and then WU dump.

2/ Investigations based on toTOW's comment on glibc library. Given the name of the download folder on FAH servers, and the name of the corresponding local install directory, I assumed the Core24 was build for CentOS 7.9.2009. After investigating:
- for this version of CentOS, versions of glibc are between 2.17_317 and 2.17_324
- Ubuntu 20.04 LTS is glibc 2.31 so appears newer

Investigation unconclusive, impossible to know if FAH client / servers have folder names that were not updated, or if an exotic patched version of glibc 2.17 was used, containing something that does not exist in glibc 2.31.

3/ Upgrade to Ubuntu 22.04.LTS, it folds on OpenCL after failing on CUDA due to old nvidia-driver-470 that is a CUDA 11.4 driver.

I tried to install from Ubuntu repositories nvidia-driver-550 and nvidia-driver-545 but it failed. I also tried to install with a shell the latest driver compatible for my card downloaded from the Nvidia website (NVIDIA-Linux-x86_64-550.120.run), but there was an issue with the 22.04 kernel (should be 6.5 but I did not check)

4/ Upgrade to Ubuntu 24.04 LTS, installation from Ubuntu repositories of nvidia-driver-550 that is a CUDA 12.4 driver, and it folds under CUDA.

"Conclusions":
- OpenCL: works on a modern distribution with 6.5 kernel
- CUDA should work on a modern distribution with a 6.5 kernel and any CUDA 12.4 driver, if possible to install
- Ubuntu 24.04 LTS works on CUDA after replacing nvidia-driver-550 over nvidia-driver-470. Since bollix47 managed to make the Core24 work under nvidia-driver-535, I digged into CUDA's documentation (https://docs.nvidia.com/deploy/cuda-compatibility/) and found out CUDA 12.x compatibility starts with Nvidia driver 525.60.13 so I would assume this is the minimum driver to have it fold with CUDA under a >= 6.5 kernel.

Sorry for the technical content. I used to do FAH beta/alpha (Ocores) stuff a few years ago, so I managed to solve my issue, but donors should not have to try to guess minimal requirement of production cores, in my humble opinion.

Best regards

Nicolas

[58Enfield]

Thank You, Nicolas for the deep dive as to how "far" we would have to update to get the appropriate gilbc - kernel - driver complex to run Core 24. That will save a lot of time and effort spent "poking around" during new experiments. We are on the last day of our move, and will make decision on continuing FAH when I am not physically and mentally exhausted. Our energetic youth passed us by quite some time ago, heh heh.