Ultimate AMD GPU Linux FAH Guide for OpenCL (FAH-Client v8 only)

[muziqaz]

Ah, it is possible you iGPU is not 100% stable, and if it is using system ram as VRAM, you system ram might not be stable.
If you don't mind getting few logs for those failed WUs

[foxpy]

Of course it uses system RAM as VRAM, it doesn't have any other options. Maybe I should check my RAM, that's a good idea, though I would like to stress out CPU folding works without any problems at all

Here is an example of a failed WU:

Code: Select all

foxpy@t14s:~/.fah/logs$ rg WU33
log-20250225-113239.txt
304:07:37:08:I1:WU33:Requesting WU assignment for user foxpy team 45032
308:07:37:10:I1:WU33:Retry #1 in 2 secs
309:07:37:12:I1:WU33:Requesting WU assignment for user foxpy team 45032
313:07:37:13:I1:WU33:Retry #2 in 4 secs
314:07:37:17:I1:WU33:Requesting WU assignment for user foxpy team 45032
318:07:37:18:I1:WU33:Retry #3 in 8 secs
319:07:37:26:I1:WU33:Requesting WU assignment for user foxpy team 45032
323:07:37:27:I1:WU33:Retry #4 in 16 secs
324:07:37:43:I1:WU33:Requesting WU assignment for user foxpy team 45032
328:07:37:44:I1:WU33:Retry #5 in 32 secs
329:07:38:16:I1:WU33:Requesting WU assignment for user foxpy team 45032
333:07:38:18:I1:WU33:Retry #6 in 1 min 4 secs
334:07:39:22:I1:WU33:Requesting WU assignment for user foxpy team 45032
338:07:39:23:I1:WU33:Retry #7 in 2 mins 8 secs
339:07:41:31:I1:WU33:Requesting WU assignment for user foxpy team 45032
343:07:41:32:I1:WU33:Retry #8 in 4 mins 16 secs
345:07:45:48:I1:WU33:Requesting WU assignment for user foxpy team 45032
349:07:45:49:I1:WU33:Retry #9 in 8 mins 32 secs
351:07:54:21:I1:WU33:Requesting WU assignment for user foxpy team 45032
355:07:54:22:I1:WU33:Retry #10 in 8 mins 32 secs
357:08:02:54:I1:WU33:Requesting WU assignment for user foxpy team 45032
361:08:02:56:I1:WU33:Retry #11 in 8 mins 32 secs
363:08:11:28:I1:WU33:Requesting WU assignment for user foxpy team 45032
367:08:11:29:I1:WU33:Retry #12 in 8 mins 32 secs
369:08:20:01:I1:WU33:Requesting WU assignment for user foxpy team 45032
373:08:20:02:I1:WU33:Retry #13 in 8 mins 32 secs
375:08:28:34:I1:WU33:Requesting WU assignment for user foxpy team 45032
378:08:28:35:I1:WU33:Received WU assignment 0iUQNbxtVeEctoMwH1MDXDmxizWpOi0NEENjVYeX5cE
379:08:28:35:I1:WU33:Downloading WU
381:08:28:36:I1:WU33:DOWNLOAD 100% 46B of 46B
384:08:28:36:I1:WU33:Retry #14 in 8 mins 32 secs
386:08:37:08:I1:WU33:Requesting WU assignment for user foxpy team 45032
390:08:37:09:I1:WU33:Retry #15 in 8 mins 32 secs
392:08:45:41:I1:WU33:Requesting WU assignment for user foxpy team 45032
396:08:45:43:I1:WU33:Retry #16 in 8 mins 32 secs
398:08:54:15:I1:WU33:Requesting WU assignment for user foxpy team 45032
402:08:54:16:I1:WU33:Retry #17 in 8 mins 32 secs
403:09:02:48:I1:WU33:Requesting WU assignment for user foxpy team 45032
407:09:02:49:I1:WU33:Received WU assignment QQWIje7RVhnNSvGHL9ouuvYBOhE66NFOcuY5xSuP98k
408:09:02:49:I1:WU33:Downloading WU
410:09:02:50:I1:WU33:DOWNLOAD 100% 46B of 46B
413:09:02:50:I1:WU33:Retry #18 in 8 mins 32 secs
414:09:11:22:I1:WU33:Requesting WU assignment for user foxpy team 45032
418:09:11:23:I1:WU33:Retry #19 in 8 mins 32 secs
420:09:19:55:I1:WU33:Requesting WU assignment for user foxpy team 45032
424:09:19:57:I1:WU33:Retry #20 in 8 mins 32 secs
426:09:28:29:I1:WU33:Requesting WU assignment for user foxpy team 45032
430:09:28:30:I1:WU33:Retry #21 in 8 mins 32 secs
432:09:37:02:I1:WU33:Requesting WU assignment for user foxpy team 45032
436:09:37:03:I1:WU33:Retry #22 in 8 mins 32 secs
438:09:45:35:I1:WU33:Requesting WU assignment for user foxpy team 45032
442:09:45:36:I1:WU33:Retry #23 in 8 mins 32 secs
444:09:54:08:I1:WU33:Requesting WU assignment for user foxpy team 45032
448:09:54:10:I1:WU33:Retry #24 in 8 mins 32 secs
450:10:02:42:I1:WU33:Requesting WU assignment for user foxpy team 45032
454:10:02:43:I1:WU33:Retry #25 in 8 mins 32 secs
456:10:11:15:I1:WU33:Requesting WU assignment for user foxpy team 45032
460:10:11:16:I1:WU33:Retry #26 in 8 mins 32 secs
462:10:19:48:I1:WU33:Requesting WU assignment for user foxpy team 45032
466:10:19:49:I1:WU33:Retry #27 in 8 mins 32 secs
468:10:28:21:I1:WU33:Requesting WU assignment for user foxpy team 45032
472:10:28:23:I1:WU33:Retry #28 in 8 mins 32 secs
474:10:36:55:I1:WU33:Requesting WU assignment for user foxpy team 45032
478:10:36:56:I1:WU33:Retry #29 in 8 mins 32 secs
480:10:45:28:I1:WU33:Requesting WU assignment for user foxpy team 45032
484:10:45:29:I1:WU33:Retry #30 in 8 mins 32 secs
486:10:54:01:I1:WU33:Requesting WU assignment for user foxpy team 45032
490:10:54:02:I1:WU33:Retry #31 in 8 mins 32 secs
492:11:02:34:I1:WU33:Requesting WU assignment for user foxpy team 45032
496:11:02:35:I1:WU33:Retry #32 in 8 mins 32 secs
498:11:11:07:I1:WU33:Requesting WU assignment for user foxpy team 45032
501:11:11:09:I1:WU33:Received WU assignment R91c1H1Xa4-E3VwcQS1CZU0SausNvmWvvduM3pKRtYA
502:11:11:09:I1:WU33:Downloading WU
504:11:11:10:I1:WU33:DOWNLOAD 2% 95.44KiB of 6.26MiB
505:11:11:11:I1:WU33:DOWNLOAD 11% 723.73KiB of 6.26MiB
506:11:11:12:I1:WU33:DOWNLOAD 30% 1.86MiB of 6.26MiB
507:11:11:13:I1:WU33:DOWNLOAD 61% 3.80MiB of 6.26MiB
508:11:11:14:I1:WU33:DOWNLOAD 100% 6.26MiB of 6.26MiB
510:11:11:14:I1:WU33:Received WU P17650 R111 C6 G581
513:11:11:15:I3:WU33:Started FahCore on PID 7827
514:11:11:15:I1:WU33:*********************** Log Started 2025-02-25T11:11:15Z ***********************
515:11:11:15:I1:WU33:*************************** Core26 Folding@home Core ***************************
516:11:11:15:I1:WU33:       Core: Core26
517:11:11:15:I1:WU33:       Type: 0x26
518:11:11:15:I1:WU33:    Version: 8.2.0
519:11:11:15:I1:WU33:     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
520:11:11:15:I1:WU33:  Copyright: 2022 foldingathome.org
521:11:11:15:I1:WU33:   Homepage: https://foldingathome.org/
522:11:11:15:I1:WU33:       Date: Jan 7 2025
523:11:11:15:I1:WU33:       Time: 00:35:47
524:11:11:15:I1:WU33:   Revision: 4f149b599caa4725076ef2de3b47c8d7ce725787
525:11:11:15:I1:WU33:     Branch: HEAD
526:11:11:15:I1:WU33:   Compiler: GNU 7.5.0
527:11:11:15:I1:WU33:    Options: -faligned-new -std=c++14 -fsigned-char -ffunction-sections
528:11:11:15:I1:WU33:             -fdata-sections -O3 -funroll-loops -fno-pie
529:11:11:15:I1:WU33:             -DOPENMM_VERSION="\"8.2.0\""
530:11:11:15:I1:WU33:   Platform: linux 6.8.0-1017-azure
531:11:11:15:I1:WU33:       Bits: 64
532:11:11:15:I1:WU33:       Mode: Release
533:11:11:15:I1:WU33:Maintainers: John Chodera <john.chodera@choderalab.org> and Peter Eastman
534:11:11:15:I1:WU33:             <peastman@stanford.edu>
535:11:11:15:I1:WU33:       Args: -dir R91c1H1Xa4-E3VwcQS1CZU0SausNvmWvvduM3pKRtYA -suffix 01
536:11:11:15:I1:WU33:             -version 8.4.9 -lifeline 4132 -gpu-platform opencl -gpu-vendor amd
537:11:11:15:I1:WU33:             -opencl-platform 0 -opencl-device 0 -gpu 0
538:11:11:15:I1:WU33:************************************ libFAH ************************************
539:11:11:15:I1:WU33:       Date: Jan 7 2025
540:11:11:15:I1:WU33:       Time: 00:29:24
541:11:11:15:I1:WU33:   Revision: c7d2824a47eb025fa8cda8968c7a5e971585d90c
542:11:11:15:I1:WU33:     Branch: HEAD
543:11:11:15:I1:WU33:   Compiler: GNU 7.5.0
544:11:11:15:I1:WU33:    Options: -faligned-new -std=c++11 -fsigned-char -ffunction-sections
545:11:11:15:I1:WU33:             -fdata-sections -O3 -funroll-loops -fno-pie
546:11:11:15:I1:WU33:   Platform: linux 6.8.0-1017-azure
547:11:11:15:I1:WU33:       Bits: 64
548:11:11:15:I1:WU33:       Mode: Release
549:11:11:15:I1:WU33:************************************ CBang *************************************
550:11:11:15:I1:WU33:    Version: 1.7.2
551:11:11:15:I1:WU33:     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
552:11:11:15:I1:WU33:        Org: Cauldron Development LLC
553:11:11:15:I1:WU33:  Copyright: Cauldron Development LLC, 2003-2024
554:11:11:15:I1:WU33:   Homepage: https://cauldrondevelopment.com/
555:11:11:15:I1:WU33:    License: LGPL-2.1-or-later
556:11:11:15:I1:WU33:       Date: Jan 7 2025
557:11:11:15:I1:WU33:       Time: 00:28:59
558:11:11:15:I1:WU33:   Revision: f1cd4c791e8c40a35dcfeab3ab85d910949cc0cb
559:11:11:15:I1:WU33:     Branch: HEAD
560:11:11:15:I1:WU33:   Compiler: GNU 7.5.0
561:11:11:15:I1:WU33:    Options: -faligned-new -std=c++11 -fsigned-char -ffunction-sections
562:11:11:15:I1:WU33:             -fdata-sections -O3 -funroll-loops -fno-pie -fPIC
563:11:11:15:I1:WU33:   Platform: linux 6.8.0-1017-azure
564:11:11:15:I1:WU33:       Bits: 64
565:11:11:15:I1:WU33:       Mode: Release
566:11:11:15:I1:WU33:************************************ System ************************************
567:11:11:15:I1:WU33:        CPU: AMD Ryzen 7 PRO 5850U with Radeon Graphics
568:11:11:15:I1:WU33:     CPU ID: AuthenticAMD Family 25 Model 80 Stepping 0
569:11:11:15:I1:WU33:       CPUs: 16
570:11:11:15:I1:WU33:     Memory: 30.18GiB
571:11:11:15:I1:WU33:Free Memory: 17.89GiB
572:11:11:15:I1:WU33: OS Version: 6.6
573:11:11:15:I1:WU33:Has Battery: false
574:11:11:15:I1:WU33: On Battery: false
575:11:11:15:I1:WU33:   Hostname: t14s
576:11:11:15:I1:WU33: UTC Offset: 2
577:11:11:15:I1:WU33:        PID: 7827
578:11:11:15:I1:WU33:        CWD: /home/foxpy/.fah/work
579:11:11:15:I1:WU33:       Exec: /home/foxpy/.fah/cores/openmm-core-26/centos-7.9.2009-64bit/release/fahcore-26-centos-7.9.2009-64bit-release-8.2.0/FahCore_26
580:11:11:15:I1:WU33:************************************ OpenMM ************************************
581:11:11:15:I1:WU33:    Version: 8.2.0
582:11:11:15:I1:WU33:********************************************************************************
583:11:11:15:I1:WU33:Project: 17650 (Run 111, Clone 6, Gen 581)
584:11:11:15:I1:WU33:Reading tar file core.xml
585:11:11:15:I1:WU33:Reading tar file integrator.xml.bz2
586:11:11:15:I1:WU33:Reading tar file state.xml.bz2
587:11:11:15:I1:WU33:Reading tar file system.xml.bz2
588:11:11:15:I1:WU33:Digital signatures verified
589:11:11:15:I1:WU33:Folding@home GPU Core26 Folding@home Core
590:11:11:15:I1:WU33:Version 8.2
591:11:11:15:I1:WU33:  Checkpoint write interval: 125000 steps (5%) [20 total]
592:11:11:15:I1:WU33:  JSON viewer frame write interval: 25000 steps (1%) [100 total]
593:11:11:15:I1:WU33:  XTC frame write interval: 12500 steps (0.5%) [200 total]
594:11:11:15:I1:WU33:  TRR frame write interval: disabled
595:11:11:15:I1:WU33:  Global context and integrator variables write interval: disabled
596:11:11:15:I1:WU33:There are 3 platforms available.
597:11:11:15:I1:WU33:Platform 0: Reference
598:11:11:15:I1:WU33:Platform 1: CPU
599:11:11:15:I1:WU33:Platform 2: OpenCL
600:11:11:15:I1:WU33:  opencl-device 0 specified
601:11:11:20:I1:WU33:Attempting to create OpenCL context:
602:11:11:20:I1:WU33:  Configuring platform OpenCL
606:11:26:20:I1:WU33:Caught signal SIGINT(2)
607:11:26:20:I1:WU33:Exiting, please wait. . .
614:11:27:22:E :WU33:Core was killed
615:11:27:22:E :WU33:Core returned FAILED_1 (0)
616:11:27:22:E :WU33:Run did not produce any results. Dumping WU
617:11:27:22:I1:WU33:Sending dump report
620:11:27:23:I1:WU33:Dumped

As I said before, it was stuck at creating OpenCL context without using almost any CPU time at all. Seems like some weird deadlock for me. I remember trying to strace the threads of fah core to see what it is doing, but there was nothing but a few timing out futex() calls in the main thread.

[muziqaz]

What Fedora version and which kernel?
How much ram is assigned to iGPU?
Have you ticked Beta box anywhere?

[foxpy]

What Fedora version and which kernel?

It's "Fedora Linux 41 (KDE Plasma)", with vanilla kernel: Linux t14s 6.13.5-200.fc41.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Feb 27 15:07:31 UTC 2025 x86_64 GNU/Linux. I recently updated it, but I think during the times when it failed, I used kernel version 6.13.4.

How much ram is assigned to iGPU?

radeontop reports that it has 991MB VRAM and 15436MB GTT (I assume that's dynamic VRAM extension?)

Have you ticked Beta box anywhere?

Nope, I never tried folding beta projects.

[muziqaz]

Core22 failed because of GLIBC mismatch, core25 issue has been encountered in testing with Nvidia and is being fixed (hopefully).
You shouldn't be getting anymore core26 work. If you are, let me know

[foxpy]

Core22 failed because of GLIBC mismatch

Why do you think so? It didn't fail at link time, it just couldn't create OpenCL context, for some reason. Or are these things actually related somehow?

You shouldn't be getting anymore core26 work

Why?

If you are, let me know

I have just started folding on this iGPU again, I will tell you what happens once it gets any assignment

[muziqaz]

Core26 is Beta only for now.

[foxpy]

It has been pinging assign server for 7 hours so far, without any job being assigned to it. I assume this iGPU is no longer viable for folding.

And speaking about core26 jobs, I am pretty sure it wasn't beta a few weeks ago, because I was getting work units for it.

[muziqaz]

It has been pinging assign server for 7 hours so far, without any job being assigned to it. I assume this iGPU is no longer viable for folding.

And speaking about core26 jobs, I am pretty sure it wasn't beta a few weeks ago, because I was getting work units for it.

core26 was moved back to beta couple of days ago. And yes, don't bother with iGPU folding

[Joe_H]

It has been pinging assign server for 7 hours so far, without any job being assigned to it. I assume this iGPU is no longer viable for folding.

And speaking about core26 jobs, I am pretty sure it wasn't beta a few weeks ago, because I was getting work units for it.

WUs for the iGPUs come in cycles as there are projects with small enough protein systems to assign to them. Some of the less powerful ones from Intel and AMD could be approaching end of usability for folding. The more powerful ones packaged in the AMD processors may still have some life left.

A few weeks ago the developers for Core_26 did release it to full FAH with a batch of test WUs to obtain additional error reporting outside the setups available from the internal and beta testers. That is supposed to have been shutdown and be beta only now.

[foxpy]

I honestly thought folding on Intel iGPUs is straight up impossible, but I guess the new Arc ones are supported.

Thank you for helping out! I will stick to 100% CPU folding on this machine then.

[Joe_H]

The latest discrete Intel GPUs, the Battlemage B570 and B580, do support double precision. So far no word if they will leave that enabled for iGPUs included in a processor package based on the Intel Arc II.

The previous generation Intel Arc chips did not include hardware double precision in either the mobile iGPU processor packages or the discrete video cards. Those are unsupported for folding. There were two data center GPUs released based on the Xe architecture, those appear to have had double precision enabled. I don't expect to see those being used for folding at a price of $8500 for the higher end model.

[foxpy]

Wait, does that mean that when I fold on modern nvidia GPUs, I use double precision math, which these GPUs handle rather poorly? I always assumed that fah cores use single precision. Well, this explains why CPUs gain a lot more points compared to iGPUs. iGPUs might be very good in single precision math, but there is barely any double precision performance capacity in them.

[muziqaz]

Fah uses mixed precision, mostly single precision.

[Joe_H]

Yes, F@h uses a small amount of double precision calculations where needed to maintain sufficient accuracy. OpenMM has an option to push those calculations to the CPU, but that is often slower than the consumer grade GPU FP64 calculations. So the F@h GPU cores have been coded to do them on the GPU and require FP64 support. The vast majority of calculations are FP32.

For the Intel Xe chips support in the driver was provided to emulate FP64 being available on the GPU. It wasn't supported on all platforms, and needed to be enabled. F@h testing of that found it was not stable enough.

The other issue with iGPUs is that they share the same TDP as the CPU on the processor package. That means intensive iGPU usage for calculations can lead to thermal throttling, especially if CPU folding is also being done at the same time.