Folding Forum

Community driven support forum for Folding@home
https://foldingforum.org/

Issues with Radeon R9 200/300 series

https://foldingforum.org/viewtopic.php?t=33371

Page 1 of 1

Issues with Radeon R9 200/300 series

Posted: Thu Mar 26, 2020 1:01 am
by smbrelay
Yet another problem related to AMD GPU as many folks reported here. GPU unit is not getting assignments or fails to start folding. My logs and setup below.

I have tried to use FAHBench-2.3.2. The benchmark runs without errors. CPU assignments are working as well.

OS: Ubuntu 18.04.2
GPU hardware: AMD Radeon (TM) R9 390 Series
FAHClient: 7.4.4


clinfo

Code: Select all

Number of platforms                               1
  Platform Name                                   AMD Accelerated Parallel Processing
  Platform Vendor                                 Advanced Micro Devices, Inc.
  Platform Version                                OpenCL 2.1 AMD-APP (3004.6)
  Platform Profile                                FULL_PROFILE
  Platform Extensions                             cl_khr_icd cl_amd_event_callback cl_amd_offline_devices 
  Platform Host timer resolution                  1ns
  Platform Extensions function suffix             AMD

  Platform Name                                   AMD Accelerated Parallel Processing
Number of devices                                 1
  Device Name                                     Hawaii
  Device Vendor                                   Advanced Micro Devices, Inc.
  Device Vendor ID                                0x1002
  Device Version                                  OpenCL 1.2 AMD-APP (3004.6)
  Driver Version                                  3004.6
  Device OpenCL C Version                         OpenCL C 1.2 
  Device Type                                     GPU
  Device Board Name (AMD)                         AMD Radeon (TM) R9 390 Series
  Device Topology (AMD)                           PCI-E, 01:00.0
  Device Profile                                  FULL_PROFILE
  Device Available                                Yes
  Compiler Available                              Yes
  Linker Available                                Yes
  Max compute units                               44
  SIMD per compute unit (AMD)                     4
  SIMD width (AMD)                                16
  SIMD instruction width (AMD)                    1
  Max clock frequency                             1080MHz
  Graphics IP (AMD)                               7.2
  Device Partition                                (core)
    Max number of sub-devices                     44
    Supported partition types                     None
  Max work item dimensions                        3
  Max work item sizes                             1024x1024x1024
  Max work group size                             256
  Preferred work group size (AMD)                 256
  Max work group size (AMD)                       1024
  Preferred work group size multiple              64
  Wavefront width (AMD)                           64
  Preferred / native vector sizes                 
    char                                                 4 / 4       
    short                                                2 / 2       
    int                                                  1 / 1       
    long                                                 1 / 1       
    half                                                 1 / 1        (n/a)
    float                                                1 / 1       
    double                                               1 / 1        (cl_khr_fp64)
  Half-precision Floating-point support           (n/a)
  Single-precision Floating-point support         (core)
    Denormals                                     No
    Infinity and NANs                             Yes
    Round to nearest                              Yes
    Round to zero                                 Yes
    Round to infinity                             Yes
    IEEE754-2008 fused multiply-add               Yes
    Support is emulated in software               No
    Correctly-rounded divide and sqrt operations  Yes
  Double-precision Floating-point support         (cl_khr_fp64)
    Denormals                                     Yes
    Infinity and NANs                             Yes
    Round to nearest                              Yes
    Round to zero                                 Yes
    Round to infinity                             Yes
    IEEE754-2008 fused multiply-add               Yes
    Support is emulated in software               No
  Address bits                                    64, Little-Endian
  Global memory size                              7610015744 (7.087GiB)
  Global free memory (AMD)                        7411880 (7.069GiB)
  Global memory channels (AMD)                    16
  Global memory banks per channel (AMD)           16
  Global memory bank width (AMD)                  256 bytes
  Error Correction support                        No
  Max memory allocation                           4244635648 (3.953GiB)
  Unified memory for Host and Device              No
  Minimum alignment for any data type             128 bytes
  Alignment of base address                       2048 bits (256 bytes)
  Global Memory cache type                        Read/Write
  Global Memory cache size                        16384 (16KiB)
  Global Memory cache line size                   64 bytes
  Image support                                   Yes
    Max number of samplers per kernel             16
    Max size for 1D images from buffer            134217728 pixels
    Max 1D or 2D image array size                 2048 images
    Base address alignment for 2D image buffers   256 bytes
    Pitch alignment for 2D image buffers          256 pixels
    Max 2D image size                             16384x16384 pixels
    Max 3D image size                             2048x2048x2048 pixels
    Max number of read image args                 128
    Max number of write image args                8
  Local memory type                               Local
  Local memory size                               32768 (32KiB)
  Local memory syze per CU (AMD)                  65536 (64KiB)
  Local memory banks (AMD)                        32
  Max number of constant args                     8
  Max constant buffer size                        4244635648 (3.953GiB)
  Preferred constant buffer size (AMD)            16384 (16KiB)
  Max size of kernel argument                     1024
  Queue properties                                
    Out-of-order execution                        No
    Profiling                                     Yes
  Prefer user sync for interop                    Yes
  Profiling timer resolution                      1ns
  Profiling timer offset since Epoch (AMD)        1585172370080340821ns (Wed Mar 25 23:39:30 2020)
  Execution capabilities                          
    Run OpenCL kernels                            Yes
    Run native kernels                            No
    Thread trace supported (AMD)                  Yes
    Number of async queues (AMD)                  2
    Max real-time compute queues (AMD)            0
    Max real-time compute units (AMD)             714066608
    SPIR versions                                 1.2
  printf() buffer size                            4194304 (4MiB)
  Built-in kernels                                
  Device Extensions                               cl_khr_fp64 cl_amd_fp64 cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_khr_3d_image_writes cl_khr_byte_addressable_store cl_khr_gl_sharing cl_amd_device_attribute_query cl_amd_vec3 cl_amd_printf cl_amd_media_ops cl_amd_media_ops2 cl_amd_popcnt cl_khr_image2d_from_buffer cl_amd_bus_addressable_memory cl_khr_spir cl_khr_gl_event 

NULL platform behavior
  clGetPlatformInfo(NULL, CL_PLATFORM_NAME, ...)  No platform
  clGetDeviceIDs(NULL, CL_DEVICE_TYPE_ALL, ...)   No platform
  clCreateContext(NULL, ...) [default]            No platform
  clCreateContext(NULL, ...) [other]              Success [AMD]
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_DEFAULT)  Success (1)
    Platform Name                                 AMD Accelerated Parallel Processing
    Device Name                                   Hawaii
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_CPU)  No devices found in platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_GPU)  Success (1)
    Platform Name                                 AMD Accelerated Parallel Processing
    Device Name                                   Hawaii
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_ACCELERATOR)  No devices found in platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_CUSTOM)  No devices found in platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_ALL)  Success (1)
    Platform Name                                 AMD Accelerated Parallel Processing
    Device Name                                   Hawaii
FAHClient logs

Code: Select all

*********************** Log Started 2020-03-25T23:26:47Z ***********************
23:26:48:WU01:FS01:Connecting to 65.254.110.245:80
23:26:51:WARNING:WU01:FS01:Failed to get assignment from '65.254.110.245:80': Empty work server assignment
23:26:51:WU01:FS01:Connecting to 18.218.241.186:80
23:26:52:WARNING:WU01:FS01:Failed to get assignment from '18.218.241.186:80': Empty work server assignment
23:26:52:ERROR:WU01:FS01:Exception: Could not get an assignment
23:26:52:WU01:FS01:Connecting to 65.254.110.245:80
23:26:53:WARNING:WU01:FS01:Failed to get assignment from '65.254.110.245:80': Empty work server assignment
23:26:53:WU01:FS01:Connecting to 18.218.241.186:80
23:26:54:WARNING:WU01:FS01:Failed to get assignment from '18.218.241.186:80': Empty work server assignment
23:26:54:ERROR:WU01:FS01:Exception: Could not get an assignment
23:27:52:WU01:FS01:Connecting to 65.254.110.245:80
23:27:53:WARNING:WU01:FS01:Failed to get assignment from '65.254.110.245:80': Empty work server assignment
23:27:53:WU01:FS01:Connecting to 18.218.241.186:80
23:27:53:WU01:FS01:Assigned to work server 40.114.52.201
23:27:53:WU01:FS01:Requesting new work unit for slot 01: READY gpu:0:Hawaii [Radeon R9 200/300X Series] from 40.114.52.201
23:27:53:WU01:FS01:Connecting to 40.114.52.201:8080
23:30:04:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
23:30:04:WU01:FS01:Connecting to 40.114.52.201:80
23:30:47:WU01:FS01:Downloading 29.59MiB
23:30:53:WU01:FS01:Download 15.42%
23:30:59:WU01:FS01:Download 33.58%
23:31:05:WU01:FS01:Download 47.95%
23:31:11:WU01:FS01:Download 65.06%
23:31:17:WU01:FS01:Download 75.19%
23:31:23:WU01:FS01:Download 91.25%
23:31:25:WU01:FS01:Download complete
23:31:25:WU01:FS01:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:11778 run:0 clone:752 gen:16 core:0x22 unit:0x00000019287234c95e73c4260075626d
23:31:25:WU01:FS01:Starting
23:31:25:WU01:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /home/robo/cores/cores.foldingathome.org/v7/lin/64bit/Core_22.fah/FahCore_22 -dir 01 -suffix 01 -version 704 -lifeline 5026 -checkpoint 15 -gpu 0 -gpu-vendor ati
23:31:25:WU01:FS01:Started FahCore on PID 5159
23:31:25:WU01:FS01:Core PID:5163
23:31:25:WU01:FS01:FahCore 0x22 started
23:31:25:WU01:FS01:0x22:*********************** Log Started 2020-03-25T23:31:25Z ***********************
23:31:25:WU01:FS01:0x22:*************************** Core22 Folding@home Core ***************************
23:31:25:WU01:FS01:0x22:       Type: 0x22
23:31:25:WU01:FS01:0x22:       Core: Core22
23:31:25:WU01:FS01:0x22:    Website: https://foldingathome.org/
23:31:25:WU01:FS01:0x22:  Copyright: (c) 2009-2018 foldingathome.org
23:31:25:WU01:FS01:0x22:     Author: John Chodera <john.chodera@choderalab.org> and Rafal Wiewiora
23:31:25:WU01:FS01:0x22:             <rafal.wiewiora@choderalab.org>
23:31:25:WU01:FS01:0x22:       Args: -dir 01 -suffix 01 -version 704 -lifeline 5159 -checkpoint 15 -gpu
23:31:25:WU01:FS01:0x22:             0 -gpu-vendor ati
23:31:25:WU01:FS01:0x22:     Config: <none>
23:31:25:WU01:FS01:0x22:************************************ Build *************************************
23:31:25:WU01:FS01:0x22:    Version: 0.0.2
23:31:25:WU01:FS01:0x22:       Date: Dec 6 2019
23:31:25:WU01:FS01:0x22:       Time: 21:20:17
23:31:25:WU01:FS01:0x22: Repository: Git
23:31:25:WU01:FS01:0x22:   Revision: f87d92b58abdf7e6bf2e173cfbc4dc3e837c7042
23:31:25:WU01:FS01:0x22:     Branch: core22
23:31:25:WU01:FS01:0x22:   Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
23:31:25:WU01:FS01:0x22:    Options: -std=gnu++98 -O3 -funroll-loops
23:31:25:WU01:FS01:0x22:   Platform: linux2 4.9.87-linuxkit-aufs
23:31:25:WU01:FS01:0x22:       Bits: 64
23:31:25:WU01:FS01:0x22:       Mode: Release
23:31:25:WU01:FS01:0x22:************************************ System ************************************
23:31:25:WU01:FS01:0x22:        CPU: Intel(R) Core(TM) i7-4790K CPU @ 4.00GHz
23:31:25:WU01:FS01:0x22:     CPU ID: GenuineIntel Family 6 Model 60 Stepping 3
23:31:25:WU01:FS01:0x22:       CPUs: 8
23:31:25:WU01:FS01:0x22:     Memory: 3.79GiB
23:31:25:WU01:FS01:0x22:Free Memory: 731.52MiB
23:31:25:WU01:FS01:0x22:    Threads: POSIX_THREADS
23:31:25:WU01:FS01:0x22: OS Version: 5.3
23:31:25:WU01:FS01:0x22:Has Battery: false
23:31:25:WU01:FS01:0x22: On Battery: false
23:31:25:WU01:FS01:0x22: UTC Offset: 2
23:31:25:WU01:FS01:0x22:        PID: 5163
23:31:25:WU01:FS01:0x22:        CWD: /home/robo/work
23:31:25:WU01:FS01:0x22:         OS: Linux 5.3.0-42-generic x86_64
23:31:25:WU01:FS01:0x22:    OS Arch: AMD64
23:31:25:WU01:FS01:0x22:********************************************************************************
23:31:25:WU01:FS01:0x22:Project: 11778 (Run 0, Clone 752, Gen 16)
23:31:25:WU01:FS01:0x22:Unit: 0x00000019287234c95e73c4260075626d
23:31:25:WU01:FS01:0x22:Reading tar file core.xml
23:31:25:WU01:FS01:0x22:Reading tar file integrator.xml
23:31:25:WU01:FS01:0x22:Reading tar file state.xml
23:31:25:WU01:FS01:0x22:Reading tar file system.xml
23:31:25:WU01:FS01:0x22:Digital signatures verified
23:31:25:WU01:FS01:0x22:Folding@home GPU Core22 Folding@home Core
23:31:25:WU01:FS01:0x22:Version 0.0.2
23:31:35:WU01:FS01:0x22:Completed 0 out of 2000000 steps (0%)
23:31:35:WU01:FS01:0x22:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
23:31:42:WU01:FS01:0x22:Bad State detected... attempting to resume from last good checkpoint. Is your system overclocked?
23:31:42:WU01:FS01:0x22:Following exception occured: Particle coordinate is nan
23:31:46:WU01:FS01:0x22:Bad State detected... attempting to resume from last good checkpoint. Is your system overclocked?
23:31:46:WU01:FS01:0x22:Following exception occured: Particle coordinate is nan
23:31:49:WU01:FS01:0x22:Bad State detected... attempting to resume from last good checkpoint. Is your system overclocked?
23:31:49:WU01:FS01:0x22:Following exception occured: Particle coordinate is nan
23:31:49:WU01:FS01:0x22:ERROR:114: Max Retries Reached
23:31:49:WU01:FS01:0x22:Saving result file ../logfile_01.txt
23:31:49:WU01:FS01:0x22:Saving result file badstate-0.xml
23:31:49:WU01:FS01:0x22:Saving result file badstate-1.xml
23:31:49:WU01:FS01:0x22:Saving result file badstate-2.xml
23:31:49:WU01:FS01:0x22:Saving result file checkpt.crc
23:31:49:WU01:FS01:0x22:Saving result file science.log
23:31:49:WU01:FS01:0x22:Folding@home Core Shutdown: BAD_WORK_UNIT
23:31:49:WARNING:WU01:FS01:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
23:31:49:WU01:FS01:Sending unit results: id:01 state:SEND error:FAULTY project:11778 run:0 clone:752 gen:16 core:0x22 unit:0x00000019287234c95e73c4260075626d
23:31:50:WU01:FS01:Uploading 43.09MiB to 40.114.52.201
23:31:50:WU01:FS01:Connecting to 40.114.52.201:8080
23:31:50:WU02:FS01:Connecting to 65.254.110.245:80
23:31:50:WARNING:WU02:FS01:Failed to get assignment from '65.254.110.245:80': Empty work server assignment
23:31:50:WU02:FS01:Connecting to 18.218.241.186:80
23:31:51:WARNING:WU02:FS01:Failed to get assignment from '18.218.241.186:80': Empty work server assignment
23:31:51:ERROR:WU02:FS01:Exception: Could not get an assignment
23:31:51:WU02:FS01:Connecting to 65.254.110.245:80
23:31:52:WARNING:WU02:FS01:Failed to get assignment from '65.254.110.245:80': Empty work server assignment
23:31:52:WU02:FS01:Connecting to 18.218.241.186:80
23:31:52:WARNING:WU02:FS01:Failed to get assignment from '18.218.241.186:80': Empty work server assignment
23:31:52:ERROR:WU02:FS01:Exception: Could not get an assignment
23:32:51:WU02:FS01:Connecting to 65.254.110.245:80
23:32:52:WU02:FS01:Assigned to work server 13.90.152.57
23:32:52:WU02:FS01:Requesting new work unit for slot 01: READY gpu:0:Hawaii [Radeon R9 200/300X Series] from 13.90.152.57
23:32:52:WU02:FS01:Connecting to 13.90.152.57:8080
23:32:53:WU02:FS01:Downloading 51.19MiB
23:32:59:WU02:FS01:Download 7.33%
23:33:05:WU02:FS01:Download 16.12%
23:33:11:WU02:FS01:Download 23.44%
23:33:17:WU02:FS01:Download 29.06%
23:33:23:WU02:FS01:Download 34.06%
23:33:29:WU02:FS01:Download 38.70%
23:33:35:WU02:FS01:Download 45.90%
23:33:41:WU02:FS01:Download 56.65%
23:33:47:WU02:FS01:Download 65.32%
23:33:53:WU02:FS01:Download 73.98%
23:33:59:WU02:FS01:Download 80.58%
23:34:00:WARNING:WU01:FS01:WorkServer connection failed on port 8080 trying 80
23:34:00:WU01:FS01:Connecting to 40.114.52.201:80
23:34:01:WU01:FS01:Upload 0.15%
23:34:05:WU02:FS01:Download 88.63%
23:34:11:WU02:FS01:Download 98.64%
23:34:11:WU02:FS01:Download complete
23:34:12:WU02:FS01:Received Unit: id:02 state:DOWNLOAD error:NO_ERROR project:11780 run:0 clone:4114 gen:16 core:0x22 unit:0x0000001b0d5a98395e73c54efa148ab6
23:34:12:WU02:FS01:Starting
23:34:12:WU02:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /home/robo/cores/cores.foldingathome.org/v7/lin/64bit/Core_22.fah/FahCore_22 -dir 02 -suffix 01 -version 704 -lifeline 5026 -checkpoint 15 -gpu 0 -gpu-vendor ati
23:34:12:WU02:FS01:Started FahCore on PID 5209
23:34:12:WU02:FS01:Core PID:5213
23:34:12:WU02:FS01:FahCore 0x22 started
23:34:12:WU02:FS01:0x22:*********************** Log Started 2020-03-25T23:34:12Z ***********************
23:34:12:WU02:FS01:0x22:*************************** Core22 Folding@home Core ***************************
23:34:12:WU02:FS01:0x22:       Type: 0x22
23:34:12:WU02:FS01:0x22:       Core: Core22
23:34:12:WU02:FS01:0x22:    Website: https://foldingathome.org/
23:34:12:WU02:FS01:0x22:  Copyright: (c) 2009-2018 foldingathome.org
23:34:12:WU02:FS01:0x22:     Author: John Chodera <john.chodera@choderalab.org> and Rafal Wiewiora
23:34:12:WU02:FS01:0x22:             <rafal.wiewiora@choderalab.org>
23:34:12:WU02:FS01:0x22:       Args: -dir 02 -suffix 01 -version 704 -lifeline 5209 -checkpoint 15 -gpu
23:34:12:WU02:FS01:0x22:             0 -gpu-vendor ati
23:34:12:WU02:FS01:0x22:     Config: <none>
23:34:12:WU02:FS01:0x22:************************************ Build *************************************
23:34:12:WU02:FS01:0x22:    Version: 0.0.2
23:34:12:WU02:FS01:0x22:       Date: Dec 6 2019
23:34:12:WU02:FS01:0x22:       Time: 21:20:17
23:34:12:WU02:FS01:0x22: Repository: Git
23:34:12:WU02:FS01:0x22:   Revision: f87d92b58abdf7e6bf2e173cfbc4dc3e837c7042
23:34:12:WU02:FS01:0x22:     Branch: core22
23:34:12:WU02:FS01:0x22:   Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
23:34:12:WU02:FS01:0x22:    Options: -std=gnu++98 -O3 -funroll-loops
23:34:12:WU02:FS01:0x22:   Platform: linux2 4.9.87-linuxkit-aufs
23:34:12:WU02:FS01:0x22:       Bits: 64
23:34:12:WU02:FS01:0x22:       Mode: Release
23:34:12:WU02:FS01:0x22:************************************ System ************************************
23:34:12:WU02:FS01:0x22:        CPU: Intel(R) Core(TM) i7-4790K CPU @ 4.00GHz
23:34:12:WU02:FS01:0x22:     CPU ID: GenuineIntel Family 6 Model 60 Stepping 3
23:34:12:WU02:FS01:0x22:       CPUs: 8
23:34:12:WU02:FS01:0x22:     Memory: 3.79GiB
23:34:12:WU02:FS01:0x22:Free Memory: 582.84MiB
23:34:12:WU02:FS01:0x22:    Threads: POSIX_THREADS
23:34:12:WU02:FS01:0x22: OS Version: 5.3
23:34:12:WU02:FS01:0x22:Has Battery: false
23:34:12:WU02:FS01:0x22: On Battery: false
23:34:12:WU02:FS01:0x22: UTC Offset: 2
23:34:12:WU02:FS01:0x22:        PID: 5213
23:34:12:WU02:FS01:0x22:        CWD: /home/robo/work
23:34:12:WU02:FS01:0x22:         OS: Linux 5.3.0-42-generic x86_64
23:34:12:WU02:FS01:0x22:    OS Arch: AMD64
23:34:12:WU02:FS01:0x22:********************************************************************************
23:34:12:WU02:FS01:0x22:Project: 11780 (Run 0, Clone 4114, Gen 16)
23:34:12:WU02:FS01:0x22:Unit: 0x0000001b0d5a98395e73c54efa148ab6
23:34:12:WU02:FS01:0x22:Reading tar file core.xml
23:34:12:WU02:FS01:0x22:Reading tar file integrator.xml
23:34:12:WU02:FS01:0x22:Reading tar file state.xml
23:34:12:WU02:FS01:0x22:Reading tar file system.xml
23:34:12:WU02:FS01:0x22:Digital signatures verified
23:34:12:WU02:FS01:0x22:Folding@home GPU Core22 Folding@home Core
23:34:12:WU02:FS01:0x22:Version 0.0.2
23:34:14:WU01:FS01:Upload 0.29%
23:34:14:WARNING:WU01:FS01:Exception: Failed to send results to work server: Transfer failed
23:34:14:WU01:FS01:Trying to send results to collection server
23:34:14:WU01:FS01:Uploading 43.09MiB to 13.90.152.57
23:34:14:WU01:FS01:Connecting to 13.90.152.57:8080
23:34:20:WU01:FS01:Upload 6.96%
23:34:26:WU02:FS01:0x22:Completed 0 out of 1000000 steps (0%)
23:34:26:WU02:FS01:0x22:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
23:34:26:WU01:FS01:Upload 12.04%
23:34:32:WU01:FS01:Upload 18.28%
23:34:38:WU02:FS01:0x22:Bad State detected... attempting to resume from last good checkpoint. Is your system overclocked?
23:34:38:WU02:FS01:0x22:Following exception occured: Particle coordinate is nan
23:34:38:WU01:FS01:Upload 24.51%
23:34:43:WU02:FS01:0x22:Bad State detected... attempting to resume from last good checkpoint. Is your system overclocked?
23:34:43:WU02:FS01:0x22:Following exception occured: Particle coordinate is nan
23:34:44:WU01:FS01:Upload 30.90%
23:34:49:WU02:FS01:0x22:Bad State detected... attempting to resume from last good checkpoint. Is your system overclocked?
23:34:49:WU02:FS01:0x22:Following exception occured: Particle coordinate is nan
23:34:49:WU02:FS01:0x22:ERROR:114: Max Retries Reached
23:34:49:WU02:FS01:0x22:Saving result file ../logfile_01.txt
23:34:49:WU02:FS01:0x22:Saving result file badstate-0.xml
23:34:49:WU02:FS01:0x22:Saving result file badstate-1.xml
23:34:49:WU02:FS01:0x22:Saving result file badstate-2.xml
23:34:50:WU02:FS01:0x22:Saving result file checkpt.crc
23:34:50:WU02:FS01:0x22:Saving result file science.log
23:34:50:WU02:FS01:0x22:Folding@home Core Shutdown: BAD_WORK_UNIT
23:34:50:WU01:FS01:Upload 36.41%
23:34:50:WARNING:WU02:FS01:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
23:34:50:WU02:FS01:Sending unit results: id:02 state:SEND error:FAULTY project:11780 run:0 clone:4114 gen:16 core:0x22 unit:0x0000001b0d5a98395e73c54efa148ab6
23:34:50:WU02:FS01:Uploading 76.67MiB to 13.90.152.57
23:34:50:WU02:FS01:Connecting to 13.90.152.57:8080
23:34:50:WU03:FS01:Connecting to 65.254.110.245:80
23:34:51:WARNING:WU03:FS01:Failed to get assignment from '65.254.110.245:80': Empty work server assignment
23:34:51:WU03:FS01:Connecting to 18.218.241.186:80
23:34:51:WARNING:WU03:FS01:Failed to get assignment from '18.218.241.186:80': Empty work server assignment
23:34:51:ERROR:WU03:FS01:Exception: Could not get an assignment
23:34:51:WU03:FS01:Connecting to 65.254.110.245:80
23:34:52:WARNING:WU03:FS01:Failed to get assignment from '65.254.110.245:80': Empty work server assignment
23:34:52:WU03:FS01:Connecting to 18.218.241.186:80
23:34:52:WARNING:WU03:FS01:Failed to get assignment from '18.218.241.186:80': Empty work server assignment
23:34:52:ERROR:WU03:FS01:Exception: Could not get an assignment
23:34:56:WU02:FS01:Upload 3.10%
23:34:56:WU01:FS01:Upload 41.49%
23:35:03:WU01:FS01:Upload 44.53%
23:35:07:WU02:FS01:Upload 5.22%
23:35:10:WU01:FS01:Upload 45.98%
23:35:13:WU02:FS01:Upload 7.58%
23:35:16:WU01:FS01:Upload 52.22%
23:35:19:WU02:FS01:Upload 11.09%
23:35:22:WU01:FS01:Upload 58.46%
23:35:25:WU02:FS01:Upload 14.59%
23:35:28:WU01:FS01:Upload 64.69%
23:35:31:WU02:FS01:Upload 17.36%
23:35:34:WU01:FS01:Upload 69.77%
23:35:37:WU02:FS01:Upload 20.46%
23:35:40:WU01:FS01:Upload 76.15%
23:35:43:WU02:FS01:Upload 23.97%
23:35:46:WU01:FS01:Upload 81.81%
23:35:49:WU02:FS01:Upload 27.47%
23:35:52:WU03:FS01:Connecting to 65.254.110.245:80
23:35:52:WU01:FS01:Upload 86.74%
23:35:52:WARNING:WU03:FS01:Failed to get assignment from '65.254.110.245:80': Empty work server assignment
23:35:52:WU03:FS01:Connecting to 18.218.241.186:80
23:35:52:WARNING:WU03:FS01:Failed to get assignment from '18.218.241.186:80': Empty work server assignment
23:35:52:ERROR:WU03:FS01:Exception: Could not get an assignment
23:35:55:WU02:FS01:Upload 29.83%
23:35:58:WU01:FS01:Upload 93.13%
23:36:01:WU02:FS01:Upload 33.34%
23:36:04:WU01:FS01:Upload 99.36%
23:36:07:WU01:FS01:Upload complete
23:36:07:WU01:FS01:Server responded WORK_ACK (400)
23:36:07:WU01:FS01:Cleaning up
23:36:10:WU02:FS01:Upload 36.84%
23:36:16:WU02:FS01:Upload 40.35%
23:36:22:WU02:FS01:Upload 44.26%
23:36:28:WU02:FS01:Upload 47.44%
23:36:34:WU02:FS01:Upload 50.95%
23:36:40:WU02:FS01:Upload 54.04%
23:36:46:WU02:FS01:Upload 57.14%
23:36:52:WU02:FS01:Upload 60.32%
23:36:58:WU02:FS01:Upload 63.83%
23:37:04:WU02:FS01:Upload 67.00%
23:37:10:WU02:FS01:Upload 70.10%
23:37:16:WU02:FS01:Upload 73.61%
23:37:22:WU02:FS01:Upload 77.11%
23:37:28:WU02:FS01:Upload 80.70%
23:37:29:WU03:FS01:Connecting to 65.254.110.245:80
23:37:29:WARNING:WU03:FS01:Failed to get assignment from '65.254.110.245:80': Empty work server assignment
23:37:29:WU03:FS01:Connecting to 18.218.241.186:80
23:37:30:WARNING:WU03:FS01:Failed to get assignment from '18.218.241.186:80': Empty work server assignment
23:37:30:ERROR:WU03:FS01:Exception: Could not get an assignment
23:37:34:WU02:FS01:Upload 84.20%
23:37:40:WU02:FS01:Upload 87.71%
23:37:46:WU02:FS01:Upload 91.21%
23:37:52:WU02:FS01:Upload 94.72%
23:37:58:WU02:FS01:Upload 98.22%
23:38:03:WU02:FS01:Upload complete
23:38:03:WU02:FS01:Server responded WORK_ACK (400)
23:38:03:WU02:FS01:Cleaning up
23:40:06:WU03:FS01:Connecting to 65.254.110.245:80
23:40:07:WARNING:WU03:FS01:Failed to get assignment from '65.254.110.245:80': Empty work server assignment
23:40:07:WU03:FS01:Connecting to 18.218.241.186:80
23:40:07:WU03:FS01:Assigned to work server 128.252.203.10
23:40:07:WU03:FS01:Requesting new work unit for slot 01: READY gpu:0:Hawaii [Radeon R9 200/300X Series] from 128.252.203.10
23:40:07:WU03:FS01:Connecting to 128.252.203.10:8080
23:40:55:WU03:FS01:Downloading 50.73MiB
23:41:01:WU03:FS01:Download 6.41%
23:41:07:WU03:FS01:Download 12.07%
23:41:13:WU03:FS01:Download 18.85%
23:41:19:WU03:FS01:Download 25.13%
23:41:25:WU03:FS01:Download 32.77%
23:41:31:WU03:FS01:Download 40.28%
23:41:37:WU03:FS01:Download 46.57%
23:41:43:WU03:FS01:Download 52.73%
23:41:49:WU03:FS01:Download 61.23%
23:41:55:WU03:FS01:Download 68.49%
23:42:01:WU03:FS01:Download 75.27%
23:42:07:WU03:FS01:Download 83.03%
23:42:13:WU03:FS01:Download 90.55%
23:42:19:WU03:FS01:Download 99.29%
23:42:19:WU03:FS01:Download complete
23:42:19:WU03:FS01:Received Unit: id:03 state:DOWNLOAD error:NO_ERROR project:11762 run:0 clone:1577 gen:17 core:0x22 unit:0x0000001b80fccb0a5e6d80db33ad3040
23:42:19:WU03:FS01:Starting
23:42:19:WU03:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /home/robo/cores/cores.foldingathome.org/v7/lin/64bit/Core_22.fah/FahCore_22 -dir 03 -suffix 01 -version 704 -lifeline 5026 -checkpoint 15 -gpu 0 -gpu-vendor ati
23:42:19:WU03:FS01:Started FahCore on PID 5968
23:42:19:WU03:FS01:Core PID:5972
23:42:19:WU03:FS01:FahCore 0x22 started
23:42:20:WU03:FS01:0x22:*********************** Log Started 2020-03-25T23:42:19Z ***********************
23:42:20:WU03:FS01:0x22:*************************** Core22 Folding@home Core ***************************
23:42:20:WU03:FS01:0x22:       Type: 0x22
23:42:20:WU03:FS01:0x22:       Core: Core22
23:42:20:WU03:FS01:0x22:    Website: https://foldingathome.org/
23:42:20:WU03:FS01:0x22:  Copyright: (c) 2009-2018 foldingathome.org
23:42:20:WU03:FS01:0x22:     Author: John Chodera <john.chodera@choderalab.org> and Rafal Wiewiora
23:42:20:WU03:FS01:0x22:             <rafal.wiewiora@choderalab.org>
23:42:20:WU03:FS01:0x22:       Args: -dir 03 -suffix 01 -version 704 -lifeline 5968 -checkpoint 15 -gpu
23:42:20:WU03:FS01:0x22:             0 -gpu-vendor ati
23:42:20:WU03:FS01:0x22:     Config: <none>
23:42:20:WU03:FS01:0x22:************************************ Build *************************************
23:42:20:WU03:FS01:0x22:    Version: 0.0.2
23:42:20:WU03:FS01:0x22:       Date: Dec 6 2019
23:42:20:WU03:FS01:0x22:       Time: 21:20:17
23:42:20:WU03:FS01:0x22: Repository: Git
23:42:20:WU03:FS01:0x22:   Revision: f87d92b58abdf7e6bf2e173cfbc4dc3e837c7042
23:42:20:WU03:FS01:0x22:     Branch: core22
23:42:20:WU03:FS01:0x22:   Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
23:42:20:WU03:FS01:0x22:    Options: -std=gnu++98 -O3 -funroll-loops
23:42:20:WU03:FS01:0x22:   Platform: linux2 4.9.87-linuxkit-aufs
23:42:20:WU03:FS01:0x22:       Bits: 64
23:42:20:WU03:FS01:0x22:       Mode: Release
23:42:20:WU03:FS01:0x22:************************************ System ************************************
23:42:20:WU03:FS01:0x22:        CPU: Intel(R) Core(TM) i7-4790K CPU @ 4.00GHz
23:42:20:WU03:FS01:0x22:     CPU ID: GenuineIntel Family 6 Model 60 Stepping 3
23:42:20:WU03:FS01:0x22:       CPUs: 8
23:42:20:WU03:FS01:0x22:     Memory: 3.79GiB
23:42:20:WU03:FS01:0x22:Free Memory: 993.79MiB
23:42:20:WU03:FS01:0x22:    Threads: POSIX_THREADS
23:42:20:WU03:FS01:0x22: OS Version: 5.3
23:42:20:WU03:FS01:0x22:Has Battery: false
23:42:20:WU03:FS01:0x22: On Battery: false
23:42:20:WU03:FS01:0x22: UTC Offset: 2
23:42:20:WU03:FS01:0x22:        PID: 5972
23:42:20:WU03:FS01:0x22:        CWD: /home/robo/work
23:42:20:WU03:FS01:0x22:         OS: Linux 5.3.0-42-generic x86_64
23:42:20:WU03:FS01:0x22:    OS Arch: AMD64
23:42:20:WU03:FS01:0x22:********************************************************************************
23:42:20:WU03:FS01:0x22:Project: 11762 (Run 0, Clone 1577, Gen 17)
23:42:20:WU03:FS01:0x22:Unit: 0x0000001b80fccb0a5e6d80db33ad3040
23:42:20:WU03:FS01:0x22:Reading tar file core.xml
23:42:20:WU03:FS01:0x22:Reading tar file integrator.xml
23:42:20:WU03:FS01:0x22:Reading tar file state.xml
23:42:20:WU03:FS01:0x22:Reading tar file system.xml
23:42:20:WU03:FS01:0x22:Digital signatures verified
23:42:20:WU03:FS01:0x22:Folding@home GPU Core22 Folding@home Core
23:42:20:WU03:FS01:0x22:Version 0.0.2
23:42:34:WU03:FS01:0x22:Completed 0 out of 1000000 steps (0%)
23:42:34:WU03:FS01:0x22:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
23:43:15:WU03:FS01:0x22:Bad State detected... attempting to resume from last good checkpoint. Is your system overclocked?
23:43:15:WU03:FS01:0x22:Following exception occured: Particle coordinate is nan
23:43:21:WU03:FS01:0x22:Bad State detected... attempting to resume from last good checkpoint. Is your system overclocked?
23:43:21:WU03:FS01:0x22:Following exception occured: Particle coordinate is nan
23:43:26:WU03:FS01:0x22:Bad State detected... attempting to resume from last good checkpoint. Is your system overclocked?
23:43:26:WU03:FS01:0x22:Following exception occured: Particle coordinate is nan
23:43:26:WU03:FS01:0x22:ERROR:114: Max Retries Reached
23:43:26:WU03:FS01:0x22:Saving result file ../logfile_01.txt
23:43:26:WU03:FS01:0x22:Saving result file badstate-0.xml
23:43:26:WU03:FS01:0x22:Saving result file badstate-1.xml
23:43:26:WU03:FS01:0x22:Saving result file badstate-2.xml
23:43:27:WU03:FS01:0x22:Saving result file checkpt.crc
23:43:27:WU03:FS01:0x22:Saving result file science.log
23:43:27:WU03:FS01:0x22:Folding@home Core Shutdown: BAD_WORK_UNIT
23:43:27:WARNING:WU03:FS01:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
23:43:27:WU03:FS01:Sending unit results: id:03 state:SEND error:FAULTY project:11762 run:0 clone:1577 gen:17 core:0x22 unit:0x0000001b80fccb0a5e6d80db33ad3040
23:43:28:WU03:FS01:Uploading 75.87MiB to 128.252.203.10
23:43:28:WU03:FS01:Connecting to 128.252.203.10:8080
23:43:28:WU01:FS01:Connecting to 65.254.110.245:80
23:43:28:WARNING:WU01:FS01:Failed to get assignment from '65.254.110.245:80': Empty work server assignment
23:43:28:WU01:FS01:Connecting to 18.218.241.186:80
23:43:29:WARNING:WU01:FS01:Failed to get assignment from '18.218.241.186:80': Empty work server assignment
23:43:29:ERROR:WU01:FS01:Exception: Could not get an assignment
23:43:29:WU01:FS01:Connecting to 65.254.110.245:80
23:43:29:WARNING:WU01:FS01:Failed to get assignment from '65.254.110.245:80': Empty work server assignment
23:43:29:WU01:FS01:Connecting to 18.218.241.186:80
23:43:30:WU01:FS01:Assigned to work server 140.163.4.231
23:43:30:WU01:FS01:Requesting new work unit for slot 01: READY gpu:0:Hawaii [Radeon R9 200/300X Series] from 140.163.4.231
23:43:30:WU01:FS01:Connecting to 140.163.4.231:8080
23:43:59:WU03:FS01:Upload 1.07%
23:44:05:WU03:FS01:Upload 29.66%
23:44:11:WU03:FS01:Upload 68.54%
23:44:17:WU03:FS01:Upload 100.00%
23:44:18:WU03:FS01:Upload complete
23:44:18:WU03:FS01:Server responded WORK_ACK (400)
23:44:18:WU03:FS01:Cleaning up
23:47:29:WU01:FS01:Downloading 13.15MiB
23:47:42:WU01:FS01:Download 1.90%
23:47:48:WU01:FS01:Download 4.75%
23:47:57:WU01:FS01:Download 9.98%
23:48:03:WU01:FS01:Download 12.83%
23:48:09:WU01:FS01:Download 34.22%
23:48:20:WU01:FS01:Download 41.34%
23:48:26:WU01:FS01:Download 49.90%
23:48:32:WU01:FS01:Download 53.22%
23:48:38:WU01:FS01:Download 59.88%
23:48:52:WU01:FS01:Download 68.43%
23:48:58:WU01:FS01:Download 73.66%
23:49:04:WU01:FS01:Download 80.79%
23:49:10:WU01:FS01:Download 84.11%
23:49:16:WU01:FS01:Download 90.77%
23:49:22:WU01:FS01:Download 98.37%
23:49:22:WU01:FS01:Download complete
23:49:22:WU01:FS01:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:11752 run:0 clone:2444 gen:5 core:0x22 unit:0x0000000e8ca304e75e6a8068ac098048
23:49:22:WU01:FS01:Starting
23:49:22:WU01:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /home/robo/cores/cores.foldingathome.org/v7/lin/64bit/Core_22.fah/FahCore_22 -dir 01 -suffix 01 -version 704 -lifeline 5026 -checkpoint 15 -gpu 0 -gpu-vendor ati
23:49:22:WU01:FS01:Started FahCore on PID 6514
23:49:22:WU01:FS01:Core PID:6518
23:49:22:WU01:FS01:FahCore 0x22 started
23:49:23:WU01:FS01:0x22:*********************** Log Started 2020-03-25T23:49:22Z ***********************
23:49:23:WU01:FS01:0x22:*************************** Core22 Folding@home Core ***************************
23:49:23:WU01:FS01:0x22:       Type: 0x22
23:49:23:WU01:FS01:0x22:       Core: Core22
23:49:23:WU01:FS01:0x22:    Website: https://foldingathome.org/
23:49:23:WU01:FS01:0x22:  Copyright: (c) 2009-2018 foldingathome.org
23:49:23:WU01:FS01:0x22:     Author: John Chodera <john.chodera@choderalab.org> and Rafal Wiewiora
23:49:23:WU01:FS01:0x22:             <rafal.wiewiora@choderalab.org>
23:49:23:WU01:FS01:0x22:       Args: -dir 01 -suffix 01 -version 704 -lifeline 6514 -checkpoint 15 -gpu
23:49:23:WU01:FS01:0x22:             0 -gpu-vendor ati
23:49:23:WU01:FS01:0x22:     Config: <none>
23:49:23:WU01:FS01:0x22:************************************ Build *************************************
23:49:23:WU01:FS01:0x22:    Version: 0.0.2
23:49:23:WU01:FS01:0x22:       Date: Dec 6 2019
23:49:23:WU01:FS01:0x22:       Time: 21:20:17
23:49:23:WU01:FS01:0x22: Repository: Git
23:49:23:WU01:FS01:0x22:   Revision: f87d92b58abdf7e6bf2e173cfbc4dc3e837c7042
23:49:23:WU01:FS01:0x22:     Branch: core22
23:49:23:WU01:FS01:0x22:   Compiler: GNU 4.8.2 20140120 (Red Hat 4.8.2-15)
23:49:23:WU01:FS01:0x22:    Options: -std=gnu++98 -O3 -funroll-loops
23:49:23:WU01:FS01:0x22:   Platform: linux2 4.9.87-linuxkit-aufs
23:49:23:WU01:FS01:0x22:       Bits: 64
23:49:23:WU01:FS01:0x22:       Mode: Release
23:49:23:WU01:FS01:0x22:************************************ System ************************************
23:49:23:WU01:FS01:0x22:        CPU: Intel(R) Core(TM) i7-4790K CPU @ 4.00GHz
23:49:23:WU01:FS01:0x22:     CPU ID: GenuineIntel Family 6 Model 60 Stepping 3
23:49:23:WU01:FS01:0x22:       CPUs: 8
23:49:23:WU01:FS01:0x22:     Memory: 3.79GiB
23:49:23:WU01:FS01:0x22:Free Memory: 912.17MiB
23:49:23:WU01:FS01:0x22:    Threads: POSIX_THREADS
23:49:23:WU01:FS01:0x22: OS Version: 5.3
23:49:23:WU01:FS01:0x22:Has Battery: false
23:49:23:WU01:FS01:0x22: On Battery: false
23:49:23:WU01:FS01:0x22: UTC Offset: 2
23:49:23:WU01:FS01:0x22:        PID: 6518
23:49:23:WU01:FS01:0x22:        CWD: /home/robo/work
23:49:23:WU01:FS01:0x22:         OS: Linux 5.3.0-42-generic x86_64
23:49:23:WU01:FS01:0x22:    OS Arch: AMD64
23:49:23:WU01:FS01:0x22:********************************************************************************
23:49:23:WU01:FS01:0x22:Project: 11752 (Run 0, Clone 2444, Gen 5)
23:49:23:WU01:FS01:0x22:Unit: 0x0000000e8ca304e75e6a8068ac098048
23:49:23:WU01:FS01:0x22:Reading tar file core.xml
23:49:23:WU01:FS01:0x22:Reading tar file integrator.xml
23:49:23:WU01:FS01:0x22:Reading tar file state.xml
23:49:26:WU01:FS01:0x22:Reading tar file system.xml
23:49:27:WU01:FS01:0x22:Digital signatures verified
23:49:27:WU01:FS01:0x22:Folding@home GPU Core22 Folding@home Core
23:49:27:WU01:FS01:0x22:Version 0.0.2
23:49:42:WU01:FS01:0x22:ERROR:exception: Error invoking kernel sortShortList: clEnqueueNDRangeKernel (-5)
23:49:42:WU01:FS01:0x22:Saving result file ../logfile_01.txt
23:49:42:WU01:FS01:0x22:Saving result file science.log
23:49:42:WU01:FS01:0x22:Folding@home Core Shutdown: BAD_WORK_UNIT
23:49:42:WARNING:WU01:FS01:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
23:49:42:WU01:FS01:Sending unit results: id:01 state:SEND error:FAULTY project:11752 run:0 clone:2444 gen:5 core:0x22 unit:0x0000000e8ca304e75e6a8068ac098048
23:49:42:WU01:FS01:Uploading 2.59KiB to 140.163.4.231
23:49:42:WU01:FS01:Connecting to 140.163.4.231:8080
23:49:43:WU02:FS01:Connecting to 65.254.110.245:80
23:49:43:WARNING:WU02:FS01:Failed to get assignment from '65.254.110.245:80': Empty work server assignment
23:49:43:WU02:FS01:Connecting to 18.218.241.186:80
23:49:44:WARNING:WU02:FS01:Failed to get assignment from '18.218.241.186:80': Empty work server assignment
23:49:44:ERROR:WU02:FS01:Exception: Could not get an assignment
23:49:44:WU02:FS01:Connecting to 65.254.110.245:80
23:49:44:WARNING:WU02:FS01:Failed to get assignment from '65.254.110.245:80': Empty work server assignment
23:49:44:WU02:FS01:Connecting to 18.218.241.186:80
23:49:45:WARNING:WU02:FS01:Failed to get assignment from '18.218.241.186:80': Empty work server assignment
23:49:45:ERROR:WU02:FS01:Exception: Could not get an assignment
23:50:44:WU02:FS01:Connecting to 65.254.110.245:80
23:50:44:WARNING:WU02:FS01:Failed to get assignment from '65.254.110.245:80': Empty work server assignment
23:50:44:WU02:FS01:Connecting to 18.218.241.186:80
23:50:45:WARNING:WU02:FS01:Failed to get assignment from '18.218.241.186:80': Empty work server assignment
23:50:45:ERROR:WU02:FS01:Exception: Could not get an assignment
23:51:05:WU01:FS01:Upload complete
23:51:05:WU01:FS01:Server responded WORK_ACK (400)
23:51:05:WU01:FS01:Cleaning up
23:52:21:WU02:FS01:Connecting to 65.254.110.245:80
23:52:22:WARNING:WU02:FS01:Failed to get assignment from '65.254.110.245:80': Empty work server assignment
23:52:22:WU02:FS01:Connecting to 18.218.241.186:80
23:52:22:WARNING:WU02:FS01:Failed to get assignment from '18.218.241.186:80': Empty work server assignment
23:52:22:ERROR:WU02:FS01:Exception: Could not get an assignment
23:54:58:WU02:FS01:Connecting to 65.254.110.245:80
23:54:59:WARNING:WU02:FS01:Failed to get assignment from '65.254.110.245:80': Empty work server assignment
23:54:59:WU02:FS01:Connecting to 18.218.241.186:80
23:54:59:WARNING:WU02:FS01:Failed to get assignment from '18.218.241.186:80': Empty work server assignment
23:54:59:ERROR:WU02:FS01:Exception: Could not get an assignment
23:59:13:WU02:FS01:Connecting to 65.254.110.245:80
23:59:13:WU02:FS01:Assigned to work server 40.114.52.201
23:59:13:WU02:FS01:Requesting new work unit for slot 01: READY gpu:0:Hawaii [Radeon R9 200/300X Series] from 40.114.52.201
23:59:13:WU02:FS01:Connecting to 40.114.52.201:8080
00:01:24:WARNING:WU02:FS01:WorkServer connection failed on port 8080 trying 80
00:01:24:WU02:FS01:Connecting to 40.114.52.201:80
00:01:51:ERROR:WU02:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
00:06:04:WU02:FS01:Connecting to 65.254.110.245:80
00:06:05:WARNING:WU02:FS01:Failed to get assignment from '65.254.110.245:80': Empty work server assignment
00:06:05:WU02:FS01:Connecting to 18.218.241.186:80
00:06:05:WARNING:WU02:FS01:Failed to get assignment from '18.218.241.186:80': Empty work server assignment
00:06:05:ERROR:WU02:FS01:Exception: Could not get an assignment
00:17:10:WU02:FS01:Connecting to 65.254.110.245:80
00:17:10:WARNING:WU02:FS01:Failed to get assignment from '65.254.110.245:80': Empty work server assignment
00:17:10:WU02:FS01:Connecting to 18.218.241.186:80
00:17:11:WARNING:WU02:FS01:Failed to get assignment from '18.218.241.186:80': Empty work server assignment
00:17:11:ERROR:WU02:FS01:Exception: Could not get an assignment
00:35:06:WU02:FS01:Connecting to 65.254.110.245:80
00:35:07:WARNING:WU02:FS01:Failed to get assignment from '65.254.110.245:80': Empty work server assignment
00:35:07:WU02:FS01:Connecting to 18.218.241.186:80
00:35:07:WU02:FS01:Assigned to work server 128.252.203.10
00:35:07:WU02:FS01:Requesting new work unit for slot 01: READY gpu:0:Hawaii [Radeon R9 200/300X Series] from 128.252.203.10
00:35:07:WU02:FS01:Connecting to 128.252.203.10:8080
00:36:09:ERROR:WU02:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0

Re: Issues with Radeon R9 200/300 series

Posted: Thu Mar 26, 2020 7:49 pm
by Frogging101
The "Could not get an assignment" errors are just due to greatly increased demand on the servers from the number of new donors recently. This is a known issue, and it is discussed at length all over the forums right now. They are working on it; just leave your client running as normal.

The "Error invoking kernel sortShortList: clEnqueueNDRangeKernel (-5)" error is a known issue on AMD devices. It happens on both Windows and Linux, and it only affects some work units. There's a thread about it here: viewtopic.php?f=74&t=32991&p=316173&hilit=clEnqueueNDRangeKernel

Now that we have the known issues out of the way, we're left with this:

Code: Select all

23:31:42:WU01:FS01:0x22:Bad State detected... attempting to resume from last good checkpoint. Is your system overclocked?
23:31:42:WU01:FS01:0x22:Following exception occured: Particle coordinate is nan
This one is something else. Could you answer the following:
  • Like the message says, is your system overclocked? Have you adjusted the voltages or clock speeds of your CPU, GPU, or anything else?
  • What OpenCL implementation are you using? Is it AMDGPU-PRO (official proprietary driver from AMD's site), or is it ROCm (official open source OpenCL implementation)? If you don't know what ROCm is, then you are probably not using it. Your clinfo output looks pretty similar to my mine on my R9 290 with AMDGPU-PRO, but I ask anyway because both platforms call themselves "AMD Accelerated Parallel Processing" in clinfo.

Re: Issues with Radeon R9 200/300 series

Posted: Thu Mar 26, 2020 9:02 pm
by MrFrizzy
Frogging101 wrote:The "Error invoking kernel sortShortList: clEnqueueNDRangeKernel (-5)" error is a known issue on AMD devices. It happens on both Windows and Linux, and it only affects some work units. There's a thread about it here: viewtopic.php?f=74&t=32991&p=316173&hilit=clEnqueueNDRangeKernel
Frogging, just so you are informed, this message has not been reported on Navi based cards (RX 5500 to 5700 XT) to my knowledge. Every single project that I have seen this error on so far has been on GCN based cards all the way from the R9 290 to the Vega 64 cards; my 5700 XT, on the other hand, has successfully completed them all.

Smbrelay, while I am no linux guru, I believe that only the official drivers are supported. So if you are using those, the next step would be to underclock your core or memory, pull 10-15% power limit, increase the fan speeds, or if you have the thermal headroom, increase the core voltage. Any time I have personally seen that message about a "bad state being detected" and the "particle coordinate is nan", it has been due to the card not being stable. I have even seen this happen with stock, out of the box settings so I would try tweaking things even if you are running stock settings.

Re: Issues with Radeon R9 200/300 series

Posted: Sat Mar 28, 2020 7:26 pm
by smbrelay
Frogging101 wrote: [*] Like the message says, is your system overclocked? Have you adjusted the voltages or clock speeds of your CPU, GPU, or anything else?
[*] What OpenCL implementation are you using? Is it AMDGPU-PRO (official proprietary driver from AMD's site), or is it ROCm (official open source OpenCL implementation)? If you don't know what ROCm is, then you are probably not using it. Your clinfo output looks pretty similar to my mine on my R9 290 with AMDGPU-PRO, but I ask anyway because both platforms call themselves "AMD Accelerated Parallel Processing" in clinfo.
1. no at all, using stock default settings;
2. official proprietary driver from AMD's site (AMDGPU-PRO). ROCm required some additional setup steps which I didn't complete. How do you think if it should increase chances to success?

Re: Issues with Radeon R9 200/300 series

Posted: Sat Mar 28, 2020 7:35 pm
by Frogging101
ROCm didn't work for me for Folding@Home. It worked for LuxMark and other loads, but with F@H it caused the system to hang, at least on kernel 5.4. The AMDGPU-PRO OpenCL implementation is what I use for Folding@Home on my R9 290 (viewtopic.php?f=81&t=33353 ).

Is your AMDGPU-PRO driver the latest version? Currently that's 19.50, which is what I'm using. It doesn't sound very likely that an older version would cause this, but it's worth a try I guess.

Aside from that, I don't have any other ideas. Perhaps someone else has an idea. MrFrizzy said something about underclocking earlier. Maybe that's something to look at.

Re: Issues with Radeon R9 200/300 series

Posted: Sat Mar 28, 2020 7:49 pm
by smbrelay
Frogging101 wrote:ROCm didn't work for me for Folding@Home. It worked for LuxMark and other loads, but with F@H it caused the system to hang, at least on kernel 5.4. The AMDGPU-PRO OpenCL implementation is what I use for Folding@Home on my R9 290 (viewtopic.php?f=81&t=33353 ).

Is your AMDGPU-PRO driver the latest version? Currently that's 19.50, which is what I'm using. It doesn't sound very likely that an older version would cause this, but it's worth a try I guess.

Aside from that, I don't have any other ideas. Perhaps someone else has an idea. MrFrizzy said something about underclocking earlier. Maybe that's something to look at.
Yes, 19.50. (amdgpu-pro-19.50-967956-ubuntu-18.04.tar.xz). Will try to adjust hardware settings.. hope this helps. Let you know if any results.
All times are UTC
Page 1 of 1

Powered by phpBB® Forum Software © phpBB Limited