Page 1 of 1

Core22 v0.0.13 return CUDA_ERROR_INVALID_PTX (218)

Posted: Fri Sep 25, 2020 8:29 am
by vmzy
when Core22 v0.0.13 run Project: 14484 return Error loading CUDA module: CUDA_ERROR_INVALID_PTX (218)

Code: Select all

*********************** Log Started 2020-09-20T01:35:39Z ***********************
01:35:39:Trying to access database...
01:35:39:Successfully acquired database lock
01:35:42:Downloading GPUs.txt from assign1.foldingathome.org:80
01:35:42:Connecting to assign1.foldingathome.org:80
01:35:46:Read GPUs.txt
01:35:47:Enabled folding slot 01: READY gpu:0:GP108 [GeForce MX150 (GT 1030) Max-Q] 1127
01:35:47:****************************** FAHClient ******************************
01:35:47:        Version: 7.6.13
01:35:47:         Author: Joseph Coffland <joseph@cauldrondevelopment.com>
01:35:47:      Copyright: 2020 foldingathome.org
01:35:47:       Homepage: https://foldingathome.org/
01:35:47:           Date: Apr 27 2020
01:35:47:           Time: 21:21:01
01:35:47:       Revision: 5a652817f46116b6e135503af97f18e094414e3b
01:35:47:         Branch: master
01:35:47:       Compiler: Visual C++ 2008
01:35:47:        Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
01:35:47:       Platform: win32 10
01:35:47:           Bits: 32
01:35:47:           Mode: Release
01:35:47:         Config: D:\Program Files (x86)\FAHData\config.xml
01:35:47:******************************** CBang ********************************
01:35:47:           Date: Apr 24 2020
01:35:47:           Time: 17:07:55
01:35:47:       Revision: ea081a3b3b0f4a37c4d0440b4f1bc184197c7797
01:35:47:         Branch: master
01:35:47:       Compiler: Visual C++ 2008
01:35:47:        Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
01:35:47:       Platform: win32 10
01:35:47:           Bits: 32
01:35:47:           Mode: Release
01:35:47:******************************* System ********************************
01:35:47:            CPU: Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz
01:35:47:         CPU ID: GenuineIntel Family 6 Model 142 Stepping 9
01:35:47:           CPUs: 4
01:35:47:         Memory: 7.91GiB
01:35:47:    Free Memory: 2.92GiB
01:35:47:        Threads: WINDOWS_THREADS
01:35:47:     OS Version: 6.2
01:35:47:    Has Battery: true
01:35:47:     On Battery: false
01:35:47:     UTC Offset: 8
01:35:47:            PID: 91232
01:35:47:            CWD: D:\Program Files (x86)\FAHData
01:35:47:  Win32 Service: false
01:35:47:             OS: Windows 10 Home China
01:35:47:        OS Arch: AMD64
01:35:47:           GPUs: 1
01:35:47:          GPU 0: Bus:1 Slot:0 Func:0 NVIDIA:5 GP108 [GeForce MX150 (GT 1030)
01:35:47:                 Max-Q] 1127
01:35:47:  CUDA Device 0: Platform:0 Device:0 Bus:1 Slot:0 Compute:6.1 Driver:9.1
01:35:47:OpenCL Device 0: Platform:0 Device:0 Bus:NA Slot:NA Compute:2.1 Driver:24.20
01:35:47:OpenCL Device 2: Platform:1 Device:0 Bus:1 Slot:0 Compute:1.2 Driver:391.24
01:35:47:******************************* libFAH ********************************
01:35:47:           Date: Apr 15 2020
01:35:47:           Time: 14:53:14
01:35:47:       Revision: 216968bc7025029c841ed6e36e81a03a316890d3
01:35:47:         Branch: master
01:35:47:       Compiler: Visual C++ 2008
01:35:47:        Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
01:35:47:       Platform: win32 10
01:35:47:           Bits: 32
01:35:47:           Mode: Release
01:35:47:***********************************************************************
01:35:47:<config>
01:35:47:  <!-- Client Control -->
01:35:47:  <disable-sleep-when-active v='false'/>
01:35:47:
01:35:47:  <!-- Folding Core -->
01:35:47:  <checkpoint v='30'/>
01:35:47:
01:35:47:  <!-- Network -->
01:35:47:  <proxy v='210.101.131.231:8080'/>
01:35:47:
01:35:47:  <!-- Slot Control -->
01:35:47:  <power v='full'/>
01:35:47:
01:35:47:  <!-- User Information -->
01:35:47:  <passkey v='*****'/>
01:35:47:  <team v='3213'/>
01:35:47:  <user v='vmzy'/>
01:35:47:
01:35:47:  <!-- Folding Slots -->
01:35:47:  <slot id='1' type='GPU'/>
01:35:47:</config>
...
05:03:24:WU00:FS01:Connecting to assign1.foldingathome.org:80
05:03:25:WU00:FS01:Assigned to work server 128.252.203.9
05:03:25:WU00:FS01:Requesting new work unit for slot 01: RUNNING gpu:0:GP108 [GeForce MX150 (GT 1030) Max-Q] 1127 from 128.252.203.9
05:03:25:WU00:FS01:Connecting to 128.252.203.9:8080
05:03:26:WU00:FS01:Downloading 20.60MiB
...
05:30:35:WU00:FS01:Download complete
05:30:35:WU00:FS01:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:14484 run:0 clone:920 gen:74 core:0x22 unit:0x0000006f80fccb095f171a3d48be36d6
05:30:35:WU00:FS01:Downloading core from http://cores.foldingathome.org/win/64bit/22-0.0.13/Core_22.fah
05:30:35:WU00:FS01:Connecting to cores.foldingathome.org:80
05:30:36:WU00:FS01:FahCore 22: Downloading 77.67MiB
...
07:23:03:WU00:FS01:FahCore 22: Download complete
07:23:03:WU00:FS01:Valid core signature
07:23:03:WU00:FS01:Unpacked 8.33MiB to cores/cores.foldingathome.org/win/64bit/22-0.0.13/Core_22.fah/FahCore_22.exe
07:23:03:WU00:FS01:Unpacked 3.06MiB to cores/cores.foldingathome.org/win/64bit/22-0.0.13/Core_22.fah/nvrtc-builtins64_92.dll
07:23:03:WU00:FS01:Unpacked 3.13MiB to cores/cores.foldingathome.org/win/64bit/22-0.0.13/Core_22.fah/OpenMM.dll
07:23:03:WU00:FS01:Unpacked 2.07MiB to cores/cores.foldingathome.org/win/64bit/22-0.0.13/Core_22.fah/OpenMMOpenCL.dll
07:23:03:WU00:FS01:Unpacked 14.82MiB to cores/cores.foldingathome.org/win/64bit/22-0.0.13/Core_22.fah/nvrtc64_92.dll
07:23:03:WU00:FS01:Unpacked 82.99MiB to cores/cores.foldingathome.org/win/64bit/22-0.0.13/Core_22.fah/cufft64_92.dll
07:23:03:WU00:FS01:Unpacked 275.00KiB to cores/cores.foldingathome.org/win/64bit/22-0.0.13/Core_22.fah/OpenMMCudaCompiler.dll
07:23:03:WU00:FS01:Unpacked 1.95MiB to cores/cores.foldingathome.org/win/64bit/22-0.0.13/Core_22.fah/OpenMMCUDA.dll
07:23:03:WU00:FS01:Unpacked 786.00KiB to cores/cores.foldingathome.org/win/64bit/22-0.0.13/Core_22.fah/OpenMMCPU.dll
07:23:03:WU00:FS01:Starting
07:23:03:WU00:FS01:Running FahCore: "D:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" "D:\Program Files (x86)\FAHData\cores/cores.foldingathome.org/win/64bit/22-0.0.13/Core_22.fah/FahCore_22.exe" -dir 00 -suffix 01 -version 706 -lifeline 91232 -checkpoint 30 -gpu-vendor nvidia -opencl-platform 1 -opencl-device 0 -cuda-device 0 -gpu 0
07:23:04:WU00:FS01:Started FahCore on PID 157352
07:23:04:WU00:FS01:Core PID:154580
07:23:04:WU00:FS01:FahCore 0x22 started
07:23:05:WU00:FS01:0x22:*********************** Log Started 2020-09-25T07:23:04Z ***********************
07:23:05:WU00:FS01:0x22:*************************** Core22 Folding@home Core ***************************
07:23:05:WU00:FS01:0x22:       Core: Core22
07:23:05:WU00:FS01:0x22:       Type: 0x22
07:23:05:WU00:FS01:0x22:    Version: 0.0.13
07:23:05:WU00:FS01:0x22:     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
07:23:05:WU00:FS01:0x22:  Copyright: 2020 foldingathome.org
07:23:05:WU00:FS01:0x22:   Homepage: https://foldingathome.org/
07:23:05:WU00:FS01:0x22:       Date: Sep 19 2020
07:23:05:WU00:FS01:0x22:       Time: 02:35:58
07:23:05:WU00:FS01:0x22:   Revision: 571cf95de6de2c592c7c3ed48fcfb2e33e9ea7d3
07:23:05:WU00:FS01:0x22:     Branch: core22-0.0.13
07:23:05:WU00:FS01:0x22:   Compiler: Visual C++ 2015
07:23:05:WU00:FS01:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
07:23:05:WU00:FS01:0x22:             -DOPENMM_GIT_HASH="\"189320d0\""
07:23:05:WU00:FS01:0x22:   Platform: win32 10
07:23:05:WU00:FS01:0x22:       Bits: 64
07:23:05:WU00:FS01:0x22:       Mode: Release
07:23:05:WU00:FS01:0x22:Maintainers: John Chodera <john.chodera@choderalab.org> and Peter Eastman
07:23:05:WU00:FS01:0x22:             <peastman@stanford.edu>
07:23:05:WU00:FS01:0x22:       Args: -dir 00 -suffix 01 -version 706 -lifeline 157352 -checkpoint 30
07:23:05:WU00:FS01:0x22:             -gpu-vendor nvidia -opencl-platform 1 -opencl-device 0 -cuda-device
07:23:05:WU00:FS01:0x22:             0 -gpu 0
07:23:05:WU00:FS01:0x22:************************************ libFAH ************************************
07:23:05:WU00:FS01:0x22:       Date: Sep 7 2020
07:23:05:WU00:FS01:0x22:       Time: 19:09:56
07:23:05:WU00:FS01:0x22:   Revision: 44301ed97b996b63fe736bb8073f22209cb2b603
07:23:05:WU00:FS01:0x22:     Branch: HEAD
07:23:05:WU00:FS01:0x22:   Compiler: Visual C++ 2015
07:23:05:WU00:FS01:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
07:23:05:WU00:FS01:0x22:   Platform: win32 10
07:23:05:WU00:FS01:0x22:       Bits: 64
07:23:05:WU00:FS01:0x22:       Mode: Release
07:23:05:WU00:FS01:0x22:************************************ CBang *************************************
07:23:05:WU00:FS01:0x22:       Date: Sep 7 2020
07:23:05:WU00:FS01:0x22:       Time: 19:08:30
07:23:05:WU00:FS01:0x22:   Revision: 33fcfc2b3ed2195a423606a264718e31e6b3903f
07:23:05:WU00:FS01:0x22:     Branch: HEAD
07:23:05:WU00:FS01:0x22:   Compiler: Visual C++ 2015
07:23:05:WU00:FS01:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
07:23:05:WU00:FS01:0x22:   Platform: win32 10
07:23:05:WU00:FS01:0x22:       Bits: 64
07:23:05:WU00:FS01:0x22:       Mode: Release
07:23:05:WU00:FS01:0x22:************************************ System ************************************
07:23:05:WU00:FS01:0x22:        CPU: Intel(R) Core(TM) i5-7200U CPU @ 2.50GHz
07:23:05:WU00:FS01:0x22:     CPU ID: GenuineIntel Family 6 Model 142 Stepping 9
07:23:05:WU00:FS01:0x22:       CPUs: 4
07:23:05:WU00:FS01:0x22:     Memory: 7.91GiB
07:23:05:WU00:FS01:0x22:Free Memory: 2.56GiB
07:23:05:WU00:FS01:0x22:    Threads: WINDOWS_THREADS
07:23:05:WU00:FS01:0x22: OS Version: 6.2
07:23:05:WU00:FS01:0x22:Has Battery: true
07:23:05:WU00:FS01:0x22: On Battery: false
07:23:05:WU00:FS01:0x22: UTC Offset: 8
07:23:05:WU00:FS01:0x22:        PID: 154580
07:23:05:WU00:FS01:0x22:        CWD: D:\Program Files (x86)\FAHData\work
07:23:05:WU00:FS01:0x22:************************************ OpenMM ************************************
07:23:05:WU00:FS01:0x22:   Revision: 189320d0
07:23:05:WU00:FS01:0x22:********************************************************************************
07:23:05:WU00:FS01:0x22:Project: 14484 (Run 0, Clone 920, Gen 74)
07:23:05:WU00:FS01:0x22:Unit: 0x0000006f80fccb095f171a3d48be36d6
07:23:05:WU00:FS01:0x22:Reading tar file core.xml
07:23:05:WU00:FS01:0x22:Reading tar file integrator.xml.bz2
07:23:05:WU00:FS01:0x22:Reading tar file state.xml.bz2
07:23:05:WU00:FS01:0x22:Reading tar file system.xml.bz2
07:23:05:WU00:FS01:0x22:Digital signatures verified
07:23:05:WU00:FS01:0x22:Folding@home GPU Core22 Folding@home Core
07:23:05:WU00:FS01:0x22:Version 0.0.13
07:23:06:WU00:FS01:0x22:  Checkpoint write interval: 25000 steps (2%) [50 total]
07:23:06:WU00:FS01:0x22:  JSON viewer frame write interval: 12500 steps (1%) [100 total]
07:23:06:WU00:FS01:0x22:  XTC frame write interval: 10000 steps (0.8%) [125 total]
07:23:06:WU00:FS01:0x22:  Global context and integrator variables write interval: disabled
07:23:06:WU00:FS01:0x22:There are 4 platforms available.
07:23:06:WU00:FS01:0x22:Platform 0: Reference
07:23:06:WU00:FS01:0x22:Platform 1: CPU
07:23:06:WU00:FS01:0x22:Platform 2: OpenCL
07:23:06:WU00:FS01:0x22:  opencl-device 0 specified
07:23:06:WU00:FS01:0x22:Platform 3: CUDA
07:23:06:WU00:FS01:0x22:  cuda-device 0 specified
07:23:31:WU00:FS01:0x22:Attempting to create CUDA context:
07:23:31:WU00:FS01:0x22:  Configuring platform CUDA
07:23:33:WU00:FS01:0x22:Failed to create CUDA context:
07:23:33:WU00:FS01:0x22:Error loading CUDA module: CUDA_ERROR_INVALID_PTX (218)
07:23:33:WU00:FS01:0x22:Attempting to create OpenCL context:
07:23:33:WU00:FS01:0x22:  Configuring platform OpenCL
07:23:42:WU00:FS01:0x22:  Using OpenCL on platformId 1 and gpu 0
07:23:42:WU00:FS01:0x22:Completed 0 out of 1250000 steps (0%)

Re: Core22 v0.0.13 return CUDA_ERROR_INVALID_PTX (218)

Posted: Fri Sep 25, 2020 12:36 pm
by foldy
It means it cannot run CUDA so it switches to OpenCL

Re: Core22 v0.0.13 return CUDA_ERROR_INVALID_PTX (218)

Posted: Fri Sep 25, 2020 2:37 pm
by Kjetil
core 22 v0.0.13 is om beta and p 14484 is not v13 beta Project. And did not see beta on in his log.

Re: Core22 v0.0.13 return CUDA_ERROR_INVALID_PTX (218)

Posted: Fri Sep 25, 2020 3:12 pm
by gunnarre
Core 22 v. 0.0.13 is apparently released to non-beta users too. I have folded a WU on project 14486 in CUDA on a GTX 1080. I have client-type = advanced set, which is technically late stage beta, but it does seem like some projects are now using Core 22 v. 0.0.13 in the general non-beta folding. Yay, CUDA?

Re: Core22 v0.0.13 return CUDA_ERROR_INVALID_PTX (218)

Posted: Fri Sep 25, 2020 3:30 pm
by aetch
I'm just a pleb and I have it too. :p

Re: Core22 v0.0.13 return CUDA_ERROR_INVALID_PTX (218)

Posted: Fri Sep 25, 2020 3:38 pm
by gunnarre
Prepare for massive speed boosts, ladies and gentlemen!

Re: Core22 v0.0.13 return CUDA_ERROR_INVALID_PTX (218)

Posted: Fri Sep 25, 2020 3:51 pm
by vmzy
Fix the problem by upgrade nVidia driver from 391.24 to latest 456.38.
But when FAH run CUDA platform successfully, it has ignored the checkpoints(has been ran 18%) generated by opencl platform previous, and restart from beginning(0%).
The TPF has been reduced from 25min to 19min.

Re: Core22 v0.0.13 return CUDA_ERROR_INVALID_PTX (218)

Posted: Fri Sep 25, 2020 8:10 pm
by Kjetil
gunnarre wrote:Core 22 v. 0.0.13 is apparently released to non-beta users too. I have folded a WU on project 14486 in CUDA on a GTX 1080. I have client-type = advanced set, which is technically late stage beta, but it does seem like some projects are now using Core 22 v. 0.0.13 in the general non-beta folding. Yay, CUDA?
Okay sorry all, on beta and only running 17xxx projects.(0013) Now i running p 134xx on c 22 0011

Re: Core22 v0.0.13 return CUDA_ERROR_INVALID_PTX (218)

Posted: Fri Sep 25, 2020 9:36 pm
by JohnChodera
> Core 22 v. 0.0.13 is apparently released to non-beta users too.

Whoops! This made it out of the testing lab accidentally. We're still doing a lot of testing, so nothing official yet...

~ John Chodera // MSKCC

Re: Core22 v0.0.13 return CUDA_ERROR_INVALID_PTX (218)

Posted: Sat Sep 26, 2020 12:13 am
by PantherX
vmzy wrote:...when FAH run CUDA platform successfully, it has ignored the checkpoints(has been ran 18%) generated by opencl platform previous, and restart from beginning(0%)...
Currently, that's expected since the checkpoints written in CUDA context can't be read by OpenCL and vice-versa. Thus, the WU will restart from 0% and that will also throw off HFM.NET (if you're using it). It is a very rare edge case that you have discovered and the likelihood of it happening outside of the dev/testers environment is exceptionally low. This won't normally happen even for Beta testers... you can consider yourself really lucky in that aspect :lol:

It would be nice to ensure that that the checkpoint written is platform independent but there's no official commitment yet if it will be available in FahCore_22.