P13454 Bad allocation error
Posted: Wed Jun 16, 2021 8:03 pm
Hey guys,
Today I noticed 2 failed units on my 2080ti, both P13454 and both units were finished to 100%.
The units received base credit, and one of them has been finished by someone else without an error.
https://apps.foldingathome.org/wu#proje ... e=58&gen=0
https://apps.foldingathome.org/wu#proje ... e=41&gen=0
Any idea what the reason could be? This card has been rock solid for as long as I own it (17 months), and the temps are around 50°C on water.
Thanks in advance,
Bastiaan
Today I noticed 2 failed units on my 2080ti, both P13454 and both units were finished to 100%.
The units received base credit, and one of them has been finished by someone else without an error.
https://apps.foldingathome.org/wu#proje ... e=58&gen=0
https://apps.foldingathome.org/wu#proje ... e=41&gen=0
Any idea what the reason could be? This card has been rock solid for as long as I own it (17 months), and the temps are around 50°C on water.
Code: Select all
18:21:29:WU01:FS00:Starting
18:21:29:WU01:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\ProgramData\FAHClient\cores/cores.foldingathome.org/win/64bit/22-0.0.13/Core_22.fah/FahCore_22.exe -dir 01 -suffix 01 -version 706 -lifeline 9516 -checkpoint 15 -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor nvidia -gpu 0 -gpu-usage 100
18:21:29:WU01:FS00:Started FahCore on PID 12980
18:21:29:WU01:FS00:Core PID:12416
18:21:29:WU01:FS00:FahCore 0x22 started
18:21:29:WU01:FS00:0x22:*********************** Log Started 2021-06-16T18:21:29Z ***********************
18:21:29:WU01:FS00:0x22:*************************** Core22 Folding@home Core ***************************
18:21:29:WU01:FS00:0x22: Core: Core22
18:21:29:WU01:FS00:0x22: Type: 0x22
18:21:29:WU01:FS00:0x22: Version: 0.0.13
18:21:29:WU01:FS00:0x22: Author: Joseph Coffland <joseph@cauldrondevelopment.com>
18:21:29:WU01:FS00:0x22: Copyright: 2020 foldingathome.org
18:21:29:WU01:FS00:0x22: Homepage: https://foldingathome.org/
18:21:29:WU01:FS00:0x22: Date: Sep 19 2020
18:21:29:WU01:FS00:0x22: Time: 02:35:58
18:21:29:WU01:FS00:0x22: Revision: 571cf95de6de2c592c7c3ed48fcfb2e33e9ea7d3
18:21:29:WU01:FS00:0x22: Branch: core22-0.0.13
18:21:29:WU01:FS00:0x22: Compiler: Visual C++ 2015
18:21:29:WU01:FS00:0x22: Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
18:21:29:WU01:FS00:0x22: -DOPENMM_GIT_HASH="\"189320d0\""
18:21:29:WU01:FS00:0x22: Platform: win32 10
18:21:29:WU01:FS00:0x22: Bits: 64
18:21:29:WU01:FS00:0x22: Mode: Release
18:21:29:WU01:FS00:0x22:Maintainers: John Chodera <john.chodera@choderalab.org> and Peter Eastman
18:21:29:WU01:FS00:0x22: <peastman@stanford.edu>
18:21:29:WU01:FS00:0x22: Args: -dir 01 -suffix 01 -version 706 -lifeline 12980 -checkpoint 15
18:21:29:WU01:FS00:0x22: -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu-vendor
18:21:29:WU01:FS00:0x22: nvidia -gpu 0 -gpu-usage 100
18:21:29:WU01:FS00:0x22:************************************ libFAH ************************************
18:21:29:WU01:FS00:0x22: Date: Sep 7 2020
18:21:29:WU01:FS00:0x22: Time: 19:09:56
18:21:29:WU01:FS00:0x22: Revision: 44301ed97b996b63fe736bb8073f22209cb2b603
18:21:29:WU01:FS00:0x22: Branch: HEAD
18:21:29:WU01:FS00:0x22: Compiler: Visual C++ 2015
18:21:29:WU01:FS00:0x22: Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
18:21:29:WU01:FS00:0x22: Platform: win32 10
18:21:29:WU01:FS00:0x22: Bits: 64
18:21:29:WU01:FS00:0x22: Mode: Release
18:21:29:WU01:FS00:0x22:************************************ CBang *************************************
18:21:29:WU01:FS00:0x22: Date: Sep 7 2020
18:21:29:WU01:FS00:0x22: Time: 19:08:30
18:21:29:WU01:FS00:0x22: Revision: 33fcfc2b3ed2195a423606a264718e31e6b3903f
18:21:29:WU01:FS00:0x22: Branch: HEAD
18:21:29:WU01:FS00:0x22: Compiler: Visual C++ 2015
18:21:29:WU01:FS00:0x22: Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
18:21:29:WU01:FS00:0x22: Platform: win32 10
18:21:29:WU01:FS00:0x22: Bits: 64
18:21:29:WU01:FS00:0x22: Mode: Release
18:21:29:WU01:FS00:0x22:************************************ System ************************************
18:21:29:WU01:FS00:0x22: CPU: Intel(R) Core(TM) i7-7700K CPU @ 4.20GHz
18:21:29:WU01:FS00:0x22: CPU ID: GenuineIntel Family 6 Model 158 Stepping 9
18:21:29:WU01:FS00:0x22: CPUs: 8
18:21:29:WU01:FS00:0x22: Memory: 7.94GiB
18:21:29:WU01:FS00:0x22:Free Memory: 3.99GiB
18:21:29:WU01:FS00:0x22: Threads: WINDOWS_THREADS
18:21:29:WU01:FS00:0x22: OS Version: 6.2
18:21:29:WU01:FS00:0x22:Has Battery: false
18:21:29:WU01:FS00:0x22: On Battery: false
18:21:29:WU01:FS00:0x22: UTC Offset: 2
18:21:29:WU01:FS00:0x22: PID: 12416
18:21:29:WU01:FS00:0x22: CWD: C:\ProgramData\FAHClient\work
18:21:29:WU01:FS00:0x22:************************************ OpenMM ************************************
18:21:29:WU01:FS00:0x22: Revision: 189320d0
18:21:29:WU01:FS00:0x22:********************************************************************************
18:21:29:WU01:FS00:0x22:Project: 13454 (Run 1090, Clone 58, Gen 0)
18:21:29:WU01:FS00:0x22:Unit: 0x00000000000000000000000000000000
18:21:29:WU01:FS00:0x22:Reading tar file core.xml
18:21:29:WU01:FS00:0x22:Reading tar file integrator.xml.bz2
18:21:29:WU01:FS00:0x22:Reading tar file state.xml.bz2
18:21:29:WU01:FS00:0x22:Reading tar file system.xml.bz2
18:21:29:WU01:FS00:0x22:Digital signatures verified
18:21:29:WU01:FS00:0x22:Folding@home GPU Core22 Folding@home Core
18:21:29:WU01:FS00:0x22:Version 0.0.13
18:21:29:WU01:FS00:0x22: Checkpoint write interval: 50000 steps (5%) [20 total]
18:21:29:WU01:FS00:0x22: JSON viewer frame write interval: 10000 steps (1%) [100 total]
18:21:29:WU01:FS00:0x22: XTC frame write interval: 250000 steps (25%) [4 total]
18:21:29:WU01:FS00:0x22: Global context and integrator variables write interval: 25000 steps (2.5%) [40 total]
18:21:29:WU01:FS00:0x22:There are 4 platforms available.
18:21:29:WU01:FS00:0x22:Platform 0: Reference
18:21:29:WU01:FS00:0x22:Platform 1: CPU
18:21:29:WU01:FS00:0x22:Platform 2: OpenCL
18:21:29:WU01:FS00:0x22: opencl-device 0 specified
18:21:29:WU01:FS00:0x22:Platform 3: CUDA
18:21:29:WU01:FS00:0x22: cuda-device 0 specified
18:21:37:WU01:FS00:0x22:Attempting to create CUDA context:
18:21:37:WU01:FS00:0x22: Configuring platform CUDA
18:21:41:WU01:FS00:0x22: Using CUDA and gpu 0
18:21:41:WU01:FS00:0x22:Completed 0 out of 1000000 steps (0%)
18:21:42:WU01:FS00:0x22:Checkpoint completed at step 0
18:22:26:WU01:FS00:0x22:Completed 10000 out of 1000000 steps (1%)
18:23:09:WU01:FS00:0x22:Completed 20000 out of 1000000 steps (2%)
18:23:52:WU01:FS00:0x22:Completed 30000 out of 1000000 steps (3%)
18:24:36:WU01:FS00:0x22:Completed 40000 out of 1000000 steps (4%)
18:25:19:WU01:FS00:0x22:Completed 50000 out of 1000000 steps (5%)
18:25:20:WU01:FS00:0x22:Checkpoint completed at step 50000
18:26:03:WU01:FS00:0x22:Completed 60000 out of 1000000 steps (6%)
18:26:46:WU01:FS00:0x22:Completed 70000 out of 1000000 steps (7%)
18:27:30:WU01:FS00:0x22:Completed 80000 out of 1000000 steps (8%)
18:28:13:WU01:FS00:0x22:Completed 90000 out of 1000000 steps (9%)
18:28:56:WU01:FS00:0x22:Completed 100000 out of 1000000 steps (10%)
18:28:57:WU01:FS00:0x22:Checkpoint completed at step 100000
18:29:40:WU01:FS00:0x22:Completed 110000 out of 1000000 steps (11%)
18:30:24:WU01:FS00:0x22:Completed 120000 out of 1000000 steps (12%)
18:31:07:WU01:FS00:0x22:Completed 130000 out of 1000000 steps (13%)
18:31:51:WU01:FS00:0x22:Completed 140000 out of 1000000 steps (14%)
18:32:34:WU01:FS00:0x22:Completed 150000 out of 1000000 steps (15%)
18:32:34:WU01:FS00:0x22:Checkpoint completed at step 150000
18:33:18:WU01:FS00:0x22:Completed 160000 out of 1000000 steps (16%)
18:34:01:WU01:FS00:0x22:Completed 170000 out of 1000000 steps (17%)
18:34:44:WU01:FS00:0x22:Completed 180000 out of 1000000 steps (18%)
18:35:28:WU01:FS00:0x22:Completed 190000 out of 1000000 steps (19%)
18:36:11:WU01:FS00:0x22:Completed 200000 out of 1000000 steps (20%)
18:36:12:WU01:FS00:0x22:Checkpoint completed at step 200000
18:36:55:WU01:FS00:0x22:Completed 210000 out of 1000000 steps (21%)
18:37:38:WU01:FS00:0x22:Completed 220000 out of 1000000 steps (22%)
18:38:22:WU01:FS00:0x22:Completed 230000 out of 1000000 steps (23%)
18:39:05:WU01:FS00:0x22:Completed 240000 out of 1000000 steps (24%)
18:39:49:WU01:FS00:0x22:Completed 250000 out of 1000000 steps (25%)
18:39:49:WU01:FS00:0x22:Checkpoint completed at step 250000
18:40:34:WU01:FS00:0x22:Completed 260000 out of 1000000 steps (26%)
18:41:19:WU01:FS00:0x22:Completed 270000 out of 1000000 steps (27%)
18:42:04:WU01:FS00:0x22:Completed 280000 out of 1000000 steps (28%)
18:42:49:WU01:FS00:0x22:Completed 290000 out of 1000000 steps (29%)
18:43:33:WU01:FS00:0x22:Completed 300000 out of 1000000 steps (30%)
18:43:34:WU01:FS00:0x22:Checkpoint completed at step 300000
18:44:19:WU01:FS00:0x22:Completed 310000 out of 1000000 steps (31%)
18:45:03:WU01:FS00:0x22:Completed 320000 out of 1000000 steps (32%)
18:45:48:WU01:FS00:0x22:Completed 330000 out of 1000000 steps (33%)
18:46:33:WU01:FS00:0x22:Completed 340000 out of 1000000 steps (34%)
18:47:18:WU01:FS00:0x22:Completed 350000 out of 1000000 steps (35%)
18:47:19:WU01:FS00:0x22:Checkpoint completed at step 350000
18:48:04:WU01:FS00:0x22:Completed 360000 out of 1000000 steps (36%)
18:48:48:WU01:FS00:0x22:Completed 370000 out of 1000000 steps (37%)
18:49:33:WU01:FS00:0x22:Completed 380000 out of 1000000 steps (38%)
18:50:18:WU01:FS00:0x22:Completed 390000 out of 1000000 steps (39%)
18:51:03:WU01:FS00:0x22:Completed 400000 out of 1000000 steps (40%)
18:51:03:WU01:FS00:0x22:Checkpoint completed at step 400000
18:51:48:WU01:FS00:0x22:Completed 410000 out of 1000000 steps (41%)
18:52:33:WU01:FS00:0x22:Completed 420000 out of 1000000 steps (42%)
18:53:18:WU01:FS00:0x22:Completed 430000 out of 1000000 steps (43%)
18:54:03:WU01:FS00:0x22:Completed 440000 out of 1000000 steps (44%)
18:54:47:WU01:FS00:0x22:Completed 450000 out of 1000000 steps (45%)
18:54:48:WU01:FS00:0x22:Checkpoint completed at step 450000
18:55:33:WU01:FS00:0x22:Completed 460000 out of 1000000 steps (46%)
18:56:18:WU01:FS00:0x22:Completed 470000 out of 1000000 steps (47%)
18:57:03:WU01:FS00:0x22:Completed 480000 out of 1000000 steps (48%)
18:57:47:WU01:FS00:0x22:Completed 490000 out of 1000000 steps (49%)
18:58:32:WU01:FS00:0x22:Completed 500000 out of 1000000 steps (50%)
18:58:33:WU01:FS00:0x22:Checkpoint completed at step 500000
18:59:16:WU01:FS00:0x22:Completed 510000 out of 1000000 steps (51%)
19:00:00:WU01:FS00:0x22:Completed 520000 out of 1000000 steps (52%)
19:00:43:WU01:FS00:0x22:Completed 530000 out of 1000000 steps (53%)
19:01:26:WU01:FS00:0x22:Completed 540000 out of 1000000 steps (54%)
19:02:10:WU01:FS00:0x22:Completed 550000 out of 1000000 steps (55%)
19:02:10:WU01:FS00:0x22:Checkpoint completed at step 550000
19:02:54:WU01:FS00:0x22:Completed 560000 out of 1000000 steps (56%)
19:03:37:WU01:FS00:0x22:Completed 570000 out of 1000000 steps (57%)
19:04:20:WU01:FS00:0x22:Completed 580000 out of 1000000 steps (58%)
19:05:04:WU01:FS00:0x22:Completed 590000 out of 1000000 steps (59%)
19:05:47:WU01:FS00:0x22:Completed 600000 out of 1000000 steps (60%)
19:05:48:WU01:FS00:0x22:Checkpoint completed at step 600000
19:06:31:WU01:FS00:0x22:Completed 610000 out of 1000000 steps (61%)
19:07:14:WU01:FS00:0x22:Completed 620000 out of 1000000 steps (62%)
19:07:58:WU01:FS00:0x22:Completed 630000 out of 1000000 steps (63%)
19:08:41:WU01:FS00:0x22:Completed 640000 out of 1000000 steps (64%)
19:09:24:WU01:FS00:0x22:Completed 650000 out of 1000000 steps (65%)
19:09:25:WU01:FS00:0x22:Checkpoint completed at step 650000
19:10:08:WU01:FS00:0x22:Completed 660000 out of 1000000 steps (66%)
19:10:52:WU01:FS00:0x22:Completed 670000 out of 1000000 steps (67%)
19:11:35:WU01:FS00:0x22:Completed 680000 out of 1000000 steps (68%)
19:12:19:WU01:FS00:0x22:Completed 690000 out of 1000000 steps (69%)
19:13:02:WU01:FS00:0x22:Completed 700000 out of 1000000 steps (70%)
19:13:02:WU01:FS00:0x22:Checkpoint completed at step 700000
19:13:46:WU01:FS00:0x22:Completed 710000 out of 1000000 steps (71%)
19:14:29:WU01:FS00:0x22:Completed 720000 out of 1000000 steps (72%)
19:15:12:WU01:FS00:0x22:Completed 730000 out of 1000000 steps (73%)
19:15:56:WU01:FS00:0x22:Completed 740000 out of 1000000 steps (74%)
19:16:39:WU01:FS00:0x22:Completed 750000 out of 1000000 steps (75%)
19:16:40:WU01:FS00:0x22:Checkpoint completed at step 750000
19:17:25:WU01:FS00:0x22:Completed 760000 out of 1000000 steps (76%)
19:18:10:WU01:FS00:0x22:Completed 770000 out of 1000000 steps (77%)
19:18:54:WU01:FS00:0x22:Completed 780000 out of 1000000 steps (78%)
19:19:39:WU01:FS00:0x22:Completed 790000 out of 1000000 steps (79%)
19:20:24:WU01:FS00:0x22:Completed 800000 out of 1000000 steps (80%)
19:20:25:WU01:FS00:0x22:Checkpoint completed at step 800000
19:21:09:WU01:FS00:0x22:Completed 810000 out of 1000000 steps (81%)
19:21:54:WU01:FS00:0x22:Completed 820000 out of 1000000 steps (82%)
19:22:39:WU01:FS00:0x22:Completed 830000 out of 1000000 steps (83%)
19:23:24:WU01:FS00:0x22:Completed 840000 out of 1000000 steps (84%)
19:24:09:WU01:FS00:0x22:Completed 850000 out of 1000000 steps (85%)
19:24:09:WU01:FS00:0x22:Checkpoint completed at step 850000
19:24:54:WU01:FS00:0x22:Completed 860000 out of 1000000 steps (86%)
19:25:39:WU01:FS00:0x22:Completed 870000 out of 1000000 steps (87%)
19:26:24:WU01:FS00:0x22:Completed 880000 out of 1000000 steps (88%)
19:27:09:WU01:FS00:0x22:Completed 890000 out of 1000000 steps (89%)
19:27:53:WU01:FS00:0x22:Completed 900000 out of 1000000 steps (90%)
19:27:54:WU01:FS00:0x22:Checkpoint completed at step 900000
19:28:39:WU01:FS00:0x22:Completed 910000 out of 1000000 steps (91%)
19:29:24:WU01:FS00:0x22:Completed 920000 out of 1000000 steps (92%)
19:30:08:WU01:FS00:0x22:Completed 930000 out of 1000000 steps (93%)
19:30:53:WU01:FS00:0x22:Completed 940000 out of 1000000 steps (94%)
19:31:38:WU01:FS00:0x22:Completed 950000 out of 1000000 steps (95%)
19:31:39:WU01:FS00:0x22:Checkpoint completed at step 950000
19:32:23:WU01:FS00:0x22:Completed 960000 out of 1000000 steps (96%)
19:33:08:WU01:FS00:0x22:Completed 970000 out of 1000000 steps (97%)
19:33:53:WU01:FS00:0x22:Completed 980000 out of 1000000 steps (98%)
19:34:38:WU01:FS00:0x22:Completed 990000 out of 1000000 steps (99%)
19:34:39:WU00:FS00:Connecting to assign1.foldingathome.org:80
19:34:40:WU00:FS00:Assigned to work server 54.157.202.86
19:34:40:WU00:FS00:Requesting new work unit for slot 00: gpu:1:0 TU102 [GeForce RTX 2080 Ti Rev. A] M 13448 from 54.157.202.86
19:34:40:WU00:FS00:Connecting to 54.157.202.86:8080
19:34:41:WU00:FS00:Downloading 6.31MiB
19:34:47:WU00:FS00:Download 99.04%
19:34:47:WU00:FS00:Download complete
19:34:47:WU00:FS00:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:13454 run:1136 clone:11 gen:0 core:0x22 unit:0x0000000b000000000000348e00000470
19:35:23:WU01:FS00:0x22:Completed 1000000 out of 1000000 steps (100%)
19:35:23:WU01:FS00:0x22:Average performance: 192.857 ns/day
19:35:23:WU01:FS00:0x22:Checkpoint completed at step 1000000
19:35:24:WU01:FS00:0x22:ERROR:exception: bad allocation
19:35:24:WU01:FS00:0x22:Saving result file ..\logfile_01.txt
19:35:24:WU01:FS00:0x22:Saving result file globals.csv
19:35:24:WU01:FS00:0x22:Saving result file positions.xtc
19:35:24:WU01:FS00:0x22:Saving result file science.log
19:35:24:WU01:FS00:0x22:Folding@home Core Shutdown: BAD_WORK_UNIT
19:35:25:WARNING:WU01:FS00:FahCore returned: BAD_WORK_UNIT (114 = 0x72)
19:35:25:WU01:FS00:Sending unit results: id:01 state:SEND error:FAULTY project:13454 run:1090 clone:58 gen:0 core:0x22 unit:0x0000003a000000000000348e00000442
19:35:25:WU01:FS00:Uploading 155.93KiB to 54.157.202.86Bastiaan
