Page 1 of 1

Project: 14225 (Run 48, Clone 1, Gen 24)

Posted: Sat Sep 28, 2019 5:18 pm
by Adam_Ford
Keeps failing to upload since 9/22

Code: Select all

19:02:30:WU02:FS01:Connecting to 65.254.110.245:8080
19:02:30:WU02:FS01:Assigned to work server 128.252.203.10
19:02:30:WU02:FS01:Requesting new work unit for slot 01: RUNNING gpu:0:TU102 [GeForce RTX 2080 Ti Rev. A] M 13448 from 128.252.203.10
19:02:30:WU02:FS01:Connecting to 128.252.203.10:8080
19:02:31:WU02:FS01:Downloading 65.96MiB
19:02:37:WU02:FS01:Download 100.00%
19:02:37:WU02:FS01:Download complete
19:02:37:WU02:FS01:Received Unit: id:02 state:DOWNLOAD error:NO_ERROR project:14225 run:48 clone:1 gen:24 core:0x21 unit:0x0000002480fccb0a5d6545dbf5b1623c
19:02:37:WU02:FS01:Starting
19:02:37:WU02:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\Users\fongo\AppData\Roaming\FAHClient\cores/cores.foldingathome.org/Win32/AMD64/NVIDIA/Fermi/Core_21.fah/FahCore_21.exe -dir 02 -suffix 01 -version 705 -lifeline 16268 -checkpoint 30 -gpu-vendor nvidia -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu 0
19:02:37:WU02:FS01:Started FahCore on PID 7348
19:02:37:WU02:FS01:Core PID:4808
19:02:37:WU02:FS01:FahCore 0x21 started
19:02:38:WU02:FS01:0x21:*********************** Log Started 2019-09-22T19:02:37Z ***********************
19:02:38:WU02:FS01:0x21:Project: 14225 (Run 48, Clone 1, Gen 24)
19:02:38:WU02:FS01:0x21:Unit: 0x0000002480fccb0a5d6545dbf5b1623c
19:02:38:WU02:FS01:0x21:CPU: 0x00000000000000000000000000000000
19:02:38:WU02:FS01:0x21:Machine: 1
19:02:38:WU02:FS01:0x21:Reading tar file core.xml
19:02:38:WU02:FS01:0x21:Reading tar file integrator.xml
19:02:38:WU02:FS01:0x21:Reading tar file state.xml
19:02:40:WU02:FS01:0x21:Reading tar file system.xml
19:02:41:WU02:FS01:0x21:Digital signatures verified
19:02:41:WU02:FS01:0x21:Folding@home GPU Core21 Folding@home Core
19:02:41:WU02:FS01:0x21:Version 0.0.20
19:03:03:WU02:FS01:0x21:Completed 0 out of 2000000 steps (0%)
19:03:03:WU02:FS01:0x21:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
19:04:13:WU02:FS01:0x21:Completed 20000 out of 2000000 steps (1%)
19:05:30:WU02:FS01:0x21:Completed 40000 out of 2000000 steps (2%)
19:06:45:WU02:FS01:0x21:Completed 60000 out of 2000000 steps (3%)
19:08:00:WU02:FS01:0x21:Completed 80000 out of 2000000 steps (4%)
19:09:15:WU02:FS01:0x21:Completed 100000 out of 2000000 steps (5%)
19:10:30:WU02:FS01:0x21:Completed 120000 out of 2000000 steps (6%)
19:11:45:WU02:FS01:0x21:Completed 140000 out of 2000000 steps (7%)
19:13:00:WU02:FS01:0x21:Completed 160000 out of 2000000 steps (8%)
19:14:15:WU02:FS01:0x21:Completed 180000 out of 2000000 steps (9%)
19:15:30:WU02:FS01:0x21:Completed 200000 out of 2000000 steps (10%)
19:16:45:WU02:FS01:0x21:Completed 220000 out of 2000000 steps (11%)
19:18:00:WU02:FS01:0x21:Completed 240000 out of 2000000 steps (12%)
19:19:15:WU02:FS01:0x21:Completed 260000 out of 2000000 steps (13%)
19:20:30:WU02:FS01:0x21:Completed 280000 out of 2000000 steps (14%)
19:21:45:WU02:FS01:0x21:Completed 300000 out of 2000000 steps (15%)
19:23:00:WU02:FS01:0x21:Completed 320000 out of 2000000 steps (16%)
19:24:15:WU02:FS01:0x21:Completed 340000 out of 2000000 steps (17%)
19:25:30:WU02:FS01:0x21:Completed 360000 out of 2000000 steps (18%)
19:26:45:WU02:FS01:0x21:Completed 380000 out of 2000000 steps (19%)
19:28:01:WU02:FS01:0x21:Completed 400000 out of 2000000 steps (20%)
19:29:16:WU02:FS01:0x21:Completed 420000 out of 2000000 steps (21%)
19:30:31:WU02:FS01:0x21:Completed 440000 out of 2000000 steps (22%)
19:31:46:WU02:FS01:0x21:Completed 460000 out of 2000000 steps (23%)
19:33:02:WU02:FS01:0x21:Completed 480000 out of 2000000 steps (24%)
19:34:17:WU02:FS01:0x21:Completed 500000 out of 2000000 steps (25%)
19:35:32:WU02:FS01:0x21:Completed 520000 out of 2000000 steps (26%)
19:36:33:WU02:FS01:0x21:Bad State detected... attempting to resume from last good checkpoint. Is your system overclocked?
19:37:42:WU02:FS01:0x21:Completed 540000 out of 2000000 steps (27%)
19:38:58:WU02:FS01:0x21:Completed 560000 out of 2000000 steps (28%)
19:40:13:WU02:FS01:0x21:Completed 580000 out of 2000000 steps (29%)
19:41:28:WU02:FS01:0x21:Completed 600000 out of 2000000 steps (30%)
19:42:44:WU02:FS01:0x21:Completed 620000 out of 2000000 steps (31%)
19:44:00:WU02:FS01:0x21:Completed 640000 out of 2000000 steps (32%)
19:45:15:WU02:FS01:0x21:Completed 660000 out of 2000000 steps (33%)
19:46:30:WU02:FS01:0x21:Completed 680000 out of 2000000 steps (34%)
19:47:45:WU02:FS01:0x21:Completed 700000 out of 2000000 steps (35%)
19:49:00:WU02:FS01:0x21:Completed 720000 out of 2000000 steps (36%)
19:50:16:WU02:FS01:0x21:Completed 740000 out of 2000000 steps (37%)
19:51:31:WU02:FS01:0x21:Completed 760000 out of 2000000 steps (38%)
19:52:46:WU02:FS01:0x21:Completed 780000 out of 2000000 steps (39%)
19:54:01:WU02:FS01:0x21:Completed 800000 out of 2000000 steps (40%)
19:55:17:WU02:FS01:0x21:Completed 820000 out of 2000000 steps (41%)
19:56:32:WU02:FS01:0x21:Completed 840000 out of 2000000 steps (42%)
19:57:47:WU02:FS01:0x21:Completed 860000 out of 2000000 steps (43%)
19:59:03:WU02:FS01:0x21:Completed 880000 out of 2000000 steps (44%)
20:00:18:WU02:FS01:0x21:Completed 900000 out of 2000000 steps (45%)
20:01:34:WU02:FS01:0x21:Completed 920000 out of 2000000 steps (46%)
20:02:49:WU02:FS01:0x21:Completed 940000 out of 2000000 steps (47%)
20:04:05:WU02:FS01:0x21:Completed 960000 out of 2000000 steps (48%)
20:05:20:WU02:FS01:0x21:Completed 980000 out of 2000000 steps (49%)
20:06:36:WU02:FS01:0x21:Completed 1000000 out of 2000000 steps (50%)
20:07:52:WU02:FS01:0x21:Completed 1020000 out of 2000000 steps (51%)
20:09:07:WU02:FS01:0x21:Completed 1040000 out of 2000000 steps (52%)
20:10:22:WU02:FS01:0x21:Completed 1060000 out of 2000000 steps (53%)
20:11:38:WU02:FS01:0x21:Completed 1080000 out of 2000000 steps (54%)
20:12:54:WU02:FS01:0x21:Completed 1100000 out of 2000000 steps (55%)
20:14:09:WU02:FS01:0x21:Completed 1120000 out of 2000000 steps (56%)
20:15:24:WU02:FS01:0x21:Completed 1140000 out of 2000000 steps (57%)
20:15:43:WU02:FS01:0x21:Bad State detected... attempting to resume from last good checkpoint. Is your system overclocked?
20:16:52:WU02:FS01:0x21:Completed 1160000 out of 2000000 steps (58%)
20:18:07:WU02:FS01:0x21:Completed 1180000 out of 2000000 steps (59%)
20:19:22:WU02:FS01:0x21:Completed 1200000 out of 2000000 steps (60%)
20:20:37:WU02:FS01:0x21:Completed 1220000 out of 2000000 steps (61%)
20:21:52:WU02:FS01:0x21:Completed 1240000 out of 2000000 steps (62%)
20:23:08:WU02:FS01:0x21:Completed 1260000 out of 2000000 steps (63%)
20:24:23:WU02:FS01:0x21:Completed 1280000 out of 2000000 steps (64%)
20:25:38:WU02:FS01:0x21:Completed 1300000 out of 2000000 steps (65%)
20:26:54:WU02:FS01:0x21:Completed 1320000 out of 2000000 steps (66%)
20:28:09:WU02:FS01:0x21:Completed 1340000 out of 2000000 steps (67%)
20:29:24:WU02:FS01:0x21:Completed 1360000 out of 2000000 steps (68%)
20:30:40:WU02:FS01:0x21:Completed 1380000 out of 2000000 steps (69%)
20:31:55:WU02:FS01:0x21:Completed 1400000 out of 2000000 steps (70%)
20:33:11:WU02:FS01:0x21:Completed 1420000 out of 2000000 steps (71%)
20:34:26:WU02:FS01:0x21:Completed 1440000 out of 2000000 steps (72%)
20:35:42:WU02:FS01:0x21:Completed 1460000 out of 2000000 steps (73%)
20:36:57:WU02:FS01:0x21:Completed 1480000 out of 2000000 steps (74%)
20:38:13:WU02:FS01:0x21:Completed 1500000 out of 2000000 steps (75%)
20:39:28:WU02:FS01:0x21:Completed 1520000 out of 2000000 steps (76%)
20:40:44:WU02:FS01:0x21:Completed 1540000 out of 2000000 steps (77%)
20:41:59:WU02:FS01:0x21:Completed 1560000 out of 2000000 steps (78%)
20:43:15:WU02:FS01:0x21:Completed 1580000 out of 2000000 steps (79%)
20:44:30:WU02:FS01:0x21:Completed 1600000 out of 2000000 steps (80%)
20:45:46:WU02:FS01:0x21:Completed 1620000 out of 2000000 steps (81%)
20:47:01:WU02:FS01:0x21:Completed 1640000 out of 2000000 steps (82%)
20:48:17:WU02:FS01:0x21:Completed 1660000 out of 2000000 steps (83%)
20:49:32:WU02:FS01:0x21:Completed 1680000 out of 2000000 steps (84%)
20:50:47:WU02:FS01:0x21:Completed 1700000 out of 2000000 steps (85%)
20:52:03:WU02:FS01:0x21:Completed 1720000 out of 2000000 steps (86%)
20:53:18:WU02:FS01:0x21:Completed 1740000 out of 2000000 steps (87%)
20:54:33:WU02:FS01:0x21:Completed 1760000 out of 2000000 steps (88%)
20:55:49:WU02:FS01:0x21:Completed 1780000 out of 2000000 steps (89%)
20:57:04:WU02:FS01:0x21:Completed 1800000 out of 2000000 steps (90%)
20:58:19:WU02:FS01:0x21:Completed 1820000 out of 2000000 steps (91%)
20:59:35:WU02:FS01:0x21:Completed 1840000 out of 2000000 steps (92%)
21:00:51:WU02:FS01:0x21:Completed 1860000 out of 2000000 steps (93%)
21:02:06:WU02:FS01:0x21:Completed 1880000 out of 2000000 steps (94%)
******************************* Date: 2019-09-22 *******************************
21:03:22:WU02:FS01:0x21:Completed 1900000 out of 2000000 steps (95%)
21:04:38:WU02:FS01:0x21:Completed 1920000 out of 2000000 steps (96%)
21:05:56:WU02:FS01:0x21:Completed 1940000 out of 2000000 steps (97%)
21:07:11:WU02:FS01:0x21:Completed 1960000 out of 2000000 steps (98%)
21:08:26:WU02:FS01:0x21:Completed 1980000 out of 2000000 steps (99%)
21:09:42:WU02:FS01:0x21:Completed 2000000 out of 2000000 steps (100%)
21:09:50:WU02:FS01:0x21:Saving result file logfile_01.txt
21:09:50:WU02:FS01:0x21:Saving result file badstate-0.xml
21:09:51:WU02:FS01:0x21:Saving result file badstate-1.xml
21:09:52:WU02:FS01:0x21:Saving result file checkpointState.xml
21:09:53:WU02:FS01:0x21:Saving result file checkpt.crc
21:09:53:WU02:FS01:0x21:Saving result file log.txt
21:09:53:WU02:FS01:0x21:Saving result file positions.xtc
21:09:57:WU02:FS01:0x21:Folding@home Core Shutdown: FINISHED_UNIT
21:09:57:WU02:FS01:FahCore returned: FINISHED_UNIT (100 = 0x64)
21:09:57:WU02:FS01:Sending unit results: id:02 state:SEND error:NO_ERROR project:14225 run:48 clone:1 gen:24 core:0x21 unit:0x0000002480fccb0a5d6545dbf5b1623c
21:09:57:WU02:FS01:Uploading 160.33MiB to 128.252.203.10
21:09:57:WU02:FS01:Connecting to 128.252.203.10:8080
21:10:17:WU02:FS01:Upload 0.12%
21:10:17:WARNING:WU02:FS01:Exception: Failed to send results to work server: Transfer failed
21:10:17:WU02:FS01:Trying to send results to collection server
21:10:17:WU02:FS01:Uploading 160.33MiB to 155.247.166.219
21:10:17:WU02:FS01:Connecting to 155.247.166.219:8080
21:10:17:ERROR:WU02:FS01:Exception: Transfer failed

Re: Project: 14225 (Run 48, Clone 1, Gen 24)

Posted: Sun Sep 29, 2019 3:24 am
by bruce
There's something strange going on with the analysis being computed for that WU. Notice that the upload is 160.33MiB. That's a log bigger that it should be, though I can't explain it. I suspect that the server is rejecting it as corrupt. Personally, I'd delete it and move on.

The same WU was assigned to someone else who was able to upload it and the server classified his result as FAULTY.

I wish the server issued a message telling you that the WU was faulty and removed it from your upload queue.