Folding Forum

Posted: **Mon Apr 27, 2020 3:32 pm**

Hello Folks,

At time: 14:37:14 GMT apparently received a job failure for Project 14437.

This GPU job started around 1009 PM CDT on Sunday, 26 April or 0309 GMT on the 27th. It was a big one, with estimated points I would have earned over 190,000 (forgot the exact number, but it was a good number). The 14437 job ran for a good 11 1/2 hours on my rather new GPU card, ending around 1435 GMT or 935 AM CDT both on April 27th. On attempting to upload the results, it appeared to be having trouble contacting the server at 1435 GMT (in addition to first unsuccessful attempts to download a new GPU job). A minute later it tried uploading the results, only to come with with an apparent failure at 14:37:14. I may have not caught this problem, but seeing the number of points that was ahead, decided to wait out the last 10 minutes of processing and see how it went. And caught it in the act of nil credit, thus this note.

Is there anyway that I could get some credit for the amount of time, electricity, etc. spent for this one? Losing +190,000 points is not nice after putting in all of that work. Is it possible that this job was bad/contaminated -- that yielded a nil outcome?

Another item that could provide more hints. For the failed GPU job 14437, I tried looking at it on the FAH Viewer option under the FAH Control window. It would not display any of the view picture what so ever for 14437. Both jobs running here now, the CPU: 14379 (2246, 3, 41) and GPU: 11763 (0, 8210, 34) both display molecules in motion. And despite explanations made earlier; now see why you award GPU projects much higher credit, !many! more molecules to work on.

Under "Expert" settings, I have open-web-control, value=true and next-unit-percentage, value=100. If there is anything else I can do from my end, please let me know.

Thanks,
Robby

Code: Select all

02:31:47:WU03:FS01:0x22:Completed 4700000 out of 5000000 steps (94%)
02:36:56:WU03:FS01:0x22:Completed 4750000 out of 5000000 steps (95%)
02:40:50:WU03:FS01:0x22:Completed 4800000 out of 5000000 steps (96%)
02:44:46:WU03:FS01:0x22:Completed 4850000 out of 5000000 steps (97%)
02:49:04:WU03:FS01:0x22:Completed 4900000 out of 5000000 steps (98%)
02:52:16:WU03:FS01:0x22:Completed 4950000 out of 5000000 steps (99%)
02:56:44:WU03:FS01:0x22:Completed 5000000 out of 5000000 steps (100%)
02:57:09:WU00:FS01:Connecting to 65.254.110.245:80
02:57:18:WARNING:WU00:FS01:Failed to get assignment from '65.254.110.245:80': No WUs available for this configuration
02:57:22:WU00:FS01:Connecting to 18.218.241.186:80
02:58:00:WARNING:WU00:FS01:Failed to get assignment from '18.218.241.186:80': No WUs available for this configuration
02:58:03:WU00:FS01:Connecting to 65.254.110.245:80
02:58:16:WARNING:WU00:FS01:Failed to get assignment from '65.254.110.245:80': No WUs available for this configuration
02:58:29:WU00:FS01:Connecting to 18.218.241.186:80
02:58:56:WU03:FS01:0x22:Saving result file ..\logfile_01.txt
02:58:56:WU03:FS01:0x22:Saving result file checkpointState.xml
02:59:19:WU03:FS01:0x22:Saving result file checkpt.crc
02:59:24:WU03:FS01:0x22:Saving result file positions.xtc
02:59:48:WU03:FS01:0x22:Saving result file science.log
02:59:52:WARNING:WU00:FS01:Failed to get assignment from '18.218.241.186:80': No WUs available for this configuration
02:59:53:ERROR:WU00:FS01:Exception: Could not get an assignment
03:00:01:WU00:FS01:Connecting to 65.254.110.245:80
03:00:02:WU03:FS01:0x22:Folding@home Core Shutdown: FINISHED_UNIT
03:00:18:WARNING:WU00:FS01:Failed to get assignment from '65.254.110.245:80': No WUs available for this configuration
03:00:19:WU00:FS01:Connecting to 18.218.241.186:80
03:00:20:WU03:FS01:FahCore returned: FINISHED_UNIT (100 = 0x64)
03:00:23:WU03:FS01:Sending unit results: id:03 state:SEND error:NO_ERROR project:14545 run:0 clone:1469 gen:20 core:0x22 unit:0x000000220d5262775e80e22486dbbc14
03:00:37:WU03:FS01:Uploading 26.21MiB to 13.82.98.119
03:00:37:WU03:FS01:Connecting to 13.82.98.119:8080
03:00:43:WU03:FS01:Upload 0.48%
03:00:43:WARNING:WU00:FS01:Failed to get assignment from '18.218.241.186:80': No WUs available for this configuration
03:00:43:WU00:FS01:Connecting to 65.254.110.245:80
03:00:56:WU03:FS01:Upload 2.15%
03:01:01:WARNING:WU00:FS01:Failed to get assignment from '65.254.110.245:80': No WUs available for this configuration
03:01:01:WU00:FS01:Connecting to 18.218.241.186:80
03:01:02:WU03:FS01:Upload 12.88%
03:01:14:WU03:FS01:Upload 15.02%
03:01:16:WARNING:WU00:FS01:Failed to get assignment from '18.218.241.186:80': No WUs available for this configuration
03:01:16:ERROR:WU00:FS01:Exception: Could not get an assignment
03:01:25:WU00:FS01:Connecting to 65.254.110.245:80
03:01:25:WU03:FS01:Upload 24.56%
03:01:31:WU03:FS01:Upload 32.19%
03:01:31:WARNING:WU00:FS01:Failed to get assignment from '65.254.110.245:80': No WUs available for this configuration
03:01:32:WU00:FS01:Connecting to 18.218.241.186:80
03:01:39:WU03:FS01:Upload 37.20%
03:01:45:WU03:FS01:Upload 38.87%
03:01:45:WARNING:WU00:FS01:Failed to get assignment from '18.218.241.186:80': No WUs available for this configuration
03:01:48:WU00:FS01:Connecting to 65.254.110.245:80
03:01:52:WU03:FS01:Upload 45.30%
03:01:58:WU03:FS01:Upload 45.54%
03:02:02:WU00:FS01:Assigned to work server 128.252.203.10
03:02:05:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:TU116 [GeForce GTX 1660 SUPER] from 128.252.203.10
03:02:05:WU00:FS01:Connecting to 128.252.203.10:8080
03:02:05:WU03:FS01:Upload 50.79%
03:02:12:WU03:FS01:Upload 51.50%
03:02:18:WU03:FS01:Upload 51.74%
03:02:24:WU03:FS01:Upload 53.89%
03:02:37:WU03:FS01:Upload 55.80%
03:02:43:WU03:FS01:Upload 56.03%
03:02:43:ERROR:WU00:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
03:02:56:WU03:FS01:Upload 62.95%
03:03:02:WU03:FS01:Upload 63.19%
03:03:05:WU00:FS01:Connecting to 65.254.110.245:80
03:03:10:WU03:FS01:Upload 67.48%
03:03:18:WU03:FS01:Upload 71.53%
03:03:19:WU00:FS01:Assigned to work server 3.133.76.19
03:03:31:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:TU116 [GeForce GTX 1660 SUPER] from 3.133.76.19
03:03:31:WU00:FS01:Connecting to 3.133.76.19:8080
03:03:31:WU03:FS01:Upload 80.36%
03:03:37:WU03:FS01:Upload 82.03%
03:03:45:WU03:FS01:Upload 84.17%
03:04:01:WU03:FS01:Upload 90.61%
03:04:14:WU03:FS01:Upload 94.42%
03:04:14:WU00:FS01:Downloading 20.24MiB
03:04:28:WU00:FS01:Download 10.19%
03:04:29:WU03:FS01:Upload complete
03:04:31:WU03:FS01:Server responded WORK_ACK (400)
03:04:31:WU03:FS01:Final credit estimate, 73229.00 points
03:04:35:WU03:FS01:Cleaning up
03:04:35:WU00:FS01:Download 58.97%
03:04:41:WU00:FS01:Download 59.59%
03:04:43:WU00:FS01:Download complete
03:06:14:WU00:FS01:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:14437 run:593 clone:3 gen:3 core:0x22 unit:0x0000000603854c135e9d2263183d8aeb
03:06:21:WU00:FS01:Starting
03:06:22:WU00:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\Users\bobb1\AppData\Roaming\FAHClient\cores/cores.foldingathome.org/v7/win/64bit/Core_22.fah/FahCore_22.exe -dir 00 -suffix 01 -version 706 -lifeline 7248 -checkpoint 15 -gpu-vendor nvidia -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu 0
03:06:36:WU00:FS01:Started FahCore on PID 20724
03:07:57:WU00:FS01:Core PID:20832
03:07:57:WU00:FS01:FahCore 0x22 started
03:09:43:WU00:FS01:0x22:*********************** Log Started 2020-04-27T03:09:39Z ***********************
03:09:43:WU00:FS01:0x22:*************************** Core22 Folding@home Core ***************************
03:09:43:WU00:FS01:0x22:       Type: 0x22
03:09:46:WU00:FS01:0x22:       Core: Core22
03:09:46:WU00:FS01:0x22:    Website: https://foldingathome.org/
03:09:46:WU00:FS01:0x22:  Copyright: (c) 2009-2018 foldingathome.org
03:09:46:WU00:FS01:0x22:     Author: John Chodera <john.chodera@choderalab.org> and Rafal Wiewiora
03:09:46:WU00:FS01:0x22:             <rafal.wiewiora@choderalab.org>
03:09:46:WU00:FS01:0x22:       Args: -dir 00 -suffix 01 -version 706 -lifeline 20724 -checkpoint 15
03:09:46:WU00:FS01:0x22:             -gpu-vendor nvidia -opencl-platform 0 -opencl-device 0 -cuda-device
03:09:46:WU00:FS01:0x22:             0 -gpu 0
03:09:46:WU00:FS01:0x22:     Config: <none>
03:09:46:WU00:FS01:0x22:************************************ Build *************************************
03:09:46:WU00:FS01:0x22:    Version: 0.0.2
03:09:46:WU00:FS01:0x22:       Date: Dec 6 2019
03:09:46:WU00:FS01:0x22:       Time: 21:30:31
03:09:46:WU00:FS01:0x22: Repository: Git
03:09:46:WU00:FS01:0x22:   Revision: abeb39247cc72df5af0f63723edafadb23d5dfbe
03:09:46:WU00:FS01:0x22:     Branch: HEAD
03:09:46:WU00:FS01:0x22:   Compiler: Visual C++ 2008
03:09:46:WU00:FS01:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
03:09:46:WU00:FS01:0x22:   Platform: win32 10
03:09:46:WU00:FS01:0x22:       Bits: 64
03:09:46:WU00:FS01:0x22:       Mode: Release
03:09:46:WU00:FS01:0x22:************************************ System ************************************
03:09:46:WU00:FS01:0x22:        CPU: Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
03:09:46:WU00:FS01:0x22:     CPU ID: GenuineIntel Family 6 Model 60 Stepping 3
03:09:46:WU00:FS01:0x22:       CPUs: 8
03:09:46:WU00:FS01:0x22:     Memory: 15.88GiB
03:09:46:WU00:FS01:0x22:Free Memory: 10.73GiB
03:09:46:WU00:FS01:0x22:    Threads: WINDOWS_THREADS
03:09:46:WU00:FS01:0x22: OS Version: 6.2
03:09:46:WU00:FS01:0x22:Has Battery: false
03:09:46:WU00:FS01:0x22: On Battery: false
03:09:46:WU00:FS01:0x22: UTC Offset: -5
03:09:46:WU00:FS01:0x22:        PID: 20832
03:09:46:WU00:FS01:0x22:        CWD: C:\Users\bobb1\AppData\Roaming\FAHClient\work
03:09:46:WU00:FS01:0x22:         OS: Windows 10 Home
03:09:46:WU00:FS01:0x22:    OS Arch: AMD64
03:09:46:WU00:FS01:0x22:********************************************************************************
03:09:51:WU00:FS01:0x22:Project: 14437 (Run 593, Clone 3, Gen 3)
03:09:51:WU00:FS01:0x22:Unit: 0x0000000603854c135e9d2263183d8aeb
03:09:51:WU00:FS01:0x22:Reading tar file core.xml
03:09:51:WU00:FS01:0x22:Reading tar file integrator.xml
03:09:51:WU00:FS01:0x22:Reading tar file state.xml
03:11:33:WU00:FS01:0x22:Reading tar file system.xml
03:14:06:WU00:FS01:0x22:Digital signatures verified
03:14:07:WU00:FS01:0x22:Folding@home GPU Core22 Folding@home Core
03:14:07:WU00:FS01:0x22:Version 0.0.2
03:23:07:WU00:FS01:0x22:Completed 0 out of 2000000 steps (0%)
03:23:10:WU00:FS01:0x22:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
03:28:30:WU00:FS01:0x22:Completed 20000 out of 2000000 steps (1%)
03:39:03:WU00:FS01:0x22:Completed 40000 out of 2000000 steps (2%)
03:46:19:WU00:FS01:0x22:Completed 60000 out of 2000000 steps (3%)
03:56:31:WU00:FS01:0x22:Completed 80000 out of 2000000 steps (4%)
04:05:57:WU00:FS01:0x22:Completed 100000 out of 2000000 steps (5%)
04:11:09:WU00:FS01:0x22:Completed 120000 out of 2000000 steps (6%)
04:16:22:WU00:FS01:0x22:Completed 140000 out of 2000000 steps (7%)
04:21:34:WU00:FS01:0x22:Completed 160000 out of 2000000 steps (8%)
04:26:40:WU00:FS01:0x22:Completed 180000 out of 2000000 steps (9%)
04:31:42:WU00:FS01:0x22:Completed 200000 out of 2000000 steps (10%)
04:36:45:WU00:FS01:0x22:Completed 220000 out of 2000000 steps (11%)
04:41:50:WU00:FS01:0x22:Completed 240000 out of 2000000 steps (12%)
04:46:50:WU00:FS01:0x22:Completed 260000 out of 2000000 steps (13%)
04:51:48:WU00:FS01:0x22:Completed 280000 out of 2000000 steps (14%)
04:56:46:WU00:FS01:0x22:Completed 300000 out of 2000000 steps (15%)
05:06:30:WU00:FS01:0x22:Completed 320000 out of 2000000 steps (16%)
05:14:13:WU00:FS01:0x22:Completed 340000 out of 2000000 steps (17%)
05:23:26:WU00:FS01:0x22:Completed 360000 out of 2000000 steps (18%)
05:32:04:WU00:FS01:0x22:Completed 380000 out of 2000000 steps (19%)
05:37:05:WU00:FS01:0x22:Completed 400000 out of 2000000 steps (20%)
05:42:05:WU00:FS01:0x22:Completed 420000 out of 2000000 steps (21%)
05:47:04:WU00:FS01:0x22:Completed 440000 out of 2000000 steps (22%)
05:52:02:WU00:FS01:0x22:Completed 460000 out of 2000000 steps (23%)
05:57:00:WU00:FS01:0x22:Completed 480000 out of 2000000 steps (24%)
06:01:59:WU00:FS01:0x22:Completed 500000 out of 2000000 steps (25%)
06:06:57:WU00:FS01:0x22:Completed 520000 out of 2000000 steps (26%)
06:11:58:WU00:FS01:0x22:Completed 540000 out of 2000000 steps (27%)
06:17:01:WU00:FS01:0x22:Completed 560000 out of 2000000 steps (28%)
06:21:59:WU00:FS01:0x22:Completed 580000 out of 2000000 steps (29%)
******************************* Date: 2020-04-27 *******************************
06:26:59:WU00:FS01:0x22:Completed 600000 out of 2000000 steps (30%)
06:31:59:WU00:FS01:0x22:Completed 620000 out of 2000000 steps (31%)
06:37:01:WU00:FS01:0x22:Completed 640000 out of 2000000 steps (32%)
06:42:01:WU00:FS01:0x22:Completed 660000 out of 2000000 steps (33%)
06:47:01:WU00:FS01:0x22:Completed 680000 out of 2000000 steps (34%)
06:52:03:WU00:FS01:0x22:Completed 700000 out of 2000000 steps (35%)
06:57:04:WU00:FS01:0x22:Completed 720000 out of 2000000 steps (36%)
07:02:11:WU00:FS01:0x22:Completed 740000 out of 2000000 steps (37%)
07:07:27:WU00:FS01:0x22:Completed 760000 out of 2000000 steps (38%)
07:12:42:WU00:FS01:0x22:Completed 780000 out of 2000000 steps (39%)
07:17:58:WU00:FS01:0x22:Completed 800000 out of 2000000 steps (40%)
07:23:11:WU00:FS01:0x22:Completed 820000 out of 2000000 steps (41%)
07:28:22:WU00:FS01:0x22:Completed 840000 out of 2000000 steps (42%)
07:33:28:WU00:FS01:0x22:Completed 860000 out of 2000000 steps (43%)
07:38:40:WU00:FS01:0x22:Completed 880000 out of 2000000 steps (44%)
07:43:59:WU00:FS01:0x22:Completed 900000 out of 2000000 steps (45%)
07:53:06:WU00:FS01:0x22:Completed 920000 out of 2000000 steps (46%)
08:02:16:WU00:FS01:0x22:Completed 940000 out of 2000000 steps (47%)
08:10:35:WU00:FS01:0x22:Completed 960000 out of 2000000 steps (48%)
08:19:05:WU00:FS01:0x22:Completed 980000 out of 2000000 steps (49%)
08:24:12:WARNING:WU00:FS01:Detected clock skew (1 mins 03 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
08:28:42:WU00:FS01:0x22:Completed 1000000 out of 2000000 steps (50%)
08:37:33:WU00:FS01:0x22:Completed 1020000 out of 2000000 steps (51%)
08:45:01:WU00:FS01:0x22:Completed 1040000 out of 2000000 steps (52%)
08:53:34:WU00:FS01:0x22:Completed 1060000 out of 2000000 steps (53%)
09:02:40:WU00:FS01:0x22:Completed 1080000 out of 2000000 steps (54%)
09:11:02:WU00:FS01:0x22:Completed 1100000 out of 2000000 steps (55%)
09:19:57:WU00:FS01:0x22:Completed 1120000 out of 2000000 steps (56%)
09:29:44:WU00:FS01:0x22:Completed 1140000 out of 2000000 steps (57%)
09:37:59:WU00:FS01:0x22:Completed 1160000 out of 2000000 steps (58%)
09:45:35:WU00:FS01:0x22:Completed 1180000 out of 2000000 steps (59%)
09:55:01:WU00:FS01:0x22:Completed 1200000 out of 2000000 steps (60%)
10:03:41:WU00:FS01:0x22:Completed 1220000 out of 2000000 steps (61%)
10:12:23:WU00:FS01:0x22:Completed 1240000 out of 2000000 steps (62%)
10:21:04:WU00:FS01:0x22:Completed 1260000 out of 2000000 steps (63%)
10:29:59:WU00:FS01:0x22:Completed 1280000 out of 2000000 steps (64%)
10:39:06:WU00:FS01:0x22:Completed 1300000 out of 2000000 steps (65%)
10:45:38:WU00:FS01:0x22:Completed 1320000 out of 2000000 steps (66%)
10:52:13:WU00:FS01:0x22:Completed 1340000 out of 2000000 steps (67%)
10:57:17:WU00:FS01:0x22:Completed 1360000 out of 2000000 steps (68%)
11:02:22:WU00:FS01:0x22:Completed 1380000 out of 2000000 steps (69%)
11:08:25:WARNING:WU00:FS01:Detected clock skew (1 mins 41 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
11:10:38:WU00:FS01:0x22:Completed 1400000 out of 2000000 steps (70%)
11:10:50:WARNING:WU00:FS01:Detected clock skew (2 mins 16 secs), I/O delay, laptop hibernation or other slowdown noted, adjusting time estimates
11:18:37:WU00:FS01:0x22:Completed 1420000 out of 2000000 steps (71%)
11:27:26:WU00:FS01:0x22:Completed 1440000 out of 2000000 steps (72%)
11:36:19:WU00:FS01:0x22:Completed 1460000 out of 2000000 steps (73%)
11:43:30:WU00:FS01:0x22:Completed 1480000 out of 2000000 steps (74%)
11:49:25:WU00:FS01:0x22:Completed 1500000 out of 2000000 steps (75%)
11:55:31:WU00:FS01:0x22:Completed 1520000 out of 2000000 steps (76%)
12:02:04:WU00:FS01:0x22:Completed 1540000 out of 2000000 steps (77%)
12:11:21:WU00:FS01:0x22:Completed 1560000 out of 2000000 steps (78%)
12:19:36:WU00:FS01:0x22:Completed 1580000 out of 2000000 steps (79%)
12:26:14:WU00:FS01:0x22:Completed 1600000 out of 2000000 steps (80%)
******************************* Date: 2020-04-27 *******************************
12:33:51:WU00:FS01:0x22:Completed 1620000 out of 2000000 steps (81%)
12:43:12:WU00:FS01:0x22:Completed 1640000 out of 2000000 steps (82%)
12:51:20:WU00:FS01:0x22:Completed 1660000 out of 2000000 steps (83%)
13:00:54:WU00:FS01:0x22:Completed 1680000 out of 2000000 steps (84%)
13:09:11:WU00:FS01:0x22:Completed 1700000 out of 2000000 steps (85%)
13:16:02:WU00:FS01:0x22:Completed 1720000 out of 2000000 steps (86%)
13:23:30:WU00:FS01:0x22:Completed 1740000 out of 2000000 steps (87%)
13:29:35:WU00:FS01:0x22:Completed 1760000 out of 2000000 steps (88%)
13:37:26:WU00:FS01:0x22:Completed 1780000 out of 2000000 steps (89%)
13:44:43:WU00:FS01:0x22:Completed 1800000 out of 2000000 steps (90%)
13:49:46:WU00:FS01:0x22:Completed 1820000 out of 2000000 steps (91%)
13:54:51:WU00:FS01:0x22:Completed 1840000 out of 2000000 steps (92%)
13:59:57:WU00:FS01:0x22:Completed 1860000 out of 2000000 steps (93%)
14:05:00:WU00:FS01:0x22:Completed 1880000 out of 2000000 steps (94%)
14:10:04:WU00:FS01:0x22:Completed 1900000 out of 2000000 steps (95%)
14:15:10:WU00:FS01:0x22:Completed 1920000 out of 2000000 steps (96%)
14:20:11:WU00:FS01:0x22:Completed 1940000 out of 2000000 steps (97%)
14:25:05:WU00:FS01:0x22:Completed 1960000 out of 2000000 steps (98%)
14:30:08:WU00:FS01:0x22:Completed 1980000 out of 2000000 steps (99%)
14:35:08:WU00:FS01:0x22:Completed 2000000 out of 2000000 steps (100%)
14:35:09:WU02:FS01:Connecting to 65.254.110.245:80
14:35:09:WARNING:WU02:FS01:Failed to get assignment from '65.254.110.245:80': No WUs available for this configuration
14:35:09:WU02:FS01:Connecting to 18.218.241.186:80
14:35:10:WU02:FS01:Assigned to work server 140.163.4.231
14:35:10:WU02:FS01:Requesting new work unit for slot 01: RUNNING gpu:0:TU116 [GeForce GTX 1660 SUPER] from 140.163.4.231
14:35:10:WU02:FS01:Connecting to 140.163.4.231:8080
14:35:19:WU00:FS01:0x22:Saving result file ..\logfile_01.txt
14:35:19:WU00:FS01:0x22:Saving result file checkpointState.xml
14:35:22:WU00:FS01:0x22:Saving result file checkpt.crc
14:35:22:WU00:FS01:0x22:Saving result file positions.xtc
14:35:28:WU00:FS01:0x22:Saving result file science.log
14:35:28:WU00:FS01:0x22:Folding@home Core Shutdown: FINISHED_UNIT
14:35:29:WU00:FS01:FahCore returned: FINISHED_UNIT (100 = 0x64)
14:35:29:WU00:FS01:Sending unit results: id:00 state:SEND error:NO_ERROR project:14437 run:593 clone:3 gen:3 core:0x22 unit:0x0000000603854c135e9d2263183d8aeb
14:35:30:WU00:FS01:Uploading 93.04MiB to 3.133.76.19
14:35:30:WU00:FS01:Connecting to 3.133.76.19:8080
14:35:51:WARNING:WU00:FS01:WorkServer connection failed on port 8080 trying 80
14:35:51:WU00:FS01:Connecting to 3.133.76.19:80
14:36:12:WARNING:WU00:FS01:Exception: Failed to send results to work server: Failed to connect to 3.133.76.19:80: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
14:36:12:WU00:FS01:Trying to send results to collection server
14:36:12:WU00:FS01:Uploading 93.04MiB to 3.21.157.11
14:36:12:WU00:FS01:Connecting to 3.21.157.11:8080
14:36:18:WU00:FS01:Upload 7.93%
14:36:24:WU00:FS01:Upload 17.87%
14:36:30:WU00:FS01:Upload 27.68%
14:36:36:WU00:FS01:Upload 37.15%
14:36:42:WU00:FS01:Upload 47.02%
14:36:48:WU00:FS01:Upload 56.49%
14:36:50:ERROR:WU02:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
14:36:50:WU02:FS01:Connecting to 65.254.110.245:80
14:36:50:WARNING:WU02:FS01:Failed to get assignment from '65.254.110.245:80': No WUs available for this configuration
14:36:50:WU02:FS01:Connecting to 18.218.241.186:80
14:36:51:WARNING:WU02:FS01:Failed to get assignment from '18.218.241.186:80': No WUs available for this configuration
14:36:51:WU02:FS01:Connecting to 65.254.110.245:80
14:36:51:WARNING:WU02:FS01:Failed to get assignment from '65.254.110.245:80': No WUs available for this configuration
14:36:51:WU02:FS01:Connecting to 18.218.241.186:80
14:36:51:WARNING:WU02:FS01:Failed to get assignment from '18.218.241.186:80': No WUs available for this configuration
14:36:51:ERROR:WU02:FS01:Exception: Could not get an assignment
14:36:54:WU00:FS01:Upload 66.44%
14:37:00:WU00:FS01:Upload 76.44%
14:37:06:WU00:FS01:Upload 86.39%
14:37:12:WU00:FS01:Upload 96.33%
14:37:14:WU00:FS01:Upload complete
14:37:14:WU00:FS01:Server responded WORK_QUIT (404)
14:37:14:WARNING:WU00:FS01:Server did not like results, dumping
14:37:15:WU00:FS01:Cleaning up
14:37:50:WU02:FS01:Connecting to 65.254.110.245:80
14:37:50:WARNING:WU02:FS01:Failed to get assignment from '65.254.110.245:80': No WUs available for this configuration
14:37:50:WU02:FS01:Connecting to 18.218.241.186:80
14:37:50:WARNING:WU02:FS01:Failed to get assignment from '18.218.241.186:80': No WUs available for this configuration
14:37:50:WU02:FS01:Connecting to 65.254.110.245:80
14:37:51:WU02:FS01:Assigned to work server 13.82.98.119
14:37:51:WU02:FS01:Requesting new work unit for slot 01: READY gpu:0:TU116 [GeForce GTX 1660 SUPER] from 13.82.98.119
14:37:51:WU02:FS01:Connecting to 13.82.98.119:8080
14:38:12:WARNING:WU02:FS01:WorkServer connection failed on port 8080 trying 80
14:38:12:WU02:FS01:Connecting to 13.82.98.119:80
14:39:21:ERROR:WU02:FS01:Exception: 10002: Received short response, expected 512 bytes, got 0
14:39:27:WU02:FS01:Connecting to 65.254.110.245:80
14:39:28:WARNING:WU02:FS01:Failed to get assignment from '65.254.110.245:80': No WUs available for this configuration
14:39:28:WU02:FS01:Connecting to 18.218.241.186:80
14:39:28:WARNING:WU02:FS01:Failed to get assignment from '18.218.241.186:80': No WUs available for this configuration
14:39:28:WU02:FS01:Connecting to 65.254.110.245:80
14:39:28:WARNING:WU02:FS01:Failed to get assignment from '65.254.110.245:80': No WUs available for this configuration
14:39:28:WU02:FS01:Connecting to 18.218.241.186:80
14:39:29:WU02:FS01:Assigned to work server 128.252.203.10
14:39:29:WU02:FS01:Requesting new work unit for slot 01: READY gpu:0:TU116 [GeForce GTX 1660 SUPER] from 128.252.203.10
14:39:29:WU02:FS01:Connecting to 128.252.203.10:8080
14:40:29:WU02:FS01:Downloading 51.20MiB
14:40:35:WU02:FS01:Download 83.14%
14:40:36:WU02:FS01:Download complete
14:40:36:WU02:FS01:Received Unit: id:02 state:DOWNLOAD error:NO_ERROR project:11763 run:0 clone:8210 gen:34 core:0x22 unit:0x0000004a80fccb0a5e71135d4eb99ce8
14:40:37:WU02:FS01:Starting
14:40:37:WU02:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\Users\bobb1\AppData\Roaming\FAHClient\cores/cores.foldingathome.org/v7/win/64bit/Core_22.fah/FahCore_22.exe -dir 02 -suffix 01 -version 706 -lifeline 7248 -checkpoint 15 -gpu-vendor nvidia -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu 0
14:40:37:WU02:FS01:Started FahCore on PID 9856
14:40:37:WU02:FS01:Core PID:18728
14:40:37:WU02:FS01:FahCore 0x22 started
14:40:37:WU02:FS01:0x22:*********************** Log Started 2020-04-27T14:40:37Z ***********************
14:40:37:WU02:FS01:0x22:*************************** Core22 Folding@home Core ***************************
14:40:37:WU02:FS01:0x22:       Type: 0x22
14:40:37:WU02:FS01:0x22:       Core: Core22
14:40:37:WU02:FS01:0x22:    Website: https://foldingathome.org/
14:40:37:WU02:FS01:0x22:  Copyright: (c) 2009-2018 foldingathome.org
14:40:37:WU02:FS01:0x22:     Author: John Chodera <john.chodera@choderalab.org> and Rafal Wiewiora
14:40:37:WU02:FS01:0x22:             <rafal.wiewiora@choderalab.org>
14:40:37:WU02:FS01:0x22:       Args: -dir 02 -suffix 01 -version 706 -lifeline 9856 -checkpoint 15
14:40:37:WU02:FS01:0x22:             -gpu-vendor nvidia -opencl-platform 0 -opencl-device 0 -cuda-device
14:40:37:WU02:FS01:0x22:             0 -gpu 0
14:40:37:WU02:FS01:0x22:     Config: <none>
14:40:37:WU02:FS01:0x22:************************************ Build *************************************
14:40:37:WU02:FS01:0x22:    Version: 0.0.2
14:40:37:WU02:FS01:0x22:       Date: Dec 6 2019
14:40:37:WU02:FS01:0x22:       Time: 21:30:31
14:40:37:WU02:FS01:0x22: Repository: Git
14:40:37:WU02:FS01:0x22:   Revision: abeb39247cc72df5af0f63723edafadb23d5dfbe
14:40:37:WU02:FS01:0x22:     Branch: HEAD
14:40:37:WU02:FS01:0x22:   Compiler: Visual C++ 2008
14:40:37:WU02:FS01:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
14:40:37:WU02:FS01:0x22:   Platform: win32 10
14:40:37:WU02:FS01:0x22:       Bits: 64
14:40:37:WU02:FS01:0x22:       Mode: Release
14:40:37:WU02:FS01:0x22:************************************ System ************************************
14:40:37:WU02:FS01:0x22:        CPU: Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
14:40:37:WU02:FS01:0x22:     CPU ID: GenuineIntel Family 6 Model 60 Stepping 3
14:40:37:WU02:FS01:0x22:       CPUs: 8
14:40:37:WU02:FS01:0x22:     Memory: 15.88GiB
14:40:37:WU02:FS01:0x22:Free Memory: 10.06GiB
14:40:37:WU02:FS01:0x22:    Threads: WINDOWS_THREADS
14:40:37:WU02:FS01:0x22: OS Version: 6.2
14:40:37:WU02:FS01:0x22:Has Battery: false
14:40:37:WU02:FS01:0x22: On Battery: false
14:40:37:WU02:FS01:0x22: UTC Offset: -5
14:40:37:WU02:FS01:0x22:        PID: 18728
14:40:37:WU02:FS01:0x22:        CWD: C:\Users\bobb1\AppData\Roaming\FAHClient\work
14:40:37:WU02:FS01:0x22:         OS: Windows 10 Home
14:40:37:WU02:FS01:0x22:    OS Arch: AMD64
14:40:37:WU02:FS01:0x22:********************************************************************************
14:40:37:WU02:FS01:0x22:Project: 11763 (Run 0, Clone 8210, Gen 34)
14:40:37:WU02:FS01:0x22:Unit: 0x0000004a80fccb0a5e71135d4eb99ce8
14:40:37:WU02:FS01:0x22:Reading tar file core.xml
14:40:37:WU02:FS01:0x22:Reading tar file integrator.xml
14:40:37:WU02:FS01:0x22:Reading tar file state.xml
14:40:37:WU02:FS01:0x22:Reading tar file system.xml
14:40:38:WU02:FS01:0x22:Digital signatures verified
14:40:38:WU02:FS01:0x22:Folding@home GPU Core22 Folding@home Core
14:40:38:WU02:FS01:0x22:Version 0.0.2
14:40:55:WU02:FS01:0x22:Completed 0 out of 1000000 steps (0%)
14:40:55:WU02:FS01:0x22:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
14:42:19:WU02:FS01:0x22:Completed 10000 out of 1000000 steps (1%)
14:43:42:WU02:FS01:0x22:Completed 20000 out of 1000000 steps (2%)
14:45:06:WU02:FS01:0x22:Completed 30000 out of 1000000 steps (3%)

Posted: **Mon Apr 27, 2020 4:08 pm**

The WU failed validation checks on upload to the server and so the server dumped the WU - Points are only awarded for validated WUs ... This happens occasionally and there are a number of potential causes which I'll let someone with better knowledge of the system explain but as I understand it things like small instabilities in the graphics card, borderline bad WU, antivirus/router/network issues, possibly even issues at the receiving server end - but the upshot is when this happens it is unfortunately "nil points"

Posted: **Mon Apr 27, 2020 7:00 pm**

Hello Neil,

Thanks for your reply. I sure would like to learn of more explanations on why this sometimes occurs. Am leaning towards that the 14437 WU was bad. I assume that the labs do some QC on what they want us to work on before releasing it to us users. I would suggest a "consolation" rewarding of points, especially if it ls likely the work unit was bad to begin with.

Back in mid March when the COVID-19 issue start becoming a big news item and a HUGE uptick in new FAH users, I got a bum CPU job that was going to take something like 10 days to complete and I'd get only 100 points some something like that. It took over a week to get my systems (this desktop and a laptop stabilized), as after deleting the data and reinstalling the older client repeatably and not getting any jobs (not knowing of the influx of new users) and shortage of normally plentiful work units. That WU was determined to be bad and was flushed from the system. viewtopic.php?f=19&t=32394

The two jobs in my first post that were running both finished successfully by the early afternoon CDT, with 56773 points from GPU run 11763, and 3255 points from the CPU run 14379. I have not been monitoring job completions and/or failures, but will start doing so more frequently. If there is a way for the admin staff to let us know of excessive failures with our systems, there could be a problem on our end.

I am not overclocking my CPU nor GPU. The GPU was added about a month ago to my system. Except for the CPU going on 6 years of age and still going strong, the computer and networking (LAN connected directly into a AT&T router) seems to be working fine. Am using AVAST for the anti-virus software! And speaking of which, I just excluded the FAHClient directory under my "roaming" directory from virus scans in Windows 10. Is that proper, or do I just exclude the data directory? Before then, I have never seen AVAST! report on FAH being contaminated...

Thanks,
Robby

Mod Edit: Fixed URL - PantherX

Posted: **Mon Apr 27, 2020 7:16 pm**

Could it be a server issue? Same server as these > viewtopic.php?f=19&t=34865

Posted: **Mon Apr 27, 2020 8:53 pm**

Thanks for suggesting a possible server issue.

My system is on a UPS/battery backup, so unlikely too low voltage input. Am not near the machine now, but the UPS usually displays ~122V to 124V; could not tell you if that is in or out of the system.

Thanks,
Robby

Posted: **Tue Apr 28, 2020 3:37 am**

Robby_Firefox wrote:...Is there anyway that I could get some credit for the amount of time, electricity, etc. spent for this one? Losing +190,000 points is not nice after putting in all of that work. Is it possible that this job was bad/contaminated -- that yielded a nil outcome?...

Unfortunately, there's no points for lost WU. Points are a measure of scientific contribution. If you didn't contribute to science, you can't be awarded points for it

Robby_Firefox wrote:...I sure would like to learn of more explanations on why this sometimes occurs. Am leaning towards that the 14437 WU was bad I assume that the labs do some QC on what they want us to work on before releasing it to us users....

The reason bad WUs exist is because of the statistical nature of the simulation. In this case, the WU might be bad: https://apps.foldingathome.org/wu#proje ... ne=3&gen=3
You can read more about the process here for WU QA: viewtopic.php?f=19&t=16526

Robby_Firefox wrote:...I would suggest a "consolation" rewarding of points, especially if it ls likely the work unit was bad to begin with. ..

I get your intention but since it is impossible to say which WU will be bad and which will be good, it can't happen. Plus, the points align with the amount of scientific work done. If no scientific work was done, how can you be rewarded points?

Robby_Firefox wrote:...I am not overclocking my CPU nor GPU. The GPU was added about a month ago to my system. Except for the CPU going on 6 years of age and still going strong, the computer and networking (LAN connected directly into a AT&T router) seems to be working fine...

Now a days, companies can sell you factory overclocked GPUs which later turn out to be unstable for folding. Thus, even on "stock" frequencies, issues can happen.

Robby_Firefox wrote:...Am using AVAST for the anti-virus software! And speaking of which, I just excluded the FAHClient directory under my "roaming" directory from virus scans in Windows 10. Is that proper, or do I just exclude the data directory? Before then, I have never seen AVAST! report on FAH being contaminated...

Yep, excluding this directory is fine:
C:\Users\bobb1\AppData\Roaming\FAHClient

You can also exclude that directory from Windows Security too.

Posted: **Tue Apr 28, 2020 8:45 am**

Robby_Firefox wrote:I assume that the labs do some QC on what they want us to work on before releasing it to us users.

The following trivialises the complexity of the process but I hope shows there is "some QC" in the release process ...

The Researchers who create the projects test them on their own rigs - but obviously these have a limited amount of variance in setup compared to the pool of folding kit they are about to be released to - at this stage they are probably more concerned with is the Project meeting the scientific needs and doing the calculations they want it to but there will no doubt be an amount of QC'ing as far as does the Project run stable.

Projects are then released to the FAH Beta Team which gets first chance to "break them" in the wild … Beta Team members accept that new Projects can/do have issues and report back their experiences with the Beta Projects allowing for issues to be corrected/avoided … Obviously the longer a Project is on test the more chance there is of spotting issues … Two things that can impair this process is when folders who are not Beta Team members use the Beta flag - which means any errors are not reported on and cannot be resolved/mitigated and means that Beta Team members may not actually get any of the Beta Project WUs because others are grabbing them - and when for scientifically urgent reasons some Projects are rushed through Beta to Advanced or even Full Release with the Researchers knowing there may have to be adjustments "on the fly" … The first of these would be improved if folders who are not Beta Team members would cease to use the Beta flag - The second is a fact of life in that sometimes the science needs to be rushed.

Once the Researchers are happy the Project has been tuned (if necessary) to a good enough level it is released to Advanced … These are pre Full Release so have a higher level of risk than Full Release but less than Beta and allow a much greater mass testing of the Project which the Researchers monitor closely … Finally Projects are set to Full Release.

The Researchers do have stats available on completed/failed and many other factors … They are the ones who make the call as to what is an acceptable failure rate dependant upon many factors … but generally they will not want high failure rates as that means they aren't getting as much science done.

Projects are adjusted or even withdrawn from release at any of these stages if issues start to arise that have too severe an impact.

Posted: **Wed Apr 29, 2020 5:15 am**

Hi PantherX and Neil,

Thanks a bunch for your replies and feedback with the issues and questions I posed. Learned a lot more things behind the scenes at FAH. Have been busy the past day or so, but getting back to the main machine and monitoring shows that all appears to be working overall OK at this time. None of the recent jobs have failed. Easier to look at them in the main FAH controller (where they can be split) rather than the older logs, which mix both CPU and GPU results.

Cheers to all!!
Robby

Folding Forum

Project: 14437 (Run 593, Clone 3, Gen 3)

Project: 14437 (Run 593, Clone 3, Gen 3)

Re: Project: 14437 (Run 593, Clone 3, Gen 3)

Re: Project: 14437 (Run 593, Clone 3, Gen 3)

Re: Project: 14437 (Run 593, Clone 3, Gen 3)

Re: Project: 14437 (Run 593, Clone 3, Gen 3)

Re: Project: 14437 (Run 593, Clone 3, Gen 3)

Re: Project: 14437 (Run 593, Clone 3, Gen 3)

Re: Project: 14437 (Run 593, Clone 3, Gen 3)