bad WU 9430 (125, 5, 100) and 9641 (0, 28, 54)

Moderators: Site Moderators, FAHC Science Team

Post Reply
rpmouton
Posts: 40
Joined: Mon Jun 23, 2008 1:09 pm
Hardware configuration: 1-MSI 990FXA-GD65V2 AM3+, AMD FX-8120 8-Core Black Edition-3.1 GHz, Mushkin Enhanced Blackline 8GB (2 x 4GB) 1600 MHz and ASUS GeForce GTX 550 Ti. Win 7 64x, 7.1x client with SMP and GPU slots ~14k ppd

2-ASUS M2NE-SLI AM2, AMD Phenom 4 @ 2.3 GHZ, 4 GB @ 800 MHz and ASUS GeForce GTX 550 Ti. Win Vista 64x, 7.1x client with SMP and GPU slots ~10k ppd

3-MSI 785GTM-E45 AM2+, AMD Phenom 4 Propus @ 3 GHZ, 4GB @ 800 MHZ, Win 7 64x, 7.1x client with SMP slots ~4k ppd

4-DELL 2950 Gen III, 2 Xeon E5405 Quad core @ 2GHz, 8 GB @ 669MHz, Ubuntu 12.04, 7.1 client with one SMP slot (bigadv) ~12k ppd
Location: Orlando, Florida

bad WU 9430 (125, 5, 100) and 9641 (0, 28, 54)

Post by rpmouton »

Been having a few connection issues on my end lately but this doesn't seem to be the issue here..

One WU wont go the the servers and the other keeps restarting after bad state detected..

UPDATE: after a reboot the second WU finished successfully..

here is just the warning portion of my log:

Code: Select all

*********************** Log Started 2015-11-08T23:53:22Z ***********************
23:53:43:WARNING:WU00:FS01:WorkServer connection failed on port 8080 trying 80
23:54:04:WARNING:WU00:FS01:Exception: Failed to send results to work server: Failed to connect to 171.64.65.104:80: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
23:54:26:WARNING:WU00:FS01:WorkServer connection failed on port 8080 trying 80
23:56:09:WARNING:WU00:FS01:Failed to send results, will try again later
23:57:52:WARNING:WU00:FS01:Server did not like results, dumping
******************************* Date: 2015-11-09 *******************************
09:25:55:WARNING:WU00:FS01:Exception: Could not get IP address for assign-GPU.stanford.edu: No such host is known. 
09:25:55:ERROR:WU00:FS01:Exception: Could not get an assignment
09:25:55:WARNING:WU00:FS01:Exception: Could not get IP address for assign-GPU.stanford.edu: No such host is known. 
09:25:55:ERROR:WU00:FS01:Exception: Could not get an assignment
09:26:55:WARNING:WU00:FS01:Exception: Could not get IP address for assign-GPU.stanford.edu: No such host is known. 
09:26:55:ERROR:WU00:FS01:Exception: Could not get an assignment
09:28:33:WARNING:WU00:FS01:Exception: Could not get IP address for assign-GPU.stanford.edu: No such host is known. 
09:28:33:ERROR:WU00:FS01:Exception: Could not get an assignment
09:31:10:WARNING:WU00:FS01:Exception: Could not get IP address for assign-GPU.stanford.edu: No such host is known. 
09:31:10:ERROR:WU00:FS01:Exception: Could not get an assignment
09:32:20:WARNING:WU02:FS01:WorkServer connection failed on port 8080 trying 80
09:32:41:WARNING:WU02:FS01:Exception: Failed to send results to work server: Failed to connect to 171.64.65.56:80: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
09:33:02:WARNING:WU02:FS01:WorkServer connection failed on port 8080 trying 80
09:35:35:WARNING:WU00:FS01:Exception: Could not get IP address for assign-GPU.stanford.edu: No such host is known. 
09:35:35:ERROR:WU00:FS01:Exception: Could not get an assignment
******************************* Date: 2015-11-09 *******************************
13:11:10:WU00:FS01:0x21:WARNING:Console control signal 1 on PID 1796
here is the complete log:

Code: Select all

*********************** Log Started 2015-11-08T23:53:22Z ***********************
23:53:22:************************* Folding@home Client *************************
23:53:22:      Website: http://folding.stanford.edu/
23:53:22:    Copyright: (c) 2009-2014 Stanford University
23:53:22:       Author: Joseph Coffland <joseph@cauldrondevelopment.com>
23:53:22:         Args: 
23:53:22:       Config: C:/ProgramData/FAHClient/config.xml
23:53:22:******************************** Build ********************************
23:53:22:      Version: 7.4.4
23:53:22:         Date: Mar 4 2014
23:53:22:         Time: 20:26:54
23:53:22:      SVN Rev: 4130
23:53:22:       Branch: fah/trunk/client
23:53:22:     Compiler: Intel(R) C++ MSVC 1500 mode 1200
23:53:22:      Options: /TP /nologo /EHa /Qdiag-disable:4297,4103,1786,279 /Ox -arch:SSE
23:53:22:               /QaxSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 /Qopenmp /Qrestrict /MT /Qmkl
23:53:22:     Platform: win32 XP
23:53:22:         Bits: 32
23:53:22:         Mode: Release
23:53:22:******************************* System ********************************
23:53:22:          CPU: AMD FX(tm)-4350 Quad-Core Processor
23:53:22:       CPU ID: AuthenticAMD Family 21 Model 2 Stepping 0
23:53:22:         CPUs: 4
23:53:22:       Memory: 7.96GiB
23:53:22:  Free Memory: 6.27GiB
23:53:22:      Threads: WINDOWS_THREADS
23:53:22:   OS Version: 6.2
23:53:22:  Has Battery: true
23:53:22:   On Battery: false
23:53:22:   UTC Offset: -5
23:53:22:          PID: 6644
23:53:22:          CWD: C:/ProgramData/FAHClient
23:53:22:           OS: Windows 10 Home
23:53:22:      OS Arch: AMD64
23:53:22:         GPUs: 1
23:53:22:        GPU 0: NVIDIA:5 GM204 [GeForce GTX 970]
23:53:22:         CUDA: 5.2
23:53:22:  CUDA Driver: 7050
23:53:22:Win32 Service: false
23:53:22:***********************************************************************
23:53:22:<config>
23:53:22:  <!-- Network -->
23:53:22:  <proxy v=':8080'/>
23:53:22:
23:53:22:  <!-- Slot Control -->
23:53:22:  <power v='FULL'/>
23:53:22:
23:53:22:  <!-- User Information -->
23:53:22:  <passkey v='********************************'/>
23:53:22:  <team v='229370'/>
23:53:22:  <user v='RPMouton'/>
23:53:22:
23:53:22:  <!-- Folding Slots -->
23:53:22:  <slot id='1' type='GPU'>
23:53:22:    <client-type v='advanced'/>
23:53:22:  </slot>
23:53:22:</config>
23:53:22:Trying to access database...
23:53:22:Successfully acquired database lock
23:53:22:Enabled folding slot 01: READY gpu:0:GM204 [GeForce GTX 970]
23:53:22:WU00:FS01:Sending unit results: id:00 state:SEND error:NO_ERROR project:9209 run:0 clone:38 gen:21 core:0x21 unit:0x0000002c664f2dd055edef618f9158aa
23:53:22:WU00:FS01:Uploading 17.50MiB to 171.64.65.104
23:53:22:WU02:FS01:Starting
23:53:22:WU00:FS01:Connecting to 171.64.65.104:8080
23:53:22:WU02:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/ProgramData/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_18.fah/FahCore_18.exe -dir 02 -suffix 01 -version 704 -lifeline 6644 -checkpoint 15 -gpu 0 -gpu-vendor nvidia
23:53:22:WU02:FS01:Started FahCore on PID 7044
23:53:23:WU02:FS01:Core PID:7068
23:53:23:WU02:FS01:FahCore 0x18 started
23:53:23:WU02:FS01:0x18:*********************** Log Started 2015-11-08T23:53:23Z ***********************
23:53:23:WU02:FS01:0x18:Project: 9430 (Run 125, Clone 5, Gen 100)
23:53:23:WU02:FS01:0x18:Unit: 0x00000075ab40413855474dd0b8b52503
23:53:23:WU02:FS01:0x18:CPU: 0x00000000000000000000000000000000
23:53:23:WU02:FS01:0x18:Machine: 1
23:53:23:WU02:FS01:0x18:Digital signatures verified
23:53:23:WU02:FS01:0x18:Folding@home GPU core18
23:53:23:WU02:FS01:0x18:Version 0.0.4
23:53:23:WU02:FS01:0x18:  Found a checkpoint file
23:53:34:WU02:FS01:0x18:Completed 350000 out of 16000000 steps (2%)
23:53:34:WU02:FS01:0x18:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
23:53:43:WARNING:WU00:FS01:WorkServer connection failed on port 8080 trying 80
23:53:43:WU00:FS01:Connecting to 171.64.65.104:80
23:54:04:WARNING:WU00:FS01:Exception: Failed to send results to work server: Failed to connect to 171.64.65.104:80: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
23:54:05:WU00:FS01:Trying to send results to collection server
23:54:05:WU00:FS01:Uploading 17.50MiB to 171.65.103.160
23:54:05:WU00:FS01:Connecting to 171.65.103.160:8080
23:54:26:WARNING:WU00:FS01:WorkServer connection failed on port 8080 trying 80
23:54:26:WU00:FS01:Connecting to 171.65.103.160:80
23:54:26:WU00:FS01:Upload 0.36%
23:54:32:WU00:FS01:Upload 6.79%
23:54:38:WU00:FS01:Upload 11.79%
23:54:44:WU00:FS01:Upload 17.50%
23:54:50:WU00:FS01:Upload 23.22%
23:54:56:WU00:FS01:Upload 28.93%
23:55:02:WU00:FS01:Upload 35.01%
23:55:08:WU00:FS01:Upload 40.37%
23:55:14:WU00:FS01:Upload 46.08%
23:55:20:WU00:FS01:Upload 52.15%
23:55:26:WU00:FS01:Upload 57.87%
23:55:32:WU00:FS01:Upload 63.94%
23:55:38:WU00:FS01:Upload 69.66%
23:55:44:WU00:FS01:Upload 75.73%
23:55:50:WU00:FS01:Upload 81.44%
23:55:56:WU00:FS01:Upload 87.16%
23:56:02:WU00:FS01:Upload 93.23%
23:56:08:WU00:FS01:Upload 98.95%
23:56:09:WU00:FS01:Upload complete
23:56:09:WU00:FS01:Server responded PLEASE_WAIT (464)
23:56:09:WARNING:WU00:FS01:Failed to send results, will try again later
23:56:10:WU00:FS01:Sending unit results: id:00 state:SEND error:NO_ERROR project:9209 run:0 clone:38 gen:21 core:0x21 unit:0x0000002c664f2dd055edef618f9158aa
23:56:10:WU00:FS01:Uploading 17.50MiB to 171.64.65.104
23:56:10:WU00:FS01:Connecting to 171.64.65.104:8080
23:56:16:WU00:FS01:Upload 6.07%
23:56:22:WU00:FS01:Upload 12.15%
23:56:28:WU00:FS01:Upload 17.86%
23:56:34:WU00:FS01:Upload 23.93%
23:56:40:WU00:FS01:Upload 30.01%
23:56:46:WU00:FS01:Upload 35.72%
23:56:52:WU00:FS01:Upload 41.79%
23:56:58:WU00:FS01:Upload 47.51%
23:57:04:WU00:FS01:Upload 53.58%
23:57:10:WU00:FS01:Upload 59.30%
23:57:16:WU00:FS01:Upload 65.37%
23:57:22:WU00:FS01:Upload 71.09%
23:57:28:WU00:FS01:Upload 77.16%
23:57:34:WU00:FS01:Upload 82.87%
23:57:40:WU00:FS01:Upload 88.95%
23:57:46:WU00:FS01:Upload 94.66%
23:57:52:WU00:FS01:Upload complete
23:57:52:WU00:FS01:Server responded GOT_ALREADY (434)
23:57:52:WARNING:WU00:FS01:Server did not like results, dumping
23:57:52:WU00:FS01:Cleaning up
23:58:25:WU02:FS01:0x18:Completed 480000 out of 16000000 steps (3%)
00:04:18:WU02:FS01:0x18:Completed 640000 out of 16000000 steps (4%)
00:10:11:WU02:FS01:0x18:Completed 800000 out of 16000000 steps (5%)
00:16:10:WU02:FS01:0x18:Completed 960000 out of 16000000 steps (6%)
00:22:03:WU02:FS01:0x18:Completed 1120000 out of 16000000 steps (7%)
00:27:56:WU02:FS01:0x18:Completed 1280000 out of 16000000 steps (8%)
00:33:50:WU02:FS01:0x18:Completed 1440000 out of 16000000 steps (9%)
00:39:43:WU02:FS01:0x18:Completed 1600000 out of 16000000 steps (10%)
00:45:41:WU02:FS01:0x18:Completed 1760000 out of 16000000 steps (11%)
00:51:35:WU02:FS01:0x18:Completed 1920000 out of 16000000 steps (12%)
00:57:28:WU02:FS01:0x18:Completed 2080000 out of 16000000 steps (13%)
01:03:22:WU02:FS01:0x18:Completed 2240000 out of 16000000 steps (14%)
01:09:16:WU02:FS01:0x18:Completed 2400000 out of 16000000 steps (15%)
01:15:14:WU02:FS01:0x18:Completed 2560000 out of 16000000 steps (16%)
01:21:07:WU02:FS01:0x18:Completed 2720000 out of 16000000 steps (17%)
01:27:01:WU02:FS01:0x18:Completed 2880000 out of 16000000 steps (18%)
01:32:54:WU02:FS01:0x18:Completed 3040000 out of 16000000 steps (19%)
01:38:48:WU02:FS01:0x18:Completed 3200000 out of 16000000 steps (20%)
01:44:46:WU02:FS01:0x18:Completed 3360000 out of 16000000 steps (21%)
01:50:39:WU02:FS01:0x18:Completed 3520000 out of 16000000 steps (22%)
01:56:33:WU02:FS01:0x18:Completed 3680000 out of 16000000 steps (23%)
02:02:27:WU02:FS01:0x18:Completed 3840000 out of 16000000 steps (24%)
02:08:20:WU02:FS01:0x18:Completed 4000000 out of 16000000 steps (25%)
02:14:19:WU02:FS01:0x18:Completed 4160000 out of 16000000 steps (26%)
02:20:12:WU02:FS01:0x18:Completed 4320000 out of 16000000 steps (27%)
02:26:06:WU02:FS01:0x18:Completed 4480000 out of 16000000 steps (28%)
02:32:00:WU02:FS01:0x18:Completed 4640000 out of 16000000 steps (29%)
02:37:53:WU02:FS01:0x18:Completed 4800000 out of 16000000 steps (30%)
02:43:52:WU02:FS01:0x18:Completed 4960000 out of 16000000 steps (31%)
02:49:45:WU02:FS01:0x18:Completed 5120000 out of 16000000 steps (32%)
02:55:39:WU02:FS01:0x18:Completed 5280000 out of 16000000 steps (33%)
03:01:33:WU02:FS01:0x18:Completed 5440000 out of 16000000 steps (34%)
03:07:26:WU02:FS01:0x18:Completed 5600000 out of 16000000 steps (35%)
03:13:25:WU02:FS01:0x18:Completed 5760000 out of 16000000 steps (36%)
03:19:18:WU02:FS01:0x18:Completed 5920000 out of 16000000 steps (37%)
03:25:12:WU02:FS01:0x18:Completed 6080000 out of 16000000 steps (38%)
03:31:06:WU02:FS01:0x18:Completed 6240000 out of 16000000 steps (39%)
03:37:00:WU02:FS01:0x18:Completed 6400000 out of 16000000 steps (40%)
03:42:58:WU02:FS01:0x18:Completed 6560000 out of 16000000 steps (41%)
03:48:52:WU02:FS01:0x18:Completed 6720000 out of 16000000 steps (42%)
03:54:46:WU02:FS01:0x18:Completed 6880000 out of 16000000 steps (43%)
04:00:40:WU02:FS01:0x18:Completed 7040000 out of 16000000 steps (44%)
04:06:33:WU02:FS01:0x18:Completed 7200000 out of 16000000 steps (45%)
04:12:31:WU02:FS01:0x18:Completed 7360000 out of 16000000 steps (46%)
04:18:25:WU02:FS01:0x18:Completed 7520000 out of 16000000 steps (47%)
04:24:19:WU02:FS01:0x18:Completed 7680000 out of 16000000 steps (48%)
04:30:13:WU02:FS01:0x18:Completed 7840000 out of 16000000 steps (49%)
04:36:06:WU02:FS01:0x18:Completed 8000000 out of 16000000 steps (50%)
04:42:05:WU02:FS01:0x18:Completed 8160000 out of 16000000 steps (51%)
04:47:58:WU02:FS01:0x18:Completed 8320000 out of 16000000 steps (52%)
04:53:52:WU02:FS01:0x18:Completed 8480000 out of 16000000 steps (53%)
04:59:46:WU02:FS01:0x18:Completed 8640000 out of 16000000 steps (54%)
05:05:40:WU02:FS01:0x18:Completed 8800000 out of 16000000 steps (55%)
05:11:38:WU02:FS01:0x18:Completed 8960000 out of 16000000 steps (56%)
05:17:32:WU02:FS01:0x18:Completed 9120000 out of 16000000 steps (57%)
05:23:26:WU02:FS01:0x18:Completed 9280000 out of 16000000 steps (58%)
05:29:20:WU02:FS01:0x18:Completed 9440000 out of 16000000 steps (59%)
05:35:14:WU02:FS01:0x18:Completed 9600000 out of 16000000 steps (60%)
05:41:12:WU02:FS01:0x18:Completed 9760000 out of 16000000 steps (61%)
05:47:06:WU02:FS01:0x18:Completed 9920000 out of 16000000 steps (62%)
05:53:00:WU02:FS01:0x18:Completed 10080000 out of 16000000 steps (63%)
******************************* Date: 2015-11-09 *******************************
05:58:54:WU02:FS01:0x18:Completed 10240000 out of 16000000 steps (64%)
06:04:48:WU02:FS01:0x18:Completed 10400000 out of 16000000 steps (65%)
06:10:47:WU02:FS01:0x18:Completed 10560000 out of 16000000 steps (66%)
06:16:40:WU02:FS01:0x18:Completed 10720000 out of 16000000 steps (67%)
06:22:34:WU02:FS01:0x18:Completed 10880000 out of 16000000 steps (68%)
06:28:28:WU02:FS01:0x18:Completed 11040000 out of 16000000 steps (69%)
06:34:22:WU02:FS01:0x18:Completed 11200000 out of 16000000 steps (70%)
06:40:21:WU02:FS01:0x18:Completed 11360000 out of 16000000 steps (71%)
06:46:15:WU02:FS01:0x18:Completed 11520000 out of 16000000 steps (72%)
06:52:09:WU02:FS01:0x18:Completed 11680000 out of 16000000 steps (73%)
06:58:03:WU02:FS01:0x18:Completed 11840000 out of 16000000 steps (74%)
07:03:57:WU02:FS01:0x18:Completed 12000000 out of 16000000 steps (75%)
07:09:55:WU02:FS01:0x18:Completed 12160000 out of 16000000 steps (76%)
07:15:49:WU02:FS01:0x18:Completed 12320000 out of 16000000 steps (77%)
07:21:43:WU02:FS01:0x18:Completed 12480000 out of 16000000 steps (78%)
07:27:37:WU02:FS01:0x18:Completed 12640000 out of 16000000 steps (79%)
07:33:30:WU02:FS01:0x18:Completed 12800000 out of 16000000 steps (80%)
07:39:29:WU02:FS01:0x18:Completed 12960000 out of 16000000 steps (81%)
07:45:23:WU02:FS01:0x18:Completed 13120000 out of 16000000 steps (82%)
07:51:17:WU02:FS01:0x18:Completed 13280000 out of 16000000 steps (83%)
07:57:11:WU02:FS01:0x18:Completed 13440000 out of 16000000 steps (84%)
08:03:05:WU02:FS01:0x18:Completed 13600000 out of 16000000 steps (85%)
08:09:04:WU02:FS01:0x18:Completed 13760000 out of 16000000 steps (86%)
08:14:57:WU02:FS01:0x18:Completed 13920000 out of 16000000 steps (87%)
08:20:51:WU02:FS01:0x18:Completed 14080000 out of 16000000 steps (88%)
08:26:45:WU02:FS01:0x18:Completed 14240000 out of 16000000 steps (89%)
08:32:39:WU02:FS01:0x18:Completed 14400000 out of 16000000 steps (90%)
08:38:38:WU02:FS01:0x18:Completed 14560000 out of 16000000 steps (91%)
08:44:32:WU02:FS01:0x18:Completed 14720000 out of 16000000 steps (92%)
08:50:26:WU02:FS01:0x18:Completed 14880000 out of 16000000 steps (93%)
08:56:20:WU02:FS01:0x18:Completed 15040000 out of 16000000 steps (94%)
09:02:14:WU02:FS01:0x18:Completed 15200000 out of 16000000 steps (95%)
09:08:13:WU02:FS01:0x18:Completed 15360000 out of 16000000 steps (96%)
09:14:06:WU02:FS01:0x18:Completed 15520000 out of 16000000 steps (97%)
09:20:00:WU02:FS01:0x18:Completed 15680000 out of 16000000 steps (98%)
09:25:54:WU02:FS01:0x18:Completed 15840000 out of 16000000 steps (99%)
09:25:55:WARNING:WU00:FS01:Exception: Could not get IP address for assign-GPU.stanford.edu: No such host is known. 
09:25:55:ERROR:WU00:FS01:Exception: Could not get an assignment
09:25:55:WARNING:WU00:FS01:Exception: Could not get IP address for assign-GPU.stanford.edu: No such host is known. 
09:25:55:ERROR:WU00:FS01:Exception: Could not get an assignment
09:26:55:WARNING:WU00:FS01:Exception: Could not get IP address for assign-GPU.stanford.edu: No such host is known. 
09:26:55:ERROR:WU00:FS01:Exception: Could not get an assignment
09:28:33:WARNING:WU00:FS01:Exception: Could not get IP address for assign-GPU.stanford.edu: No such host is known. 
09:28:33:ERROR:WU00:FS01:Exception: Could not get an assignment
09:31:10:WARNING:WU00:FS01:Exception: Could not get IP address for assign-GPU.stanford.edu: No such host is known. 
09:31:10:ERROR:WU00:FS01:Exception: Could not get an assignment
09:31:48:WU02:FS01:0x18:Completed 16000000 out of 16000000 steps (100%)
09:31:53:WU02:FS01:0x18:Saving result file logfile_01.txt
09:31:53:WU02:FS01:0x18:Saving result file checkpointState.xml
09:31:54:WU02:FS01:0x18:Saving result file checkpt.crc
09:31:54:WU02:FS01:0x18:Saving result file log.txt
09:31:54:WU02:FS01:0x18:Saving result file positions.xtc
09:31:59:WU02:FS01:0x18:Folding@home Core Shutdown: FINISHED_UNIT
09:31:59:WU02:FS01:FahCore returned: FINISHED_UNIT (100 = 0x64)
09:31:59:WU02:FS01:Sending unit results: id:02 state:SEND error:NO_ERROR project:9430 run:125 clone:5 gen:100 core:0x18 unit:0x00000075ab40413855474dd0b8b52503
09:31:59:WU02:FS01:Uploading 24.11MiB to 171.64.65.56
09:31:59:WU02:FS01:Connecting to 171.64.65.56:8080
09:32:20:WARNING:WU02:FS01:WorkServer connection failed on port 8080 trying 80
09:32:20:WU02:FS01:Connecting to 171.64.65.56:80
09:32:41:WARNING:WU02:FS01:Exception: Failed to send results to work server: Failed to connect to 171.64.65.56:80: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
09:32:41:WU02:FS01:Trying to send results to collection server
09:32:41:WU02:FS01:Uploading 24.11MiB to 171.65.103.160
09:32:41:WU02:FS01:Connecting to 171.65.103.160:8080
09:33:02:WARNING:WU02:FS01:WorkServer connection failed on port 8080 trying 80
09:33:02:WU02:FS01:Connecting to 171.65.103.160:80
09:33:12:WU02:FS01:Upload 0.26%
09:33:18:WU02:FS01:Upload 4.15%
09:33:24:WU02:FS01:Upload 8.04%
09:33:30:WU02:FS01:Upload 12.18%
09:33:36:WU02:FS01:Upload 16.33%
09:33:42:WU02:FS01:Upload 20.48%
09:33:48:WU02:FS01:Upload 24.62%
09:33:54:WU02:FS01:Upload 28.77%
09:34:00:WU02:FS01:Upload 32.92%
09:34:06:WU02:FS01:Upload 37.32%
09:34:12:WU02:FS01:Upload 41.47%
09:34:18:WU02:FS01:Upload 45.62%
09:34:24:WU02:FS01:Upload 50.03%
09:34:30:WU02:FS01:Upload 54.17%
09:34:36:WU02:FS01:Upload 58.58%
09:34:42:WU02:FS01:Upload 62.73%
09:34:48:WU02:FS01:Upload 67.13%
09:34:54:WU02:FS01:Upload 71.28%
09:35:00:WU02:FS01:Upload 75.43%
09:35:06:WU02:FS01:Upload 79.83%
09:35:12:WU02:FS01:Upload 83.98%
09:35:18:WU02:FS01:Upload 88.39%
09:35:24:WU02:FS01:Upload 92.53%
09:35:30:WU02:FS01:Upload 96.68%
09:35:35:WARNING:WU00:FS01:Exception: Could not get IP address for assign-GPU.stanford.edu: No such host is known. 
09:35:35:ERROR:WU00:FS01:Exception: Could not get an assignment
09:42:16:WU00:FS01:Connecting to 171.67.108.45:80
09:42:16:WU00:FS01:Assigned to work server 171.67.108.155
09:42:16:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:GM204 [GeForce GTX 970] from 171.67.108.155
09:42:16:WU00:FS01:Connecting to 171.67.108.155:8080
09:42:17:WU00:FS01:Downloading 51.59MiB
09:42:23:WU00:FS01:Download 8.12%
09:42:29:WU00:FS01:Download 17.93%
09:42:35:WU00:FS01:Download 28.59%
09:42:41:WU00:FS01:Download 39.13%
09:42:47:WU00:FS01:Download 49.43%
09:42:53:WU00:FS01:Download 59.97%
09:42:59:WU00:FS01:Download 70.02%
09:43:05:WU00:FS01:Download 79.47%
09:43:11:WU00:FS01:Download 90.13%
09:43:16:WU00:FS01:Download complete
09:43:16:WU00:FS01:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:9641 run:0 clone:28 gen:54 core:0x21 unit:0x00000041ab436c9b5609bee4fc6cd172
09:43:16:WU00:FS01:Starting
09:43:16:WU00:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/ProgramData/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_21.fah/FahCore_21.exe -dir 00 -suffix 01 -version 704 -lifeline 6644 -checkpoint 15 -gpu 0 -gpu-vendor nvidia
09:43:16:WU00:FS01:Started FahCore on PID 1516
09:43:16:WU00:FS01:Core PID:1796
09:43:16:WU00:FS01:FahCore 0x21 started
09:43:17:WU00:FS01:0x21:*********************** Log Started 2015-11-09T09:43:16Z ***********************
09:43:17:WU00:FS01:0x21:Project: 9641 (Run 0, Clone 28, Gen 54)
09:43:17:WU00:FS01:0x21:Unit: 0x00000041ab436c9b5609bee4fc6cd172
09:43:17:WU00:FS01:0x21:CPU: 0x00000000000000000000000000000000
09:43:17:WU00:FS01:0x21:Machine: 1
09:43:17:WU00:FS01:0x21:Reading tar file core.xml
09:43:17:WU00:FS01:0x21:Reading tar file integrator.xml
09:43:17:WU00:FS01:0x21:Reading tar file state.xml
09:43:17:WU00:FS01:0x21:Reading tar file system.xml
09:43:18:WU00:FS01:0x21:Digital signatures verified
09:43:18:WU00:FS01:0x21:Folding@home GPU Core21 Folding@home Core
09:43:18:WU00:FS01:0x21:Version 0.0.12
09:44:07:WU00:FS01:0x21:Completed 0 out of 2000000 steps (0%)
09:44:07:WU00:FS01:0x21:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
09:46:25:WU00:FS01:0x21:Completed 20000 out of 2000000 steps (1%)
09:48:29:WU00:FS01:0x21:Completed 40000 out of 2000000 steps (2%)
09:50:36:WU00:FS01:0x21:Completed 60000 out of 2000000 steps (3%)
09:52:46:WU00:FS01:0x21:Completed 80000 out of 2000000 steps (4%)
09:54:54:WU00:FS01:0x21:Completed 100000 out of 2000000 steps (5%)
09:57:18:WU00:FS01:0x21:Completed 120000 out of 2000000 steps (6%)
09:59:29:WU00:FS01:0x21:Completed 140000 out of 2000000 steps (7%)
10:01:41:WU00:FS01:0x21:Completed 160000 out of 2000000 steps (8%)
10:03:52:WU00:FS01:0x21:Completed 180000 out of 2000000 steps (9%)
10:06:01:WU00:FS01:0x21:Completed 200000 out of 2000000 steps (10%)
10:08:24:WU00:FS01:0x21:Completed 220000 out of 2000000 steps (11%)
10:10:33:WU00:FS01:0x21:Completed 240000 out of 2000000 steps (12%)
10:12:42:WU00:FS01:0x21:Completed 260000 out of 2000000 steps (13%)
10:14:53:WU00:FS01:0x21:Completed 280000 out of 2000000 steps (14%)
10:17:07:WU00:FS01:0x21:Completed 300000 out of 2000000 steps (15%)
10:19:35:WU00:FS01:0x21:Completed 320000 out of 2000000 steps (16%)
10:21:46:WU00:FS01:0x21:Completed 340000 out of 2000000 steps (17%)
10:23:58:WU00:FS01:0x21:Completed 360000 out of 2000000 steps (18%)
10:26:08:WU00:FS01:0x21:Completed 380000 out of 2000000 steps (19%)
10:28:19:WU00:FS01:0x21:Completed 400000 out of 2000000 steps (20%)
10:30:43:WU00:FS01:0x21:Completed 420000 out of 2000000 steps (21%)
10:32:55:WU00:FS01:0x21:Completed 440000 out of 2000000 steps (22%)
10:35:05:WU00:FS01:0x21:Completed 460000 out of 2000000 steps (23%)
10:37:17:WU00:FS01:0x21:Completed 480000 out of 2000000 steps (24%)
10:39:26:WU00:FS01:0x21:Completed 500000 out of 2000000 steps (25%)
10:41:51:WU00:FS01:0x21:Completed 520000 out of 2000000 steps (26%)
10:44:03:WU00:FS01:0x21:Completed 540000 out of 2000000 steps (27%)
10:46:13:WU00:FS01:0x21:Completed 560000 out of 2000000 steps (28%)
10:48:15:WU00:FS01:0x21:Completed 580000 out of 2000000 steps (29%)
10:50:09:WU00:FS01:0x21:Completed 600000 out of 2000000 steps (30%)
10:52:15:WU00:FS01:0x21:Completed 620000 out of 2000000 steps (31%)
10:54:09:WU00:FS01:0x21:Completed 640000 out of 2000000 steps (32%)
10:56:03:WU00:FS01:0x21:Completed 660000 out of 2000000 steps (33%)
10:57:57:WU00:FS01:0x21:Completed 680000 out of 2000000 steps (34%)
10:59:50:WU00:FS01:0x21:Completed 700000 out of 2000000 steps (35%)
11:01:55:WU00:FS01:0x21:Completed 720000 out of 2000000 steps (36%)
11:03:49:WU00:FS01:0x21:Completed 740000 out of 2000000 steps (37%)
11:05:43:WU00:FS01:0x21:Completed 760000 out of 2000000 steps (38%)
11:07:37:WU00:FS01:0x21:Completed 780000 out of 2000000 steps (39%)
11:09:30:WU00:FS01:0x21:Completed 800000 out of 2000000 steps (40%)
11:11:35:WU00:FS01:0x21:Completed 820000 out of 2000000 steps (41%)
11:13:30:WU00:FS01:0x21:Completed 840000 out of 2000000 steps (42%)
11:15:24:WU00:FS01:0x21:Completed 860000 out of 2000000 steps (43%)
11:17:17:WU00:FS01:0x21:Completed 880000 out of 2000000 steps (44%)
11:19:11:WU00:FS01:0x21:Completed 900000 out of 2000000 steps (45%)
11:21:16:WU00:FS01:0x21:Completed 920000 out of 2000000 steps (46%)
11:23:09:WU00:FS01:0x21:Completed 940000 out of 2000000 steps (47%)
11:25:03:WU00:FS01:0x21:Completed 960000 out of 2000000 steps (48%)
11:26:56:WU00:FS01:0x21:Completed 980000 out of 2000000 steps (49%)
11:28:49:WU00:FS01:0x21:Completed 1000000 out of 2000000 steps (50%)
11:30:54:WU00:FS01:0x21:Completed 1020000 out of 2000000 steps (51%)
11:32:48:WU00:FS01:0x21:Completed 1040000 out of 2000000 steps (52%)
11:34:42:WU00:FS01:0x21:Completed 1060000 out of 2000000 steps (53%)
11:36:35:WU00:FS01:0x21:Completed 1080000 out of 2000000 steps (54%)
11:38:28:WU00:FS01:0x21:Completed 1100000 out of 2000000 steps (55%)
11:40:33:WU00:FS01:0x21:Completed 1120000 out of 2000000 steps (56%)
11:42:27:WU00:FS01:0x21:Completed 1140000 out of 2000000 steps (57%)
11:44:20:WU00:FS01:0x21:Completed 1160000 out of 2000000 steps (58%)
11:46:14:WU00:FS01:0x21:Completed 1180000 out of 2000000 steps (59%)
11:48:07:WU00:FS01:0x21:Completed 1200000 out of 2000000 steps (60%)
11:48:16:WU00:FS01:0x21:Bad State detected... attempting to resume from last good checkpoint
11:50:09:WU00:FS01:0x21:Completed 1120000 out of 2000000 steps (56%)
11:52:03:WU00:FS01:0x21:Completed 1140000 out of 2000000 steps (57%)
11:53:56:WU00:FS01:0x21:Completed 1160000 out of 2000000 steps (58%)
11:55:49:WU00:FS01:0x21:Completed 1180000 out of 2000000 steps (59%)
11:57:42:WU00:FS01:0x21:Completed 1200000 out of 2000000 steps (60%)
******************************* Date: 2015-11-09 *******************************
11:59:47:WU00:FS01:0x21:Completed 1220000 out of 2000000 steps (61%)
12:01:41:WU00:FS01:0x21:Completed 1240000 out of 2000000 steps (62%)
12:03:34:WU00:FS01:0x21:Completed 1260000 out of 2000000 steps (63%)
12:05:27:WU00:FS01:0x21:Completed 1280000 out of 2000000 steps (64%)
12:07:20:WU00:FS01:0x21:Completed 1300000 out of 2000000 steps (65%)
12:09:25:WU00:FS01:0x21:Completed 1320000 out of 2000000 steps (66%)
12:11:19:WU00:FS01:0x21:Completed 1340000 out of 2000000 steps (67%)
12:13:13:WU00:FS01:0x21:Completed 1360000 out of 2000000 steps (68%)
12:15:06:WU00:FS01:0x21:Completed 1380000 out of 2000000 steps (69%)
12:16:59:WU00:FS01:0x21:Completed 1400000 out of 2000000 steps (70%)
12:19:04:WU00:FS01:0x21:Completed 1420000 out of 2000000 steps (71%)
12:20:58:WU00:FS01:0x21:Completed 1440000 out of 2000000 steps (72%)
12:22:52:WU00:FS01:0x21:Completed 1460000 out of 2000000 steps (73%)
12:24:45:WU00:FS01:0x21:Completed 1480000 out of 2000000 steps (74%)
12:26:38:WU00:FS01:0x21:Completed 1500000 out of 2000000 steps (75%)
12:28:43:WU00:FS01:0x21:Completed 1520000 out of 2000000 steps (76%)
12:30:36:WU00:FS01:0x21:Completed 1540000 out of 2000000 steps (77%)
12:32:30:WU00:FS01:0x21:Completed 1560000 out of 2000000 steps (78%)
12:34:23:WU00:FS01:0x21:Completed 1580000 out of 2000000 steps (79%)
12:36:16:WU00:FS01:0x21:Completed 1600000 out of 2000000 steps (80%)
12:38:20:WU00:FS01:0x21:Completed 1620000 out of 2000000 steps (81%)
12:40:14:WU00:FS01:0x21:Completed 1640000 out of 2000000 steps (82%)
12:42:07:WU00:FS01:0x21:Completed 1660000 out of 2000000 steps (83%)
12:44:00:WU00:FS01:0x21:Completed 1680000 out of 2000000 steps (84%)
12:45:54:WU00:FS01:0x21:Completed 1700000 out of 2000000 steps (85%)
12:47:58:WU00:FS01:0x21:Completed 1720000 out of 2000000 steps (86%)
12:49:52:WU00:FS01:0x21:Completed 1740000 out of 2000000 steps (87%)
12:51:45:WU00:FS01:0x21:Completed 1760000 out of 2000000 steps (88%)
12:53:39:WU00:FS01:0x21:Completed 1780000 out of 2000000 steps (89%)
12:55:32:WU00:FS01:0x21:Completed 1800000 out of 2000000 steps (90%)
12:57:37:WU00:FS01:0x21:Completed 1820000 out of 2000000 steps (91%)
12:59:30:WU00:FS01:0x21:Completed 1840000 out of 2000000 steps (92%)
13:01:35:WU00:FS01:0x21:Completed 1860000 out of 2000000 steps (93%)
13:04:20:WU00:FS01:0x21:Completed 1880000 out of 2000000 steps (94%)
13:07:00:WU00:FS01:0x21:Completed 1900000 out of 2000000 steps (95%)
13:07:21:WU00:FS01:0x21:Bad State detected... attempting to resume from last good checkpoint
13:09:57:WU00:FS01:0x21:Completed 1820000 out of 2000000 steps (91%)
13:11:10:FS01:Paused
13:11:10:FS01:Shutting core down
13:11:10:WU00:FS01:0x21:WARNING:Console control signal 1 on PID 1796
13:11:10:WU00:FS01:0x21:Exiting, please wait. . .
13:11:11:WU00:FS01:0x21:Folding@home Core Shutdown: INTERRUPTED
13:11:11:WU00:FS01:FahCore returned: INTERRUPTED (102 = 0x66)
13:11:15:FS01:Unpaused
13:11:15:WU00:FS01:Starting
13:11:15:WU00:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/ProgramData/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_21.fah/FahCore_21.exe -dir 00 -suffix 01 -version 704 -lifeline 6644 -checkpoint 15 -gpu 0 -gpu-vendor nvidia
13:11:15:WU00:FS01:Started FahCore on PID 4952
13:11:15:WU00:FS01:Core PID:652
13:11:15:WU00:FS01:FahCore 0x21 started
13:11:15:WU00:FS01:0x21:*********************** Log Started 2015-11-09T13:11:15Z ***********************
13:11:15:WU00:FS01:0x21:Project: 9641 (Run 0, Clone 28, Gen 54)
13:11:15:WU00:FS01:0x21:Unit: 0x00000041ab436c9b5609bee4fc6cd172
13:11:15:WU00:FS01:0x21:CPU: 0x00000000000000000000000000000000
13:11:15:WU00:FS01:0x21:Machine: 1
13:11:15:WU00:FS01:0x21:Digital signatures verified
13:11:15:WU00:FS01:0x21:Folding@home GPU Core21 Folding@home Core
13:11:15:WU00:FS01:0x21:Version 0.0.12
13:11:16:WU00:FS01:0x21:  Found a checkpoint file
13:12:48:WU00:FS01:0x21:Completed 1800000 out of 2000000 steps (90%)
13:12:48:WU00:FS01:0x21:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
13:15:48:WU00:FS01:0x21:Completed 1820000 out of 2000000 steps (91%)
13:18:24:WU00:FS01:0x21:Completed 1840000 out of 2000000 steps (92%)
Any suggestions to resolve these hanging WUs?

thanks
Roger
Top
58Enfield
Posts: 26
Joined: Sun Dec 02, 2007 1:35 pm
Location: Cedar Wilds of North Central Arizona

Re: bad WU 9430 (125, 5, 100) and 9641 (0, 28, 54)

Post by 58Enfield »

Two things to try based on problems I've had over the last couple of weeks. When the larger core 21 uploads arrived, my uploads would trail off between 75% to 99% and print no message in the log as to completion or not. They would show as not sent in fahcontrol so I would stop the current wu and run a send script which would show the "already have....dumping" dialog. The failure to print in the log on a vary slow or hung upload was a fah problem; but the slow upload itself was at my end.

The upload problem was a very old (prior to 2010) DNS server entry in my modem. Changing to a more up to date DNS entry solved that problem, but I still had 4 plus minute uploads with ~20mb core 21 workunits.

New modem, new data plan and a different self inflicted problem. Couldn't send to fah servers on port 80 or 8080. Turned out the port (something in the 5000 range, can't find in my notes) fah was using was included in a range of ports that I had turned off in the firewall as something I had no interest in and no reason to be allowed through the firewall. Re-enabling the range containing that port allowed uploads to proceed smoothly.

Hope that helps.
Top
bruce
Posts: 20824
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: bad WU 9430 (125, 5, 100) and 9641 (0, 28, 54)

Post by bruce »

I suspect that Stanford has been having some network problems. One of the servers I often use was giving me time-outs for long enough that I started checking what might be wrong. It did get resolved, though. and obviously might be unrelated. Ordinarily I'd blame a "no such host is known" on your local DNS but it might have been due to some kind of change made at Stanford. Anyway, assign-GPU.stanford.edu seems to be pingable now from here (though your report was from a long time ago).

The issue of "Bad State detected... attempting to resume from last good checkpoint" is a known problem and development is attempting to diagnose and fix it in a future version of FahCore_21. Originally, such an error would abort the WU giving you no credit. Now most of the work you've done up to that point is recoverable and a high percentage of WU with this type of error are completed successfully. Pausing and resuming manually after such an error probably doesn't change the outcome.

You did successfully complete that WU so we can't call it a bad WU.
Your WU (P9641 R0 C28 G54) was added to the stats database on 2015-11-09 06:08:36 for 36541.4 points of credit.

I can't explain the other WU (You did get an unusual error indication.) but you did get credit for it and the next Gen has already been completed by someone, as well.
Your WU (P9430 R125 C5 G100) was added to the stats database on 2015-11-09 04:07:37 for 105436 points of credit.
Top
Post Reply

Return to “Issues with a specific WU”