Page 1 of 1
171.67.108.52
Posted: Sat Feb 28, 2015 5:40 pm
by qqqq
I got this WU's on Windows 8.1 - 64bit, Windows 7 - 64 bit and there is no problem but on Ubuntu it slow as a hell, was about 10 minutes but it's sends on EXT4 jour sys, now on EXT3 it's even slower and in the end says about upload error to work server (work server seriously ?)... but after that it start upload to collection server.
Code: Select all
17:20:16:Saving configuration to /etc/fahclient/config.xml
17:20:28:WU01:FS01:Upload 7.46%
17:20:44:WU00:FS00:Upload 1.50%
17:20:55:WU00:FS00:Upload 2.24%
17:20:55:WU01:FS01:Upload 8.20%
17:21:07:WU01:FS01:Upload 8.95%
17:21:14:Saving configuration to /etc/fahclient/config.xml
17:21:15:WU00:FS00:Upload 2.99%
17:21:15:WU01:FS01:Upload 9.70%
17:21:30:WU00:FS00:Upload 3.74%
17:21:36:WU01:FS01:Upload 10.44%
17:22:00:WU00:FS00:Upload 4.49%
17:22:14:WU01:FS01:Upload 11.19%
17:22:20:WU00:FS00:Upload 5.23%
17:22:36:WU00:FS00:Upload 5.98%
17:22:37:WU01:FS01:Upload 11.93%
17:22:55:WU01:FS01:Upload 12.68%
17:23:09:WU00:FS00:Upload 6.73%
17:23:14:WU01:FS01:Upload 13.42%
17:23:24:WU00:FS00:Upload 7.48%
17:23:27:WU01:FS01:Upload 14.17%
17:23:41:WU01:FS01:Upload 14.92%
17:23:54:WU00:FS00:Upload 8.22%
17:24:01:WU01:FS01:Upload 15.66%
17:24:12:WU00:FS00:Upload 8.97%
17:24:22:WU01:FS01:Upload 16.41%
17:24:26:WU00:FS00:Upload 9.72%
17:24:46:WU00:FS00:Upload 10.47%
17:24:55:WU01:FS01:Upload 17.15%
17:25:01:WU00:FS00:Upload 11.21%
17:25:14:WU01:FS01:Upload 17.90%
17:25:33:WU00:FS00:Upload 11.96%
17:25:40:WU01:FS01:Upload 18.64%
17:25:50:WU00:FS00:Upload 12.71%
17:26:07:WU01:FS01:Upload 19.39%
17:26:07:WU00:FS00:Upload 13.46%
17:26:25:WU01:FS01:Upload 20.14%
17:26:38:WU01:FS01:Upload 20.88%
17:26:38:WU00:FS00:Upload 14.20%
17:26:56:WU00:FS00:Upload 14.95%
17:27:18:WU01:FS01:Upload 21.63%
17:27:25:WU00:FS00:Upload 15.70%
17:27:42:WU00:FS00:Upload 16.45%
17:27:48:WU01:FS01:Upload 22.37%
17:27:59:WU00:FS00:Upload 17.19%
17:28:18:WU01:FS01:Upload 23.12%
17:28:37:WU01:FS01:Upload 23.87%
17:28:51:WU00:FS00:Upload 17.94%
17:28:53:WU01:FS01:Upload 24.61%
17:29:09:WU00:FS00:Upload 18.69%
17:29:17:WU01:FS01:Upload 25.36%
17:29:28:WU00:FS00:Upload 19.44%
17:29:41:WU01:FS01:Upload 26.10%
17:29:45:WU00:FS00:Upload 20.19%
17:30:08:WU01:FS01:Upload 26.85%
17:30:20:WU00:FS00:Upload 20.93%
17:31:13:WU01:FS01:Upload 27.59%
17:31:13:WARNING:WU01:FS01:Exception: Failed to send results to work server: Transfer failed
17:31:13:WU01:FS01:Trying to send results to collection server
17:31:13:WU01:FS01:Uploading 8.38MiB to 171.65.103.160
17:31:19:WU01:FS01:Upload 15.66%
17:31:25:WU01:FS01:Upload 30.58%
17:31:31:WU01:FS01:Upload 45.49%
17:31:37:WU01:FS01:Upload 61.15%
17:31:43:WU01:FS01:Upload 76.82%
17:31:49:WU01:FS01:Upload 90.99%
17:31:53:WU01:FS01:Upload complete
17:31:54:WU01:FS01:Server responded WORK_ACK (400)
17:31:54:WU01:FS01:Final credit estimate, 35293.00 points
17:31:54:WU01:FS01:Cleaning up
17:32:54:WU00:FS00:Upload 21.68%
17:32:54:WARNING:WU00:FS00:Exception: Failed to send results to work server: Transfer failed
17:32:54:WU00:FS00:Trying to send results to collection server
17:32:54:WU00:FS00:Uploading 8.36MiB to 171.65.103.160
17:33:00:WU00:FS00:Upload 14.95%
17:33:06:WU00:FS00:Upload 29.16%
17:33:12:WU00:FS00:Upload 45.60%
17:33:18:WU00:FS00:Upload 59.81%
17:33:24:WU00:FS00:Upload 76.26%
17:33:30:WU00:FS00:Upload 91.21%
17:33:35:WU00:FS00:Upload complete
17:33:35:WU00:FS00:Server responded WORK_ACK (400)
17:33:35:WU00:FS00:Final credit estimate, 35221.00 points
17:33:35:WU00:FS00:Cleaning up
Re: Slow Upload Complete 9201 on Ubuntu 14.10 (64 bit)
Posted: Sat Feb 28, 2015 5:49 pm
by bruce
Welcome to foldingforum.org, qqqq
The log you posted shows that two or more WUs are being uploaded concurrently. That may not be the only reason it's slow, but it's certainly a contributing factor.
Stanford is having trouble on one of their subnets and people are already working on the problem so it may have nothing to do with Windows vs. Linux. The segment of the log that you posted doesn't include the IP addresses of the servers involved and I can't do a more thorough job of checking the status of those servers without that information.
Re: Slow Upload Complete 9201 on Ubuntu 14.10 (64 bit)
Posted: Sat Feb 28, 2015 6:17 pm
by qqqq
Ok, i set verbosity 3, it's another two 9201 on the way. By the way 9411 Project has not this type of problem, it's uploading about 5-10 sec.
Work server was 171.67.108.52
Re: Slow Upload Complete 9201 on Ubuntu 14.10 (64 bit)
Posted: Sat Feb 28, 2015 8:10 pm
by bruce
171.67.108.52 is working fine right now but the logs show that it clearly has had some problems on-and-off over the last several days.
Re: Slow Upload Complete 9201 on Ubuntu 14.10 (64 bit)
Posted: Sat Feb 28, 2015 9:02 pm
by qqqq
As you can see WU01 upload via Collecting Server and it's pretty fast. But WU02 uploads through the Work server
20:52:16:WU02:FS00:Connecting to 171.67.108.52:8080, is it normal ? I mean why Finished unit sends through the
Work server... So only 20 % was uploading for 10 minutes, it not include in the log.
Code: Select all
20:33:36: <!-- Folding Slots -->
20:33:36: <slot id='1' type='GPU'/>
20:33:36: <slot id='0' type='GPU'>
20:33:36: <gpu-index v='1'/>
20:33:36: </slot>
20:33:36:</config>
20:34:18:WU02:FS00:0x17:Completed 4500000 out of 5000000 steps (90%)
20:35:15:WU00:FS01:0x17:Completed 4600000 out of 5000000 steps (92%)
20:36:05:WU02:FS00:0x17:Completed 4550000 out of 5000000 steps (91%)
20:37:00:WU00:FS01:0x17:Completed 4650000 out of 5000000 steps (93%)
20:37:52:WU02:FS00:0x17:Completed 4600000 out of 5000000 steps (92%)
20:38:45:WU00:FS01:0x17:Completed 4700000 out of 5000000 steps (94%)
20:39:39:WU02:FS00:0x17:Completed 4650000 out of 5000000 steps (93%)
20:40:31:WU00:FS01:0x17:Completed 4750000 out of 5000000 steps (95%)
20:41:25:WU02:FS00:0x17:Completed 4700000 out of 5000000 steps (94%)
20:42:16:WU00:FS01:0x17:Completed 4800000 out of 5000000 steps (96%)
20:43:12:WU02:FS00:0x17:Completed 4750000 out of 5000000 steps (95%)
20:44:01:WU00:FS01:0x17:Completed 4850000 out of 5000000 steps (97%)
20:44:59:WU02:FS00:0x17:Completed 4800000 out of 5000000 steps (96%)
20:45:46:WU00:FS01:0x17:Completed 4900000 out of 5000000 steps (98%)
20:46:46:WU02:FS00:0x17:Completed 4850000 out of 5000000 steps (97%)
20:47:32:WU00:FS01:0x17:Completed 4950000 out of 5000000 steps (99%)
20:47:32:WU01:FS01:Connecting to 171.67.108.200:80
20:47:33:WU01:FS01:Assigned to work server 171.67.108.52
20:47:33:WU01:FS01:Requesting new work unit for slot 01: RUNNING gpu:0:GM204 [GeForce GTX 970] from 171.67.108.52
20:47:33:WU01:FS01:Connecting to 171.67.108.52:8080
20:47:33:WU01:FS01:Downloading 1.53MiB
20:47:37:WU01:FS01:Download complete
20:47:37:WU01:FS01:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:9201 run:577 clone:0 gen:502 core:0x17 unit:0x000002946652edc45399ecb3998ae1a3
20:48:33:WU02:FS00:0x17:Completed 4900000 out of 5000000 steps (98%)
20:49:17:WU00:FS01:0x17:Completed 5000000 out of 5000000 steps (100%)
20:49:20:WU00:FS01:0x17:Saving result file logfile_01.txt
20:49:20:WU00:FS01:0x17:Saving result file checkpointState.xml
20:49:21:WU00:FS01:0x17:Saving result file checkpt.crc
20:49:21:WU00:FS01:0x17:Saving result file log.txt
20:49:21:WU00:FS01:0x17:Saving result file positions.xtc
20:49:25:WU00:FS01:0x17:Folding@home Core Shutdown: FINISHED_UNIT
20:49:25:WU00:FS01:FahCore returned: FINISHED_UNIT (100 = 0x64)
20:49:25:WU00:FS01:Sending unit results: id:00 state:SEND error:NO_ERROR project:9201 run:497 clone:2 gen:225 core:0x17 unit:0x000001356652edc45399e994879bfd54
20:49:25:WU00:FS01:Uploading 8.42MiB to 171.67.108.52
20:49:25:WU00:FS01:Connecting to 171.67.108.52:8080
20:49:25:WU01:FS01:Starting
20:49:25:WU01:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/web.stanford.edu/~pande/Linux/AMD64/NVIDIA/Fermi/Core_17.fah/FahCore_17 -dir 01 -suffix 01 -version 704 -lifeline 1442 -checkpoint 30 -gpu 0 -gpu-vendor nvidia --client-type=advanced
20:49:25:WU01:FS01:Started FahCore on PID 15261
20:49:25:WU01:FS01:Core PID:15265
20:49:25:WU01:FS01:FahCore 0x17 started
20:49:26:WU01:FS01:0x17:*********************** Log Started 2015-02-28T20:49:25Z ***********************
20:49:26:WU01:FS01:0x17:Project: 9201 (Run 577, Clone 0, Gen 502)
20:49:26:WU01:FS01:0x17:Unit: 0x000002946652edc45399ecb3998ae1a3
20:49:26:WU01:FS01:0x17:CPU: 0x00000000000000000000000000000000
20:49:26:WU01:FS01:0x17:Machine: 1
20:49:26:WU01:FS01:0x17:Reading tar file state.xml
20:49:26:WU01:FS01:0x17:Reading tar file system.xml
20:49:26:WU01:FS01:0x17:Reading tar file integrator.xml
20:49:26:WU01:FS01:0x17:Reading tar file core.xml
20:49:26:WU01:FS01:0x17:Digital signatures verified
20:50:05:WU01:FS01:0x17:Completed 0 out of 5000000 steps (0%)
20:50:09:WU00:FS01:Upload 1.49%
20:50:19:WU00:FS01:Upload 2.23%
20:50:20:WU02:FS00:0x17:Completed 4950000 out of 5000000 steps (99%)
20:50:21:WU03:FS00:Connecting to 171.67.108.200:80
20:50:23:WU03:FS00:Assigned to work server 171.67.108.52
20:50:23:WU03:FS00:Requesting new work unit for slot 00: RUNNING gpu:1:GM204 [GeForce GTX 970] from 171.67.108.52
20:50:23:WU03:FS00:Connecting to 171.67.108.52:8080
20:50:24:WU03:FS00:Downloading 1.53MiB
20:50:27:WU03:FS00:Download complete
20:50:27:WU03:FS00:Received Unit: id:03 state:DOWNLOAD error:NO_ERROR project:9201 run:599 clone:4 gen:195 core:0x17 unit:0x000001116652edc45399ed977eaf7d30
20:51:49:WU01:FS01:0x17:Completed 50000 out of 5000000 steps (1%)
20:52:02:WU00:FS01:Upload 2.97%
20:52:02:WARNING:WU00:FS01:Exception: Failed to send results to work server: Transfer failed
20:52:02:WU00:FS01:Trying to send results to collection server
20:52:02:WU00:FS01:Uploading 8.42MiB to 171.65.103.160
20:52:02:WU00:FS01:Connecting to 171.65.103.160:8080
20:52:07:WU02:FS00:0x17:Completed 5000000 out of 5000000 steps (100%)
20:52:08:WU00:FS01:Upload 23.02%
20:52:11:WU02:FS00:0x17:Saving result file logfile_01.txt
20:52:11:WU02:FS00:0x17:Saving result file checkpointState.xml
20:52:12:WU02:FS00:0x17:Saving result file checkpt.crc
20:52:12:WU02:FS00:0x17:Saving result file log.txt
20:52:12:WU02:FS00:0x17:Saving result file positions.xtc
20:52:14:WU00:FS01:Upload 44.56%
20:52:15:WU02:FS00:0x17:Folding@home Core Shutdown: FINISHED_UNIT
20:52:16:WU02:FS00:FahCore returned: FINISHED_UNIT (100 = 0x64)
20:52:16:WU02:FS00:Sending unit results: id:02 state:SEND error:NO_ERROR project:9201 run:536 clone:4 gen:288 core:0x17 unit:0x000001766652edc45399eb206ee27c3e
20:52:16:WU02:FS00:Uploading 8.40MiB to 171.67.108.52
20:52:16:WU02:FS00:Connecting to 171.67.108.52:8080
20:52:16:WU03:FS00:Starting
20:52:16:WU03:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/web.stanford.edu/~pande/Linux/AMD64/NVIDIA/Fermi/Core_17.fah/FahCore_17 -dir 03 -suffix 01 -version 704 -lifeline 1442 -checkpoint 30 -gpu 1 -gpu-vendor nvidia --client-type=advanced
20:52:16:WU03:FS00:Started FahCore on PID 15276
20:52:16:WU03:FS00:Core PID:15280
20:52:16:WU03:FS00:FahCore 0x17 started
20:52:16:WU03:FS00:0x17:*********************** Log Started 2015-02-28T20:52:16Z ***********************
20:52:16:WU03:FS00:0x17:Project: 9201 (Run 599, Clone 4, Gen 195)
20:52:16:WU03:FS00:0x17:Unit: 0x000001116652edc45399ed977eaf7d30
20:52:16:WU03:FS00:0x17:CPU: 0x00000000000000000000000000000000
20:52:16:WU03:FS00:0x17:Machine: 0
20:52:16:WU03:FS00:0x17:Reading tar file state.xml
20:52:16:WU03:FS00:0x17:Reading tar file system.xml
20:52:16:WU03:FS00:0x17:Reading tar file integrator.xml
20:52:16:WU03:FS00:0x17:Reading tar file core.xml
20:52:16:WU03:FS00:0x17:Digital signatures verified
20:52:20:WU00:FS01:Upload 68.32%
20:52:22:WU02:FS00:Upload 0.74%
20:52:26:WU00:FS01:Upload 89.85%
20:52:29:WU00:FS01:Upload complete
20:52:29:WU00:FS01:Server responded WORK_ACK (400)
20:52:29:WU00:FS01:Final credit estimate, 35646.00 points
20:52:29:WU00:FS01:Cleaning up
20:52:45:WU02:FS00:Upload 1.49%
20:52:55:WU03:FS00:0x17:Completed 0 out of 5000000 steps (0%)
20:52:57:WU02:FS00:Upload 2.23%
20:53:07:WU02:FS00:Upload 2.98%
20:53:22:WU02:FS00:Upload 3.72%
20:53:35:WU01:FS01:0x17:Completed 100000 out of 5000000 steps (2%)
20:53:35:WU02:FS00:Upload 4.46%
20:53:55:WU02:FS00:Upload 5.21%
20:54:14:WU02:FS00:Upload 5.95%
20:54:35:WU02:FS00:Upload 6.69%
20:54:41:WU03:FS00:0x17:Completed 50000 out of 5000000 steps (1%)
20:55:20:WU01:FS01:0x17:Completed 150000 out of 5000000 steps (3%)
20:55:36:WU02:FS00:Upload 7.44%
After stuck on ~30 % of WU02 it says again WU02 Exceprion Failed to send result to work server: Transfer failed
And then it connected to Collection server and uploaded about 30 secs. So guys it's a problem needs to be fixed.
Re: Slow Upload Complete 9201 on Ubuntu 14.10 (64 bit)
Posted: Sat Feb 28, 2015 10:09 pm
by bruce
You've indicated that the problem is associate with a specific server, not with project 9201 or Ubuntu so I'm going to change the title of the first post and move the discussion to the forum where server problems are discussed.
All WUs are returned to the server which issued them (the Work Server). If the Work Server is down, the WU can be uploaded to a Collection Server which will forward the WU to the WS. In this case, there are communications problems associated with the WS, but it's not "down." If it were, the WU would go directly to the CS.
Re: 171.67.108.52
Posted: Sun Mar 01, 2015 6:38 am
by qqqq
Ok i see. The work server still show this error (05:57:30:WARNING:WU03:FS00:Exception: Failed to send results to work server: Transfer failed)
Re: 171.67.108.52
Posted: Sun Mar 01, 2015 8:16 am
by jmrios
I'm having the same upload issue with this server since a few days ago, takes forever to upload a finished WU. Since I thought that was mi ISP, but all the other things related to my ISP are working fine. I just came home, and found a WU in Send state, but with no upload % in the log for a long time, so I've restarted the PC
My PC specs:
OS: Ubuntu 14.04 LTS 64bits
i7 3770k (no folding on SMP)
A pair of Nvidia GTX 970
Driver: 346.35
Code: Select all
*********************** Log Started 2015-03-01T07:49:38Z ***********************
07:49:38:************************* Folding@home Client *************************
07:49:38: Website: http://folding.stanford.edu/
07:49:38: Copyright: (c) 2009-2014 Stanford University
07:49:38: Author: Joseph Coffland <joseph@cauldrondevelopment.com>
07:49:38: Args:
07:49:38: Config: /home/jmrios/config.xml
07:49:38:******************************** Build ********************************
07:49:38: Version: 7.4.4
07:49:38: Date: Mar 4 2014
07:49:38: Time: 12:02:38
07:49:38: SVN Rev: 4130
07:49:38: Branch: fah/trunk/client
07:49:38: Compiler: GNU 4.4.7
07:49:38: Options: -std=gnu++98 -O3 -funroll-loops -mfpmath=sse -ffast-math
07:49:38: -fno-unsafe-math-optimizations -msse2
07:49:38: Platform: linux2 3.2.0-1-amd64
07:49:38: Bits: 64
07:49:38: Mode: Release
07:49:38:******************************* System ********************************
07:49:38: CPU: Intel(R) Core(TM) i7-3770K CPU @ 3.50GHz
07:49:38: CPU ID: GenuineIntel Family 6 Model 58 Stepping 9
07:49:38: CPUs: 8
07:49:38: Memory: 7.75GiB
07:49:38:Free Memory: 6.88GiB
07:49:38: Threads: POSIX_THREADS
07:49:38: OS Version: 3.13
07:49:38:Has Battery: false
07:49:38: On Battery: false
07:49:38: UTC Offset: -3
07:49:38: PID: 2532
07:49:38: CWD: /home/jmrios
07:49:38: OS: Linux 3.13.0-45-generic x86_64
07:49:38: OS Arch: AMD64
07:49:38: GPUs: 2
07:49:38: GPU 0: NVIDIA:5 GM204 [GeForce GTX 970]
07:49:38: GPU 1: NVIDIA:5 GM204 [GeForce GTX 970]
07:49:38: CUDA: 5.2
07:49:38:CUDA Driver: 7000
07:49:38:***********************************************************************
07:49:38:<config>
07:49:38: <!-- Network -->
07:49:38: <proxy v=':8080'/>
07:49:38:
07:49:38: <!-- Slot Control -->
07:49:38: <power v='full'/>
07:49:38:
07:49:38: <!-- User Information -->
07:49:38: <passkey v='********************************'/>
07:49:38: <team v='196420'/>
07:49:38: <user v='jmrios'/>
07:49:38:
07:49:38: <!-- Folding Slots -->
07:49:38: <slot id='1' type='GPU'>
07:49:38: <next-unit-percentage v='100'/>
07:49:38: </slot>
07:49:38: <slot id='2' type='GPU'>
07:49:38: <next-unit-percentage v='100'/>
07:49:38: </slot>
07:49:38:</config>
07:49:38:Trying to access database...
07:49:38:Successfully acquired database lock
07:49:38:Enabled folding slot 01: READY gpu:0:GM204 [GeForce GTX 970]
07:49:38:Enabled folding slot 02: READY gpu:1:GM204 [GeForce GTX 970]
07:49:38:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:9201 run:921 clone:4 gen:191 core:0x17 unit:0x0000010f6652edc45399fa41509a79cf
07:49:38:WU01:FS01:Uploading 8.33MiB to 171.67.108.52
07:49:38:WU01:FS01:Connecting to 171.67.108.52:8080
07:49:38:WU00:FS02:Starting
07:49:38:WU00:FS02:Running FahCore: /usr/bin/FAHCoreWrapper /home/jmrios/cores/web.stanford.edu/~pande/Linux/AMD64/NVIDIA/Fermi/Core_17.fah/FahCore_17 -dir 00 -suffix 01 -version 704 -lifeline 2532 -checkpoint 15 -gpu 1 -gpu-vendor nvidia
07:49:38:WU00:FS02:Started FahCore on PID 2550
07:49:38:WU00:FS02:Core PID:2554
07:49:38:WU00:FS02:FahCore 0x17 started
07:49:38:WU03:FS01:Starting
07:49:38:WU03:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /home/jmrios/cores/web.stanford.edu/~pande/Linux/AMD64/NVIDIA/Fermi/Core_17.fah/FahCore_17 -dir 03 -suffix 01 -version 704 -lifeline 2532 -checkpoint 15 -gpu 0 -gpu-vendor nvidia
07:49:38:WU03:FS01:Started FahCore on PID 2555
07:49:38:WU03:FS01:Core PID:2559
07:49:38:WU03:FS01:FahCore 0x17 started
07:49:39:WU00:FS02:0x17:*********************** Log Started 2015-03-01T07:49:38Z ***********************
07:49:39:WU00:FS02:0x17:Project: 9201 (Run 628, Clone 4, Gen 194)
07:49:39:WU00:FS02:0x17:Unit: 0x000000fc6652edc45399eebdb0cd3a2d
07:49:39:WU00:FS02:0x17:CPU: 0x00000000000000000000000000000000
07:49:39:WU00:FS02:0x17:Machine: 2
07:49:39:WU00:FS02:0x17:Digital signatures verified
07:49:39:WU00:FS02:0x17: Found a checkpoint file
07:49:39:WU03:FS01:0x17:*********************** Log Started 2015-03-01T07:49:38Z ***********************
07:49:39:WU03:FS01:0x17:Project: 9201 (Run 607, Clone 4, Gen 250)
07:49:39:WU03:FS01:0x17:Unit: 0x000001406652edc45399edeaa2090a63
07:49:39:WU03:FS01:0x17:CPU: 0x00000000000000000000000000000000
07:49:39:WU03:FS01:0x17:Machine: 1
07:49:39:WU03:FS01:0x17:Digital signatures verified
07:49:39:WU03:FS01:0x17: Found a checkpoint file
07:50:01:WU00:FS02:0x17:Completed 1600000 out of 5000000 steps (32%)
07:50:01:WU03:FS01:0x17:Completed 1100000 out of 5000000 steps (22%)
07:50:17:WU01:FS01:Upload 0.75%
07:51:42:WU03:FS01:0x17:Completed 1150000 out of 5000000 steps (23%)
07:51:48:WU00:FS02:0x17:Completed 1650000 out of 5000000 steps (33%)
07:53:23:WU03:FS01:0x17:Completed 1200000 out of 5000000 steps (24%)
07:53:36:WU00:FS02:0x17:Completed 1700000 out of 5000000 steps (34%)
07:55:06:WU03:FS01:0x17:Completed 1250000 out of 5000000 steps (25%)
07:55:23:WU00:FS02:0x17:Completed 1750000 out of 5000000 steps (35%)
07:56:46:WU03:FS01:0x17:Completed 1300000 out of 5000000 steps (26%)
07:57:10:WU00:FS02:0x17:Completed 1800000 out of 5000000 steps (36%)
07:58:26:WU03:FS01:0x17:Completed 1350000 out of 5000000 steps (27%)
07:58:57:WU00:FS02:0x17:Completed 1850000 out of 5000000 steps (37%)
08:00:05:WU03:FS01:0x17:Completed 1400000 out of 5000000 steps (28%)
08:00:44:WU00:FS02:0x17:Completed 1900000 out of 5000000 steps (38%)
08:01:45:WU03:FS01:0x17:Completed 1450000 out of 5000000 steps (29%)
08:02:31:WU00:FS02:0x17:Completed 1950000 out of 5000000 steps (39%)
08:03:24:WU03:FS01:0x17:Completed 1500000 out of 5000000 steps (30%)
08:04:19:WU00:FS02:0x17:Completed 2000000 out of 5000000 steps (40%)
08:05:05:WU03:FS01:0x17:Completed 1550000 out of 5000000 steps (31%)
08:06:08:WU00:FS02:0x17:Completed 2050000 out of 5000000 steps (41%)
08:06:43:WU01:FS01:Upload 1.50%
08:06:43:WARNING:WU01:FS01:Exception: Failed to send results to work server: Transfer failed
08:06:43:WU01:FS01:Trying to send results to collection server
08:06:43:WU01:FS01:Uploading 8.33MiB to 171.65.103.160
08:06:43:WU01:FS01:Connecting to 171.65.103.160:8080
08:06:45:WU03:FS01:0x17:Completed 1600000 out of 5000000 steps (32%)
08:06:49:WU01:FS01:Upload 7.50%
08:06:55:WU01:FS01:Upload 18.75%
08:07:01:WU01:FS01:Upload 27.75%
08:07:07:WU01:FS01:Upload 38.25%
08:07:13:WU01:FS01:Upload 48.00%
08:07:19:WU01:FS01:Upload 57.75%
08:07:25:WU01:FS01:Upload 66.75%
08:07:31:WU01:FS01:Upload 77.25%
08:07:37:WU01:FS01:Upload 87.00%
08:07:43:WU01:FS01:Upload 96.75%
08:07:47:WU01:FS01:Upload complete
08:07:47:WU01:FS01:Server responded WORK_ACK (400)
08:07:47:WU01:FS01:Final credit estimate, 32554.00 points
08:07:47:WU01:FS01:Cleaning up
08:07:56:WU00:FS02:0x17:Completed 2100000 out of 5000000 steps (42%)
08:08:26:WU03:FS01:0x17:Completed 1650000 out of 5000000 steps (33%)
08:09:45:WU00:FS02:0x17:Completed 2150000 out of 5000000 steps (43%)
08:10:08:WU03:FS01:0x17:Completed 1700000 out of 5000000 steps (34%)
08:11:32:WU00:FS02:0x17:Completed 2200000 out of 5000000 steps (44%)
08:11:49:WU03:FS01:0x17:Completed 1750000 out of 5000000 steps (35%)
08:13:19:WU00:FS02:0x17:Completed 2250000 out of 5000000 steps (45%)
08:13:32:WU03:FS01:0x17:Completed 1800000 out of 5000000 steps (36%)
I'll check tomorrow the log after more finished WU's to provide more information.
Re: 171.67.108.52
Posted: Sun Mar 01, 2015 6:36 pm
by jmrios
New logs, same issue:
Code: Select all
*********************** Log Started 2015-03-01T15:12:22Z ***********************
15:12:22:************************* Folding@home Client *************************
15:12:22: Website: http://folding.stanford.edu/
15:12:22: Copyright: (c) 2009-2014 Stanford University
15:12:22: Author: Joseph Coffland <joseph@cauldrondevelopment.com>
15:12:22: Args:
15:12:22: Config: /home/jmrios/config.xml
15:12:22:******************************** Build ********************************
15:12:22: Version: 7.4.4
15:12:22: Date: Mar 4 2014
15:12:22: Time: 12:02:38
15:12:22: SVN Rev: 4130
15:12:22: Branch: fah/trunk/client
15:12:22: Compiler: GNU 4.4.7
15:12:22: Options: -std=gnu++98 -O3 -funroll-loops -mfpmath=sse -ffast-math
15:12:22: -fno-unsafe-math-optimizations -msse2
15:12:22: Platform: linux2 3.2.0-1-amd64
15:12:22: Bits: 64
15:12:22: Mode: Release
15:12:22:******************************* System ********************************
15:12:22: CPU: Intel(R) Core(TM) i7-3770K CPU @ 3.50GHz
15:12:22: CPU ID: GenuineIntel Family 6 Model 58 Stepping 9
15:12:22: CPUs: 8
15:12:22: Memory: 7.75GiB
15:12:22:Free Memory: 6.89GiB
15:12:22: Threads: POSIX_THREADS
15:12:22: OS Version: 3.13
15:12:22:Has Battery: false
15:12:22: On Battery: false
15:12:22: UTC Offset: -3
15:12:22: PID: 2476
15:12:22: CWD: /home/jmrios
15:12:22: OS: Linux 3.13.0-45-generic x86_64
15:12:22: OS Arch: AMD64
15:12:22: GPUs: 2
15:12:22: GPU 0: NVIDIA:5 GM204 [GeForce GTX 970]
15:12:22: GPU 1: NVIDIA:5 GM204 [GeForce GTX 970]
15:12:22: CUDA: 5.2
15:12:22:CUDA Driver: 7000
15:12:22:***********************************************************************
15:12:22:<config>
15:12:22: <!-- Network -->
15:12:22: <proxy v=':8080'/>
15:12:22:
15:12:22: <!-- Slot Control -->
15:12:22: <power v='full'/>
15:12:22:
15:12:22: <!-- User Information -->
15:12:22: <passkey v='********************************'/>
15:12:22: <team v='196420'/>
15:12:22: <user v='jmrios'/>
15:12:22:
15:12:22: <!-- Folding Slots -->
15:12:22: <slot id='1' type='GPU'>
15:12:22: <next-unit-percentage v='100'/>
15:12:22: <paused v='true'/>
15:12:22: </slot>
15:12:22: <slot id='2' type='GPU'>
15:12:22: <next-unit-percentage v='100'/>
15:12:22: <paused v='true'/>
15:12:22: </slot>
15:12:22:</config>
15:12:22:Trying to access database...
15:12:22:Successfully acquired database lock
15:12:22:Enabled folding slot 01: PAUSED gpu:0:GM204 [GeForce GTX 970] (by user)
15:12:22:Enabled folding slot 02: PAUSED gpu:1:GM204 [GeForce GTX 970] (by user)
15:12:28:FS01:Unpaused
15:12:28:FS02:Unpaused
15:12:28:WU00:FS01:Starting
15:12:28:WU00:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /home/jmrios/cores/web.stanford.edu/~pande/Linux/AMD64/NVIDIA/Fermi/Core_17.fah/FahCore_17 -dir 00 -suffix 01 -version 704 -lifeline 2476 -checkpoint 15 -gpu 0 -gpu-vendor nvidia
15:12:28:WU00:FS01:Started FahCore on PID 2548
15:12:28:WU00:FS01:Core PID:2552
15:12:28:WU00:FS01:FahCore 0x17 started
15:12:28:WU02:FS02:Starting
15:12:28:WU02:FS02:Running FahCore: /usr/bin/FAHCoreWrapper /home/jmrios/cores/web.stanford.edu/~pande/Linux/AMD64/NVIDIA/Fermi/Core_17.fah/FahCore_17 -dir 02 -suffix 01 -version 704 -lifeline 2476 -checkpoint 15 -gpu 1 -gpu-vendor nvidia
15:12:28:WU02:FS02:Started FahCore on PID 2553
15:12:28:WU02:FS02:Core PID:2557
15:12:28:WU02:FS02:FahCore 0x17 started
15:12:29:WU00:FS01:0x17:*********************** Log Started 2015-03-01T15:12:28Z ***********************
15:12:29:WU00:FS01:0x17:Project: 9201 (Run 832, Clone 3, Gen 191)
15:12:29:WU00:FS01:0x17:Unit: 0x000001186652edc45399f6c3346ce98f
15:12:29:WU00:FS01:0x17:CPU: 0x00000000000000000000000000000000
15:12:29:WU00:FS01:0x17:Machine: 1
15:12:29:WU00:FS01:0x17:Digital signatures verified
15:12:29:WU00:FS01:0x17: Found a checkpoint file
15:12:29:WU02:FS02:0x17:*********************** Log Started 2015-03-01T15:12:28Z ***********************
15:12:29:WU02:FS02:0x17:Project: 9201 (Run 307, Clone 2, Gen 318)
15:12:29:WU02:FS02:0x17:Unit: 0x0000019b6652edc45399e220e1a5ed65
15:12:29:WU02:FS02:0x17:CPU: 0x00000000000000000000000000000000
15:12:29:WU02:FS02:0x17:Machine: 2
15:12:29:WU02:FS02:0x17:Digital signatures verified
15:12:29:WU02:FS02:0x17: Found a checkpoint file
15:12:50:WU02:FS02:0x17:Completed 3900000 out of 5000000 steps (78%)
15:12:50:WU00:FS01:0x17:Completed 1450000 out of 5000000 steps (29%)
15:13:23:Removing old file 'configs/config-20150219-224625.xml'
15:13:23:Saving configuration to config.xml
15:13:23:<config>
15:13:23: <!-- Network -->
15:13:23: <proxy v=':8080'/>
15:13:23:
15:13:23: <!-- Slot Control -->
15:13:23: <power v='full'/>
15:13:23:
15:13:23: <!-- User Information -->
15:13:23: <passkey v='********************************'/>
15:13:23: <team v='196420'/>
15:13:23: <user v='jmrios'/>
15:13:23:
15:13:23: <!-- Folding Slots -->
15:13:23: <slot id='1' type='GPU'>
15:13:23: <next-unit-percentage v='100'/>
15:13:23: </slot>
15:13:23: <slot id='2' type='GPU'>
15:13:23: <next-unit-percentage v='100'/>
15:13:23: </slot>
15:13:23:</config>
15:14:31:WU00:FS01:0x17:Completed 1500000 out of 5000000 steps (30%)
15:14:37:WU02:FS02:0x17:Completed 3950000 out of 5000000 steps (79%)
15:16:12:WU00:FS01:0x17:Completed 1550000 out of 5000000 steps (31%)
15:16:24:WU02:FS02:0x17:Completed 4000000 out of 5000000 steps (80%)
15:17:52:WU00:FS01:0x17:Completed 1600000 out of 5000000 steps (32%)
15:18:11:WU02:FS02:0x17:Completed 4050000 out of 5000000 steps (81%)
15:19:31:WU00:FS01:0x17:Completed 1650000 out of 5000000 steps (33%)
15:19:58:WU02:FS02:0x17:Completed 4100000 out of 5000000 steps (82%)
15:21:10:WU00:FS01:0x17:Completed 1700000 out of 5000000 steps (34%)
15:21:44:WU02:FS02:0x17:Completed 4150000 out of 5000000 steps (83%)
15:22:49:WU00:FS01:0x17:Completed 1750000 out of 5000000 steps (35%)
15:23:32:WU02:FS02:0x17:Completed 4200000 out of 5000000 steps (84%)
15:24:30:WU00:FS01:0x17:Completed 1800000 out of 5000000 steps (36%)
15:25:20:WU02:FS02:0x17:Completed 4250000 out of 5000000 steps (85%)
15:26:10:WU00:FS01:0x17:Completed 1850000 out of 5000000 steps (37%)
15:27:08:WU02:FS02:0x17:Completed 4300000 out of 5000000 steps (86%)
15:27:51:WU00:FS01:0x17:Completed 1900000 out of 5000000 steps (38%)
15:28:56:WU02:FS02:0x17:Completed 4350000 out of 5000000 steps (87%)
15:29:31:WU00:FS01:0x17:Completed 1950000 out of 5000000 steps (39%)
15:30:44:WU02:FS02:0x17:Completed 4400000 out of 5000000 steps (88%)
15:31:12:WU00:FS01:0x17:Completed 2000000 out of 5000000 steps (40%)
15:32:32:WU02:FS02:0x17:Completed 4450000 out of 5000000 steps (89%)
15:32:52:WU00:FS01:0x17:Completed 2050000 out of 5000000 steps (41%)
15:34:21:WU02:FS02:0x17:Completed 4500000 out of 5000000 steps (90%)
15:34:33:WU00:FS01:0x17:Completed 2100000 out of 5000000 steps (42%)
15:36:09:WU02:FS02:0x17:Completed 4550000 out of 5000000 steps (91%)
15:36:13:WU00:FS01:0x17:Completed 2150000 out of 5000000 steps (43%)
15:37:53:WU00:FS01:0x17:Completed 2200000 out of 5000000 steps (44%)
15:37:57:WU02:FS02:0x17:Completed 4600000 out of 5000000 steps (92%)
15:39:33:WU00:FS01:0x17:Completed 2250000 out of 5000000 steps (45%)
15:39:45:WU02:FS02:0x17:Completed 4650000 out of 5000000 steps (93%)
15:41:14:WU00:FS01:0x17:Completed 2300000 out of 5000000 steps (46%)
15:41:33:WU02:FS02:0x17:Completed 4700000 out of 5000000 steps (94%)
15:42:54:WU00:FS01:0x17:Completed 2350000 out of 5000000 steps (47%)
15:43:21:WU02:FS02:0x17:Completed 4750000 out of 5000000 steps (95%)
15:44:34:WU00:FS01:0x17:Completed 2400000 out of 5000000 steps (48%)
15:45:09:WU02:FS02:0x17:Completed 4800000 out of 5000000 steps (96%)
15:46:15:WU00:FS01:0x17:Completed 2450000 out of 5000000 steps (49%)
15:46:57:WU02:FS02:0x17:Completed 4850000 out of 5000000 steps (97%)
15:47:55:WU00:FS01:0x17:Completed 2500000 out of 5000000 steps (50%)
15:48:46:WU02:FS02:0x17:Completed 4900000 out of 5000000 steps (98%)
15:49:36:WU00:FS01:0x17:Completed 2550000 out of 5000000 steps (51%)
15:50:34:WU02:FS02:0x17:Completed 4950000 out of 5000000 steps (99%)
15:51:16:WU00:FS01:0x17:Completed 2600000 out of 5000000 steps (52%)
15:52:22:WU02:FS02:0x17:Completed 5000000 out of 5000000 steps (100%)
15:52:24:WU01:FS02:Connecting to 171.67.108.200:80
15:52:24:WU02:FS02:0x17:Saving result file logfile_01.txt
15:52:24:WU02:FS02:0x17:Saving result file checkpointState.xml
15:52:25:WU02:FS02:0x17:Saving result file checkpt.crc
15:52:25:WU02:FS02:0x17:Saving result file log.txt
15:52:25:WU02:FS02:0x17:Saving result file positions.xtc
15:52:26:WU02:FS02:0x17:Folding@home Core Shutdown: FINISHED_UNIT
15:52:26:WU02:FS02:FahCore returned: FINISHED_UNIT (100 = 0x64)
15:52:26:WU02:FS02:Sending unit results: id:02 state:SEND error:NO_ERROR project:9201 run:307 clone:2 gen:318 core:0x17 unit:0x0000019b6652edc45399e220e1a5ed65
15:52:26:WU02:FS02:Uploading 8.42MiB to 171.67.108.52
15:52:26:WU01:FS02:Assigned to work server 171.67.108.52
15:52:26:WU02:FS02:Connecting to 171.67.108.52:8080
15:52:26:WU01:FS02:Requesting new work unit for slot 02: READY gpu:1:GM204 [GeForce GTX 970] from 171.67.108.52
15:52:26:WU01:FS02:Connecting to 171.67.108.52:8080
15:52:28:WU01:FS02:Downloading 1.53MiB
15:52:34:WU01:FS02:Download 61.38%
15:52:36:WU01:FS02:Download complete
15:52:36:WU01:FS02:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:9201 run:310 clone:4 gen:262 core:0x17 unit:0x0000015a6652edc45399e24265c61366
15:52:36:WU01:FS02:Starting
15:52:36:WU01:FS02:Running FahCore: /usr/bin/FAHCoreWrapper /home/jmrios/cores/web.stanford.edu/~pande/Linux/AMD64/NVIDIA/Fermi/Core_17.fah/FahCore_17 -dir 01 -suffix 01 -version 704 -lifeline 2476 -checkpoint 15 -gpu 1 -gpu-vendor nvidia
15:52:36:WU01:FS02:Started FahCore on PID 3226
15:52:36:WU01:FS02:Core PID:3230
15:52:36:WU01:FS02:FahCore 0x17 started
15:52:37:WU01:FS02:0x17:*********************** Log Started 2015-03-01T15:52:36Z ***********************
15:52:37:WU01:FS02:0x17:Project: 9201 (Run 310, Clone 4, Gen 262)
15:52:37:WU01:FS02:0x17:Unit: 0x0000015a6652edc45399e24265c61366
15:52:37:WU01:FS02:0x17:CPU: 0x00000000000000000000000000000000
15:52:37:WU01:FS02:0x17:Machine: 2
15:52:37:WU01:FS02:0x17:Reading tar file state.xml
15:52:37:WU01:FS02:0x17:Reading tar file system.xml
15:52:37:WU01:FS02:0x17:Reading tar file integrator.xml
15:52:37:WU01:FS02:0x17:Reading tar file core.xml
15:52:37:WU01:FS02:0x17:Digital signatures verified
15:52:50:WU01:FS02:0x17:Completed 0 out of 5000000 steps (0%)
15:52:56:WU00:FS01:0x17:Completed 2650000 out of 5000000 steps (53%)
15:53:09:WU02:FS02:Upload 0.74%
15:54:03:WU02:FS02:Upload 1.48%
15:54:35:WU00:FS01:0x17:Completed 2700000 out of 5000000 steps (54%)
15:54:37:WU01:FS02:0x17:Completed 50000 out of 5000000 steps (1%)
15:54:54:WU02:FS02:Upload 2.23%
15:56:14:WU00:FS01:0x17:Completed 2750000 out of 5000000 steps (55%)
15:56:16:WU02:FS02:Upload 2.97%
15:56:17:WARNING:WU02:FS02:Exception: Failed to send results to work server: Transfer failed
15:56:17:WU02:FS02:Trying to send results to collection server
15:56:17:WU02:FS02:Uploading 8.42MiB to 171.65.103.160
15:56:17:WU02:FS02:Connecting to 171.65.103.160:8080
15:56:23:WU02:FS02:Upload 6.68%
15:56:24:WU01:FS02:0x17:Completed 100000 out of 5000000 steps (2%)
15:56:29:WU02:FS02:Upload 13.36%
15:56:36:WU02:FS02:Upload 19.30%
15:56:42:WU02:FS02:Upload 24.50%
15:56:48:WU02:FS02:Upload 31.18%
15:56:54:WU02:FS02:Upload 36.37%
15:57:00:WU02:FS02:Upload 42.31%
15:57:06:WU02:FS02:Upload 47.51%
15:57:12:WU02:FS02:Upload 54.19%
15:57:18:WU02:FS02:Upload 61.61%
15:57:24:WU02:FS02:Upload 68.29%
15:57:30:WU02:FS02:Upload 74.98%
15:57:36:WU02:FS02:Upload 80.91%
15:57:42:WU02:FS02:Upload 88.34%
15:57:48:WU02:FS02:Upload 95.76%
15:57:53:WU00:FS01:0x17:Completed 2800000 out of 5000000 steps (56%)
15:57:54:WU02:FS02:Upload complete
15:57:54:WU02:FS02:Server responded WORK_ACK (400)
15:57:54:WU02:FS02:Final credit estimate, 35343.00 points
15:57:54:WU02:FS02:Cleaning up
15:58:11:WU01:FS02:0x17:Completed 150000 out of 5000000 steps (3%)
15:59:32:WU00:FS01:0x17:Completed 2850000 out of 5000000 steps (57%)
15:59:58:WU01:FS02:0x17:Completed 200000 out of 5000000 steps (4%)
16:01:11:WU00:FS01:0x17:Completed 2900000 out of 5000000 steps (58%)
16:01:45:WU01:FS02:0x17:Completed 250000 out of 5000000 steps (5%)
16:02:49:WU00:FS01:0x17:Completed 2950000 out of 5000000 steps (59%)
16:03:32:WU01:FS02:0x17:Completed 300000 out of 5000000 steps (6%)
16:04:28:WU00:FS01:0x17:Completed 3000000 out of 5000000 steps (60%)
16:05:19:WU01:FS02:0x17:Completed 350000 out of 5000000 steps (7%)
16:06:07:WU00:FS01:0x17:Completed 3050000 out of 5000000 steps (61%)
16:07:06:WU01:FS02:0x17:Completed 400000 out of 5000000 steps (8%)
16:07:46:WU00:FS01:0x17:Completed 3100000 out of 5000000 steps (62%)
16:08:53:WU01:FS02:0x17:Completed 450000 out of 5000000 steps (9%)
16:09:25:WU00:FS01:0x17:Completed 3150000 out of 5000000 steps (63%)
16:10:40:WU01:FS02:0x17:Completed 500000 out of 5000000 steps (10%)
16:11:04:WU00:FS01:0x17:Completed 3200000 out of 5000000 steps (64%)
16:12:27:WU01:FS02:0x17:Completed 550000 out of 5000000 steps (11%)
16:12:43:WU00:FS01:0x17:Completed 3250000 out of 5000000 steps (65%)
16:14:14:WU01:FS02:0x17:Completed 600000 out of 5000000 steps (12%)
16:14:22:WU00:FS01:0x17:Completed 3300000 out of 5000000 steps (66%)
16:16:00:WU00:FS01:0x17:Completed 3350000 out of 5000000 steps (67%)
16:16:01:WU01:FS02:0x17:Completed 650000 out of 5000000 steps (13%)
16:17:39:WU00:FS01:0x17:Completed 3400000 out of 5000000 steps (68%)
16:17:48:WU01:FS02:0x17:Completed 700000 out of 5000000 steps (14%)
16:19:18:WU00:FS01:0x17:Completed 3450000 out of 5000000 steps (69%)
16:19:35:WU01:FS02:0x17:Completed 750000 out of 5000000 steps (15%)
16:20:57:WU00:FS01:0x17:Completed 3500000 out of 5000000 steps (70%)
16:21:22:WU01:FS02:0x17:Completed 800000 out of 5000000 steps (16%)
16:22:36:WU00:FS01:0x17:Completed 3550000 out of 5000000 steps (71%)
16:23:09:WU01:FS02:0x17:Completed 850000 out of 5000000 steps (17%)
16:24:15:WU00:FS01:0x17:Completed 3600000 out of 5000000 steps (72%)
16:24:56:WU01:FS02:0x17:Completed 900000 out of 5000000 steps (18%)
16:25:54:WU00:FS01:0x17:Completed 3650000 out of 5000000 steps (73%)
16:26:44:WU01:FS02:0x17:Completed 950000 out of 5000000 steps (19%)
16:27:33:WU00:FS01:0x17:Completed 3700000 out of 5000000 steps (74%)
16:28:31:WU01:FS02:0x17:Completed 1000000 out of 5000000 steps (20%)
16:29:12:WU00:FS01:0x17:Completed 3750000 out of 5000000 steps (75%)
16:30:17:WU01:FS02:0x17:Completed 1050000 out of 5000000 steps (21%)
16:30:50:WU00:FS01:0x17:Completed 3800000 out of 5000000 steps (76%)
16:32:04:WU01:FS02:0x17:Completed 1100000 out of 5000000 steps (22%)
16:32:29:WU00:FS01:0x17:Completed 3850000 out of 5000000 steps (77%)
16:33:51:WU01:FS02:0x17:Completed 1150000 out of 5000000 steps (23%)
16:34:08:WU00:FS01:0x17:Completed 3900000 out of 5000000 steps (78%)
16:35:38:WU01:FS02:0x17:Completed 1200000 out of 5000000 steps (24%)
16:35:47:WU00:FS01:0x17:Completed 3950000 out of 5000000 steps (79%)
16:37:25:WU01:FS02:0x17:Completed 1250000 out of 5000000 steps (25%)
16:37:26:WU00:FS01:0x17:Completed 4000000 out of 5000000 steps (80%)
16:39:05:WU00:FS01:0x17:Completed 4050000 out of 5000000 steps (81%)
16:39:12:WU01:FS02:0x17:Completed 1300000 out of 5000000 steps (26%)
16:40:44:WU00:FS01:0x17:Completed 4100000 out of 5000000 steps (82%)
16:40:59:WU01:FS02:0x17:Completed 1350000 out of 5000000 steps (27%)
16:42:23:WU00:FS01:0x17:Completed 4150000 out of 5000000 steps (83%)
16:42:46:WU01:FS02:0x17:Completed 1400000 out of 5000000 steps (28%)
16:44:01:WU00:FS01:0x17:Completed 4200000 out of 5000000 steps (84%)
16:44:33:WU01:FS02:0x17:Completed 1450000 out of 5000000 steps (29%)
16:45:40:WU00:FS01:0x17:Completed 4250000 out of 5000000 steps (85%)
16:46:20:WU01:FS02:0x17:Completed 1500000 out of 5000000 steps (30%)
16:47:19:WU00:FS01:0x17:Completed 4300000 out of 5000000 steps (86%)
16:48:07:WU01:FS02:0x17:Completed 1550000 out of 5000000 steps (31%)
16:48:58:WU00:FS01:0x17:Completed 4350000 out of 5000000 steps (87%)
16:49:54:WU01:FS02:0x17:Completed 1600000 out of 5000000 steps (32%)
16:50:37:WU00:FS01:0x17:Completed 4400000 out of 5000000 steps (88%)
16:51:41:WU01:FS02:0x17:Completed 1650000 out of 5000000 steps (33%)
16:52:16:WU00:FS01:0x17:Completed 4450000 out of 5000000 steps (89%)
16:53:28:WU01:FS02:0x17:Completed 1700000 out of 5000000 steps (34%)
16:53:55:WU00:FS01:0x17:Completed 4500000 out of 5000000 steps (90%)
16:55:15:WU01:FS02:0x17:Completed 1750000 out of 5000000 steps (35%)
16:55:33:WU00:FS01:0x17:Completed 4550000 out of 5000000 steps (91%)
16:57:02:WU01:FS02:0x17:Completed 1800000 out of 5000000 steps (36%)
16:57:12:WU00:FS01:0x17:Completed 4600000 out of 5000000 steps (92%)
16:58:49:WU01:FS02:0x17:Completed 1850000 out of 5000000 steps (37%)
16:58:51:WU00:FS01:0x17:Completed 4650000 out of 5000000 steps (93%)
17:00:30:WU00:FS01:0x17:Completed 4700000 out of 5000000 steps (94%)
17:00:36:WU01:FS02:0x17:Completed 1900000 out of 5000000 steps (38%)
17:02:09:WU00:FS01:0x17:Completed 4750000 out of 5000000 steps (95%)
17:02:23:WU01:FS02:0x17:Completed 1950000 out of 5000000 steps (39%)
17:03:48:WU00:FS01:0x17:Completed 4800000 out of 5000000 steps (96%)
17:04:10:WU01:FS02:0x17:Completed 2000000 out of 5000000 steps (40%)
17:05:27:WU00:FS01:0x17:Completed 4850000 out of 5000000 steps (97%)
17:05:57:WU01:FS02:0x17:Completed 2050000 out of 5000000 steps (41%)
17:07:06:WU00:FS01:0x17:Completed 4900000 out of 5000000 steps (98%)
17:07:44:WU01:FS02:0x17:Completed 2100000 out of 5000000 steps (42%)
17:08:44:WU00:FS01:0x17:Completed 4950000 out of 5000000 steps (99%)
17:09:31:WU01:FS02:0x17:Completed 2150000 out of 5000000 steps (43%)
17:10:23:WU00:FS01:0x17:Completed 5000000 out of 5000000 steps (100%)
17:10:25:WU00:FS01:0x17:Saving result file logfile_01.txt
17:10:25:WU00:FS01:0x17:Saving result file checkpointState.xml
17:10:25:WU02:FS01:Connecting to 171.67.108.200:80
17:10:25:WU00:FS01:0x17:Saving result file checkpt.crc
17:10:25:WU00:FS01:0x17:Saving result file log.txt
17:10:25:WU00:FS01:0x17:Saving result file positions.xtc
17:10:26:WARNING:WU02:FS01:Failed to get assignment from '171.67.108.200:80': Empty work server assignment
17:10:26:WU02:FS01:Connecting to 171.67.108.204:80
17:10:27:WU00:FS01:0x17:Folding@home Core Shutdown: FINISHED_UNIT
17:10:27:WU00:FS01:FahCore returned: FINISHED_UNIT (100 = 0x64)
17:10:27:WU00:FS01:Sending unit results: id:00 state:SEND error:NO_ERROR project:9201 run:832 clone:3 gen:191 core:0x17 unit:0x000001186652edc45399f6c3346ce98f
17:10:27:WU00:FS01:Uploading 8.37MiB to 171.67.108.52
17:10:27:WU02:FS01:Assigned to work server 171.67.108.52
17:10:27:WU00:FS01:Connecting to 171.67.108.52:8080
17:10:27:WU02:FS01:Requesting new work unit for slot 01: READY gpu:0:GM204 [GeForce GTX 970] from 171.67.108.52
17:10:27:WU02:FS01:Connecting to 171.67.108.52:8080
17:10:28:WU02:FS01:Downloading 1.52MiB
17:10:34:WU02:FS01:Download 57.45%
17:10:36:WU02:FS01:Download complete
17:10:36:WU02:FS01:Received Unit: id:02 state:DOWNLOAD error:NO_ERROR project:9201 run:263 clone:0 gen:464 core:0x17 unit:0x000002726652edc45399e0579b41ae21
17:10:36:WU02:FS01:Starting
17:10:36:WU02:FS01:Running FahCore: /usr/bin/FAHCoreWrapper /home/jmrios/cores/web.stanford.edu/~pande/Linux/AMD64/NVIDIA/Fermi/Core_17.fah/FahCore_17 -dir 02 -suffix 01 -version 704 -lifeline 2476 -checkpoint 15 -gpu 0 -gpu-vendor nvidia
17:10:36:WU02:FS01:Started FahCore on PID 3388
17:10:36:WU02:FS01:Core PID:3392
17:10:36:WU02:FS01:FahCore 0x17 started
17:10:37:WU02:FS01:0x17:*********************** Log Started 2015-03-01T17:10:36Z ***********************
17:10:37:WU02:FS01:0x17:Project: 9201 (Run 263, Clone 0, Gen 464)
17:10:37:WU02:FS01:0x17:Unit: 0x000002726652edc45399e0579b41ae21
17:10:37:WU02:FS01:0x17:CPU: 0x00000000000000000000000000000000
17:10:37:WU02:FS01:0x17:Machine: 1
17:10:37:WU02:FS01:0x17:Reading tar file state.xml
17:10:37:WU02:FS01:0x17:Reading tar file system.xml
17:10:37:WU02:FS01:0x17:Reading tar file integrator.xml
17:10:37:WU02:FS01:0x17:Reading tar file core.xml
17:10:37:WU02:FS01:0x17:Digital signatures verified
17:10:52:WU02:FS01:0x17:Completed 0 out of 5000000 steps (0%)
17:11:00:WU00:FS01:Upload 0.75%
17:11:18:WU01:FS02:0x17:Completed 2200000 out of 5000000 steps (44%)
17:12:25:WU00:FS01:Upload 1.49%
17:12:25:WARNING:WU00:FS01:Exception: Failed to send results to work server: Transfer failed
17:12:25:WU00:FS01:Trying to send results to collection server
17:12:25:WU00:FS01:Uploading 8.37MiB to 171.65.103.160
17:12:25:WU00:FS01:Connecting to 171.65.103.160:8080
17:12:31:WU02:FS01:0x17:Completed 50000 out of 5000000 steps (1%)
17:12:31:WU00:FS01:Upload 5.97%
17:12:37:WU00:FS01:Upload 11.94%
17:12:43:WU00:FS01:Upload 17.17%
17:12:49:WU00:FS01:Upload 21.64%
17:12:55:WU00:FS01:Upload 28.36%
17:13:01:WU00:FS01:Upload 33.59%
17:13:05:WU01:FS02:0x17:Completed 2250000 out of 5000000 steps (45%)
17:13:07:WU00:FS01:Upload 38.81%
17:13:13:WU00:FS01:Upload 44.03%
17:13:19:WU00:FS01:Upload 49.26%
17:13:26:WU00:FS01:Upload 55.23%
17:13:32:WU00:FS01:Upload 60.45%
17:13:39:WU00:FS01:Upload 66.42%
17:13:45:WU00:FS01:Upload 72.40%
17:13:52:WU00:FS01:Upload 78.37%
17:13:58:WU00:FS01:Upload 83.59%
17:14:04:WU00:FS01:Upload 88.82%
17:14:09:WU02:FS01:0x17:Completed 100000 out of 5000000 steps (2%)
17:14:10:WU00:FS01:Upload 94.79%
17:14:16:WU00:FS01:Upload 100.00%
17:14:19:WU00:FS01:Upload complete
17:14:19:WU00:FS01:Server responded WORK_ACK (400)
17:14:19:WU00:FS01:Final credit estimate, 28363.00 points
17:14:19:WU00:FS01:Cleaning up
Re: 171.67.108.52
Posted: Sun Mar 01, 2015 7:03 pm
by jmrios
Just finished another 9201, same issue:
Code: Select all
18:51:11:WU01:FS02:0x17:Completed 5000000 out of 5000000 steps (100%)
18:51:12:WU01:FS02:0x17:Saving result file logfile_01.txt
18:51:12:WU01:FS02:0x17:Saving result file checkpointState.xml
18:51:13:WU01:FS02:0x17:Saving result file checkpt.crc
18:51:13:WU01:FS02:0x17:Saving result file log.txt
18:51:13:WU01:FS02:0x17:Saving result file positions.xtc
18:51:13:WU00:FS02:Connecting to 171.67.108.200:80
18:51:14:WU01:FS02:0x17:Folding@home Core Shutdown: FINISHED_UNIT
18:51:14:WU01:FS02:FahCore returned: FINISHED_UNIT (100 = 0x64)
18:51:14:WU01:FS02:Sending unit results: id:01 state:SEND error:NO_ERROR project:9201 run:310 clone:4 gen:262 core:0x17 unit:0x0000015a6652edc45399e24265c61366
18:51:14:WU01:FS02:Uploading 8.43MiB to 171.67.108.52
18:51:14:WU00:FS02:Assigned to work server 171.67.108.52
18:51:14:WU01:FS02:Connecting to 171.67.108.52:8080
18:51:14:WU00:FS02:Requesting new work unit for slot 02: READY gpu:1:GM204 [GeForce GTX 970] from 171.67.108.52
18:51:14:WU00:FS02:Connecting to 171.67.108.52:8080
18:51:15:WU00:FS02:Downloading 1.52MiB
18:51:16:WU02:FS01:0x17:Completed 3050000 out of 5000000 steps (61%)
18:51:21:WU00:FS02:Download 45.12%
18:51:23:WU00:FS02:Download complete
18:51:23:WU00:FS02:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:9201 run:309 clone:4 gen:166 core:0x17 unit:0x000000ef6652edc45399e23859dcf41a
18:51:24:WU00:FS02:Starting
18:51:24:WU00:FS02:Running FahCore: /usr/bin/FAHCoreWrapper /home/jmrios/cores/web.stanford.edu/~pande/Linux/AMD64/NVIDIA/Fermi/Core_17.fah/FahCore_17 -dir 00 -suffix 01 -version 704 -lifeline 2476 -checkpoint 15 -gpu 1 -gpu-vendor nvidia
18:51:24:WU00:FS02:Started FahCore on PID 3674
18:51:24:WU00:FS02:Core PID:3678
18:51:24:WU00:FS02:FahCore 0x17 started
18:51:24:WU00:FS02:0x17:*********************** Log Started 2015-03-01T18:51:24Z ***********************
18:51:24:WU00:FS02:0x17:Project: 9201 (Run 309, Clone 4, Gen 166)
18:51:24:WU00:FS02:0x17:Unit: 0x000000ef6652edc45399e23859dcf41a
18:51:24:WU00:FS02:0x17:CPU: 0x00000000000000000000000000000000
18:51:24:WU00:FS02:0x17:Machine: 2
18:51:24:WU00:FS02:0x17:Reading tar file state.xml
18:51:24:WU00:FS02:0x17:Reading tar file system.xml
18:51:24:WU00:FS02:0x17:Reading tar file integrator.xml
18:51:24:WU00:FS02:0x17:Reading tar file core.xml
18:51:24:WU00:FS02:0x17:Digital signatures verified
18:51:26:WU01:FS02:Upload 0.74%
18:51:37:WU00:FS02:0x17:Completed 0 out of 5000000 steps (0%)
18:52:10:WU01:FS02:Upload 1.48%
18:52:34:WU01:FS02:Upload 2.22%
18:52:55:WU02:FS01:0x17:Completed 3100000 out of 5000000 steps (62%)
18:53:01:WU01:FS02:Upload 2.97%
18:53:23:WU00:FS02:0x17:Completed 50000 out of 5000000 steps (1%)
18:53:41:WU01:FS02:Upload 3.71%
18:54:34:WU02:FS01:0x17:Completed 3150000 out of 5000000 steps (63%)
18:54:47:WU01:FS02:Upload 4.45%
18:55:10:WU00:FS02:0x17:Completed 100000 out of 5000000 steps (2%)
18:55:57:WU01:FS02:Upload 5.19%
18:56:12:WU02:FS01:0x17:Completed 3200000 out of 5000000 steps (64%)
18:56:57:WU00:FS02:0x17:Completed 150000 out of 5000000 steps (3%)
18:57:01:WU01:FS02:Upload 5.93%
18:57:02:WARNING:WU01:FS02:Exception: Failed to send results to work server: Transfer failed
18:57:02:WU01:FS02:Trying to send results to collection server
18:57:02:WU01:FS02:Uploading 8.43MiB to 171.65.103.160
18:57:02:WU01:FS02:Connecting to 171.65.103.160:8080
18:57:08:WU01:FS02:Upload 7.41%
18:57:15:WU01:FS02:Upload 14.09%
18:57:21:WU01:FS02:Upload 20.02%
18:57:27:WU01:FS02:Upload 25.21%
18:57:33:WU01:FS02:Upload 31.14%
18:57:39:WU01:FS02:Upload 37.07%
18:57:45:WU01:FS02:Upload 42.26%
18:57:51:WU01:FS02:Upload 48.19%
18:57:51:WU02:FS01:0x17:Completed 3250000 out of 5000000 steps (65%)
18:57:57:WU01:FS02:Upload 54.12%
18:58:03:WU01:FS02:Upload 60.79%
18:58:09:WU01:FS02:Upload 66.73%
18:58:15:WU01:FS02:Upload 73.40%
18:58:21:WU01:FS02:Upload 79.33%
18:58:27:WU01:FS02:Upload 85.26%
18:58:34:WU01:FS02:Upload 90.45%
18:58:41:WU01:FS02:Upload 97.86%
18:58:43:WU00:FS02:0x17:Completed 200000 out of 5000000 steps (4%)
18:58:45:WU01:FS02:Upload complete
18:58:46:WU01:FS02:Server responded WORK_ACK (400)
18:58:46:WU01:FS02:Final credit estimate, 35516.00 points
18:58:46:WU01:FS02:Cleaning up
The other server is receiving the WU " at the normal speed"
Re: 171.67.108.52
Posted: Fri Mar 27, 2015 10:42 am
by wilding2004
Server in "reject". 39 WU's available, but nothing being assigned to ATI. My Nvidia cards are however getting WU's from this server.
Re: 171.67.108.52
Posted: Fri Mar 27, 2015 1:11 pm
by bruce
1) The server is having trouble communinating with serverstat. If you look at the overnight history log, you'll find that some records show the name vsp13a (with data) and other records show the name VSP13 (with no data). The number of WUs varies in various reports, so SOMETHING is working, although at this point, serverstat is unreliable for that server. I'll notify jadeshi but I doubt anybody will be in the office for a few more hours.
2) Some projects work on all kinds of GPUs and some fail on one type but not another. The Pande Group restricts assignments that are likely to fail. This might be an explanation to your NVidia/ATI differences. Unfortunately I know of no way to tell which type of GPU can be assigned those 39 WUs without a detailed search of all the projects that might be assigned by that server.
Rewriting serverstat (making it more reliable) is on the list of projects needing attention from Development.
Re: 171.67.108.52
Posted: Fri Mar 27, 2015 5:47 pm
by jadeshi
The WS was having some issues communicating with the AS this morning for some unknown reason, but it just now came back online. Hopefully this fixes the "reject" issue from wilding2004's post. I'm still uncertain as to the cause for the slow upload speeds, however. It may just be due to the fact that this is one the few WS's that hosts core17 projects at the moment and so there's an abnormally high amount of traffic.
Re: 171.67.108.52
Posted: Mon May 18, 2015 7:56 pm
by qqqq
Got problem with this server again! So two 290 is idle right now.
Code: Select all
19:49:24:WARNING:WU00:FS00:Failed to get assignment from '171.67.108.200:80': Empty work server assignment
19:49:25:WU00:FS00:Assigned to work server 171.67.108.52
19:49:25:WU00:FS00:Requesting new work unit for slot 00: READY gpu:1:Hawaii [Radeon R9 200 Series] from 171.67.108.52
19:49:27:WU00:FS00:Downloading 1.52MiB
19:49:27:WARNING:WU01:FS01:Failed to get assignment from '171.67.108.200:80': Empty work server assignment
19:49:28:WU01:FS01:Assigned to work server 171.67.108.52
19:49:28:WU01:FS01:Requesting new work unit for slot 01: READY gpu:2:Caicos [AMD RADEON HD 6450] from 171.67.108.52
19:49:29:WU01:FS01:Downloading 1.53MiB
19:49:39:WU03:FS02:0x17:Completed 1950000 out of 5000000 steps (39%)
19:49:39:WU03:FS02:0x17:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
19:49:40:WU01:FS01:Download 4.10%
19:49:52:Saving configuration to config.xml
19:49:53:WU01:FS01:Download 8.19%
19:49:55:WU00:FS00:Download 4.12%
19:50:04:WU00:FS00:Download 8.23%
19:50:05:WU01:FS01:Download 12.29%
19:50:11:WU00:FS00:Download 12.35%
19:50:25:WU01:FS01:Download 16.39%
19:50:27:WU00:FS00:Download 16.46%
19:50:41:WU00:FS00:Download 20.58%
19:50:51:WU00:FS00:Download 24.69%
19:51:00:WU00:FS00:Download 28.81%
19:51:10:WU00:FS00:Download 32.92%
19:51:27:WU00:FS00:Download 37.04%
19:51:27:WU01:FS01:Download 20.48%
19:51:38:WU00:FS00:Download 41.15%
19:51:41:WU01:FS01:Download 24.58%
19:51:47:WU00:FS00:Download 45.27%
19:51:58:WU01:FS01:Download 28.67%
19:52:04:WU03:FS02:0x17:Completed 2000000 out of 5000000 steps (40%)
19:54:07:WU01:FS01:Download 32.77%
19:54:07:ERROR:WU01:FS01:Exception: Transfer failed
19:54:09:WARNING:WU01:FS01:Failed to get assignment from '171.67.108.200:80': Empty work server assignment
19:54:10:WARNING:WU01:FS01:Failed to get assignment from '171.67.108.204:80': Empty work server assignment
19:54:10:ERROR:WU01:FS01:Exception: Could not get an assignment
19:54:29:WU03:FS02:0x17:Completed 2050000 out of 5000000 steps (41%)
19:55:14:WARNING:WU01:FS01:Failed to get assignment from '171.67.108.200:80': Empty work server assignment
19:55:36:WARNING:WU01:FS01:Failed to get assignment from '171.67.108.204:80': Empty work server assignment
19:55:36:ERROR:WU01:FS01:Exception: Could not get an assignment
19:55:39:WU00:FS00:Download 47.23%
19:55:39:ERROR:WU00:FS00:Exception: Transfer failed
19:55:43:WARNING:WU00:FS00:Failed to get assignment from '171.67.108.200:80': Empty work server assignment
19:55:47:WARNING:WU00:FS00:Failed to get assignment from '171.67.108.204:80': Empty work server assignment
19:55:47:ERROR:WU00:FS00:Exception: Could not get an assignment
19:56:44:WARNING:WU00:FS00:Failed to get assignment from '171.67.108.200:80': Empty work server assignment
19:56:44:WARNING:WU00:FS00:Failed to get assignment from '171.67.108.204:80': Empty work server assignment
19:56:44:ERROR:WU00:FS00:Exception: Could not get an assignment
19:56:46:WARNING:WU01:FS01:Failed to get assignment from '171.67.108.200:80': Empty work server assignment
19:56:51:WARNING:WU01:FS01:Failed to get assignment from '171.67.108.204:80': Empty work server assignment
19:56:51:ERROR:WU01:FS01:Exception: Could not get an assignment
19:56:54:WU03:FS02:0x17:Completed 2100000 out of 5000000 steps (42%)
Re: 171.67.108.52
Posted: Tue May 19, 2015 1:59 am
by jadeshi
Thanks for bringing this up. Judging from the logs, it looks like the server is running out of WU's again, most likely because I think this is one of few servers, if not the only, left that still hosts x17 units and is getting all the WU requests from those donors. I'm creating more jobs on the WS right now that should be available to donors by tomorrow that should fix this issue.