Page 1 of 1
140.163.4.233 super slow downloads some time out
Posted: Wed Aug 02, 2017 11:34 pm
by jimerickson
Code: Select all
*********************** Log Started 2017-08-02T23:25:31Z ***********************
23:25:31:************************* Folding@home Client *************************
23:25:31: Website: http://folding.stanford.edu/
23:25:31: Copyright: (c) 2009-2016 Stanford University
23:25:31: Author: Joseph Coffland <joseph@cauldrondevelopment.com>
23:25:31: Args:
23:25:31: Config: /home/jim/fah/config.xml
23:25:31:******************************** Build ********************************
23:25:31: Version: 7.4.16
23:25:31: Date: Jan 6 2017
23:25:31: Time: 08:08:33
23:25:31: Repository: Git
23:25:31: Revision: e12187cbb0bd6937c067b9749af011374563b7b9
23:25:31: Branch: master
23:25:31: Compiler: GNU 4.9.2
23:25:31: Options: -std=gnu++98 -O3 -funroll-loops -ffast-math -mfpmath=sse
23:25:31: -fno-unsafe-math-optimizations -msse2
23:25:31: Platform: linux2 4.8.0-2-amd64
23:25:31: Bits: 64
23:25:31: Mode: Release
23:25:31:******************************* System ********************************
23:25:31: CPU: Intel(R) Core(TM) i7-4960X CPU @ 3.60GHz
23:25:31: CPU ID: GenuineIntel Family 6 Model 62 Stepping 4
23:25:31: CPUs: 12
23:25:31: Memory: 31.36GiB
23:25:31: Free Memory: 29.12GiB
23:25:31: Threads: POSIX_THREADS
23:25:31: OS Version: 4.4
23:25:31: Has Battery: false
23:25:31: On Battery: false
23:25:31: UTC Offset: -5
23:25:31: PID: 1994
23:25:31: CWD: /home/jim/fah
23:25:31: OS: Linux 4.4.0-87-generic x86_64
23:25:31: OS Arch: AMD64
23:25:31: GPUs: 3
23:25:31: GPU 0: Bus:8 Slot:0 Func:0 NVIDIA:7 GP104 [GeForce GTX 1080] 8873
23:25:31: GPU 1: Bus:4 Slot:0 Func:0 NVIDIA:7 GP104 [GeForce GTX 1080] 8873
23:25:31: GPU 2: Bus:3 Slot:0 Func:0 NVIDIA:7 GP104 [GeForce GTX 1080] 8873
23:25:31: CUDA Device 0: Platform:0 Device:0 Bus:8 Slot:0 Compute:6.1 Driver:8.0
23:25:31: CUDA Device 1: Platform:0 Device:1 Bus:4 Slot:0 Compute:6.1 Driver:8.0
23:25:31: CUDA Device 2: Platform:0 Device:2 Bus:3 Slot:0 Compute:6.1 Driver:8.0
23:25:31:OpenCL Device 0: Platform:0 Device:0 Bus:8 Slot:0 Compute:1.2 Driver:375.66
23:25:31:OpenCL Device 1: Platform:0 Device:1 Bus:4 Slot:0 Compute:1.2 Driver:375.66
23:25:31:OpenCL Device 2: Platform:0 Device:2 Bus:3 Slot:0 Compute:1.2 Driver:375.66
23:25:31:***********************************************************************
23:25:31:<config>
23:25:31: <!-- Folding Slot Configuration -->
23:25:31: <client-type v='beta'/>
23:25:31:
23:25:31: <!-- HTTP Server -->
23:25:31: <allow v='127.0.0.1 192.168.1.0/24'/>
23:25:31:
23:25:31: <!-- Network -->
23:25:31: <proxy v=':8080'/>
23:25:31:
23:25:31: <!-- Remote Command Server -->
23:25:31: <password v='******'/>
23:25:31:
23:25:31: <!-- Slot Control -->
23:25:31: <power v='full'/>
23:25:31:
23:25:31: <!-- User Information -->
23:25:31: <passkey v='********************************'/>
23:25:31: <team v='172800'/>
23:25:31: <user v='jimerickson'/>
23:25:31:
23:25:31: <!-- Folding Slots -->
23:25:31: <slot id='1' type='GPU'>
23:25:31: <paused v='true'/>
23:25:31: </slot>
23:25:31: <slot id='2' type='GPU'>
23:25:31: <paused v='true'/>
23:25:31: </slot>
23:25:31: <slot id='0' type='GPU'>
23:25:31: <paused v='true'/>
23:25:31: </slot>
23:25:31:</config>
23:25:31:Trying to access database...
23:25:31:Successfully acquired database lock
23:25:31:Enabled folding slot 01: PAUSED gpu:0:GP104 [GeForce GTX 1080] 8873 (by user)
23:25:31:Enabled folding slot 02: PAUSED gpu:1:GP104 [GeForce GTX 1080] 8873 (by user)
23:25:31:Enabled folding slot 00: PAUSED gpu:2:GP104 [GeForce GTX 1080] 8873 (by user)
23:25:42:FS01:Unpaused
23:25:42:FS02:Unpaused
23:25:42:FS00:Unpaused
23:25:42:WU02:FS00:Starting
23:25:42:WU02:FS00:Running FahCore: /home/jim/fah/FAHCoreWrapper /home/jim/fah/cores/fahwebx.stanford.edu/cores/Linux/AMD64/NVIDIA/Fermi/beta/Core_21.fah/FahCore_21 -dir 02 -suffix 01 -version 704 -lifeline 1994 -checkpoint 15 -gpu-vendor nvidia -opencl-platform 0 -opencl-device 2 -cuda-device 2 -gpu 2
23:25:42:WU02:FS00:Started FahCore on PID 2016
23:25:42:WU02:FS00:Core PID:2020
23:25:42:WU02:FS00:FahCore 0x21 started
23:25:42:WU00:FS01:Connecting to 171.67.108.45:80
23:25:43:WU02:FS00:0x21:*********************** Log Started 2017-08-02T23:25:42Z ***********************
23:25:43:WU02:FS00:0x21:Project: 10496 (Run 163, Clone 73, Gen 40)
23:25:43:WU02:FS00:0x21:Unit: 0x0000002d8ca304f5588983cd4577bf4f
23:25:43:WU02:FS00:0x21:CPU: 0x00000000000000000000000000000000
23:25:43:WU02:FS00:0x21:Machine: 0
23:25:43:WU02:FS00:0x21:Digital signatures verified
23:25:43:WU02:FS00:0x21:Folding@home GPU Core21 Folding@home Core
23:25:43:WU02:FS00:0x21:Version 0.0.20
23:25:43:WU02:FS00:0x21: Found a checkpoint file
23:25:43:WU01:FS02:Connecting to 171.67.108.45:80
23:25:43:WU00:FS01:Assigned to work server 140.163.4.245
23:25:43:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:GP104 [GeForce GTX 1080] 8873 from 140.163.4.245
23:25:43:WU00:FS01:Connecting to 140.163.4.245:8080
23:25:43:WU01:FS02:Assigned to work server 140.163.4.232
23:25:43:WU01:FS02:Requesting new work unit for slot 02: READY gpu:1:GP104 [GeForce GTX 1080] 8873 from 140.163.4.232
23:25:43:WU01:FS02:Connecting to 140.163.4.232:8080
23:25:46:WU00:FS01:Downloading 14.47MiB
23:25:46:WU01:FS02:Downloading 30.53MiB
23:25:52:WU00:FS01:Download 2.59%
23:25:52:WU01:FS02:Download 1.02%
23:25:56:WU02:FS00:0x21:Completed 375000 out of 2000000 steps (18%)
23:25:56:WU02:FS00:0x21:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
23:25:58:WU01:FS02:Download 2.05%
23:25:58:WU00:FS01:Download 6.91%
23:26:04:WU00:FS01:Download 10.37%
23:26:04:WU01:FS02:Download 3.89%
23:26:10:WU01:FS02:Download 5.94%
23:26:10:WU00:FS01:Download 13.82%
23:26:16:WU01:FS02:Download 7.98%
23:26:16:WU00:FS01:Download 17.28%
23:26:22:WU00:FS01:Download 22.46%
23:26:22:WU01:FS02:Download 10.85%
23:26:26:WU02:FS00:0x21:Completed 380000 out of 2000000 steps (19%)
23:26:28:WU01:FS02:Download 12.49%
23:26:29:WU00:FS01:Download 26.35%
23:26:32:Removing old file 'configs/config-20170526-220725.xml'
23:26:32:Saving configuration to config.xml
23:26:32:<config>
23:26:32: <!-- Folding Slot Configuration -->
23:26:32: <client-type v='beta'/>
23:26:32:
23:26:32: <!-- HTTP Server -->
23:26:32: <allow v='127.0.0.1 192.168.1.0/24'/>
23:26:32:
23:26:32: <!-- Network -->
23:26:32: <proxy v=':8080'/>
23:26:32:
23:26:32: <!-- Remote Command Server -->
23:26:32: <password v='******'/>
23:26:32:
23:26:32: <!-- Slot Control -->
23:26:32: <power v='full'/>
23:26:32:
23:26:32: <!-- User Information -->
23:26:32: <passkey v='********************************'/>
23:26:32: <team v='172800'/>
23:26:32: <user v='jimerickson'/>
23:26:32:
23:26:32: <!-- Folding Slots -->
23:26:32: <slot id='1' type='GPU'/>
23:26:32: <slot id='2' type='GPU'/>
23:26:32: <slot id='0' type='GPU'/>
23:26:32:</config>
23:26:35:WU00:FS01:Download 27.21%
23:26:35:WU01:FS02:Download 13.10%
23:26:43:WU01:FS02:Download 13.31%
23:26:44:WU00:FS01:Download 27.64%
23:26:51:WU01:FS02:Download 13.51%
23:26:55:WU00:FS01:Download 28.07%
23:26:57:WU01:FS02:Download 14.53%
23:27:01:WU00:FS01:Download 29.37%
23:27:03:WU01:FS02:Download 14.94%
23:27:18:WU00:FS01:Download 29.80%
23:27:22:WU01:FS02:Download 15.15%
23:27:25:WU00:FS01:Download 31.53%
23:27:28:WU01:FS02:Download 15.97%
23:27:31:WU00:FS01:Download 34.55%
23:27:38:WU01:FS02:Download 17.40%
23:27:38:WU00:FS01:Download 36.71%
23:27:45:WU00:FS01:Download 38.44%
23:27:46:WU01:FS02:Download 18.42%
23:27:51:WU00:FS01:Download 44.92%
23:27:52:WU01:FS02:Download 19.86%
23:27:57:WU00:FS01:Download 51.40%
23:27:58:WU01:FS02:Download 22.31%
23:28:03:WU00:FS01:Download 53.99%
23:28:04:WU01:FS02:Download 24.16%
23:28:27:WU02:FS00:0x21:Completed 400000 out of 2000000 steps (20%)
Re: 140.163.4.233 super slow downloads some time out
Posted: Thu Aug 03, 2017 2:28 am
by bruce
I don't see any timeouts in what you posted.
WU01:FS02 ... 140.163.4.232
downloads about 24% of 14.47MiB from 23:28:04 to 23:25:46 or 2m18s which is 3.47 MB in 138s which is 27,7 Mb/s
WU00:FS01 ... 140.163.4.245
downloads about 54% of 30.53MiB from 23:25:43 to 23:28:03 or 2m20s which is 16.5 MB in 140s which is 132 Mb/s
These two downloads are both going on at the same time, from different VMs on the same server and through your ISP connection.
Have you measured the actual bandwidth of your internet connection?
According to the information I have, 140.163.4.232 is servicing downloads at and assign rate of ~380
and 140.163.4.232 is servicing downloads at an assign rate of ~600
Re: 140.163.4.233 super slow downloads some time out
Posted: Thu Aug 03, 2017 3:21 am
by jimerickson
it was an example of a slow download, now quit obfuscating. if you want more logs just say so.
Re: 140.163.4.233 super slow downloads some time out
Posted: Thu Aug 03, 2017 3:38 am
by jimerickson
Code: Select all
*********************** Log Started 2017-08-03T03:29:31Z ***********************
03:29:31:************************* Folding@home Client *************************
03:29:31: Website: http://folding.stanford.edu/
03:29:31: Copyright: (c) 2009-2016 Stanford University
03:29:31: Author: Joseph Coffland <joseph@cauldrondevelopment.com>
03:29:31: Args:
03:29:31: Config: /home/jim/fah/config.xml
03:29:31:******************************** Build ********************************
03:29:31: Version: 7.4.16
03:29:31: Date: Jan 6 2017
03:29:31: Time: 08:08:33
03:29:31: Repository: Git
03:29:31: Revision: e12187cbb0bd6937c067b9749af011374563b7b9
03:29:31: Branch: master
03:29:31: Compiler: GNU 4.9.2
03:29:31: Options: -std=gnu++98 -O3 -funroll-loops -ffast-math -mfpmath=sse
03:29:31: -fno-unsafe-math-optimizations -msse2
03:29:31: Platform: linux2 4.8.0-2-amd64
03:29:31: Bits: 64
03:29:31: Mode: Release
03:29:31:******************************* System ********************************
03:29:31: CPU: Intel(R) Core(TM) i7-4960X CPU @ 3.60GHz
03:29:31: CPU ID: GenuineIntel Family 6 Model 62 Stepping 4
03:29:31: CPUs: 12
03:29:31: Memory: 31.36GiB
03:29:31: Free Memory: 28.73GiB
03:29:31: Threads: POSIX_THREADS
03:29:31: OS Version: 4.4
03:29:31: Has Battery: false
03:29:31: On Battery: false
03:29:31: UTC Offset: -5
03:29:31: PID: 12865
03:29:31: CWD: /home/jim/fah
03:29:31: OS: Linux 4.4.0-87-generic x86_64
03:29:31: OS Arch: AMD64
03:29:31: GPUs: 3
03:29:31: GPU 0: Bus:8 Slot:0 Func:0 NVIDIA:7 GP104 [GeForce GTX 1080] 8873
03:29:31: GPU 1: Bus:4 Slot:0 Func:0 NVIDIA:7 GP104 [GeForce GTX 1080] 8873
03:29:31: GPU 2: Bus:3 Slot:0 Func:0 NVIDIA:7 GP104 [GeForce GTX 1080] 8873
03:29:31: CUDA Device 0: Platform:0 Device:0 Bus:8 Slot:0 Compute:6.1 Driver:8.0
03:29:31: CUDA Device 1: Platform:0 Device:1 Bus:4 Slot:0 Compute:6.1 Driver:8.0
03:29:31: CUDA Device 2: Platform:0 Device:2 Bus:3 Slot:0 Compute:6.1 Driver:8.0
03:29:31:OpenCL Device 0: Platform:0 Device:0 Bus:8 Slot:0 Compute:1.2 Driver:375.66
03:29:31:OpenCL Device 1: Platform:0 Device:1 Bus:4 Slot:0 Compute:1.2 Driver:375.66
03:29:31:OpenCL Device 2: Platform:0 Device:2 Bus:3 Slot:0 Compute:1.2 Driver:375.66
03:29:31:***********************************************************************
03:29:31:<config>
03:29:31: <!-- Folding Slot Configuration -->
03:29:31: <client-type v='beta'/>
03:29:31:
03:29:31: <!-- HTTP Server -->
03:29:31: <allow v='127.0.0.1 192.168.1.0/24'/>
03:29:31:
03:29:31: <!-- Network -->
03:29:31: <proxy v=':8080'/>
03:29:31:
03:29:31: <!-- Remote Command Server -->
03:29:31: <password v='******'/>
03:29:31:
03:29:31: <!-- Slot Control -->
03:29:31: <power v='full'/>
03:29:31:
03:29:31: <!-- User Information -->
03:29:31: <passkey v='********************************'/>
03:29:31: <team v='172800'/>
03:29:31: <user v='jimerickson'/>
03:29:31:
03:29:31: <!-- Folding Slots -->
03:29:31: <slot id='0' type='GPU'/>
03:29:31: <slot id='1' type='GPU'/>
03:29:31: <slot id='2' type='GPU'/>
03:29:31:</config>
03:29:31:Trying to access database...
03:29:31:Successfully acquired database lock
03:29:31:Enabled folding slot 00: READY gpu:0:GP104 [GeForce GTX 1080] 8873
03:29:31:Enabled folding slot 01: READY gpu:1:GP104 [GeForce GTX 1080] 8873
03:29:31:Enabled folding slot 02: READY gpu:2:GP104 [GeForce GTX 1080] 8873
03:29:31:WU00:FS01:Starting
03:29:31:WU00:FS01:Running FahCore: /home/jim/fah/FAHCoreWrapper /home/jim/fah/cores/fahwebx.stanford.edu/cores/Linux/AMD64/NVIDIA/Fermi/beta/Core_21.fah/FahCore_21 -dir 00 -suffix 01 -version 704 -lifeline 12865 -checkpoint 15 -gpu-vendor nvidia -opencl-platform 0 -opencl-device 1 -cuda-device 1 -gpu 1
03:29:31:WU00:FS01:Started FahCore on PID 12879
03:29:31:WU00:FS01:Core PID:12883
03:29:31:WU00:FS01:FahCore 0x21 started
03:29:31:WU00:FS01:0x21:*********************** Log Started 2017-08-03T03:29:31Z ***********************
03:29:31:WU00:FS01:0x21:Project: 9431 (Run 1096, Clone 0, Gen 173)
03:29:31:WU00:FS01:0x21:Unit: 0x000000d5ab436c9d586fdd3cecc513aa
03:29:31:WU00:FS01:0x21:CPU: 0x00000000000000000000000000000000
03:29:31:WU00:FS01:0x21:Machine: 1
03:29:31:WU00:FS01:0x21:Digital signatures verified
03:29:31:WU00:FS01:0x21:Folding@home GPU Core21 Folding@home Core
03:29:31:WU00:FS01:0x21:Version 0.0.20
03:29:31:WU00:FS01:0x21: Found a checkpoint file
03:29:32:WU00:FS01:0x21:Completed 3800000 out of 6250000 steps (60%)
03:29:32:WU00:FS01:0x21:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
03:29:38:WU01:FS00:Connecting to 171.67.108.45:80
03:29:38:WU02:FS02:Connecting to 171.67.108.45:80
03:29:39:WU01:FS00:Assigned to work server 140.163.4.232
03:29:39:WU01:FS00:Requesting new work unit for slot 00: READY gpu:0:GP104 [GeForce GTX 1080] 8873 from 140.163.4.232
03:29:39:WU02:FS02:Assigned to work server 140.163.4.245
03:29:39:WU01:FS00:Connecting to 140.163.4.232:8080
03:29:39:WU02:FS02:Requesting new work unit for slot 02: READY gpu:2:GP104 [GeForce GTX 1080] 8873 from 140.163.4.245
03:29:39:WU02:FS02:Connecting to 140.163.4.245:8080
03:29:40:WU02:FS02:Downloading 14.49MiB
03:29:41:WU01:FS00:Downloading 12.21MiB
03:29:44:WU00:FS01:0x21:Completed 3812500 out of 6250000 steps (61%)
03:30:07:WU01:FS00:Download 0.51%
03:30:08:WU02:FS02:Download 0.43%
03:30:43:WU00:FS01:0x21:Completed 3875000 out of 6250000 steps (62%)
03:31:13:WU02:FS02:Download 0.86%
03:31:13:ERROR:WU02:FS02:Exception: Transfer failed
03:31:13:WU02:FS02:Connecting to 171.67.108.45:80
03:31:14:WU02:FS02:Assigned to work server 171.67.108.157
03:31:14:WU02:FS02:Requesting new work unit for slot 02: READY gpu:2:GP104 [GeForce GTX 1080] 8873 from 171.67.108.157
03:31:14:WU02:FS02:Connecting to 171.67.108.157:8080
03:31:14:WU02:FS02:Downloading 5.15MiB
03:31:21:WU02:FS02:Download 1.21%
03:31:41:WU02:FS02:Download 2.43%
03:31:43:WU00:FS01:0x21:Completed 3937500 out of 6250000 steps (63%)
this one failed to 140.163.4.245:8080. same set of mskcc servers. now why don't you contact chodera's people and see about having it fixed.
Re: 140.163.4.233 super slow downloads some time out
Posted: Thu Aug 03, 2017 7:08 am
by rwh202
Something off in Bruce's calcs too - try 25 and 135 kB/s
I've seen plenty of slow downloads, but would need to do some digging to see if there was a pattern of particular guilty servers.
Taking nearly 10 minutes to download a 14.47 MB WU is ridiculous though. It's probably adding 10% to the turnaround time for the WU (assuming upload then goes at a sensible rate). Given the efforts that thousands of volunteers go to to eek out 1% improvements, perhaps it's worth the project taking a look at that server configuration / logs and checking if something needs optimising.
Re: 140.163.4.233 super slow downloads some time out
Posted: Sun Aug 13, 2017 7:28 pm
by bruce
OK. Fix my calculations, but the issue here is how long a connection can be interrupted.
In the log, there are multiple messages from each connection -- typically every few seconds or less. Once there's a long pause (say a minute or so) for whatever reason, it's fair to assume that connection is hung and FAHClient will not restart the connection until you restart FAHClient.
As I've said elsewhere, when an internet connection hangs, it's impossible to tell whether it's a problem at the server or a problem with your ISP or somewhere in-between. If you're having repeated problems, the problem is likely on your end. If multiple people are seeing problems with the same server(s), then the problem is probably on the server's end. Restarting the computer on whichever end is having the problems is the only known "cure" without a deeper investigation.