Page 4 of 6
Re: Slow WU downloads & failed transfer [140.163.4.231]
Posted: Wed Jul 25, 2018 12:23 am
by lknl
bruce wrote:It sounds like you're trying to somehow overcome the Windows restriction that GPUs can only be used by FAH when the FAHCore is running in the memory-space of the logged on user by (re-)starting FAHClient when user A logs off an user B logs on. I don't know if anybody has done that successfully. I'm sure that Microsoft had a reason to build that restriction into Windows.
Have you though about Linux -- which is designed with genuine multi-user support built in.
actually I'm the only user, and my intention is to let FAHClient run without waiting until I finish logging in, because sometimes I start my PC and go away to do other things without waiting for Windows to finish booting.
When setting up the scheduled task, I need to fill in my logon info, probably that's how the GPU client can start normally.
At the moment, I don't have plan to switch to Linux.
Re: Slow WU downloads & failed transfer [140.163.4.231]
Posted: Wed Jul 25, 2018 12:53 am
by SteveWillis
Unless you really need one, just remove your password and windows will automatically log you in at boot. I know that works with windows 10, not sure about other versions. Another thing you might can do is go into your bios and set your pc to automatically reboot after a power failure when power is restored.
Re: Slow WU downloads & failed transfer [140.163.4.231]
Posted: Wed Jul 25, 2018 1:12 am
by lknl
SteveWillis wrote:Unless you really need one, just remove your password and windows will automatically log you in at boot. I know that works with windows 10, not sure about other versions. Another thing you might can do is go into your bios and set your pc to automatically reboot after a power failure when power is restored.
I remember a few years ago I ran into problem while sharing files with other PCs in local network because of blank password even when I turned off password protected sharing, so I set one.
Re: Slow WU downloads & failed transfer [140.163.4.231]
Posted: Wed Jul 25, 2018 1:29 am
by lknl
It seems like the setting max_packet_size=9 has little effect for me.
GPU-only log:
Code: Select all
*********************** Log Started 2018-07-24T17:07:56Z ***********************
17:07:56:WU01:FS01:Connecting to 65.254.110.245:8080
17:07:57:WU01:FS01:Assigned to work server 140.163.4.231
17:07:57:WU01:FS01:Requesting new work unit for slot 01: READY gpu:0:GP102 [GeForce GTX 1080 Ti] 11380 from 140.163.4.231
17:07:57:WU01:FS01:Connecting to 140.163.4.231:8080
17:07:59:WU01:FS01:Downloading 5.42MiB
17:08:11:WU01:FS01:Download 1.15%
17:08:24:WU01:FS01:Download 2.31%
17:08:37:WU01:FS01:Download 3.46%
17:08:50:WU01:FS01:Download 4.61%
17:09:02:WU01:FS01:Download 5.77%
17:09:14:WU01:FS01:Download 6.92%
17:09:26:WU01:FS01:Download 8.08%
17:09:38:WU01:FS01:Download 9.23%
17:09:51:WU01:FS01:Download 10.38%
17:10:04:WU01:FS01:Download 11.54%
17:10:17:WU01:FS01:Download 12.69%
17:10:29:WU01:FS01:Download 13.84%
17:10:41:WU01:FS01:Download 15.00%
17:10:54:WU01:FS01:Download 16.15%
17:11:29:WU01:FS01:Download 17.31%
17:12:09:WU01:FS01:Download 18.46%
17:12:51:WU01:FS01:Download 19.61%
17:13:33:WU01:FS01:Download 20.77%
17:14:14:WU01:FS01:Download 21.92%
17:14:52:WU01:FS01:Download 23.07%
17:15:33:WU01:FS01:Download 24.23%
17:16:16:WU01:FS01:Download 25.38%
17:16:58:WU01:FS01:Download 26.53%
17:17:42:WU01:FS01:Download 27.69%
17:18:24:WU01:FS01:Download 28.84%
17:19:06:WU01:FS01:Download 30.00%
17:19:46:WU01:FS01:Download 31.15%
17:20:26:WU01:FS01:Download 32.30%
17:21:06:WU01:FS01:Download 33.46%
17:21:46:WU01:FS01:Download 34.61%
17:22:28:WU01:FS01:Download 35.76%
17:23:10:WU01:FS01:Download 36.92%
17:23:53:WU01:FS01:Download 38.07%
17:24:35:WU01:FS01:Download 39.22%
17:25:16:WU01:FS01:Download 40.38%
17:25:59:WU01:FS01:Download 41.53%
17:26:38:WU01:FS01:Download 42.69%
17:27:20:WU01:FS01:Download 43.84%
17:28:04:WU01:FS01:Download 44.99%
17:28:46:WU01:FS01:Download 46.15%
17:29:29:WU01:FS01:Download 47.30%
17:30:12:WU01:FS01:Download 48.45%
17:30:54:WU01:FS01:Download 49.61%
17:31:37:WU01:FS01:Download 50.76%
17:32:18:WU01:FS01:Download 51.92%
17:32:53:WU01:FS01:Download 53.07%
17:33:32:WU01:FS01:Download 54.22%
17:34:16:WU01:FS01:Download 55.38%
17:35:00:WU01:FS01:Download 56.53%
17:35:44:WU01:FS01:Download 57.68%
17:36:29:WU01:FS01:Download 58.84%
17:37:14:WU01:FS01:Download 59.99%
17:38:00:WU01:FS01:Download 61.14%
17:38:44:WU01:FS01:Download 62.30%
17:39:29:WU01:FS01:Download 63.45%
17:40:13:WU01:FS01:Download 64.61%
17:40:58:WU01:FS01:Download 65.76%
17:41:42:WU01:FS01:Download 66.91%
17:42:26:WU01:FS01:Download 68.07%
17:43:11:WU01:FS01:Download 69.22%
17:43:55:WU01:FS01:Download 70.37%
17:44:39:WU01:FS01:Download 71.53%
17:45:24:WU01:FS01:Download 72.68%
17:46:08:WU01:FS01:Download 73.84%
17:46:53:WU01:FS01:Download 74.99%
17:47:38:WU01:FS01:Download 76.14%
17:48:22:WU01:FS01:Download 77.30%
17:49:07:WU01:FS01:Download 78.45%
17:49:50:WU01:FS01:Download 79.60%
17:50:35:WU01:FS01:Download 80.76%
17:51:19:WU01:FS01:Download 81.91%
17:52:03:WU01:FS01:Download 83.06%
17:52:48:WU01:FS01:Download 84.22%
17:53:01:WU01:FS01:Download 84.22%
17:53:01:ERROR:WU01:FS01:Exception: Transfer failed
17:53:02:WU01:FS01:Connecting to 65.254.110.245:8080
17:53:03:WU01:FS01:Assigned to work server 140.163.4.231
17:53:03:WU01:FS01:Requesting new work unit for slot 01: READY gpu:0:GP102 [GeForce GTX 1080 Ti] 11380 from 140.163.4.231
17:53:03:WU01:FS01:Connecting to 140.163.4.231:8080
17:53:04:WU01:FS01:Downloading 16.53MiB
17:53:16:WU01:FS01:Download 0.38%
17:53:29:WU01:FS01:Download 0.76%
17:53:42:WU01:FS01:Download 1.13%
17:53:55:WU01:FS01:Download 1.51%
17:54:08:WU01:FS01:Download 1.89%
17:54:20:WU01:FS01:Download 2.27%
17:54:33:WU01:FS01:Download 2.65%
17:54:46:WU01:FS01:Download 3.02%
17:54:59:WU01:FS01:Download 3.40%
17:55:12:WU01:FS01:Download 3.78%
17:55:24:WU01:FS01:Download 4.16%
17:55:37:WU01:FS01:Download 4.54%
17:55:50:WU01:FS01:Download 4.92%
17:56:03:WU01:FS01:Download 5.29%
17:56:16:WU01:FS01:Download 5.67%
17:56:29:WU01:FS01:Download 6.05%
17:56:41:WU01:FS01:Download 6.43%
17:56:54:WU01:FS01:Download 6.81%
17:57:07:WU01:FS01:Download 7.18%
17:57:20:WU01:FS01:Download 7.56%
17:57:33:WU01:FS01:Download 7.94%
17:57:45:WU01:FS01:Download 8.32%
17:57:58:WU01:FS01:Download 8.70%
17:58:11:WU01:FS01:Download 9.07%
17:58:24:WU01:FS01:Download 9.45%
17:58:37:WU01:FS01:Download 9.83%
17:58:50:WU01:FS01:Download 10.21%
17:59:02:WU01:FS01:Download 10.59%
17:59:15:WU01:FS01:Download 10.96%
17:59:28:WU01:FS01:Download 11.34%
17:59:41:WU01:FS01:Download 11.72%
17:59:54:WU01:FS01:Download 12.10%
18:00:06:WU01:FS01:Download 12.48%
18:00:19:WU01:FS01:Download 12.85%
18:00:32:WU01:FS01:Download 13.23%
18:00:45:WU01:FS01:Download 13.61%
18:00:58:WU01:FS01:Download 13.99%
18:01:11:WU01:FS01:Download 14.37%
18:01:23:WU01:FS01:Download 14.75%
18:01:36:WU01:FS01:Download 15.12%
18:01:49:WU01:FS01:Download 15.50%
18:02:02:WU01:FS01:Download 15.88%
18:02:15:WU01:FS01:Download 16.26%
18:02:28:WU01:FS01:Download 16.64%
18:02:40:WU01:FS01:Download 17.01%
18:02:53:WU01:FS01:Download 17.39%
18:03:06:WU01:FS01:Download 17.77%
18:03:19:WU01:FS01:Download 18.15%
18:03:32:WU01:FS01:Download 18.53%
18:03:45:WU01:FS01:Download 18.90%
18:03:57:WU01:FS01:Download 19.28%
18:04:10:WU01:FS01:Download 19.66%
18:04:23:WU01:FS01:Download 20.04%
18:04:36:WU01:FS01:Download 20.42%
18:04:49:WU01:FS01:Download 20.79%
18:05:02:WU01:FS01:Download 21.17%
18:05:14:WU01:FS01:Download 21.55%
18:05:27:WU01:FS01:Download 21.93%
18:05:40:WU01:FS01:Download 22.31%
18:05:53:WU01:FS01:Download 22.68%
18:06:06:WU01:FS01:Download 23.06%
18:06:18:WU01:FS01:Download 23.44%
18:06:31:WU01:FS01:Download 23.82%
18:06:44:WU01:FS01:Download 24.20%
18:06:57:WU01:FS01:Download 24.58%
18:07:10:WU01:FS01:Download 24.95%
18:07:22:WU01:FS01:Download 25.33%
18:07:35:WU01:FS01:Download 25.71%
18:07:48:WU01:FS01:Download 26.09%
18:08:01:WU01:FS01:Download 26.47%
18:08:14:WU01:FS01:Download 26.84%
18:08:26:WU01:FS01:Download 27.22%
18:08:39:WU01:FS01:Download 27.60%
18:08:52:WU01:FS01:Download 27.98%
18:09:05:WU01:FS01:Download 28.36%
18:09:18:WU01:FS01:Download 28.73%
18:09:31:WU01:FS01:Download 29.11%
18:09:43:WU01:FS01:Download 29.49%
18:09:56:WU01:FS01:Download 29.87%
18:10:09:WU01:FS01:Download 30.25%
18:10:22:WU01:FS01:Download 30.62%
18:10:35:WU01:FS01:Download 31.00%
18:10:47:WU01:FS01:Download 31.38%
18:11:00:WU01:FS01:Download 31.76%
18:11:13:WU01:FS01:Download 32.14%
18:11:26:WU01:FS01:Download 32.52%
18:11:39:WU01:FS01:Download 32.89%
18:11:51:WU01:FS01:Download 33.27%
18:12:04:WU01:FS01:Download 33.65%
18:12:17:WU01:FS01:Download 34.03%
18:12:30:WU01:FS01:Download 34.41%
18:12:43:WU01:FS01:Download 34.78%
18:12:56:WU01:FS01:Download 35.16%
18:13:08:WU01:FS01:Download 35.54%
18:13:21:WU01:FS01:Download 35.92%
18:13:34:WU01:FS01:Download 36.30%
18:13:47:WU01:FS01:Download 36.67%
18:14:00:WU01:FS01:Download 37.05%
18:14:13:WU01:FS01:Download 37.43%
18:14:25:WU01:FS01:Download 37.81%
18:14:38:WU01:FS01:Download 38.19%
18:14:51:WU01:FS01:Download 38.56%
18:15:04:WU01:FS01:Download 38.94%
18:15:17:WU01:FS01:Download 39.32%
18:15:30:WU01:FS01:Download 39.70%
18:15:42:WU01:FS01:Download 40.08%
18:15:55:WU01:FS01:Download 40.45%
18:16:08:WU01:FS01:Download 40.83%
18:16:21:WU01:FS01:Download 41.21%
18:16:34:WU01:FS01:Download 41.59%
18:16:47:WU01:FS01:Download 41.97%
18:16:59:WU01:FS01:Download 42.35%
18:17:12:WU01:FS01:Download 42.72%
18:17:25:WU01:FS01:Download 43.10%
18:17:38:WU01:FS01:Download 43.48%
18:17:50:WU01:FS01:Download 43.86%
18:18:03:WU01:FS01:Download 44.24%
18:18:16:WU01:FS01:Download 44.61%
18:18:29:WU01:FS01:Download 44.99%
18:18:42:WU01:FS01:Download 45.37%
18:18:55:WU01:FS01:Download 45.75%
18:19:07:WU01:FS01:Download 46.13%
18:19:20:WU01:FS01:Download 46.50%
18:19:33:WU01:FS01:Download 46.88%
18:19:46:WU01:FS01:Download 47.26%
18:19:59:WU01:FS01:Download 47.64%
18:20:12:WU01:FS01:Download 48.02%
18:20:25:WU01:FS01:Download 48.39%
18:20:38:WU01:FS01:Download 48.77%
18:20:51:WU01:FS01:Download 49.15%
18:21:03:WU01:FS01:Download 49.53%
18:21:16:WU01:FS01:Download 49.91%
18:21:29:WU01:FS01:Download 50.29%
18:21:42:WU01:FS01:Download 50.66%
18:21:55:WU01:FS01:Download 51.04%
18:22:07:WU01:FS01:Download 51.42%
18:22:20:WU01:FS01:Download 51.80%
18:22:33:WU01:FS01:Download 52.18%
18:22:46:WU01:FS01:Download 52.55%
18:22:59:WU01:FS01:Download 52.93%
18:23:12:WU01:FS01:Download 53.31%
18:23:25:WU01:FS01:Download 53.69%
18:23:37:WU01:FS01:Download 54.07%
18:23:50:WU01:FS01:Download 54.44%
18:24:03:WU01:FS01:Download 54.82%
18:24:16:WU01:FS01:Download 55.20%
18:24:28:WU01:FS01:Download 55.58%
18:24:41:WU01:FS01:Download 55.96%
18:24:54:WU01:FS01:Download 56.33%
18:25:07:WU01:FS01:Download 56.71%
18:25:20:WU01:FS01:Download 57.09%
18:25:33:WU01:FS01:Download 57.47%
18:25:45:WU01:FS01:Download 57.85%
18:25:58:WU01:FS01:Download 58.22%
18:26:11:WU01:FS01:Download 58.60%
18:26:24:WU01:FS01:Download 58.98%
18:26:37:WU01:FS01:Download 59.36%
18:26:50:WU01:FS01:Download 59.74%
18:27:02:WU01:FS01:Download 60.12%
18:27:15:WU01:FS01:Download 60.49%
18:27:28:WU01:FS01:Download 60.87%
18:27:41:WU01:FS01:Download 61.25%
18:27:54:WU01:FS01:Download 61.63%
18:28:06:WU01:FS01:Download 62.01%
18:28:19:WU01:FS01:Download 62.38%
18:28:32:WU01:FS01:Download 62.76%
18:28:45:WU01:FS01:Download 63.14%
18:28:58:WU01:FS01:Download 63.52%
18:29:10:WU01:FS01:Download 63.90%
18:29:23:WU01:FS01:Download 64.27%
18:29:36:WU01:FS01:Download 64.65%
18:29:49:WU01:FS01:Download 65.03%
18:30:02:WU01:FS01:Download 65.41%
18:30:15:WU01:FS01:Download 65.79%
18:30:27:WU01:FS01:Download 66.16%
18:30:40:WU01:FS01:Download 66.54%
18:30:53:WU01:FS01:Download 66.92%
18:31:06:WU01:FS01:Download 67.30%
18:31:19:WU01:FS01:Download 67.68%
18:31:32:WU01:FS01:Download 68.05%
18:31:44:WU01:FS01:Download 68.43%
18:31:57:WU01:FS01:Download 68.81%
18:32:10:WU01:FS01:Download 69.19%
18:32:23:WU01:FS01:Download 69.57%
18:32:36:WU01:FS01:Download 69.95%
18:32:48:WU01:FS01:Download 70.32%
18:33:01:WU01:FS01:Download 70.70%
18:33:14:WU01:FS01:Download 71.08%
18:33:27:WU01:FS01:Download 71.46%
18:33:40:WU01:FS01:Download 71.84%
18:33:53:WU01:FS01:Download 72.21%
18:34:05:WU01:FS01:Download 72.59%
18:34:18:WU01:FS01:Download 72.97%
18:34:31:WU01:FS01:Download 73.35%
18:34:44:WU01:FS01:Download 73.73%
18:34:57:WU01:FS01:Download 74.10%
18:35:10:WU01:FS01:Download 74.48%
18:35:23:WU01:FS01:Download 74.86%
18:35:35:WU01:FS01:Download 75.24%
18:35:48:WU01:FS01:Download 75.62%
18:36:01:WU01:FS01:Download 75.99%
18:36:14:WU01:FS01:Download 76.37%
18:36:27:WU01:FS01:Download 76.75%
18:36:40:WU01:FS01:Download 77.13%
18:36:52:WU01:FS01:Download 77.51%
18:37:05:WU01:FS01:Download 77.89%
18:37:18:WU01:FS01:Download 78.26%
18:37:31:WU01:FS01:Download 78.64%
18:37:44:WU01:FS01:Download 79.02%
18:37:56:WU01:FS01:Download 79.40%
18:38:09:WU01:FS01:Download 79.78%
18:38:22:WU01:FS01:Download 80.15%
18:38:35:WU01:FS01:Download 80.53%
18:38:48:WU01:FS01:Download 80.91%
18:39:00:WU01:FS01:Download 81.29%
18:39:13:WU01:FS01:Download 81.67%
18:39:26:WU01:FS01:Download 82.04%
18:39:39:WU01:FS01:Download 82.42%
18:39:52:WU01:FS01:Download 82.80%
18:40:05:WU01:FS01:Download 83.18%
18:40:17:WU01:FS01:Download 83.56%
18:40:30:WU01:FS01:Download 83.93%
18:40:43:WU01:FS01:Download 84.31%
18:40:56:WU01:FS01:Download 84.69%
18:41:08:WU01:FS01:Download 85.07%
18:41:21:WU01:FS01:Download 85.45%
18:41:34:WU01:FS01:Download 85.82%
18:41:47:WU01:FS01:Download 86.20%
18:41:59:WU01:FS01:Download 86.58%
18:42:12:WU01:FS01:Download 86.96%
18:42:25:WU01:FS01:Download 87.34%
18:42:38:WU01:FS01:Download 87.72%
18:42:51:WU01:FS01:Download 88.09%
18:43:04:WU01:FS01:Download 88.47%
18:43:17:WU01:FS01:Download 88.85%
18:43:29:WU01:FS01:Download 89.23%
18:43:42:WU01:FS01:Download 89.61%
18:43:55:WU01:FS01:Download 89.98%
18:44:08:WU01:FS01:Download 90.36%
18:44:21:WU01:FS01:Download 90.74%
18:44:33:WU01:FS01:Download 91.12%
18:44:46:WU01:FS01:Download 91.50%
18:44:59:WU01:FS01:Download 91.87%
18:45:12:WU01:FS01:Download 92.25%
18:45:25:WU01:FS01:Download 92.63%
18:45:38:WU01:FS01:Download 93.01%
18:45:50:WU01:FS01:Download 93.39%
18:46:03:WU01:FS01:Download 93.76%
18:46:16:WU01:FS01:Download 94.14%
18:46:29:WU01:FS01:Download 94.52%
18:46:42:WU01:FS01:Download 94.90%
18:46:54:WU01:FS01:Download 95.28%
18:47:07:WU01:FS01:Download 95.65%
18:47:20:WU01:FS01:Download 96.03%
18:47:33:WU01:FS01:Download 96.41%
18:47:46:WU01:FS01:Download 96.79%
18:47:59:WU01:FS01:Download 97.17%
18:48:12:WU01:FS01:Download 97.55%
18:48:25:WU01:FS01:Download 97.92%
18:48:37:WU01:FS01:Download 98.30%
18:48:50:WU01:FS01:Download 98.68%
18:49:03:WU01:FS01:Download 99.06%
18:49:16:WU01:FS01:Download 99.44%
18:49:29:WU01:FS01:Download 99.81%
18:49:34:WU01:FS01:Download complete
18:49:35:WU01:FS01:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:11713 run:7 clone:18 gen:268 core:0x21 unit:0x0000014d8ca304e759cee15aaec0b40a
18:49:35:WU01:FS01:Starting
18:49:35:WU01:FS01:Running FahCore: E:\FAHClient/FAHCoreWrapper.exe E:\FAH\cores/cores.foldingathome.org/Win32/AMD64/NVIDIA/Fermi/Core_21.fah/FahCore_21.exe -dir 01 -suffix 01 -version 705 -lifeline 1164 -checkpoint 30 -gpu-vendor nvidia -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu 0
18:49:35:WU01:FS01:Started FahCore on PID 8296
18:49:35:WU01:FS01:Core PID:6548
18:49:35:WU01:FS01:FahCore 0x21 started
18:49:35:WU01:FS01:0x21:*********************** Log Started 2018-07-24T18:49:35Z ***********************
18:49:35:WU01:FS01:0x21:Project: 11713 (Run 7, Clone 18, Gen 268)
18:49:35:WU01:FS01:0x21:Unit: 0x0000014d8ca304e759cee15aaec0b40a
18:49:35:WU01:FS01:0x21:CPU: 0x00000000000000000000000000000000
18:49:35:WU01:FS01:0x21:Machine: 1
18:49:35:WU01:FS01:0x21:Reading tar file core.xml
18:49:35:WU01:FS01:0x21:Reading tar file integrator.xml
18:49:35:WU01:FS01:0x21:Reading tar file state.xml
18:49:49:WU01:FS01:0x21:Reading tar file system.xml
18:50:04:WU01:FS01:0x21:Digital signatures verified
18:50:04:WU01:FS01:0x21:Folding@home GPU Core21 Folding@home Core
18:50:04:WU01:FS01:0x21:Version 0.0.18
18:50:08:WU01:FS01:0x21:Completed 0 out of 7500000 steps (0%)
18:50:08:WU01:FS01:0x21:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
18:51:44:WU01:FS01:0x21:Completed 75000 out of 7500000 steps (1%)
18:53:19:WU01:FS01:0x21:Completed 150000 out of 7500000 steps (2%)
18:54:54:WU01:FS01:0x21:Completed 225000 out of 7500000 steps (3%)
18:55:47:WARNING:FS01:Size of positions 2582 does not match topology 2580
18:56:45:WU01:FS01:0x21:Completed 300000 out of 7500000 steps (4%)
18:58:21:WU01:FS01:0x21:Completed 375000 out of 7500000 steps (5%)
18:59:56:WU01:FS01:0x21:Completed 450000 out of 7500000 steps (6%)
19:01:23:WARNING:FS01:Size of positions 2582 does not match topology 2580
19:01:48:WU01:FS01:0x21:Completed 525000 out of 7500000 steps (7%)
19:03:23:WU01:FS01:0x21:Completed 600000 out of 7500000 steps (8%)
19:04:59:WU01:FS01:0x21:Completed 675000 out of 7500000 steps (9%)
19:06:35:WU01:FS01:0x21:Completed 750000 out of 7500000 steps (10%)
19:06:57:WARNING:FS01:Size of positions 2582 does not match topology 2580
19:08:26:WU01:FS01:0x21:Completed 825000 out of 7500000 steps (11%)
19:10:02:WU01:FS01:0x21:Completed 900000 out of 7500000 steps (12%)
19:11:38:WU01:FS01:0x21:Completed 975000 out of 7500000 steps (13%)
19:12:31:WARNING:FS01:Size of positions 2582 does not match topology 2580
19:13:29:WU01:FS01:0x21:Completed 1050000 out of 7500000 steps (14%)
19:15:05:WU01:FS01:0x21:Completed 1125000 out of 7500000 steps (15%)
19:16:40:WU01:FS01:0x21:Completed 1200000 out of 7500000 steps (16%)
19:18:07:WARNING:FS01:Size of positions 2582 does not match topology 2580
19:18:32:WU01:FS01:0x21:Completed 1275000 out of 7500000 steps (17%)
19:20:08:WU01:FS01:0x21:Completed 1350000 out of 7500000 steps (18%)
19:21:43:WU01:FS01:0x21:Completed 1425000 out of 7500000 steps (19%)
19:23:19:WU01:FS01:0x21:Completed 1500000 out of 7500000 steps (20%)
19:23:47:WARNING:FS01:Size of positions 2582 does not match topology 2580
19:25:16:WU01:FS01:0x21:Completed 1575000 out of 7500000 steps (21%)
19:26:52:WU01:FS01:0x21:Completed 1650000 out of 7500000 steps (22%)
19:28:27:WU01:FS01:0x21:Completed 1725000 out of 7500000 steps (23%)
19:29:21:WARNING:FS01:Size of positions 2582 does not match topology 2580
19:30:18:WU01:FS01:0x21:Completed 1800000 out of 7500000 steps (24%)
19:31:54:WU01:FS01:0x21:Completed 1875000 out of 7500000 steps (25%)
19:33:29:WU01:FS01:0x21:Completed 1950000 out of 7500000 steps (26%)
19:34:55:WARNING:FS01:Size of positions 2582 does not match topology 2580
19:35:20:WU01:FS01:0x21:Completed 2025000 out of 7500000 steps (27%)
19:36:56:WU01:FS01:0x21:Completed 2100000 out of 7500000 steps (28%)
19:38:32:WU01:FS01:0x21:Completed 2175000 out of 7500000 steps (29%)
19:40:07:WU01:FS01:0x21:Completed 2250000 out of 7500000 steps (30%)
19:40:29:WARNING:FS01:Size of positions 2582 does not match topology 2580
19:41:59:WU01:FS01:0x21:Completed 2325000 out of 7500000 steps (31%)
19:43:34:WU01:FS01:0x21:Completed 2400000 out of 7500000 steps (32%)
19:45:10:WU01:FS01:0x21:Completed 2475000 out of 7500000 steps (33%)
19:46:03:WARNING:FS01:Size of positions 2582 does not match topology 2580
19:47:01:WU01:FS01:0x21:Completed 2550000 out of 7500000 steps (34%)
19:48:36:WU01:FS01:0x21:Completed 2625000 out of 7500000 steps (35%)
19:50:12:WU01:FS01:0x21:Completed 2700000 out of 7500000 steps (36%)
19:51:37:WARNING:FS01:Size of positions 2582 does not match topology 2580
19:52:03:WU01:FS01:0x21:Completed 2775000 out of 7500000 steps (37%)
19:53:39:WU01:FS01:0x21:Completed 2850000 out of 7500000 steps (38%)
19:55:14:WU01:FS01:0x21:Completed 2925000 out of 7500000 steps (39%)
19:56:49:WU01:FS01:0x21:Completed 3000000 out of 7500000 steps (40%)
19:57:11:WARNING:FS01:Size of positions 2582 does not match topology 2580
19:58:40:WU01:FS01:0x21:Completed 3075000 out of 7500000 steps (41%)
20:00:16:WU01:FS01:0x21:Completed 3150000 out of 7500000 steps (42%)
20:01:51:WU01:FS01:0x21:Completed 3225000 out of 7500000 steps (43%)
20:02:45:WARNING:FS01:Size of positions 2582 does not match topology 2580
20:03:42:WU01:FS01:0x21:Completed 3300000 out of 7500000 steps (44%)
20:05:18:WU01:FS01:0x21:Completed 3375000 out of 7500000 steps (45%)
20:06:53:WU01:FS01:0x21:Completed 3450000 out of 7500000 steps (46%)
20:08:19:WARNING:FS01:Size of positions 2582 does not match topology 2580
20:08:45:WU01:FS01:0x21:Completed 3525000 out of 7500000 steps (47%)
20:10:20:WU01:FS01:0x21:Completed 3600000 out of 7500000 steps (48%)
20:11:55:WU01:FS01:0x21:Completed 3675000 out of 7500000 steps (49%)
20:13:31:WU01:FS01:0x21:Completed 3750000 out of 7500000 steps (50%)
20:13:53:WARNING:FS01:Size of positions 2582 does not match topology 2580
20:15:22:WU01:FS01:0x21:Completed 3825000 out of 7500000 steps (51%)
20:16:58:WU01:FS01:0x21:Completed 3900000 out of 7500000 steps (52%)
20:18:33:WU01:FS01:0x21:Completed 3975000 out of 7500000 steps (53%)
20:19:27:WARNING:FS01:Size of positions 2582 does not match topology 2580
20:20:25:WU01:FS01:0x21:Completed 4050000 out of 7500000 steps (54%)
20:22:01:WU01:FS01:0x21:Completed 4125000 out of 7500000 steps (55%)
20:23:36:WU01:FS01:0x21:Completed 4200000 out of 7500000 steps (56%)
20:25:01:WARNING:FS01:Size of positions 2582 does not match topology 2580
20:25:27:WU01:FS01:0x21:Completed 4275000 out of 7500000 steps (57%)
20:27:03:WU01:FS01:0x21:Completed 4350000 out of 7500000 steps (58%)
20:28:38:WU01:FS01:0x21:Completed 4425000 out of 7500000 steps (59%)
20:30:14:WU01:FS01:0x21:Completed 4500000 out of 7500000 steps (60%)
20:30:37:WARNING:FS01:Size of positions 2582 does not match topology 2580
20:32:05:WU01:FS01:0x21:Completed 4575000 out of 7500000 steps (61%)
20:33:41:WU01:FS01:0x21:Completed 4650000 out of 7500000 steps (62%)
20:35:16:WU01:FS01:0x21:Completed 4725000 out of 7500000 steps (63%)
20:36:11:WARNING:FS01:Size of positions 2582 does not match topology 2580
20:37:07:WU01:FS01:0x21:Completed 4800000 out of 7500000 steps (64%)
20:38:43:WU01:FS01:0x21:Completed 4875000 out of 7500000 steps (65%)
20:40:19:WU01:FS01:0x21:Completed 4950000 out of 7500000 steps (66%)
20:41:45:WARNING:FS01:Size of positions 2582 does not match topology 2580
20:42:10:WU01:FS01:0x21:Completed 5025000 out of 7500000 steps (67%)
20:43:46:WU01:FS01:0x21:Completed 5100000 out of 7500000 steps (68%)
20:45:21:WU01:FS01:0x21:Completed 5175000 out of 7500000 steps (69%)
20:46:57:WU01:FS01:0x21:Completed 5250000 out of 7500000 steps (70%)
20:47:19:WARNING:FS01:Size of positions 2582 does not match topology 2580
20:48:48:WU01:FS01:0x21:Completed 5325000 out of 7500000 steps (71%)
20:50:23:WU01:FS01:0x21:Completed 5400000 out of 7500000 steps (72%)
20:51:59:WU01:FS01:0x21:Completed 5475000 out of 7500000 steps (73%)
20:52:53:WARNING:FS01:Size of positions 2582 does not match topology 2580
20:53:50:WU01:FS01:0x21:Completed 5550000 out of 7500000 steps (74%)
20:55:26:WU01:FS01:0x21:Completed 5625000 out of 7500000 steps (75%)
20:57:01:WU01:FS01:0x21:Completed 5700000 out of 7500000 steps (76%)
20:58:27:WARNING:FS01:Size of positions 2582 does not match topology 2580
20:58:52:WU01:FS01:0x21:Completed 5775000 out of 7500000 steps (77%)
21:00:28:WU01:FS01:0x21:Completed 5850000 out of 7500000 steps (78%)
21:02:03:WU01:FS01:0x21:Completed 5925000 out of 7500000 steps (79%)
21:03:39:WU01:FS01:0x21:Completed 6000000 out of 7500000 steps (80%)
21:04:01:WARNING:FS01:Size of positions 2582 does not match topology 2580
21:05:29:WU01:FS01:0x21:Completed 6075000 out of 7500000 steps (81%)
21:07:03:WU01:FS01:0x21:Completed 6150000 out of 7500000 steps (82%)
21:08:38:WU01:FS01:0x21:Completed 6225000 out of 7500000 steps (83%)
21:09:31:WARNING:FS01:Size of positions 2582 does not match topology 2580
21:10:28:WU01:FS01:0x21:Completed 6300000 out of 7500000 steps (84%)
21:12:04:WU01:FS01:0x21:Completed 6375000 out of 7500000 steps (85%)
21:13:40:WU01:FS01:0x21:Completed 6450000 out of 7500000 steps (86%)
21:15:05:WARNING:FS01:Size of positions 2582 does not match topology 2580
21:15:30:WU01:FS01:0x21:Completed 6525000 out of 7500000 steps (87%)
21:17:06:WU01:FS01:0x21:Completed 6600000 out of 7500000 steps (88%)
21:18:41:WU01:FS01:0x21:Completed 6675000 out of 7500000 steps (89%)
21:20:17:WU01:FS01:0x21:Completed 6750000 out of 7500000 steps (90%)
21:20:18:WU00:FS01:Connecting to 65.254.110.245:8080
21:20:19:WU00:FS01:Assigned to work server 128.252.203.4
21:20:19:WU00:FS01:Requesting new work unit for slot 01: RUNNING gpu:0:GP102 [GeForce GTX 1080 Ti] 11380 from 128.252.203.4
21:20:19:WU00:FS01:Connecting to 128.252.203.4:8080
21:20:20:WU00:FS01:Downloading 11.66MiB
21:20:33:WU00:FS01:Download 0.54%
21:20:37:WARNING:FS01:Size of positions 2582 does not match topology 2580
21:20:45:WU00:FS01:Download 1.07%
21:20:58:WU00:FS01:Download 1.61%
21:21:40:WU00:FS01:Download 2.14%
21:22:07:WU01:FS01:0x21:Completed 6825000 out of 7500000 steps (91%)
21:22:24:WU00:FS01:Download 2.68%
21:23:08:WU00:FS01:Download 3.22%
21:23:43:WU01:FS01:0x21:Completed 6900000 out of 7500000 steps (92%)
21:23:52:WU00:FS01:Download 3.75%
21:24:36:WU00:FS01:Download 4.29%
21:25:19:WU01:FS01:0x21:Completed 6975000 out of 7500000 steps (93%)
21:25:20:WU00:FS01:Download 4.83%
21:26:04:WU00:FS01:Download 5.36%
21:26:11:WARNING:FS01:Size of positions 2582 does not match topology 2580
21:26:49:WU00:FS01:Download 5.90%
21:27:09:WU01:FS01:0x21:Completed 7050000 out of 7500000 steps (94%)
21:27:34:WU00:FS01:Download 6.43%
21:28:19:WU00:FS01:Download 6.97%
21:28:46:WU01:FS01:0x21:Completed 7125000 out of 7500000 steps (95%)
21:29:04:WU00:FS01:Download 7.51%
21:29:48:WU00:FS01:Download 8.04%
21:30:22:WU01:FS01:0x21:Completed 7200000 out of 7500000 steps (96%)
21:30:32:WU00:FS01:Download 8.58%
21:31:16:WU00:FS01:Download 9.12%
21:31:41:WU00:FS01:Download 9.65%
21:31:45:WARNING:FS01:Size of positions 2582 does not match topology 2580
21:32:03:WU00:FS01:Download 10.19%
21:32:12:WU01:FS01:0x21:Completed 7275000 out of 7500000 steps (97%)
21:32:47:WU00:FS01:Download 10.72%
21:33:30:WU00:FS01:Download 11.26%
21:33:48:WU01:FS01:0x21:Completed 7350000 out of 7500000 steps (98%)
21:34:15:WU00:FS01:Download 11.80%
21:34:59:WU00:FS01:Download 12.33%
21:35:24:WU01:FS01:0x21:Completed 7425000 out of 7500000 steps (99%)
21:35:44:WU00:FS01:Download 12.87%
21:36:28:WU00:FS01:Download 13.40%
21:37:00:WU01:FS01:0x21:Completed 7500000 out of 7500000 steps (100%)
21:37:09:WU00:FS01:Download 13.94%
21:37:14:WU01:FS01:0x21:Saving result file logfile_01.txt
21:37:14:WU01:FS01:0x21:Saving result file checkpointState.xml
21:37:14:WU01:FS01:0x21:Saving result file checkpt.crc
21:37:14:WU01:FS01:0x21:Saving result file log.txt
21:37:14:WU01:FS01:0x21:Saving result file positions.xtc
21:37:15:WU01:FS01:0x21:Folding@home Core Shutdown: FINISHED_UNIT
21:37:15:WU01:FS01:FahCore returned: FINISHED_UNIT (100 = 0x64)
21:37:15:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:11713 run:7 clone:18 gen:268 core:0x21 unit:0x0000014d8ca304e759cee15aaec0b40a
21:37:15:WU01:FS01:Uploading 11.78MiB to 140.163.4.231
21:37:15:WU01:FS01:Connecting to 140.163.4.231:8080
21:37:24:WU00:FS01:Download 14.48%
21:37:49:WU00:FS01:Download 15.01%
21:38:32:WU01:FS01:Upload complete
21:38:32:WU01:FS01:Server responded WORK_ACK (400)
21:38:32:WU01:FS01:Final credit estimate, 110490.00 points
21:38:32:WU01:FS01:Cleaning up
21:38:33:WU00:FS01:Download 15.55%
21:39:17:WU00:FS01:Download 16.09%
21:40:02:WU00:FS01:Download 16.62%
21:40:18:WU00:FS01:Download 17.16%
21:40:45:WU00:FS01:Download 17.69%
21:41:30:WU00:FS01:Download 18.23%
21:41:57:WU00:FS01:Download 18.77%
21:42:09:WU00:FS01:Download 19.30%
21:42:48:WU00:FS01:Download 19.84%
21:43:11:WU00:FS01:Download 20.38%
21:43:49:WU00:FS01:Download 20.91%
21:44:16:WU00:FS01:Download 21.45%
21:44:28:WU00:FS01:Download 21.98%
21:44:43:WU00:FS01:Download 22.52%
21:44:56:WU00:FS01:Download 23.06%
21:45:08:WU00:FS01:Download 23.59%
21:45:21:WU00:FS01:Download 24.13%
21:45:44:WU00:FS01:Download 24.66%
21:45:57:WU00:FS01:Download 25.20%
21:46:09:WU00:FS01:Download 25.74%
21:46:22:WU00:FS01:Download 26.27%
21:47:01:WU00:FS01:Download 26.81%
21:47:29:WU00:FS01:Download 27.35%
21:47:41:WU00:FS01:Download 27.88%
21:47:54:WU00:FS01:Download 28.42%
21:48:07:WU00:FS01:Download 28.95%
21:48:20:WU00:FS01:Download 29.49%
21:48:37:WU00:FS01:Download 30.03%
21:48:50:WU00:FS01:Download 30.56%
21:49:03:WU00:FS01:Download 31.10%
21:49:36:WU00:FS01:Download 31.64%
21:49:49:WU00:FS01:Download 32.17%
21:50:02:WU00:FS01:Download 32.71%
21:50:15:WU00:FS01:Download 33.24%
21:50:43:WU00:FS01:Download 33.78%
21:50:56:WU00:FS01:Download 34.32%
21:51:09:WU00:FS01:Download 34.85%
21:51:22:WU00:FS01:Download 35.39%
21:51:43:WU00:FS01:Download 35.92%
21:52:07:WU00:FS01:Download 36.46%
21:52:19:WU00:FS01:Download 37.00%
21:52:48:WU00:FS01:Download 37.53%
21:53:29:WU00:FS01:Download 38.07%
21:54:06:WU00:FS01:Download 38.61%
21:54:51:WU00:FS01:Download 39.14%
21:55:36:WU00:FS01:Download 39.68%
21:55:57:WU00:FS01:Download 40.21%
21:56:09:WU00:FS01:Download 40.75%
21:56:22:WU00:FS01:Download 41.29%
21:56:48:WU00:FS01:Download 41.82%
21:57:00:WU00:FS01:Download 42.36%
21:57:13:WU00:FS01:Download 42.90%
21:57:26:WU00:FS01:Download 43.43%
21:57:39:WU00:FS01:Download 43.97%
21:57:52:WU00:FS01:Download 44.50%
21:58:04:WU00:FS01:Download 45.04%
21:58:17:WU00:FS01:Download 45.58%
21:58:30:WU00:FS01:Download 46.11%
21:58:43:WU00:FS01:Download 46.65%
21:58:56:WU00:FS01:Download 47.18%
21:59:09:WU00:FS01:Download 47.72%
21:59:21:WU00:FS01:Download 48.26%
21:59:34:WU00:FS01:Download 48.79%
22:00:02:WU00:FS01:Download 49.33%
22:00:15:WU00:FS01:Download 49.87%
22:00:35:WU00:FS01:Download 50.40%
22:00:48:WU00:FS01:Download 50.94%
22:01:01:WU00:FS01:Download 51.47%
22:01:14:WU00:FS01:Download 52.01%
22:01:27:WU00:FS01:Download 52.55%
22:01:39:WU00:FS01:Download 53.08%
22:02:06:WU00:FS01:Download 53.62%
22:02:31:WU00:FS01:Download 54.16%
22:02:43:WU00:FS01:Download 54.69%
22:02:56:WU00:FS01:Download 55.23%
22:03:09:WU00:FS01:Download 55.76%
22:03:22:WU00:FS01:Download 56.30%
22:03:34:WU00:FS01:Download 56.84%
22:03:47:WU00:FS01:Download 57.37%
22:04:00:WU00:FS01:Download 57.91%
22:04:13:WU00:FS01:Download 58.45%
22:04:26:WU00:FS01:Download 58.98%
22:04:39:WU00:FS01:Download 59.52%
22:04:51:WU00:FS01:Download 60.05%
22:05:04:WU00:FS01:Download 60.59%
22:05:22:WU00:FS01:Download 60.59%
22:05:22:ERROR:WU00:FS01:Exception: Transfer failed
22:05:22:WU00:FS01:Connecting to 65.254.110.245:8080
22:05:23:WU00:FS01:Assigned to work server 140.163.4.231
22:05:23:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:GP102 [GeForce GTX 1080 Ti] 11380 from 140.163.4.231
22:05:23:WU00:FS01:Connecting to 140.163.4.231:8080
22:05:26:WU00:FS01:Downloading 16.52MiB
22:06:09:WU00:FS01:Download 0.38%
22:06:54:WU00:FS01:Download 0.76%
22:07:40:WU00:FS01:Download 1.14%
22:07:57:WU00:FS01:Download 1.51%
22:08:10:WU00:FS01:Download 1.89%
22:08:23:WU00:FS01:Download 2.27%
22:08:36:WU00:FS01:Download 2.65%
22:08:48:WU00:FS01:Download 3.03%
22:09:01:WU00:FS01:Download 3.41%
22:09:14:WU00:FS01:Download 3.78%
22:09:27:WU00:FS01:Download 4.16%
22:10:09:WU00:FS01:Download 4.54%
22:10:55:WU00:FS01:Download 4.92%
22:11:39:WU00:FS01:Download 5.30%
22:12:24:WU00:FS01:Download 5.68%
22:13:08:WU00:FS01:Download 6.05%
22:13:52:WU00:FS01:Download 6.43%
22:14:37:WU00:FS01:Download 6.81%
22:15:23:WU00:FS01:Download 7.19%
22:16:08:WU00:FS01:Download 7.57%
22:16:53:WU00:FS01:Download 7.95%
22:17:39:WU00:FS01:Download 8.32%
22:18:23:WU00:FS01:Download 8.70%
22:19:08:WU00:FS01:Download 9.08%
22:19:53:WU00:FS01:Download 9.46%
22:20:38:WU00:FS01:Download 9.84%
22:21:22:WU00:FS01:Download 10.22%
22:22:08:WU00:FS01:Download 10.60%
22:22:52:WU00:FS01:Download 10.97%
22:23:37:WU00:FS01:Download 11.35%
22:24:22:WU00:FS01:Download 11.73%
22:25:07:WU00:FS01:Download 12.11%
22:25:52:WU00:FS01:Download 12.49%
22:26:37:WU00:FS01:Download 12.87%
22:27:22:WU00:FS01:Download 13.24%
22:28:07:WU00:FS01:Download 13.62%
22:28:52:WU00:FS01:Download 14.00%
22:29:37:WU00:FS01:Download 14.38%
22:30:21:WU00:FS01:Download 14.76%
22:31:06:WU00:FS01:Download 15.14%
22:31:52:WU00:FS01:Download 15.51%
22:32:36:WU00:FS01:Download 15.89%
22:33:21:WU00:FS01:Download 16.27%
22:34:06:WU00:FS01:Download 16.65%
22:34:52:WU00:FS01:Download 17.03%
22:35:36:WU00:FS01:Download 17.41%
22:36:21:WU00:FS01:Download 17.78%
22:37:05:WU00:FS01:Download 18.16%
22:37:51:WU00:FS01:Download 18.54%
22:38:36:WU00:FS01:Download 18.92%
22:39:20:WU00:FS01:Download 19.30%
22:40:05:WU00:FS01:Download 19.68%
22:40:51:WU00:FS01:Download 20.05%
22:41:35:WU00:FS01:Download 20.43%
22:42:20:WU00:FS01:Download 20.81%
22:43:04:WU00:FS01:Download 21.19%
22:43:49:WU00:FS01:Download 21.57%
22:44:34:WU00:FS01:Download 21.95%
22:45:18:WU00:FS01:Download 22.33%
22:46:02:WU00:FS01:Download 22.70%
22:46:47:WU00:FS01:Download 23.08%
22:47:32:WU00:FS01:Download 23.46%
22:48:17:WU00:FS01:Download 23.84%
22:49:01:WU00:FS01:Download 24.22%
22:49:47:WU00:FS01:Download 24.60%
22:50:31:WU00:FS01:Download 24.97%
22:51:16:WU00:FS01:Download 25.35%
22:52:02:WU00:FS01:Download 25.73%
22:52:46:WU00:FS01:Download 26.11%
22:53:31:WU00:FS01:Download 26.49%
22:54:16:WU00:FS01:Download 26.87%
22:55:01:WU00:FS01:Download 27.24%
22:55:46:WU00:FS01:Download 27.62%
22:56:31:WU00:FS01:Download 28.00%
22:57:15:WU00:FS01:Download 28.38%
22:58:00:WU00:FS01:Download 28.76%
22:58:45:WU00:FS01:Download 29.14%
22:59:30:WU00:FS01:Download 29.51%
23:00:16:WU00:FS01:Download 29.89%
23:01:01:WU00:FS01:Download 30.27%
23:01:45:WU00:FS01:Download 30.65%
23:02:30:WU00:FS01:Download 31.03%
23:03:15:WU00:FS01:Download 31.41%
23:04:00:WU00:FS01:Download 31.79%
23:04:45:WU00:FS01:Download 32.16%
23:05:29:WU00:FS01:Download 32.54%
23:06:14:WU00:FS01:Download 32.92%
23:06:59:WU00:FS01:Download 33.30%
23:07:44:WU00:FS01:Download 33.68%
******************************* Date: 2018-07-24 *******************************
23:08:28:WU00:FS01:Download 34.06%
23:09:14:WU00:FS01:Download 34.43%
23:09:58:WU00:FS01:Download 34.81%
23:10:43:WU00:FS01:Download 35.19%
23:11:28:WU00:FS01:Download 35.57%
23:12:13:WU00:FS01:Download 35.95%
23:12:26:WU00:FS01:Download 36.33%
23:12:39:WU00:FS01:Download 36.70%
23:12:52:WU00:FS01:Download 37.08%
23:13:05:WU00:FS01:Download 37.46%
23:13:18:WU00:FS01:Download 37.84%
23:13:30:WU00:FS01:Download 38.22%
23:13:43:WU00:FS01:Download 38.60%
23:13:56:WU00:FS01:Download 38.97%
23:14:09:WU00:FS01:Download 39.35%
23:14:22:WU00:FS01:Download 39.73%
23:14:35:WU00:FS01:Download 40.11%
23:14:48:WU00:FS01:Download 40.49%
23:15:01:WU00:FS01:Download 40.87%
23:15:14:WU00:FS01:Download 41.25%
23:15:26:WU00:FS01:Download 41.62%
23:15:39:WU00:FS01:Download 42.00%
23:15:52:WU00:FS01:Download 42.38%
23:16:05:WU00:FS01:Download 42.76%
23:16:18:WU00:FS01:Download 43.14%
23:16:31:WU00:FS01:Download 43.52%
23:16:44:WU00:FS01:Download 43.89%
23:16:57:WU00:FS01:Download 44.27%
23:17:10:WU00:FS01:Download 44.65%
23:17:22:WU00:FS01:Download 45.03%
23:17:35:WU00:FS01:Download 45.41%
23:17:48:WU00:FS01:Download 45.79%
23:18:01:WU00:FS01:Download 46.16%
23:18:14:WU00:FS01:Download 46.54%
23:18:27:WU00:FS01:Download 46.92%
23:18:40:WU00:FS01:Download 47.30%
23:18:53:WU00:FS01:Download 47.68%
23:19:06:WU00:FS01:Download 48.06%
23:19:19:WU00:FS01:Download 48.43%
23:19:32:WU00:FS01:Download 48.81%
23:19:45:WU00:FS01:Download 49.19%
23:19:57:WU00:FS01:Download 49.57%
23:20:10:WU00:FS01:Download 49.95%
23:20:23:WU00:FS01:Download 50.33%
23:20:36:WU00:FS01:Download 50.71%
23:20:49:WU00:FS01:Download 51.08%
23:21:02:WU00:FS01:Download 51.46%
23:21:15:WU00:FS01:Download 51.84%
23:21:28:WU00:FS01:Download 52.22%
23:21:40:WU00:FS01:Download 52.60%
23:21:53:WU00:FS01:Download 52.98%
23:22:06:WU00:FS01:Download 53.35%
23:22:19:WU00:FS01:Download 53.73%
23:22:32:WU00:FS01:Download 54.11%
23:22:45:WU00:FS01:Download 54.49%
23:22:58:WU00:FS01:Download 54.87%
23:23:11:WU00:FS01:Download 55.25%
23:23:24:WU00:FS01:Download 55.62%
23:23:37:WU00:FS01:Download 56.00%
23:23:50:WU00:FS01:Download 56.38%
23:24:02:WU00:FS01:Download 56.76%
23:24:15:WU00:FS01:Download 57.14%
23:24:28:WU00:FS01:Download 57.52%
23:24:41:WU00:FS01:Download 57.89%
23:24:54:WU00:FS01:Download 58.27%
23:25:07:WU00:FS01:Download 58.65%
23:25:20:WU00:FS01:Download 59.03%
23:25:33:WU00:FS01:Download 59.41%
23:25:46:WU00:FS01:Download 59.79%
23:25:59:WU00:FS01:Download 60.16%
23:26:13:WU00:FS01:Download 60.54%
23:26:25:WU00:FS01:Download 60.92%
23:26:55:WU00:FS01:Download 61.30%
23:27:40:WU00:FS01:Download 61.68%
23:28:18:WU00:FS01:Download 62.06%
23:29:01:WU00:FS01:Download 62.44%
23:29:39:WU00:FS01:Download 62.81%
23:30:24:WU00:FS01:Download 63.19%
23:31:09:WU00:FS01:Download 63.57%
23:31:55:WU00:FS01:Download 63.95%
23:32:27:WU00:FS01:Download 64.33%
23:33:12:WU00:FS01:Download 64.71%
23:33:56:WU00:FS01:Download 65.08%
23:34:41:WU00:FS01:Download 65.46%
23:35:25:WU00:FS01:Download 65.84%
23:36:11:WU00:FS01:Download 66.22%
23:36:56:WU00:FS01:Download 66.60%
23:37:41:WU00:FS01:Download 66.98%
23:38:25:WU00:FS01:Download 67.35%
23:39:10:WU00:FS01:Download 67.73%
23:39:55:WU00:FS01:Download 68.11%
23:40:39:WU00:FS01:Download 68.49%
23:41:23:WU00:FS01:Download 68.87%
23:42:07:WU00:FS01:Download 69.25%
23:42:52:WU00:FS01:Download 69.62%
23:43:36:WU00:FS01:Download 70.00%
23:44:21:WU00:FS01:Download 70.38%
23:45:05:WU00:FS01:Download 70.76%
23:45:50:WU00:FS01:Download 71.14%
23:46:35:WU00:FS01:Download 71.52%
23:47:20:WU00:FS01:Download 71.90%
23:48:05:WU00:FS01:Download 72.27%
23:48:50:WU00:FS01:Download 72.65%
23:49:35:WU00:FS01:Download 73.03%
23:50:19:WU00:FS01:Download 73.41%
23:51:04:WU00:FS01:Download 73.79%
23:51:49:WU00:FS01:Download 74.17%
23:52:34:WU00:FS01:Download 74.54%
23:53:19:WU00:FS01:Download 74.92%
23:54:03:WU00:FS01:Download 75.30%
23:54:47:WU00:FS01:Download 75.68%
23:55:32:WU00:FS01:Download 76.06%
23:56:16:WU00:FS01:Download 76.44%
23:57:01:WU00:FS01:Download 76.81%
23:57:45:WU00:FS01:Download 77.19%
23:58:31:WU00:FS01:Download 77.57%
23:59:15:WU00:FS01:Download 77.95%
23:59:52:WU00:FS01:Download 78.33%
00:00:36:WU00:FS01:Download 78.71%
00:01:20:WU00:FS01:Download 79.08%
00:02:05:WU00:FS01:Download 79.46%
00:02:50:WU00:FS01:Download 79.84%
00:03:35:WU00:FS01:Download 80.22%
00:04:19:WU00:FS01:Download 80.60%
00:05:05:WU00:FS01:Download 80.98%
00:05:49:WU00:FS01:Download 81.36%
00:06:26:WU00:FS01:Download 81.36%
00:06:26:ERROR:WU00:FS01:Exception: Transfer failed
00:06:26:WU00:FS01:Connecting to 65.254.110.245:8080
00:06:27:WU00:FS01:Assigned to work server 128.252.203.4
00:06:27:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:GP102 [GeForce GTX 1080 Ti] 11380 from 128.252.203.4
00:06:27:WU00:FS01:Connecting to 128.252.203.4:8080
00:06:28:WU00:FS01:Downloading 11.66MiB
00:07:01:WU00:FS01:Download 0.54%
00:07:45:WU00:FS01:Download 1.07%
00:08:29:WU00:FS01:Download 1.61%
00:09:14:WU00:FS01:Download 2.14%
00:09:59:WU00:FS01:Download 2.68%
00:10:44:WU00:FS01:Download 3.22%
00:11:28:WU00:FS01:Download 3.75%
00:12:12:WU00:FS01:Download 4.29%
00:12:58:WU00:FS01:Download 4.82%
00:13:43:WU00:FS01:Download 5.36%
00:14:28:WU00:FS01:Download 5.90%
00:15:12:WU00:FS01:Download 6.43%
00:15:56:WU00:FS01:Download 6.97%
00:16:41:WU00:FS01:Download 7.50%
00:17:26:WU00:FS01:Download 8.04%
00:18:10:WU00:FS01:Download 8.58%
00:18:34:WU00:FS01:Download 9.11%
00:19:19:WU00:FS01:Download 9.65%
00:20:03:WU00:FS01:Download 10.18%
00:20:48:WU00:FS01:Download 10.72%
00:21:33:WU00:FS01:Download 11.26%
00:22:17:WU00:FS01:Download 11.79%
00:23:02:WU00:FS01:Download 12.33%
00:23:47:WU00:FS01:Download 12.86%
00:24:32:WU00:FS01:Download 13.40%
00:25:17:WU00:FS01:Download 13.94%
00:26:01:WU00:FS01:Download 14.47%
00:26:46:WU00:FS01:Download 15.01%
00:27:31:WU00:FS01:Download 15.55%
00:28:15:WU00:FS01:Download 16.08%
00:29:00:WU00:FS01:Download 16.62%
00:29:44:WU00:FS01:Download 17.15%
00:30:28:WU00:FS01:Download 17.69%
00:31:12:WU00:FS01:Download 18.23%
00:31:56:WU00:FS01:Download 18.76%
00:32:41:WU00:FS01:Download 19.30%
00:33:25:WU00:FS01:Download 19.83%
00:34:10:WU00:FS01:Download 20.37%
00:34:54:WU00:FS01:Download 20.91%
00:35:39:WU00:FS01:Download 21.44%
00:36:24:WU00:FS01:Download 21.98%
00:37:09:WU00:FS01:Download 22.51%
00:37:54:WU00:FS01:Download 23.05%
00:38:38:WU00:FS01:Download 23.59%
00:39:24:WU00:FS01:Download 24.12%
00:40:08:WU00:FS01:Download 24.66%
00:40:52:WU00:FS01:Download 25.19%
00:41:37:WU00:FS01:Download 25.73%
00:42:22:WU00:FS01:Download 26.27%
00:43:07:WU00:FS01:Download 26.80%
00:43:51:WU00:FS01:Download 27.34%
00:44:35:WU00:FS01:Download 27.87%
00:45:20:WU00:FS01:Download 28.41%
00:46:04:WU00:FS01:Download 28.95%
00:46:49:WU00:FS01:Download 29.48%
00:47:33:WU00:FS01:Download 30.02%
00:48:19:WU00:FS01:Download 30.55%
00:49:04:WU00:FS01:Download 31.09%
00:49:49:WU00:FS01:Download 31.63%
00:50:34:WU00:FS01:Download 32.16%
00:51:06:WU00:FS01:Download 32.70%
00:51:30:WU00:FS01:Download 32.70%
00:51:30:ERROR:WU00:FS01:Exception: Transfer failed
00:51:30:WU00:FS01:Connecting to 65.254.110.245:8080
00:51:31:WU00:FS01:Assigned to work server 140.163.4.231
00:51:31:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:GP102 [GeForce GTX 1080 Ti] 11380 from 140.163.4.231
00:51:31:WU00:FS01:Connecting to 140.163.4.231:8080
00:51:34:WU00:FS01:Downloading 5.42MiB
00:52:20:WU00:FS01:Download 1.15%
00:53:04:WU00:FS01:Download 2.31%
00:53:50:WU00:FS01:Download 3.46%
00:54:34:WU00:FS01:Download 4.62%
00:55:20:WU00:FS01:Download 5.77%
00:56:05:WU00:FS01:Download 6.92%
00:56:49:WU00:FS01:Download 8.08%
00:57:33:WU00:FS01:Download 9.23%
00:58:18:WU00:FS01:Download 10.38%
00:58:47:WU00:FS01:Download 11.54%
00:59:20:WU00:FS01:Download 12.69%
01:00:05:WU00:FS01:Download 13.85%
01:00:49:WU00:FS01:Download 15.00%
01:01:33:WU00:FS01:Download 16.15%
01:02:17:WU00:FS01:Download 17.31%
01:03:01:WU00:FS01:Download 18.46%
01:03:46:WU00:FS01:Download 19.61%
01:04:30:WU00:FS01:Download 20.77%
01:05:15:WU00:FS01:Download 21.92%
01:06:00:WU00:FS01:Download 23.08%
01:06:45:WU00:FS01:Download 24.23%
01:07:20:WU00:FS01:Download 25.38%
01:07:56:WU00:FS01:Download 26.54%
01:08:41:WU00:FS01:Download 27.69%
01:09:19:WU00:FS01:Download 28.84%
01:10:00:WU00:FS01:Download 30.00%
01:10:42:WU00:FS01:Download 31.15%
01:11:22:WU00:FS01:Download 32.31%
01:12:03:WU00:FS01:Download 33.46%
01:12:43:WU00:FS01:Download 34.61%
01:13:24:WU00:FS01:Download 35.77%
01:14:07:WU00:FS01:Download 36.92%
01:14:46:WU00:FS01:Download 38.07%
01:15:26:WU00:FS01:Download 39.23%
01:16:07:WU00:FS01:Download 40.38%
01:16:48:WU00:FS01:Download 41.54%
01:17:26:WU00:FS01:Download 42.69%
01:18:06:WU00:FS01:Download 43.84%
01:18:48:WU00:FS01:Download 45.00%
01:19:28:WU00:FS01:Download 46.15%
01:20:06:WU00:FS01:Download 47.30%
01:20:50:WU00:FS01:Download 48.46%
01:21:34:WU00:FS01:Download 49.61%
01:22:19:WU00:FS01:Download 50.77%
01:22:58:WU00:FS01:Download 51.92%
01:23:39:WU00:FS01:Download 53.07%
01:24:17:WU00:FS01:Download 54.23%
01:25:02:WU00:FS01:Download 55.38%
Full log:
https://pastebin.com/raw/qiUcx0pq
Re: Slow WU downloads & failed transfer [140.163.4.231]
Posted: Fri Jul 27, 2018 4:41 am
by bruce
lknl wrote:It seems like the setting max_packet_size=9 has little effect for me.
GPU-only log:
Code: Select all
17:07:57:WU01:FS01:Assigned to work server 140.163.4.231
17:07:57:WU01:FS01:Requesting new work unit for slot 01: READY gpu:0:GP102 [GeForce GTX 1080 Ti] 11380 from 140.163.4.231
17:07:57:WU01:FS01:Connecting to 140.163.4.231:8080
17:07:59:WU01:FS01:Downloading 5.42MiB
17:08:11:WU01:FS01:Download 1.15%
17:08:24:WU01:FS01:Download 2.31%
...
17:52:03:WU01:FS01:Download 83.06%
17:52:48:WU01:FS01:Download 84.22%
17:53:01:WU01:FS01:Download 84.22% >>>>> note: 13 seconds passed during which zero progress was made.
17:53:01:ERROR:WU01:FS01:Exception: Transfer failed
>>>>>>> so after 45m04s it has successfully downloaded 84% of 5.42 MB or 4.55 mb. from 140.163.4.231 and then it timed out. that's terribly slow.
17:53:03:WU01:FS01:Assigned to work server 140.163.4.231
17:53:03:WU01:FS01:Requesting new work unit for slot 01: READY gpu:0:GP102 [GeForce GTX 1080 Ti] 11380 from 140.163.4.231
17:53:03:WU01:FS01:Connecting to 140.163.4.231:8080
17:53:04:WU01:FS01:Downloading 16.53MiB
17:53:16:WU01:FS01:Download 0.38%
17:53:29:WU01:FS01:Download 0.76%
17:53:42:WU01:FS01:Download 1.13%
...
18:49:03:WU01:FS01:Download 99.06%
18:49:16:WU01:FS01:Download 99.44%
18:49:29:WU01:FS01:Download 99.81%
18:49:34:WU01:FS01:Download complete
>>>>>> this time after 56m31s it successfully download 100% of 16.53 MB from 140.163.4.231
>>>>>> As expected, the WU starts up and processes normally:
18:49:35:WU01:FS01:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:11713 run:7 clone:18 gen:268 core:0x21 unit:0x0000014d8ca304e759cee15aaec0b40a
18:49:35:WU01:FS01:Starting
18:49:35:WU01:FS01:Running FahCore: E:\FAHClient/FAHCoreWrapper.exe E:\FAH\cores/cores.foldingathome.org/Win32/AMD64/NVIDIA/Fermi/Core_21.fah/FahCore_21.exe -dir 01 -suffix 01 -version 705 -lifeline 1164 -checkpoint 30 -gpu-vendor nvidia -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu 0
18:49:35:WU01:FS01:Started FahCore on PID 8296
18:49:35:WU01:FS01:Core PID:6548
18:49:35:WU01:FS01:FahCore 0x21 started
18:49:35:WU01:FS01:0x21:*********************** Log Started 2018-07-24T18:49:35Z ***********************
18:49:35:WU01:FS01:0x21:Project: 11713 (Run 7, Clone 18, Gen 268)
18:49:35:WU01:FS01:0x21:Unit: 0x0000014d8ca304e759cee15aaec0b40a
18:49:35:WU01:FS01:0x21:CPU: 0x00000000000000000000000000000000
18:49:35:WU01:FS01:0x21:Machine: 1
18:49:35:WU01:FS01:0x21:Reading tar file core.xml
18:49:35:WU01:FS01:0x21:Reading tar file integrator.xml
18:49:35:WU01:FS01:0x21:Reading tar file state.xml
18:49:49:WU01:FS01:0x21:Reading tar file system.xml
18:50:04:WU01:FS01:0x21:Digital signatures verified
18:50:04:WU01:FS01:0x21:Folding@home GPU Core21 Folding@home Core
18:50:04:WU01:FS01:0x21:Version 0.0.18
18:50:08:WU01:FS01:0x21:Completed 0 out of 7500000 steps (0%)
18:50:08:WU01:FS01:0x21:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
18:51:44:WU01:FS01:0x21:Completed 75000 out of 7500000 steps (1%)
18:53:19:WU01:FS01:0x21:Completed 150000 out of 7500000 steps (2%)
18:54:54:WU01:FS01:0x21:Completed 225000 out of 7500000 steps (3%)
18:55:47:WARNING:FS01:Size of positions 2582 does not match topology 2580
18:56:45:WU01:FS01:0x21:Completed 300000 out of 7500000 steps (4%)
18:58:21:WU01:FS01:0x21:Completed 375000 out of 7500000 steps (5%)
18:59:56:WU01:FS01:0x21:Completed 450000 out of 7500000 steps (6%)
19:01:23:WARNING:FS01:Size of positions 2582 does not match topology 2580
19:01:48:WU01:FS01:0x21:Completed 525000 out of 7500000 steps (7%)
19:03:23:WU01:FS01:0x21:Completed 600000 out of 7500000 steps (8%)
19:04:59:WU01:FS01:0x21:Completed 675000 out of 7500000 steps (9%)
19:06:35:WU01:FS01:0x21:Completed 750000 out of 7500000 steps (10%)
19:06:57:WARNING:FS01:Size of positions 2582 does not match topology 2580
19:08:26:WU01:FS01:0x21:Completed 825000 out of 7500000 steps (11%)
19:10:02:WU01:FS01:0x21:Completed 900000 out of 7500000 steps (12%)
19:11:38:WU01:FS01:0x21:Completed 975000 out of 7500000 steps (13%)
19:12:31:WARNING:FS01:Size of positions 2582 does not match topology 2580
19:13:29:WU01:FS01:0x21:Completed 1050000 out of 7500000 steps (14%)
19:15:05:WU01:FS01:0x21:Completed 1125000 out of 7500000 steps (15%)
19:16:40:WU01:FS01:0x21:Completed 1200000 out of 7500000 steps (16%)
19:18:07:WARNING:FS01:Size of positions 2582 does not match topology 2580
19:18:32:WU01:FS01:0x21:Completed 1275000 out of 7500000 steps (17%)
19:20:08:WU01:FS01:0x21:Completed 1350000 out of 7500000 steps (18%)
19:21:43:WU01:FS01:0x21:Completed 1425000 out of 7500000 steps (19%)
19:23:19:WU01:FS01:0x21:Completed 1500000 out of 7500000 steps (20%)
19:23:47:WARNING:FS01:Size of positions 2582 does not match topology 2580
19:25:16:WU01:FS01:0x21:Completed 1575000 out of 7500000 steps (21%)
19:26:52:WU01:FS01:0x21:Completed 1650000 out of 7500000 steps (22%)
19:28:27:WU01:FS01:0x21:Completed 1725000 out of 7500000 steps (23%)
19:29:21:WARNING:FS01:Size of positions 2582 does not match topology 2580
19:30:18:WU01:FS01:0x21:Completed 1800000 out of 7500000 steps (24%)
19:31:54:WU01:FS01:0x21:Completed 1875000 out of 7500000 steps (25%)
19:33:29:WU01:FS01:0x21:Completed 1950000 out of 7500000 steps (26%)
19:34:55:WARNING:FS01:Size of positions 2582 does not match topology 2580
19:35:20:WU01:FS01:0x21:Completed 2025000 out of 7500000 steps (27%)
19:36:56:WU01:FS01:0x21:Completed 2100000 out of 7500000 steps (28%)
19:38:32:WU01:FS01:0x21:Completed 2175000 out of 7500000 steps (29%)
19:40:07:WU01:FS01:0x21:Completed 2250000 out of 7500000 steps (30%)
19:40:29:WARNING:FS01:Size of positions 2582 does not match topology 2580
19:41:59:WU01:FS01:0x21:Completed 2325000 out of 7500000 steps (31%)
19:43:34:WU01:FS01:0x21:Completed 2400000 out of 7500000 steps (32%)
19:45:10:WU01:FS01:0x21:Completed 2475000 out of 7500000 steps (33%)
19:46:03:WARNING:FS01:Size of positions 2582 does not match topology 2580
19:47:01:WU01:FS01:0x21:Completed 2550000 out of 7500000 steps (34%)
19:48:36:WU01:FS01:0x21:Completed 2625000 out of 7500000 steps (35%)
19:50:12:WU01:FS01:0x21:Completed 2700000 out of 7500000 steps (36%)
19:51:37:WARNING:FS01:Size of positions 2582 does not match topology 2580
19:52:03:WU01:FS01:0x21:Completed 2775000 out of 7500000 steps (37%)
19:53:39:WU01:FS01:0x21:Completed 2850000 out of 7500000 steps (38%)
19:55:14:WU01:FS01:0x21:Completed 2925000 out of 7500000 steps (39%)
19:56:49:WU01:FS01:0x21:Completed 3000000 out of 7500000 steps (40%)
19:57:11:WARNING:FS01:Size of positions 2582 does not match topology 2580
19:58:40:WU01:FS01:0x21:Completed 3075000 out of 7500000 steps (41%)
20:00:16:WU01:FS01:0x21:Completed 3150000 out of 7500000 steps (42%)
20:01:51:WU01:FS01:0x21:Completed 3225000 out of 7500000 steps (43%)
20:02:45:WARNING:FS01:Size of positions 2582 does not match topology 2580
20:03:42:WU01:FS01:0x21:Completed 3300000 out of 7500000 steps (44%)
20:05:18:WU01:FS01:0x21:Completed 3375000 out of 7500000 steps (45%)
20:06:53:WU01:FS01:0x21:Completed 3450000 out of 7500000 steps (46%)
20:08:19:WARNING:FS01:Size of positions 2582 does not match topology 2580
20:08:45:WU01:FS01:0x21:Completed 3525000 out of 7500000 steps (47%)
20:10:20:WU01:FS01:0x21:Completed 3600000 out of 7500000 steps (48%)
20:11:55:WU01:FS01:0x21:Completed 3675000 out of 7500000 steps (49%)
20:13:31:WU01:FS01:0x21:Completed 3750000 out of 7500000 steps (50%)
20:13:53:WARNING:FS01:Size of positions 2582 does not match topology 2580
20:15:22:WU01:FS01:0x21:Completed 3825000 out of 7500000 steps (51%)
20:16:58:WU01:FS01:0x21:Completed 3900000 out of 7500000 steps (52%)
20:18:33:WU01:FS01:0x21:Completed 3975000 out of 7500000 steps (53%)
20:19:27:WARNING:FS01:Size of positions 2582 does not match topology 2580
20:20:25:WU01:FS01:0x21:Completed 4050000 out of 7500000 steps (54%)
20:22:01:WU01:FS01:0x21:Completed 4125000 out of 7500000 steps (55%)
20:23:36:WU01:FS01:0x21:Completed 4200000 out of 7500000 steps (56%)
20:25:01:WARNING:FS01:Size of positions 2582 does not match topology 2580
20:25:27:WU01:FS01:0x21:Completed 4275000 out of 7500000 steps (57%)
20:27:03:WU01:FS01:0x21:Completed 4350000 out of 7500000 steps (58%)
20:28:38:WU01:FS01:0x21:Completed 4425000 out of 7500000 steps (59%)
20:30:14:WU01:FS01:0x21:Completed 4500000 out of 7500000 steps (60%)
20:30:37:WARNING:FS01:Size of positions 2582 does not match topology 2580
20:32:05:WU01:FS01:0x21:Completed 4575000 out of 7500000 steps (61%)
20:33:41:WU01:FS01:0x21:Completed 4650000 out of 7500000 steps (62%)
20:35:16:WU01:FS01:0x21:Completed 4725000 out of 7500000 steps (63%)
20:36:11:WARNING:FS01:Size of positions 2582 does not match topology 2580
20:37:07:WU01:FS01:0x21:Completed 4800000 out of 7500000 steps (64%)
20:38:43:WU01:FS01:0x21:Completed 4875000 out of 7500000 steps (65%)
20:40:19:WU01:FS01:0x21:Completed 4950000 out of 7500000 steps (66%)
20:41:45:WARNING:FS01:Size of positions 2582 does not match topology 2580
20:42:10:WU01:FS01:0x21:Completed 5025000 out of 7500000 steps (67%)
20:43:46:WU01:FS01:0x21:Completed 5100000 out of 7500000 steps (68%)
20:45:21:WU01:FS01:0x21:Completed 5175000 out of 7500000 steps (69%)
20:46:57:WU01:FS01:0x21:Completed 5250000 out of 7500000 steps (70%)
20:47:19:WARNING:FS01:Size of positions 2582 does not match topology 2580
20:48:48:WU01:FS01:0x21:Completed 5325000 out of 7500000 steps (71%)
20:50:23:WU01:FS01:0x21:Completed 5400000 out of 7500000 steps (72%)
20:51:59:WU01:FS01:0x21:Completed 5475000 out of 7500000 steps (73%)
20:52:53:WARNING:FS01:Size of positions 2582 does not match topology 2580
20:53:50:WU01:FS01:0x21:Completed 5550000 out of 7500000 steps (74%)
20:55:26:WU01:FS01:0x21:Completed 5625000 out of 7500000 steps (75%)
20:57:01:WU01:FS01:0x21:Completed 5700000 out of 7500000 steps (76%)
20:58:27:WARNING:FS01:Size of positions 2582 does not match topology 2580
20:58:52:WU01:FS01:0x21:Completed 5775000 out of 7500000 steps (77%)
21:00:28:WU01:FS01:0x21:Completed 5850000 out of 7500000 steps (78%)
21:02:03:WU01:FS01:0x21:Completed 5925000 out of 7500000 steps (79%)
21:03:39:WU01:FS01:0x21:Completed 6000000 out of 7500000 steps (80%)
21:04:01:WARNING:FS01:Size of positions 2582 does not match topology 2580
21:05:29:WU01:FS01:0x21:Completed 6075000 out of 7500000 steps (81%)
21:07:03:WU01:FS01:0x21:Completed 6150000 out of 7500000 steps (82%)
21:08:38:WU01:FS01:0x21:Completed 6225000 out of 7500000 steps (83%)
21:09:31:WARNING:FS01:Size of positions 2582 does not match topology 2580
21:10:28:WU01:FS01:0x21:Completed 6300000 out of 7500000 steps (84%)
21:12:04:WU01:FS01:0x21:Completed 6375000 out of 7500000 steps (85%)
21:13:40:WU01:FS01:0x21:Completed 6450000 out of 7500000 steps (86%)
21:15:05:WARNING:FS01:Size of positions 2582 does not match topology 2580
21:15:30:WU01:FS01:0x21:Completed 6525000 out of 7500000 steps (87%)
21:17:06:WU01:FS01:0x21:Completed 6600000 out of 7500000 steps (88%)
21:18:41:WU01:FS01:0x21:Completed 6675000 out of 7500000 steps (89%)
21:20:17:WU01:FS01:0x21:Completed 6750000 out of 7500000 steps (90%)
21:20:18:WU00:FS01:Connecting to 65.254.110.245:8080
21:20:19:WU00:FS01:Assigned to work server 128.252.203.4
21:20:19:WU00:FS01:Requesting new work unit for slot 01: RUNNING gpu:0:GP102 [GeForce GTX 1080 Ti] 11380 from 128.252.203.4
21:20:19:WU00:FS01:Connecting to 128.252.203.4:8080
21:20:20:WU00:FS01:Downloading 11.66MiB
21:20:33:WU00:FS01:Download 0.54%
21:20:37:WARNING:FS01:Size of positions 2582 does not match topology 2580
21:20:45:WU00:FS01:Download 1.07%
21:20:58:WU00:FS01:Download 1.61%
21:21:40:WU00:FS01:Download 2.14%
21:22:07:WU01:FS01:0x21:Completed 6825000 out of 7500000 steps (91%)
21:22:24:WU00:FS01:Download 2.68%
21:23:08:WU00:FS01:Download 3.22%
21:23:43:WU01:FS01:0x21:Completed 6900000 out of 7500000 steps (92%)
21:23:52:WU00:FS01:Download 3.75%
21:24:36:WU00:FS01:Download 4.29%
21:25:19:WU01:FS01:0x21:Completed 6975000 out of 7500000 steps (93%)
21:25:20:WU00:FS01:Download 4.83%
21:26:04:WU00:FS01:Download 5.36%
21:26:11:WARNING:FS01:Size of positions 2582 does not match topology 2580
21:26:49:WU00:FS01:Download 5.90%
21:27:09:WU01:FS01:0x21:Completed 7050000 out of 7500000 steps (94%)
21:27:34:WU00:FS01:Download 6.43%
21:28:19:WU00:FS01:Download 6.97%
21:28:46:WU01:FS01:0x21:Completed 7125000 out of 7500000 steps (95%)
21:29:04:WU00:FS01:Download 7.51%
21:29:48:WU00:FS01:Download 8.04%
21:30:22:WU01:FS01:0x21:Completed 7200000 out of 7500000 steps (96%)
21:30:32:WU00:FS01:Download 8.58%
21:31:16:WU00:FS01:Download 9.12%
21:31:41:WU00:FS01:Download 9.65%
21:31:45:WARNING:FS01:Size of positions 2582 does not match topology 2580
21:32:03:WU00:FS01:Download 10.19%
21:32:12:WU01:FS01:0x21:Completed 7275000 out of 7500000 steps (97%)
21:32:47:WU00:FS01:Download 10.72%
21:33:30:WU00:FS01:Download 11.26%
21:33:48:WU01:FS01:0x21:Completed 7350000 out of 7500000 steps (98%)
21:34:15:WU00:FS01:Download 11.80%
21:34:59:WU00:FS01:Download 12.33%
21:35:24:WU01:FS01:0x21:Completed 7425000 out of 7500000 steps (99%)
21:35:44:WU00:FS01:Download 12.87%
21:36:28:WU00:FS01:Download 13.40%
21:37:00:WU01:FS01:0x21:Completed 7500000 out of 7500000 steps (100%)
21:37:09:WU00:FS01:Download 13.94%
>>>>> After 16m50s it has successfully download only 13.94% of 11.66 MB or 1.63 MB from 128.252.203.4.
>>>>> Same extremely slow performance but from a different server.
21:37:14:WU01:FS01:0x21:Saving result file logfile_01.txt
21:37:14:WU01:FS01:0x21:Saving result file checkpointState.xml
21:37:14:WU01:FS01:0x21:Saving result file checkpt.crc
21:37:14:WU01:FS01:0x21:Saving result file log.txt
21:37:14:WU01:FS01:0x21:Saving result file positions.xtc
21:37:15:WU01:FS01:0x21:Folding@home Core Shutdown: FINISHED_UNIT
21:37:15:WU01:FS01:FahCore returned: FINISHED_UNIT (100 = 0x64)
21:37:15:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:11713 run:7 clone:18 gen:268 core:0x21 unit:0x0000014d8ca304e759cee15aaec0b40a
21:37:15:WU01:FS01:Uploading 11.78MiB to 140.163.4.231
21:37:15:WU01:FS01:Connecting to 140.163.4.231:8080
21:37:24:WU00:FS01:Download 14.48%
21:37:49:WU00:FS01:Download 15.01%
21:38:32:WU01:FS01:Upload complete
21:38:32:WU01:FS01:Server responded WORK_ACK (400)
21:38:32:WU01:FS01:Final credit estimate, 110490.00 points
21:38:32:WU01:FS01:Cleaning up
21:38:33:WU00:FS01:Download 15.55%
21:39:17:WU00:FS01:Download 16.09%
,,,
22:04:51:WU00:FS01:Download 60.05%
22:05:04:WU00:FS01:Download 60.59%
22:05:22:WU00:FS01:Download 60.59%
22:05:22:ERROR:WU00:FS01:Exception: Transfer failed
>>>> after 22:05:22 - 21:20:19 = 45m02s it times out again after downloading 69,59% of 11.66 MB from 128.252.203.4.
>>>> Same terrible internet performance but
22:05:22:WU00:FS01:Connecting to 65.254.110.245:8080
22:05:23:WU00:FS01:Assigned to work server 140.163.4.231
22:05:23:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:GP102 [GeForce GTX 1080 Ti] 11380 from 140.163.4.231
22:05:23:WU00:FS01:Connecting to 140.163.4.231:8080
22:05:26:WU00:FS01:Downloading 16.52MiB
22:06:09:WU00:FS01:Download 0.38%
...
00:05:49:WU00:FS01:Download 81.36%
00:06:26:WU00:FS01:Download 81.36%
00:06:26:ERROR:WU00:FS01:Exception: Transfer failed
>>>>>>
00:06:26:WU00:FS01:Connecting to 65.254.110.245:8080
00:06:27:WU00:FS01:Assigned to work server 128.252.203.4
00:06:27:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:GP102 [GeForce GTX 1080 Ti] 11380 from 128.252.203.4
00:06:27:WU00:FS01:Connecting to 128.252.203.4:8080
00:06:28:WU00:FS01:Downloading 11.66MiB
00:07:01:WU00:FS01:Download 0.54%
00:07:45:WU00:FS01:Download 1.07%
00:08:29:WU00:FS01:Download 1.61%
00:09:14:WU00:FS01:Download 2.14%
00:09:59:WU00:FS01:Download 2.68%
00:10:44:WU00:FS01:Download 3.22%
00:11:28:WU00:FS01:Download 3.75%
00:12:12:WU00:FS01:Download 4.29%
00:12:58:WU00:FS01:Download 4.82%
00:13:43:WU00:FS01:Download 5.36%
00:14:28:WU00:FS01:Download 5.90%
00:15:12:WU00:FS01:Download 6.43%
00:15:56:WU00:FS01:Download 6.97%
00:16:41:WU00:FS01:Download 7.50%
00:17:26:WU00:FS01:Download 8.04%
00:18:10:WU00:FS01:Download 8.58%
00:18:34:WU00:FS01:Download 9.11%
00:19:19:WU00:FS01:Download 9.65%
00:20:03:WU00:FS01:Download 10.18%
00:20:48:WU00:FS01:Download 10.72%
00:21:33:WU00:FS01:Download 11.26%
00:22:17:WU00:FS01:Download 11.79%
00:23:02:WU00:FS01:Download 12.33%
00:23:47:WU00:FS01:Download 12.86%
00:24:32:WU00:FS01:Download 13.40%
00:25:17:WU00:FS01:Download 13.94%
00:26:01:WU00:FS01:Download 14.47%
00:26:46:WU00:FS01:Download 15.01%
00:27:31:WU00:FS01:Download 15.55%
00:28:15:WU00:FS01:Download 16.08%
00:29:00:WU00:FS01:Download 16.62%
00:29:44:WU00:FS01:Download 17.15%
00:30:28:WU00:FS01:Download 17.69%
00:31:12:WU00:FS01:Download 18.23%
00:31:56:WU00:FS01:Download 18.76%
00:32:41:WU00:FS01:Download 19.30%
00:33:25:WU00:FS01:Download 19.83%
00:34:10:WU00:FS01:Download 20.37%
00:34:54:WU00:FS01:Download 20.91%
00:35:39:WU00:FS01:Download 21.44%
00:36:24:WU00:FS01:Download 21.98%
00:37:09:WU00:FS01:Download 22.51%
00:37:54:WU00:FS01:Download 23.05%
00:38:38:WU00:FS01:Download 23.59%
00:39:24:WU00:FS01:Download 24.12%
00:40:08:WU00:FS01:Download 24.66%
00:40:52:WU00:FS01:Download 25.19%
00:41:37:WU00:FS01:Download 25.73%
00:42:22:WU00:FS01:Download 26.27%
00:43:07:WU00:FS01:Download 26.80%
00:43:51:WU00:FS01:Download 27.34%
00:44:35:WU00:FS01:Download 27.87%
00:45:20:WU00:FS01:Download 28.41%
00:46:04:WU00:FS01:Download 28.95%
00:46:49:WU00:FS01:Download 29.48%
00:47:33:WU00:FS01:Download 30.02%
00:48:19:WU00:FS01:Download 30.55%
00:49:04:WU00:FS01:Download 31.09%
00:49:49:WU00:FS01:Download 31.63%
00:50:34:WU00:FS01:Download 32.16%
00:51:06:WU00:FS01:Download 32.70%
00:51:30:WU00:FS01:Download 32.70%
00:51:30:ERROR:WU00:FS01:Exception: Transfer failed
>>>>>>
00:51:30:WU00:FS01:Connecting to 65.254.110.245:8080
00:51:31:WU00:FS01:Assigned to work server 140.163.4.231
00:51:31:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:GP102 [GeForce GTX 1080 Ti] 11380 from 140.163.4.231
00:51:31:WU00:FS01:Connecting to 140.163.4.231:8080
00:51:34:WU00:FS01:Downloading 5.42MiB
00:52:20:WU00:FS01:Download 1.15%
00:53:04:WU00:FS01:Download 2.31%
...
Full log:
https://pastebin.com/raw/qiUcx0pq
Since you're getting terrible performance from several different FAH servers, I conclude it's probably that your internet connection is simply too slow or it's completely saturated with other data going through the same connection.
What do you get if you run one of the popular speed tests?
Re: Slow WU downloads & failed transfer [140.163.4.231]
Posted: Fri Jul 27, 2018 5:47 am
by lknl
bruce wrote:lknl wrote:It seems like the setting max_packet_size=9 has little effect for me.
GPU-only log:
Code: Select all
17:07:57:WU01:FS01:Assigned to work server 140.163.4.231
17:07:57:WU01:FS01:Requesting new work unit for slot 01: READY gpu:0:GP102 [GeForce GTX 1080 Ti] 11380 from 140.163.4.231
17:07:57:WU01:FS01:Connecting to 140.163.4.231:8080
17:07:59:WU01:FS01:Downloading 5.42MiB
17:08:11:WU01:FS01:Download 1.15%
17:08:24:WU01:FS01:Download 2.31%
...
17:52:03:WU01:FS01:Download 83.06%
17:52:48:WU01:FS01:Download 84.22%
17:53:01:WU01:FS01:Download 84.22% >>>>> note: 13 seconds passed during which zero progress was made.
17:53:01:ERROR:WU01:FS01:Exception: Transfer failed
>>>>>>> so after 45m04s it has successfully downloaded 84% of 5.42 MB or 4.55 mb. from 140.163.4.231 and then it timed out. that's terribly slow.
17:53:03:WU01:FS01:Assigned to work server 140.163.4.231
17:53:03:WU01:FS01:Requesting new work unit for slot 01: READY gpu:0:GP102 [GeForce GTX 1080 Ti] 11380 from 140.163.4.231
17:53:03:WU01:FS01:Connecting to 140.163.4.231:8080
17:53:04:WU01:FS01:Downloading 16.53MiB
17:53:16:WU01:FS01:Download 0.38%
17:53:29:WU01:FS01:Download 0.76%
17:53:42:WU01:FS01:Download 1.13%
...
18:49:03:WU01:FS01:Download 99.06%
18:49:16:WU01:FS01:Download 99.44%
18:49:29:WU01:FS01:Download 99.81%
18:49:34:WU01:FS01:Download complete
>>>>>> this time after 56m31s it successfully download 100% of 16.53 MB from 140.163.4.231
>>>>>> As expected, the WU starts up and processes normally:
18:49:35:WU01:FS01:Received Unit: id:01 state:DOWNLOAD error:NO_ERROR project:11713 run:7 clone:18 gen:268 core:0x21 unit:0x0000014d8ca304e759cee15aaec0b40a
18:49:35:WU01:FS01:Starting
18:49:35:WU01:FS01:Running FahCore: E:\FAHClient/FAHCoreWrapper.exe E:\FAH\cores/cores.foldingathome.org/Win32/AMD64/NVIDIA/Fermi/Core_21.fah/FahCore_21.exe -dir 01 -suffix 01 -version 705 -lifeline 1164 -checkpoint 30 -gpu-vendor nvidia -opencl-platform 0 -opencl-device 0 -cuda-device 0 -gpu 0
18:49:35:WU01:FS01:Started FahCore on PID 8296
18:49:35:WU01:FS01:Core PID:6548
18:49:35:WU01:FS01:FahCore 0x21 started
18:49:35:WU01:FS01:0x21:*********************** Log Started 2018-07-24T18:49:35Z ***********************
18:49:35:WU01:FS01:0x21:Project: 11713 (Run 7, Clone 18, Gen 268)
18:49:35:WU01:FS01:0x21:Unit: 0x0000014d8ca304e759cee15aaec0b40a
18:49:35:WU01:FS01:0x21:CPU: 0x00000000000000000000000000000000
18:49:35:WU01:FS01:0x21:Machine: 1
18:49:35:WU01:FS01:0x21:Reading tar file core.xml
18:49:35:WU01:FS01:0x21:Reading tar file integrator.xml
18:49:35:WU01:FS01:0x21:Reading tar file state.xml
18:49:49:WU01:FS01:0x21:Reading tar file system.xml
18:50:04:WU01:FS01:0x21:Digital signatures verified
18:50:04:WU01:FS01:0x21:Folding@home GPU Core21 Folding@home Core
18:50:04:WU01:FS01:0x21:Version 0.0.18
18:50:08:WU01:FS01:0x21:Completed 0 out of 7500000 steps (0%)
18:50:08:WU01:FS01:0x21:Temperature control disabled. Requirements: single Nvidia GPU, tmax must be < 110 and twait >= 900
18:51:44:WU01:FS01:0x21:Completed 75000 out of 7500000 steps (1%)
18:53:19:WU01:FS01:0x21:Completed 150000 out of 7500000 steps (2%)
18:54:54:WU01:FS01:0x21:Completed 225000 out of 7500000 steps (3%)
18:55:47:WARNING:FS01:Size of positions 2582 does not match topology 2580
18:56:45:WU01:FS01:0x21:Completed 300000 out of 7500000 steps (4%)
18:58:21:WU01:FS01:0x21:Completed 375000 out of 7500000 steps (5%)
18:59:56:WU01:FS01:0x21:Completed 450000 out of 7500000 steps (6%)
19:01:23:WARNING:FS01:Size of positions 2582 does not match topology 2580
19:01:48:WU01:FS01:0x21:Completed 525000 out of 7500000 steps (7%)
19:03:23:WU01:FS01:0x21:Completed 600000 out of 7500000 steps (8%)
19:04:59:WU01:FS01:0x21:Completed 675000 out of 7500000 steps (9%)
19:06:35:WU01:FS01:0x21:Completed 750000 out of 7500000 steps (10%)
19:06:57:WARNING:FS01:Size of positions 2582 does not match topology 2580
19:08:26:WU01:FS01:0x21:Completed 825000 out of 7500000 steps (11%)
19:10:02:WU01:FS01:0x21:Completed 900000 out of 7500000 steps (12%)
19:11:38:WU01:FS01:0x21:Completed 975000 out of 7500000 steps (13%)
19:12:31:WARNING:FS01:Size of positions 2582 does not match topology 2580
19:13:29:WU01:FS01:0x21:Completed 1050000 out of 7500000 steps (14%)
19:15:05:WU01:FS01:0x21:Completed 1125000 out of 7500000 steps (15%)
19:16:40:WU01:FS01:0x21:Completed 1200000 out of 7500000 steps (16%)
19:18:07:WARNING:FS01:Size of positions 2582 does not match topology 2580
19:18:32:WU01:FS01:0x21:Completed 1275000 out of 7500000 steps (17%)
19:20:08:WU01:FS01:0x21:Completed 1350000 out of 7500000 steps (18%)
19:21:43:WU01:FS01:0x21:Completed 1425000 out of 7500000 steps (19%)
19:23:19:WU01:FS01:0x21:Completed 1500000 out of 7500000 steps (20%)
19:23:47:WARNING:FS01:Size of positions 2582 does not match topology 2580
19:25:16:WU01:FS01:0x21:Completed 1575000 out of 7500000 steps (21%)
19:26:52:WU01:FS01:0x21:Completed 1650000 out of 7500000 steps (22%)
19:28:27:WU01:FS01:0x21:Completed 1725000 out of 7500000 steps (23%)
19:29:21:WARNING:FS01:Size of positions 2582 does not match topology 2580
19:30:18:WU01:FS01:0x21:Completed 1800000 out of 7500000 steps (24%)
19:31:54:WU01:FS01:0x21:Completed 1875000 out of 7500000 steps (25%)
19:33:29:WU01:FS01:0x21:Completed 1950000 out of 7500000 steps (26%)
19:34:55:WARNING:FS01:Size of positions 2582 does not match topology 2580
19:35:20:WU01:FS01:0x21:Completed 2025000 out of 7500000 steps (27%)
19:36:56:WU01:FS01:0x21:Completed 2100000 out of 7500000 steps (28%)
19:38:32:WU01:FS01:0x21:Completed 2175000 out of 7500000 steps (29%)
19:40:07:WU01:FS01:0x21:Completed 2250000 out of 7500000 steps (30%)
19:40:29:WARNING:FS01:Size of positions 2582 does not match topology 2580
19:41:59:WU01:FS01:0x21:Completed 2325000 out of 7500000 steps (31%)
19:43:34:WU01:FS01:0x21:Completed 2400000 out of 7500000 steps (32%)
19:45:10:WU01:FS01:0x21:Completed 2475000 out of 7500000 steps (33%)
19:46:03:WARNING:FS01:Size of positions 2582 does not match topology 2580
19:47:01:WU01:FS01:0x21:Completed 2550000 out of 7500000 steps (34%)
19:48:36:WU01:FS01:0x21:Completed 2625000 out of 7500000 steps (35%)
19:50:12:WU01:FS01:0x21:Completed 2700000 out of 7500000 steps (36%)
19:51:37:WARNING:FS01:Size of positions 2582 does not match topology 2580
19:52:03:WU01:FS01:0x21:Completed 2775000 out of 7500000 steps (37%)
19:53:39:WU01:FS01:0x21:Completed 2850000 out of 7500000 steps (38%)
19:55:14:WU01:FS01:0x21:Completed 2925000 out of 7500000 steps (39%)
19:56:49:WU01:FS01:0x21:Completed 3000000 out of 7500000 steps (40%)
19:57:11:WARNING:FS01:Size of positions 2582 does not match topology 2580
19:58:40:WU01:FS01:0x21:Completed 3075000 out of 7500000 steps (41%)
20:00:16:WU01:FS01:0x21:Completed 3150000 out of 7500000 steps (42%)
20:01:51:WU01:FS01:0x21:Completed 3225000 out of 7500000 steps (43%)
20:02:45:WARNING:FS01:Size of positions 2582 does not match topology 2580
20:03:42:WU01:FS01:0x21:Completed 3300000 out of 7500000 steps (44%)
20:05:18:WU01:FS01:0x21:Completed 3375000 out of 7500000 steps (45%)
20:06:53:WU01:FS01:0x21:Completed 3450000 out of 7500000 steps (46%)
20:08:19:WARNING:FS01:Size of positions 2582 does not match topology 2580
20:08:45:WU01:FS01:0x21:Completed 3525000 out of 7500000 steps (47%)
20:10:20:WU01:FS01:0x21:Completed 3600000 out of 7500000 steps (48%)
20:11:55:WU01:FS01:0x21:Completed 3675000 out of 7500000 steps (49%)
20:13:31:WU01:FS01:0x21:Completed 3750000 out of 7500000 steps (50%)
20:13:53:WARNING:FS01:Size of positions 2582 does not match topology 2580
20:15:22:WU01:FS01:0x21:Completed 3825000 out of 7500000 steps (51%)
20:16:58:WU01:FS01:0x21:Completed 3900000 out of 7500000 steps (52%)
20:18:33:WU01:FS01:0x21:Completed 3975000 out of 7500000 steps (53%)
20:19:27:WARNING:FS01:Size of positions 2582 does not match topology 2580
20:20:25:WU01:FS01:0x21:Completed 4050000 out of 7500000 steps (54%)
20:22:01:WU01:FS01:0x21:Completed 4125000 out of 7500000 steps (55%)
20:23:36:WU01:FS01:0x21:Completed 4200000 out of 7500000 steps (56%)
20:25:01:WARNING:FS01:Size of positions 2582 does not match topology 2580
20:25:27:WU01:FS01:0x21:Completed 4275000 out of 7500000 steps (57%)
20:27:03:WU01:FS01:0x21:Completed 4350000 out of 7500000 steps (58%)
20:28:38:WU01:FS01:0x21:Completed 4425000 out of 7500000 steps (59%)
20:30:14:WU01:FS01:0x21:Completed 4500000 out of 7500000 steps (60%)
20:30:37:WARNING:FS01:Size of positions 2582 does not match topology 2580
20:32:05:WU01:FS01:0x21:Completed 4575000 out of 7500000 steps (61%)
20:33:41:WU01:FS01:0x21:Completed 4650000 out of 7500000 steps (62%)
20:35:16:WU01:FS01:0x21:Completed 4725000 out of 7500000 steps (63%)
20:36:11:WARNING:FS01:Size of positions 2582 does not match topology 2580
20:37:07:WU01:FS01:0x21:Completed 4800000 out of 7500000 steps (64%)
20:38:43:WU01:FS01:0x21:Completed 4875000 out of 7500000 steps (65%)
20:40:19:WU01:FS01:0x21:Completed 4950000 out of 7500000 steps (66%)
20:41:45:WARNING:FS01:Size of positions 2582 does not match topology 2580
20:42:10:WU01:FS01:0x21:Completed 5025000 out of 7500000 steps (67%)
20:43:46:WU01:FS01:0x21:Completed 5100000 out of 7500000 steps (68%)
20:45:21:WU01:FS01:0x21:Completed 5175000 out of 7500000 steps (69%)
20:46:57:WU01:FS01:0x21:Completed 5250000 out of 7500000 steps (70%)
20:47:19:WARNING:FS01:Size of positions 2582 does not match topology 2580
20:48:48:WU01:FS01:0x21:Completed 5325000 out of 7500000 steps (71%)
20:50:23:WU01:FS01:0x21:Completed 5400000 out of 7500000 steps (72%)
20:51:59:WU01:FS01:0x21:Completed 5475000 out of 7500000 steps (73%)
20:52:53:WARNING:FS01:Size of positions 2582 does not match topology 2580
20:53:50:WU01:FS01:0x21:Completed 5550000 out of 7500000 steps (74%)
20:55:26:WU01:FS01:0x21:Completed 5625000 out of 7500000 steps (75%)
20:57:01:WU01:FS01:0x21:Completed 5700000 out of 7500000 steps (76%)
20:58:27:WARNING:FS01:Size of positions 2582 does not match topology 2580
20:58:52:WU01:FS01:0x21:Completed 5775000 out of 7500000 steps (77%)
21:00:28:WU01:FS01:0x21:Completed 5850000 out of 7500000 steps (78%)
21:02:03:WU01:FS01:0x21:Completed 5925000 out of 7500000 steps (79%)
21:03:39:WU01:FS01:0x21:Completed 6000000 out of 7500000 steps (80%)
21:04:01:WARNING:FS01:Size of positions 2582 does not match topology 2580
21:05:29:WU01:FS01:0x21:Completed 6075000 out of 7500000 steps (81%)
21:07:03:WU01:FS01:0x21:Completed 6150000 out of 7500000 steps (82%)
21:08:38:WU01:FS01:0x21:Completed 6225000 out of 7500000 steps (83%)
21:09:31:WARNING:FS01:Size of positions 2582 does not match topology 2580
21:10:28:WU01:FS01:0x21:Completed 6300000 out of 7500000 steps (84%)
21:12:04:WU01:FS01:0x21:Completed 6375000 out of 7500000 steps (85%)
21:13:40:WU01:FS01:0x21:Completed 6450000 out of 7500000 steps (86%)
21:15:05:WARNING:FS01:Size of positions 2582 does not match topology 2580
21:15:30:WU01:FS01:0x21:Completed 6525000 out of 7500000 steps (87%)
21:17:06:WU01:FS01:0x21:Completed 6600000 out of 7500000 steps (88%)
21:18:41:WU01:FS01:0x21:Completed 6675000 out of 7500000 steps (89%)
21:20:17:WU01:FS01:0x21:Completed 6750000 out of 7500000 steps (90%)
21:20:18:WU00:FS01:Connecting to 65.254.110.245:8080
21:20:19:WU00:FS01:Assigned to work server 128.252.203.4
21:20:19:WU00:FS01:Requesting new work unit for slot 01: RUNNING gpu:0:GP102 [GeForce GTX 1080 Ti] 11380 from 128.252.203.4
21:20:19:WU00:FS01:Connecting to 128.252.203.4:8080
21:20:20:WU00:FS01:Downloading 11.66MiB
21:20:33:WU00:FS01:Download 0.54%
21:20:37:WARNING:FS01:Size of positions 2582 does not match topology 2580
21:20:45:WU00:FS01:Download 1.07%
21:20:58:WU00:FS01:Download 1.61%
21:21:40:WU00:FS01:Download 2.14%
21:22:07:WU01:FS01:0x21:Completed 6825000 out of 7500000 steps (91%)
21:22:24:WU00:FS01:Download 2.68%
21:23:08:WU00:FS01:Download 3.22%
21:23:43:WU01:FS01:0x21:Completed 6900000 out of 7500000 steps (92%)
21:23:52:WU00:FS01:Download 3.75%
21:24:36:WU00:FS01:Download 4.29%
21:25:19:WU01:FS01:0x21:Completed 6975000 out of 7500000 steps (93%)
21:25:20:WU00:FS01:Download 4.83%
21:26:04:WU00:FS01:Download 5.36%
21:26:11:WARNING:FS01:Size of positions 2582 does not match topology 2580
21:26:49:WU00:FS01:Download 5.90%
21:27:09:WU01:FS01:0x21:Completed 7050000 out of 7500000 steps (94%)
21:27:34:WU00:FS01:Download 6.43%
21:28:19:WU00:FS01:Download 6.97%
21:28:46:WU01:FS01:0x21:Completed 7125000 out of 7500000 steps (95%)
21:29:04:WU00:FS01:Download 7.51%
21:29:48:WU00:FS01:Download 8.04%
21:30:22:WU01:FS01:0x21:Completed 7200000 out of 7500000 steps (96%)
21:30:32:WU00:FS01:Download 8.58%
21:31:16:WU00:FS01:Download 9.12%
21:31:41:WU00:FS01:Download 9.65%
21:31:45:WARNING:FS01:Size of positions 2582 does not match topology 2580
21:32:03:WU00:FS01:Download 10.19%
21:32:12:WU01:FS01:0x21:Completed 7275000 out of 7500000 steps (97%)
21:32:47:WU00:FS01:Download 10.72%
21:33:30:WU00:FS01:Download 11.26%
21:33:48:WU01:FS01:0x21:Completed 7350000 out of 7500000 steps (98%)
21:34:15:WU00:FS01:Download 11.80%
21:34:59:WU00:FS01:Download 12.33%
21:35:24:WU01:FS01:0x21:Completed 7425000 out of 7500000 steps (99%)
21:35:44:WU00:FS01:Download 12.87%
21:36:28:WU00:FS01:Download 13.40%
21:37:00:WU01:FS01:0x21:Completed 7500000 out of 7500000 steps (100%)
21:37:09:WU00:FS01:Download 13.94%
>>>>> After 16m50s it has successfully download only 13.94% of 11.66 MB or 1.63 MB from 128.252.203.4.
>>>>> Same extremely slow performance but from a different server.
21:37:14:WU01:FS01:0x21:Saving result file logfile_01.txt
21:37:14:WU01:FS01:0x21:Saving result file checkpointState.xml
21:37:14:WU01:FS01:0x21:Saving result file checkpt.crc
21:37:14:WU01:FS01:0x21:Saving result file log.txt
21:37:14:WU01:FS01:0x21:Saving result file positions.xtc
21:37:15:WU01:FS01:0x21:Folding@home Core Shutdown: FINISHED_UNIT
21:37:15:WU01:FS01:FahCore returned: FINISHED_UNIT (100 = 0x64)
21:37:15:WU01:FS01:Sending unit results: id:01 state:SEND error:NO_ERROR project:11713 run:7 clone:18 gen:268 core:0x21 unit:0x0000014d8ca304e759cee15aaec0b40a
21:37:15:WU01:FS01:Uploading 11.78MiB to 140.163.4.231
21:37:15:WU01:FS01:Connecting to 140.163.4.231:8080
21:37:24:WU00:FS01:Download 14.48%
21:37:49:WU00:FS01:Download 15.01%
21:38:32:WU01:FS01:Upload complete
21:38:32:WU01:FS01:Server responded WORK_ACK (400)
21:38:32:WU01:FS01:Final credit estimate, 110490.00 points
21:38:32:WU01:FS01:Cleaning up
21:38:33:WU00:FS01:Download 15.55%
21:39:17:WU00:FS01:Download 16.09%
,,,
22:04:51:WU00:FS01:Download 60.05%
22:05:04:WU00:FS01:Download 60.59%
22:05:22:WU00:FS01:Download 60.59%
22:05:22:ERROR:WU00:FS01:Exception: Transfer failed
>>>> after 22:05:22 - 21:20:19 = 45m02s it times out again after downloading 69,59% of 11.66 MB from 128.252.203.4.
>>>> Same terrible internet performance but
22:05:22:WU00:FS01:Connecting to 65.254.110.245:8080
22:05:23:WU00:FS01:Assigned to work server 140.163.4.231
22:05:23:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:GP102 [GeForce GTX 1080 Ti] 11380 from 140.163.4.231
22:05:23:WU00:FS01:Connecting to 140.163.4.231:8080
22:05:26:WU00:FS01:Downloading 16.52MiB
22:06:09:WU00:FS01:Download 0.38%
...
00:05:49:WU00:FS01:Download 81.36%
00:06:26:WU00:FS01:Download 81.36%
00:06:26:ERROR:WU00:FS01:Exception: Transfer failed
>>>>>>
00:06:26:WU00:FS01:Connecting to 65.254.110.245:8080
00:06:27:WU00:FS01:Assigned to work server 128.252.203.4
00:06:27:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:GP102 [GeForce GTX 1080 Ti] 11380 from 128.252.203.4
00:06:27:WU00:FS01:Connecting to 128.252.203.4:8080
00:06:28:WU00:FS01:Downloading 11.66MiB
00:07:01:WU00:FS01:Download 0.54%
00:07:45:WU00:FS01:Download 1.07%
00:08:29:WU00:FS01:Download 1.61%
00:09:14:WU00:FS01:Download 2.14%
00:09:59:WU00:FS01:Download 2.68%
00:10:44:WU00:FS01:Download 3.22%
00:11:28:WU00:FS01:Download 3.75%
00:12:12:WU00:FS01:Download 4.29%
00:12:58:WU00:FS01:Download 4.82%
00:13:43:WU00:FS01:Download 5.36%
00:14:28:WU00:FS01:Download 5.90%
00:15:12:WU00:FS01:Download 6.43%
00:15:56:WU00:FS01:Download 6.97%
00:16:41:WU00:FS01:Download 7.50%
00:17:26:WU00:FS01:Download 8.04%
00:18:10:WU00:FS01:Download 8.58%
00:18:34:WU00:FS01:Download 9.11%
00:19:19:WU00:FS01:Download 9.65%
00:20:03:WU00:FS01:Download 10.18%
00:20:48:WU00:FS01:Download 10.72%
00:21:33:WU00:FS01:Download 11.26%
00:22:17:WU00:FS01:Download 11.79%
00:23:02:WU00:FS01:Download 12.33%
00:23:47:WU00:FS01:Download 12.86%
00:24:32:WU00:FS01:Download 13.40%
00:25:17:WU00:FS01:Download 13.94%
00:26:01:WU00:FS01:Download 14.47%
00:26:46:WU00:FS01:Download 15.01%
00:27:31:WU00:FS01:Download 15.55%
00:28:15:WU00:FS01:Download 16.08%
00:29:00:WU00:FS01:Download 16.62%
00:29:44:WU00:FS01:Download 17.15%
00:30:28:WU00:FS01:Download 17.69%
00:31:12:WU00:FS01:Download 18.23%
00:31:56:WU00:FS01:Download 18.76%
00:32:41:WU00:FS01:Download 19.30%
00:33:25:WU00:FS01:Download 19.83%
00:34:10:WU00:FS01:Download 20.37%
00:34:54:WU00:FS01:Download 20.91%
00:35:39:WU00:FS01:Download 21.44%
00:36:24:WU00:FS01:Download 21.98%
00:37:09:WU00:FS01:Download 22.51%
00:37:54:WU00:FS01:Download 23.05%
00:38:38:WU00:FS01:Download 23.59%
00:39:24:WU00:FS01:Download 24.12%
00:40:08:WU00:FS01:Download 24.66%
00:40:52:WU00:FS01:Download 25.19%
00:41:37:WU00:FS01:Download 25.73%
00:42:22:WU00:FS01:Download 26.27%
00:43:07:WU00:FS01:Download 26.80%
00:43:51:WU00:FS01:Download 27.34%
00:44:35:WU00:FS01:Download 27.87%
00:45:20:WU00:FS01:Download 28.41%
00:46:04:WU00:FS01:Download 28.95%
00:46:49:WU00:FS01:Download 29.48%
00:47:33:WU00:FS01:Download 30.02%
00:48:19:WU00:FS01:Download 30.55%
00:49:04:WU00:FS01:Download 31.09%
00:49:49:WU00:FS01:Download 31.63%
00:50:34:WU00:FS01:Download 32.16%
00:51:06:WU00:FS01:Download 32.70%
00:51:30:WU00:FS01:Download 32.70%
00:51:30:ERROR:WU00:FS01:Exception: Transfer failed
>>>>>>
00:51:30:WU00:FS01:Connecting to 65.254.110.245:8080
00:51:31:WU00:FS01:Assigned to work server 140.163.4.231
00:51:31:WU00:FS01:Requesting new work unit for slot 01: READY gpu:0:GP102 [GeForce GTX 1080 Ti] 11380 from 140.163.4.231
00:51:31:WU00:FS01:Connecting to 140.163.4.231:8080
00:51:34:WU00:FS01:Downloading 5.42MiB
00:52:20:WU00:FS01:Download 1.15%
00:53:04:WU00:FS01:Download 2.31%
...
Full log:
https://pastebin.com/raw/qiUcx0pq
Since you're getting terrible performance from several different FAH servers, I conclude it's probably that your internet connection is simply too slow or it's completely saturated with other data going through the same connection.
What do you get if you run one of the popular speed tests?
Some speed tests:
Local:
US:
US:
US:
When uploading completed WU, it only took a few seconds to finish 10MB, as you can see in the log.
I don't think the downstream bandwidth is occupied by something else, because I can still download ~14MB/s.
Also, WU download failed regardless of day time or night time, as you can see in the logs:
http://s000.tinyupload.com/index.php?fi ... 7678740470
Do you have any test file on these servers for me to test the download speed?
Re: Slow WU downloads & failed transfer [140.163.4.231]
Posted: Fri Jul 27, 2018 4:09 pm
by bruce
At the very beginning of xfinity speed test, it reports the latency but they don't show it on the results page.
The AT&T test reports 113 ms but I don't know how far away the remote site was.
The third test shows short bursts of high speed data transfers that are repeatedly interrupted.
I presume that means you're connecting my satellite. Satellite connections tend to send large packets at a very high speed with lots of intervening pauses to overcome long latency. The AVERAGE speed may be good, but the big,fast packets may be slowed by intermediate steps. The latency can be a killer.
That third test looks like it failed but you didn't show that information. Note that it reports receiving multiple small files without explaining how long each one took and it apparently never finished. How many times did it say "Receiving data" and what happened after a total of 81% was downloaded? I'd say that looks a lot like your dowloads from FAH ... maybe less than 1% downloaded with frequent pauses for confirmation.
What does the "Speed Limiter" tab do?
At least it's something that our tech people can look into.
No, FAH does not support a speed test to/from any of it's servers.
Re: Slow WU downloads & failed transfer [140.163.4.231]
Posted: Fri Jul 27, 2018 5:25 pm
by lknl
bruce wrote:At the very beginning of xfinity speed test, it reports the latency but they don't show it on the results page.
at the beginning of xfinity speed test, there is latency value, but that value seems to be the same as the value shown at the result page, I'm not sure if that's what you are looking for.
bruce wrote:The AT&T test reports 113 ms but I don't know how far away the remote site was.
I'm in Singapore, I presume at&t test server is in the US, and I also presume the specific location of at&t test server in the US has little impact on latency in my case. But if you need to get the result with the specific server or specific location or specific speed test website, please tell me specifically.
bruce wrote:The third test shows short bursts of high speed data transfers that are repeatedly interrupted.
I'm neither a networking expert nor a researcher, and that's how common folks test the internet speed around here. I'm happy to learn something new today, but I'm also curious about how much more reliable the result from speed test websites is, because I notice that the speed shown during these tests also fluctuates.
bruce wrote:I presume that means you're connecting my satellite.
No, it's a cheap fiber internet plan. Sorry for not informing in the first post. I presume we are here to provide/obtain support and to solve problems in an effective way, so if you need more info, please let me know, I will try to provide as much as I can, there is no need to waste time posting unwanted info or guessing at readily available info.
bruce wrote:That third test looks like it failed but you didn't show that information. Note that it reports receiving multiple small files without explaining how long each one took and it apparently never finished. How many times did it say "Receiving data" and what happened after a total of 81% was downloaded? I'd say that looks a lot like your dowloads from FAH ... maybe less than 1% downloaded with a pause for confirmation every 5-6 seconds.
That screenshot was from a download manager software. Because you had a concern that my internet bandwidth was occupied by something else, so I arbitrarily downloaded some big file to show that the available bandwidth was more than adequate. After completed, the software didn't show the download speed, so I had to capture while it's still downloading. Of course, the download finished without any error.
If interested, you can read more about the "multiple small files" from the of the software's website
http://www.internetdownloadmanager.com/ ... ation.html
bruce wrote:What does the "Speed Limiter" tab do?
It's a feature of the download manager software to reduce the bandwidth allocated for downloading, so that there is bandwidth for other applications, in case the total bandwidth of the internet connection is low.
More info on their website
http://www.internetdownloadmanager.com/ ... ons11.html
bruce wrote:No, FAH does not support a speed test to/from any of it's servers.
Is there any other way to figure out whether the issue is at my side or server side?
Re: Slow WU downloads & failed transfer [140.163.4.231]
Posted: Fri Jul 27, 2018 5:38 pm
by bruce
lknl wrote:[...I'm neither a networking expert nor a researcher, and that's how common folks test the internet speed around here. I'm happy to learn something new today, but I'm also curious about how much more reliable the result from speed test websites is, because I notice that the speed shown during these tests also fluctuates.
bruce wrote:I presume that means you're connecting my satellite.
No, it's a cheap fiber internet plan. Sorry for not informing in the first post. I presume we are here to provide/obtain support and to solve problems in an effective way, so if you need more info, please let me know, I will try to provide as much as I can, there is no need to waste time posting unwanted info or guessing at readily available info.
bruce wrote:No, FAH does not support a speed test to/from any of it's servers.
Is there any other way to figure out whether the issue is at my side or server side?
Neither am I but we're both learning as we go. I suspect "cheap fiber" covers what's often called "the last mile" (or maybe that should be "km") The fiber only gets that data to a nearby satellite site for the long hop across the Pacific Ocean and then it probably goes back to fiber for the rest of the path. I have a meeting with some FAH technical people in a couple hours. I'll see if they know more than I do and/or if they need to do some testing or research.
On other network tests, I've seen numbers for "packet loss" Can you find any numbers?
Re: Slow WU downloads & failed transfer [140.163.4.231]
Posted: Fri Jul 27, 2018 5:51 pm
by bruce
We may learn something from traceroute (Linux) or tracert (Windows). It measures latency and jitter (variations in latency) using multiple pimgs. Unfortunately may intermediate routers drop ping packets so it's hard to find a suitable test site.
When I do a traceroute 140.163.4.231. it never gets to 140.163.4.231 but rather stops at 74.8.57.6 after 15 hops. What happens from Singapore?
Re: Slow WU downloads & failed transfer [140.163.4.231]
Posted: Fri Jul 27, 2018 6:08 pm
by lknl
for me, tracert also stopped at step 15 - 74.8.57.6 - but maybe the numbers can tell something.
rafwiewiora wrote:They're dropping pings and traceroutes - you won't get anything from those.
Code: Select all
tracert 140.163.4.231
Tracing route to plfah1-1.mskcc.org [140.163.4.231]
over a maximum of 30 hops:
1 1 ms 1 ms 1 ms 192.168.1.254
2 2 ms 2 ms 2 ms bb118-201-229-254.singnet.com.sg [118.201.229.254]
3 4 ms 3 ms 3 ms 202.166.123.134
4 2 ms 2 ms 2 ms 202.166.123.133
5 2 ms 2 ms 2 ms ae8-0.tp-cr03.singnet.com.sg [202.166.122.50]
6 3 ms 4 ms 3 ms ae4-0.tp-er03.singnet.com.sg [202.166.123.70]
7 2 ms 2 ms 2 ms 203.208.145.233
8 186 ms 186 ms 186 ms 203.208.182.122
9 190 ms 189 ms 189 ms ae3.cr0-pao1.ip4.gtt.net [216.221.158.129]
10 252 ms 252 ms 251 ms et-0-0-65.cr3-was1.ip4.gtt.net [89.149.142.242]
11 239 ms 239 ms 240 ms ip4.gtt.net [173.205.47.226]
12 242 ms 242 ms 242 ms ae13-0.cr02.asbn01-va.us.windstream.net [40.128.10.173]
13 259 ms 259 ms 259 ms be1.agr04.nwrk01-nj.us.windstream.net [40.128.248.7]
14 258 ms 258 ms 258 ms xe1-0-0-0.pe07.nwrk01-nj.us.windstream.net [40.128.249.134]
15 258 ms 258 ms 258 ms 74.8.57.6
16 * * * Request timed out.
17 * * * Request timed out.
18 * * * Request timed out.
19 * * * Request timed out.
20 * * * Request timed out.
21 * * * Request timed out.
22 * * * Request timed out.
23 * * * Request timed out.
24 * * * Request timed out.
25 * * * Request timed out.
26 * * * Request timed out.
27 * * * Request timed out.
28 * * * Request timed out.
29 * * * Request timed out.
30 * * * Request timed out.
Trace complete.
Re: Slow WU downloads & failed transfer [140.163.4.231]
Posted: Fri Jul 27, 2018 6:25 pm
by bruce
your data goes by fiber to step 7 and then your transpacific satellite is between step 7 and 8. Steps 8 to 15 are in the USA.
It takes 184 ms to make the transpacific hop (or really, half of that)..
Re: Slow WU downloads & failed transfer [140.163.4.231]
Posted: Fri Jul 27, 2018 6:39 pm
by lknl
That's very interesting to know - thanks for enlightening me.
Maybe I will dig more about this satellite hopping (for personal knowledge) when I have time.
Re: Slow WU downloads & failed transfer [140.163.4.231]
Posted: Fri Jul 27, 2018 7:49 pm
by Joe_H
Just one clarification, the longer delay starting at step 8 is not due to satellite transmission, that would be at least in the range of 250 ms. It is most likely due to delays in the router(s) connecting Singapore to the rest of the Internet and/or the Singapore Firewall added to the transmission time over the Pacific Ocean undersea cables.
In any case, at step 14 the response is from a router in NJ, almost to MSKCC. Unfortunately the traceroute does not give any further information at that point.
Re: Slow WU downloads & failed transfer [140.163.4.231]
Posted: Sat Jul 28, 2018 10:50 pm
by bruce
I'm somewhat familiar with the so-called "Great Firewall of China" and am aware that other countries may have rather restrictive firewalls. They also have legal penalties if citizens attempt to bypass the national firewall -- and we do not advocate that people attempt to do so illegally. In rare instances, special permission might be granted.
Is the situation in Singapore similar? Could your data be spending 184 ms being processed by a national firewall? If so, there's nothing we can do.