work units not sending. Also, idle not working properly

[Volnaiskra]

So I've now reinstalled FAH using the express install. Here is the new log file. Does it seem correct, compared to the last one?

Code: Select all

*********************** Log Started 2014-05-03T04:00:45Z ***********************
04:00:45:************************* Folding@home Client *************************
04:00:45:      Website: http://folding.stanford.edu/
04:00:45:    Copyright: (c) 2009-2014 Stanford University
04:00:45:       Author: Joseph Coffland <joseph@cauldrondevelopment.com>
04:00:45:         Args: --open-web-control
04:00:45:       Config: <none>
04:00:45:******************************** Build ********************************
04:00:45:      Version: 7.4.4
04:00:45:         Date: Mar 4 2014
04:00:45:         Time: 20:26:54
04:00:45:      SVN Rev: 4130
04:00:45:       Branch: fah/trunk/client
04:00:45:     Compiler: Intel(R) C++ MSVC 1500 mode 1200
04:00:45:      Options: /TP /nologo /EHa /Qdiag-disable:4297,4103,1786,279 /Ox -arch:SSE
04:00:45:               /QaxSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 /Qopenmp /Qrestrict /MT /Qmkl
04:00:45:     Platform: win32 XP
04:00:45:         Bits: 32
04:00:45:         Mode: Release
04:00:45:******************************* System ********************************
04:00:45:          CPU: Intel(R) Core(TM) i7-4770K CPU @ 3.50GHz
04:00:45:       CPU ID: GenuineIntel Family 6 Model 60 Stepping 3
04:00:45:         CPUs: 8
04:00:45:       Memory: 15.93GiB
04:00:45:  Free Memory: 13.27GiB
04:00:45:      Threads: WINDOWS_THREADS
04:00:45:   OS Version: 6.1
04:00:45:  Has Battery: false
04:00:45:   On Battery: false
04:00:45:   UTC Offset: 10
04:00:45:          PID: 2820
04:00:45:          CWD: C:/Users/Dave/AppData/Roaming/FAHClient
04:00:45:           OS: Windows 7 Home Premium
04:00:45:      OS Arch: AMD64
04:00:45:         GPUs: 3
04:00:45:        GPU 0: NVIDIA:3 GK110 [GeForce GTX Titan]
04:00:45:        GPU 1: NVIDIA:3 GK110 [GeForce GTX Titan]
04:00:45:        GPU 2: NVIDIA:3 GK106 [GeForce GTX 650 Ti]
04:00:45:         CUDA: 3.5
04:00:45:  CUDA Driver: 6000
04:00:45:Win32 Service: false
04:00:45:***********************************************************************
04:00:45:<config>
04:00:45:  <!-- Folding Slots -->
04:00:45:</config>
04:00:45:Connecting to assign-GPU.stanford.edu:80
04:00:48:Updated GPUs.txt
04:00:48:Read GPUs.txt
04:00:48:Trying to access database...
04:00:48:Successfully acquired database lock
04:00:48:Enabled folding slot 00: PAUSED cpu:4 (not configured)
04:00:48:Enabled folding slot 01: PAUSED gpu:0:GK110 [GeForce GTX Titan] (not configured)
04:00:48:Enabled folding slot 02: PAUSED gpu:1:GK110 [GeForce GTX Titan] (not configured)
04:00:48:Enabled folding slot 03: PAUSED gpu:2:GK106 [GeForce GTX 650 Ti] (not configured)
04:00:55:16:127.0.0.1:New Web connection
04:01:49:Saving configuration to config.xml
04:01:49:<config>
04:01:49:  <!-- Folding Slots -->
04:01:49:  <slot id='0' type='CPU'/>
04:01:49:  <slot id='1' type='GPU'/>
04:01:49:  <slot id='2' type='GPU'/>
04:01:49:  <slot id='3' type='GPU'/>
04:01:49:</config>
04:01:49:Set client configured
04:01:49:WU00:FS00:Connecting to 171.67.108.200:8080
04:01:49:WU01:FS01:Connecting to 171.67.108.200:8080
04:01:49:WU02:FS02:Connecting to 171.67.108.200:8080
04:01:50:WU03:FS03:Connecting to 171.67.108.200:8080
04:01:50:WU00:FS00:Connecting to 171.67.108.200:8080
04:01:50:WU01:FS01:Connecting to 171.67.108.201:80
04:01:50:WU02:FS02:Connecting to 171.67.108.201:80
04:01:50:WU03:FS03:Connecting to 171.67.108.201:80
04:01:51:WU00:FS00:Assigned to work server 171.64.65.124
04:01:51:WU00:FS00:Requesting new work unit for slot 00: READY cpu:4 from 171.64.65.124
04:01:51:WU00:FS00:Connecting to 171.64.65.124:8080
04:01:51:WU01:FS01:Assigned to work server 140.163.4.231
04:01:51:WU01:FS01:Requesting new work unit for slot 01: READY gpu:0:GK110 [GeForce GTX Titan] from 140.163.4.231
04:01:51:WU01:FS01:Connecting to 140.163.4.231:8080
04:01:51:WU02:FS02:Assigned to work server 140.163.4.231
04:01:51:WU02:FS02:Requesting new work unit for slot 02: READY gpu:1:GK110 [GeForce GTX Titan] from 140.163.4.231
04:01:51:WU02:FS02:Connecting to 140.163.4.231:8080
04:01:51:WU03:FS03:Assigned to work server 140.163.4.231
04:01:51:WU03:FS03:Requesting new work unit for slot 03: READY gpu:2:GK106 [GeForce GTX 650 Ti] from 140.163.4.231
04:01:51:WU03:FS03:Connecting to 140.163.4.231:8080
04:01:52:WU01:FS01:Downloading 4.84MiB
04:01:52:WU02:FS02:Downloading 4.83MiB
04:01:52:WU03:FS03:Downloading 4.84MiB
04:01:52:WU00:FS00:Downloading 855.02KiB
04:01:58:WU01:FS01:Download 10.34%
04:01:58:WU02:FS02:Download 9.05%
04:01:58:WU00:FS00:Download 29.94%
04:01:58:WU03:FS03:Download 10.34%
04:02:04:WU00:FS00:Download 67.37%
04:02:04:WU02:FS02:Download 16.81%
04:02:04:WU03:FS03:Download 20.68%
04:02:04:WU01:FS01:Download 18.09%
04:02:09:WU00:FS00:Download complete
04:02:09:WU00:FS00:Received Unit: id:00 state:DOWNLOAD error:NO_ERROR project:9006 run:493 clone:2 gen:42 core:0xa4 unit:0x0000002c664f2de4533b2baa5d7556e5
04:02:09:WU00:FS00:Downloading core from http://www.stanford.edu/~pande/Win32/AMD64/Core_a4.fah
04:02:09:WU00:FS00:Connecting to www.stanford.edu:80
04:02:10:WU03:FS03:Download 31.02%
04:02:10:WU01:FS01:Download 25.85%
04:02:10:WU02:FS02:Download 27.15%
04:02:11:WU00:FS00:FahCore a4: Downloading 2.89MiB
04:02:16:WU01:FS01:Download 36.19%
04:02:16:WU03:FS03:Download 41.35%
04:02:16:WU02:FS02:Download 34.90%
04:02:17:WU00:FS00:FahCore a4: 8.66%
04:02:22:WU01:FS01:Download 43.94%
04:02:22:WU03:FS03:Download 54.28%
04:02:22:WU02:FS02:Download 42.66%
04:02:23:WU00:FS00:FahCore a4: 19.48%
04:02:28:WU03:FS03:Download 64.62%
04:02:28:WU01:FS01:Download 51.69%
04:02:28:WU02:FS02:Download 51.71%
04:02:31:WU00:FS00:FahCore a4: 23.81%
04:02:34:WU02:FS02:Download 59.47%
04:02:34:WU01:FS01:Download 60.74%
04:02:35:WU03:FS03:Download 76.25%
04:02:37:WU00:FS00:FahCore a4: 30.30%
04:02:40:WU01:FS01:Download 71.08%
04:02:40:WU02:FS02:Download 67.22%
04:02:41:WU03:FS03:Download 84.00%
04:02:44:WU00:FS00:FahCore a4: 38.96%
04:02:46:WU02:FS02:Download 72.39%
04:02:46:WU01:FS01:Download 80.13%
04:02:47:WU03:FS03:Download 98.22%
04:02:47:WU03:FS03:Download complete
04:02:47:WU03:FS03:Received Unit: id:03 state:DOWNLOAD error:NO_ERROR project:13001 run:276 clone:5 gen:9 core:0x17 unit:0x00000013538b3db753289e0c5beea249
04:02:47:WU03:FS03:Downloading core from http://www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_17.fah
04:02:47:WU03:FS03:Connecting to www.stanford.edu:80
04:02:48:WU03:FS03:FahCore 17: Downloading 2.55MiB
04:02:50:WU00:FS00:FahCore a4: 45.45%
04:02:52:WU01:FS01:Download 90.47%
04:02:52:WU02:FS02:Download 80.15%
04:02:55:WU03:FS03:FahCore 17: 12.25%
04:02:58:WU00:FS00:FahCore a4: 54.10%
04:02:58:WU02:FS02:Download 94.37%
04:03:01:WU03:FS03:FahCore 17: 26.96%
04:03:04:WU00:FS00:FahCore a4: 77.91%
04:03:07:WU03:FS03:FahCore 17: 58.81%
04:03:09:WU00:FS00:FahCore a4: Download complete
04:03:09:WU00:FS00:Valid core signature
04:03:09:WU00:FS00:Unpacked 9.59MiB to cores/www.stanford.edu/~pande/Win32/AMD64/Core_a4.fah/FahCore_a4.exe
04:03:09:WU00:FS00:Starting
04:03:09:WU00:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Dave/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/Core_a4.fah/FahCore_a4.exe -dir 00 -suffix 01 -version 704 -lifeline 2820 -checkpoint 15 -np 4
04:03:09:WU00:FS00:Started FahCore on PID 6124
04:03:10:WU00:FS00:Core PID:1820
04:03:10:WU00:FS00:FahCore 0xa4 started
04:03:10:WU00:FS00:0xa4:
04:03:10:WU00:FS00:0xa4:*------------------------------*
04:03:10:WU00:FS00:0xa4:Folding@Home Gromacs GB Core
04:03:10:WU00:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
04:03:10:WU00:FS00:0xa4:
04:03:10:WU00:FS00:0xa4:Preparing to commence simulation
04:03:10:WU00:FS00:0xa4:- Looking at optimizations...
04:03:10:WU00:FS00:0xa4:- Created dyn
04:03:10:WU00:FS00:0xa4:- Files status OK
04:03:10:WU00:FS00:0xa4:- Expanded 875031 -> 1434864 (decompressed 163.9 percent)
04:03:10:WU00:FS00:0xa4:Called DecompressByteArray: compressed_data_size=875031 data_size=1434864, decompressed_data_size=1434864 diff=0
04:03:10:WU00:FS00:0xa4:- Digital signature verified
04:03:10:WU00:FS00:0xa4:
04:03:10:WU00:FS00:0xa4:Project: 9006 (Run 493, Clone 2, Gen 42)
04:03:10:WU00:FS00:0xa4:
04:03:10:WU00:FS00:0xa4:Assembly optimizations on if available.
04:03:10:WU00:FS00:0xa4:Entering M.D.
04:03:13:WU03:FS03:FahCore 17: 90.67%
04:03:14:WU03:FS03:FahCore 17: Download complete
04:03:14:WU03:FS03:Valid core signature
04:03:14:WU03:FS03:Unpacked 8.60MiB to cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_17.fah/FahCore_17.exe
04:03:14:WU03:FS03:Starting
04:03:14:WU03:FS03:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Dave/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_17.fah/FahCore_17.exe -dir 03 -suffix 01 -version 704 -lifeline 2820 -checkpoint 15 -gpu 2 -gpu-vendor nvidia
04:03:14:WU03:FS03:Started FahCore on PID 4164
04:03:15:WU03:FS03:Core PID:5524
04:03:15:WU03:FS03:FahCore 0x17 started
04:03:15:WU03:FS03:0x17:*********************** Log Started 2014-05-03T04:03:15Z ***********************
04:03:15:WU03:FS03:0x17:Project: 13001 (Run 276, Clone 5, Gen 9)
04:03:15:WU03:FS03:0x17:Unit: 0x00000013538b3db753289e0c5beea249
04:03:15:WU03:FS03:0x17:CPU: 0x00000000000000000000000000000000
04:03:15:WU03:FS03:0x17:Machine: 3
04:03:15:WU03:FS03:0x17:Reading tar file state.xml
04:03:15:WU03:FS03:0x17:Reading tar file system.xml
04:03:16:WU03:FS03:0x17:Reading tar file integrator.xml
04:03:16:WU03:FS03:0x17:Reading tar file core.xml
04:03:16:WU03:FS03:0x17:Digital signatures verified
04:03:16:WU03:FS03:0x17:Folding@home GPU core17
04:03:16:WU03:FS03:0x17:Version 0.0.52
04:03:16:WU00:FS00:0xa4:Mapping NT from 4 to 4 
04:03:16:WU00:FS00:0xa4:Completed 0 out of 250000 steps  (0%)
04:03:51:Saving configuration to config.xml
04:03:51:<config>
04:03:51:  <!-- User Information -->
04:03:51:  <passkey v='********************************'/>
04:03:51:  <team v='225663'/>
04:03:51:  <user v='Volnaiskra'/>
04:03:51:
04:03:51:  <!-- Folding Slots -->
04:03:51:  <slot id='0' type='CPU'/>
04:03:51:  <slot id='1' type='GPU'/>
04:03:51:  <slot id='2' type='GPU'/>
04:03:51:  <slot id='3' type='GPU'/>
04:03:51:</config>

[P5-133XL]

All those WU's were successfully uploaded. Why you are not getting acknowledgements, I can't say nor why it didn't continue folding unless you pulled the log before it was all done.

Hi Volnaiskra (team 225663),
Your WU (P13001 R93 C6 G11) was added to the stats database on 2014-04-28 13:03:15 for 60170.2 points of credit.

Hi Volnaiskra (team 225663),
Your WU (P13000 R1038 C1 G12) was added to the stats database on 2014-04-28 13:03:15 for 60136 points of credit.

Hi Volnaiskra (team 225663),
Your WU (P13001 R537 C5 G9) was added to the stats database on 2014-04-30 07:03:50 for 68320.8 points of credit.

Hi Volnaiskra (team 225663),
Your WU (P13000 R3 C4 G13) was added to the stats database on 2014-04-30 07:03:50 for 72961.1 points of credit.

Hi Volnaiskra (team 225663),
Your WU (P9406 R234 C0 G20) was added to the stats database on 2014-04-30 00:02:05 for 42136.5 points of credit.

Hi Volnaiskra (team 225663),
Your WU (P8570 R1 C4 G321) was added to the stats database on 2014-04-28 21:09:19 for 10166.2 points of credit.

[P5-133XL]

So I've now reinstalled FAH using the express install. Here is the new log file. Does it seem correct, compared to the last one?

I'd cap the CPU to a maximum of 5 threads and preferably 4 threads in the CPU slot.

[Volnaiskra]

All those WU's were successfully uploaded. Why you are not getting acknowledgements, I can't say nor why it didn't continue folding unless you pulled the log before it was all done.

Hi Volnaiskra (team 225663),
Your WU (P13001 R93 C6 G11) was added to the stats database on 2014-04-28 13:03:15 for 60170.2 points of credit.

Hi Volnaiskra (team 225663),
Your WU (P13000 R1038 C1 G12) was added to the stats database on 2014-04-28 13:03:15 for 60136 points of credit.

Hi Volnaiskra (team 225663),
Your WU (P13001 R537 C5 G9) was added to the stats database on 2014-04-30 07:03:50 for 68320.8 points of credit.

Hi Volnaiskra (team 225663),
Your WU (P13000 R3 C4 G13) was added to the stats database on 2014-04-30 07:03:50 for 72961.1 points of credit.

Hi Volnaiskra (team 225663),
Your WU (P9406 R234 C0 G20) was added to the stats database on 2014-04-30 00:02:05 for 42136.5 points of credit.

Hi Volnaiskra (team 225663),
Your WU (P8570 R1 C4 G321) was added to the stats database on 2014-04-28 21:09:19 for 10166.2 points of credit.

Where did you get those "Hi Volnaiskra" messages? Are you saying I got the points, but my FAHClient wasn't informed of the upload?

[P5-133XL]

I checked the database and that is what PG says you got for each of those WU's. The Hi Volnaiskra ... is just the format of the output that I get from their query program after I entered the Project (Run, Gen, Clone) numbers and then I simply copy and pasted it here.

To answer your question before you ask it: No, you do not have access to the database. For performance reasons it is not available to the general public but is limited to moderator access.

The best explanation as to why it didn't appear in the log is that you pulled the log and posted it here before it was done. However, I suppose there are other possible explanations as to why the client stopped logging. Do you have follow checkmarked ...

[bruce]

Some ISP connections (I think with a proxy) block the acknowledgement message. We need to figure out how to make sure the client receives the confirmation messages from the server. I remember that we figured that out once before but I don't remember the solution right now. Hopefully somebody else will provide that information.

[Volnaiskra]

So what you're saying is that if this happens again, there may be nothing I can do about it. But I CAN ignore the problem, because folding and points will continue properly. Even though so-called 'incomplete' WUs will build up in the display of FAHcontrol, they won't actually be incomplete in reality?

But actually, before we blame the ISP, I remind you that my other PCs don't have this problem. They complete units and assign points without any hitches. And they're using the same router.

[PantherX]

One possible reason that you aren't getting the acknowledgement message is that your bandwidth is being saturated. It is a known issue where multiple uploads can cause an issue especially if you have a slow/unstable internet connection (https://fah.stanford.edu/projects/FAHClient/ticket/1038).

Since you have two systems using the same internet connection, only one system is facing the issue, right? In that case, what is unique about this problematic system, heavy internet use, firewall issue, data-capping application, etc? To remove a possibility of a faulty cable/port, swap the cables then the port between the systems (if possible) and see if the issue is resolved or not. Moreover, is there a pattern when the issue appears, i.e. only at certain times of the day, when a particular application is running, etc.

Regarding the build-up of WUs, since the Servers receive the WUs, the science can continue and you are being awarded points. However, since the completed ones aren't removed, they will continue to build-up until the newest WUs can't be uploaded. If you are certain that the completed WU has been successfully sent, you can manually delete the WU (go to the Data Directory and in the Work Folder, delete the folder which matches the Work Queue ID).

[bruce]

Repeatedly uploading WUs which have already been received is NOT a problem which should be ignored and if anything I said was taken to imply that it should be ignored, I want to correct that impression right now. It does reduce the urgency of the problem but it still needs to be solved, and we're happy to work with you to solve it. It's just that my faulty (organic) memory remembers we figured it out once but doesn't remember what we "learned."

[Volnaiskra]

OK, the problem seems to be happening on both my main PCs. I'd never seen it happen on the secondary PC before, but I've checked now, and there are two "sending 100%" WUs stuck on it.

On my main PC, since reinstalling FAH using the express install, the client is no longer greyed out, and is folding properly. But it is still getting "sending 100%" stuck WUs (three of them currently).

I haven't noticed any pattern. It usually seems to happen at night, but that may be simply because my PCs are idle then and that's when they do the most folding. Though now I think about it, it does seem to often happen when there's more than one upload at once, which would support your 'multiple uploads' theory. It also tends to happen mainly to my Titans. I previously thought that was because they're my strongest hardware and therefore get through the most WUs. But it may be because they're identical cards, and therefore tend to start and finish WUs at the same time (hence, they tend to both upload at the same time)

My internet is pretty reliable, with very infrequent dropouts (a few times a year at most)

My internet speed is decent by local (Australian) standards, but probably abysmal by global standards, because our internet infrastructure is in the Dark Ages, with a government to match. I have ADSL, and my upload speed according to speedtest.net is 0.53Mbps. Could that account for it?

[PantherX]

My upload speed varies from 0.1 Mbps to 0.41 Mbps (with very frequent dropouts) and in most cases, I always get the confirmation. However, there are occasions where I don't receive the confirmation message. A simple test that you can perform is to set all your F@H Slots to Finish (on all your systems) and once you have ensured that no bandwidth heavy applications are running (bit-torrent, video streaming, etc), only start folding on a single Titan and let it fold and see if you get the confirmation message or not.

[Volnaiskra]

And now two of my earlier problems have resurfaced. Firstly, FAHcontrol is permanently greyed out. Secondly, it's currently folding all three GPUs (but CPU activity is low) - it's not supposed to, because it's set to "idle".

Here's a shot of task manager, AFTER I quit FAH from the system tray. All GPUS are still on 100%; CPU usage is low.


The startup link to FAH has a target of "C:\Program Files (x86)\FAHClient\HideConsole.exe" C:\Program Files (x86)\FAHClient\FAHClient.exe
and a "start in" value of "C:\Users\Dave\AppData\Roaming\FAHClient"

This is the latest log file, dated a couple of minutes ago
The entire log is too big to upload. So this is the start of it:

Code: Select all

*********************** Log Started 2014-05-04T13:50:05Z ***********************
13:50:05:************************* Folding@home Client *************************
13:50:05:      Website: http://folding.stanford.edu/
13:50:05:    Copyright: (c) 2009-2014 Stanford University
13:50:05:       Author: Joseph Coffland <joseph@cauldrondevelopment.com>
13:50:05:         Args: 
13:50:05:       Config: C:/Users/Dave/AppData/Roaming/FAHClient/config.xml
13:50:05:******************************** Build ********************************
13:50:05:      Version: 7.4.4
13:50:05:         Date: Mar 4 2014
13:50:05:         Time: 20:26:54
13:50:05:      SVN Rev: 4130
13:50:05:       Branch: fah/trunk/client
13:50:05:     Compiler: Intel(R) C++ MSVC 1500 mode 1200
13:50:05:      Options: /TP /nologo /EHa /Qdiag-disable:4297,4103,1786,279 /Ox -arch:SSE
13:50:05:               /QaxSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 /Qopenmp /Qrestrict /MT /Qmkl
13:50:05:     Platform: win32 XP
13:50:05:         Bits: 32
13:50:05:         Mode: Release
13:50:05:******************************* System ********************************
13:50:05:          CPU: Intel(R) Core(TM) i7-4770K CPU @ 3.50GHz
13:50:05:       CPU ID: GenuineIntel Family 6 Model 60 Stepping 3
13:50:05:         CPUs: 8
13:50:05:       Memory: 15.93GiB
13:50:05:  Free Memory: 14.13GiB
13:50:05:      Threads: WINDOWS_THREADS
13:50:05:   OS Version: 6.1
13:50:05:  Has Battery: false
13:50:05:   On Battery: false
13:50:05:   UTC Offset: 10
13:50:05:          PID: 4272
13:50:05:          CWD: C:/Users/Dave/AppData/Roaming/FAHClient
13:50:05:           OS: Windows 7 Home Premium
13:50:05:      OS Arch: AMD64
13:50:05:         GPUs: 3
13:50:05:        GPU 0: NVIDIA:3 GK110 [GeForce GTX Titan]
13:50:05:        GPU 1: NVIDIA:3 GK110 [GeForce GTX Titan]
13:50:05:        GPU 2: NVIDIA:3 GK106 [GeForce GTX 650 Ti]
13:50:05:         CUDA: 3.5
13:50:05:  CUDA Driver: 6000
13:50:05:Win32 Service: false
13:50:05:***********************************************************************
13:50:05:<config>
13:50:05:  <!-- Slot Control -->
13:50:05:  <power v='FULL'/>
13:50:05:
13:50:05:  <!-- User Information -->
13:50:05:  <passkey v='********************************'/>
13:50:05:  <team v='225663'/>
13:50:05:  <user v='Volnaiskra'/>
13:50:05:
13:50:05:  <!-- Folding Slots -->
13:50:05:  <slot id='0' type='CPU'>
13:50:05:    <idle v='true'/>
13:50:05:  </slot>
13:50:05:  <slot id='1' type='GPU'>
13:50:05:    <idle v='true'/>
13:50:05:  </slot>
13:50:05:  <slot id='2' type='GPU'>
13:50:05:    <idle v='true'/>
13:50:05:  </slot>
13:50:05:  <slot id='3' type='GPU'>
13:50:05:    <idle v='true'/>
13:50:05:  </slot>
13:50:05:</config>
13:50:05:Trying to access database...
13:50:12:Successfully acquired database lock
13:50:13:Enabled folding slot 00: PAUSED cpu:5 (waiting for idle)
13:50:13:Enabled folding slot 01: PAUSED gpu:0:GK110 [GeForce GTX Titan] (waiting for idle)
13:50:13:Enabled folding slot 02: PAUSED gpu:1:GK110 [GeForce GTX Titan] (waiting for idle)
13:50:13:Enabled folding slot 03: PAUSED gpu:2:GK106 [GeForce GTX 650 Ti] (waiting for idle)
13:50:15:WU01:FS02:Sending unit results: id:01 state:SEND error:NO_ERROR project:13001 run:498 clone:6 gen:6 core:0x17 unit:0x00000015538b3db7532c8978f965ee3d
13:50:18:WU01:FS02:Uploading 12.83MiB to 140.163.4.231
13:50:18:WU00:FS01:Sending unit results: id:00 state:SEND error:NO_ERROR project:13000 run:435 clone:0 gen:14 core:0x17 unit:0x00000023538b3db7531016a3eb7582ed
13:50:18:WU01:FS02:Connecting to 140.163.4.231:8080
13:50:19:WU00:FS01:Uploading 12.83MiB to 140.163.4.231
13:50:19:WU00:FS01:Connecting to 140.163.4.231:8080
13:50:19:WU02:FS00:Sending unit results: id:02 state:SEND error:NO_ERROR project:7504 run:3 clone:98 gen:185 core:0xa3 unit:0x000000e8fbcb017d4e39b5e8e564325d
13:50:19:WU02:FS00:Uploading 9.71MiB to 128.143.199.97
13:50:19:WU02:FS00:Connecting to 128.143.199.97:8080
13:50:25:WU01:FS02:Upload 1.46%

And this is the end of it

Code: Select all

06:44:46:WU10:FS02:0x15:Completed  28800000 out of 40000000 steps (72%).
06:45:23:WU06:FS00:0xa3:Completed 465000 out of 500000 steps  (93%)
06:47:48:WU09:FS01:0x15:Completed  28000000 out of 40000000 steps (70%).
06:48:36:WU10:FS02:0x15:Completed  29200000 out of 40000000 steps (73%).
06:49:30:WU06:FS00:0xa3:Completed 470000 out of 500000 steps  (94%)
06:51:38:WU09:FS01:0x15:Completed  28400000 out of 40000000 steps (71%).
06:52:26:WU10:FS02:0x15:Completed  29600000 out of 40000000 steps (74%).
06:53:36:WU06:FS00:0xa3:Completed 475000 out of 500000 steps  (95%)
06:55:28:WU09:FS01:0x15:Completed  28800000 out of 40000000 steps (72%).
06:56:16:WU10:FS02:0x15:Completed  30000000 out of 40000000 steps (75%).
06:57:41:WU06:FS00:0xa3:Completed 480000 out of 500000 steps  (96%)
06:59:29:WU09:FS01:0x15:Completed  29200000 out of 40000000 steps (73%).
07:00:06:WU10:FS02:0x15:Completed  30400000 out of 40000000 steps (76%).
07:01:47:WU06:FS00:0xa3:Completed 485000 out of 500000 steps  (97%)
07:03:22:WU09:FS01:0x15:Completed  29600000 out of 40000000 steps (74%).
07:03:56:WU10:FS02:0x15:Completed  30800000 out of 40000000 steps (77%).
07:05:54:WU06:FS00:0xa3:Completed 490000 out of 500000 steps  (98%)
07:07:42:WU09:FS01:0x15:Completed  30000000 out of 40000000 steps (75%).
07:07:46:WU10:FS02:0x15:Completed  31200000 out of 40000000 steps (78%).
07:09:59:WU06:FS00:0xa3:Completed 495000 out of 500000 steps  (99%)
07:10:00:WU05:FS00:Connecting to 171.67.108.200:8080
07:10:01:WU05:FS00:Assigned to work server 171.64.65.124
07:10:01:WU05:FS00:Requesting new work unit for slot 00: RUNNING cpu:5 from 171.64.65.124
07:10:01:WU05:FS00:Connecting to 171.64.65.124:8080
07:10:02:WU05:FS00:Downloading 862.05KiB
07:10:08:WU05:FS00:Download 74.24%
07:10:09:WU05:FS00:Download complete
07:10:09:WU05:FS00:Received Unit: id:05 state:DOWNLOAD error:NO_ERROR project:9005 run:702 clone:2 gen:55 core:0xa4 unit:0x0000003f664f2de4533b27d5a71dd782
07:11:32:WU09:FS01:0x15:Completed  30400000 out of 40000000 steps (76%).
07:11:37:WU10:FS02:0x15:Completed  31600000 out of 40000000 steps (79%).
07:12:42:WU03:FS03:0x17:Completed 2700000 out of 5000000 steps (54%)
07:14:06:WU06:FS00:0xa3:Completed 500000 out of 500000 steps  (100%)
07:14:06:WU06:FS00:0xa3:DynamicWrapper: Finished Work Unit: sleep=10000
07:14:16:WU06:FS00:0xa3:
07:14:16:WU06:FS00:0xa3:Finished Work Unit:
07:14:16:WU06:FS00:0xa3:- Reading up to 5030784 from "06/wudata_01.trr": Read 5030784
07:14:16:WU06:FS00:0xa3:trr file hash check passed.
07:14:16:WU06:FS00:0xa3:- Reading up to 5366712 from "06/wudata_01.xtc": Read 5366712
07:14:16:WU06:FS00:0xa3:xtc file hash check passed.
07:14:16:WU06:FS00:0xa3:edr file hash check passed.
07:14:16:WU06:FS00:0xa3:logfile size: 325618
07:14:16:WU06:FS00:0xa3:Leaving Run
07:14:19:WU06:FS00:0xa3:- Writing 10755518 bytes of core data to disk...
07:14:20:WU06:FS00:0xa3:Done: 10755006 -> 10180552 (compressed to 94.6 percent)
07:14:20:WU06:FS00:0xa3:  ... Done.
07:14:20:WU06:FS00:0xa3:- Shutting down core
07:14:20:WU06:FS00:0xa3:
07:14:20:WU06:FS00:0xa3:Folding@home Core Shutdown: FINISHED_UNIT
07:14:21:WU06:FS00:FahCore returned: FINISHED_UNIT (100 = 0x64)
07:14:21:WU06:FS00:Sending unit results: id:06 state:SEND error:NO_ERROR project:7504 run:5 clone:129 gen:38 core:0xa3 unit:0x0000002cfbcb017d4e39b98b4c13dc18
07:14:21:WU06:FS00:Uploading 9.71MiB to 128.143.199.97
07:14:21:WU06:FS00:Connecting to 128.143.199.97:8080
07:14:28:WU06:FS00:Upload 2.57%
07:14:36:WU06:FS00:Upload 4.51%
07:14:43:WU06:FS00:Upload 6.44%
07:14:50:WU06:FS00:Upload 8.37%
07:14:58:WU06:FS00:Upload 10.30%
07:15:05:WU06:FS00:Upload 12.23%
07:15:13:WU06:FS00:Upload 14.16%
07:15:20:WU06:FS00:Upload 16.09%
07:15:22:WU09:FS01:0x15:Completed  30800000 out of 40000000 steps (77%).
07:15:27:WU10:FS02:0x15:Completed  32000000 out of 40000000 steps (80%).
07:15:27:WU06:FS00:Upload 18.02%
07:15:35:WU06:FS00:Upload 19.95%
07:15:42:WU06:FS00:Upload 21.89%
07:15:49:WU06:FS00:Upload 23.82%
07:15:57:WU06:FS00:Upload 25.75%
07:16:04:WU06:FS00:Upload 27.68%
07:16:11:WU06:FS00:Upload 29.61%
07:16:19:WU06:FS00:Upload 31.54%
07:16:26:WU06:FS00:Upload 33.47%
07:16:34:WU06:FS00:Upload 35.40%
07:16:41:WU06:FS00:Upload 37.33%
07:16:48:WU06:FS00:Upload 39.27%
07:16:56:WU06:FS00:Upload 41.20%
07:17:03:WU06:FS00:Upload 43.13%
07:17:10:WU06:FS00:Upload 45.06%
07:17:18:WU06:FS00:Upload 46.99%
07:17:25:WU06:FS00:Upload 48.92%
07:17:33:WU06:FS00:Upload 50.85%
07:17:40:WU06:FS00:Upload 52.78%
07:17:47:WU06:FS00:Upload 54.71%
07:17:55:WU06:FS00:Upload 56.65%
07:18:02:WU06:FS00:Upload 58.58%
07:18:09:WU06:FS00:Upload 60.51%
07:18:17:WU06:FS00:Upload 62.44%
07:18:24:WU06:FS00:Upload 64.37%
07:18:31:WU06:FS00:Upload 66.30%
07:18:39:WU06:FS00:Upload 68.23%
07:18:46:WU06:FS00:Upload 70.16%
07:18:53:WU06:FS00:Upload 72.09%
07:19:01:WU06:FS00:Upload 74.03%
07:19:08:WU06:FS00:Upload 75.96%
07:19:16:WU06:FS00:Upload 77.89%
07:19:17:WU10:FS02:0x15:Completed  32400000 out of 40000000 steps (81%).
07:19:20:WU09:FS01:0x15:Completed  31200000 out of 40000000 steps (78%).
07:19:23:WU06:FS00:Upload 79.82%
07:19:30:WU06:FS00:Upload 81.75%
07:19:38:WU06:FS00:Upload 83.68%
07:19:45:WU06:FS00:Upload 85.61%
07:19:52:WU06:FS00:Upload 87.54%
07:20:00:WU06:FS00:Upload 89.47%
07:20:07:WU06:FS00:Upload 91.41%
07:20:14:WU06:FS00:Upload 93.34%
07:20:22:WU06:FS00:Upload 95.27%
07:20:29:WU06:FS00:Upload 97.20%
07:20:37:WU06:FS00:Upload 99.13%
07:23:07:WU10:FS02:0x15:Completed  32800000 out of 40000000 steps (82%).
07:23:37:WU09:FS01:0x15:Completed  31600000 out of 40000000 steps (79%).
07:26:57:WU10:FS02:0x15:Completed  33200000 out of 40000000 steps (83%).
07:27:34:WU09:FS01:0x15:Completed  32000000 out of 40000000 steps (80%).
07:30:47:WU10:FS02:0x15:Completed  33600000 out of 40000000 steps (84%).
07:31:28:WU09:FS01:0x15:Completed  32400000 out of 40000000 steps (81%).
07:34:37:WU10:FS02:0x15:Completed  34000000 out of 40000000 steps (85%).
07:35:40:WU09:FS01:0x15:Completed  32800000 out of 40000000 steps (82%).
07:38:27:WU10:FS02:0x15:Completed  34400000 out of 40000000 steps (86%).
07:39:35:WU09:FS01:0x15:Completed  33200000 out of 40000000 steps (83%).
07:42:17:WU10:FS02:0x15:Completed  34800000 out of 40000000 steps (87%).
07:43:36:WU09:FS01:0x15:Completed  33600000 out of 40000000 steps (84%).
07:46:07:WU10:FS02:0x15:Completed  35200000 out of 40000000 steps (88%).
07:47:26:WU09:FS01:0x15:Completed  34000000 out of 40000000 steps (85%).
07:49:57:WU10:FS02:0x15:Completed  35600000 out of 40000000 steps (89%).
******************************* Date: 2014-05-05 *******************************
07:51:37:WU09:FS01:0x15:Completed  34400000 out of 40000000 steps (86%).
07:53:36:WU03:FS03:0x17:Completed 2750000 out of 5000000 steps (55%)
07:53:47:WU10:FS02:0x15:Completed  36000000 out of 40000000 steps (90%).
07:55:28:WU09:FS01:0x15:Completed  34800000 out of 40000000 steps (87%).
07:57:37:WU10:FS02:0x15:Completed  36400000 out of 40000000 steps (91%).
07:59:25:WU09:FS01:0x15:Completed  35200000 out of 40000000 steps (88%).
08:01:27:WU10:FS02:0x15:Completed  36800000 out of 40000000 steps (92%).
08:03:16:WU09:FS01:0x15:Completed  35600000 out of 40000000 steps (89%).
08:05:18:WU10:FS02:0x15:Completed  37200000 out of 40000000 steps (93%).
08:07:07:WU09:FS01:0x15:Completed  36000000 out of 40000000 steps (90%).
08:09:11:WU10:FS02:0x15:Completed  37600000 out of 40000000 steps (94%).
08:10:59:WU09:FS01:0x15:Completed  36400000 out of 40000000 steps (91%).
08:13:05:WU10:FS02:0x15:Completed  38000000 out of 40000000 steps (95%).
08:14:51:WU09:FS01:0x15:Completed  36800000 out of 40000000 steps (92%).
08:16:57:WU10:FS02:0x15:Completed  38400000 out of 40000000 steps (96%).
08:18:43:WU09:FS01:0x15:Completed  37200000 out of 40000000 steps (93%).
08:20:51:WU10:FS02:0x15:Completed  38800000 out of 40000000 steps (97%).
08:22:36:WU09:FS01:0x15:Completed  37600000 out of 40000000 steps (94%).
08:24:45:WU10:FS02:0x15:Completed  39200000 out of 40000000 steps (98%).

[PantherX]

The task manager doesn't list FahControl. Even it is greyed out, it should still show up in Task Manager.

According to the initial log file, FAHClient is waiting for the system to be idle before it will start folding:
13:50:13:Enabled folding slot 00: PAUSED cpu:5 (waiting for idle)
13:50:13:Enabled folding slot 01: PAUSED gpu:0:GK110 [GeForce GTX Titan] (waiting for idle)
13:50:13:Enabled folding slot 02: PAUSED gpu:1:GK110 [GeForce GTX Titan] (waiting for idle)
13:50:13:Enabled folding slot 03: PAUSED gpu:2:GK106 [GeForce GTX 650 Ti] (waiting for idle)

Did you later change the configuration or not?

The initial section shows that it is attempting to upload WU01, WU00 and WU02. The second log file shows an upload of WU06 which stops at 99.13% upload. Do you still have the previous WUs in the queue or not? Were you running any bandwidth intensive application while WU06 was being uploaded (or any other WU for that matter)?

[Volnaiskra]

I guess FAHcontrol isn't listed, because I closed it from the system tray. It seems that it was the only FAH program that managed to close - all others remained in RAM.

No, I didn't change the configuration. It was set to idle, and the icon had "zzz" on it, but the GPUs were nevertheless churning away on 100% usage.

I can't tell if those WUs are in the queue, because it's still greyed out (after a windows restart). But the 99.13% upload just seems like more of the same thing that's been happening throughout the past few days: WUs upload all the way till about 99%, then all record of them on the logs are halted (but from what you've said, the WUs do actually get successfully uploaded to the server).

I don't believe I was using any bandwidth intensive application at all at that time.

[PantherX]

The WU was successfully returned:

Hi Volnaiskra (team 225663),
Your WU (P7504 R5 C129 G38) was added to the stats database on 2014-05-05 01:11:09 for 4568.54 points of credit.

Do note that FAHControl doesn't have an icon in the Notification Tray, only FAHClient icon is present in the Notification Tray. With the recent V7 release, it is acting more like a service than a program so it is expected that FAHClient runs continuously in the background. What is not expected is that it folds when the system is in use since you have explicitly told it to fold only when idle. Since you can't connect to FAHControl, can you try Web Control:
1) http://localhost:7396/
2) http://folding.stanford.edu/client/

The zzz icon means that all Slots are paused which is expected with the above log file (all slots are paused waiting for idle). I do hope that you have installed only a single instance of V7 instead of multiple instances by accident.