Questions on optimising threads for CPU/GPU

[Breach]

Further to some feedback I have received on another thread I'd like to ask a few basic questions on how to best allocate CPU threads when co-folding with a GPU (nVidia 670 in my case). My processor is 3770k (HT enabled, thus 8 logical cores).

I understand that when running both CPU/GPU at the same time it's best to configure the CPU threads to Max - 1 (and even Max - 2 in case of 2 AMD cards for example) to give a dedicated (logical) core to the GPU. I read somewhere that SMP folding is only as fast as the slowest thread - is this true?

I also understand that 7 threads is also not a good idea in all cases as some WUs would fail on 7 threads (though so far I haven't seen such a case).

My second question pertains to the default FAH assignment behaviour (when CPU thread allocation is set to -1). When I just run CPU folding it seems that FAH automatically assigns only 7 out of the 8 threads - why is that? I assume this is so other apps work on the available core hence not interrupting the FAH ones...?

What makes this however even more unclear is why when I start a GPU job the allocation moves the number of CPU threads from 7 to 8 threads? This seems counter-intuitive - in that case I would expect that FAH should actually keep/decrease the number of CPU threads to 7 or 6, not increase them to 8 which would cause the GPU job to interrupt the CPU folding...? Am I missing something here?

My final question concerns multi-threaded scaling and essentially what is the actual relative % drop in performance, say when I have 8 CPUs threads folding compared to 6 CPU threads folding the same SMP WU.

Many thanks.

[bruce]

Two fundamental facts that generally answer all your questions:
1) Your CPU has 4 cores and 8 threads. CPU folding depends on getting full utilization of the 4 cores, so CPU:4 through CPU:8 provide relatively minor changes to the CPU slot. (The're not zero, but they're small)
2) The NVidia GPU core_15 uses a relatively small amount of CPU time and it's quite happy with a single thread, adding almost nothing to the pair of threads that constitute a core. It won't slow up the companion CPU thread but it really benefits from having immediate access to a thread rather than having to interrupt a thread that's doing something else. (For the NV 670, measurable processing time has been known to report more CPU time. Windows TaskMan isn't always as accurate as it's supposed to be.)

I have not seen the number of CPU threads change the way you've described. Please provide a more complete explanaion of what you're changing and how you know it changed.

[Breach]

Thanks. So, for the my first question - the overall (CPU+GPU) performance with a GPU dedicated thread and 7 CPU threads is better compared to no dedicated GPU thread and 8 CPU threads, correct? If so I'll leave the CPU threads to 6 which is what I've already been told to do.

For my second question:

When I start processing a CPU WU unit (at -1 thread allocation) and GPU paused I see in the log:
21:38:59:WU00:FS01:0xa4:Mapping NT from 7 to 7
When I start the GPU slot I receive:
21:39:43:WARNING:WU00:FS01:Changed SMP threads from 7 to 8 this can cause some work units to fail

Hence my question and below is the complete log:

Code: Select all

*********************** Log Started 2013-04-12T21:38:52Z ***********************
21:38:52:************************* Folding@home Client *************************
21:38:52:      Website: http://folding.stanford.edu/
21:38:52:    Copyright: (c) 2009-2013 Stanford University
21:38:52:       Author: Joseph Coffland <joseph@cauldrondevelopment.com>
21:38:52:         Args: --open-web-control
21:38:52:       Config: C:/ProgramData/FAHClient/config.xml
21:38:52:******************************** Build ********************************
21:38:52:      Version: 7.3.6
21:38:52:         Date: Feb 18 2013
21:38:52:         Time: 15:25:17
21:38:52:      SVN Rev: 3923
21:38:52:       Branch: fah/trunk/client
21:38:52:     Compiler: Intel(R) C++ MSVC 1500 mode 1200
21:38:52:      Options: /TP /nologo /EHa /Qdiag-disable:4297,4103,1786,279 /Ox -arch:SSE
21:38:52:               /QaxSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 /Qopenmp /Qrestrict /MT /Qmkl
21:38:52:     Platform: win32 XP
21:38:52:         Bits: 32
21:38:52:         Mode: Release
21:38:52:******************************* System ********************************
21:38:52:          CPU: Intel(R) Core(TM) i7-3770K CPU @ 3.50GHz
21:38:52:       CPU ID: GenuineIntel Family 6 Model 58 Stepping 9
21:38:52:         CPUs: 8
21:38:52:       Memory: 7.88GiB
21:38:52:  Free Memory: 5.92GiB
21:38:52:      Threads: WINDOWS_THREADS
21:38:52:  Has Battery: false
21:38:52:   On Battery: false
21:38:52:   UTC offset: 2
21:38:52:          PID: 1068
21:38:52:          CWD: C:/ProgramData/FAHClient
21:38:52:           OS: Windows 8 Pro with Media Center
21:38:52:      OS Arch: AMD64
21:38:52:         GPUs: 1
21:38:52:        GPU 0: NVIDIA:3 GK104 [GeForce GTX 670]
21:38:52:         CUDA: 3.0
21:38:52:  CUDA Driver: 5000
21:38:52:Win32 Service: false
21:38:52:***********************************************************************
21:38:52:<config>
21:38:52:  <!-- Network -->
21:38:52:  <proxy v=':8080'/>
21:38:52:
21:38:52:  <!-- User Information -->
21:38:52:  <passkey v='********************************'/>
21:38:52:  <team v='845'/>
21:38:52:  <user v='Alexander_Ivanchev'/>
21:38:52:
21:38:52:  <!-- Folding Slots -->
21:38:52:  <slot id='0' type='GPU'/>
21:38:52:  <slot id='1' type='CPU'>
21:38:52:    <cpus v='-1'/>
21:38:52:  </slot>
21:38:52:</config>
21:38:52:Trying to access database...
21:38:52:Successfully acquired database lock
21:38:52:Enabled folding slot 00: PAUSED gpu:0:GK104 [GeForce GTX 670] (waiting for idle)
21:38:52:Enabled folding slot 01: READY cpu:7
21:38:52:WU00:FS01:Starting
21:38:52:WU00:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/ProgramData/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/Core_a4.fah/FahCore_a4.exe -dir 00 -suffix 01 -version 703 -lifeline 1068 -checkpoint 15 -np 7
21:38:52:WU00:FS01:Started FahCore on PID 6404
21:38:52:WU00:FS01:Core PID:2512
21:38:52:WU00:FS01:FahCore 0xa4 started
21:38:53:WU00:FS01:0xa4:
21:38:53:WU00:FS01:0xa4:*------------------------------*
21:38:53:WU00:FS01:0xa4:Folding@Home Gromacs GB Core
21:38:53:WU00:FS01:0xa4:Version 2.27 (Dec. 15, 2010)
21:38:53:WU00:FS01:0xa4:
21:38:53:WU00:FS01:0xa4:Preparing to commence simulation
21:38:53:WU00:FS01:0xa4:- Looking at optimizations...
21:38:53:WU00:FS01:0xa4:- Files status OK
21:38:53:WU00:FS01:0xa4:- Expanded 2053966 -> 5365960 (decompressed 261.2 percent)
21:38:53:WU00:FS01:0xa4:Called DecompressByteArray: compressed_data_size=2053966 data_size=5365960, decompressed_data_size=5365960 diff=0
21:38:53:WU00:FS01:0xa4:- Digital signature verified
21:38:53:WU00:FS01:0xa4:
21:38:53:WU00:FS01:0xa4:Project: 7808 (Run 7, Clone 197, Gen 33)
21:38:53:WU00:FS01:0xa4:
21:38:53:WU00:FS01:0xa4:Assembly optimizations on if available.
21:38:53:WU00:FS01:0xa4:Entering M.D.
21:38:55:3:127.0.0.1:New Web connection
21:38:59:WU00:FS01:0xa4:Mapping NT from 7 to 7 
21:38:59:WU00:FS01:0xa4:Completed 0 out of 1500000 steps  (0%)
21:39:01:WARNING:Exception: 8:127.0.0.1: Send error: 10053: An established connection was aborted by the software in your host machine.
21:39:35:FS01:Shutting core down
21:39:35:WU01:FS00:Starting
21:39:35:WU01:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/ProgramData/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/NVIDIA/Fermi/Core_15.fah/FahCore_15.exe -dir 01 -suffix 01 -version 703 -lifeline 1068 -checkpoint 15 -gpu 0 -gpu-vendor nvidia
21:39:35:WU01:FS00:Started FahCore on PID 2948
21:39:35:WU01:FS00:Core PID:5700
21:39:35:WU01:FS00:FahCore 0x15 started
21:39:35:WU01:FS00:0x15:
21:39:35:WU01:FS00:0x15:*------------------------------*
21:39:35:WU01:FS00:0x15:Folding@Home GPU Core
21:39:35:WU01:FS00:0x15:Version                2.25 (Wed May 9 17:03:01 EDT 2012)
21:39:35:WU01:FS00:0x15:Build host             AmoebaRemote
21:39:35:WU01:FS00:0x15:Board Type             NVIDIA/CUDA
21:39:35:WU01:FS00:0x15:Core                   15
21:39:35:WU01:FS00:0x15:
21:39:35:WU01:FS00:0x15:Window's signal control handler registered.
21:39:35:WU01:FS00:0x15:Preparing to commence simulation
21:39:35:WU01:FS00:0x15:- Looking at optimizations...
21:39:35:WU01:FS00:0x15:- Files status OK
21:39:35:WU01:FS00:0x15:sizeof(CORE_PACKET_HDR) = 512 file=<>
21:39:35:WU01:FS00:0x15:- Expanded 58300 -> 257358 (decompressed 441.4 percent)
21:39:35:WU01:FS00:0x15:Called DecompressByteArray: compressed_data_size=58300 data_size=257358, decompressed_data_size=257358 diff=0
21:39:35:WU01:FS00:0x15:- Digital signature verified
21:39:35:WU01:FS00:0x15:
21:39:35:WU01:FS00:0x15:Project: 8070 (Run 74, Clone 11, Gen 43)
21:39:35:WU01:FS00:0x15:
21:39:35:WU01:FS00:0x15:Assembly optimizations on if available.
21:39:35:WU01:FS00:0x15:Entering M.D.
21:39:37:WU01:FS00:0x15:Tpr hash 01/wudata_01.tpr:  643569616 2775577685 2716874991 2400108752 1712827670
21:39:37:WU01:FS00:0x15:GPU device id=0
21:39:37:WU01:FS00:0x15:Working on Gallium Rubidium Oxygen Manganese Argon Carbon Silicon t=  74.00000
21:39:37:WU01:FS00:0x15:Client config unavailable.
21:39:37:WU01:FS00:0x15:Starting GUI Server
21:39:43:WU00:FS01:FahCore returned: INTERRUPTED (102 = 0x66)
21:39:43:WU00:FS01:Starting
21:39:43:WARNING:WU00:FS01:Changed SMP threads from 7 to 8 this can cause some work units to fail
21:39:43:WU00:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/ProgramData/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/Core_a4.fah/FahCore_a4.exe -dir 00 -suffix 01 -version 703 -lifeline 1068 -checkpoint 15 -np 8
21:39:43:WU00:FS01:Started FahCore on PID 3744
21:39:43:WU00:FS01:Core PID:6664
21:39:43:WU00:FS01:FahCore 0xa4 started
21:39:43:WU00:FS01:0xa4:
21:39:43:WU00:FS01:0xa4:*------------------------------*
21:39:43:WU00:FS01:0xa4:Folding@Home Gromacs GB Core
21:39:43:WU00:FS01:0xa4:Version 2.27 (Dec. 15, 2010)
21:39:43:WU00:FS01:0xa4:
21:39:43:WU00:FS01:0xa4:Preparing to commence simulation
21:39:43:WU00:FS01:0xa4:- Looking at optimizations...
21:39:43:WU00:FS01:0xa4:- Files status OK
21:39:43:WU00:FS01:0xa4:- Expanded 2053966 -> 5365960 (decompressed 261.2 percent)
21:39:43:WU00:FS01:0xa4:Called DecompressByteArray: compressed_data_size=2053966 data_size=5365960, decompressed_data_size=5365960 diff=0
21:39:43:WU00:FS01:0xa4:- Digital signature verified
21:39:43:WU00:FS01:0xa4:
21:39:43:WU00:FS01:0xa4:Project: 7808 (Run 7, Clone 197, Gen 33)
21:39:43:WU00:FS01:0xa4:
21:39:43:WU00:FS01:0xa4:Assembly optimizations on if available.
21:39:43:WU00:FS01:0xa4:Entering M.D.
21:39:49:WU00:FS01:0xa4:Mapping NT from 8 to 8 
21:39:50:WU00:FS01:0xa4:Completed 0 out of 1500000 steps  (0%)

[P5-133XL]

The first thing I would like to address is the power slider. You will note that the number of CPU cores used depends on that slider unless you have manually forced the number of cores to be used using the advanced control. If you are manually setting the number of cores to be used then I would recommend SMP:6 as a good starting point for your 8-logical core CPU. There are several good reasons for that starting point.

The first is that like you said above, the individual threads normally run at the speed of the slowest. SMP threads are highly synchronized. The second characteristic is that SMP folding is designed with a very low priority so that if anything else requests some CPU resource then one or more threads will be suspended so that you can do what want with your computer. When an SMP thread is suspended, the other threads typically sit in an idle-loop waiting for the thread to get reactivated. Even small external CPU usage can have a dramatic effect on TPF's/PPD's. It is good to have a little unused CPU capability so that normal usage doesn't produce huge TPF impacts.

Next is the issue of Virtual-cores (hyper threading) and real cores. your specific CPU has 4 real cores and 4 simulated cores. The purpose of simulated cores is to keep all the internal capability of your CPU fully used For example if at a particular instant a true core is not using the floating point processor then one of the virtual cores can use it. As such, a virtual core normally runs much slower than a true core but if the instruction-mix is good, there is a total speed increase. The result is that not using two or more virtual cores for folding does not harm the folding PPD much but will potentially significantly improve it by allowing other activities to not suspend a true core from folding.

The reason why SMP:7 is discouraged is because a few projects/WU's do not like to be broken up in specific pieces. typically, large prime numbers of cores but other numbers are possible too. What they do is called decompensation and what happens is they eventually EUE and die. No one like to get 90% done and watch the WU die, and then see it happen again and again so we try to avoid it happening in the first place by saying you should avoid SMP:7, 11, 13, 19 (lower than 7 is not normally a problem) ... Almost always the person is using a virtual-core so the PPD loss is minimal, for the same reason as above.

The next issue is that of GPU processing combined with SMP. Under the standard GPU cores, Nvidia video cards typically uses a very small amount of CPU, While AMD/ATI cards use a full CPU core. With the brand new Core_17, that seems to be totally reversed. If your GPU is using a full core and you run your SMP using all the cores that your CPU is capable then SMP then Windows will suspend a SMP thread to allow your GPU to run. This will cause a very dramatic drop in your SMP productivity as the other SMP threads will sit in an idle loop waiting for your suspended SMP thread to reactivate. It is always better to run at SMP-1 (or even SMP-2) to prevent your CPU from being over-subscribed. If the GPU folding is using a nominal amount of the CPU then feel free to run SMP at your full CPU's capability.

The real reason that starting a GPU job moves the CPU threads from 7 to 8, for your CPU, is that you are probably doing so by moving the power slider. By default the slider runs GPU only when idle and SMP-1 but if you move the slider to full then SMP will be at maximum cores and the GPU folding will run at all times.

As to SMP scaling for SMP:6 vs SMP:8 no one can answer it accurately. There are just too many different variables to answer it universally for everybody. The drop to 6 cores will drop your PPD/TPF's somewhat but not a lot because you are just eliminating the virtual cores. I would suggest that you simply record what your PPD is at 8 for a few frames (3-5), then on the same WU move it to 6 and then let it run for a few more frames to allow for the PPD to settle down. That way for your machine you know...

[Breach]

Thanks a lot for the VERY comprehensive write up, appreciate it! Indeed at least with my GPU and the current Core 15 the CPU utilisation I am seeing is between 3-5%. I understand I have to measure whether 8 CPU threads and desktop apps + GPU interruptions to the SMP threads is better or worse as compared to 6 CPU threads, apps and a dedicated GPU thread. At any rate I understand results will be more or less comparable - so I'll just leave the CPU threads at 6.

Now, just two other points to clarify based on your answer purely because I am curious:

- You said that some WUs would fail with SMP threads = 7 - then why would that be the default assignment of threads at Medium given the dangers...?
- Indeed, what I am seeing matches the described behaviour of the power slider at Full which I was doing in this case. But wouldn't that behaviour on Full (8 SMP cores + GPU on) become an out of the box issue for folders running AMD/ATI cards, which are, as you said, more ' CPU hungry'?

Thanks.

[P5-133XL]

I have no idea why seven was chosen as the default. I suspect it was chosen because of AMD/ATI using a full core on conventional GPU cores so that when the GPU kicked in, when the client determined the machine was idle, it wouldn't kill-off the SMP productivity. However, to me it feels like a mistake and that -2 would have been a better default.

P.S. One also has to consider what happens on machines that have no virtual cores and those that only have either one or two total cores. SMP:-1 is probably just a compromise between all the possible negative consequences.

[PantherX]

Do note that those Projects which fail on SMP:7 are usually blocked on Server so aren't assigned to donors. Since only a relatively small number of projects have this issue, it can be managed if the project owners are notified. Moreover, the larger bad numbers like 11, etc are blacklisted in the FahCore itself so even if you configure your SMP slot with 11, the FahCore will remap it to 10 and fold on your system.