Configurable affinity changer for Windows

[baobab]

I imagine most of you have heard of the other affinity changer program and its PPD benefits when used with the supported folding clients/numbers of cores. And I think there are also many who would like a more flexible affinity changer that works with other non-standard configurations.

I was one of these people when I bought my Intel Quad core about half a year ago. I wanted to run a Linux SMP client in a VMWare virtual machine on two of the cores, and a Windows SMP client on the other two cores. At that time I decided to write my own affinity changer which would deal with my particular setup. I also wanted the program to automatically detect the two pairs of cores that share an L2 cache on the Intel quad, and schedule each client on one of the pairs of cores. This task took only one week-end at that time, and I used this code for the last 6 months without any problems. However, my affinity changer was hard coded for my particular setup.

Three weeks ago when I decided to give a try to the new GPU client, I realized that I needed a more flexible affinity changer program. In the last 2-3 weeks I transformed my affinity changer in a configurable program that can be used to manage affinities for arbitrary applications and arbitrary number of cores, with the goal to also make a public release.

The philosophy behind this new affinity changer was to be as flexible as possible (unfortunately that are still things that you cannot do), at the expense of requiring some cooperation from the user who must specify which applications should be managed by the affinity changer, how many CPUs to assign to each application, and optionally what policy to use for assigning processes to CPUs.

In addition, the code uses an heuristic to detect eventual asymmetries in the CPU configuration, in other words if certain CPUs have a higher affinity for each other. This heuristic detects the cores that share an L2 cache on an Intel Quad with very high confidence, creating logical CPU IDs to identify them, i.e. PAIR0 and PAIR1 for an Intel quad.

These logical CPU IDs can then be used in the affinity profiles that the user writes in a configuration file. These application profiles are very concise and easy to write once you understand the syntax. For example, to specify that a VMWare virtual machine should run on two cores that share an L2 cache, you write a profile as the one below:
*\vmware-vmx.exe := PAIR1

Without going into further detail now, I point you to the first public release of this program, which is distributed under the GNU public license.
The program can be downloaded from: http://sourceforge.net/project/platform ... _id=238550
Some lengthy documentation which hopefully will give you an idea of how the program works, is included with the program, but can also be viewed online at: http://affinitychanger.sourceforge.net/. The documentation includes also 6 simple configuration files to give you a better idea of how things are glued together.

Finally, you might have noticed that in all my examples I referred to Intel Quads. The reason for this is that I own such a CPU and it was the only architecture that I could test the program on.
However, the program aims to be general and it works with any architecture and number of cores, at least with physical CPU IDs (see the link above for an explanation). The only thing of which I am uncertain at this time is if the heuristic that detects asymmetric CPU configurations works well with AMD quad cores, or with dual socket systems. Some discussion about this topic can be found here: http://affinitychanger.sourceforge.net/#section_3

So I would like to hear any reports from people with more "exotic" configurations, where by exotic I mean a system with more than 4 cores, or a 4 core system that is not an Intel quad.
Of course, I am interested in positive or negative reports from other people as well. This is the first release, and some things might not work as expected in the wild.

[l8nights]

I can't wait to see some results and maybe a guide. Thanks for this contribution.

[MoneyGuyBK]

I can't wait to see some results and maybe a guide. Thanks for this contribution.

+1




Peace

[baobab]

I am also looking forward to some reports. But people need to try it and post if they had any issue or not.

Regarding your second point, there is a guide of sorts at the second link included in my first post. But I agree that that documentation is a bit dense (to be kind).

I wonder if it is easier for you (and I mean any of you) to just tell me what type of configuration you are running on your machine. I mean what folding clients, and how many clients of each type, and also what type of CPU(s) do you have in your machine, and I can explain how to write the configuration file for your concrete case.

You can also read my comments for the provided sample configuration files, and maybe the section that describes the configuration file format.

I do plan to rewrite part of the online documentation, but I am not optimistic that it will be something very short and clear for everybody. So maybe having some concrete examples and then specific questions and answers will be more helpful.

I am in the Houston area and I have no power at home currently. I just stopped by my office to check on a few things and I do have power here. Probably I will be able to check back on this thread tomorrow afternoon or evening.

[MoneyGuyBK]

baobab.... first off, my hats off to you for caring to check on us while you are in the Big "H" with no power at home.... I hope all turns out well for you and your loved ones.
I lived there from 78 to 95 and my worst hurricane was Alicia in 83

I wonder if it is easier for you (and I mean any of you) to just tell me what type of configuration you are running on your machine. I mean what folding clients, and how many clients of each type, and also what type of CPU(s) do you have in your machine, and I can explain how to write the configuration file for your concrete case.

My setups.....
1) Beast = Dell XPS720H2C Vista 64-UE ... QX6800, VM (Running Linux) plus GPU Client using 8800 Ultra ... 8Gig RAM ... this is the machine I do lots of things with
2) Monster = DIY Folding Exclusive, running 2 currently: 9800GX2 cards, hoping to add a third; might add a CPU client or two if I can!!! ... Q6600, 4Gig RAM

TIA




Peace

[l8nights]

I'm hoping all is okay down there in H-town, and watching the news I assume you are still without power for a bit

but I'm running it (hopefully succesfully LOL) all in all it seems to do all that is intended.

my question is the lock cpu setting should be turned off in fah clients correct?

basically I just copied the "start in" from my shortcut properties and added "\FahCore_*.exe :=" followed by one of these options..

"CPU, PAIR, TRIO, or QUAD, followed by a number (with no space in-between). " from the read me.

thanks again for the contribution, and hope you faired well in Ike.

[baobab]

First, thanks for your concerns. Personally I did not suffer any property damage and the only issue I have is the lack of electricity.
There are people who are flooded or suffered other damage, so while I miss electricity, it could have been worse.

I am stopping by my office these days because I have power and AC here. So there is no problem checking on this thread as well.

@MoneyGuy:
The good news about your setup is that you are using the Intel quads for which the program can detect with 100% accuracy the two pairs of cores that share an L2 cache and we can use this info for your affinity profiles.
The not so good news is that you are running Vista 64 and I do not know if there is going to be some incompatibility or not. I welcome any feedback if the program runs for you or not on Vista 64.

1) Beast = Dell XPS720H2C Vista 64-UE ... QX6800, VM (Running Linux) plus GPU Client using 8800 Ultra ... 8Gig RAM ... this is the machine I do lots of things with

1) The Beast:
Are you using a VMWare virtual machine? If so, the VM process is called vmware-vmx.exe and you get better performance if you run it on two cores that share a L2 cache. The profile for the VM machine is very simple:

Code: Select all

*\vmware-vmx.exe := PAIR1

This tells AffinityChanger to set affinity for vmware-vmx.exe to the two cores that are part of the second logical PAIR. As I mentioned before, for Intel quads the program detects the two pairs of cores that share an L2 cache and we can refer to these pairs through the logical names PAIR0 and PAIR1.

The program matches running processes based on their full name which includes the full path to the executable. So you cannot just say "vmware-vmx.exe := PAIR1" because it will not match anything. You can either type the full path where vmware-vmx.exe is located, or because we can safely assume there is no other program called vmware-vmx.exe installed on your system, we can just use a wildcard ('*') which will match any path. That's why I wrote it that way.

Because PAIR1 is used for the VM, we are going to run the GPU client on one of the cores of the first pair, PAIR0. We can refer to the individual cores of a PAIR using the resolution operator "::". Thus, PAIR0::CPU0 and PAIR0::CPU1 are the logical names of the two cores of the first pair.
Let's say we run the GPU client on the second core of PAIR0. The profile is:

Code: Select all

C:\Path_to_GPU_client\FahCore_*.exe := PAIR0::CPU1

You have to replace "C:\Path_to_GPU_client" with the actual path where the GPU client is installed on your machine. Personally I have seen only FahCore_11.exe cores for the GPU2 client. But I wrote the name as "FahCore_*.exe" to match other GPU cores if Stanford decides to release a "FahCore_12.exe" or something else. You can also write separate profiles for each FahCore version if that is what you want.

So, all you need is a file called "affinityinput.txt" which includes these two lines:

Code: Select all

*\vmware-vmx.exe := PAIR1
C:\Path_to_GPU_client\FahCore_*.exe := PAIR0::CPU1

You can also add a line:

Code: Select all

TestMode = 1

anywhere in the file if you want to check the actions of the program without having any affinities modified. But be sure to remove or comment that line if you want to actually have the affinities changed based on those profiles.

2) Monster = DIY Folding Exclusive, running 2 currently: 9800GX2 cards, hoping to add a third; might add a CPU client or two if I can!!! ... Q6600, 4Gig RAM

I understand you got two 9800GX2 cards? Each card has two GPUs. So are you running four GPU folding clients or two? I am going to write something for each case.
Each folding client is in a separate folder. We are going to use this info to distinguish between them. To use any of these configurations, you need to copy (plus modify the paths as needed) those lines into a file called "affinityinput.txt" located in the same folder where AffinityChangerConsole.exe is.

a) If you run four folding clients, we assign one to each CPU, so we can specify physical CPU identifiers (we do not care about which cores share an L2 cache or not). There are going to be four profiles, one per line:

Code: Select all

C:\Path_to_GPU_client_1\FahCore_*.exe := CPU0
C:\Path_to_GPU_client_2\FahCore_*.exe := CPU1
C:\Path_to_GPU_client_3\FahCore_*.exe := CPU2
C:\Path_to_GPU_client_4\FahCore_*.exe := CPU3

Again, you need to replace those fictional paths with the actual paths on your machine.

b) Still with four GPU clients, but you can try different ways to group them to make better use of the L2 cache on each PAIR.
If different work units require different amounts of memory, another way of running them if to balance the use of the L2 cache.
There are

b1) We can group the client using most memory and the client using least memory onto the same logical PAIR, and the two middle clients on the other PAIR. You can specify something like that with one profile:

Code: Select all

*\FahCore_11.exe := PAIR0::CPU0+PAIR1+PAIR0::CPU1 [assign=1,resource=MEMUSE,policy=PSEUDOBALANCED]

Because the path is specified as a wildcard, this profile will match all FahCore_11.exe running, wherever they are installed. So it will match all four GPU clients.
The part "PAIR0::CPU0+PAIR1+PAIR0::CPU1" is the CPU mask for this profile and it includes all four CPUs, just ordered in a funny way.

The "assign=1" attribute specifies how many CPUs to assign per process, in this case we want one CPU/process.
Flags "resource=MEMUSE,policy=PSEUDOBALANCED" causes all matching processes to be sorted in descending order by memory use.
CPUs are going to be assigned to processes in the order in which they are listed in the CPU mask. So I wrote it in that funny way to make sure the largest and the smallest processes are assigned to PAIR0 and the two middle ones to PAIR1.

b2) Or we can group the largest two clients onto the same PAIR, so the smaller ones are not slowed-down by the large ones:

Code: Select all

*\FahCore_11.exe := PAIR0+PAIR1 [assign=1,resource=MEMUSE,policy=PSEUDOBALANCED]

This profile is similar to the one in (b1), with just the CPU mask written in a different way to have the two largest processes on PAIR0.

I do not know which configuration will produce most PPDs. You can try each one for a limited time and see if there is any benefit from running one of the balanced profiles over just running the dumber configuration from (a).

c) If you are using only two GPU clients, then it might be better to run each of them on cores that are on different PAIRs, so they can use the full L2 cache of each PAIR.

Code: Select all

C:\Path_to_GPU_client_1\FahCore_*.exe := PAIR0::CPU0
C:\Path_to_GPU_client_2\FahCore_*.exe := PAIR1::CPU0

d) If you run two GPU clients and one VMware, then it is better to have the VMWare on a PAIR by itself and the two GPUs on the other PAIR:

Code: Select all

C:\Path_to_GPU_client_1\FahCore_*.exe := PAIR0::CPU0
C:\Path_to_GPU_client_2\FahCore_*.exe := PAIR0::CPU1
*\vmware-vmx.exe := PAIR1

Well, I hope this gives you an idea of how you can write these profiles.
AffinityChanger does not have any predefined profiles for any application. But it offers a fair amount of flexibility in the way of writing custom profiles.
One can also try different configurations and test which one produces the best PPD. But the barrier of entry is understanding the syntax for writing these profiles.

@l8nights:
I think that lock cpu setting just sets some affinity when the client starts. But it is not locked in the sense that somebody else cannot change it.
So, it does not really matter if you turn it off or not. AffinityChanger checks for processes every 30 seconds, so there is going to be a 15 seconds delay on average before affinities are adjusted for a new process.

You can tell if the program works or not by looking at the output on the console. The program prints on console the names of all processes whose affinities are adjusted. But also make sure TestMode is not enabled, or it will print on console, but not change any affinities. Once you get the configuration file the way you want, comment out the line which says "TestMode = 1" if you have one.

[MoneyGuyBK]

baobab ..... Thanx for the wonderful teachings.....
Being or at least considering myself a noob.... I will have to do a lot of reading, and get my nerves up to doing what all you have so very hard worked on to customize for my Beast and Monster.... I definitely appreciate that, Kudos.

And to note, yes you did guess correct VM = vmWare Workstation for Windows, and I run Linux_Client (SMP) in Ubuntu... That's The Beast.
The Monster, I am hoping to up it to 3 cards (9800GX2) for Six GPU clients, and add a SMP, Winodws, but more so leaning towards Linux/VM again.

Glad to hear you are ok, my friend in Houston tells me he will not have power until Tuesday at the earliest, he was thinking of traveling to San Antonio to get away, maybe he has.... I will call him...... God Bless.



Peace

[baobab]

That's cool MG. Thanks for bringing some life into this thread anyway.

I've finally read through some other parts of this forum and I noticed your Monster thread. I hope you get that 3rd card recognized by the OS.
You can have the clients assigned in any way you want, but you'd have to decide on the configuration because I did not play with that many clients on a single machine. My guess is that filling two cores with 6 GPU clients (3 per core), and then running a VM on the other two cores, would work fine from what I've read in your thread.

And BTW, I got power here on Monday. I hope your friend got power back as well. Things are slowly getting better.

[Xilikon]

Didn't have time to read everything but I'm really happy to see this being released. I need a robust affinity changer program since I tried a lot in the net and none works really good (delay to change core affinity or missed a new process).

In my home farm, there is 2 cases I need this :

1-Running 2 VM, each set to a pair (a pair need to have the priority lowered to feed the GPU) + 1 GPU client set to one core.
2- 2 VM set to 3 cores (both share all 3 cores) and 2 GPU client sharing the 4th core. This is where I'm often having issues with reliable affinity change.

[baobab]

Thanks for the comment Xilikon.
Your second system can be easily configured. You need just two profile lines in the affinityinput.txt configuration file.

Code: Select all

*\vmware-vmx.exe := CPU1+CPU2+CPU3
*\FahCore_11.exe := CPU0

The program does not change process priority right now. I did not think of that before but it makes sense to be able to change priority as well. That is a 10 minute change to the code.

I plan to release a new version in about 2 days to handle this, and also to release a 64-bit version because I think current version cannot handle 64-bit processes. Current version should still be able to manage affinity of other 32-bit process running in XP64 or Vista64 (nobody tested this and I do not have a 64 bit Windows). So I plan to do these two things, which are both fairly easy.

[Xilikon]

As soon as you implement priority changes, I will be your beta tester since I wanted yours to work perfectly after being disappointed with the other choices (I do have Vista 64, WHS and XP in my farm so I'm your perfect candidate)

[l8nights]

The program does not change process priority right now. I did not think of that before but it makes sense to be able to change priority as well. That is a 10 minute change to the code.

I plan to release a new version in about 2 days to handle this, and also to release a 64-bit version because I think current version cannot handle 64-bit processes. Current version should still be able to manage affinity of other 32-bit process running in XP64 or Vista64 (nobody tested this and I do not have a 64 bit Windows). So I plan to do these two things, which are both fairly easy.

sounds like it's going well

could I request a "cosmetic change" it's not of great signifigance but when I first set it up and didnt' get the "xxx set to use cpux" I thought the process was simply already set correctly, when really I had included the wrong path in the config. DOH!
then I noticed that it displays the same whether it doesn't find the processes, or it doesnt need the change.

Maybe a simple "xxx Process not found" dialogue would have helped a newb like me get set up faster LOL.

but over all good work

[baobab]

Xilikon I will take you on your offer

@l8nights:
You are the only one who posted that he's using the program, so thanks for the trust.

I can see why you would want that warning, but let me explain why it is this way and then maybe a solution if you still want it.

If you let the application run, it will keep monitoring your system for new processes, or adjustments to old processes, every 30 seconds.
I have more profiles in my configuration file than applications running at some point in time. The motivation for me was to use it to automatically set the affinity for new processes whenever I start them.

My desktop is not a dedicated folder. I have a VMware that is running basically 24/7 on PAIR1 and then a GPU client on one core of PAIR0. But I have also profiles for some games I play. When I start a game, I want the program to set their affinity to PAIR0, so they do not interfere with my VMWare. The GPU client is toast anyway because I need the GPU for the game.
So one can use the program for stuff like this, not expecting to find processes matching every profile all the time.

I could add such messages on a higher log level, let's say level 4. So you have to also set LogLevel (or only LogLevelConsole) to value 4 in the config file to get these messages. But let me know if you want these warnings printed only when the program checks processes for the very first time after it starts, or on every check (every 30 seconds). It is not hard to implement it either way.
Personally I think the first option would make more sense, otherwise you get a lot of messages of "No process found for profile xxx" if you have an extra profile in there.

[l8nights]

I have the log set to 9 dont' know what the max is, but personally to much info is better than not enough =)

I've looked over the source ( a little past my current skills tho hehe) and have got it up and running on two machines,works like a charm.

the only logical reason for displaying this would be for assurance that it has been set up correctly. the user will of course not suspect anything if a process isn't found from a program that's not running, but not finding one that is running would be an extra check.

start-up message should be enough for that purpose, and it's just cosmetics after you get to know the program.

but I think the priority settings are a higher "priority" than such a small change.