HFM.NET - Monitoring Application for Folding@Home v7

[bruce]

The FAH client and server code were built on Cosm as far back as I remember. It was an essential toolkit before the V3 client -- and probably the V1 client -- but I wasn't around back then.

Dick developed several important third-party tools, including qd, in the V3 time period and updated it as FAH developed (after I began folding). Smoking2000 has maintained Dick's tools since he passed away.

[MtM]

Dick deserves the grattitude of each and everyone who folds, not only those using tools he developed or where developed on his blueprints but because of how he did it and why and what effect it had on the whole of the community. Though I think his example might need continuation and that in a way makes me feel a bit more sad.

[harlam357]

2) The CPU Type is what is recorded by Stanford in the queue.dat file. It is not read directly by HFM through any other means. That particular piece of the UI is the Queue Viewer and it displays the data contained within the queue.dat file. I'm sure your Core 2 or Core i5/7 cpu is reporting as a Pentium II/III. In all fairness to Stanford, the "Core" generation of cpus are descendants of the PII/III generations. A lot of the same "markers" identifying the PII/III are still in place in the "Core" generation. So to Stanford that's what they look like.

Again, the nomenclature here was something I took from the maintainer of qd.c - http://linuxminded.xs4all.nl/?target=so ... -tools.plc - a gentleman here by the name of smoking2000. I agree that data may be better reported with a different name, like "Total Cores" or "Available Cores". I assume you're running with the -smp 7 flag?

That nomenclature is not from Smoking2000, it's from Cosm.

Cosm is used by f@h, Dick Howell disected queue.dat for the most important parts and found the references to Cosm API. It's not the other way around, Dick/Smoking2000 did not come up with Cosm.

The nomenclature I'm referring to is, "SMP Cores" and "Cores To Use", which I'm sure I got from the qd output and I'm sure SMP wasn't around when Dick was alive. Thus, I'll blame smoking2000.

If the reasoning behind a Core series CPU being detected as a Pentium II/III is Cosm, then so be it. Doesn't change the fact that Stanford hasn't done anything to change the detection algorithms.

[k1wi]

One quick question while you're around...

Is it possible to get HFM to parse the Core #? By that I mean differentiate between A2, A3 etc. I know it names the core eg GROCVS, but it would be handy if it could also list the Core number.

Otherwise I'm really enjoying the app and have learned a few things about xslt (Namely that you shouldn't edit the active template file while HFM.net is running!). Nowadays when I edit the template I make a copy of my running .xslt (with the original files stored separately) and then switch over to it afterwards...

[Russ_64]

A3's are already shown as GRO-A3 in HFM.net

[smoking2000]

The nomenclature I'm referring to is, "SMP Cores" and "Cores To Use", which I'm sure I got from the qd output and I'm sure SMP wasn't around when Dick was alive. Thus, I'll blame smoking2000.

You are correct, I am to blame for all (mis)naming of output fields since qd fr 033 and up. The cores to use is probably not named correctly, but I haven't been able to figure out a better name since I don't understand why the detected number of cores is now stored twice in the queue.dat where it was stored only once in the early days of the SMP client.

The GPU memory reported by qd is likely not correct either (it's more likely the GPU bit rate), but Dick Howells disclaimer still applies: "Also, this program is not in any official way connected to the Stanford code, so if it calls data items the wrong thing, it is purely an error of my own interpretation."

If the reasoning behind a Core series CPU being detected as a Pentium II/III is Cosm, then so be it. Doesn't change the fact that Stanford hasn't done anything to change the detection algorithms.

Correct again. The Cosm library hasn't seen any (public) update since almost forever. So its built-in CPU and OS tables haven't been updated either. There have been some updates behind the scenes since I had to add Win2K3 to the OS table in qd (which mimics the Cosm table) back in 2005 (and generic Windows in 2006), and AMD64 to the CPU table in 2007.

[harlam357]

One quick question while you're around...

Is it possible to get HFM to parse the Core #? By that I mean differentiate between A2, A3 etc. I know it names the core eg GROCVS, but it would be handy if it could also list the Core number.

Otherwise I'm really enjoying the app and have learned a few things about xslt (Namely that you shouldn't edit the active template file while HFM.net is running!). Nowadays when I edit the template I make a copy of my running .xslt (with the original files stored separately) and then switch over to it afterwards...

The Core #, or really ID - it's a hex value, is available in the queue.dat file. There really isn't a good place in the queue viewer to place that information. I guess it could be appended to the Core string as seen in the main grid. I'll think about it. In some cases this information would be redundant. For example, GRO-A3 (A3).

Thanks very much! What happened when you edited the active xslt template file? That's something I've never tried before.

The nomenclature I'm referring to is, "SMP Cores" and "Cores To Use", which I'm sure I got from the qd output and I'm sure SMP wasn't around when Dick was alive. Thus, I'll blame smoking2000.

You are correct, I am to blame for all (mis)naming of output fields since qd fr 033 and up. The cores to use is probably not named correctly, but I haven't been able to figure out a better name since I don't understand why the detected number of cores is now stored twice in the queue.dat where it was stored only once in the early days of the SMP client.

The GPU memory reported by qd is likely not correct either (it's more likely the GPU bit rate), but Dick Howells disclaimer still applies: "Also, this program is not in any official way connected to the Stanford code, so if it calls data items the wrong thing, it is purely an error of my own interpretation."

If the reasoning behind a Core series CPU being detected as a Pentium II/III is Cosm, then so be it. Doesn't change the fact that Stanford hasn't done anything to change the detection algorithms.

Correct again. The Cosm library hasn't seen any (public) update since almost forever. So its built-in CPU and OS tables haven't been updated either. There have been some updates behind the scenes since I had to add Win2K3 to the OS table in qd (which mimics the Cosm table) back in 2005 (and generic Windows in 2006), and AMD64 to the CPU table in 2007.

Maybe those values get renamed to, "Cores In Use" and "Cores Available"... or something in that regard. So one of the numbers is the actual detected number of cores (based on the hardware) and the other is the requested number of threads (A3) or core processes (A1/A2). The latter will always be at least 4, but with i7s w/HT being very prevalent now, many folks are running with -smp 7, for example, which will cause those two values to differ.

Ah, I always wondered why the "GPU Memory" value seemed off. I knew it wasn't a representation of the memory available to the GPU because the values always seem to be the same regardless of which card I'm running. Nor did it seem likely that it is the requested amount of memory... so bitrate seems like a much better interpretation of the value. It just makes more sense.

And your CPU and OS tables are where I got my values as well. I did my best to completely clone *how* the qd code interprets the values. But obviously it's implemented quite differently. I *could* possibly even create a .NET equivalent of qd that uses my HFM.Queue.dll to read the queue.dat contents and display it in the same manner as qd. That may be helpful in my personal debugging of my implementation vs. qd... however, we already have qd thanks to you and it would seem redundant to release such a tool into the wild.

[PantherX]

I always thought that the GPU Memory was the RAM on my system (I could be wrong) which I know is inaccurate but as the Client is still in BETA Stage, I assumed that it would be fixed along the way. Right now, my 4 GB RAM (4091 MB) is being displayed in both the GPU Client and the SMP2 Client.

[smoking2000]

Maybe those values get renamed to, "Cores In Use" and "Cores Available"... or something in that regard. So one of the numbers is the actual detected number of cores (based on the hardware) and the other is the requested number of threads (A3) or core processes (A1/A2). The latter will always be at least 4, but with i7s w/HT being very prevalent now, many folks are running with -smp 7, for example, which will cause those two values to differ.

In all my queue.dat testcases I have never seen a case where SMP Cores != Cores To Use, so I'm very interested in the queue.dat and FAHlog of the cases where these values differ.

The second SMP Cores values started to appear in the queue.dat at the time when the -smp N parameter was introduced, it's seemed obvious that this value in the queue.dat was the N parameter. But this so far has not been so. I received some testcases from Jason of EOC Stats fame from the dual quad server he bought from the donations, we tested -smp, -smp 4 and -smp 8, and in all cases both values in the queue.dat were 8 and no the expected 8/8, 8/4 & 8/8.

Ah, I always wondered why the "GPU Memory" value seemed off. I knew it wasn't a representation of the memory available to the GPU because the values always seem to be the same regardless of which card I'm running. Nor did it seem likely that it is the requested amount of memory... so bitrate seems like a much better interpretation of the value. It just makes more sense.

Even if treated as bitrate the values are a little off, in all my GPU testcases I have only two different values: 258 & 513. These are not the expected 256 & 512, but are off 2 & 1 respectively. Maybe that difference has some meaning, or I need to interpret the value differently, I don't know.

I always thought that the GPU Memory was the RAM on my system (I could be wrong) which I know is inaccurate but as the Client is still in BETA Stage, I assumed that it would be fixed along the way. Right now, my 4 GB RAM (4091 MB) is being displayed in both the GPU Client and the SMP2 Client.

The amount of RAM detected by the FAH client, or configured in client.cfg is what's reported as Memory, both the GPU & SMP client detect this value, but only the GPU clients also have the GPU Memory value stored in the queue.dat.

[bruce]

The Core #, or really ID - it's a hex value, is available in the queue.dat file. There really isn't a good place in the queue viewer to place that information. I guess it could be appended to the Core string as seen in the main grid. I'll think about it. In some cases this information would be redundant. For example, GRO-A3 (A3).

Personally, I like k1wi's suggestion.

My personal recommendation would be to replace the core name with the core number. I spend a lot of time with FAH, so I know that GROCVS means the same thing as A2 (which is less obvious than A3 being equivalent to GRO-A3. My problem is that the I often look at the output on a narrow screen so I'm continueally looking for ways to shrink the width of the display, and 79 takes up a lot less column width than DGROMACS.

(or, just put both fields there and we can turn off whichever columns we don't want.)

Feel free to disagree and outvote me. It's just a suggestion and I recognize that this may not be what others want. (This is probably as good a place to ask for feedback as any.)

[7im]

I like having both options. Core #s for the pro's, and/or core names for the newbs.

[harlam357]

Maybe those values get renamed to, "Cores In Use" and "Cores Available"... or something in that regard. So one of the numbers is the actual detected number of cores (based on the hardware) and the other is the requested number of threads (A3) or core processes (A1/A2). The latter will always be at least 4, but with i7s w/HT being very prevalent now, many folks are running with -smp 7, for example, which will cause those two values to differ.

In all my queue.dat testcases I have never seen a case where SMP Cores != Cores To Use, so I'm very interested in the queue.dat and FAHlog of the cases where these values differ.

The second SMP Cores values started to appear in the queue.dat at the time when the -smp N parameter was introduced, it's seemed obvious that this value in the queue.dat was the N parameter. But this so far has not been so. I received some testcases from Jason of EOC Stats fame from the dual quad server he bought from the donations, we tested -smp, -smp 4 and -smp 8, and in all cases both values in the queue.dat were 8 and no the expected 8/8, 8/4 & 8/8.

Well... I personally haven't tried it either. But I'm just going by what PantherX said that kicked off this discussion.

I do have a suggestion for the SMP Client as in the Queue, it states "SMP Cores" (it reports correctly 7) and then it states "Cores To Use" (reports 8) and it would make more sense to rename the latter to "Total Cores"

The Core #, or really ID - it's a hex value, is available in the queue.dat file. There really isn't a good place in the queue viewer to place that information. I guess it could be appended to the Core string as seen in the main grid. I'll think about it. In some cases this information would be redundant. For example, GRO-A3 (A3).

Personally, I like k1wi's suggestion.

My personal recommendation would be to replace the core name with the core number. I spend a lot of time with FAH, so I know that GROCVS means the same thing as A2 (which is less obvious than A3 being equivalent to GRO-A3. My problem is that the I often look at the output on a narrow screen so I'm continueally looking for ways to shrink the width of the display, and 79 takes up a lot less column width than DGROMACS.

(or, just put both fields there and we can turn off whichever columns we don't want.)

Feel free to disagree and outvote me. It's just a suggestion and I recognize that this may not be what others want. (This is probably as good a place to ask for feedback as any.)

Ok... I like the idea to put both fields on the grid and this kinda works out good... here's the scoop.

I plan to combine the Core and Core Version columns. The Client Type column will also be getting blessed with the client version (been capturing it, just not displaying it). So the Core Version column will become the Core ID (or Code) column. Then, as you say, one can turn it on or off depending on preference.

How do y'all feel about that?

[k1wi]

Sounds great - I look forward to updating

I think the problem I had with editing the active XSLT template was that it had all sorts of issues with saving the changes. For background sake- I SSH into the templates (stored on a network shared linux folder so I can SSH into it and edit) and edit using nano. A couple of times when I attempted to save them it actually froze up my SSH session, and then when the app updated the webpage it would come up blank.

At first I thought it was just because my code had been wrong, but then I made a fresh copy of the original template, made my edits (basically just added my EOC badge to the bottom), then save as a new file and linked the app to the new file things worked fine and dandy. I'm not a coder, but my impression was that the app was keeping the template open/active the whole time? I.e. Between updates.

[harlam357]

Not sure about the XSLT file locking, but I'll double check it. Thanks for the report, I'll post back if that is indeed the case.

[PantherX]

Today I was babysitting the uploading of my 6041 WU which was 54 MB and took roughly 1.25 hours. During this time, I was getting some ideas about some "features" - if you may call it- that would be interesting to have in the future versions of the Client: (programming is easier said than done so I do respect all your handwork and dedication that you have put in to making such amazing software that too for FREE )

1) WU History: This would be an interesting feature where the Client records all the successfully completed WU done by the Client. It could be grouped according to the Projects done. Something that was initially available in the Official Stats but was later removed. The idea is that F@H users complete unique WUs so there isn't any duplication on general basis unless you miss the Preferred Deadline or got errors. Now I am expecting this to be low on the list of your to-do as this doesn't really have any actual usage but would be interesting.

2) The feature of showing the FAHLog for the selected WU is an interesting one but it only works for FAHLog.txt. It would be interesting if it could also read the FAHLog-Prev.txt Also the color coding is nice but not uniform, maybe in later versions, this would be fixed completely (looking forward to that). Also I would like to add when the WU is over, the Fahlog should be displayed till "[14:46:53] + Number of Units Completed: X" as it indicates a successful completion rather than "[14:49:46] + Closed connections" as this is technically the next WU (I don't mean to be rude or anything rather telling what I think would be more appropriate). The rest of the Log should be displayed for the next WU. Also when I fail to receive WU from the Server, and restart the Client, HFM.NET displays "No Log Available". While it is true for the selected WU, (I got scared the first time I saw it and took me some time to figure out) I would suggest that the WU Info be automatically blank when the displaying the FAHLog. I hope this is on you to-do list once the Client is out of BETA Stage.

Well these are my observations and I decided to share them. Feel free to comment.