FCI: Folding@Home Client Info (Web based client monitor)

[jimerickson]

smoking2000:

i am getting the following when fci-1.6 updates xml files:

Code: Select all

Changing working directory to: /var/www/fci/

Reading XML files:
 xml-data/project-summary.xml
  read in: 2.76104 wallclock secs ( 2.58 usr +  0.04 sys =  2.62 CPU)
 xml-data/jmol-projects.xml
  read in: 2.75107 wallclock secs ( 2.60 usr +  0.00 sys =  2.60 CPU)
 xml-data/known-projects.xml
  read in: 0.00240397 wallclock secs ( 0.00 usr +  0.00 sys =  0.00 CPU)
 xml-data/client-list.xml
  read in: 0.00707006 wallclock secs ( 0.01 usr +  0.00 sys =  0.01 CPU)
 xml-data/username-list.xml
  read in: 0.00123692 wallclock secs ( 0.00 usr +  0.00 sys =  0.00 CPU)
 xml-data/team-list.xml
  read in: 0.000968933 wallclock secs ( 0.00 usr +  0.00 sys =  0.00 CPU)
XML files read in: 5.52444 wallclock secs ( 5.19 usr +  0.04 sys =  5.23 CPU)

Processing client: odysseus

Checking last update ...
Previous: 2010-01-30 18:10:02
 Current: 2010-01-30 18:10:02
     Now: 2010-01-30 18:36:42
The last upload is less than 1 hour old.
Client is considered: active
Client was first seen on: 2010-01-29 18:32:40

Files for this client:
 client-data/odysseus/qd-data.xml.new         (qd_data_new)
 client-data/odysseus/MyFolding.html.new      (myfolding_new)
 client-data/odysseus/FAHlog-Complete.txt.new (fahlog_complete_new)
 client-data/odysseus/FAHlog-Prev.txt.new     (fahlog_prev_new)
 client-data/odysseus/FAHlog.txt.new          (fahlog_new)
 client-data/odysseus/client.xml              (client_xml)
 client-data/odysseus/client.cfg.new          (client_cfg_new)
 client-data/odysseus/unitinfo.txt.new        (unitinfo_new)

File to parse: client-data/odysseus/qd-data.xml.new
Parsing qd-data.xml file: client-data/odysseus/qd-data.xml.new
Copying qd-data.xml.new to: client-data/odysseus/qd-data.xml

File to parse: client-data/odysseus/unitinfo.txt.new
Copying unitinfo.txt.new to: client-data/odysseus/unitinfo.txt

File to parse: client-data/odysseus/client.cfg.new
Parsing client.cfg file: client-data/odysseus/client.cfg.new
Copying client.cfg.new to: client-data/odysseus/client.cfg

File to parse: client-data/odysseus/FAHlog.txt.new
Parsing FAHlog.txt file: client-data/odysseus/FAHlog.txt.new
Copying FAHlog.txt.new to: client-data/odysseus/FAHlog.txt

File to parse: client-data/odysseus/FAHlog-Prev.txt.new
Copying FAHlog-Prev.txt.new to: client-data/odysseus/FAHlog-Prev.txt

File to parse: client-data/odysseus/FAHlog-Complete.txt.new
Parsing FAHlog-Complete.txt file: client-data/odysseus/FAHlog-Complete.txt.new
Copying FAHlog-Complete.txt.new to: client-data/odysseus/FAHlog-Complete.txt

File to parse: client-data/odysseus/MyFolding.html.new
Parsing MyFolding.html file: client-data/odysseus/MyFolding.html.new
Copying MyFolding.html.new to: client-data/odysseus/MyFolding.html

Updating Stanford user- & teampage URLs.

Saving updated client information to: client-data/odysseus/client.xml

Successfully updated the client XML file for: odysseus
Client processed in: 0.286732 wallclock secs ( 0.27 usr +  0.00 sys =  0.27 CPU)


Client processing completed in: 0.287007 wallclock secs ( 0.27 usr +  0.00 sys =  0.27 CPU)

Adding data from project summary to client projects list.
Completed in: 2.00272e-05 wallclock secs ( 0.00 usr +  0.00 sys =  0.00 CPU)

Adding data from project summary to known projects list.
Completed in: 0.009238 wallclock secs ( 0.01 usr +  0.00 sys =  0.01 CPU)

Parsing EXTREME Overclocking Folding@Home Stats data.
 Looking for XML file for user: jimerickson team: 45104 ... Found.
 File to parse: eoc-data/jimerickson-45104.xml
 Parsing EOC User Summary file: eoc-data/jimerickson-45104.xml

Completed in: 0.00290298 wallclock secs ( 0.00 usr +  0.00 sys =  0.00 CPU)

Adding kakaostats.com User Summary URLs.

Completed in: 0.000983953 wallclock secs ( 0.00 usr +  0.00 sys =  0.00 CPU)

Updating Stanford user- & teampage URLs.
Completed in: 2.69413e-05 wallclock secs ( 0.00 usr +  0.00 sys =  0.00 CPU)

Looking for stats file for team 45104 ... Found.
File to parse: stanford-data/team45104.txt
Parsing teamstats file: stanford-data/team45104.txt
Date::Calc::Decode_Month(): argument is not a string at /usr/local/bin/fci-update-xml-files.pl line 3337,  line 4.

what am i doing wrong here??

[smoking2000]

Good question, as you don't appear to be doing anything wrong at first glance.

I cannot reproduce the error on my FCI development server, there it parses the team45104.txt file correctly.

But looking at your FCI server, I see more issues indicating other problems.
Your odysseus client is still listed as a new client, which it only is on the first upload. The next time fci-update-xml-files.pl runs, it will lose this status.
The TPF data for odysseus is wrong too, so I think your upgrade of FCI did no go well.

Since you don't have much history in FCI yet, so I recommend to do a clean install of FCI v1.6. So first remove the existing install (assuming you used to default paths):

Code: Select all

rm -rf /usr/local/bin/fci-* /usr/local/bin/qd /usr/local/bin/rasmol /usr/local/bin/xyz2pdb /usr/local/etc/fci/ /usr/local/share/fci/ /var/www/fci/

And do the clean install (substituting your username in place of <user>):

Code: Select all

./install.pl --client --server --owner <user>

When you upgrade to a new version of FCI, use the install command you used previously and append --upgrade to overwrite files for upgrade (without --upgrade existing files will not be overwritten by the installer):

Code: Select all

./install.pl --client --server --owner <user> --upgrade

[smoking2000]

Yeah, I know I was a member twice.

Was? Why not any more? Seems like it would be an advantage for a developer like you to participate in a group like that?

If I remember correctly, the second time I got booted from the beta team was after I released an update of qd including the ability to display the client type. The mapping of the client types included two special client types that were considered secret, or at least not for public consumption. The fuss this caused with bruce and others, convinced me that there is no place in the beta team for someone with my kind of involvement.

I think I'm missing out on some crucial details, but I cannot look up what happened exactly back then since FCF died. But all the negative feelings these kind of encounters with the likes of you and bruce caused me to decrease my involvement in the project significantly, focusing only one the parts and people I care about. That's why I only announce qd updates on the Team Mac OS X forum, because that caused a fuss here (on FCF actually) too many times, and why there is still no new qd topic on this new forum. I don't want to restore the passionate involvement in the FAH project I had back then anymore, it caused me too much stress and misery. That's not what a hobby is supposed to be about.

Since the links to the other psummary pages were removed again, I don't think they are considered public by now. AFAIK they are still kept for beta team members only, for the same reasons as all those years ago, non-beta members will complain about beta WUs showing more credits than they are getting. If this situation has changed, please tell me so I can make the use of the other project summary pages public in FCI.

That is a falicy! PPD is based on how fast your computer folds, not on how many credits a work unit is worth, and the PSummary page does NOT list PPD, and DOES only list points for each WU. It doesn't matter if the WU credit is 100 points or 1,000 points. A 100 point WU takes 1 day to fold, the 1,000 point WU takes 10 days to fold. Both only score 100 PPD. Clearly that is not why.

That was the primary argument for not making the beta and comprehensive project summaries publicly available. People were afraid of the support nightmare this would cause with the influx of people asking why they are not getting the new super WU worth more than 1000 points (which was a lot back then), that were at that time new beta only projects.

If this argument has been dropped these days, it has been many years since, then why were the links to the other project summary pages removed again? Vijay asked me to only use project data sources that are publicly available in 3rd party applications, i.e. files linked from the project website (not the links to special files in the beta forum). I still think this is a silly requirement, and didn't not want to stop using the other project summary pages. As it was one of my new features for FCI 1.0, but I removed this to comply with Vijays request anyway as a sign of good faith. I got fed up with waiting on the promised changes, so I've reinstated this functionality but adapted the implementation to better fit the privilege model used for the beta team, in the hope that it won't cause a fuss again.

[bruce]

non-beta members will complain about beta WUs showing more credits than they are getting.

This is a persistent myth.

For the most part, the projects that are being processed by the beta team are exactly the same projects which are released to the general public a few weeks later, and the PPD is not changed just because a beta project is released to advmethods or an advmethods project is released to generic FAH.

There is one notable exception, and that has nothing to do with the projects themselves. When a new FahCore is nearing release, it and all WUs associated with it may be tested by any one of several methods, one of which involves the beta team. During that testing phase, the members so get a preview of the upcoming code. If the code runs smoothly, it is moved into public testing, along with the projects associated with it. Again, there is no change in PPD -- except for the fact that the new code may run more efficiently. If the code does NOT run smoothly, saying that they get more points than the general public is clearly incorrect because there are lots more cases where no points are awarded due to the errors encountered.

Is it correct to say that Linux gets more PPD than Windows? It depends on what you're comparing. For classic WUs, the PPD is identical. If you compare FahCore_a1 with FahCore_a1, the PPD is identical. If you compare FahCore_a2 (Linux) with FahCore_a1(Windows) then clearly Linux has an advantage. If you compare FahCore_11(Windows GPU) with no GPU support on Linux, the clearly Windows has an advantage. In both cases, the FAH community has responded with things like NotFred and a shell for WINE, and some degree of equity has been restored. Nevertheless, it depends on exactly what you're comparing and, for the most part, whether the cries of "unfair" are designed to provoke controversy or to provide accurate information.

Personally, I hate partisan politics. Public statements about "My Party" and "The Other Party" are invariably biased and are, in one way or another, propaganda. During times of war, governments often use propaganda, too, and we often ignore the inaccuracies if they come from our own government while attaking the statements if they're made by another government. Hopefully, some intelligent people on each side see through to the truth.

The statement that be beta team earns more points than the general public is also propaganda. and hopefully you're willing to see through to the truth.

The goal of the beta team is to attract people who are genuinely interested in testing new things ... finding and reporting errors ... and contributing to the development of reliable software and reliable WUs. The beta team is NOT interested in people who only goal is to maximize their points. Most people who join for the mythical increase in PPD drift away rather quickly after they find out that good testing is hard work and that membership does not provide a clear path to higher PPD.

[smoking2000]

non-beta members will complain about beta WUs showing more credits than they are getting.

This is a persistent myth.

Can I then go ahead and include the URLs to the other project summaries in my application and its documentation, thereby making the existence of these pages public knowledge?

Or are there other arguments for not wanting non-beta team members to see the beta projects listed on the alternate psummary pages, preventing the general use of these pages in 3rd party apps?

[jimerickson]

smoking2000:

after a clean install of fci-1.6 everything works except it does not generate any project images (i.e. static, rotating, and xyz molecule). how can i get it to do this?

[smoking2000]

Project images can only be generated if a current.xyz is uploaded for the project in question, but since the introduction of the GPU client many new cores stopped producing this file so projects cannot always be visualized anymore.

In FCI 1.6 the upload logic in FCI client was improved to stop uploading the current.xyz if it likely doesn't belong to the current WU, because it was modified too long ago (the last core to generate the file didn't clean it up). Because the cleanup of the current.xyz didn't work, the previous FCI releases generated an image for a project to which the current.xyz didn't belong.

[jimerickson]

yes you are right my current.xyz is old. thank you for the explanation smoking2000.

[7im]

non-beta members will complain about beta WUs showing more credits than they are getting.

This is a persistent myth.

Can I then go ahead and include the URLs to the other project summaries in my application and its documentation, thereby making the existence of these pages public knowledge?

Or are there other arguments for not wanting non-beta team members to see the beta projects listed on the alternate psummary pages, preventing the general use of these pages in 3rd party apps?

I would recommend against it, if for nothing more than to simply follow the recommendation of the project not to publish those links. The second answer was already mentioned, to avoid the clutter. No need to list work units that 99% of the contributors will never see until the WUs are released publicly. And the other 1% already know how to find them, if needed. And by not listing those links publicly, you can avoid having to answer questions about those links, and the forum can avoid having to answer those questions. And I assume we both have better things to do than cause more questions instead of answering more questions.

[smoking2000]

For the next release, I'm planning to include support for the new (BigAdv) bonus scheme, but for this I need testers with dual quad machines to test an experimental qd release that I'm preparing for that. I'm hoping that jimerickson, Magic-Michael, and possibly others are willing to help out with testing this. Please get in touch via PM if you want to help test the new qd with bigadv support.

As announced on the TeamMacOSX forum, I released functional revision 081 of qd today, with support for the new bonus scheme. This is the prerequisite for support in FCI as I wrote about before.

This qd release also adds support for the BigAdv client types (normal & beta), which will be shown in the (expanded) Current Work Unit and Queue sections of FCI client pages. Due to a bug in fci-update-xml-files.pl this is not properly shown in the Configuration section of client pages yet (but this is fixed in the development branch already).

If you don't run your FCI client with --update-qd, to have it automatically download the latest qd binary, it's recommended to do this once now. Or to copy the most recent qd binary you downloaded (or compiled) manually to ~/.fci/ where the FCI client configuration and updated qd & qdinfo.dat live.

I'd like to thank jimerickson of TeamUbuntu for his help testing this release on his dual quad box running BigAdv WUs.

[rhavern]

FWIW, the 1.6 upgrade went smoothly on all my clients and server. The new client status markers have already proved valuable in helping me troubleshoot an odd client issue. Again, many thanks for writing this and your help in the past.

[smoking2000]

I'm almost done integrating the new bonus scheme in FCI, but the question rises which way to include these into the PPD graph.

Which one do you prefer?

Single Y-axis


Double Y-axis


Please tell me your preference via the Doodle poll.

[smoking2000]

The results are in, thanks to all who have participated.

There is a clear preference for the single Y-axis:



So I've settled for the single Y-axis, but with some small enhancements:

[smoking2000]

New release of FCI: v1.6.1

New Feature: Support for the new Bonus Points

Since qd is able to calculate the bonus points from functional revision 081, FCI can use this data as well. The Bonus Points, Bonus PPD and Bonus Factor are displayed in additional columns next to the normal Points and PPD. As seen in my earlier posts, the Bonus PPD is also included in the PPD Graphs. RRDs for these new values are also added to the client queue history.

For the bonus points data to be available, you'll have to update qd on your FCI clients to at least fr 081. qd fr082 introduces an alternate PPD calculation which doesn't account for the progress, but asssumes the WU is completed at the expected date. This data is preferred for PPD values used in FCI, if the alternate PPD values are not availble, it will fallback to the points per hour and multiply by 24 as was done before.

So you'll want to run all your FCI clients at least once with --update-qd if you don't always do so already, to have the FCI client download the latest qd from your FCI server.

New Dependency: Math::Round

With the work to add support for the Bonus Points to the PPD Graphs also came the opportunity to set smarter maximum values for the graphs. I chose to use nhimult() from Math::Round for this. Most OSes on which FCI Server is supported provide a package for Math::Round, although some require some more work (Gentoo: g-cpan & OpenSUSE: OpenSUSE Build Service). The installation instructions have been updated to include instructions for Math::Round.

The Great Renaming: Credits -> Points

As Usenet saw the Great Renaming back in 1987 to provide a better naming scheme, so sees FCI its own Great Renaming today.

Due to popular demand, the column formally known as Credit has been renamed to Points to be more in line with the naming used by most users .

This renaming includes the keys used in the XML files generated by FCI. The exception are the XML files populated with data from the psummary and team/user stats files, whose upstream naming is adopted.

New Feature: Extended Team Information

Similar to the extinfo used in Nagios, this version of FCI ships with support for additional team information. For now this are only the team website URL, and URL to its logo. These will be shown on the team page in FCI next to the team information, smiliar to how they're shown atop of the team stats page at Stanford.

I've provided a pre-populated team-extinfo.xml with the team info of a handful of teams. A management interface is available in the Settings web interface, with which you can add more.

Improved Functionality: Show 'Passkey: Yes' in client configuration section

When a clients uses a passkey, this now also displayed on its client page in the configuration section as 'Passkey: Yes', similar to the 'Passkey found' line logged in FAHlog.txt.

Bugfix: Client Type parsing from client.cfg

fci-update-xml-files.pl will no longer overwrite the client type when parsing the client.cfg if the client type was already parsed from the qd data. This caused an incorrect type (0 No Preference) to be reported for clients which do have a client type configured (Advmethods, BigAdv, etc), but only stored in the queue.dat not also in the client.cfg (e.g. because it was set using a commandline argument).

Bugfix: Argument parsing from FAHlog.txt

fci-update-xml-files.pl will forget about the arguments it parsed from a previous FAH client startup, to properly reflect the current situation. It now also properly merges arguments from FAHlog with those in the client.cfg.

Bugfix: Don't show disfunctional TPF graph in IE

As expected, the jQuery Flot TPF graphs are not support in Internet Explorer. So these will no longer be shown for users of IE. Advanced features, require advanced browers such as Firefox, Safari, Chrome, or Opera.

Flot v0.6 possibly improves support for IE browers, but I haven't tested this version yet.

Minor Changes

clients.plc Align filesizes to the right in the Client Files list on detailed client pages.


The lastest FCI release (and previous releases) can be downloaded from the project website, or you can use the direct link.

[jimerickson]

time per frame is no longer being calculated as of today. yesterday it was. i notice qd updated today. has that got something to do with it?