PeddlerOfFlesh wrote:It appears that known-projects.xml is empty. Clients have made uploads, though. It shows them as unprocessed in settings -> clients.
Try running one or more of your FCI clients with --verbose and check if in the
Composing list of files to upload: section what the status for the
work/current.xyz is. This file is used by the FCI server in the final stage of the upload procedure to generate an image using rasmol for the project in question. This project is then added to the known-projects.xml file.
Also check the
#####[ Server Response ]##### section, which normally includes something like this near the end:
Code: Select all
Saved current.xyz file to disc.
Saving uploaded client information in client.xml.
Saved client information to: client-data/osiris/client.xml.new
Checking if project of xyz file is seen before.
Extracted from xyz file:
Work Unit : p3064_lambda5_2003
Project : 3064
Project already known.
Or in the case of a project not yet listed in the known-projects.xml:
Code: Select all
Checking if project of xyz file is seen before.
Extracted from xyz file:
Work Unit : p3064_lambda5_2003
Project : 3064
Project not seen before.
Copying xyz file to project data directory.
client-data/osiris/current.xyz -> project-data/p3064.xyz
Adding project to list of known projects.
Project added at 2009-04-03 19:55:38.
Successfully converted .xyz to .pdb for project 3064
Successfully generated image of type still for project 3064
This output will be missing if you're uploading data of a GPU client.
Recently (since
FCI v1.4, 2009-03-21) projects being worked on by clients that are missing a current.xyz in the upload are also added to the known-projects.xml, to have these projects also show up in the project list under the Projects menu option in the main web interface. This change was made because the GPU clients don't produce the current.xyz anymore, and whose projects would therefor never be preserved in the known-projects list.
I tried to reproduce your situation, but I cannot trigger the warnings you reported using the following procedure:
* Clean install of the latest FCI client & server (
v1.4.2)
* Upload Linux SMP client data to FCI server by FCI client
* Executing the FCI server cronjobs listed in
documentation/INSTALL.Debian in that order, but twice:
** fci-update-stanford-files.pl
** fci-update-qd-files.pl
** fci-update-fahstats.pl
** fci-jmol-missing-projects.pl
** fci-update-xml-files.pl
** fci-generate-queue-graphs.pl
** fci-update-jmol-projects.pl
** fci-update-eoc-stats.pl
** fci-update-project-images.pl
When you do a clean install of an FCI server, most of the data still needs to be collected.
fci-update-stanford-files.pl needs to be run first to download the data files from Stanford so the FCI server knows about the various projects, and the user & team stats. The FCI server only has empty files serving as placeholders for the actual data from Stanford. The first time this script is download it will not also download the teamstats files because fci-update-xml-files.pl has not run yet, which updates the team-list.xml file listing the teams known on the FCI server. This also means that the first time fci-update-xml-files.pl is run, it will not be able to fetch the stats for a username from the much smaller teamstats text file, and will have to get it by parsing the much bigger daily_user_summary.txt. Because the performance overhead of parsing the daily_user_summary.txt is quite big, this is only done if there are no teamstats files for the usernames known on the FCI server.
fci-update-qd-files.pl is not as immediately required, but I don't always include the latest qdinfo.dat and qd binaries with FCI, so you'll want to download the latest and greatest, so your FCI clients will download the most recent qdinfo.dat containing the most recent projects. You want this to avoid missing PPD info for your clients which is provided by qd.
fci-update-fahstats.pl is not useful at this point yet, but running it anyway is harmless. It will download the UserID used in the Kakaostats (formerly FAHstats) for the usernames listed in the username-list.xml (this XML file is updated by fci-update-xml-files.pl (the FCI server hearth) with the usernames listed in the FAH client configuration uploaded by the FCI clients). But because the client data has not been processed by fci-update-xml-files.pl yet, the script won't have any usernames to download the UserID for. It will in the next run.
fci-jmol-missing-projects.pl needs to be run too, to download the latest copy of the Jmol missingProjects.txt file and to check the FCI known-projects.xml file for projects listed in the Jmol missingProjects.txt. If the script finds Jmol missing projects in the known-projects.xml file, and these projects have and xyz file. The xyz file will be send via email to NicoV so he can include it in
http://jmol.sourceforge.net/fah/. This script uses the Email Settings you can configured via the Jmol Missing Projects in the FCI Settings web interface. You'll want to change the From address to your email address so NicoV can credit you properly, and to contact you in case of problems.
fci-update-xml-files.pl is the hearth of the FCI server which combines the various data sources and updates the xml-data files used by the FCI Main web interface. It processes the uploaded client-data, the stanford data, and more (see the
INSTALL file). As mentioned before, some scripts can only do something useful after fci-update-xml-files.pl has processed all the data and updated the XML files whose content they require.
fci-generate-queue-graphs.pl is also only useful after fci-update-xml-files.pl has run at least once, because it uses the client-list.xml composed by fci-update-xml-files.pl listing the clients who data has been processed.
fci-update-jmol-projects.pl is needed to download the Jmol fah-projects.xml which is used to update the FCI project-summary.xml with projects not listed on the Stanford psummary.html. This keeps the project data available in FCI useful for approved betateam members too, or those that run with -advmethods whose projects tend to be listed only on the alternate psummary pages every now an then.
fci-update-eoc-stats.pl is something you really want to run, especially in the next release with the inclusion of RRD graphs using the EOC 24hr Average. This script also depends on fci-update-xml-files.pl to have done its job, because like fci-update-fahstats.pl it downloads the EOC stats for all the users listed in the usernames.xml updated by fci-update-xml-files.pl.
fci-update-project-images.pl does not require fci-update-xml-files.pl to have done its job, it only generates the missing image types for all the projects listed in the known-projects.xml that have an xyz file. During the FCI client upload only the still image type is generated because it's not very demanding to generate a single image using rasmol. Generating the animated GIFs for the rotate & xyz180 image types is bit more demanding because all the frames need to be generated, and compiled into the actual animation. This is best done at a quiet time, not when all the FCI clients are uploading (most probably all at the same time or nearly).
Because of the dependencies between the various FCI server scripts, I highly recommend that you do not enable the FCI client nor the FCI server cronjobs before you've run all the commands in question (both client and server-side) at least once manually. When you've verified that everything works correctly, you can enable the cronjobs. This way you are sure that scripts do not run before their dependencies, because you hardly ever install an FCI server at xx:59:59 so that all the scripts run in the correct sequence automatically. You'll need to run the sequence manually at least once, but preferably twice, to get all the data in their most complete form. When this is done, you can let the schedulers on the FCI clients and server handle all the subsequent runs of the scripts.
PeddlerOfFlesh wrote:Now I noticed that running the update also causes errors.
Code: Select all
Constant subroutine main::NULL redefined at D:/Programs/Misc/PERL/lib/Exporter.pm line 66.
at C:\Program Files\FCI\fci-client.pl line 98
This is a harmless warning in a module used by FCI client on Windows only, caused by the diverted use of the Win32 Perl module (to avoid requiring this module be installed on non-Windows OSes):
Code: Select all
78 require Win32;
79 import Win32;
PeddlerOfFlesh wrote:Edit: I went through and checked for errors with all the things in crontab and this is what I found:
Code: Select all
user@computer:/var/www/fci$ /usr/local/bin/fci-update-stanford-files.pl --dir /var/www/fci/
Warning: Cannot parse project line (252):
<snipped a bunch of HTML>
These warnings are shown because the psummary.html downloaded from the Stanford servers contains invalid project lines. Projects listed on the psummary are considered invalid if they don't have data in all its colums.
This has been the case for p5732-5747 for quite some time now, the script generating the psummary pages at stanford contains a bug whereby its includes both the valid and invalid project lines:
Code: Select all
5732 171.64.65.102 p5732_ACBP_ff03_300K 1392 2.00 3.00 511.00 100 GROGPU2 Description vvoelz
5732 171.64.65.102 p5732_ACBP_ff03_300K 1392 2.00 3.00 511.00 TINKER Description vvoelz
5733 171.64.65.102 p5733_ACBP_ff03_330K 1392 2.00 3.00 511.00 100 GROGPU2 Description vvoelz
5733 171.64.65.102 p5733_ACBP_ff03_330K 1392 2.00 3.00 511.00 TINKER Description vvoelz
5733 171.64.65.102 p5733_ACBP_ff03_330K 1392 2.00 3.00 511.00 100 GROGPU2 Description vvoelz
5734 171.64.65.102 p5734_ACBP_ff03_370K 1392 2.00 3.00 511.00 100 GROGPU2 Description vvoelz
5734 171.64.65.102 p5734_ACBP_ff03_370K 1392 2.00 3.00 511.00 TINKER Description vvoelz
5735 171.64.65.102 p5735_ACBP_ff03_450K 1392 2.00 3.00 511.00 100 GROGPU2 Description vvoelz
5735 171.64.65.102 p5735_ACBP_ff03_450K 1392 2.00 3.00 511.00 TINKER Description vvoelz
5735 171.64.65.102 p5735_ACBP_ff03_450K 1392 2.00 3.00 511.00 100 GROGPU2 Description vvoelz
5736 171.64.65.102 p5736_ACBP_ff96_300K 1392 2.00 3.00 511.00 100 GROGPU2 Description vvoelz
5736 171.64.65.102 p5736_ACBP_ff96_300K 1392 2.00 3.00 511.00 TINKER Description vvoelz
5737 171.64.65.102 p5737_ACBP_ff96_330K 1392 2.00 3.00 511.00 100 GROGPU2 Description vvoelz
5737 171.64.65.102 p5737_ACBP_ff96_330K 1392 2.00 3.00 511.00 TINKER Description vvoelz
5737 171.64.65.102 p5737_ACBP_ff96_330K 1392 2.00 3.00 511.00 100 GROGPU2 Description vvoelz
5738 171.64.65.102 p5738_ACBP_ff96_370K 1392 2.00 3.00 511.00 100 GROGPU2 Description vvoelz
5738 171.64.65.102 p5738_ACBP_ff96_370K 1392 2.00 3.00 511.00 TINKER Description vvoelz
5739 171.64.65.102 p5739_ACBP_ff96_450K 1392 2.00 3.00 511.00 100 GROGPU2 Description vvoelz
5739 171.64.65.102 p5739_ACBP_ff96_450K 1392 2.00 3.00 511.00 TINKER Description vvoelz
5739 171.64.65.102 p5739_ACBP_ff96_450K 1392 2.00 3.00 511.00 100 GROGPU2 Description vvoelz
5740 171.64.65.102 p5740_proL_ff03_300K 973 2.00 3.00 384.00 100 GROGPU2 Description vvoelz
5740 171.64.65.102 p5740_proL_ff03_300K 973 2.00 3.00 384.00 TINKER Description vvoelz
5741 171.64.65.102 p5741_proL_ff03_330K 973 2.00 3.00 384.00 100 GROGPU2 Description vvoelz
5741 171.64.65.102 p5741_proL_ff03_330K 973 2.00 3.00 384.00 TINKER Description vvoelz
5741 171.64.65.102 p5741_proL_ff03_330K 973 2.00 3.00 384.00 100 GROGPU2 Description vvoelz
5742 171.64.65.102 p5742_proL_ff03_370K 973 2.00 3.00 384.00 100 GROGPU2 Description vvoelz
5742 171.64.65.102 p5742_proL_ff03_370K 973 2.00 3.00 384.00 TINKER Description vvoelz
5743 171.64.65.102 p5743_proL_ff03_450K 973 2.00 3.00 384.00 100 GROGPU2 Description vvoelz
5743 171.64.65.102 p5743_proL_ff03_450K 973 2.00 3.00 384.00 TINKER Description vvoelz
5743 171.64.65.102 p5743_proL_ff03_450K 973 2.00 3.00 384.00 100 GROGPU2 Description vvoelz
5744 171.64.65.102 p5744_proL_ff96_300K 973 2.00 3.00 384.00 100 GROGPU2 Description vvoelz
5744 171.64.65.102 p5744_proL_ff96_300K 973 2.00 3.00 384.00 TINKER Description vvoelz
5745 171.64.65.102 p5745_proL_ff96_330K 973 2.00 3.00 384.00 100 GROGPU2 Description vvoelz
5745 171.64.65.102 p5745_proL_ff96_330K 973 2.00 3.00 384.00 TINKER Description vvoelz
5745 171.64.65.102 p5745_proL_ff96_330K 973 2.00 3.00 384.00 100 GROGPU2 Description vvoelz
5746 171.64.65.102 p5746_proL_ff96_370K 973 2.00 3.00 384.00 100 GROGPU2 Description vvoelz
5746 171.64.65.102 p5746_proL_ff96_370K 973 2.00 3.00 384.00 TINKER Description vvoelz
5747 171.64.65.102 p5747_proL_ff96_450K 973 2.00 3.00 384.00 100 GROGPU2 Description vvoelz
5747 171.64.65.102 p5747_proL_ff96_450K 973 2.00 3.00 384.00 TINKER Description vvoelz
5747 171.64.65.102 p5747_proL_ff96_450K 973 2.00 3.00 384.00 100 GROGPU2 Description vvoelz
This warning is unrelated to the one reported by fci-update-xml-files due to an empty known-projects.xml file.
PeddlerOfFlesh wrote:Code: Select all
user@computer:/var/www/fci$ /usr/local/bin/fci-jmol-missing-projects.pl --dir /var/www/fci/
Warning: <project> element has no 'number' key attribute at /usr/local/bin/fci-jmol-missing-projects.pl line 488
Not a HASH reference at /usr/local/bin/fci-jmol-missing-projects.pl line 258.
user@computer:/var/www/fci$ /usr/local/bin/fci-update-project-images.pl --dir /var/www/fci/
Warning: <project> element has no 'number' key attribute at /usr/local/bin/fci-update-project-images.pl line 548
Not a HASH reference at /usr/local/bin/fci-update-project-images.pl line 174.
These warnings are also due to the empty known-projects.xml file, like you encounted before while running fci-update-xml-files.pl.