Project 5771 slowing down overnight [by factor of 4]

Moderators: Site Moderators, FAHC Science Team

Post Reply
new08
Posts: 188
Joined: Fri Jan 04, 2008 11:02 pm
Hardware configuration: Hewlett-Packard 1494 Win10 Build 1836
GeForce [MSI] GTX 950
Runs F@H Ver7.6.21
[As of Jan 2021]
Location: England

Project 5771 slowing down overnight [by factor of 4]

Post by new08 »

My system usually runs steady on 353 point units, more than any other. GPU nV gt 240
Last night after loading a new 5771 unit and running OK [just over 1 min /step] I left m/c for overnight run [something I don't do all the time in the summer].
This one took over 8 hrs, instead of less than 2 hrs, to complete [with nothing else running of note] @the 36% mark extending step times, notably.

The next 353 unit is now running ok -as a follow on.
PC is overclocked but runs fine usually, 450 units sometimes giving eue -but not lately.
I suppose I'll be lucky to get comment on this, other than it's an o/c problem, but I don't think it is...

Code: Select all

[23:00:21] *------------------------------*
[23:00:21] Folding@Home GPU Core
[23:00:21] Version 1.31 (Tue Sep 15 10:57:42 PDT 2009)
[23:00:21] 
[23:00:21] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[23:00:21] Build host: amoeba
[23:00:21] Board Type: Nvidia
[23:00:21] Core      : 
[23:00:21] Preparing to commence simulation
[23:00:21] - Looking at optimizations...
[23:00:21] DeleteFrameFiles: successfully deleted file=work/wudata_04.ckp
[23:00:21] - Created dyn
[23:00:21] - Files status OK
[23:00:21] - Expanded 45400 -> 251112 (decompressed 553.1 percent)
[23:00:21] Called DecompressByteArray: compressed_data_size=45400 data_size=251112, decompressed_data_size=251112 diff=0
[23:00:21] - Digital signature verified
[23:00:22] 
[23:00:22] Project: 5771 (Run 4, Clone 191, Gen 1922)
[23:00:22] 
[23:00:22] Assembly optimizations on if available.
[23:00:22] Entering M.D.
[23:00:28] Tpr hash work/wudata_04.tpr:  586361484 216708831 28206916 2059230783 331972735
[23:00:28] 
[23:00:28] Calling fah_main args: 14 usage=100
[23:00:28] 
[23:00:28] Working on Protein
[23:00:29] Client config found, loading data.
[23:00:30] Starting GUI Server
[23:01:30] Completed 1%
[23:02:30] Completed 2%
[23:03:31] Completed 3%
[23:04:32] Completed 4%
[23:05:32] Completed 5%
[23:06:33] Completed 6%
[23:07:34] Completed 7%
[23:08:34] Completed 8%
[23:09:35] Completed 9%
[23:10:35] Completed 10%
[23:11:35] Completed 11%
[23:12:36] Completed 12%
[23:13:37] Completed 13%
[23:14:37] Completed 14%
[23:15:38] Completed 15%
[23:16:39] Completed 16%
[23:17:40] Completed 17%
[23:18:40] Completed 18%
[23:19:40] Completed 19%
[23:20:41] Completed 20%
[23:21:42] Completed 21%
[23:22:42] Completed 22%
[23:23:42] Completed 23%
[23:24:43] Completed 24%
[23:25:43] Completed 25%
[23:26:44] Completed 26%
[23:27:44] Completed 27%
[23:28:45] Completed 28%
[23:29:46] Completed 29%
[23:30:46] Completed 30%
[23:31:46] Completed 31%
[23:32:48] Completed 32%
[23:33:48] Completed 33%
[23:34:49] Completed 34%
[23:35:49] Completed 35%
[23:36:50] Completed 36%
[23:41:20] Completed 37%
[23:48:46] Completed 38%
[23:56:42] Completed 39%
[00:04:08] Completed 40%
[00:11:33] Completed 41%
[00:18:59] Completed 42%
[00:26:25] Completed 43%
[00:33:51] Completed 44%
[00:41:16] Completed 45%
[00:48:42] Completed 46%
[00:56:07] Completed 47%
[01:03:33] Completed 48%
[01:10:58] Completed 49%
[01:18:24] Completed 50%
[01:25:50] Completed 51%
[01:33:15] Completed 52%
[01:40:41] Completed 53%
[01:48:07] Completed 54%
[01:55:32] Completed 55%
[02:02:58] Completed 56%
[02:10:24] Completed 57%
[02:17:49] Completed 58%
[02:25:14] Completed 59%
[02:32:40] Completed 60%
[02:40:06] Completed 61%
[02:47:31] Completed 62%
[02:54:57] Completed 63%
[03:02:25] Completed 64%
[03:09:51] Completed 65%
[03:17:17] Completed 66%
[03:24:43] Completed 67%
[03:32:09] Completed 68%
[03:37:45] + Working...
[03:39:37] Completed 69%
[03:47:03] Completed 70%
[03:54:54] Completed 71%
[04:02:24] Completed 72%
[04:09:50] Completed 73%
[04:17:15] Completed 74%
[04:24:41] Completed 75%
[04:32:07] Completed 76%
[04:39:32] Completed 77%
[04:46:58] Completed 78%
[04:54:23] Completed 79%
[05:01:49] Completed 80%
[05:09:15] Completed 81%
[05:16:40] Completed 82%
[05:24:06] Completed 83%
[05:31:32] Completed 84%
[05:38:57] Completed 85%
[05:46:23] Completed 86%
[05:53:49] Completed 87%
[06:01:14] Completed 88%
[06:08:40] Completed 89%
[06:16:05] Completed 90%
[06:23:31] Completed 91%
[06:30:57] Completed 92%
[06:38:22] Completed 93%
[06:45:48] Completed 94%
[06:53:14] Completed 95%
[07:00:40] Completed 96%
[07:08:05] Completed 97%
[07:15:31] Completed 98%
[07:22:58] Completed 99%
[07:30:23] Completed 100%
[07:30:24] Successful run
[07:30:24] DynamicWrapper: Finished Work Unit: sleep=10000
[07:30:34] Reserved 75628 bytes for xtc file; Cosm status=0
[07:30:34] Allocated 75628 bytes for xtc file
[07:30:34] - Reading up to 75628 from "work/wudata_04.xtc": Read 75628
[07:30:34] Read 75628 bytes from xtc file; available packet space=786354836
[07:30:34] xtc file hash check passed.
[07:30:34] Reserved 15168 15168 786354836 bytes for arc file=<work/wudata_04.trr> Cosm status=0
[07:30:34] Allocated 15168 bytes for arc file
[07:30:34] - Reading up to 15168 from "work/wudata_04.trr": Read 15168
[07:30:34] Read 15168 bytes from arc file; available packet space=786339668
[07:30:34] trr file hash check passed.
[07:30:34] Allocated 560 bytes for edr file
[07:30:34] Read bedfile
[07:30:34] edr file hash check passed.
[07:30:34] Allocated 33418 bytes for logfile
[07:30:34] Read logfile
[07:30:34] GuardedRun: success in DynamicWrapper
[07:30:34] GuardedRun: done
[07:30:34] Run: GuardedRun completed.
[07:30:35] + Opened results file
[07:30:35] - Writing 125286 bytes of core data to disk...
[07:30:35] Done: 124774 -> 99165 (compressed to 79.4 percent)
[07:30:35]   ... Done.
[07:30:35] DeleteFrameFiles: successfully deleted file=work/wudata_04.ckp
[07:30:35] Shutting down core 
[07:30:35] 
[07:30:35] Folding@home Core Shutdown: FINISHED_UNIT
[07:30:37] CoreStatus = 64 (100)
[07:30:37] Sending work to server
[07:30:37] Project: 5771 (Run 4, Clone 191, Gen 1922)


[07:30:37] + Attempting to send results [August 26 07:30:37 UTC]
[07:30:37] Gpu type=2 species=12.
[07:30:39] + Results successfully sent
[07:30:39] Thank you for your contribution to Folding@Home.
[07:30:39] + Number of Units Completed: 526
Image
P5-133XL
Posts: 2948
Joined: Sun Dec 02, 2007 4:36 am
Hardware configuration: Machine #1:

Intel Q9450; 2x2GB=8GB Ram; Gigabyte GA-X48-DS4 Motherboard; PC Power and Cooling Q750 PS; 2x GTX 460; Windows Server 2008 X64 (SP1).

Machine #2:

Intel Q6600; 2x2GB=4GB Ram; Gigabyte GA-X48-DS4 Motherboard; PC Power and Cooling Q750 PS; 2x GTX 460 video card; Windows 7 X64.

Machine 3:

Dell Dimension 8400, 3.2GHz P4 4x512GB Ram, Video card GTX 460, Windows 7 X32

I am currently folding just on the 5x GTX 460's for aprox. 70K PPD
Location: Salem. OR USA

Re: Project 5771 slowing down overnight [by factor of 4]

Post by P5-133XL »

When I see something like this I start looking for a scheduled application that is designed to run when someone is not on their machine. The classic one is a screen saver, but there are lots of other alternatives such as defragging your disk, or an anti-virus scan.

Since this is a GPU WU, I'd also check to see if a display reset didn't drop your video card's clock rates.
Image
new08
Posts: 188
Joined: Fri Jan 04, 2008 11:02 pm
Hardware configuration: Hewlett-Packard 1494 Win10 Build 1836
GeForce [MSI] GTX 950
Runs F@H Ver7.6.21
[As of Jan 2021]
Location: England

Re: Project 5771 slowing down overnight [by factor of 4]

Post by new08 »

Fair comment P5. The next unit kicked in, unattended, totally as normal -without reset of GPU clock rates. No Screen Saver.
The AV updates ultra quick and defragging keeps low, too and finishes in minutes.
I suspected what you say, but no evidence as to this sustained slow down till the unit finished.[See changeover in log below]
If it doesn't happen again, all well and good- but I report this as it may herald something 'in the woodwork' and could impact across a lot of work, in my guesstimation, if easily unnoticed / under reported!

Code: Select all

07:00:40] Completed 96%
[07:08:05] Completed 97%
[07:15:31] Completed 98%
[07:22:58] Completed 99%
[07:30:23] Completed 100%
[07:30:24] Successful run
[07:30:24] DynamicWrapper: Finished Work Unit: sleep=10000
[07:30:34] Reserved 75628 bytes for xtc file; Cosm status=0
[07:30:34] Allocated 75628 bytes for xtc file
[07:30:34] - Reading up to 75628 from "work/wudata_04.xtc": Read 75628
[07:30:34] Read 75628 bytes from xtc file; available packet space=786354836
[07:30:34] xtc file hash check passed.
[07:30:34] Reserved 15168 15168 786354836 bytes for arc file=<work/wudata_04.trr> Cosm status=0
[07:30:34] Allocated 15168 bytes for arc file
[07:30:34] - Reading up to 15168 from "work/wudata_04.trr": Read 15168
[07:30:34] Read 15168 bytes from arc file; available packet space=786339668
[07:30:34] trr file hash check passed.
[07:30:34] Allocated 560 bytes for edr file
[07:30:34] Read bedfile
[07:30:34] edr file hash check passed.
[07:30:34] Allocated 33418 bytes for logfile
[07:30:34] Read logfile
[07:30:34] GuardedRun: success in DynamicWrapper
[07:30:34] GuardedRun: done
[07:30:34] Run: GuardedRun completed.
[07:30:35] + Opened results file
[07:30:35] - Writing 125286 bytes of core data to disk...
[07:30:35] Done: 124774 -> 99165 (compressed to 79.4 percent)
[07:30:35]   ... Done.
[07:30:35] DeleteFrameFiles: successfully deleted file=work/wudata_04.ckp
[07:30:35] Shutting down core 
[07:30:35] 
[07:30:35] Folding@home Core Shutdown: FINISHED_UNIT
[07:30:37] CoreStatus = 64 (100)
[07:30:37] Sending work to server
[07:30:37] Project: 5771 (Run 4, Clone 191, Gen 1922)


[07:30:37] + Attempting to send results [August 26 07:30:37 UTC]
[07:30:37] Gpu type=2 species=12.
[07:30:39] + Results successfully sent
[07:30:39] Thank you for your contribution to Folding@Home.
[07:30:39] + Number of Units Completed: 526

[07:30:44] - Preparing to get new work unit...
[07:30:44] Cleaning up work directory
[07:30:44] + Attempting to get work packet
[07:30:44] Gpu type=2 species=12.
[07:30:44] - Connecting to assignment server
[07:31:00] - Successful: assigned to (171.67.108.11).
[07:31:00] + News From Folding@Home: Welcome to Folding@Home
[07:31:00] Loaded queue successfully.
[07:31:00] Gpu type=2 species=12.
[07:31:01] + Closed connections
[07:31:01] 
[07:31:01] + Processing work unit
[07:31:01] Core required: FahCore_11.exe
[07:31:01] Core found.
[07:31:01] Working on queue slot 05 [August 26 07:31:01 UTC]
[07:31:01] + Working ...
[07:31:01] 
[07:31:01] *------------------------------*
[07:31:01] Folding@Home GPU Core
[07:31:01] Version 1.31 (Tue Sep 15 10:57:42 PDT 2009)
[07:31:01] 
[07:31:01] Compiler  : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 14.00.50727.762 for 80x86 
[07:31:01] Build host: amoeba
[07:31:01] Board Type: Nvidia
[07:31:01] Core      : 
[07:31:01] Preparing to commence simulation
[07:31:01] - Looking at optimizations...
[07:31:01] DeleteFrameFiles: successfully deleted file=work/wudata_05.ckp
[07:31:01] - Created dyn
[07:31:01] - Files status OK
[07:31:01] - Expanded 46728 -> 252912 (decompressed 541.2 percent)
[07:31:01] Called DecompressByteArray: compressed_data_size=46728 data_size=252912, decompressed_data_size=252912 diff=0
[07:31:01] - Digital signature verified
[07:31:02] 
[07:31:02] Project: 5768 (Run 12, Clone 144, Gen 1681)
[07:31:02] 
[07:31:02] Assembly optimizations on if available.
[07:31:02] Entering M.D.
[07:31:08] Tpr hash work/wudata_05.tpr:  2485658627 1070680096 3713587499 1430668302 2730813239
[07:31:08] 
[07:31:08] Calling fah_main args: 14 usage=100
[07:31:08] 
[07:31:08] Working on Protein
[07:31:09] Client config found, loading data.
[07:31:10] Starting GUI Server
[07:32:10] Completed 1%
[07:33:11] Completed 2%
[07:34:12] Completed 3%
[07:35:12] Completed 4%
Image
bruce
Posts: 20822
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Project 5771 slowing down overnight [by factor of 4]

Post by bruce »

The other thing you might suspect is some kind of power-saving feature kicking in. Whether that's something that applies to the CPU (such as "sleep") or something that happens to the GPU, your system may be trying to save power by cutting back in a way that changes the speed that FAH's processing. If it doesn't happen consistently, though, then that's not it.
new08
Posts: 188
Joined: Fri Jan 04, 2008 11:02 pm
Hardware configuration: Hewlett-Packard 1494 Win10 Build 1836
GeForce [MSI] GTX 950
Runs F@H Ver7.6.21
[As of Jan 2021]
Location: England

Re: Project 5771 slowing down overnight [by factor of 4]

Post by new08 »

No- It's running the same config as for months past.
It appears as if the unit went into a 'brown study' for the remaining time -for unknown reason[s].
Puzzled as to this- is why I posted, not because I'm bothered about one unit as such and 'flagged' is 'bagged', next time :).
Image
Post Reply