Upload compression

[butc8]

Is it possible to implement upload compression especially when you getting 15MB result files for the SMP client p30xx.

[uncle_fungus]

Results are already compressed before upload.

Code: Select all

[06:39:15] Finished Work Unit:
[06:39:15] - Reading up to 230280 from "work/wudata_00.arc": Read 230280
[06:39:15] - Reading up to 173868 from "work/wudata_00.xtc": Read 173868
[06:39:15] goefile size: 0
[06:39:15] logfile size: 119813
[06:39:15] Leaving Run
[06:39:20] - Writing 594025 bytes of core data to disk...
[06:39:20] Done: 593513 -> 449135 (compressed to 75.6 percent)
[06:39:20]   ... Done.
[06:39:21] - Shutting down core
[06:39:21]
[06:39:21] Folding@home Core Shutdown: FINISHED_UNIT

[butc8]

Does it still compress when it does this

Code: Select all

[06:44:27] Finished Work Unit:
[06:44:27] - Reading up to 3721632 from "work/wudata_02.arc": Read 3721632
[06:44:27] - Reading up to 1776020 from "work/wudata_02.xtc": Read 1776020
[06:44:27] goefile size: 0
[06:44:27] logfile size: 25510
[06:44:27] Leaving Run
[06:44:29] - Writing 5527562 bytes of core data to disk...
[06:44:29]   ... Done.
[06:44:29] - Failed to delete work/wudata_02.sas
[06:44:29] - Failed to delete work/wudata_02.goe
[06:44:29] Warning:  check for stray files
[06:44:29] - Shutting down core
[06:46:29] 
[06:46:29] Folding@home Core Shutdown: FINISHED_UNIT
[06:46:29] 
[06:46:29] Folding@home Core Shutdown: FINISHED_UNIT
[06:46:32] CoreStatus = 64 (100)
[06:46:32] Unit 2 finished with 80 percent of time to deadline remaining.
[06:46:32] Updated performance fraction: 0.750828
[06:46:32] Sending work to server

[7im]

-verbosity 9 on in that example?

[alancabler]

Is it possible to implement upload compression especially when you getting 15MB result files for the SMP client p30xx.

Some "WUs greater than 5MB in size", or single- client Bonus WUs are known to be around 45MB in size... not for dial- up.

[butc8]

-verbosity 9 on in that example?

yip

[Reddog99]

I have two computers running SMP units and the size of the results files returned is about to cause me to be penalized by my ISP because of the excess bandwidth being used. I'm on a satellite connection and they allow me 2.3 gb upload bandwidth, and I'm using about 3.0 gb.

I experimented with the smp result files and found that the 22 mb file sizes can be reduced to about 15 mb by compressing using "7zip". So why isn't Stanford compressing the files? With result file sizes seeming to grow, this could become a problem for more people than just me.

Pat


Merged new post with existing topic. PM Sent. 7im

[David]

I suspect they need to find a balance between computational power needed to compress units (and hence time to compress) vs file sizes. I'm not sure how old the compression method used is (ie is it the same as was used for v4? v3? v2? but it may be worth looking at alernatives).