Uploads Timing Out/Error

[KWSN_Sir_Frank]

i have the results of two work units re-trying their uploads (13 times so far)...
WUs are 10624 (0,2,7) and 10642 (0,338,1)...
both are going to 128.252.203.2:8080...
when the first one was trying alone, it made it to aprox 83% or 84%...
now that both are trying at the same time, they fail at 39% to 43%...
they seem to time out at aprox 16 min...

can anything be done to allow more time to upload ???

frank

[Joe_H]

Welcome to the folding support forum.

Could you post your log file as described in this topic - viewtopic.php?f=61&t=26036. The first 100 or so lines that show your system information and configuration plus a section showing the upload attempts would help determine what is happening. It would also show the size of the file being uploaded, and give us an idea of how fast your upload speed is. 15 minutes is the normal upload timeout value on the work servers, usually it is not increased except for exceptionally large results files.

The project numbers you listed do not appear to be active, did you write down the correct numbers?

[KWSN_Sir_Frank]

hello joe

whoops...correct numbers are: 11624 (0,2,7) and 11642 (0,338,1)...
the results file sizes are aprox 28 meg each...
my connection is a DSL line...
thanks for your help...

frank

Code: Select all

*********************** Log Started 2016-01-23T02:40:00Z ***********************
02:40:00:************************* Folding@home Client *************************
02:40:00:      Website: http://folding.stanford.edu/
02:40:00:    Copyright: (c) 2009-2014 Stanford University
02:40:00:       Author: Joseph Coffland <joseph@cauldrondevelopment.com>
02:40:00:         Args: 
02:40:00:       Config: C:/Users/Frank/AppData/Roaming/FAHClient/config.xml
02:40:00:******************************** Build ********************************
02:40:00:      Version: 7.4.4
02:40:00:         Date: Mar 4 2014
02:40:00:         Time: 20:26:54
02:40:00:      SVN Rev: 4130
02:40:00:       Branch: fah/trunk/client
02:40:00:     Compiler: Intel(R) C++ MSVC 1500 mode 1200
02:40:00:      Options: /TP /nologo /EHa /Qdiag-disable:4297,4103,1786,279 /Ox -arch:SSE
02:40:00:               /QaxSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 /Qopenmp /Qrestrict /MT /Qmkl
02:40:00:     Platform: win32 XP
02:40:00:         Bits: 32
02:40:00:         Mode: Release
02:40:00:******************************* System ********************************
02:40:00:          CPU: Intel(R) Core(TM)2 Quad CPU Q9500 @ 2.83GHz
02:40:00:       CPU ID: GenuineIntel Family 6 Model 23 Stepping 10
02:40:00:         CPUs: 4
02:40:00:       Memory: 3.90GiB
02:40:00:  Free Memory: 3.07GiB
02:40:00:      Threads: WINDOWS_THREADS
02:40:00:   OS Version: 6.1
02:40:00:  Has Battery: false
02:40:00:   On Battery: false
02:40:00:   UTC Offset: -5
02:40:00:          PID: 2988
02:40:00:          CWD: C:/Users/Frank/AppData/Roaming/FAHClient
02:40:00:           OS: Windows 7 Professional
02:40:00:      OS Arch: AMD64
02:40:00:         GPUs: 0
02:40:00:         CUDA: Not detected
02:40:00:Win32 Service: false
02:40:00:***********************************************************************
02:40:00:<config>
02:40:00:  <!-- Network -->
02:40:00:  <proxy v=':8080'/>
02:40:00:
02:40:00:  <!-- Slot Control -->
02:40:00:  <power v='full'/>
02:40:00:
02:40:00:  <!-- User Information -->
02:40:00:  <passkey v='********************************'/>
02:40:00:  <team v='117'/>
02:40:00:  <user v='KWSN.Sir.Frank.of.the.Wood'/>
02:40:00:
02:40:00:  <!-- Folding Slots -->
02:40:00:  <slot id='0' type='CPU'/>
02:40:00:</config>
02:40:00:Trying to access database...
02:40:02:Enabled folding slot 00: READY cpu:4
02:40:03:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:11624 run:0 clone:2 gen:7 core:0xa4 unit:0x000000074c71bbb0569681725d1b79c3
02:40:04:WU00:FS00:Uploading 27.84MiB to 128.252.203.2
02:40:04:WU01:FS00:Sending unit results: id:01 state:SEND error:NO_ERROR project:11642 run:0 clone:338 gen:1 core:0xa4 unit:0x000000014c71bbb0569e899507d96b39
02:40:04:WU00:FS00:Connecting to 128.252.203.2:8080
02:40:04:WU01:FS00:Uploading 27.87MiB to 128.252.203.2
02:40:04:WU01:FS00:Connecting to 128.252.203.2:8080
02:40:04:WU02:FS00:Starting
02:40:04:WU02:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/Frank/AppData/Roaming/FAHClient/cores/web.stanford.edu/~pande/Win32/AMD64/Core_a4.fah/FahCore_a4.exe -dir 02 -suffix 01 -version 704 -lifeline 2988 -checkpoint 15 -np 4
02:40:04:WU02:FS00:Started FahCore on PID 2548
02:40:05:WU02:FS00:Core PID:2488
02:40:05:WU02:FS00:FahCore 0xa4 started
02:40:07:WU02:FS00:0xa4:
02:40:07:WU02:FS00:0xa4:*------------------------------*
02:40:07:WU02:FS00:0xa4:Folding@Home Gromacs GB Core
02:40:07:WU02:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
02:40:07:WU02:FS00:0xa4:
02:40:07:WU02:FS00:0xa4:Preparing to commence simulation
02:40:07:WU02:FS00:0xa4:- Ensuring status. Please wait.
02:40:14:WU01:FS00:Upload 0.45%
02:40:14:WU00:FS00:Upload 0.45%
02:40:16:WU02:FS00:0xa4:- Looking at optimizations...
02:40:16:WU02:FS00:0xa4:- Working with standard loops on this execution.
02:40:16:WU02:FS00:0xa4:- Previous termination of core was improper.
02:40:16:WU02:FS00:0xa4:- Going to use standard loops.
02:40:16:WU02:FS00:0xa4:- Files status OK
02:40:20:WU01:FS00:Upload 0.67%
02:40:20:WU00:FS00:Upload 0.67%
02:40:21:WU02:FS00:0xa4:- Expanded 6523533 -> 22366436 (decompressed 342.8 percent)
02:40:21:WU02:FS00:0xa4:Called DecompressByteArray: compressed_data_size=6523533 data_size=22366436, decompressed_data_size=22366436 diff=0
02:40:21:WU02:FS00:0xa4:- Digital signature verified
02:40:21:WU02:FS00:0xa4:
02:40:21:WU02:FS00:0xa4:Project: 9752 (Run 155, Clone 0, Gen 630)
02:40:21:WU02:FS00:0xa4:
02:40:22:WU02:FS00:0xa4:Entering M.D.
02:40:26:WU00:FS00:Upload 0.90%
02:40:28:WU02:FS00:0xa4:Using Gromacs checkpoints
02:40:29:WU02:FS00:0xa4:Mapping NT from 4 to 4 
02:40:30:WU01:FS00:Upload 1.12%
02:40:33:WU02:FS00:0xa4:Resuming from checkpoint
02:40:33:WU02:FS00:0xa4:Verified 02/wudata_01.log
02:40:33:WU02:FS00:0xa4:Verified 02/wudata_01.trr
02:40:33:WU02:FS00:0xa4:Verified 02/wudata_01.xtc
02:40:33:WU02:FS00:0xa4:Verified 02/wudata_01.edr
02:40:34:WU02:FS00:0xa4:Completed 56220 out of 80000 steps  (70%)
02:40:36:WU01:FS00:Upload 1.35%
02:40:36:WU00:FS00:Upload 1.35%
02:40:45:WU01:FS00:Upload 1.79%
02:40:46:WU00:FS00:Upload 1.80%
(break/snip)
02:54:54:WU00:FS00:Upload 37.72%
02:55:00:WU01:FS00:Upload 37.90%
02:56:30:WU00:FS00:Upload 38.17%
02:56:30:WARNING:WU00:FS00:Exception: Failed to send results to work server: Transfer failed
02:56:30:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:11624 run:0 clone:2 gen:7 core:0xa4 unit:0x000000074c71bbb0569681725d1b79c3
02:56:30:WU00:FS00:Uploading 27.84MiB to 128.252.203.2
02:56:30:WU00:FS00:Connecting to 128.252.203.2:8080
02:56:37:WU01:FS00:Upload 38.13%
02:56:37:WARNING:WU01:FS00:Exception: Failed to send results to work server: Transfer failed
02:56:37:WU01:FS00:Sending unit results: id:01 state:SEND error:NO_ERROR project:11642 run:0 clone:338 gen:1 core:0xa4 unit:0x000000014c71bbb0569e899507d96b39
02:56:37:WU01:FS00:Uploading 27.87MiB to 128.252.203.2
02:56:37:WU01:FS00:Connecting to 128.252.203.2:8080
02:56:39:WU00:FS00:Upload 0.67%
02:56:45:WU01:FS00:Upload 0.22%
02:56:48:WU00:FS00:Upload 1.12%
02:56:54:WU00:FS00:Upload 1.35%

[toTOW]

How fast can your DSL connection send data ?

I suspect that you might be hitting the connection timeout if your connection is too slow ...

[KWSN_Sir_Frank]

hello toTOW

just ran a speed test:
download speed: 0.95 mbps
upload speed: 0.22 mbps

fiber is coming soon, but they are still in the digging/boring stage at the moment...

frank

[7im]

Is that your typical speed? Rebooting the modem may get you a higher connection rate. Happens to me all the time.

[KWSN_Sir_Frank]

hello 7im

just re-booted and ran another speed test...no improvement...

frank

[toTOW]

I sincerely sympathize ...

Uploading 28 MB with your upload speed would require a minimum of 17 minutes ... I don't know how this server is set up, but you're definitely hitting the connection timeout (I don't know how it is set on this server, but it is usually something between 15 and 30 minutes)

[KWSN_Sir_Frank]

Is there any way to request an increase in the time-out limit ???
or are these two work units (about 5000 points total) just going to be flushed down the pipes ???
and i wonder how many other folks have run into this same problem and not realized what was happening ???

(I have stopped the job so as to not hit the re-try limit...)

any help will be really appreciated...

thanks in advance

frank

[davidcoton]

toTOW has passed on the timeout issue to the PG team.

[mmonnin]

small packets maybe?

[KWSN_Sir_Frank]

toTOW: thank you for forwarding the details of the problem to the folks who can fix it...

mmonnin: if you are thinking (as i am) of sending the results in two or four sections,
i think it would be good, but the division and re-assembly may be a can of worms...
if the division/re-assembly is not difficult, it would solve the problem...

raising the time-out limit AND sending the results one-at-a-time should also solve the problem...

frank

[mmonnin]

toTOW: thank you for forwarding the details of the problem to the folks who can fix it...

mmonnin: if you are thinking (as i am) of sending the results in two or four sections,
i think it would be good, but the division and re-assembly may be a can of worms...
if the division/re-assembly is not difficult, it would solve the problem...

raising the time-out limit AND sending the results one-at-a-time should also solve the problem...

frank

Just small WUs. max-packet-size = small.
viewtopic.php?t=18187

[Joe_H]

That option is currently obsolete for CPU WU's, it mostly applied to the older end of life Core_78 projects. Currently the only CPU size distinction is for the large WU's using the bigadv setting for 24+ core Linux machines.

[mmonnin]

Ah sorry. SOL.