Page 1 of 2

16434 (581, 1, 0) & (737, 1 2) - Dumped

Posted: Mon Apr 27, 2020 3:09 pm
by ToeBlister
Received 2x of 16434. Both went to CS and dumped. 104.36MiB.

System Config:
RTX 2060 Mobile Stock
Win 10 Home

Code: Select all

02:24:14:WU00:FS00:FahCore 0x22 started
02:24:14:WU00:FS00:0x22:*********************** Log Started 2020-04-27T02:24:14Z ***********************
02:24:14:WU00:FS00:0x22:*************************** Core22 Folding@home Core ***************************
02:24:14:WU00:FS00:0x22:       Type: 0x22
02:24:14:WU00:FS00:0x22:       Core: Core22
02:24:14:WU00:FS00:0x22:    Website: https://foldingathome.org/
02:24:14:WU00:FS00:0x22:  Copyright: (c) 2009-2018 foldingathome.org
02:24:14:WU00:FS00:0x22:     Author: John Chodera <john.chodera@choderalab.org> and Rafal Wiewiora
02:24:14:WU00:FS00:0x22:             <rafal.wiewiora@choderalab.org>
02:24:14:WU00:FS00:0x22:       Args: -dir 00 -suffix 01 -version 706 -lifeline 4320 -checkpoint 15
02:24:14:WU00:FS00:0x22:             -gpu-vendor nvidia -opencl-platform 0 -opencl-device 0 -cuda-device
02:24:14:WU00:FS00:0x22:             0 -gpu 0
02:24:14:WU00:FS00:0x22:     Config: <none>
02:24:14:WU00:FS00:0x22:************************************ Build *************************************
02:24:14:WU00:FS00:0x22:    Version: 0.0.2
02:24:14:WU00:FS00:0x22:       Date: Dec 6 2019
02:24:14:WU00:FS00:0x22:       Time: 21:30:31
02:24:14:WU00:FS00:0x22: Repository: Git
02:24:14:WU00:FS00:0x22:   Revision: abeb39247cc72df5af0f63723edafadb23d5dfbe
02:24:14:WU00:FS00:0x22:     Branch: HEAD
02:24:14:WU00:FS00:0x22:   Compiler: Visual C++ 2008
02:24:14:WU00:FS00:0x22:    Options: /TP /nologo /EHa /wd4297 /wd4103 /Ox /MT
02:24:14:WU00:FS00:0x22:   Platform: win32 10
02:24:14:WU00:FS00:0x22:       Bits: 64
02:24:14:WU00:FS00:0x22:       Mode: Release
02:24:14:WU00:FS00:0x22:************************************ System ************************************
02:24:14:WU00:FS00:0x22:        CPU: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
02:24:14:WU00:FS00:0x22:     CPU ID: GenuineIntel Family 6 Model 158 Stepping 10
02:24:14:WU00:FS00:0x22:       CPUs: 12
02:24:14:WU00:FS00:0x22:     Memory: 31.86GiB
02:24:14:WU00:FS00:0x22:Free Memory: 19.25GiB
02:24:14:WU00:FS00:0x22:    Threads: WINDOWS_THREADS
02:24:14:WU00:FS00:0x22: OS Version: 6.2
02:24:14:WU00:FS00:0x22:Has Battery: true
02:24:14:WU00:FS00:0x22: On Battery: false
02:24:14:WU00:FS00:0x22: UTC Offset: 8
02:24:14:WU00:FS00:0x22:        PID: 10416
02:24:14:WU00:FS00:0x22:        CWD: C:\Users\XXX\AppData\Roaming\FAHClient\work
02:24:14:WU00:FS00:0x22:         OS: Windows 10 Home
02:24:14:WU00:FS00:0x22:    OS Arch: AMD64
02:24:14:WU00:FS00:0x22:********************************************************************************
(581, 1, 0)

Code: Select all

08:09:47:WU00:FS00:FahCore returned: FINISHED_UNIT (100 = 0x64)
08:09:47:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:16434 run:581 clone:1 gen:0 core:0x22 unit:0x0000000003854c135e9cbaccc8f8e356
08:09:47:WU00:FS00:Uploading 104.36MiB to 3.133.76.19
08:09:47:WU00:FS00:Connecting to 3.133.76.19:8080
08:10:02:WU00:FS00:Upload 2.40%
08:10:02:WARNING:WU00:FS00:Exception: Failed to send results to work server: Transfer failed
08:10:02:WU00:FS00:Trying to send results to collection server
08:10:02:WU00:FS00:Uploading 104.36MiB to 3.21.157.11
08:10:02:WU00:FS00:Connecting to 3.21.157.11:8080
08:10:08:WU00:FS00:Upload 2.16%
.
.
.
08:18:05:WU00:FS00:Upload 99.23%
08:18:14:WU00:FS00:Upload complete
08:18:14:WU00:FS00:Server responded WORK_QUIT (404)
08:18:14:WARNING:WU00:FS00:Server did not like results, dumping

(737, 1 2)

Code: Select all

14:13:46:WU00:FS00:FahCore returned: FINISHED_UNIT (100 = 0x64)
14:13:46:WU00:FS00:Sending unit results: id:00 state:SEND error:NO_ERROR project:16434 run:737 clone:1 gen:2 core:0x22 unit:0x0000000203854c135e9cbaccb5f4d0a8
14:13:46:WU00:FS00:Uploading 104.36MiB to 3.133.76.19
14:13:46:WU00:FS00:Connecting to 3.133.76.19:8080
14:14:10:WU00:FS00:Upload 2.40%
14:14:10:WARNING:WU00:FS00:Exception: Failed to send results to work server: Transfer failed
14:14:10:WU00:FS00:Trying to send results to collection server
14:14:10:WU00:FS00:Uploading 104.36MiB to 3.21.157.11
14:14:10:WU00:FS00:Connecting to 3.21.157.11:8080
14:14:16:WU00:FS00:Upload 2.40%
.
.
.
14:22:13:WU00:FS00:Upload 99.78%
14:22:20:WU00:FS00:Upload complete
14:22:20:WU00:FS00:Server responded WORK_QUIT (404)
14:22:20:WARNING:WU00:FS00:Server did not like results, dumping

Re: 16434 (581, 1, 0) & (737, 1 2) - Dumped

Posted: Mon Apr 27, 2020 6:45 pm
by 1TM
Same here. After a 5 hours work, the server responded WORK_QUIT (404) and assigned no points (some 210K was estimated).
Have another one of these 16434 running and half-done (215773 estimated credit).

How can I dump it / kill it?
This server will also fail, and it would waste another 104MB of my metered connection.

Code: Select all

18:09:29:WU00:FS02:0x22:Completed 2475000 out of 2500000 steps (99%)

18:12:41:WU00:FS02:0x22:Completed 2500000 out of 2500000 steps (100%)
18:12:47:WU00:FS02:0x22:Saving result file ..\logfile_01.txt
18:12:47:WU00:FS02:0x22:Saving result file checkpointState.xml
18:12:47:WU00:FS02:0x22:Saving result file checkpt.crc
18:12:47:WU00:FS02:0x22:Saving result file positions.xtc
18:12:48:WU00:FS02:0x22:Saving result file science.log
18:12:48:WU00:FS02:0x22:Folding@home Core Shutdown: FINISHED_UNIT
18:12:48:WU00:FS02:FahCore returned: FINISHED_UNIT (100 = 0x64)
18:12:48:WU00:FS02:Sending unit results: id:00 state:SEND error:NO_ERROR project:16434 run:372 clone:3 gen:0 core:0x22 unit:0x0000000003854c135e9cbaccf429a179
18:12:48:WU00:FS02:Uploading 104.36MiB to 3.133.76.19
18:12:56:WU00:FS02:Upload 0.06%
18:13:28:WU00:FS02:Upload 0.12%
18:13:28:WARNING:WU00:FS02:Exception: Failed to send results to work server: Transfer failed
18:13:28:WU00:FS02:Trying to send results to collection server
18:13:28:WU00:FS02:Uploading 104.36MiB to 3.21.157.11
18:13:34:WU00:FS02:Upload 0.78%

18:23:46:WU00:FS02:Upload 99.42%
18:23:50:WU00:FS02:Upload complete
18:23:50:WU00:FS02:Server responded WORK_QUIT (404)
18:23:50:WARNING:WU00:FS02:Server did not like results, dumping

Re: 16434 (581, 1, 0) & (737, 1 2) - Dumped

Posted: Mon Apr 27, 2020 6:54 pm
by iceman1992
Are your systems overclocked/undervolted at all? I had the same issue => viewtopic.php?f=19&t=34871 could have been because it's undervolted, although it has completed many WUs before without problems.
Same server: 3.21.157.11:8080

Re: 16434 (581, 1, 0) & (737, 1 2) - Dumped

Posted: Mon Apr 27, 2020 7:18 pm
by ManInTheSun
Same here. Dumped once. Linux, standard settings on rtx2070S, 3.21.157.11.
It went through the second time though.

Re: 16434 (581, 1, 0) & (737, 1 2) - Dumped

Posted: Mon Apr 27, 2020 7:30 pm
by 1TM
@iceman1992 - thank you for suggesting this as a possibility. No, this GPU was not undervolted. Actually, the other GPU was so it was a correct decision to kill the second run.

Used this opportunity to update from 7.5.1 to 7.6.9 control. Also new tasks kept coming from the same server 3.133.76.19, so I temporarily blacklisted it in my router.

Re: 16434 (581, 1, 0) & (737, 1 2) - Dumped

Posted: Tue Apr 28, 2020 12:12 am
by ToeBlister
Mine was stocked. No undervolt nor overclocked.
Most of my other WUs (not this project) completed without errors.

Re: 16434 (581, 1, 0) & (737, 1 2) - Dumped

Posted: Tue Apr 28, 2020 4:01 am
by PantherX
Please note that there's a possibility of a new server feature contributing to this problem which is being investigated: viewtopic.php?p=330294#p330294

Re: 16434 (581, 1, 0) & (737, 1 2) - Dumped

Posted: Tue Apr 28, 2020 4:08 am
by iceman1992
PantherX wrote:Please note that there's a possibility of a new server feature contributing to this problem which is being investigated: viewtopic.php?p=330294#p330294
That's for project 13400, does it apply here too? And should we avoid the server for the time being?

Re: 16434 (581, 1, 0) & (737, 1 2) - Dumped

Posted: Tue Apr 28, 2020 4:20 am
by PantherX
It's not a "Project" issue, rather a "Server" issue. Thus, if it is an issue, it's on the Server so can potentially impact all or some of the Projects hosted by that Server. We will have to wait and see if there's any update on that or not.

Re: 16434 (581, 1, 0) & (737, 1 2) - Dumped

Posted: Tue Apr 28, 2020 5:50 am
by alien88

Code: Select all

14:09:37:WU03:FS01:0x22:Project: 16434 (Run 1006, Clone 2, Gen 1)
14:09:37:WU03:FS01:0x22:Unit: 0x0000000103854c135e9cbacb26bbc991
14:09:37:WU03:FS01:0x22:Reading tar file core.xml
14:09:37:WU03:FS01:0x22:Reading tar file integrator.xml
14:09:37:WU03:FS01:0x22:Reading tar file state.xml
14:09:38:WU03:FS01:0x22:Reading tar file system.xml
14:09:38:WU03:FS01:0x22:Digital signatures verified
14:09:38:WU03:FS01:0x22:Folding@home GPU Core22 Folding@home Core
14:09:38:WU03:FS01:0x22:Version 0.0.2
... crunching - no errors ... 
17:35:18:WU03:FS01:0x22:Completed 2500000 out of 2500000 steps (100%)
17:35:24:WU03:FS01:0x22:Saving result file ..\logfile_01.txt
17:35:24:WU03:FS01:0x22:Saving result file checkpointState.xml
17:35:24:WU03:FS01:0x22:Saving result file checkpt.crc
17:35:24:WU03:FS01:0x22:Saving result file positions.xtc
17:35:25:WU03:FS01:0x22:Saving result file science.log
17:35:25:WU03:FS01:0x22:Folding@home Core Shutdown: FINISHED_UNIT
17:35:26:WU03:FS01:FahCore returned: FINISHED_UNIT (100 = 0x64)
17:35:26:WU03:FS01:Sending unit results: id:03 state:SEND error:NO_ERROR project:16434 run:1006 clone:2 gen:1 core:0x22 unit:0x0000000103854c135e9cbacb26bbc991
17:35:26:WU03:FS01:Uploading 104.36MiB to 3.133.76.19
17:35:26:WU03:FS01:Connecting to 3.133.76.19:8080
17:35:58:WU03:FS01:Upload 0.12%
17:35:58:WARNING:WU03:FS01:Exception: Failed to send results to work server: Transfer failed
17:35:58:WU03:FS01:Trying to send results to collection server
17:35:58:WU03:FS01:Uploading 104.36MiB to 3.21.157.11
17:35:58:WU03:FS01:Connecting to 3.21.157.11:8080
17:36:04:WU03:FS01:Upload 1.62%
17:36:10:WU03:FS01:Upload 3.35%
... more upload ... 
17:40:40:WU03:FS01:Upload 83.78%
17:40:46:WU03:FS01:Upload 85.58%
17:40:52:WU03:FS01:Upload 87.32%
17:40:58:WU03:FS01:Upload 89.11%
17:41:04:WU03:FS01:Upload 90.85%
17:41:10:WU03:FS01:Upload 92.71%
17:41:16:WU03:FS01:Upload 94.38%
17:41:22:WU03:FS01:Upload 96.18%
17:41:28:WU03:FS01:Upload 97.98%
17:41:34:WU03:FS01:Upload 99.71%
17:41:35:WU03:FS01:Upload complete
17:41:35:WU03:FS01:Server responded WORK_QUIT (404)
17:41:35:WARNING:WU03:FS01:Server did not like results, dumping
17:41:35:WU03:FS01:Cleaning up
Same issue as other have reported, no overclocking, etc.

Re: 16434 (581, 1, 0) & (737, 1 2) - Dumped

Posted: Tue Apr 28, 2020 6:10 am
by ToeBlister
Can I get a quick show of those that got their WUs dumpes are folding outside of US?

In the meanwhile, I'll go back onto VPS.

Re: 16434 (581, 1, 0) & (737, 1 2) - Dumped

Posted: Tue Apr 28, 2020 7:09 am
by iceman1992
ToeBlister wrote:Can I get a quick show of those that got their WUs dumpes are folding outside of US?

In the meanwhile, I'll go back onto VPS.
I'm outside of US.

Re: 16434 (581, 1, 0) & (737, 1 2) - Dumped

Posted: Tue Apr 28, 2020 7:10 am
by 1TM
currently folding outside of US

Re: 16434 (581, 1, 0) & (737, 1 2) - Dumped

Posted: Tue Apr 28, 2020 8:27 am
by ToeBlister
Good. Me too.
I just folded 16434 (311, 0, 4) and returned it successfully ON PROXY.
I used Hotspot Shield and exiting server in US.

I think that is the key. The changes made to AS/WS lately is not accepting connections outside of US correctly, thus causing our WUs to be dumped.
Can someone let Dr. John Chodera know? He was asking about this in another thread too.

Re: 16434 (581, 1, 0) & (737, 1 2) - Dumped

Posted: Tue Apr 28, 2020 12:52 pm
by jcabana
I had two consecutives run of this project last night, and both were rejected by the collection server.
First was Run 491, Clone 1, Gen 2
Second was Run 635, Clone 4, Gen 0

I am also outside the US.
Hope this info helps.