New WU behaving strangely after GPU swap
Posted: Sun Jun 13, 2021 3:02 am
Hello,
I am new to interacting with the folding community, please let me know if I can provide further info or am posting incorrectly.
I swapped my R9 280X for a 660ti into one of my slots on the same board with a GTX 1050. The 1050 is folding as per normal, but the 660ti at first started at 99.99% and now is reaching "checkpoints" every few minutes and climbing by whole percents (1.00% to 2.00%).
I assumed maybe it picked up from a WU that the R9 hadnt completed, but I cant seem to figure out how to dump the WU and start over. Or if that is even what needs to be done.
Here is the log from the 660ti:
Any help is greatly appreciated.
I am new to interacting with the folding community, please let me know if I can provide further info or am posting incorrectly.
I swapped my R9 280X for a 660ti into one of my slots on the same board with a GTX 1050. The 1050 is folding as per normal, but the 660ti at first started at 99.99% and now is reaching "checkpoints" every few minutes and climbing by whole percents (1.00% to 2.00%).
I assumed maybe it picked up from a WU that the R9 hadnt completed, but I cant seem to figure out how to dump the WU and start over. Or if that is even what needs to be done.
Here is the log from the 660ti:
Code: Select all
*********************** Log Started 2021-06-13T02:27:12Z ***********************
02:27:12:FS01:Initialized folding slot 01: gpu:1:0 GK104 [GeForce GTX 660 Ti] 2634
02:27:12:WU00:FS01:Starting
02:27:12:WU00:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\ProgramData\FAHClient\cores/cores.foldingathome.org/win/64bit/22-0.0.13/Core_22.fah/FahCore_22.exe -dir 00 -suffix 01 -version 706 -lifeline 9340 -checkpoint 15 -opencl-platform 0 -opencl-device 1 -cuda-device 1 -gpu-vendor nvidia -gpu 1 -gpu-usage 100
02:27:12:WU00:FS01:Started FahCore on PID 9444
02:27:12:WU00:FS01:Core PID:9468
02:27:12:WU00:FS01:FahCore 0x22 started
02:27:13:WU00:FS01:0x22:*********************** Log Started 2021-06-13T02:27:12Z ***********************
02:27:13:WU00:FS01:0x22:*************************** Core22 Folding@home Core ***************************
02:27:13:WU00:FS01:0x22: Core: Core22
02:27:13:WU00:FS01:0x22: Type: 0x22
02:27:13:WU00:FS01:0x22: Version: 0.0.13
02:27:13:WU00:FS01:0x22: Author: Joseph Coffland <joseph@cauldrondevelopment.com>
02:27:13:WU00:FS01:0x22: Copyright: 2020 foldingathome.org
02:27:13:WU00:FS01:0x22: Homepage: https://foldingathome.org/
02:27:13:WU00:FS01:0x22: Date: Sep 19 2020
02:27:13:WU00:FS01:0x22: Time: 02:35:58
02:27:13:WU00:FS01:0x22: Revision: 571cf95de6de2c592c7c3ed48fcfb2e33e9ea7d3
02:27:13:WU00:FS01:0x22: Branch: core22-0.0.13
02:27:13:WU00:FS01:0x22: Compiler: Visual C++ 2015
02:27:13:WU00:FS01:0x22: Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
02:27:13:WU00:FS01:0x22: -DOPENMM_GIT_HASH="\"189320d0\""
02:27:13:WU00:FS01:0x22: Platform: win32 10
02:27:13:WU00:FS01:0x22: Bits: 64
02:27:13:WU00:FS01:0x22: Mode: Release
02:27:13:WU00:FS01:0x22:Maintainers: John Chodera <john.chodera@choderalab.org> and Peter Eastman
02:27:13:WU00:FS01:0x22: <peastman@stanford.edu>
02:27:13:WU00:FS01:0x22: Args: -dir 00 -suffix 01 -version 706 -lifeline 9444 -checkpoint 15
02:27:13:WU00:FS01:0x22: -opencl-platform 0 -opencl-device 1 -cuda-device 1 -gpu-vendor
02:27:13:WU00:FS01:0x22: nvidia -gpu 1 -gpu-usage 100
02:27:13:WU00:FS01:0x22:************************************ libFAH ************************************
02:27:13:WU00:FS01:0x22: Date: Sep 7 2020
02:27:13:WU00:FS01:0x22: Time: 19:09:56
02:27:13:WU00:FS01:0x22: Revision: 44301ed97b996b63fe736bb8073f22209cb2b603
02:27:13:WU00:FS01:0x22: Branch: HEAD
02:27:13:WU00:FS01:0x22: Compiler: Visual C++ 2015
02:27:13:WU00:FS01:0x22: Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
02:27:13:WU00:FS01:0x22: Platform: win32 10
02:27:13:WU00:FS01:0x22: Bits: 64
02:27:13:WU00:FS01:0x22: Mode: Release
02:27:13:WU00:FS01:0x22:************************************ CBang *************************************
02:27:13:WU00:FS01:0x22: Date: Sep 7 2020
02:27:13:WU00:FS01:0x22: Time: 19:08:30
02:27:13:WU00:FS01:0x22: Revision: 33fcfc2b3ed2195a423606a264718e31e6b3903f
02:27:13:WU00:FS01:0x22: Branch: HEAD
02:27:13:WU00:FS01:0x22: Compiler: Visual C++ 2015
02:27:13:WU00:FS01:0x22: Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
02:27:13:WU00:FS01:0x22: Platform: win32 10
02:27:13:WU00:FS01:0x22: Bits: 64
02:27:13:WU00:FS01:0x22: Mode: Release
02:27:13:WU00:FS01:0x22:************************************ System ************************************
02:27:13:WU00:FS01:0x22: CPU: Intel(R) Core(TM) i7-4790K CPU @ 4.00GHz
02:27:13:WU00:FS01:0x22: CPU ID: GenuineIntel Family 6 Model 60 Stepping 3
02:27:13:WU00:FS01:0x22: CPUs: 8
02:27:13:WU00:FS01:0x22: Memory: 15.86GiB
02:27:13:WU00:FS01:0x22:Free Memory: 13.89GiB
02:27:13:WU00:FS01:0x22: Threads: WINDOWS_THREADS
02:27:13:WU00:FS01:0x22: OS Version: 6.2
02:27:13:WU00:FS01:0x22:Has Battery: false
02:27:13:WU00:FS01:0x22: On Battery: false
02:27:13:WU00:FS01:0x22: UTC Offset: -7
02:27:13:WU00:FS01:0x22: PID: 9468
02:27:13:WU00:FS01:0x22: CWD: C:\ProgramData\FAHClient\work
02:27:13:WU00:FS01:0x22:************************************ OpenMM ************************************
02:27:13:WU00:FS01:0x22: Revision: 189320d0
02:27:13:WU00:FS01:0x22:********************************************************************************
02:27:13:WU00:FS01:0x22:Project: 18002 (Run 0, Clone 1075, Gen 37)
02:27:13:WU00:FS01:0x22:Unit: 0x00000000000000000000000000000000
02:27:13:WU00:FS01:0x22:Digital signatures verified
02:27:13:WU00:FS01:0x22:Folding@home GPU Core22 Folding@home Core
02:27:13:WU00:FS01:0x22:Version 0.0.13
02:27:13:WU00:FS01:0x22: Checkpoint write interval: 200000 steps (2%) [50 total]
02:27:13:WU00:FS01:0x22: JSON viewer frame write interval: 100000 steps (1%) [100 total]
02:27:13:WU00:FS01:0x22: XTC frame write interval: 5000000 steps (50%) [2 total]
02:27:13:WU00:FS01:0x22: Global context and integrator variables write interval: disabled
02:27:14:WU00:FS01:0x22:There are 4 platforms available.
02:27:14:WU00:FS01:0x22:Platform 0: Reference
02:27:14:WU00:FS01:0x22:Platform 1: CPU
02:27:14:WU00:FS01:0x22:Platform 2: OpenCL
02:27:14:WU00:FS01:0x22: opencl-device 1 specified
02:27:14:WU00:FS01:0x22:Platform 3: CUDA
02:27:14:WU00:FS01:0x22: cuda-device 1 specified
02:27:17:WU00:FS01:0x22:Attempting to create CUDA context:
02:27:17:WU00:FS01:0x22: Configuring platform CUDA
02:27:18:WU00:FS01:0x22: Using CUDA and gpu 1
02:27:18:WU00:FS01:0x22:Completed 0 out of 10000000 steps (0%)
02:27:18:WU00:FS01:0x22:Checkpoint completed at step 0
02:28:56:FS01:Paused
02:28:56:FS01:Shutting core down
02:28:56:WU00:FS01:0x22:WARNING:Console control signal 1 on PID 9468
02:28:56:WU00:FS01:0x22:Exiting, please wait. . .
02:28:56:WU00:FS01:0x22:Folding@home Core Shutdown: INTERRUPTED
02:28:57:WU00:FS01:FahCore returned: INTERRUPTED (102 = 0x66)
02:29:02:FS01:Unpaused
02:29:02:WU00:FS01:Starting
02:29:02:WU00:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\ProgramData\FAHClient\cores/cores.foldingathome.org/win/64bit/22-0.0.13/Core_22.fah/FahCore_22.exe -dir 00 -suffix 01 -version 706 -lifeline 9340 -checkpoint 15 -opencl-platform 0 -opencl-device 1 -cuda-device 1 -gpu-vendor nvidia -gpu 1 -gpu-usage 100
02:29:02:WU00:FS01:Started FahCore on PID 10556
02:29:02:WU00:FS01:Core PID:10580
02:29:02:WU00:FS01:FahCore 0x22 started
02:29:03:WU00:FS01:0x22:*********************** Log Started 2021-06-13T02:29:02Z ***********************
02:29:03:WU00:FS01:0x22:*************************** Core22 Folding@home Core ***************************
02:29:03:WU00:FS01:0x22: Core: Core22
02:29:03:WU00:FS01:0x22: Type: 0x22
02:29:03:WU00:FS01:0x22: Version: 0.0.13
02:29:03:WU00:FS01:0x22: Author: Joseph Coffland <joseph@cauldrondevelopment.com>
02:29:03:WU00:FS01:0x22: Copyright: 2020 foldingathome.org
02:29:03:WU00:FS01:0x22: Homepage: https://foldingathome.org/
02:29:03:WU00:FS01:0x22: Date: Sep 19 2020
02:29:03:WU00:FS01:0x22: Time: 02:35:58
02:29:03:WU00:FS01:0x22: Revision: 571cf95de6de2c592c7c3ed48fcfb2e33e9ea7d3
02:29:03:WU00:FS01:0x22: Branch: core22-0.0.13
02:29:03:WU00:FS01:0x22: Compiler: Visual C++ 2015
02:29:03:WU00:FS01:0x22: Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
02:29:03:WU00:FS01:0x22: -DOPENMM_GIT_HASH="\"189320d0\""
02:29:03:WU00:FS01:0x22: Platform: win32 10
02:29:03:WU00:FS01:0x22: Bits: 64
02:29:03:WU00:FS01:0x22: Mode: Release
02:29:03:WU00:FS01:0x22:Maintainers: John Chodera <john.chodera@choderalab.org> and Peter Eastman
02:29:03:WU00:FS01:0x22: <peastman@stanford.edu>
02:29:03:WU00:FS01:0x22: Args: -dir 00 -suffix 01 -version 706 -lifeline 10556 -checkpoint 15
02:29:03:WU00:FS01:0x22: -opencl-platform 0 -opencl-device 1 -cuda-device 1 -gpu-vendor
02:29:03:WU00:FS01:0x22: nvidia -gpu 1 -gpu-usage 100
02:29:03:WU00:FS01:0x22:************************************ libFAH ************************************
02:29:03:WU00:FS01:0x22: Date: Sep 7 2020
02:29:03:WU00:FS01:0x22: Time: 19:09:56
02:29:03:WU00:FS01:0x22: Revision: 44301ed97b996b63fe736bb8073f22209cb2b603
02:29:03:WU00:FS01:0x22: Branch: HEAD
02:29:03:WU00:FS01:0x22: Compiler: Visual C++ 2015
02:29:03:WU00:FS01:0x22: Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
02:29:03:WU00:FS01:0x22: Platform: win32 10
02:29:03:WU00:FS01:0x22: Bits: 64
02:29:03:WU00:FS01:0x22: Mode: Release
02:29:03:WU00:FS01:0x22:************************************ CBang *************************************
02:29:03:WU00:FS01:0x22: Date: Sep 7 2020
02:29:03:WU00:FS01:0x22: Time: 19:08:30
02:29:03:WU00:FS01:0x22: Revision: 33fcfc2b3ed2195a423606a264718e31e6b3903f
02:29:03:WU00:FS01:0x22: Branch: HEAD
02:29:03:WU00:FS01:0x22: Compiler: Visual C++ 2015
02:29:03:WU00:FS01:0x22: Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
02:29:03:WU00:FS01:0x22: Platform: win32 10
02:29:03:WU00:FS01:0x22: Bits: 64
02:29:03:WU00:FS01:0x22: Mode: Release
02:29:03:WU00:FS01:0x22:************************************ System ************************************
02:29:03:WU00:FS01:0x22: CPU: Intel(R) Core(TM) i7-4790K CPU @ 4.00GHz
02:29:03:WU00:FS01:0x22: CPU ID: GenuineIntel Family 6 Model 60 Stepping 3
02:29:03:WU00:FS01:0x22: CPUs: 8
02:29:03:WU00:FS01:0x22: Memory: 15.86GiB
02:29:03:WU00:FS01:0x22:Free Memory: 13.10GiB
02:29:03:WU00:FS01:0x22: Threads: WINDOWS_THREADS
02:29:03:WU00:FS01:0x22: OS Version: 6.2
02:29:03:WU00:FS01:0x22:Has Battery: false
02:29:03:WU00:FS01:0x22: On Battery: false
02:29:03:WU00:FS01:0x22: UTC Offset: -7
02:29:03:WU00:FS01:0x22: PID: 10580
02:29:03:WU00:FS01:0x22: CWD: C:\ProgramData\FAHClient\work
02:29:03:WU00:FS01:0x22:************************************ OpenMM ************************************
02:29:03:WU00:FS01:0x22: Revision: 189320d0
02:29:03:WU00:FS01:0x22:********************************************************************************
02:29:03:WU00:FS01:0x22:Project: 18002 (Run 0, Clone 1075, Gen 37)
02:29:03:WU00:FS01:0x22:Unit: 0x00000000000000000000000000000000
02:29:03:WU00:FS01:0x22:Digital signatures verified
02:29:03:WU00:FS01:0x22:Folding@home GPU Core22 Folding@home Core
02:29:03:WU00:FS01:0x22:Version 0.0.13
02:29:03:WU00:FS01:0x22: Checkpoint write interval: 200000 steps (2%) [50 total]
02:29:03:WU00:FS01:0x22: JSON viewer frame write interval: 100000 steps (1%) [100 total]
02:29:03:WU00:FS01:0x22: XTC frame write interval: 5000000 steps (50%) [2 total]
02:29:03:WU00:FS01:0x22: Global context and integrator variables write interval: disabled
02:29:03:WU00:FS01:0x22:There are 4 platforms available.
02:29:03:WU00:FS01:0x22:Platform 0: Reference
02:29:03:WU00:FS01:0x22:Platform 1: CPU
02:29:03:WU00:FS01:0x22:Platform 2: OpenCL
02:29:03:WU00:FS01:0x22: opencl-device 1 specified
02:29:03:WU00:FS01:0x22:Platform 3: CUDA
02:29:03:WU00:FS01:0x22: cuda-device 1 specified
02:29:06:WU00:FS01:0x22:Attempting to create CUDA context:
02:29:06:WU00:FS01:0x22: Configuring platform CUDA
02:29:07:WU00:FS01:0x22: Using CUDA and gpu 1
02:29:07:WU00:FS01:0x22:Completed 0 out of 10000000 steps (0%)
02:29:07:WU00:FS01:0x22:Checkpoint completed at step 0
02:36:02:FS01:Paused
02:36:02:FS01:Shutting core down
02:36:02:WU00:FS01:0x22:WARNING:Console control signal 1 on PID 10580
02:36:02:WU00:FS01:0x22:Exiting, please wait. . .
02:36:02:WU00:FS01:0x22:Folding@home Core Shutdown: INTERRUPTED
02:36:02:WU00:FS01:FahCore returned: INTERRUPTED (102 = 0x66)
02:38:09:FS01:Unpaused
02:38:09:WU00:FS01:Starting
02:38:09:WU00:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:\ProgramData\FAHClient\cores/cores.foldingathome.org/win/64bit/22-0.0.13/Core_22.fah/FahCore_22.exe -dir 00 -suffix 01 -version 706 -lifeline 9340 -checkpoint 15 -opencl-platform 0 -opencl-device 1 -cuda-device 1 -gpu-vendor nvidia -gpu 1 -gpu-usage 100
02:38:10:WU00:FS01:Started FahCore on PID 7236
02:38:10:WU00:FS01:Core PID:3672
02:38:10:WU00:FS01:FahCore 0x22 started
02:38:10:WU00:FS01:0x22:*********************** Log Started 2021-06-13T02:38:10Z ***********************
02:38:10:WU00:FS01:0x22:*************************** Core22 Folding@home Core ***************************
02:38:10:WU00:FS01:0x22: Core: Core22
02:38:10:WU00:FS01:0x22: Type: 0x22
02:38:10:WU00:FS01:0x22: Version: 0.0.13
02:38:10:WU00:FS01:0x22: Author: Joseph Coffland <joseph@cauldrondevelopment.com>
02:38:10:WU00:FS01:0x22: Copyright: 2020 foldingathome.org
02:38:10:WU00:FS01:0x22: Homepage: https://foldingathome.org/
02:38:10:WU00:FS01:0x22: Date: Sep 19 2020
02:38:10:WU00:FS01:0x22: Time: 02:35:58
02:38:10:WU00:FS01:0x22: Revision: 571cf95de6de2c592c7c3ed48fcfb2e33e9ea7d3
02:38:10:WU00:FS01:0x22: Branch: core22-0.0.13
02:38:10:WU00:FS01:0x22: Compiler: Visual C++ 2015
02:38:10:WU00:FS01:0x22: Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
02:38:10:WU00:FS01:0x22: -DOPENMM_GIT_HASH="\"189320d0\""
02:38:10:WU00:FS01:0x22: Platform: win32 10
02:38:10:WU00:FS01:0x22: Bits: 64
02:38:10:WU00:FS01:0x22: Mode: Release
02:38:10:WU00:FS01:0x22:Maintainers: John Chodera <john.chodera@choderalab.org> and Peter Eastman
02:38:10:WU00:FS01:0x22: <peastman@stanford.edu>
02:38:10:WU00:FS01:0x22: Args: -dir 00 -suffix 01 -version 706 -lifeline 7236 -checkpoint 15
02:38:10:WU00:FS01:0x22: -opencl-platform 0 -opencl-device 1 -cuda-device 1 -gpu-vendor
02:38:10:WU00:FS01:0x22: nvidia -gpu 1 -gpu-usage 100
02:38:10:WU00:FS01:0x22:************************************ libFAH ************************************
02:38:10:WU00:FS01:0x22: Date: Sep 7 2020
02:38:10:WU00:FS01:0x22: Time: 19:09:56
02:38:10:WU00:FS01:0x22: Revision: 44301ed97b996b63fe736bb8073f22209cb2b603
02:38:10:WU00:FS01:0x22: Branch: HEAD
02:38:10:WU00:FS01:0x22: Compiler: Visual C++ 2015
02:38:10:WU00:FS01:0x22: Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
02:38:10:WU00:FS01:0x22: Platform: win32 10
02:38:10:WU00:FS01:0x22: Bits: 64
02:38:10:WU00:FS01:0x22: Mode: Release
02:38:10:WU00:FS01:0x22:************************************ CBang *************************************
02:38:10:WU00:FS01:0x22: Date: Sep 7 2020
02:38:10:WU00:FS01:0x22: Time: 19:08:30
02:38:10:WU00:FS01:0x22: Revision: 33fcfc2b3ed2195a423606a264718e31e6b3903f
02:38:10:WU00:FS01:0x22: Branch: HEAD
02:38:10:WU00:FS01:0x22: Compiler: Visual C++ 2015
02:38:10:WU00:FS01:0x22: Options: /TP /nologo /EHa /wd4297 /wd4103 /O2 /Ob3 /Zc:throwingNew /MT
02:38:10:WU00:FS01:0x22: Platform: win32 10
02:38:10:WU00:FS01:0x22: Bits: 64
02:38:10:WU00:FS01:0x22: Mode: Release
02:38:10:WU00:FS01:0x22:************************************ System ************************************
02:38:10:WU00:FS01:0x22: CPU: Intel(R) Core(TM) i7-4790K CPU @ 4.00GHz
02:38:10:WU00:FS01:0x22: CPU ID: GenuineIntel Family 6 Model 60 Stepping 3
02:38:10:WU00:FS01:0x22: CPUs: 8
02:38:10:WU00:FS01:0x22: Memory: 15.86GiB
02:38:10:WU00:FS01:0x22:Free Memory: 12.99GiB
02:38:10:WU00:FS01:0x22: Threads: WINDOWS_THREADS
02:38:10:WU00:FS01:0x22: OS Version: 6.2
02:38:10:WU00:FS01:0x22:Has Battery: false
02:38:10:WU00:FS01:0x22: On Battery: false
02:38:10:WU00:FS01:0x22: UTC Offset: -7
02:38:10:WU00:FS01:0x22: PID: 3672
02:38:10:WU00:FS01:0x22: CWD: C:\ProgramData\FAHClient\work
02:38:10:WU00:FS01:0x22:************************************ OpenMM ************************************
02:38:10:WU00:FS01:0x22: Revision: 189320d0
02:38:10:WU00:FS01:0x22:********************************************************************************
02:38:10:WU00:FS01:0x22:Project: 18002 (Run 0, Clone 1075, Gen 37)
02:38:10:WU00:FS01:0x22:Unit: 0x00000000000000000000000000000000
02:38:10:WU00:FS01:0x22:Digital signatures verified
02:38:10:WU00:FS01:0x22:Folding@home GPU Core22 Folding@home Core
02:38:10:WU00:FS01:0x22:Version 0.0.13
02:38:10:WU00:FS01:0x22: Checkpoint write interval: 200000 steps (2%) [50 total]
02:38:10:WU00:FS01:0x22: JSON viewer frame write interval: 100000 steps (1%) [100 total]
02:38:10:WU00:FS01:0x22: XTC frame write interval: 5000000 steps (50%) [2 total]
02:38:10:WU00:FS01:0x22: Global context and integrator variables write interval: disabled
02:38:11:WU00:FS01:0x22:There are 4 platforms available.
02:38:11:WU00:FS01:0x22:Platform 0: Reference
02:38:11:WU00:FS01:0x22:Platform 1: CPU
02:38:11:WU00:FS01:0x22:Platform 2: OpenCL
02:38:11:WU00:FS01:0x22: opencl-device 1 specified
02:38:11:WU00:FS01:0x22:Platform 3: CUDA
02:38:11:WU00:FS01:0x22: cuda-device 1 specified
02:38:14:WU00:FS01:0x22:Attempting to create CUDA context:
02:38:14:WU00:FS01:0x22: Configuring platform CUDA
02:38:15:WU00:FS01:0x22: Using CUDA and gpu 1
02:38:15:WU00:FS01:0x22:Completed 0 out of 10000000 steps (0%)
02:38:15:WU00:FS01:0x22:Checkpoint completed at step 0
02:42:03:FS01:Finishing
02:47:00:WU00:FS01:0x22:Completed 100000 out of 10000000 steps (1%)
02:55:45:WU00:FS01:0x22:Completed 200000 out of 10000000 steps (2%)
02:55:46:WU00:FS01:0x22:Checkpoint completed at step 200000
Any help is greatly appreciated.