This recently started happening. I get the following crash. Constantly. Sometimes one of my GPUs, Sometimes my CPU.
Code: Select all
Problem signature:
Problem Event Name: APPCRASH
Application Name: FahCore_a4.exe
Application Version: 0.0.0.0
Application Timestamp: 4d23eafc
Fault Module Name: ntdll.dll
Fault Module Version: 6.1.7601.17725
Fault Module Timestamp: 4ec49b8f
Exception Code: c0000029
Exception Offset: 00090812
OS Version: 6.1.7601.2.1.0.256.1
Locale ID: 1033
Additional Information 1: 0a9e
Additional Information 2: 0a9e372d3b4ad19135b953a78882e789
Additional Information 3: 0a9e
Additional Information 4: 0a9e372d3b4ad19135b953a78882e789
Code: Select all
*********************** Log Started 2012-02-02T21:28:06 ************************
21:28:06:************************* Folding@home Client *************************
21:28:06: Website: http://folding.stanford.edu/
21:28:06: Copyright: (c) 2009-2012 Stanford University
21:28:06: Author: Joseph Coffland <joseph@cauldrondevelopment.com>
21:28:06: Args: --lifeline 3000 --command-port=36330
21:28:06: Config: C:/Users/D.Tillman/AppData/Roaming/FAHClient/config.xml
21:28:06:******************************** Build ********************************
21:28:06: Version: 7.1.43
21:28:06: Date: Jan 2 2012
21:28:06: Time: 12:33:05
21:28:06: SVN Rev: 3223
21:28:06: Branch: fah/trunk/client
21:28:06: Compiler: Intel(R) C++ MSVC 1500 mode 1200
21:28:06: Options: /TP /nologo /EHa /Qdiag-disable:4297,4103,1786,279 /Ox -arch:SSE
21:28:06: /QaxSSE2,SSE3,SSSE3,SSE4.1,SSE4.2 /Qopenmp /Qrestrict /MT
21:28:06: Platform: win32 XP
21:28:06: Bits: 32
21:28:06: Mode: Release
21:28:06:******************************* System ********************************
21:28:06: CPU: Intel(R) Core(TM) i7 CPU X 940 @ 2.13GHz
21:28:06: CPU ID: GenuineIntel Family 6 Model 30 Stepping 5
21:28:06: CPUs: 8
21:28:06: Memory: 15.99GiB
21:28:06: Free Memory: 14.02GiB
21:28:06: Threads: WINDOWS_THREADS
21:28:06: On Battery: false
21:28:06: UTC offset: -5
21:28:06: PID: 5664
21:28:06: CWD: C:/Users/D.Tillman/AppData/Roaming/FAHClient
21:28:06: OS: Windows 7 Ultimate
21:28:06: OS Arch: AMD64
21:28:06: GPUs: 2
21:28:06: GPU 0: ATI:4 Whistler XT [AMD Radeon HD 6970M]
21:28:06: GPU 1: ATI:4 Whistler XT [AMD Radeon HD 6970M]
21:28:06: CUDA: Not detected
21:28:06:Win32 Service: false
21:28:06:***********************************************************************
21:28:07:<config>
21:28:07: <!-- FahCore Control -->
21:28:07: <checkpoint v='20'/>
21:28:07:
21:28:07: <!-- Folding Slot Configuration -->
21:28:07: <gpu v='true'/>
21:28:07:
21:28:07: <!-- Network -->
21:28:07: <proxy v=':8080'/>
21:28:07:
21:28:07: <!-- User Information -->
21:28:07: <passkey v='********************************'/>
21:28:07: <team v='213698'/>
21:28:07: <user v='DR650SE'/>
21:28:07:
21:28:07: <!-- Folding Slots -->
21:28:07: <slot id='0' type='GPU'>
21:28:07: <next-unit-percentage v='90'/>
21:28:07: </slot>
21:28:07: <slot id='1' type='GPU'>
21:28:07: <next-unit-percentage v='90'/>
21:28:07: </slot>
21:28:07: <slot id='2' type='SMP'>
21:28:07: <cpus v='6'/>
21:28:07: <next-unit-percentage v='90'/>
21:28:07: </slot>
21:28:07:</config>
21:28:07:Trying to access database...
21:28:07:Successfully acquired database lock
21:28:07:Enabled folding slot 00: READY gpu:0:"Whistler XT [AMD Radeon HD 6970M]"
21:28:07:Enabled folding slot 01: READY gpu:1:"Whistler XT [AMD Radeon HD 6970M]"
21:28:07:Enabled folding slot 02: READY smp:6
21:28:07:WU03:FS01:Starting
21:28:07:WU03:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/D.Tillman/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/ATI/R600/Core_16.fah/FahCore_16.exe -dir 03 -suffix 01 -version 701 -checkpoint 20 -gpu 1
21:28:07:WU03:FS01:Started FahCore on PID 1692
21:28:07:WU03:FS01:Core PID:4016
21:28:07:WU03:FS01:FahCore 0x16 started
21:28:07:WU01:FS00:Starting
21:28:07:WU01:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/D.Tillman/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/ATI/R600/Core_16.fah/FahCore_16.exe -dir 01 -suffix 01 -version 701 -checkpoint 20 -gpu 0
21:28:07:WU01:FS00:Started FahCore on PID 1980
21:28:07:WU01:FS00:Core PID:4212
21:28:07:WU01:FS00:FahCore 0x16 started
21:28:07:WU02:FS02:Starting
21:28:07:WU02:FS02:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/D.Tillman/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/Core_a4.fah/FahCore_a4.exe -dir 02 -suffix 01 -version 701 -checkpoint 20 -np 6
21:28:07:WU02:FS02:Started FahCore on PID 3940
21:28:07:WU02:FS02:Core PID:6856
21:28:07:WU02:FS02:FahCore 0xa4 started
21:28:07:WU03:FS01:0x16:
21:28:07:WU03:FS01:0x16:*------------------------------*
21:28:07:WU03:FS01:0x16:Folding@Home GPU Core
21:28:07:WU03:FS01:0x16:Version 2.11 (Thu Dec 9 15:00:14 PST 2010)
21:28:07:WU03:FS01:0x16:
21:28:07:WU01:FS00:0x16:
21:28:07:WU03:FS01:0x16:Compiler : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 15.00.30729.01 for 80x86
21:28:07:WU01:FS00:0x16:*------------------------------*
21:28:07:WU03:FS01:0x16:Build host: user-f6d030f24f
21:28:07:WU01:FS00:0x16:Folding@Home GPU Core
21:28:07:WU03:FS01:0x16:Board Type: AMD/OpenCL
21:28:07:WU01:FS00:0x16:Version 2.11 (Thu Dec 9 15:00:14 PST 2010)
21:28:07:WU03:FS01:0x16:Core : x=16
21:28:07:WU02:FS02:0xa4:
21:28:07:WU01:FS00:0x16:
21:28:07:WU03:FS01:0x16: Window's signal control handler registered.
21:28:07:WU02:FS02:0xa4:*------------------------------*
21:28:07:WU01:FS00:0x16:Compiler : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 15.00.30729.01 for 80x86
21:28:07:WU03:FS01:0x16:Preparing to commence simulation
21:28:07:WU02:FS02:0xa4:Folding@Home Gromacs GB Core
21:28:07:WU01:FS00:0x16:Build host: user-f6d030f24f
21:28:07:WU03:FS01:0x16:- Looking at optimizations...
21:28:07:WU02:FS02:0xa4:Version 2.27 (Dec. 15, 2010)
21:28:07:WU01:FS00:0x16:Board Type: AMD/OpenCL
21:28:07:WU03:FS01:0x16:- Files status OK
21:28:07:WU02:FS02:0xa4:
21:28:07:WU01:FS00:0x16:Core : x=16
21:28:07:WU03:FS01:0x16:sizeof(CORE_PACKET_HDR) = 512 file=<>
21:28:07:WU02:FS02:0xa4:Preparing to commence simulation
21:28:07:WU01:FS00:0x16: Window's signal control handler registered.
21:28:07:WU03:FS01:0x16:- Expanded 42495 -> 171163 (decompressed 402.7 percent)
21:28:07:WU02:FS02:0xa4:- Looking at optimizations...
21:28:07:WU01:FS00:0x16:Preparing to commence simulation
21:28:08:WU03:FS01:0x16:Called DecompressByteArray: compressed_data_size=42495 data_size=171163, decompressed_data_size=171163 diff=0
21:28:08:WU02:FS02:0xa4:- Files status OK
21:28:08:WU01:FS00:0x16:- Looking at optimizations...
21:28:08:WU03:FS01:0x16:- Digital signature verified
21:28:08:WU02:FS02:0xa4:- Expanded 2079520 -> 5386224 (decompressed 259.0 percent)
21:28:08:WU01:FS00:0x16:- Files status OK
21:28:08:WU03:FS01:0x16:
21:28:08:WU02:FS02:0xa4:Called DecompressByteArray: compressed_data_size=2079520 data_size=5386224, decompressed_data_size=5386224 diff=0
21:28:08:WU01:FS00:0x16:sizeof(CORE_PACKET_HDR) = 512 file=<>
21:28:08:WU03:FS01:0x16:Project: 11293 (Run 1, Clone 407, Gen 0)
21:28:08:WU02:FS02:0xa4:- Digital signature verified
21:28:08:WU01:FS00:0x16:- Expanded 44853 -> 171163 (decompressed 381.6 percent)
21:28:08:WU03:FS01:0x16:
21:28:08:WU02:FS02:0xa4:
21:28:08:WU01:FS00:0x16:Called DecompressByteArray: compressed_data_size=44853 data_size=171163, decompressed_data_size=171163 diff=0
21:28:08:WU03:FS01:0x16:Assembly optimizations on if available.
21:28:08:WU02:FS02:0xa4:Project: 7809 (Run 3, Clone 327, Gen 33)
21:28:08:WU01:FS00:0x16:- Digital signature verified
21:28:08:WU03:FS01:0x16:Entering M.D.
21:28:08:WU02:FS02:0xa4:
21:28:08:WU01:FS00:0x16:
21:28:08:WU02:FS02:0xa4:Assembly optimizations on if available.
21:28:08:WU01:FS00:0x16:Project: 11293 (Run 0, Clone 243, Gen 2)
21:28:08:WU02:FS02:0xa4:Entering M.D.
21:28:08:WU01:FS00:0x16:
21:28:08:WU01:FS00:0x16:Assembly optimizations on if available.
21:28:08:WU01:FS00:0x16:Entering M.D.
21:28:09:WU03:FS01:0x16:Will resume from checkpoint file 03/wudata_01.ckp
21:28:09:WU03:FS01:0x16:Tpr hash 03/wudata_01.tpr: 428829383 4171448299 3993183890 4073803069 1795910398
21:28:09:WU03:FS01:0x16:Working on ALZHEIMER DISEASE AMYLOID
21:28:09:WU03:FS01:0x16:Client config unavailable.
21:28:09:WU03:FS01:0x16:Starting GUI Server
21:28:09:WU01:FS00:0x16:Will resume from checkpoint file 01/wudata_01.ckp
21:28:09:WU01:FS00:0x16:Tpr hash 01/wudata_01.tpr: 537407621 3550239141 1616066183 4199144504 2505667728
21:28:09:WU01:FS00:0x16:Working on ALZHEIMER DISEASE AMYLOID
21:28:09:WU01:FS00:0x16:Client config unavailable.
21:28:09:WU01:FS00:0x16:Starting GUI Server
21:28:10:Server connection id=1 on 0.0.0.0:36330 from 127.0.0.1
21:28:13:WU02:FS02:0xa4:Using Gromacs checkpoints
21:28:13:WU02:FS02:0xa4:Mapping NT from 6 to 6
21:28:14:WU02:FS02:0xa4:Resuming from checkpoint
21:28:14:WU02:FS02:0xa4:Verified 02/wudata_01.log
21:28:14:WU02:FS02:0xa4:Verified 02/wudata_01.trr
21:28:14:WU02:FS02:0xa4:Verified 02/wudata_01.xtc
21:28:14:WU02:FS02:0xa4:Verified 02/wudata_01.edr
21:28:14:WU02:FS02:0xa4:Completed 775880 out of 1500000 steps (51%)
21:28:16:WU03:FS01:0x16:Resuming from checkpoint
21:28:16:WU03:FS01:0x16:fcCheckPointResume: retreived and current tpr file hash:
21:28:16:WU03:FS01:0x16: 0 428829383 428829383
21:28:16:WU03:FS01:0x16: 1 4171448299 4171448299
21:28:16:WU03:FS01:0x16: 2 3993183890 3993183890
21:28:16:WU03:FS01:0x16: 3 4073803069 4073803069
21:28:16:WU03:FS01:0x16: 4 1795910398 1795910398
21:28:16:WU03:FS01:0x16:fcCheckPointResume: file hashes same.
21:28:16:WU03:FS01:0x16:fcCheckPointResume: state restored.
21:28:16:WU03:FS01:0x16:fcCheckPointResume: name 03/wudata_01.log Verified 03/wudata_01.log
21:28:16:WU03:FS01:0x16:fcCheckPointResume: name 03/wudata_01.trr Verified 03/wudata_01.trr
21:28:16:WU03:FS01:0x16:fcCheckPointResume: name 03/wudata_01.xtc Verified 03/wudata_01.xtc
21:28:16:WU03:FS01:0x16:fcCheckPointResume: name 03/wudata_01.edr Verified 03/wudata_01.edr
21:28:16:WU03:FS01:0x16:fcCheckPointResume: state restored 2
21:28:16:WU03:FS01:0x16:Resumed from checkpoint
21:28:16:WU03:FS01:0x16:Setting checkpoint frequency: 500000
21:28:16:WU03:FS01:0x16:Completed 41500001 out of 50000000 steps (83%).
21:28:16:WU01:FS00:0x16:Resuming from checkpoint
21:28:16:WU01:FS00:0x16:fcCheckPointResume: retreived and current tpr file hash:
21:28:16:WU01:FS00:0x16: 0 537407621 537407621
21:28:16:WU01:FS00:0x16: 1 3550239141 3550239141
21:28:16:WU01:FS00:0x16: 2 1616066183 1616066183
21:28:16:WU01:FS00:0x16: 3 4199144504 4199144504
21:28:16:WU01:FS00:0x16: 4 2505667728 2505667728
21:28:16:WU01:FS00:0x16:fcCheckPointResume: file hashes same.
21:28:16:WU01:FS00:0x16:fcCheckPointResume: state restored.
21:28:16:WU01:FS00:0x16:fcCheckPointResume: name 01/wudata_01.log Verified 01/wudata_01.log
21:28:16:WU01:FS00:0x16:fcCheckPointResume: name 01/wudata_01.trr Verified 01/wudata_01.trr
21:28:16:WU01:FS00:0x16:fcCheckPointResume: name 01/wudata_01.xtc Verified 01/wudata_01.xtc
21:28:16:WU01:FS00:0x16:fcCheckPointResume: name 01/wudata_01.edr Verified 01/wudata_01.edr
21:28:17:WU01:FS00:0x16:fcCheckPointResume: state restored 2
21:28:17:WU01:FS00:0x16:Resumed from checkpoint
21:28:17:WU01:FS00:0x16:Setting checkpoint frequency: 500000
21:28:17:WU01:FS00:0x16:Completed 37500001 out of 50000000 steps (75%).
21:32:28:WU03:FS01:0x16:Completed 42000000 out of 50000000 steps (84%).
21:32:29:WU01:FS00:0x16:Completed 38000000 out of 50000000 steps (76%).
21:33:27:WU02:FS02:0xa4:Completed 780000 out of 1500000 steps (52%)
21:37:06:WU03:FS01:0x16:Completed 42500000 out of 50000000 steps (85%).
21:37:06:WU01:FS00:0x16:Completed 38500000 out of 50000000 steps (77%).
21:41:43:WU03:FS01:0x16:Completed 43000000 out of 50000000 steps (86%).
21:41:43:WU01:FS00:0x16:Completed 39000000 out of 50000000 steps (78%).
21:46:21:WU03:FS01:0x16:Completed 43500000 out of 50000000 steps (87%).
21:46:22:WU01:FS00:0x16:Completed 39500000 out of 50000000 steps (79%).
21:51:42:WU02:FS02:0xa4:Completed 795000 out of 1500000 steps (53%)
22:08:26:WU02:FS02:0xa4:Completed 810000 out of 1500000 steps (54%)
22:24:52:WU03:FS01:FahCore returned: UNKNOWN_ENUM (-1073741819 = 0xc0000005)
22:24:52:WARNING:WU03:FS01:FahCore returned an unknown error code which probably indicates that it crashed
22:24:52:WU03:FS01:Starting
22:24:52:WU03:FS01:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/D.Tillman/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/ATI/R600/Core_16.fah/FahCore_16.exe -dir 03 -suffix 01 -version 701 -checkpoint 20 -gpu 1
22:24:52:WU03:FS01:Started FahCore on PID 5624
22:24:52:WU03:FS01:Core PID:5680
22:24:52:WU03:FS01:FahCore 0x16 started
22:24:52:WU03:FS01:0x16:
22:24:52:WU03:FS01:0x16:*------------------------------*
22:24:52:WU03:FS01:0x16:Folding@Home GPU Core
22:24:52:WU03:FS01:0x16:Version 2.11 (Thu Dec 9 15:00:14 PST 2010)
22:24:52:WU03:FS01:0x16:
22:24:52:WU03:FS01:0x16:Compiler : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 15.00.30729.01 for 80x86
22:24:52:WU03:FS01:0x16:Build host: user-f6d030f24f
22:24:52:WU03:FS01:0x16:Board Type: AMD/OpenCL
22:24:52:WU03:FS01:0x16:Core : x=16
22:24:52:WU03:FS01:0x16: Window's signal control handler registered.
22:24:52:WU03:FS01:0x16:Preparing to commence simulation
22:24:52:WU03:FS01:0x16:- Ensuring status. Please wait.
22:24:53:WU01:FS00:FahCore returned: UNKNOWN_ENUM (-1073741819 = 0xc0000005)
22:24:53:WARNING:WU01:FS00:FahCore returned an unknown error code which probably indicates that it crashed
22:24:53:WU01:FS00:Starting
22:24:53:WU01:FS00:Running FahCore: "C:\Program Files (x86)\FAHClient/FAHCoreWrapper.exe" C:/Users/D.Tillman/AppData/Roaming/FAHClient/cores/www.stanford.edu/~pande/Win32/AMD64/ATI/R600/Core_16.fah/FahCore_16.exe -dir 01 -suffix 01 -version 701 -checkpoint 20 -gpu 0
22:24:53:WU01:FS00:Started FahCore on PID 4040
22:24:53:WU01:FS00:Core PID:5512
22:24:53:WU01:FS00:FahCore 0x16 started
22:24:53:WU01:FS00:0x16:
22:24:53:WU01:FS00:0x16:*------------------------------*
22:24:53:WU01:FS00:0x16:Folding@Home GPU Core
22:24:53:WU01:FS00:0x16:Version 2.11 (Thu Dec 9 15:00:14 PST 2010)
22:24:53:WU01:FS00:0x16:
22:24:53:WU01:FS00:0x16:Compiler : Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 15.00.30729.01 for 80x86
22:24:53:WU01:FS00:0x16:Build host: user-f6d030f24f
22:24:53:WU01:FS00:0x16:Board Type: AMD/OpenCL
22:24:53:WU01:FS00:0x16:Core : x=16
22:24:53:WU01:FS00:0x16: Window's signal control handler registered.
22:24:53:WU01:FS00:0x16:Preparing to commence simulation
22:24:53:WU01:FS00:0x16:- Ensuring status. Please wait.
22:25:01:WU03:FS01:0x16:- Looking at optimizations...
22:25:01:WU03:FS01:0x16:- Working with standard loops on this execution.
22:25:01:WU03:FS01:0x16:- Previous termination of core was improper.
22:25:01:WU03:FS01:0x16:- Files status OK
22:25:01:WU03:FS01:0x16:sizeof(CORE_PACKET_HDR) = 512 file=<>
22:25:01:WU03:FS01:0x16:- Expanded 42495 -> 171163 (decompressed 402.7 percent)
22:25:01:WU03:FS01:0x16:Called DecompressByteArray: compressed_data_size=42495 data_size=171163, decompressed_data_size=171163 diff=0
22:25:01:WU03:FS01:0x16:- Digital signature verified
22:25:01:WU03:FS01:0x16:
22:25:01:WU03:FS01:0x16:Project: 11293 (Run 1, Clone 407, Gen 0)
22:25:02:WU03:FS01:0x16:
22:25:02:WU03:FS01:0x16:Entering M.D.
22:25:02:WU01:FS00:0x16:- Looking at optimizations...
22:25:02:WU01:FS00:0x16:- Working with standard loops on this execution.
22:25:02:WU01:FS00:0x16:- Previous termination of core was improper.
22:25:02:WU01:FS00:0x16:- Files status OK
22:25:02:WU01:FS00:0x16:sizeof(CORE_PACKET_HDR) = 512 file=<>
22:25:02:WU01:FS00:0x16:- Expanded 44853 -> 171163 (decompressed 381.6 percent)
22:25:02:WU01:FS00:0x16:Called DecompressByteArray: compressed_data_size=44853 data_size=171163, decompressed_data_size=171163 diff=0
22:25:02:WU01:FS00:0x16:- Digital signature verified
22:25:02:WU01:FS00:0x16:
22:25:02:WU01:FS00:0x16:Project: 11293 (Run 0, Clone 243, Gen 2)
22:25:02:WU01:FS00:0x16:
22:25:02:WU01:FS00:0x16:Entering M.D.
22:25:04:WU03:FS01:0x16:Will resume from checkpoint file 03/wudata_01.ckp
22:25:04:WU03:FS01:0x16:Tpr hash 03/wudata_01.tpr: 428829383 4171448299 3993183890 4073803069 1795910398
22:25:04:WU03:FS01:0x16:Working on ALZHEIMER DISEASE AMYLOID
22:25:04:WU03:FS01:0x16:Client config unavailable.
22:25:04:WU03:FS01:0x16:Starting GUI Server
22:25:04:WU01:FS00:0x16:Will resume from checkpoint file 01/wudata_01.ckp
22:25:04:WU01:FS00:0x16:Tpr hash 01/wudata_01.tpr: 537407621 3550239141 1616066183 4199144504 2505667728
22:25:04:WU01:FS00:0x16:Working on ALZHEIMER DISEASE AMYLOID
22:25:04:WU01:FS00:0x16:Client config unavailable.
22:25:05:WU01:FS00:0x16:Starting GUI Server
22:25:10:WU02:FS02:0xa4:Completed 825000 out of 1500000 steps (55%)
22:25:16:WU03:FS01:0x16:Resuming from checkpoint
22:25:16:WU03:FS01:0x16:fcCheckPointResume: retreived and current tpr file hash:
22:25:16:WU03:FS01:0x16: 0 428829383 428829383
22:25:16:WU03:FS01:0x16: 1 4171448299 4171448299
22:25:16:WU03:FS01:0x16: 2 3993183890 3993183890
22:25:16:WU03:FS01:0x16: 3 4073803069 4073803069
22:25:16:WU03:FS01:0x16: 4 1795910398 1795910398
22:25:16:WU03:FS01:0x16:fcCheckPointResume: file hashes same.
22:25:16:WU03:FS01:0x16:fcCheckPointResume: state restored.
22:25:16:WU03:FS01:0x16:fcCheckPointResume: name 03/wudata_01.log Verified 03/wudata_01.log
22:25:16:WU03:FS01:0x16:fcCheckPointResume: name 03/wudata_01.trr Verified 03/wudata_01.trr
22:25:16:WU03:FS01:0x16:fcCheckPointResume: name 03/wudata_01.xtc Verified 03/wudata_01.xtc
22:25:16:WU03:FS01:0x16:fcCheckPointResume: name 03/wudata_01.edr Verified 03/wudata_01.edr
22:25:16:WU03:FS01:0x16:fcCheckPointResume: state restored 2
22:25:16:WU03:FS01:0x16:Resumed from checkpoint
22:25:16:WU03:FS01:0x16:Setting checkpoint frequency: 500000
22:25:16:WU03:FS01:0x16:Completed 43000001 out of 50000000 steps (86%).
22:25:19:WU01:FS00:0x16:Resuming from checkpoint
22:25:19:WU01:FS00:0x16:fcCheckPointResume: retreived and current tpr file hash:
22:25:19:WU01:FS00:0x16: 0 537407621 537407621
22:25:19:WU01:FS00:0x16: 1 3550239141 3550239141
22:25:19:WU01:FS00:0x16: 2 1616066183 1616066183
22:25:19:WU01:FS00:0x16: 3 4199144504 4199144504
22:25:19:WU01:FS00:0x16: 4 2505667728 2505667728
22:25:19:WU01:FS00:0x16:fcCheckPointResume: file hashes same.
22:25:19:WU01:FS00:0x16:fcCheckPointResume: state restored.
22:25:19:WU01:FS00:0x16:fcCheckPointResume: name 01/wudata_01.log Verified 01/wudata_01.log
22:25:19:WU01:FS00:0x16:fcCheckPointResume: name 01/wudata_01.trr Verified 01/wudata_01.trr
22:25:19:WU01:FS00:0x16:fcCheckPointResume: name 01/wudata_01.xtc Verified 01/wudata_01.xtc
22:25:19:WU01:FS00:0x16:fcCheckPointResume: name 01/wudata_01.edr Verified 01/wudata_01.edr
22:25:19:WU01:FS00:0x16:fcCheckPointResume: state restored 2
22:25:19:WU01:FS00:0x16:Resumed from checkpoint
22:25:19:WU01:FS00:0x16:Setting checkpoint frequency: 500000
22:25:19:WU01:FS00:0x16:Completed 39000001 out of 50000000 steps (78%).
22:29:27:WU01:FS00:0x16:Completed 39500000 out of 50000000 steps (79%).
22:29:33:WU03:FS01:0x16:Completed 43500000 out of 50000000 steps (87%).
22:33:57:WU01:FS00:0x16:Completed 40000000 out of 50000000 steps (80%).
22:34:07:WU03:FS01:0x16:Completed 44000000 out of 50000000 steps (88%).