FahCore returned: INTERRUPTED (102 = 0x66)

[TomJohnson]

Any one experiencing this problem:

Code: Select all

*********************** Log Started 2014-04-18T05:57:19Z ***********************
05:57:19:************************* Folding@home Client *************************
05:57:19:    Website: http://folding.stanford.edu/
05:57:19:  Copyright: (c) 2009-2014 Stanford University
05:57:19:     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
05:57:19:       Args: --child --lifeline 48609 --respawn
05:57:19:     Config: /Library/Application Support/FAHClient/config.xml
05:57:19:******************************** Build ********************************
05:57:19:    Version: 7.4.4
05:57:19:       Date: Mar 4 2014
05:57:19:       Time: 20:27:54
05:57:19:    SVN Rev: 4130
05:57:19:     Branch: fah/trunk/client
05:57:19:   Compiler: GNU 4.2.1 (Apple Inc. build 5666) (dot 3)
05:57:19:    Options: -std=gnu++98 -O3 -funroll-loops -mfpmath=sse -ffast-math
05:57:19:             -fno-unsafe-math-optimizations -msse3 -arch x86_64
05:57:19:             -mmacosx-version-min=10.6
05:57:19:   Platform: darwin 10.8.0
05:57:19:       Bits: 64
05:57:19:       Mode: Release
05:57:19:******************************* System ********************************
05:57:19:        CPU: Intel(R) Core(TM) i7-4771 CPU @ 3.50GHz
05:57:19:     CPU ID: GenuineIntel Family 6 Model 60 Stepping 3
05:57:19:       CPUs: 8
05:57:19:     Memory: 24.00GiB
05:57:19:Free Memory: 17.10GiB
05:57:19:    Threads: POSIX_THREADS
05:57:19: OS Version: 10.9
05:57:19:Has Battery: false
05:57:19: On Battery: false
05:57:19: UTC Offset: -7
05:57:19:        PID: 48646
05:57:19:        CWD: /Library/Application Support/FAHClient
05:57:19:         OS: Darwin 13.1.0 x86_64
05:57:19:    OS Arch: AMD64
05:57:19:       GPUs: 1
05:57:19:      GPU 0: NVIDIA:3 GK104 [GeForce GTX 780M]
05:57:19:       CUDA: Not detected
05:57:19:***********************************************************************
05:57:19:<config>
05:57:19:  <!-- Folding Core -->
05:57:19:  <checkpoint v='30'/>
05:57:19:
05:57:19:  <!-- Folding Slot Configuration -->
05:57:19:  <cause v='CANCER'/>
05:57:19:
05:57:19:  <!-- Network -->
05:57:19:  <proxy v=':8080'/>
05:57:19:
05:57:19:  <!-- Slot Control -->
05:57:19:  <pause-on-battery v='false'/>
05:57:19:  <power v='full'/>
05:57:19:
05:57:19:  <!-- User Information -->
05:57:19:  <passkey v='********************************'/>
05:57:19:  <team v='1971'/>
05:57:19:  <user v='Tom_Johnson'/>
05:57:19:
05:57:19:  <!-- Folding Slots -->
05:57:19:  <slot id='0' type='CPU'/>
05:57:19:</config>
05:57:19:Trying to access database...
05:57:19:Successfully acquired database lock
05:57:19:Enabled folding slot 00: READY cpu:8
05:57:19:WU01:FS00:Starting
05:57:19:WU01:FS00:Removing old file './work/01/logfile_01-20140418-052450.txt'
05:57:19:WU01:FS00:Running FahCore: /usr/bin/FAHCoreWrapper "/Library/Application Support/FAHClient/cores/www.stanford.edu/~pande/OSX/AMD64/Core_a4.fah/FahCore_a4" -dir 01 -suffix 01 -version 704 -lifeline 48646 -checkpoint 30 -np 8
05:57:19:WU01:FS00:Started FahCore on PID 48654
05:57:20:WU01:FS00:Core PID:48666
05:57:20:WU01:FS00:FahCore 0xa4 started
05:57:20:WU01:FS00:0xa4:
05:57:20:WU01:FS00:0xa4:*------------------------------*
05:57:20:WU01:FS00:0xa4:Folding@Home Gromacs Core
05:57:20:WU01:FS00:0xa4:Version 2.27 (Thu Feb 10 09:46:40 PST 2011)
05:57:20:WU01:FS00:0xa4:
05:57:20:WU01:FS00:0xa4:Preparing to commence simulation
05:57:20:WU01:FS00:0xa4:- Ensuring status. Please wait.
05:57:29:WU01:FS00:0xa4:- Looking at optimizations...
05:57:29:WU01:FS00:0xa4:- Working with standard loops on this execution.
05:57:29:WU01:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
05:57:29:WU01:FS00:0xa4:- Expanded 476208 -> 1434864 (decompressed 301.3 percent)
05:57:29:WU01:FS00:0xa4:Called DecompressByteArray: compressed_data_size=476208 data_size=1434864, decompressed_data_size=1434864 diff=0
05:57:29:WU01:FS00:0xa4:- Digital signature verified
05:57:29:WU01:FS00:0xa4:
05:57:29:WU01:FS00:0xa4:Project: 9006 (Run 811, Clone 3, Gen 0)
05:57:29:WU01:FS00:0xa4:
05:57:29:WU01:FS00:0xa4:Entering M.D.
05:57:35:WU01:FS00:0xa4:Mapping NT from 8 to 8 
05:57:36:WU01:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
05:57:36:WU01:FS00:Starting
05:57:36:WU01:FS00:Removing old file './work/01/logfile_01-20140418-052551.txt'
05:57:36:WU01:FS00:Running FahCore: /usr/bin/FAHCoreWrapper "/Library/Application Support/FAHClient/cores/www.stanford.edu/~pande/OSX/AMD64/Core_a4.fah/FahCore_a4" -dir 01 -suffix 01 -version 704 -lifeline 48646 -checkpoint 30 -np 8
05:57:36:WU01:FS00:Started FahCore on PID 48721
05:57:36:WU01:FS00:Core PID:48722
05:57:36:WU01:FS00:FahCore 0xa4 started
05:57:37:WU01:FS00:0xa4:
05:57:37:WU01:FS00:0xa4:*------------------------------*
05:57:37:WU01:FS00:0xa4:Folding@Home Gromacs Core
05:57:37:WU01:FS00:0xa4:Version 2.27 (Thu Feb 10 09:46:40 PST 2011)
05:57:37:WU01:FS00:0xa4:
05:57:37:WU01:FS00:0xa4:Preparing to commence simulation
05:57:37:WU01:FS00:0xa4:- Ensuring status. Please wait.
05:57:46:WU01:FS00:0xa4:- Looking at optimizations...
05:57:46:WU01:FS00:0xa4:- Working with standard loops on this execution.
05:57:46:WU01:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
05:57:46:WU01:FS00:0xa4:- Expanded 476208 -> 1434864 (decompressed 301.3 percent)
05:57:46:WU01:FS00:0xa4:Called DecompressByteArray: compressed_data_size=476208 data_size=1434864, decompressed_data_size=1434864 diff=0
05:57:46:WU01:FS00:0xa4:- Digital signature verified
05:57:46:WU01:FS00:0xa4:
05:57:46:WU01:FS00:0xa4:Project: 9006 (Run 811, Clone 3, Gen 0)
05:57:46:WU01:FS00:0xa4:
05:57:46:WU01:FS00:0xa4:Entering M.D.
05:57:52:WU01:FS00:0xa4:Mapping NT from 8 to 8 
05:57:52:WU01:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
05:58:36:WU01:FS00:Starting
05:58:36:WU01:FS00:Removing old file './work/01/logfile_01-20140418-052651.txt'
05:58:36:WU01:FS00:Running FahCore: /usr/bin/FAHCoreWrapper "/Library/Application Support/FAHClient/cores/www.stanford.edu/~pande/OSX/AMD64/Core_a4.fah/FahCore_a4" -dir 01 -suffix 01 -version 704 -lifeline 48646 -checkpoint 30 -np 8
05:58:36:WU01:FS00:Started FahCore on PID 48730
05:58:36:WU01:FS00:Core PID:48731
05:58:36:WU01:FS00:FahCore 0xa4 started
05:58:37:WU01:FS00:0xa4:
05:58:37:WU01:FS00:0xa4:*------------------------------*
05:58:37:WU01:FS00:0xa4:Folding@Home Gromacs Core
05:58:37:WU01:FS00:0xa4:Version 2.27 (Thu Feb 10 09:46:40 PST 2011)
05:58:37:WU01:FS00:0xa4:
05:58:37:WU01:FS00:0xa4:Preparing to commence simulation
05:58:37:WU01:FS00:0xa4:- Ensuring status. Please wait.
05:58:46:WU01:FS00:0xa4:- Looking at optimizations...
05:58:46:WU01:FS00:0xa4:- Working with standard loops on this execution.
05:58:46:WU01:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
05:58:46:WU01:FS00:0xa4:- Expanded 476208 -> 1434864 (decompressed 301.3 percent)
05:58:46:WU01:FS00:0xa4:Called DecompressByteArray: compressed_data_size=476208 data_size=1434864, decompressed_data_size=1434864 diff=0
05:58:46:WU01:FS00:0xa4:- Digital signature verified
05:58:46:WU01:FS00:0xa4:
05:58:46:WU01:FS00:0xa4:Project: 9006 (Run 811, Clone 3, Gen 0)
05:58:46:WU01:FS00:0xa4:
05:58:46:WU01:FS00:0xa4:Entering M.D.
05:58:52:WU01:FS00:0xa4:Mapping NT from 8 to 8 
05:58:52:WU01:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
05:59:36:WU01:FS00:Starting
05:59:36:WU01:FS00:Removing old file './work/01/logfile_01-20140418-052751.txt'
05:59:36:WU01:FS00:Running FahCore: /usr/bin/FAHCoreWrapper "/Library/Application Support/FAHClient/cores/www.stanford.edu/~pande/OSX/AMD64/Core_a4.fah/FahCore_a4" -dir 01 -suffix 01 -version 704 -lifeline 48646 -checkpoint 30 -np 8
05:59:36:WU01:FS00:Started FahCore on PID 48737
05:59:36:WU01:FS00:Core PID:48738
05:59:36:WU01:FS00:FahCore 0xa4 started
05:59:37:WU01:FS00:0xa4:
05:59:37:WU01:FS00:0xa4:*------------------------------*
05:59:37:WU01:FS00:0xa4:Folding@Home Gromacs Core
05:59:37:WU01:FS00:0xa4:Version 2.27 (Thu Feb 10 09:46:40 PST 2011)
05:59:37:WU01:FS00:0xa4:
05:59:37:WU01:FS00:0xa4:Preparing to commence simulation
05:59:37:WU01:FS00:0xa4:- Ensuring status. Please wait.
05:59:46:WU01:FS00:0xa4:- Looking at optimizations...
05:59:46:WU01:FS00:0xa4:- Working with standard loops on this execution.
05:59:46:WU01:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
05:59:46:WU01:FS00:0xa4:- Expanded 476208 -> 1434864 (decompressed 301.3 percent)
05:59:46:WU01:FS00:0xa4:Called DecompressByteArray: compressed_data_size=476208 data_size=1434864, decompressed_data_size=1434864 diff=0
05:59:46:WU01:FS00:0xa4:- Digital signature verified
05:59:46:WU01:FS00:0xa4:
05:59:46:WU01:FS00:0xa4:Project: 9006 (Run 811, Clone 3, Gen 0)
05:59:46:WU01:FS00:0xa4:
05:59:46:WU01:FS00:0xa4:Entering M.D.
05:59:52:WU01:FS00:0xa4:Mapping NT from 8 to 8 
05:59:53:WU01:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
06:00:37:WU01:FS00:Starting
06:00:37:WU01:FS00:Removing old file './work/01/logfile_01-20140418-052851.txt'
06:00:37:WU01:FS00:Running FahCore: /usr/bin/FAHCoreWrapper "/Library/Application Support/FAHClient/cores/www.stanford.edu/~pande/OSX/AMD64/Core_a4.fah/FahCore_a4" -dir 01 -suffix 01 -version 704 -lifeline 48646 -checkpoint 30 -np 8
06:00:37:WU01:FS00:Started FahCore on PID 48754
06:00:37:WU01:FS00:Core PID:48755
06:00:37:WU01:FS00:FahCore 0xa4 started
06:00:37:WU01:FS00:0xa4:
06:00:37:WU01:FS00:0xa4:*------------------------------*
06:00:37:WU01:FS00:0xa4:Folding@Home Gromacs Core
06:00:37:WU01:FS00:0xa4:Version 2.27 (Thu Feb 10 09:46:40 PST 2011)
06:00:37:WU01:FS00:0xa4:
06:00:37:WU01:FS00:0xa4:Preparing to commence simulation
06:00:37:WU01:FS00:0xa4:- Ensuring status. Please wait.
06:00:46:WU01:FS00:0xa4:- Looking at optimizations...
06:00:46:WU01:FS00:0xa4:- Working with standard loops on this execution.
06:00:46:WU01:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
06:00:46:WU01:FS00:0xa4:- Expanded 476208 -> 1434864 (decompressed 301.3 percent)
06:00:46:WU01:FS00:0xa4:Called DecompressByteArray: compressed_data_size=476208 data_size=1434864, decompressed_data_size=1434864 diff=0
06:00:46:WU01:FS00:0xa4:- Digital signature verified
06:00:46:WU01:FS00:0xa4:
06:00:46:WU01:FS00:0xa4:Project: 9006 (Run 811, Clone 3, Gen 0)
06:00:46:WU01:FS00:0xa4:
06:00:46:WU01:FS00:0xa4:Entering M.D.
06:00:52:WU01:FS00:0xa4:Mapping NT from 8 to 8 
06:00:53:WU01:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
06:01:37:WU01:FS00:Starting
06:01:37:WU01:FS00:Removing old file './work/01/logfile_01-20140418-052951.txt'
06:01:37:WU01:FS00:Running FahCore: /usr/bin/FAHCoreWrapper "/Library/Application Support/FAHClient/cores/www.stanford.edu/~pande/OSX/AMD64/Core_a4.fah/FahCore_a4" -dir 01 -suffix 01 -version 704 -lifeline 48646 -checkpoint 30 -np 8
06:01:37:WU01:FS00:Started FahCore on PID 48771
06:01:37:WU01:FS00:Core PID:48772
06:01:37:WU01:FS00:FahCore 0xa4 started
06:01:37:WU01:FS00:0xa4:
06:01:37:WU01:FS00:0xa4:*------------------------------*
06:01:37:WU01:FS00:0xa4:Folding@Home Gromacs Core
06:01:37:WU01:FS00:0xa4:Version 2.27 (Thu Feb 10 09:46:40 PST 2011)
06:01:37:WU01:FS00:0xa4:
06:01:37:WU01:FS00:0xa4:Preparing to commence simulation
06:01:37:WU01:FS00:0xa4:- Ensuring status. Please wait.
06:01:46:WU01:FS00:0xa4:- Looking at optimizations...
06:01:46:WU01:FS00:0xa4:- Working with standard loops on this execution.
06:01:46:WU01:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
06:01:47:WU01:FS00:0xa4:- Expanded 476208 -> 1434864 (decompressed 301.3 percent)
06:01:47:WU01:FS00:0xa4:Called DecompressByteArray: compressed_data_size=476208 data_size=1434864, decompressed_data_size=1434864 diff=0
06:01:47:WU01:FS00:0xa4:- Digital signature verified
06:01:47:WU01:FS00:0xa4:
06:01:47:WU01:FS00:0xa4:Project: 9006 (Run 811, Clone 3, Gen 0)
06:01:47:WU01:FS00:0xa4:
06:01:47:WU01:FS00:0xa4:Entering M.D.
06:01:53:WU01:FS00:0xa4:Mapping NT from 8 to 8 
06:01:53:WU01:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
06:02:37:WU01:FS00:Starting
06:02:37:WU01:FS00:Removing old file './work/01/logfile_01-20140418-053051.txt'
06:02:37:WU01:FS00:Running FahCore: /usr/bin/FAHCoreWrapper "/Library/Application Support/FAHClient/cores/www.stanford.edu/~pande/OSX/AMD64/Core_a4.fah/FahCore_a4" -dir 01 -suffix 01 -version 704 -lifeline 48646 -checkpoint 30 -np 8
06:02:37:WU01:FS00:Started FahCore on PID 48778
06:02:37:WU01:FS00:Core PID:48779
06:02:37:WU01:FS00:FahCore 0xa4 started
06:02:37:WU01:FS00:0xa4:
06:02:37:WU01:FS00:0xa4:*------------------------------*
06:02:37:WU01:FS00:0xa4:Folding@Home Gromacs Core
06:02:37:WU01:FS00:0xa4:Version 2.27 (Thu Feb 10 09:46:40 PST 2011)
06:02:37:WU01:FS00:0xa4:
06:02:37:WU01:FS00:0xa4:Preparing to commence simulation
06:02:37:WU01:FS00:0xa4:- Ensuring status. Please wait.
06:02:46:WU01:FS00:0xa4:- Looking at optimizations...
06:02:46:WU01:FS00:0xa4:- Working with standard loops on this execution.
06:02:46:WU01:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
06:02:46:WU01:FS00:0xa4:- Expanded 476208 -> 1434864 (decompressed 301.3 percent)
06:02:46:WU01:FS00:0xa4:Called DecompressByteArray: compressed_data_size=476208 data_size=1434864, decompressed_data_size=1434864 diff=0
06:02:47:WU01:FS00:0xa4:- Digital signature verified
06:02:47:WU01:FS00:0xa4:
06:02:47:WU01:FS00:0xa4:Project: 9006 (Run 811, Clone 3, Gen 0)
06:02:47:WU01:FS00:0xa4:
06:02:47:WU01:FS00:0xa4:Entering M.D.
06:02:53:WU01:FS00:0xa4:Mapping NT from 8 to 8 
06:02:53:WU01:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
06:03:37:WU01:FS00:Starting
06:03:37:WU01:FS00:Removing old file './work/01/logfile_01-20140418-053151.txt'
06:03:37:WU01:FS00:Running FahCore: /usr/bin/FAHCoreWrapper "/Library/Application Support/FAHClient/cores/www.stanford.edu/~pande/OSX/AMD64/Core_a4.fah/FahCore_a4" -dir 01 -suffix 01 -version 704 -lifeline 48646 -checkpoint 30 -np 8
06:03:37:WU01:FS00:Started FahCore on PID 48784
06:03:37:WU01:FS00:Core PID:48785
06:03:37:WU01:FS00:FahCore 0xa4 started
06:03:37:WU01:FS00:0xa4:
06:03:37:WU01:FS00:0xa4:*------------------------------*
06:03:37:WU01:FS00:0xa4:Folding@Home Gromacs Core
06:03:37:WU01:FS00:0xa4:Version 2.27 (Thu Feb 10 09:46:40 PST 2011)
06:03:37:WU01:FS00:0xa4:
06:03:37:WU01:FS00:0xa4:Preparing to commence simulation
06:03:37:WU01:FS00:0xa4:- Ensuring status. Please wait.
06:03:46:WU01:FS00:0xa4:- Looking at optimizations...
06:03:46:WU01:FS00:0xa4:- Working with standard loops on this execution.
06:03:46:WU01:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
06:03:46:WU01:FS00:0xa4:- Expanded 476208 -> 1434864 (decompressed 301.3 percent)
06:03:46:WU01:FS00:0xa4:Called DecompressByteArray: compressed_data_size=476208 data_size=1434864, decompressed_data_size=1434864 diff=0
06:03:47:WU01:FS00:0xa4:- Digital signature verified
06:03:47:WU01:FS00:0xa4:
06:03:47:WU01:FS00:0xa4:Project: 9006 (Run 811, Clone 3, Gen 0)
06:03:47:WU01:FS00:0xa4:
06:03:47:WU01:FS00:0xa4:Entering M.D.
06:03:53:WU01:FS00:0xa4:Mapping NT from 8 to 8 
06:03:53:WU01:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
06:04:37:WU01:FS00:Starting
06:04:37:WU01:FS00:Removing old file './work/01/logfile_01-20140418-053252.txt'
06:04:37:WU01:FS00:Running FahCore: /usr/bin/FAHCoreWrapper "/Library/Application Support/FAHClient/cores/www.stanford.edu/~pande/OSX/AMD64/Core_a4.fah/FahCore_a4" -dir 01 -suffix 01 -version 704 -lifeline 48646 -checkpoint 30 -np 8
06:04:37:WU01:FS00:Started FahCore on PID 48789
06:04:37:WU01:FS00:Core PID:48790
06:04:37:WU01:FS00:FahCore 0xa4 started
06:04:37:WU01:FS00:0xa4:
06:04:37:WU01:FS00:0xa4:*------------------------------*
06:04:37:WU01:FS00:0xa4:Folding@Home Gromacs Core
06:04:37:WU01:FS00:0xa4:Version 2.27 (Thu Feb 10 09:46:40 PST 2011)
06:04:37:WU01:FS00:0xa4:
06:04:37:WU01:FS00:0xa4:Preparing to commence simulation
06:04:37:WU01:FS00:0xa4:- Ensuring status. Please wait.
06:04:46:WU01:FS00:0xa4:- Looking at optimizations...
06:04:46:WU01:FS00:0xa4:- Working with standard loops on this execution.
06:04:46:WU01:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
06:04:46:WU01:FS00:0xa4:- Expanded 476208 -> 1434864 (decompressed 301.3 percent)
06:04:46:WU01:FS00:0xa4:Called DecompressByteArray: compressed_data_size=476208 data_size=1434864, decompressed_data_size=1434864 diff=0
06:04:47:WU01:FS00:0xa4:- Digital signature verified
06:04:47:WU01:FS00:0xa4:
06:04:47:WU01:FS00:0xa4:Project: 9006 (Run 811, Clone 3, Gen 0)
06:04:47:WU01:FS00:0xa4:
06:04:47:WU01:FS00:0xa4:Entering M.D.
06:04:53:WU01:FS00:0xa4:Mapping NT from 8 to 8 
06:04:53:WU01:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
06:05:37:WU01:FS00:Starting
06:05:37:WU01:FS00:Removing old file './work/01/logfile_01-20140418-053352.txt'
06:05:37:WU01:FS00:Running FahCore: /usr/bin/FAHCoreWrapper "/Library/Application Support/FAHClient/cores/www.stanford.edu/~pande/OSX/AMD64/Core_a4.fah/FahCore_a4" -dir 01 -suffix 01 -version 704 -lifeline 48646 -checkpoint 30 -np 8
06:05:37:WU01:FS00:Started FahCore on PID 48795
06:05:37:WU01:FS00:Core PID:48796
06:05:37:WU01:FS00:FahCore 0xa4 started
06:05:37:WU01:FS00:0xa4:
06:05:37:WU01:FS00:0xa4:*------------------------------*
06:05:37:WU01:FS00:0xa4:Folding@Home Gromacs Core
06:05:37:WU01:FS00:0xa4:Version 2.27 (Thu Feb 10 09:46:40 PST 2011)
06:05:37:WU01:FS00:0xa4:
06:05:37:WU01:FS00:0xa4:Preparing to commence simulation
06:05:37:WU01:FS00:0xa4:- Ensuring status. Please wait.
06:05:47:WU01:FS00:0xa4:- Looking at optimizations...
06:05:47:WU01:FS00:0xa4:- Working with standard loops on this execution.
06:05:47:WU01:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
06:05:47:WU01:FS00:0xa4:- Expanded 476208 -> 1434864 (decompressed 301.3 percent)
06:05:47:WU01:FS00:0xa4:Called DecompressByteArray: compressed_data_size=476208 data_size=1434864, decompressed_data_size=1434864 diff=0
06:05:47:WU01:FS00:0xa4:- Digital signature verified
06:05:47:WU01:FS00:0xa4:
06:05:47:WU01:FS00:0xa4:Project: 9006 (Run 811, Clone 3, Gen 0)
06:05:47:WU01:FS00:0xa4:
06:05:47:WU01:FS00:0xa4:Entering M.D.
06:05:53:WU01:FS00:0xa4:Mapping NT from 8 to 8 
06:05:54:WU01:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
06:06:37:WU01:FS00:Starting
06:06:37:WU01:FS00:Removing old file './work/01/logfile_01-20140418-053452.txt'
06:06:37:WU01:FS00:Running FahCore: /usr/bin/FAHCoreWrapper "/Library/Application Support/FAHClient/cores/www.stanford.edu/~pande/OSX/AMD64/Core_a4.fah/FahCore_a4" -dir 01 -suffix 01 -version 704 -lifeline 48646 -checkpoint 30 -np 8
06:06:37:WU01:FS00:Started FahCore on PID 48800
06:06:37:WU01:FS00:Core PID:48801
06:06:37:WU01:FS00:FahCore 0xa4 started
06:06:38:WU01:FS00:0xa4:
06:06:38:WU01:FS00:0xa4:*------------------------------*
06:06:38:WU01:FS00:0xa4:Folding@Home Gromacs Core
06:06:38:WU01:FS00:0xa4:Version 2.27 (Thu Feb 10 09:46:40 PST 2011)
06:06:38:WU01:FS00:0xa4:
06:06:38:WU01:FS00:0xa4:Preparing to commence simulation
06:06:38:WU01:FS00:0xa4:- Ensuring status. Please wait.
06:06:47:WU01:FS00:0xa4:- Looking at optimizations...
06:06:47:WU01:FS00:0xa4:- Working with standard loops on this execution.
06:06:47:WU01:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
06:06:47:WU01:FS00:0xa4:- Expanded 476208 -> 1434864 (decompressed 301.3 percent)
06:06:47:WU01:FS00:0xa4:Called DecompressByteArray: compressed_data_size=476208 data_size=1434864, decompressed_data_size=1434864 diff=0
06:06:47:WU01:FS00:0xa4:- Digital signature verified
06:06:47:WU01:FS00:0xa4:
06:06:47:WU01:FS00:0xa4:Project: 9006 (Run 811, Clone 3, Gen 0)
06:06:47:WU01:FS00:0xa4:
06:06:47:WU01:FS00:0xa4:Entering M.D.
06:06:53:WU01:FS00:0xa4:Mapping NT from 8 to 8 
06:06:54:WU01:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
06:07:37:WU01:FS00:Starting
06:07:37:WU01:FS00:Removing old file './work/01/logfile_01-20140418-053552.txt'
06:07:37:WU01:FS00:Running FahCore: /usr/bin/FAHCoreWrapper "/Library/Application Support/FAHClient/cores/www.stanford.edu/~pande/OSX/AMD64/Core_a4.fah/FahCore_a4" -dir 01 -suffix 01 -version 704 -lifeline 48646 -checkpoint 30 -np 8
06:07:37:WU01:FS00:Started FahCore on PID 48805
06:07:37:WU01:FS00:Core PID:48806
06:07:37:WU01:FS00:FahCore 0xa4 started
06:07:38:WU01:FS00:0xa4:
06:07:38:WU01:FS00:0xa4:*------------------------------*
06:07:38:WU01:FS00:0xa4:Folding@Home Gromacs Core
06:07:38:WU01:FS00:0xa4:Version 2.27 (Thu Feb 10 09:46:40 PST 2011)
06:07:38:WU01:FS00:0xa4:
06:07:38:WU01:FS00:0xa4:Preparing to commence simulation
06:07:38:WU01:FS00:0xa4:- Ensuring status. Please wait.
06:07:47:WU01:FS00:0xa4:- Looking at optimizations...
06:07:47:WU01:FS00:0xa4:- Working with standard loops on this execution.
06:07:47:WU01:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
06:07:47:WU01:FS00:0xa4:- Expanded 476208 -> 1434864 (decompressed 301.3 percent)
06:07:47:WU01:FS00:0xa4:Called DecompressByteArray: compressed_data_size=476208 data_size=1434864, decompressed_data_size=1434864 diff=0
06:07:47:WU01:FS00:0xa4:- Digital signature verified
06:07:47:WU01:FS00:0xa4:
06:07:47:WU01:FS00:0xa4:Project: 9006 (Run 811, Clone 3, Gen 0)
06:07:47:WU01:FS00:0xa4:
06:07:47:WU01:FS00:0xa4:Entering M.D.
06:07:53:WU01:FS00:0xa4:Mapping NT from 8 to 8 
06:07:54:WU01:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
06:08:37:WU01:FS00:Starting
06:08:37:WU01:FS00:Removing old file './work/01/logfile_01-20140418-053652.txt'
06:08:37:WU01:FS00:Running FahCore: /usr/bin/FAHCoreWrapper "/Library/Application Support/FAHClient/cores/www.stanford.edu/~pande/OSX/AMD64/Core_a4.fah/FahCore_a4" -dir 01 -suffix 01 -version 704 -lifeline 48646 -checkpoint 30 -np 8
06:08:37:WU01:FS00:Started FahCore on PID 48811
06:08:37:WU01:FS00:Core PID:48812
06:08:37:WU01:FS00:FahCore 0xa4 started
06:08:38:WU01:FS00:0xa4:
06:08:38:WU01:FS00:0xa4:*------------------------------*
06:08:38:WU01:FS00:0xa4:Folding@Home Gromacs Core
06:08:38:WU01:FS00:0xa4:Version 2.27 (Thu Feb 10 09:46:40 PST 2011)
06:08:38:WU01:FS00:0xa4:
06:08:38:WU01:FS00:0xa4:Preparing to commence simulation
06:08:38:WU01:FS00:0xa4:- Ensuring status. Please wait.
06:08:47:WU01:FS00:0xa4:- Looking at optimizations...
06:08:47:WU01:FS00:0xa4:- Working with standard loops on this execution.
06:08:47:WU01:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
06:08:47:WU01:FS00:0xa4:- Expanded 476208 -> 1434864 (decompressed 301.3 percent)
06:08:47:WU01:FS00:0xa4:Called DecompressByteArray: compressed_data_size=476208 data_size=1434864, decompressed_data_size=1434864 diff=0
06:08:47:WU01:FS00:0xa4:- Digital signature verified
06:08:47:WU01:FS00:0xa4:
06:08:47:WU01:FS00:0xa4:Project: 9006 (Run 811, Clone 3, Gen 0)
06:08:47:WU01:FS00:0xa4:
06:08:47:WU01:FS00:0xa4:Entering M.D.
06:08:53:WU01:FS00:0xa4:Mapping NT from 8 to 8 
06:08:54:WU01:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
06:09:38:WU01:FS00:Starting
06:09:38:WU01:FS00:Removing old file './work/01/logfile_01-20140418-053752.txt'
06:09:38:WU01:FS00:Running FahCore: /usr/bin/FAHCoreWrapper "/Library/Application Support/FAHClient/cores/www.stanford.edu/~pande/OSX/AMD64/Core_a4.fah/FahCore_a4" -dir 01 -suffix 01 -version 704 -lifeline 48646 -checkpoint 30 -np 8
06:09:38:WU01:FS00:Started FahCore on PID 48817
06:09:38:WU01:FS00:Core PID:48818
06:09:38:WU01:FS00:FahCore 0xa4 started
06:09:38:WU01:FS00:0xa4:
06:09:38:WU01:FS00:0xa4:*------------------------------*
06:09:38:WU01:FS00:0xa4:Folding@Home Gromacs Core
06:09:38:WU01:FS00:0xa4:Version 2.27 (Thu Feb 10 09:46:40 PST 2011)
06:09:38:WU01:FS00:0xa4:
06:09:38:WU01:FS00:0xa4:Preparing to commence simulation
06:09:38:WU01:FS00:0xa4:- Ensuring status. Please wait.
06:09:47:WU01:FS00:0xa4:- Looking at optimizations...
06:09:47:WU01:FS00:0xa4:- Working with standard loops on this execution.
06:09:47:WU01:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
06:09:47:WU01:FS00:0xa4:- Expanded 476208 -> 1434864 (decompressed 301.3 percent)
06:09:47:WU01:FS00:0xa4:Called DecompressByteArray: compressed_data_size=476208 data_size=1434864, decompressed_data_size=1434864 diff=0
06:09:47:WU01:FS00:0xa4:- Digital signature verified
06:09:47:WU01:FS00:0xa4:
06:09:47:WU01:FS00:0xa4:Project: 9006 (Run 811, Clone 3, Gen 0)
06:09:47:WU01:FS00:0xa4:
06:09:47:WU01:FS00:0xa4:Entering M.D.
06:09:53:WU01:FS00:0xa4:Mapping NT from 8 to 8 
06:09:54:WU01:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
06:10:38:WU01:FS00:Starting
06:10:38:WU01:FS00:Removing old file './work/01/logfile_01-20140418-053853.txt'
06:10:38:WU01:FS00:Running FahCore: /usr/bin/FAHCoreWrapper "/Library/Application Support/FAHClient/cores/www.stanford.edu/~pande/OSX/AMD64/Core_a4.fah/FahCore_a4" -dir 01 -suffix 01 -version 704 -lifeline 48646 -checkpoint 30 -np 8
06:10:38:WU01:FS00:Started FahCore on PID 48822
06:10:38:WU01:FS00:Core PID:48823
06:10:38:WU01:FS00:FahCore 0xa4 started
06:10:38:WU01:FS00:0xa4:
06:10:38:WU01:FS00:0xa4:*------------------------------*
06:10:38:WU01:FS00:0xa4:Folding@Home Gromacs Core
06:10:38:WU01:FS00:0xa4:Version 2.27 (Thu Feb 10 09:46:40 PST 2011)
06:10:38:WU01:FS00:0xa4:
06:10:38:WU01:FS00:0xa4:Preparing to commence simulation
06:10:38:WU01:FS00:0xa4:- Ensuring status. Please wait.
06:10:47:WU01:FS00:0xa4:- Looking at optimizations...
06:10:47:WU01:FS00:0xa4:- Working with standard loops on this execution.
06:10:47:WU01:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
06:10:47:WU01:FS00:0xa4:- Expanded 476208 -> 1434864 (decompressed 301.3 percent)
06:10:47:WU01:FS00:0xa4:Called DecompressByteArray: compressed_data_size=476208 data_size=1434864, decompressed_data_size=1434864 diff=0
06:10:48:WU01:FS00:0xa4:- Digital signature verified
06:10:48:WU01:FS00:0xa4:
06:10:48:WU01:FS00:0xa4:Project: 9006 (Run 811, Clone 3, Gen 0)
06:10:48:WU01:FS00:0xa4:
06:10:48:WU01:FS00:0xa4:Entering M.D.
06:10:54:WU01:FS00:0xa4:Mapping NT from 8 to 8 
06:10:54:WU01:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
06:11:38:WU01:FS00:Starting
06:11:38:WU01:FS00:Removing old file './work/01/logfile_01-20140418-053953.txt'
06:11:38:WU01:FS00:Running FahCore: /usr/bin/FAHCoreWrapper "/Library/Application Support/FAHClient/cores/www.stanford.edu/~pande/OSX/AMD64/Core_a4.fah/FahCore_a4" -dir 01 -suffix 01 -version 704 -lifeline 48646 -checkpoint 30 -np 8
06:11:38:WU01:FS00:Started FahCore on PID 48829
06:11:38:WU01:FS00:Core PID:48830
06:11:38:WU01:FS00:FahCore 0xa4 started
06:11:38:WU01:FS00:0xa4:
06:11:38:WU01:FS00:0xa4:*------------------------------*
06:11:38:WU01:FS00:0xa4:Folding@Home Gromacs Core
06:11:38:WU01:FS00:0xa4:Version 2.27 (Thu Feb 10 09:46:40 PST 2011)
06:11:38:WU01:FS00:0xa4:
06:11:38:WU01:FS00:0xa4:Preparing to commence simulation
06:11:38:WU01:FS00:0xa4:- Ensuring status. Please wait.
06:11:47:WU01:FS00:0xa4:- Looking at optimizations...
06:11:47:WU01:FS00:0xa4:- Working with standard loops on this execution.
06:11:47:WU01:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
06:11:48:WU01:FS00:0xa4:- Expanded 476208 -> 1434864 (decompressed 301.3 percent)
06:11:48:WU01:FS00:0xa4:Called DecompressByteArray: compressed_data_size=476208 data_size=1434864, decompressed_data_size=1434864 diff=0
06:11:48:WU01:FS00:0xa4:- Digital signature verified
06:11:48:WU01:FS00:0xa4:
06:11:48:WU01:FS00:0xa4:Project: 9006 (Run 811, Clone 3, Gen 0)
06:11:48:WU01:FS00:0xa4:
06:11:48:WU01:FS00:0xa4:Entering M.D.
06:11:54:WU01:FS00:0xa4:Mapping NT from 8 to 8 
06:11:54:WU01:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
06:12:38:WU01:FS00:Starting
06:12:38:WU01:FS00:Removing old file './work/01/logfile_01-20140418-054053.txt'
06:12:38:WU01:FS00:Running FahCore: /usr/bin/FAHCoreWrapper "/Library/Application Support/FAHClient/cores/www.stanford.edu/~pande/OSX/AMD64/Core_a4.fah/FahCore_a4" -dir 01 -suffix 01 -version 704 -lifeline 48646 -checkpoint 30 -np 8
06:12:38:WU01:FS00:Started FahCore on PID 48834
06:12:38:WU01:FS00:Core PID:48835
06:12:38:WU01:FS00:FahCore 0xa4 started
06:12:39:WU01:FS00:0xa4:
06:12:39:WU01:FS00:0xa4:*------------------------------*
06:12:39:WU01:FS00:0xa4:Folding@Home Gromacs Core
06:12:39:WU01:FS00:0xa4:Version 2.27 (Thu Feb 10 09:46:40 PST 2011)
06:12:39:WU01:FS00:0xa4:
06:12:39:WU01:FS00:0xa4:Preparing to commence simulation
06:12:39:WU01:FS00:0xa4:- Ensuring status. Please wait.
06:12:48:WU01:FS00:0xa4:- Looking at optimizations...
06:12:48:WU01:FS00:0xa4:- Working with standard loops on this execution.
06:12:48:WU01:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
06:12:48:WU01:FS00:0xa4:- Expanded 476208 -> 1434864 (decompressed 301.3 percent)
06:12:48:WU01:FS00:0xa4:Called DecompressByteArray: compressed_data_size=476208 data_size=1434864, decompressed_data_size=1434864 diff=0
06:12:48:WU01:FS00:0xa4:- Digital signature verified
06:12:48:WU01:FS00:0xa4:
06:12:48:WU01:FS00:0xa4:Project: 9006 (Run 811, Clone 3, Gen 0)
06:12:48:WU01:FS00:0xa4:
06:12:48:WU01:FS00:0xa4:Entering M.D.
06:12:54:WU01:FS00:0xa4:Mapping NT from 8 to 8 
06:12:54:WU01:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
06:13:38:WU01:FS00:Starting
06:13:38:WU01:FS00:Removing old file './work/01/logfile_01-20140418-054153.txt'
06:13:38:WU01:FS00:Running FahCore: /usr/bin/FAHCoreWrapper "/Library/Application Support/FAHClient/cores/www.stanford.edu/~pande/OSX/AMD64/Core_a4.fah/FahCore_a4" -dir 01 -suffix 01 -version 704 -lifeline 48646 -checkpoint 30 -np 8
06:13:38:WU01:FS00:Started FahCore on PID 48842
06:13:38:WU01:FS00:Core PID:48843
06:13:38:WU01:FS00:FahCore 0xa4 started
06:13:39:WU01:FS00:0xa4:
06:13:39:WU01:FS00:0xa4:*------------------------------*
06:13:39:WU01:FS00:0xa4:Folding@Home Gromacs Core
06:13:39:WU01:FS00:0xa4:Version 2.27 (Thu Feb 10 09:46:40 PST 2011)
06:13:39:WU01:FS00:0xa4:
06:13:39:WU01:FS00:0xa4:Preparing to commence simulation
06:13:39:WU01:FS00:0xa4:- Ensuring status. Please wait.
06:13:48:WU01:FS00:0xa4:- Looking at optimizations...
06:13:48:WU01:FS00:0xa4:- Working with standard loops on this execution.
06:13:48:WU01:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
06:13:48:WU01:FS00:0xa4:- Expanded 476208 -> 1434864 (decompressed 301.3 percent)
06:13:48:WU01:FS00:0xa4:Called DecompressByteArray: compressed_data_size=476208 data_size=1434864, decompressed_data_size=1434864 diff=0
06:13:48:WU01:FS00:0xa4:- Digital signature verified
06:13:48:WU01:FS00:0xa4:
06:13:48:WU01:FS00:0xa4:Project: 9006 (Run 811, Clone 3, Gen 0)
06:13:48:WU01:FS00:0xa4:
06:13:48:WU01:FS00:0xa4:Entering M.D.
06:13:54:WU01:FS00:0xa4:Mapping NT from 8 to 8 
06:13:55:WU01:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
06:14:38:WU01:FS00:Starting
06:14:38:WU01:FS00:Removing old file './work/01/logfile_01-20140418-054253.txt'
06:14:38:WU01:FS00:Running FahCore: /usr/bin/FAHCoreWrapper "/Library/Application Support/FAHClient/cores/www.stanford.edu/~pande/OSX/AMD64/Core_a4.fah/FahCore_a4" -dir 01 -suffix 01 -version 704 -lifeline 48646 -checkpoint 30 -np 8
06:14:38:WU01:FS00:Started FahCore on PID 48909
06:14:38:WU01:FS00:Core PID:48910
06:14:38:WU01:FS00:FahCore 0xa4 started
06:14:39:WU01:FS00:0xa4:
06:14:39:WU01:FS00:0xa4:*------------------------------*
06:14:39:WU01:FS00:0xa4:Folding@Home Gromacs Core
06:14:39:WU01:FS00:0xa4:Version 2.27 (Thu Feb 10 09:46:40 PST 2011)
06:14:39:WU01:FS00:0xa4:
06:14:39:WU01:FS00:0xa4:Preparing to commence simulation
06:14:39:WU01:FS00:0xa4:- Ensuring status. Please wait.
06:14:48:WU01:FS00:0xa4:- Looking at optimizations...
06:14:48:WU01:FS00:0xa4:- Working with standard loops on this execution.
06:14:48:WU01:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
06:14:48:WU01:FS00:0xa4:- Expanded 476208 -> 1434864 (decompressed 301.3 percent)
06:14:48:WU01:FS00:0xa4:Called DecompressByteArray: compressed_data_size=476208 data_size=1434864, decompressed_data_size=1434864 diff=0
06:14:48:WU01:FS00:0xa4:- Digital signature verified
06:14:48:WU01:FS00:0xa4:
06:14:48:WU01:FS00:0xa4:Project: 9006 (Run 811, Clone 3, Gen 0)
06:14:48:WU01:FS00:0xa4:
06:14:48:WU01:FS00:0xa4:Entering M.D.
06:14:54:WU01:FS00:0xa4:Mapping NT from 8 to 8 
06:14:55:WU01:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
06:15:39:WU01:FS00:Starting
06:15:39:WU01:FS00:Removing old file './work/01/logfile_01-20140418-054354.txt'
06:15:39:WU01:FS00:Running FahCore: /usr/bin/FAHCoreWrapper "/Library/Application Support/FAHClient/cores/www.stanford.edu/~pande/OSX/AMD64/Core_a4.fah/FahCore_a4" -dir 01 -suffix 01 -version 704 -lifeline 48646 -checkpoint 30 -np 8
06:15:39:WU01:FS00:Started FahCore on PID 48918
06:15:39:WU01:FS00:Core PID:48919
06:15:39:WU01:FS00:FahCore 0xa4 started
06:15:39:WU01:FS00:0xa4:
06:15:39:WU01:FS00:0xa4:*------------------------------*
06:15:39:WU01:FS00:0xa4:Folding@Home Gromacs Core
06:15:39:WU01:FS00:0xa4:Version 2.27 (Thu Feb 10 09:46:40 PST 2011)
06:15:39:WU01:FS00:0xa4:
06:15:39:WU01:FS00:0xa4:Preparing to commence simulation
06:15:39:WU01:FS00:0xa4:- Ensuring status. Please wait.
06:15:48:WU01:FS00:0xa4:- Looking at optimizations...
06:15:48:WU01:FS00:0xa4:- Working with standard loops on this execution.
06:15:48:WU01:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
06:15:48:WU01:FS00:0xa4:- Expanded 476208 -> 1434864 (decompressed 301.3 percent)
06:15:48:WU01:FS00:0xa4:Called DecompressByteArray: compressed_data_size=476208 data_size=1434864, decompressed_data_size=1434864 diff=0
06:15:48:WU01:FS00:0xa4:- Digital signature verified
06:15:48:WU01:FS00:0xa4:
06:15:48:WU01:FS00:0xa4:Project: 9006 (Run 811, Clone 3, Gen 0)
06:15:48:WU01:FS00:0xa4:
06:15:48:WU01:FS00:0xa4:Entering M.D.
06:15:54:WU01:FS00:0xa4:Mapping NT from 8 to 8 
06:15:55:WU01:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
06:16:39:WU01:FS00:Starting
06:16:39:WU01:FS00:Removing old file './work/01/logfile_01-20140418-054454.txt'
06:16:39:WU01:FS00:Running FahCore: /usr/bin/FAHCoreWrapper "/Library/Application Support/FAHClient/cores/www.stanford.edu/~pande/OSX/AMD64/Core_a4.fah/FahCore_a4" -dir 01 -suffix 01 -version 704 -lifeline 48646 -checkpoint 30 -np 8
06:16:39:WU01:FS00:Started FahCore on PID 48923
06:16:39:WU01:FS00:Core PID:48924
06:16:39:WU01:FS00:FahCore 0xa4 started
06:16:39:WU01:FS00:0xa4:
06:16:39:WU01:FS00:0xa4:*------------------------------*
06:16:39:WU01:FS00:0xa4:Folding@Home Gromacs Core
06:16:39:WU01:FS00:0xa4:Version 2.27 (Thu Feb 10 09:46:40 PST 2011)
06:16:39:WU01:FS00:0xa4:
06:16:39:WU01:FS00:0xa4:Preparing to commence simulation
06:16:39:WU01:FS00:0xa4:- Ensuring status. Please wait.
06:16:48:WU01:FS00:0xa4:- Looking at optimizations...
06:16:48:WU01:FS00:0xa4:- Working with standard loops on this execution.
06:16:48:WU01:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
06:16:48:WU01:FS00:0xa4:- Expanded 476208 -> 1434864 (decompressed 301.3 percent)
06:16:48:WU01:FS00:0xa4:Called DecompressByteArray: compressed_data_size=476208 data_size=1434864, decompressed_data_size=1434864 diff=0
06:16:48:WU01:FS00:0xa4:- Digital signature verified
06:16:48:WU01:FS00:0xa4:
06:16:48:WU01:FS00:0xa4:Project: 9006 (Run 811, Clone 3, Gen 0)
06:16:48:WU01:FS00:0xa4:
06:16:48:WU01:FS00:0xa4:Entering M.D.
06:16:54:WU01:FS00:0xa4:Mapping NT from 8 to 8 
06:16:55:WU01:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
06:17:39:WU01:FS00:Starting
06:17:39:WU01:FS00:Removing old file './work/01/logfile_01-20140418-054554.txt'
06:17:39:WU01:FS00:Running FahCore: /usr/bin/FAHCoreWrapper "/Library/Application Support/FAHClient/cores/www.stanford.edu/~pande/OSX/AMD64/Core_a4.fah/FahCore_a4" -dir 01 -suffix 01 -version 704 -lifeline 48646 -checkpoint 30 -np 8
06:17:39:WU01:FS00:Started FahCore on PID 48931
06:17:39:WU01:FS00:Core PID:48932
06:17:39:WU01:FS00:FahCore 0xa4 started
06:17:39:WU01:FS00:0xa4:
06:17:39:WU01:FS00:0xa4:*------------------------------*
06:17:39:WU01:FS00:0xa4:Folding@Home Gromacs Core
06:17:39:WU01:FS00:0xa4:Version 2.27 (Thu Feb 10 09:46:40 PST 2011)
06:17:39:WU01:FS00:0xa4:
06:17:39:WU01:FS00:0xa4:Preparing to commence simulation
06:17:39:WU01:FS00:0xa4:- Ensuring status. Please wait.
06:17:48:WU01:FS00:0xa4:- Looking at optimizations...
06:17:48:WU01:FS00:0xa4:- Working with standard loops on this execution.
06:17:48:WU01:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
06:17:49:WU01:FS00:0xa4:- Expanded 476208 -> 1434864 (decompressed 301.3 percent)
06:17:49:WU01:FS00:0xa4:Called DecompressByteArray: compressed_data_size=476208 data_size=1434864, decompressed_data_size=1434864 diff=0
06:17:49:WU01:FS00:0xa4:- Digital signature verified
06:17:49:WU01:FS00:0xa4:
06:17:49:WU01:FS00:0xa4:Project: 9006 (Run 811, Clone 3, Gen 0)
06:17:49:WU01:FS00:0xa4:
06:17:49:WU01:FS00:0xa4:Entering M.D.
06:17:55:WU01:FS00:0xa4:Mapping NT from 8 to 8 
06:17:55:WU01:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
06:18:39:WU01:FS00:Starting
06:18:39:WU01:FS00:Removing old file './work/01/logfile_01-20140418-054654.txt'
06:18:39:WU01:FS00:Running FahCore: /usr/bin/FAHCoreWrapper "/Library/Application Support/FAHClient/cores/www.stanford.edu/~pande/OSX/AMD64/Core_a4.fah/FahCore_a4" -dir 01 -suffix 01 -version 704 -lifeline 48646 -checkpoint 30 -np 8
06:18:39:WU01:FS00:Started FahCore on PID 48956
06:18:39:WU01:FS00:Core PID:48957
06:18:39:WU01:FS00:FahCore 0xa4 started
06:18:39:WU01:FS00:0xa4:
06:18:39:WU01:FS00:0xa4:*------------------------------*
06:18:39:WU01:FS00:0xa4:Folding@Home Gromacs Core
06:18:39:WU01:FS00:0xa4:Version 2.27 (Thu Feb 10 09:46:40 PST 2011)
06:18:39:WU01:FS00:0xa4:
06:18:39:WU01:FS00:0xa4:Preparing to commence simulation
06:18:39:WU01:FS00:0xa4:- Ensuring status. Please wait.
06:18:49:WU01:FS00:0xa4:- Looking at optimizations...
06:18:49:WU01:FS00:0xa4:- Working with standard loops on this execution.
06:18:49:WU01:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
06:18:49:WU01:FS00:0xa4:- Expanded 476208 -> 1434864 (decompressed 301.3 percent)
06:18:49:WU01:FS00:0xa4:Called DecompressByteArray: compressed_data_size=476208 data_size=1434864, decompressed_data_size=1434864 diff=0
06:18:49:WU01:FS00:0xa4:- Digital signature verified
06:18:49:WU01:FS00:0xa4:
06:18:49:WU01:FS00:0xa4:Project: 9006 (Run 811, Clone 3, Gen 0)
06:18:49:WU01:FS00:0xa4:
06:18:49:WU01:FS00:0xa4:Entering M.D.
06:18:55:WU01:FS00:0xa4:Mapping NT from 8 to 8 
06:18:56:WU01:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
06:19:39:WU01:FS00:Starting
06:19:39:WU01:FS00:Removing old file './work/01/logfile_01-20140418-054754.txt'
06:19:39:WU01:FS00:Running FahCore: /usr/bin/FAHCoreWrapper "/Library/Application Support/FAHClient/cores/www.stanford.edu/~pande/OSX/AMD64/Core_a4.fah/FahCore_a4" -dir 01 -suffix 01 -version 704 -lifeline 48646 -checkpoint 30 -np 8
06:19:39:WU01:FS00:Started FahCore on PID 48982
06:19:39:WU01:FS00:Core PID:48983
06:19:39:WU01:FS00:FahCore 0xa4 started
06:19:40:WU01:FS00:0xa4:
06:19:40:WU01:FS00:0xa4:*------------------------------*
06:19:40:WU01:FS00:0xa4:Folding@Home Gromacs Core
06:19:40:WU01:FS00:0xa4:Version 2.27 (Thu Feb 10 09:46:40 PST 2011)
06:19:40:WU01:FS00:0xa4:
06:19:40:WU01:FS00:0xa4:Preparing to commence simulation
06:19:40:WU01:FS00:0xa4:- Ensuring status. Please wait.
06:19:49:WU01:FS00:0xa4:- Looking at optimizations...
06:19:49:WU01:FS00:0xa4:- Working with standard loops on this execution.
06:19:49:WU01:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
06:19:49:WU01:FS00:0xa4:- Expanded 476208 -> 1434864 (decompressed 301.3 percent)
06:19:49:WU01:FS00:0xa4:Called DecompressByteArray: compressed_data_size=476208 data_size=1434864, decompressed_data_size=1434864 diff=0
06:19:49:WU01:FS00:0xa4:- Digital signature verified
06:19:49:WU01:FS00:0xa4:
06:19:49:WU01:FS00:0xa4:Project: 9006 (Run 811, Clone 3, Gen 0)
06:19:49:WU01:FS00:0xa4:
06:19:49:WU01:FS00:0xa4:Entering M.D.
06:19:55:WU01:FS00:0xa4:Mapping NT from 8 to 8 
06:19:55:WU01:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
06:20:39:WU01:FS00:Starting
06:20:39:WU01:FS00:Removing old file './work/01/logfile_01-20140418-054855.txt'
06:20:39:WU01:FS00:Running FahCore: /usr/bin/FAHCoreWrapper "/Library/Application Support/FAHClient/cores/www.stanford.edu/~pande/OSX/AMD64/Core_a4.fah/FahCore_a4" -dir 01 -suffix 01 -version 704 -lifeline 48646 -checkpoint 30 -np 8
06:20:39:WU01:FS00:Started FahCore on PID 48989
06:20:39:WU01:FS00:Core PID:48990
06:20:39:WU01:FS00:FahCore 0xa4 started
06:20:40:WU01:FS00:0xa4:
06:20:40:WU01:FS00:0xa4:*------------------------------*
06:20:40:WU01:FS00:0xa4:Folding@Home Gromacs Core
06:20:40:WU01:FS00:0xa4:Version 2.27 (Thu Feb 10 09:46:40 PST 2011)
06:20:40:WU01:FS00:0xa4:
06:20:40:WU01:FS00:0xa4:Preparing to commence simulation
06:20:40:WU01:FS00:0xa4:- Ensuring status. Please wait.
06:20:49:WU01:FS00:0xa4:- Looking at optimizations...
06:20:49:WU01:FS00:0xa4:- Working with standard loops on this execution.
06:20:49:WU01:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
06:20:49:WU01:FS00:0xa4:- Expanded 476208 -> 1434864 (decompressed 301.3 percent)
06:20:49:WU01:FS00:0xa4:Called DecompressByteArray: compressed_data_size=476208 data_size=1434864, decompressed_data_size=1434864 diff=0
06:20:49:WU01:FS00:0xa4:- Digital signature verified
06:20:49:WU01:FS00:0xa4:
06:20:49:WU01:FS00:0xa4:Project: 9006 (Run 811, Clone 3, Gen 0)
06:20:49:WU01:FS00:0xa4:
06:20:49:WU01:FS00:0xa4:Entering M.D.
06:20:55:WU01:FS00:0xa4:Mapping NT from 8 to 8 
06:20:56:WU01:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
06:21:39:WU01:FS00:Starting
06:21:39:WU01:FS00:Removing old file './work/01/logfile_01-20140418-054955.txt'
06:21:39:WU01:FS00:Running FahCore: /usr/bin/FAHCoreWrapper "/Library/Application Support/FAHClient/cores/www.stanford.edu/~pande/OSX/AMD64/Core_a4.fah/FahCore_a4" -dir 01 -suffix 01 -version 704 -lifeline 48646 -checkpoint 30 -np 8
06:21:39:WU01:FS00:Started FahCore on PID 49000
06:21:39:WU01:FS00:Core PID:49001
06:21:39:WU01:FS00:FahCore 0xa4 started
06:21:40:WU01:FS00:0xa4:
06:21:40:WU01:FS00:0xa4:*------------------------------*
06:21:40:WU01:FS00:0xa4:Folding@Home Gromacs Core
06:21:40:WU01:FS00:0xa4:Version 2.27 (Thu Feb 10 09:46:40 PST 2011)
06:21:40:WU01:FS00:0xa4:
06:21:40:WU01:FS00:0xa4:Preparing to commence simulation
06:21:40:WU01:FS00:0xa4:- Ensuring status. Please wait.
06:21:49:WU01:FS00:0xa4:- Looking at optimizations...
06:21:49:WU01:FS00:0xa4:- Working with standard loops on this execution.
06:21:49:WU01:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
06:21:49:WU01:FS00:0xa4:- Expanded 476208 -> 1434864 (decompressed 301.3 percent)
06:21:49:WU01:FS00:0xa4:Called DecompressByteArray: compressed_data_size=476208 data_size=1434864, decompressed_data_size=1434864 diff=0
06:21:49:WU01:FS00:0xa4:- Digital signature verified
06:21:49:WU01:FS00:0xa4:
06:21:49:WU01:FS00:0xa4:Project: 9006 (Run 811, Clone 3, Gen 0)
06:21:49:WU01:FS00:0xa4:
06:21:49:WU01:FS00:0xa4:Entering M.D.
06:21:55:WU01:FS00:0xa4:Mapping NT from 8 to 8 
06:21:56:WU01:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
06:22:40:WU01:FS00:Starting
06:22:40:WU01:FS00:Removing old file './work/01/logfile_01-20140418-055055.txt'
06:22:40:WU01:FS00:Running FahCore: /usr/bin/FAHCoreWrapper "/Library/Application Support/FAHClient/cores/www.stanford.edu/~pande/OSX/AMD64/Core_a4.fah/FahCore_a4" -dir 01 -suffix 01 -version 704 -lifeline 48646 -checkpoint 30 -np 8
06:22:40:WU01:FS00:Started FahCore on PID 49006
06:22:40:WU01:FS00:Core PID:49007
06:22:40:WU01:FS00:FahCore 0xa4 started
06:22:40:WU01:FS00:0xa4:
06:22:40:WU01:FS00:0xa4:*------------------------------*
06:22:40:WU01:FS00:0xa4:Folding@Home Gromacs Core
06:22:40:WU01:FS00:0xa4:Version 2.27 (Thu Feb 10 09:46:40 PST 2011)
06:22:40:WU01:FS00:0xa4:
06:22:40:WU01:FS00:0xa4:Preparing to commence simulation
06:22:40:WU01:FS00:0xa4:- Ensuring status. Please wait.

Unable to process the WU.

[Joe_H]

So far there is one failure report for this WU in the database.

I have seen this repeating failure a few times before with the OS X version of the A4 core and the V7 client on my Mac's. The WU fails immediately, but just keeps retrying to start instead of detecting the failed processing and sending it back as a failed WU. You can dump this WU, it most likely is bad. If you want, you can try and see if it will process with a different core count, but that may make no difference.

P.S. I said most likely bad, of the WU's that I have seen do this about 2/3 were never successfully processed by others. But a few did get completed successfully.

[7im]

Not picking on anyone in particular, as I've seen some really big and unnecessary log posts, so when posting a log, please note that trimming the log down to just the important bits helps everyone trying to help you.

The people trying to help don't need to see 100 copies of the same error. 1 or 2 is sufficient.

They don't want to have to scroll through all the frames of 3 completed work units to see the details of the 4th one that failed.

Unless you are having upload or download problems those lines can go too.

And if all the frame times are about the same, snip out all but the first few and the last few.

It's good forum decorum, as are always using the code tags.

[TomJohnson]

Does the time interval between interrupted cycles and number of interrupts that occur in a 30 minutes time frame give the trouble shooters a better understanding of the problem ?

The interrupt problem was still repeating this morning so I deleted the problem WU and the computer is now Folding the new WU properly.

FAH programmers should have a "delete WU after so many interrupts" line of code in the Client software rather than relying on the User to monitor their computers to discover a problem which is causing their computer to stop Folding.
This is an important step for those Users who have a large number of computers Folding that are not being monitored on a regular basis.

[7im]

IMO, the cycle may help, hence the 1 to 2 copies of the error. The number of cycles per time frame rarely helps. Seeing the one cycle time from the first instance of the error to the optional second instance might help.

You could also just show one example and then state the same error repeats every x minutes.

[PantherX]

...FAH programmers should have a "delete WU after so many interrupts" line of code in the Client software rather than relying on the User to monitor their computers to discover a problem which is causing their computer to stop Folding.
This is an important step for those Users who have a large number of computers Folding that are not being monitored on a regular basis.

Do note that the feature is present from V7.4.0 upwards but it is not enabled by default. The feature contains; stall-detection, stall-timeout and stall-percent. Before being enabled by default, further testing needs to be done to ensure that it works as intended. You may try to use it put please note that it fits in the "experimental feature" section so unexpected results may occur.

[7im]

We also don't want the stall detection to kick in too quickly if someone has overclocked to an unstable level. It would just make the problem worse and drain some work units.

[Joe_H]

FAH programmers should have a "delete WU after so many interrupts" line of code in the Client software rather than relying on the User to monitor their computers to discover a problem which is causing their computer to stop Folding.
This is an important step for those Users who have a large number of computers Folding that are not being monitored on a regular basis.

As mentioned in the posts by 7im and PantherX, code in the client has been included to try and detect this problem and keep it from stalling a machine. However there are a number of notes in tickets related to this that show it has had problems, and that is why it is currently not enabled.

As for this specific error and repeating attempts, there are a number of older reports of this here in the forum and a ticket was opened over a year ago. I believe the problem was thought to actually lie in the code in the folding core, it is failing out and not handling the repeated failure correctly. Look for the lines where it reports

Examination of work files indicates 8 consecutive improper terminations of core.

At that point the WU should have been failed out by the core and the client should have received an an error code that would cause it to dump the work unit. The wrong error code is apparently being set by the core. A future revision to the core might fix this, but their has been no indication if that would happen.

[TomJohnson]

Thanks for the replies from the program/programmers perspective.
The problem from the Users viewpoint is that some of us with many remote computers available do not have the option to remotely view WU Folding progress on a regular basis and this can cause some of these computer to be running idle, not Folding, for an indefinite amount of time until we can physically check on these remotely Folding computers. The activation of the fix discussed below could resolve this problem.
FYI - I have only seen this problem on my newest 27" iMac, a 14,2 CTO i7 3.1 GHz Dual Quad 4771 with 24 GB of RAM running OS 10.9.2; not over clocked and Folding 24/7; with "Prevent App Nap" not selected; also with 128 GB SSD/1 TB Fusion hard drive.
Don't know if the Processing/Memory/L1,L2,L3 Cache usage associated with the Fusion drive might have anything to do with this problem.
Does the Error Code 102=0x66 give any hint as to where the problem stems from ?

[Joe_H]

I have seen this on both my iMac and my son's Mac Pro. I don't recall if I ever saw it happen on my MacBook, but I may look through my old logs to see at some point. As for OS version, I have seen it on 10.6.8 on both machines and once after my son upgraded to 10.8. I don't recall the oldest version of the V7 client where I have seen this. So I don't think the problem has much to do with your specific configuration.

As for frequency, this is not a common problem. I have about 2 years worth of old logs over three different machines, and there were less than a dozen occurrences of this when I last checked months ago. So it has been reported, but I don't know if the core programmers have enough information to isolate the problem and fix it. And given the low frequency of occurrence, the resources might be focused on fixing problems with bigger impact.

Finally, I don't recall reports of this same issue affecting the Linux and Windows version of this core. but may have overlooked them. It may just be an issue with OS X's version of the core. The stalled WU code they are trying to get working in the client might or might not take care of this specific error as I understand it is being designed for the general case of handling multiple failures or lack of progress of a WU. At this point in the worst case the WU will get dumped eventually when it reaches the final deadline, they are trying to come up with a better solution.

[bruce]

INTERRUPTED (102 = 0x66) is a normal message that doesn't tell you anything useful. Something is telling it to stop running.

If you use FAHControl and you press PAUSE as I did below, it shuts down the core with that kind of interruption. If you kill a FAHCore with task manager, you'll get that same indication. If you have the Console window open and you close it or if you issue a CTRL-C to interrupt FAHClient, I think you'll get the same indication when the FAHCore has to close. If your antivirus program decides the FAHCore is using too much processor time and concludes it's a run-away process and kills it, you'll also get that INTERRUPTED return code.

Code: Select all

20:42:14:FS01:Paused
20:42:14:FS01:Shutting core down
20:42:14:WU02:FS01:0x17:WARNING:Console control signal 1 on PID 4508
20:42:14:WU02:FS01:0x17:Exiting, please wait. . .
20:43:15:WARNING:FS01:Killing WU02
20:43:15:WU02:FS01:FahCore returned: INTERRUPTED (102 = 0x66)
20:44:01:FS01:Unpaused
20:44:01:WU02:FS01:Starting

[Joe_H]

Yes, the INTERRUPTED (102 = 0x66) message is normal under some circumstances. However in this case it is the visible symptom of an underlying problem connected with the core failing to engage and process the WU. It occurs immediately as the core starts, and is not due to any external signal such as telling the core to pause or intervention from an antivirus program. In my case I know there is no antivirus program running to send any such signal.

[TomJohnson]

New WU not downloaded before current WU 9009 completes - keep getting following interrupt log text:

Code: Select all

Core_a4.fah/FahCore_a4" -dir 00 -suffix 01 -version 704 -lifeline 117 -checkpoint 30 -np 24
16:42:38:WU00:FS00:Started FahCore on PID 364
16:42:38:WU00:FS00:Core PID:367
16:42:38:WU00:FS00:FahCore 0xa4 started
16:42:39:WU00:FS00:0xa4:
16:42:39:WU00:FS00:0xa4:*------------------------------*
16:42:39:WU00:FS00:0xa4:Folding@Home Gromacs Core
16:42:39:WU00:FS00:0xa4:Version 2.27 (Thu Feb 10 09:46:40 PST 2011)
16:42:39:WU00:FS00:0xa4:
16:42:39:WU00:FS00:0xa4:Preparing to commence simulation
16:42:39:WU00:FS00:0xa4:- Ensuring status. Please wait.
16:42:48:WU00:FS00:0xa4:- Looking at optimizations...
16:42:48:WU00:FS00:0xa4:- Working with standard loops on this execution.
16:42:48:WU00:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
16:42:48:WU00:FS00:0xa4:- Expanded 705209 -> 1476032 (decompressed 209.3 percent)
16:42:48:WU00:FS00:0xa4:Called DecompressByteArray: compressed_data_size=705209 data_size=1476032, decompressed_data_size=1476032 diff=0
16:42:48:WU00:FS00:0xa4:- Digital signature verified
16:42:48:WU00:FS00:0xa4:
16:42:48:WU00:FS00:0xa4:Project: 9009 (Run 724, Clone 0, Gen 0)
16:42:48:WU00:FS00:0xa4:
16:42:48:WU00:FS00:0xa4:Entering M.D.
16:42:54:WU00:FS00:0xa4:Mapping NT from 24 to 24 
16:42:55:WU00:FS00:0xa4:Completed 0 out of 250000 steps  (0%)
16:42:55:WU00:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
16:43:38:WU00:FS00:Starting
16:43:38:WU00:FS00:Removing old file './work/00/logfile_01-20140612-161107.txt'
16:43:38:WU00:FS00:Running FahCore: /usr/bin/FAHCoreWrapper "/Library/Application Support/FAHClient/cores/web.stanford.edu/~pande/OSX/AMD64/Core_a4.fah/FahCore_a4" -dir 00 -suffix 01 -version 704 -lifeline 117 -checkpoint 30 -np 24
16:43:38:WU00:FS00:Started FahCore on PID 380
16:43:39:WU00:FS00:Core PID:383
16:43:39:WU00:FS00:FahCore 0xa4 started
16:43:39:WU00:FS00:0xa4:
16:43:39:WU00:FS00:0xa4:*------------------------------*
16:43:39:WU00:FS00:0xa4:Folding@Home Gromacs Core
16:43:39:WU00:FS00:0xa4:Version 2.27 (Thu Feb 10 09:46:40 PST 2011)
16:43:39:WU00:FS00:0xa4:
16:43:39:WU00:FS00:0xa4:Preparing to commence simulation
16:43:39:WU00:FS00:0xa4:- Ensuring status. Please wait.
16:43:48:WU00:FS00:0xa4:- Looking at optimizations...
16:43:48:WU00:FS00:0xa4:- Working with standard loops on this execution.
16:43:48:WU00:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
16:43:48:WU00:FS00:0xa4:- Expanded 705209 -> 1476032 (decompressed 209.3 percent)
16:43:48:WU00:FS00:0xa4:Called DecompressByteArray: compressed_data_size=705209 data_size=1476032, decompressed_data_size=1476032 diff=0
16:43:48:WU00:FS00:0xa4:- Digital signature verified
16:43:48:WU00:FS00:0xa4:
16:43:48:WU00:FS00:0xa4:Project: 9009 (Run 724, Clone 0, Gen 0)
16:43:48:WU00:FS00:0xa4:
16:43:49:WU00:FS00:0xa4:Entering M.D.
16:43:55:WU00:FS00:0xa4:Mapping NT from 24 to 24 
16:43:55:WU00:FS00:0xa4:Completed 0 out of 250000 steps  (0%)
16:43:55:WU00:FS00:FahCore returned: INTERRUPTED (102 = 0x66)

[PantherX]

Please note that there isn't any record in the WU Database. It has been marked for a follow-up.

[bruce]

I've merged this new topic with another that looks similar enough that it might be the same problem. In the case of Project: 9006 (Run 811, Clone 3, Gen 0) which is now at the top of the topic, the client reported the problem to the server and the WU was reassigned. Everybody who attempted to run that WU encountered an error.

Both Project: 9006 and Project: 9009 are studies by the same PI, so I think it makes sense to ask him to look into the nature of those two errors and see if the INTERRUPTED was caused by a similar event.