Project 8690 bad WU

[bruce]

Maybe I missed it, but I don't see an indication of what kind of GPU you have. It would be reported (only) near the bottom of the first page of the log.

As a wild guess, perhaps your GPU does not support DoublePrecision and OpenCL 1.2. This is a requirement for (almost) every project now but it's not well enforced. Except for change in FAH's initialization panel, I don't know what other symptoms might be indicated; if it could be causing the INTERRUPTED error.

Does FAHBench run in double precision on your system?

[vincent89147]

My slot is of the CPU type, not GPU so I'm not sure that that matters. The GPU is an NVidia 8300 (on board the M3N78 PRO motherboard) but not properly configured for use as anything but graphics. I have added this slot using "slot-add cpu", so I'm not sure that I should get WUs that require a GPU slot. If I do, though, and this project requires a GPU to run, then that may explain the interruptions.

FAHBench does not work:

Code: Select all

vv@nestor:~/FAHBench-2.3.2-Linux/bin$ ./FAHBench-cmd 
FAHBench Simulation
-------------------
Plugin directory: "/home/vv/FAHBench-2.3.2-Linux/lib/openmm"
Work unit: dhfr
WU Name: Dihydrofolate reductase
WU Description: A common system for benchmarking molecular dynamics
System XML: /home/vv/FAHBench-2.3.2-Linux/share/fahbench/workunits/dhfr/system.xml
Integrator XML: /home/vv/FAHBench-2.3.2-Linux/share/fahbench/workunits/dhfr/integrator.xml
State XML: /home/vv/FAHBench-2.3.2-Linux/share/fahbench/workunits/dhfr/state.xml
Step chunk: 40
Device ID 0; Platform OpenCL; Platform ID 0
Run length: 60s

Loading plugins from plugin directory
Number of registered plugins: 2
Deserializing input files: system
Deserializing input files: state
Deserializing input files: integrator
Creating context (may take several minutes)

Something went wrong:
Error initializing context: clGetPlatformIDs (-1001)

Thanks!

[vincent89147]

I took some time today to fix the GPU configuration and after some tinkering and installing the nvidia-legacy-340xx drivers, CUDA and openCL libraries I have gotten it to work properly:

Code: Select all

19:50:22:******************************* System ********************************
19:50:22:            CPU: AMD Phenom(tm) 9950 Quad-Core Processor
19:50:22:         CPU ID: AuthenticAMD Family 16 Model 2 Stepping 3
19:50:22:           CPUs: 4
19:50:22:         Memory: 7.54GiB
19:50:22:    Free Memory: 7.09GiB
19:50:22:        Threads: POSIX_THREADS
19:50:22:     OS Version: 4.19
19:50:22:    Has Battery: false
19:50:22:     On Battery: false
19:50:22:     UTC Offset: -7
19:50:22:            PID: 1050
19:50:22:            CWD: /var/lib/fahclient
19:50:22:             OS: Linux 4.19.0-5-amd64 x86_64
19:50:22:        OS Arch: AMD64
19:50:22:           GPUs: 1
19:50:22:          GPU 0: Bus:2 Slot:0 Func:0 NVIDIA:1 C77 [GeForce 8300]
19:50:22:  CUDA Device 0: Platform:0 Device:0 Bus:0 Slot:0 Compute:1.1 Driver:6.5
19:50:22:OpenCL Device 0: Platform:0 Device:0 Bus:0 Slot:0 Compute:1.0 Driver:340.107
19:50:22:***********************************************************************

However, FAHBench-cmd will not finish successfully:

Code: Select all

vv@nestor:~/FAHBench-2.3.2-Linux/bin$ ./FAHBench-cmd 
FAHBench Simulation
-------------------
Plugin directory: "/home/vv/FAHBench-2.3.2-Linux/lib/openmm"
Work unit: dhfr
WU Name: Dihydrofolate reductase
WU Description: A common system for benchmarking molecular dynamics
System XML: /home/vv/FAHBench-2.3.2-Linux/share/fahbench/workunits/dhfr/system.xml
Integrator XML: /home/vv/FAHBench-2.3.2-Linux/share/fahbench/workunits/dhfr/integrator.xml
State XML: /home/vv/FAHBench-2.3.2-Linux/share/fahbench/workunits/dhfr/state.xml
Step chunk: 40
Device ID 0; Platform OpenCL; Platform ID 0
Run length: 60s

Loading plugins from plugin directory
Number of registered plugins: 2
Deserializing input files: system
Deserializing input files: state
Deserializing input files: integrator
Creating context (may take several minutes)
Checking accuracy against reference code
Creating reference context (may take several minutes)
Comparing forces and energy

Something went wrong:
Error downloading array interactionCount: clEnqueueReadBuffer (-36)

[vincent89147]

After several days I finally got another 8690 WU, but even with openCL and CUDA now properly detected, the only thing that happens is that it gets interrupted as before:

Code: Select all

14:46:20:******************************* System ********************************
14:46:20:            CPU: AMD Phenom(tm) 9950 Quad-Core Processor
14:46:20:         CPU ID: AuthenticAMD Family 16 Model 2 Stepping 3
14:46:20:           CPUs: 4
14:46:20:         Memory: 7.54GiB
14:46:20:    Free Memory: 7.05GiB
14:46:20:        Threads: POSIX_THREADS
14:46:20:     OS Version: 4.19
14:46:20:    Has Battery: false
14:46:20:     On Battery: false
14:46:20:     UTC Offset: -7
14:46:20:            PID: 1121
14:46:20:            CWD: /var/lib/fahclient
14:46:20:             OS: Linux 4.19.0-5-amd64 x86_64
14:46:20:        OS Arch: AMD64
14:46:20:           GPUs: 1
14:46:20:          GPU 0: Bus:2 Slot:0 Func:0 NVIDIA:0 C77 [GeForce 8300]
14:46:20:  CUDA Device 0: Platform:0 Device:0 Bus:0 Slot:0 Compute:1.1 Driver:6.5
14:46:20:OpenCL Device 0: Platform:0 Device:0 Bus:0 Slot:0 Compute:1.0 Driver:340.107
14:46:20:***********************************************************************

Code: Select all

$ grep INTERRUPTED /var/lib/fahclient/log.txt 
04:58:53:WU03:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
04:59:09:WU03:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
05:00:09:WU03:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
05:01:09:WU03:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
05:02:09:WU03:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
05:03:09:WU03:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
05:04:09:WU03:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
05:05:09:WU03:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
05:06:09:WU03:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
05:07:09:WU03:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
05:08:09:WU03:FS00:FahCore returned: INTERRUPTED (102 = 0x66)

The following files are in the WU's work directory:

Code: Select all

drwxrwxrwx 2 fahclient root   4096 Aug  3 22:05 .
drwxrwxrwx 6 fahclient root     82 Aug  3 21:58 ..
-rw-rw-rw- 1 fahclient root      0 Aug  3 21:58 core78.sta
-rwxr-x--- 1 fahclient root    482 Aug  3 21:58 logfile_01-20190804-045853.txt
-rwxr-x--- 1 fahclient root    554 Aug  3 21:59 logfile_01-20190804-045953.txt
-rwxr-x--- 1 fahclient root    585 Aug  3 22:00 logfile_01-20190804-050053.txt
-rwxr-x--- 1 fahclient root    585 Aug  3 22:01 logfile_01-20190804-050153.txt
-rwxr-x--- 1 fahclient root    585 Aug  3 22:02 logfile_01-20190804-050253.txt
-rwxr-x--- 1 fahclient root    585 Aug  3 22:03 logfile_01-20190804-050353.txt
-rwxr-x--- 1 fahclient root    585 Aug  3 22:04 logfile_01-20190804-050453.txt
-rwxr-x--- 1 fahclient root    585 Aug  3 22:05 logfile_01-20190804-050553.txt
-rwxr-x--- 1 fahclient root    572 Aug  3 22:06 logfile_01.txt
-rw-r--r-- 1 fahclient root 127884 Aug  3 21:42 wudata_01.dat
-rw-rw-rw- 1 fahclient root     64 Aug  3 22:05 wudata_01.dyn
-rw-rw-rw- 1 fahclient root      0 Aug  3 22:06 wudata_01.log
-rw-rw-rw- 1 fahclient root 264024 Aug  3 22:06 wudata_01.tpr
-rwxr-x--- 1 fahclient root    512 Aug  3 22:06 wuinfo_01.dat

The logfiles all look the same:

Code: Select all

$ sudo cat logfile_01-20190804-050553.txt

*------------------------------*
Folding@Home Gromacs GB Core
Version 2.27 (Dec. 15, 2010)

Preparing to commence simulation
- Ensuring status. Please wait.
- Looking at optimizations...
- Working with standard loops on this execution.
- Previous termination of core was improper.
- Going to use standard loops.
- Files status OK
- Expanded 127372 -> 264024 (decompressed 207.2 percent)
Called DecompressByteArray: compressed_data_size=127372 data_size=264024, decompressed_data_size=264024 diff=0
- Digital signature verified

Project: 8690 (Run 0, Clone 832, Gen 40)

Entering M.D.
vv@nestor:/var/lib/fahclient/work/03$

I'm not in a hurry with this, and if there's not enough interest in looking into this then I can easily dump 8690 WUs when I get them. But I'll also be happy to troubleshoot, if only to see why the WU doesn't get skipped after a number of times.

Thanks!

[vincent89147]

Another thing I just found is that these INTERRUPTED messages generate segfaults in the system log:

Code: Select all

[Sat Aug  3 22:19:08 2019] FahCore_a4[2800] vsyscall attempted with vsyscall=none ip:ffffffffff600400 cs:33 sp:7f58d7c23d88 ax:ffffffffff600400 si:7f58d7c23c28 di:7f58d7c23dc8
[Sat Aug  3 22:19:08 2019] FahCore_a4[2800]: segfault at ffffffffff600400 ip ffffffffff600400 sp 00007f58d7c23d88 error 15
[Sat Aug  3 22:19:08 2019] Code: Bad RIP value.

Code: Select all

$ sudo dmesg -T | grep "Bad RIP value"
[Sat Aug  3 21:58:52 2019] Code: Bad RIP value.
[Sat Aug  3 21:59:08 2019] Code: Bad RIP value.
[Sat Aug  3 22:00:08 2019] Code: Bad RIP value.
[Sat Aug  3 22:01:08 2019] Code: Bad RIP value.
[Sat Aug  3 22:02:08 2019] Code: Bad RIP value.
[Sat Aug  3 22:03:08 2019] Code: Bad RIP value.
[Sat Aug  3 22:04:08 2019] Code: Bad RIP value.
[Sat Aug  3 22:05:08 2019] Code: Bad RIP value.
[Sat Aug  3 22:06:08 2019] Code: Bad RIP value.
[Sat Aug  3 22:07:08 2019] Code: Bad RIP value.
[Sat Aug  3 22:08:08 2019] Code: Bad RIP value.
[Sat Aug  3 22:09:08 2019] Code: Bad RIP value.
[Sat Aug  3 22:10:08 2019] Code: Bad RIP value.
[Sat Aug  3 22:11:08 2019] Code: Bad RIP value.
[Sat Aug  3 22:12:08 2019] Code: Bad RIP value.
[Sat Aug  3 22:13:08 2019] Code: Bad RIP value.
[Sat Aug  3 22:14:08 2019] Code: Bad RIP value.
[Sat Aug  3 22:15:08 2019] Code: Bad RIP value.
[Sat Aug  3 22:16:08 2019] Code: Bad RIP value.
[Sat Aug  3 22:17:08 2019] Code: Bad RIP value.
[Sat Aug  3 22:18:08 2019] Code: Bad RIP value.
[Sat Aug  3 22:19:08 2019] Code: Bad RIP value.
[Sat Aug  3 22:20:08 2019] Code: Bad RIP value.

[vincent89147]

I found the cause of the problem: https://einsteinathome.org/content/vsys ... ux-distros

When I load the kernel using 'vsyscall=emulate' on the command line, there are no interrupts.

[bruce]

Please find the segment of your historic logs where the WU was downloaded (before the first interrupt failure) and post that segment of the log.

According to the history of Project: 8690 (Run 0, Clone 459, Gen 21), it was assigned to vincent and it was dumped. It was reassigned to another person who successfully completed it. I conclude the WU is not bad ... that there's something wrong with your hardware.

Does your system support Double Precision Floating point?

[vincent89147]

Thanks Bruce,

My hardware is fine and the WU is fine as well. The problem is that the Linux 0xa4 core binary can't and won't run when the kernel is not booted with vsyscall support. For the past three years, Debian kernels (and perhaps others, too) by default have support for vsyscall disabled at boot time, but it is possible to configure the kernel to boot with vsyscall support.

When I turned on vsyscall and rebooted, the 0xa4 WU that was downloaded the other day completed without problems on my hardware. If my dumped WU of two or three weeks ago was picked up by another client that was either a: running Windows, (I don't know whether that uses anything similar to vsyscall or not, or whether it is on by default), or b: running a Linux kernel with vsyscall enabled, then I would indeed expect that that same WU would finish without problems.

Had my kernel had vsyscall support enabled, then this same WU that I dumped would have finished successfully on my hardware.

The log of the latest 8690 run is here:

Code: Select all

$ grep WU03 log-20190804-052539.txt
04:42:33:WU03:FS00:Connecting to 65.254.110.245:8080
04:42:33:WU03:FS00:Assigned to work server 155.247.166.220
04:42:33:WU03:FS00:Requesting new work unit for slot 00: RUNNING cpu:1 from 155.247.166.220
04:42:33:WU03:FS00:Connecting to 155.247.166.220:8080
04:42:37:WU03:FS00:Downloading 124.89KiB
04:42:38:WU03:FS00:Download complete
04:42:38:WU03:FS00:Received Unit: id:03 state:DOWNLOAD error:NO_ERROR project:8690 run:0 clone:832 gen:40 core:0xa4 unit:0x000000300002894c5d24f90198be4f75
04:58:47:WU03:FS00:Starting
04:58:47:WU03:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/Linux/AMD64/Core_a4.fah/FahCore_a4 -dir 03 -suffix 01 -version 705 -lifeline 1121 -checkpoint 15 -np 1
04:58:47:WU03:FS00:Started FahCore on PID 2521
04:58:47:WU03:FS00:Core PID:2525
04:58:47:WU03:FS00:FahCore 0xa4 started
04:58:47:WU03:FS00:0xa4:
04:58:47:WU03:FS00:0xa4:*------------------------------*
04:58:47:WU03:FS00:0xa4:Folding@Home Gromacs GB Core
04:58:47:WU03:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
04:58:47:WU03:FS00:0xa4:
04:58:47:WU03:FS00:0xa4:Preparing to commence simulation
04:58:47:WU03:FS00:0xa4:- Looking at optimizations...
04:58:47:WU03:FS00:0xa4:- Created dyn
04:58:47:WU03:FS00:0xa4:- Files status OK
04:58:47:WU03:FS00:0xa4:- Expanded 127372 -> 264024 (decompressed 207.2 percent)
04:58:47:WU03:FS00:0xa4:Called DecompressByteArray: compressed_data_size=127372 data_size=264024, decompressed_data_size=264024 diff=0
04:58:47:WU03:FS00:0xa4:- Digital signature verified
04:58:47:WU03:FS00:0xa4:
04:58:47:WU03:FS00:0xa4:Project: 8690 (Run 0, Clone 832, Gen 40)
04:58:47:WU03:FS00:0xa4:
04:58:47:WU03:FS00:0xa4:Assembly optimizations on if available.
04:58:47:WU03:FS00:0xa4:Entering M.D.
04:58:53:WU03:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
04:58:53:WU03:FS00:Starting
04:58:53:WU03:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/Linux/AMD64/Core_a4.fah/FahCore_a4 -dir 03 -suffix 01 -version 705 -lifeline 1121 -checkpoint 15 -np 1
04:58:53:WU03:FS00:Started FahCore on PID 2529
04:58:53:WU03:FS00:Core PID:2533
04:58:53:WU03:FS00:FahCore 0xa4 started
04:58:54:WU03:FS00:0xa4:
04:58:54:WU03:FS00:0xa4:*------------------------------*
04:58:54:WU03:FS00:0xa4:Folding@Home Gromacs GB Core
04:58:54:WU03:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
04:58:54:WU03:FS00:0xa4:
04:58:54:WU03:FS00:0xa4:Preparing to commence simulation
04:58:54:WU03:FS00:0xa4:- Ensuring status. Please wait.
04:59:03:WU03:FS00:0xa4:- Looking at optimizations...
04:59:03:WU03:FS00:0xa4:- Working with standard loops on this execution.
04:59:03:WU03:FS00:0xa4:- Previous termination of core was improper.
04:59:03:WU03:FS00:0xa4:- Files status OK
04:59:03:WU03:FS00:0xa4:- Expanded 127372 -> 264024 (decompressed 207.2 percent)
04:59:03:WU03:FS00:0xa4:Called DecompressByteArray: compressed_data_size=127372 data_size=264024, decompressed_data_size=264024 diff=0
04:59:03:WU03:FS00:0xa4:- Digital signature verified
04:59:03:WU03:FS00:0xa4:
04:59:03:WU03:FS00:0xa4:Project: 8690 (Run 0, Clone 832, Gen 40)
04:59:03:WU03:FS00:0xa4:
04:59:03:WU03:FS00:0xa4:Entering M.D.
04:59:09:WU03:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
04:59:53:WU03:FS00:Starting
04:59:53:WU03:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/Linux/AMD64/Core_a4.fah/FahCore_a4 -dir 03 -suffix 01 -version 705 -lifeline 1121 -checkpoint 15 -np 1
04:59:53:WU03:FS00:Started FahCore on PID 2551
04:59:53:WU03:FS00:Core PID:2555
04:59:53:WU03:FS00:FahCore 0xa4 started
04:59:54:WU03:FS00:0xa4:
04:59:54:WU03:FS00:0xa4:*------------------------------*
04:59:54:WU03:FS00:0xa4:Folding@Home Gromacs GB Core
04:59:54:WU03:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
04:59:54:WU03:FS00:0xa4:
04:59:54:WU03:FS00:0xa4:Preparing to commence simulation
04:59:54:WU03:FS00:0xa4:- Ensuring status. Please wait.
05:00:03:WU03:FS00:0xa4:- Looking at optimizations...
05:00:03:WU03:FS00:0xa4:- Working with standard loops on this execution.
05:00:03:WU03:FS00:0xa4:- Previous termination of core was improper.
05:00:03:WU03:FS00:0xa4:- Going to use standard loops.
05:00:03:WU03:FS00:0xa4:- Files status OK
05:00:03:WU03:FS00:0xa4:- Expanded 127372 -> 264024 (decompressed 207.2 percent)
05:00:03:WU03:FS00:0xa4:Called DecompressByteArray: compressed_data_size=127372 data_size=264024, decompressed_data_size=264024 diff=0
05:00:03:WU03:FS00:0xa4:- Digital signature verified
05:00:03:WU03:FS00:0xa4:
05:00:03:WU03:FS00:0xa4:Project: 8690 (Run 0, Clone 832, Gen 40)
05:00:03:WU03:FS00:0xa4:
05:00:03:WU03:FS00:0xa4:Entering M.D.
05:00:09:WU03:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
05:00:53:WU03:FS00:Starting
05:00:53:WU03:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/Linux/AMD64/Core_a4.fah/FahCore_a4 -dir 03 -suffix 01 -version 705 -lifeline 1121 -checkpoint 15 -np 1
05:00:53:WU03:FS00:Started FahCore on PID 2563
05:00:53:WU03:FS00:Core PID:2567
05:00:53:WU03:FS00:FahCore 0xa4 started
05:00:54:WU03:FS00:0xa4:
05:00:54:WU03:FS00:0xa4:*------------------------------*
05:00:54:WU03:FS00:0xa4:Folding@Home Gromacs GB Core
05:00:54:WU03:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
05:00:54:WU03:FS00:0xa4:
05:00:54:WU03:FS00:0xa4:Preparing to commence simulation
05:00:54:WU03:FS00:0xa4:- Ensuring status. Please wait.
05:01:03:WU03:FS00:0xa4:- Looking at optimizations...
05:01:03:WU03:FS00:0xa4:- Working with standard loops on this execution.
05:01:03:WU03:FS00:0xa4:- Previous termination of core was improper.
05:01:03:WU03:FS00:0xa4:- Going to use standard loops.
05:01:03:WU03:FS00:0xa4:- Files status OK
05:01:03:WU03:FS00:0xa4:- Expanded 127372 -> 264024 (decompressed 207.2 percent)
05:01:03:WU03:FS00:0xa4:Called DecompressByteArray: compressed_data_size=127372 data_size=264024, decompressed_data_size=264024 diff=0
05:01:03:WU03:FS00:0xa4:- Digital signature verified
05:01:03:WU03:FS00:0xa4:
05:01:03:WU03:FS00:0xa4:Project: 8690 (Run 0, Clone 832, Gen 40)
05:01:03:WU03:FS00:0xa4:
05:01:03:WU03:FS00:0xa4:Entering M.D.
05:01:09:WU03:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
05:01:53:WU03:FS00:Starting
05:01:53:WU03:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/Linux/AMD64/Core_a4.fah/FahCore_a4 -dir 03 -suffix 01 -version 705 -lifeline 1121 -checkpoint 15 -np 1
05:01:53:WU03:FS00:Started FahCore on PID 2573
05:01:53:WU03:FS00:Core PID:2577
05:01:53:WU03:FS00:FahCore 0xa4 started
05:01:54:WU03:FS00:0xa4:
05:01:54:WU03:FS00:0xa4:*------------------------------*
05:01:54:WU03:FS00:0xa4:Folding@Home Gromacs GB Core
05:01:54:WU03:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
05:01:54:WU03:FS00:0xa4:
05:01:54:WU03:FS00:0xa4:Preparing to commence simulation
05:01:54:WU03:FS00:0xa4:- Ensuring status. Please wait.
05:02:03:WU03:FS00:0xa4:- Looking at optimizations...
05:02:03:WU03:FS00:0xa4:- Working with standard loops on this execution.
05:02:03:WU03:FS00:0xa4:- Previous termination of core was improper.
05:02:03:WU03:FS00:0xa4:- Going to use standard loops.
05:02:03:WU03:FS00:0xa4:- Files status OK
05:02:03:WU03:FS00:0xa4:- Expanded 127372 -> 264024 (decompressed 207.2 percent)
05:02:03:WU03:FS00:0xa4:Called DecompressByteArray: compressed_data_size=127372 data_size=264024, decompressed_data_size=264024 diff=0
05:02:03:WU03:FS00:0xa4:- Digital signature verified
05:02:03:WU03:FS00:0xa4:
05:02:03:WU03:FS00:0xa4:Project: 8690 (Run 0, Clone 832, Gen 40)
05:02:03:WU03:FS00:0xa4:
05:02:03:WU03:FS00:0xa4:Entering M.D.
05:02:09:WU03:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
05:02:53:WU03:FS00:Starting
05:02:53:WU03:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/Linux/AMD64/Core_a4.fah/FahCore_a4 -dir 03 -suffix 01 -version 705 -lifeline 1121 -checkpoint 15 -np 1
05:02:53:WU03:FS00:Started FahCore on PID 2583
05:02:53:WU03:FS00:Core PID:2587
05:02:53:WU03:FS00:FahCore 0xa4 started
05:02:54:WU03:FS00:0xa4:
05:02:54:WU03:FS00:0xa4:*------------------------------*
05:02:54:WU03:FS00:0xa4:Folding@Home Gromacs GB Core
05:02:54:WU03:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
05:02:54:WU03:FS00:0xa4:
05:02:54:WU03:FS00:0xa4:Preparing to commence simulation
05:02:54:WU03:FS00:0xa4:- Ensuring status. Please wait.
05:03:03:WU03:FS00:0xa4:- Looking at optimizations...
05:03:03:WU03:FS00:0xa4:- Working with standard loops on this execution.
05:03:03:WU03:FS00:0xa4:- Previous termination of core was improper.
05:03:03:WU03:FS00:0xa4:- Going to use standard loops.
05:03:03:WU03:FS00:0xa4:- Files status OK
05:03:03:WU03:FS00:0xa4:- Expanded 127372 -> 264024 (decompressed 207.2 percent)
05:03:03:WU03:FS00:0xa4:Called DecompressByteArray: compressed_data_size=127372 data_size=264024, decompressed_data_size=264024 diff=0
05:03:03:WU03:FS00:0xa4:- Digital signature verified
05:03:03:WU03:FS00:0xa4:
05:03:03:WU03:FS00:0xa4:Project: 8690 (Run 0, Clone 832, Gen 40)
05:03:03:WU03:FS00:0xa4:
05:03:03:WU03:FS00:0xa4:Entering M.D.
05:03:09:WU03:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
05:03:53:WU03:FS00:Starting
05:03:53:WU03:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/Linux/AMD64/Core_a4.fah/FahCore_a4 -dir 03 -suffix 01 -version 705 -lifeline 1121 -checkpoint 15 -np 1
05:03:53:WU03:FS00:Started FahCore on PID 2605
05:03:53:WU03:FS00:Core PID:2609
05:03:53:WU03:FS00:FahCore 0xa4 started
05:03:54:WU03:FS00:0xa4:
05:03:54:WU03:FS00:0xa4:*------------------------------*
05:03:54:WU03:FS00:0xa4:Folding@Home Gromacs GB Core
05:03:54:WU03:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
05:03:54:WU03:FS00:0xa4:
05:03:54:WU03:FS00:0xa4:Preparing to commence simulation
05:03:54:WU03:FS00:0xa4:- Ensuring status. Please wait.
05:04:03:WU03:FS00:0xa4:- Looking at optimizations...
05:04:03:WU03:FS00:0xa4:- Working with standard loops on this execution.
05:04:03:WU03:FS00:0xa4:- Previous termination of core was improper.
05:04:03:WU03:FS00:0xa4:- Going to use standard loops.
05:04:03:WU03:FS00:0xa4:- Files status OK
05:04:03:WU03:FS00:0xa4:- Expanded 127372 -> 264024 (decompressed 207.2 percent)
05:04:03:WU03:FS00:0xa4:Called DecompressByteArray: compressed_data_size=127372 data_size=264024, decompressed_data_size=264024 diff=0
05:04:03:WU03:FS00:0xa4:- Digital signature verified
05:04:03:WU03:FS00:0xa4:
05:04:03:WU03:FS00:0xa4:Project: 8690 (Run 0, Clone 832, Gen 40)
05:04:03:WU03:FS00:0xa4:
05:04:03:WU03:FS00:0xa4:Entering M.D.
05:04:09:WU03:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
05:04:53:WU03:FS00:Starting
05:04:53:WU03:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/Linux/AMD64/Core_a4.fah/FahCore_a4 -dir 03 -suffix 01 -version 705 -lifeline 1121 -checkpoint 15 -np 1
05:04:53:WU03:FS00:Started FahCore on PID 2621
05:04:53:WU03:FS00:Core PID:2625
05:04:53:WU03:FS00:FahCore 0xa4 started
05:04:54:WU03:FS00:0xa4:
05:04:54:WU03:FS00:0xa4:*------------------------------*
05:04:54:WU03:FS00:0xa4:Folding@Home Gromacs GB Core
05:04:54:WU03:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
05:04:54:WU03:FS00:0xa4:
05:04:54:WU03:FS00:0xa4:Preparing to commence simulation
05:04:54:WU03:FS00:0xa4:- Ensuring status. Please wait.
05:05:03:WU03:FS00:0xa4:- Looking at optimizations...
05:05:03:WU03:FS00:0xa4:- Working with standard loops on this execution.
05:05:03:WU03:FS00:0xa4:- Previous termination of core was improper.
05:05:03:WU03:FS00:0xa4:- Going to use standard loops.
05:05:03:WU03:FS00:0xa4:- Files status OK
05:05:03:WU03:FS00:0xa4:- Expanded 127372 -> 264024 (decompressed 207.2 percent)
05:05:03:WU03:FS00:0xa4:Called DecompressByteArray: compressed_data_size=127372 data_size=264024, decompressed_data_size=264024 diff=0
05:05:03:WU03:FS00:0xa4:- Digital signature verified
05:05:03:WU03:FS00:0xa4:
05:05:03:WU03:FS00:0xa4:Project: 8690 (Run 0, Clone 832, Gen 40)
05:05:03:WU03:FS00:0xa4:
05:05:03:WU03:FS00:0xa4:Entering M.D.
05:05:09:WU03:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
05:05:53:WU03:FS00:Starting
05:05:53:WU03:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/Linux/AMD64/Core_a4.fah/FahCore_a4 -dir 03 -suffix 01 -version 705 -lifeline 1121 -checkpoint 15 -np 1
05:05:53:WU03:FS00:Started FahCore on PID 2661
05:05:53:WU03:FS00:Core PID:2665
05:05:53:WU03:FS00:FahCore 0xa4 started
05:05:54:WU03:FS00:0xa4:
05:05:54:WU03:FS00:0xa4:*------------------------------*
05:05:54:WU03:FS00:0xa4:Folding@Home Gromacs GB Core
05:05:54:WU03:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
05:05:54:WU03:FS00:0xa4:
05:05:54:WU03:FS00:0xa4:Preparing to commence simulation
05:05:54:WU03:FS00:0xa4:- Ensuring status. Please wait.
05:06:03:WU03:FS00:0xa4:- Looking at optimizations...
05:06:03:WU03:FS00:0xa4:- Working with standard loops on this execution.
05:06:03:WU03:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
05:06:03:WU03:FS00:0xa4:- Expanded 127372 -> 264024 (decompressed 207.2 percent)
05:06:03:WU03:FS00:0xa4:Called DecompressByteArray: compressed_data_size=127372 data_size=264024, decompressed_data_size=264024 diff=0
05:06:03:WU03:FS00:0xa4:- Digital signature verified
05:06:03:WU03:FS00:0xa4:
05:06:03:WU03:FS00:0xa4:Project: 8690 (Run 0, Clone 832, Gen 40)
05:06:03:WU03:FS00:0xa4:
05:06:03:WU03:FS00:0xa4:Entering M.D.
05:06:09:WU03:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
05:06:53:WU03:FS00:Starting
05:06:53:WU03:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/Linux/AMD64/Core_a4.fah/FahCore_a4 -dir 03 -suffix 01 -version 705 -lifeline 1121 -checkpoint 15 -np 1
05:06:53:WU03:FS00:Started FahCore on PID 2673
05:06:53:WU03:FS00:Core PID:2677
05:06:53:WU03:FS00:FahCore 0xa4 started
05:06:54:WU03:FS00:0xa4:
05:06:54:WU03:FS00:0xa4:*------------------------------*
05:06:54:WU03:FS00:0xa4:Folding@Home Gromacs GB Core
05:06:54:WU03:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
05:06:54:WU03:FS00:0xa4:
05:06:54:WU03:FS00:0xa4:Preparing to commence simulation
05:06:54:WU03:FS00:0xa4:- Ensuring status. Please wait.
05:07:03:WU03:FS00:0xa4:- Looking at optimizations...
05:07:03:WU03:FS00:0xa4:- Working with standard loops on this execution.
05:07:03:WU03:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
05:07:03:WU03:FS00:0xa4:- Expanded 127372 -> 264024 (decompressed 207.2 percent)
05:07:03:WU03:FS00:0xa4:Called DecompressByteArray: compressed_data_size=127372 data_size=264024, decompressed_data_size=264024 diff=0
05:07:03:WU03:FS00:0xa4:- Digital signature verified
05:07:03:WU03:FS00:0xa4:
05:07:03:WU03:FS00:0xa4:Project: 8690 (Run 0, Clone 832, Gen 40)
05:07:03:WU03:FS00:0xa4:
05:07:03:WU03:FS00:0xa4:Entering M.D.
05:07:09:WU03:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
05:07:53:WU03:FS00:Starting
05:07:53:WU03:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/Linux/AMD64/Core_a4.fah/FahCore_a4 -dir 03 -suffix 01 -version 705 -lifeline 1121 -checkpoint 15 -np 1
05:07:53:WU03:FS00:Started FahCore on PID 2682
05:07:53:WU03:FS00:Core PID:2686
05:07:53:WU03:FS00:FahCore 0xa4 started
05:07:54:WU03:FS00:0xa4:
05:07:54:WU03:FS00:0xa4:*------------------------------*
05:07:54:WU03:FS00:0xa4:Folding@Home Gromacs GB Core
05:07:54:WU03:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
05:07:54:WU03:FS00:0xa4:
05:07:54:WU03:FS00:0xa4:Preparing to commence simulation
05:07:54:WU03:FS00:0xa4:- Ensuring status. Please wait.
05:08:03:WU03:FS00:0xa4:- Looking at optimizations...
05:08:03:WU03:FS00:0xa4:- Working with standard loops on this execution.
05:08:03:WU03:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
05:08:03:WU03:FS00:0xa4:- Expanded 127372 -> 264024 (decompressed 207.2 percent)
05:08:03:WU03:FS00:0xa4:Called DecompressByteArray: compressed_data_size=127372 data_size=264024, decompressed_data_size=264024 diff=0
05:08:03:WU03:FS00:0xa4:- Digital signature verified
05:08:03:WU03:FS00:0xa4:
05:08:03:WU03:FS00:0xa4:Project: 8690 (Run 0, Clone 832, Gen 40)
05:08:03:WU03:FS00:0xa4:
05:08:03:WU03:FS00:0xa4:Entering M.D.
05:08:09:WU03:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
05:08:53:WU03:FS00:Starting
05:08:53:WU03:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/Linux/AMD64/Core_a4.fah/FahCore_a4 -dir 03 -suffix 01 -version 705 -lifeline 1121 -checkpoint 15 -np 1
05:08:53:WU03:FS00:Started FahCore on PID 2699
05:08:53:WU03:FS00:Core PID:2703
05:08:53:WU03:FS00:FahCore 0xa4 started
05:08:54:WU03:FS00:0xa4:
05:08:54:WU03:FS00:0xa4:*------------------------------*
05:08:54:WU03:FS00:0xa4:Folding@Home Gromacs GB Core
05:08:54:WU03:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
05:08:54:WU03:FS00:0xa4:
05:08:54:WU03:FS00:0xa4:Preparing to commence simulation
05:08:54:WU03:FS00:0xa4:- Ensuring status. Please wait.
05:09:03:WU03:FS00:0xa4:- Looking at optimizations...
05:09:03:WU03:FS00:0xa4:- Working with standard loops on this execution.
05:09:03:WU03:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
05:09:03:WU03:FS00:0xa4:- Expanded 127372 -> 264024 (decompressed 207.2 percent)
05:09:03:WU03:FS00:0xa4:Called DecompressByteArray: compressed_data_size=127372 data_size=264024, decompressed_data_size=264024 diff=0
05:09:03:WU03:FS00:0xa4:- Digital signature verified
05:09:03:WU03:FS00:0xa4:
05:09:03:WU03:FS00:0xa4:Project: 8690 (Run 0, Clone 832, Gen 40)
05:09:03:WU03:FS00:0xa4:
05:09:03:WU03:FS00:0xa4:Entering M.D.
05:09:09:WU03:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
05:09:53:WU03:FS00:Starting
05:09:53:WU03:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/Linux/AMD64/Core_a4.fah/FahCore_a4 -dir 03 -suffix 01 -version 705 -lifeline 1121 -checkpoint 15 -np 1
05:09:53:WU03:FS00:Started FahCore on PID 2709
05:09:53:WU03:FS00:Core PID:2713
05:09:53:WU03:FS00:FahCore 0xa4 started
05:09:54:WU03:FS00:0xa4:
05:09:54:WU03:FS00:0xa4:*------------------------------*
05:09:54:WU03:FS00:0xa4:Folding@Home Gromacs GB Core
05:09:54:WU03:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
05:09:54:WU03:FS00:0xa4:
05:09:54:WU03:FS00:0xa4:Preparing to commence simulation
05:09:54:WU03:FS00:0xa4:- Ensuring status. Please wait.
05:10:03:WU03:FS00:0xa4:- Looking at optimizations...
05:10:03:WU03:FS00:0xa4:- Working with standard loops on this execution.
05:10:03:WU03:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
05:10:03:WU03:FS00:0xa4:- Expanded 127372 -> 264024 (decompressed 207.2 percent)
05:10:03:WU03:FS00:0xa4:Called DecompressByteArray: compressed_data_size=127372 data_size=264024, decompressed_data_size=264024 diff=0
05:10:03:WU03:FS00:0xa4:- Digital signature verified
05:10:03:WU03:FS00:0xa4:
05:10:03:WU03:FS00:0xa4:Project: 8690 (Run 0, Clone 832, Gen 40)
05:10:03:WU03:FS00:0xa4:
05:10:03:WU03:FS00:0xa4:Entering M.D.
05:10:09:WU03:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
05:10:53:WU03:FS00:Starting
05:10:53:WU03:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/Linux/AMD64/Core_a4.fah/FahCore_a4 -dir 03 -suffix 01 -version 705 -lifeline 1121 -checkpoint 15 -np 1
05:10:53:WU03:FS00:Started FahCore on PID 2719
05:10:53:WU03:FS00:Core PID:2723
05:10:53:WU03:FS00:FahCore 0xa4 started
05:10:54:WU03:FS00:0xa4:
05:10:54:WU03:FS00:0xa4:*------------------------------*
05:10:54:WU03:FS00:0xa4:Folding@Home Gromacs GB Core
05:10:54:WU03:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
05:10:54:WU03:FS00:0xa4:
05:10:54:WU03:FS00:0xa4:Preparing to commence simulation
05:10:54:WU03:FS00:0xa4:- Ensuring status. Please wait.
05:11:03:WU03:FS00:0xa4:- Looking at optimizations...
05:11:03:WU03:FS00:0xa4:- Working with standard loops on this execution.
05:11:03:WU03:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
05:11:03:WU03:FS00:0xa4:- Expanded 127372 -> 264024 (decompressed 207.2 percent)
05:11:03:WU03:FS00:0xa4:Called DecompressByteArray: compressed_data_size=127372 data_size=264024, decompressed_data_size=264024 diff=0
05:11:03:WU03:FS00:0xa4:- Digital signature verified
05:11:03:WU03:FS00:0xa4:
05:11:03:WU03:FS00:0xa4:Project: 8690 (Run 0, Clone 832, Gen 40)
05:11:03:WU03:FS00:0xa4:
05:11:03:WU03:FS00:0xa4:Entering M.D.
05:11:09:WU03:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
05:11:53:WU03:FS00:Starting
05:11:53:WU03:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/Linux/AMD64/Core_a4.fah/FahCore_a4 -dir 03 -suffix 01 -version 705 -lifeline 1121 -checkpoint 15 -np 1
05:11:54:WU03:FS00:Started FahCore on PID 2727
05:11:54:WU03:FS00:Core PID:2731
05:11:54:WU03:FS00:FahCore 0xa4 started
05:11:54:WU03:FS00:0xa4:
05:11:54:WU03:FS00:0xa4:*------------------------------*
05:11:54:WU03:FS00:0xa4:Folding@Home Gromacs GB Core
05:11:54:WU03:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
05:11:54:WU03:FS00:0xa4:
05:11:54:WU03:FS00:0xa4:Preparing to commence simulation
05:11:54:WU03:FS00:0xa4:- Ensuring status. Please wait.
05:12:03:WU03:FS00:0xa4:- Looking at optimizations...
05:12:03:WU03:FS00:0xa4:- Working with standard loops on this execution.
05:12:03:WU03:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
05:12:03:WU03:FS00:0xa4:- Expanded 127372 -> 264024 (decompressed 207.2 percent)
05:12:03:WU03:FS00:0xa4:Called DecompressByteArray: compressed_data_size=127372 data_size=264024, decompressed_data_size=264024 diff=0
05:12:03:WU03:FS00:0xa4:- Digital signature verified
05:12:03:WU03:FS00:0xa4:
05:12:03:WU03:FS00:0xa4:Project: 8690 (Run 0, Clone 832, Gen 40)
05:12:03:WU03:FS00:0xa4:
05:12:03:WU03:FS00:0xa4:Entering M.D.
05:12:10:WU03:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
05:12:54:WU03:FS00:Starting
05:12:54:WU03:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/Linux/AMD64/Core_a4.fah/FahCore_a4 -dir 03 -suffix 01 -version 705 -lifeline 1121 -checkpoint 15 -np 1
05:12:54:WU03:FS00:Started FahCore on PID 2736
05:12:54:WU03:FS00:Core PID:2740
05:12:54:WU03:FS00:FahCore 0xa4 started
05:12:54:WU03:FS00:0xa4:
05:12:54:WU03:FS00:0xa4:*------------------------------*
05:12:54:WU03:FS00:0xa4:Folding@Home Gromacs GB Core
05:12:54:WU03:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
05:12:54:WU03:FS00:0xa4:
05:12:54:WU03:FS00:0xa4:Preparing to commence simulation
05:12:54:WU03:FS00:0xa4:- Ensuring status. Please wait.
05:13:03:WU03:FS00:0xa4:- Looking at optimizations...
05:13:03:WU03:FS00:0xa4:- Working with standard loops on this execution.
05:13:03:WU03:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
05:13:03:WU03:FS00:0xa4:- Expanded 127372 -> 264024 (decompressed 207.2 percent)
05:13:03:WU03:FS00:0xa4:Called DecompressByteArray: compressed_data_size=127372 data_size=264024, decompressed_data_size=264024 diff=0
05:13:03:WU03:FS00:0xa4:- Digital signature verified
05:13:03:WU03:FS00:0xa4:
05:13:03:WU03:FS00:0xa4:Project: 8690 (Run 0, Clone 832, Gen 40)
05:13:03:WU03:FS00:0xa4:
05:13:03:WU03:FS00:0xa4:Entering M.D.
05:13:10:WU03:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
05:13:54:WU03:FS00:Starting
05:13:54:WU03:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/Linux/AMD64/Core_a4.fah/FahCore_a4 -dir 03 -suffix 01 -version 705 -lifeline 1121 -checkpoint 15 -np 1
05:13:54:WU03:FS00:Started FahCore on PID 2745
05:13:54:WU03:FS00:Core PID:2749
05:13:54:WU03:FS00:FahCore 0xa4 started
05:13:54:WU03:FS00:0xa4:
05:13:54:WU03:FS00:0xa4:*------------------------------*
05:13:54:WU03:FS00:0xa4:Folding@Home Gromacs GB Core
05:13:54:WU03:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
05:13:54:WU03:FS00:0xa4:
05:13:54:WU03:FS00:0xa4:Preparing to commence simulation
05:13:54:WU03:FS00:0xa4:- Ensuring status. Please wait.
05:14:03:WU03:FS00:0xa4:- Looking at optimizations...
05:14:03:WU03:FS00:0xa4:- Working with standard loops on this execution.
05:14:03:WU03:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
05:14:03:WU03:FS00:0xa4:- Expanded 127372 -> 264024 (decompressed 207.2 percent)
05:14:03:WU03:FS00:0xa4:Called DecompressByteArray: compressed_data_size=127372 data_size=264024, decompressed_data_size=264024 diff=0
05:14:03:WU03:FS00:0xa4:- Digital signature verified
05:14:03:WU03:FS00:0xa4:
05:14:03:WU03:FS00:0xa4:Project: 8690 (Run 0, Clone 832, Gen 40)
05:14:03:WU03:FS00:0xa4:
05:14:03:WU03:FS00:0xa4:Entering M.D.
05:14:10:WU03:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
05:14:54:WU03:FS00:Starting
05:14:54:WU03:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/Linux/AMD64/Core_a4.fah/FahCore_a4 -dir 03 -suffix 01 -version 705 -lifeline 1121 -checkpoint 15 -np 1
05:14:54:WU03:FS00:Started FahCore on PID 2753
05:14:54:WU03:FS00:Core PID:2757
05:14:54:WU03:FS00:FahCore 0xa4 started
05:14:54:WU03:FS00:0xa4:
05:14:54:WU03:FS00:0xa4:*------------------------------*
05:14:54:WU03:FS00:0xa4:Folding@Home Gromacs GB Core
05:14:54:WU03:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
05:14:54:WU03:FS00:0xa4:
05:14:54:WU03:FS00:0xa4:Preparing to commence simulation
05:14:54:WU03:FS00:0xa4:- Ensuring status. Please wait.
05:15:03:WU03:FS00:0xa4:- Looking at optimizations...
05:15:03:WU03:FS00:0xa4:- Working with standard loops on this execution.
05:15:03:WU03:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
05:15:03:WU03:FS00:0xa4:- Expanded 127372 -> 264024 (decompressed 207.2 percent)
05:15:03:WU03:FS00:0xa4:Called DecompressByteArray: compressed_data_size=127372 data_size=264024, decompressed_data_size=264024 diff=0
05:15:03:WU03:FS00:0xa4:- Digital signature verified
05:15:03:WU03:FS00:0xa4:
05:15:03:WU03:FS00:0xa4:Project: 8690 (Run 0, Clone 832, Gen 40)
05:15:03:WU03:FS00:0xa4:
05:15:03:WU03:FS00:0xa4:Entering M.D.
05:15:10:WU03:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
05:15:54:WU03:FS00:Starting
05:15:54:WU03:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/Linux/AMD64/Core_a4.fah/FahCore_a4 -dir 03 -suffix 01 -version 705 -lifeline 1121 -checkpoint 15 -np 1
05:15:54:WU03:FS00:Started FahCore on PID 2764
05:15:54:WU03:FS00:Core PID:2768
05:15:54:WU03:FS00:FahCore 0xa4 started
05:15:54:WU03:FS00:0xa4:
05:15:54:WU03:FS00:0xa4:*------------------------------*
05:15:54:WU03:FS00:0xa4:Folding@Home Gromacs GB Core
05:15:54:WU03:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
05:15:54:WU03:FS00:0xa4:
05:15:54:WU03:FS00:0xa4:Preparing to commence simulation
05:15:54:WU03:FS00:0xa4:- Ensuring status. Please wait.
05:16:03:WU03:FS00:0xa4:- Looking at optimizations...
05:16:03:WU03:FS00:0xa4:- Working with standard loops on this execution.
05:16:03:WU03:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
05:16:03:WU03:FS00:0xa4:- Expanded 127372 -> 264024 (decompressed 207.2 percent)
05:16:03:WU03:FS00:0xa4:Called DecompressByteArray: compressed_data_size=127372 data_size=264024, decompressed_data_size=264024 diff=0
05:16:03:WU03:FS00:0xa4:- Digital signature verified
05:16:03:WU03:FS00:0xa4:
05:16:03:WU03:FS00:0xa4:Project: 8690 (Run 0, Clone 832, Gen 40)
05:16:03:WU03:FS00:0xa4:
05:16:03:WU03:FS00:0xa4:Entering M.D.
05:16:10:WU03:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
05:16:54:WU03:FS00:Starting
05:16:54:WU03:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/Linux/AMD64/Core_a4.fah/FahCore_a4 -dir 03 -suffix 01 -version 705 -lifeline 1121 -checkpoint 15 -np 1
05:16:54:WU03:FS00:Started FahCore on PID 2775
05:16:54:WU03:FS00:Core PID:2779
05:16:54:WU03:FS00:FahCore 0xa4 started
05:16:54:WU03:FS00:0xa4:
05:16:54:WU03:FS00:0xa4:*------------------------------*
05:16:54:WU03:FS00:0xa4:Folding@Home Gromacs GB Core
05:16:54:WU03:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
05:16:54:WU03:FS00:0xa4:
05:16:54:WU03:FS00:0xa4:Preparing to commence simulation
05:16:54:WU03:FS00:0xa4:- Ensuring status. Please wait.
05:17:03:WU03:FS00:0xa4:- Looking at optimizations...
05:17:03:WU03:FS00:0xa4:- Working with standard loops on this execution.
05:17:03:WU03:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
05:17:03:WU03:FS00:0xa4:- Expanded 127372 -> 264024 (decompressed 207.2 percent)
05:17:03:WU03:FS00:0xa4:Called DecompressByteArray: compressed_data_size=127372 data_size=264024, decompressed_data_size=264024 diff=0
05:17:03:WU03:FS00:0xa4:- Digital signature verified
05:17:03:WU03:FS00:0xa4:
05:17:03:WU03:FS00:0xa4:Project: 8690 (Run 0, Clone 832, Gen 40)
05:17:03:WU03:FS00:0xa4:
05:17:03:WU03:FS00:0xa4:Entering M.D.
05:17:10:WU03:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
05:17:54:WU03:FS00:Starting
05:17:54:WU03:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/Linux/AMD64/Core_a4.fah/FahCore_a4 -dir 03 -suffix 01 -version 705 -lifeline 1121 -checkpoint 15 -np 1
05:17:54:WU03:FS00:Started FahCore on PID 2787
05:17:54:WU03:FS00:Core PID:2791
05:17:54:WU03:FS00:FahCore 0xa4 started
05:17:54:WU03:FS00:0xa4:
05:17:54:WU03:FS00:0xa4:*------------------------------*
05:17:54:WU03:FS00:0xa4:Folding@Home Gromacs GB Core
05:17:54:WU03:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
05:17:54:WU03:FS00:0xa4:
05:17:54:WU03:FS00:0xa4:Preparing to commence simulation
05:17:54:WU03:FS00:0xa4:- Ensuring status. Please wait.
05:18:03:WU03:FS00:0xa4:- Looking at optimizations...
05:18:03:WU03:FS00:0xa4:- Working with standard loops on this execution.
05:18:03:WU03:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
05:18:03:WU03:FS00:0xa4:- Expanded 127372 -> 264024 (decompressed 207.2 percent)
05:18:03:WU03:FS00:0xa4:Called DecompressByteArray: compressed_data_size=127372 data_size=264024, decompressed_data_size=264024 diff=0
05:18:03:WU03:FS00:0xa4:- Digital signature verified
05:18:03:WU03:FS00:0xa4:
05:18:03:WU03:FS00:0xa4:Project: 8690 (Run 0, Clone 832, Gen 40)
05:18:03:WU03:FS00:0xa4:
05:18:03:WU03:FS00:0xa4:Entering M.D.
05:18:10:WU03:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
05:18:54:WU03:FS00:Starting
05:18:54:WU03:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/Linux/AMD64/Core_a4.fah/FahCore_a4 -dir 03 -suffix 01 -version 705 -lifeline 1121 -checkpoint 15 -np 1
05:18:54:WU03:FS00:Started FahCore on PID 2795
05:18:54:WU03:FS00:Core PID:2799
05:18:54:WU03:FS00:FahCore 0xa4 started
05:18:54:WU03:FS00:0xa4:
05:18:54:WU03:FS00:0xa4:*------------------------------*
05:18:54:WU03:FS00:0xa4:Folding@Home Gromacs GB Core
05:18:54:WU03:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
05:18:54:WU03:FS00:0xa4:
05:18:54:WU03:FS00:0xa4:Preparing to commence simulation
05:18:54:WU03:FS00:0xa4:- Ensuring status. Please wait.
05:19:03:WU03:FS00:0xa4:- Looking at optimizations...
05:19:03:WU03:FS00:0xa4:- Working with standard loops on this execution.
05:19:03:WU03:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
05:19:03:WU03:FS00:0xa4:- Expanded 127372 -> 264024 (decompressed 207.2 percent)
05:19:03:WU03:FS00:0xa4:Called DecompressByteArray: compressed_data_size=127372 data_size=264024, decompressed_data_size=264024 diff=0
05:19:03:WU03:FS00:0xa4:- Digital signature verified
05:19:03:WU03:FS00:0xa4:
05:19:03:WU03:FS00:0xa4:Project: 8690 (Run 0, Clone 832, Gen 40)
05:19:03:WU03:FS00:0xa4:
05:19:03:WU03:FS00:0xa4:Entering M.D.
05:19:10:WU03:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
05:19:54:WU03:FS00:Starting
05:19:54:WU03:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/Linux/AMD64/Core_a4.fah/FahCore_a4 -dir 03 -suffix 01 -version 705 -lifeline 1121 -checkpoint 15 -np 1
05:19:54:WU03:FS00:Started FahCore on PID 2807
05:19:54:WU03:FS00:Core PID:2811
05:19:54:WU03:FS00:FahCore 0xa4 started
05:19:54:WU03:FS00:0xa4:
05:19:54:WU03:FS00:0xa4:*------------------------------*
05:19:54:WU03:FS00:0xa4:Folding@Home Gromacs GB Core
05:19:54:WU03:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
05:19:54:WU03:FS00:0xa4:
05:19:54:WU03:FS00:0xa4:Preparing to commence simulation
05:19:54:WU03:FS00:0xa4:- Ensuring status. Please wait.
05:20:03:WU03:FS00:0xa4:- Looking at optimizations...
05:20:03:WU03:FS00:0xa4:- Working with standard loops on this execution.
05:20:03:WU03:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
05:20:03:WU03:FS00:0xa4:- Expanded 127372 -> 264024 (decompressed 207.2 percent)
05:20:03:WU03:FS00:0xa4:Called DecompressByteArray: compressed_data_size=127372 data_size=264024, decompressed_data_size=264024 diff=0
05:20:03:WU03:FS00:0xa4:- Digital signature verified
05:20:03:WU03:FS00:0xa4:
05:20:03:WU03:FS00:0xa4:Project: 8690 (Run 0, Clone 832, Gen 40)
05:20:03:WU03:FS00:0xa4:
05:20:03:WU03:FS00:0xa4:Entering M.D.
05:20:10:WU03:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
05:20:54:WU03:FS00:Starting
05:20:54:WU03:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/Linux/AMD64/Core_a4.fah/FahCore_a4 -dir 03 -suffix 01 -version 705 -lifeline 1121 -checkpoint 15 -np 1
05:20:54:WU03:FS00:Started FahCore on PID 2818
05:20:54:WU03:FS00:Core PID:2822
05:20:54:WU03:FS00:FahCore 0xa4 started
05:20:54:WU03:FS00:0xa4:
05:20:54:WU03:FS00:0xa4:*------------------------------*
05:20:54:WU03:FS00:0xa4:Folding@Home Gromacs GB Core
05:20:54:WU03:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
05:20:54:WU03:FS00:0xa4:
05:20:54:WU03:FS00:0xa4:Preparing to commence simulation
05:20:54:WU03:FS00:0xa4:- Ensuring status. Please wait.
05:21:03:WU03:FS00:0xa4:- Looking at optimizations...
05:21:03:WU03:FS00:0xa4:- Working with standard loops on this execution.
05:21:03:WU03:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
05:21:03:WU03:FS00:0xa4:- Expanded 127372 -> 264024 (decompressed 207.2 percent)
05:21:03:WU03:FS00:0xa4:Called DecompressByteArray: compressed_data_size=127372 data_size=264024, decompressed_data_size=264024 diff=0
05:21:03:WU03:FS00:0xa4:- Digital signature verified
05:21:03:WU03:FS00:0xa4:
05:21:03:WU03:FS00:0xa4:Project: 8690 (Run 0, Clone 832, Gen 40)
05:21:03:WU03:FS00:0xa4:
05:21:03:WU03:FS00:0xa4:Entering M.D.
05:21:10:WU03:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
05:21:54:WU03:FS00:Starting
05:21:54:WU03:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/Linux/AMD64/Core_a4.fah/FahCore_a4 -dir 03 -suffix 01 -version 705 -lifeline 1121 -checkpoint 15 -np 1
05:21:54:WU03:FS00:Started FahCore on PID 2827
05:21:54:WU03:FS00:Core PID:2831
05:21:54:WU03:FS00:FahCore 0xa4 started
05:21:54:WU03:FS00:0xa4:
05:21:54:WU03:FS00:0xa4:*------------------------------*
05:21:54:WU03:FS00:0xa4:Folding@Home Gromacs GB Core
05:21:54:WU03:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
05:21:54:WU03:FS00:0xa4:
05:21:54:WU03:FS00:0xa4:Preparing to commence simulation
05:21:54:WU03:FS00:0xa4:- Ensuring status. Please wait.
05:22:03:WU03:FS00:0xa4:- Looking at optimizations...
05:22:03:WU03:FS00:0xa4:- Working with standard loops on this execution.
05:22:03:WU03:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
05:22:03:WU03:FS00:0xa4:- Expanded 127372 -> 264024 (decompressed 207.2 percent)
05:22:03:WU03:FS00:0xa4:Called DecompressByteArray: compressed_data_size=127372 data_size=264024, decompressed_data_size=264024 diff=0
05:22:03:WU03:FS00:0xa4:- Digital signature verified
05:22:03:WU03:FS00:0xa4:
05:22:03:WU03:FS00:0xa4:Project: 8690 (Run 0, Clone 832, Gen 40)
05:22:03:WU03:FS00:0xa4:
05:22:03:WU03:FS00:0xa4:Entering M.D.
05:22:10:WU03:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
05:22:54:WU03:FS00:Starting
05:22:54:WU03:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/Linux/AMD64/Core_a4.fah/FahCore_a4 -dir 03 -suffix 01 -version 705 -lifeline 1121 -checkpoint 15 -np 1
05:22:54:WU03:FS00:Started FahCore on PID 2835
05:22:54:WU03:FS00:Core PID:2839
05:22:54:WU03:FS00:FahCore 0xa4 started
05:22:54:WU03:FS00:0xa4:
05:22:54:WU03:FS00:0xa4:*------------------------------*
05:22:54:WU03:FS00:0xa4:Folding@Home Gromacs GB Core
05:22:54:WU03:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
05:22:54:WU03:FS00:0xa4:
05:22:54:WU03:FS00:0xa4:Preparing to commence simulation
05:22:54:WU03:FS00:0xa4:- Ensuring status. Please wait.
05:23:03:WU03:FS00:0xa4:- Looking at optimizations...
05:23:03:WU03:FS00:0xa4:- Working with standard loops on this execution.
05:23:03:WU03:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
05:23:03:WU03:FS00:0xa4:- Expanded 127372 -> 264024 (decompressed 207.2 percent)
05:23:03:WU03:FS00:0xa4:Called DecompressByteArray: compressed_data_size=127372 data_size=264024, decompressed_data_size=264024 diff=0
05:23:03:WU03:FS00:0xa4:- Digital signature verified
05:23:03:WU03:FS00:0xa4:
05:23:03:WU03:FS00:0xa4:Project: 8690 (Run 0, Clone 832, Gen 40)
05:23:03:WU03:FS00:0xa4:
05:23:03:WU03:FS00:0xa4:Entering M.D.
05:23:10:WU03:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
05:23:54:WU03:FS00:Starting
05:23:54:WU03:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/Linux/AMD64/Core_a4.fah/FahCore_a4 -dir 03 -suffix 01 -version 705 -lifeline 1121 -checkpoint 15 -np 1
05:23:54:WU03:FS00:Started FahCore on PID 2879
05:23:54:WU03:FS00:Core PID:2883
05:23:54:WU03:FS00:FahCore 0xa4 started
05:23:54:WU03:FS00:0xa4:
05:23:54:WU03:FS00:0xa4:*------------------------------*
05:23:54:WU03:FS00:0xa4:Folding@Home Gromacs GB Core
05:23:54:WU03:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
05:23:54:WU03:FS00:0xa4:
05:23:54:WU03:FS00:0xa4:Preparing to commence simulation
05:23:54:WU03:FS00:0xa4:- Ensuring status. Please wait.
05:24:03:WU03:FS00:0xa4:- Looking at optimizations...
05:24:03:WU03:FS00:0xa4:- Working with standard loops on this execution.
05:24:03:WU03:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
05:24:03:WU03:FS00:0xa4:- Expanded 127372 -> 264024 (decompressed 207.2 percent)
05:24:03:WU03:FS00:0xa4:Called DecompressByteArray: compressed_data_size=127372 data_size=264024, decompressed_data_size=264024 diff=0
05:24:03:WU03:FS00:0xa4:- Digital signature verified
05:24:03:WU03:FS00:0xa4:
05:24:03:WU03:FS00:0xa4:Project: 8690 (Run 0, Clone 832, Gen 40)
05:24:03:WU03:FS00:0xa4:
05:24:03:WU03:FS00:0xa4:Entering M.D.
05:24:10:WU03:FS00:FahCore returned: INTERRUPTED (102 = 0x66)

I then configured the kernel to enable vsyscall and rebooted. After that, this 8690 WU ran successfully on my hardware:

Code: Select all

$ grep WU03 log-20190804-165746.txt
05:25:40:WU03:FS00:Starting
05:25:40:WU03:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/Linux/AMD64/Core_a4.fah/FahCore_a4 -dir 03 -suffix 01 -version 705 -lifeline 1093 -checkpoint 15 -np 1
05:25:40:WU03:FS00:Started FahCore on PID 1134
05:25:40:WU03:FS00:Core PID:1139
05:25:40:WU03:FS00:FahCore 0xa4 started
05:25:40:WU03:FS00:0xa4:
05:25:40:WU03:FS00:0xa4:*------------------------------*
05:25:40:WU03:FS00:0xa4:Folding@Home Gromacs GB Core
05:25:40:WU03:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
05:25:40:WU03:FS00:0xa4:
05:25:40:WU03:FS00:0xa4:Preparing to commence simulation
05:25:40:WU03:FS00:0xa4:- Ensuring status. Please wait.
05:25:49:WU03:FS00:0xa4:- Looking at optimizations...
05:25:49:WU03:FS00:0xa4:- Working with standard loops on this execution.
05:25:49:WU03:FS00:0xa4:Examination of work files indicates 8 consecutive improper terminations of core.
05:25:49:WU03:FS00:0xa4:- Expanded 127372 -> 264024 (decompressed 207.2 percent)
05:25:49:WU03:FS00:0xa4:Called DecompressByteArray: compressed_data_size=127372 data_size=264024, decompressed_data_size=264024 diff=0
05:25:49:WU03:FS00:0xa4:- Digital signature verified
05:25:49:WU03:FS00:0xa4:
05:25:49:WU03:FS00:0xa4:Project: 8690 (Run 0, Clone 832, Gen 40)
05:25:49:WU03:FS00:0xa4:
05:25:49:WU03:FS00:0xa4:Entering M.D.
05:25:55:WU03:FS00:0xa4:Completed 0 out of 2500000 steps  (0%)
05:32:57:WU03:FS00:0xa4:Completed 25000 out of 2500000 steps  (1%)
05:40:32:WU03:FS00:0xa4:Completed 50000 out of 2500000 steps  (2%)
05:47:34:WU03:FS00:0xa4:Completed 75000 out of 2500000 steps  (3%)
05:54:35:WU03:FS00:0xa4:Completed 100000 out of 2500000 steps  (4%)
06:01:37:WU03:FS00:0xa4:Completed 125000 out of 2500000 steps  (5%)
06:08:38:WU03:FS00:0xa4:Completed 150000 out of 2500000 steps  (6%)
06:15:39:WU03:FS00:0xa4:Completed 175000 out of 2500000 steps  (7%)
06:23:10:WU03:FS00:0xa4:Completed 200000 out of 2500000 steps  (8%)
06:30:11:WU03:FS00:0xa4:Completed 225000 out of 2500000 steps  (9%)
06:37:13:WU03:FS00:0xa4:Completed 250000 out of 2500000 steps  (10%)
06:44:30:WU03:FS00:0xa4:Completed 275000 out of 2500000 steps  (11%)
06:51:44:WU03:FS00:0xa4:Completed 300000 out of 2500000 steps  (12%)
06:58:45:WU03:FS00:0xa4:Completed 325000 out of 2500000 steps  (13%)
07:05:46:WU03:FS00:0xa4:Completed 350000 out of 2500000 steps  (14%)
07:13:16:WU03:FS00:0xa4:Completed 375000 out of 2500000 steps  (15%)
07:20:46:WU03:FS00:0xa4:Completed 400000 out of 2500000 steps  (16%)
07:27:48:WU03:FS00:0xa4:Completed 425000 out of 2500000 steps  (17%)
07:35:12:WU03:FS00:0xa4:Completed 450000 out of 2500000 steps  (18%)
07:42:44:WU03:FS00:0xa4:Completed 475000 out of 2500000 steps  (19%)
07:49:53:WU03:FS00:0xa4:Completed 500000 out of 2500000 steps  (20%)
07:56:54:WU03:FS00:0xa4:Completed 525000 out of 2500000 steps  (21%)
08:05:43:WU03:FS00:0xa4:Completed 550000 out of 2500000 steps  (22%)
08:15:32:WU03:FS00:0xa4:Completed 575000 out of 2500000 steps  (23%)
08:24:33:WU03:FS00:0xa4:Completed 600000 out of 2500000 steps  (24%)
08:33:33:WU03:FS00:0xa4:Completed 625000 out of 2500000 steps  (25%)
08:43:16:WU03:FS00:0xa4:Completed 650000 out of 2500000 steps  (26%)
08:52:22:WU03:FS00:0xa4:Completed 675000 out of 2500000 steps  (27%)
09:01:33:WU03:FS00:0xa4:Completed 700000 out of 2500000 steps  (28%)
09:11:20:WU03:FS00:0xa4:Completed 725000 out of 2500000 steps  (29%)
09:20:20:WU03:FS00:0xa4:Completed 750000 out of 2500000 steps  (30%)
09:29:14:WU03:FS00:0xa4:Completed 775000 out of 2500000 steps  (31%)
09:38:27:WU03:FS00:0xa4:Completed 800000 out of 2500000 steps  (32%)
09:47:35:WU03:FS00:0xa4:Completed 825000 out of 2500000 steps  (33%)
09:55:55:WU03:FS00:0xa4:Completed 850000 out of 2500000 steps  (34%)
10:04:42:WU03:FS00:0xa4:Completed 875000 out of 2500000 steps  (35%)
10:12:16:WU03:FS00:0xa4:Completed 900000 out of 2500000 steps  (36%)
10:19:40:WU03:FS00:0xa4:Completed 925000 out of 2500000 steps  (37%)
10:26:37:WU03:FS00:0xa4:Completed 950000 out of 2500000 steps  (38%)
10:34:03:WU03:FS00:0xa4:Completed 975000 out of 2500000 steps  (39%)
10:41:31:WU03:FS00:0xa4:Completed 1000000 out of 2500000 steps  (40%)
10:48:58:WU03:FS00:0xa4:Completed 1025000 out of 2500000 steps  (41%)
10:55:55:WU03:FS00:0xa4:Completed 1050000 out of 2500000 steps  (42%)
11:03:21:WU03:FS00:0xa4:Completed 1075000 out of 2500000 steps  (43%)
11:10:47:WU03:FS00:0xa4:Completed 1100000 out of 2500000 steps  (44%)
11:18:12:WU03:FS00:0xa4:Completed 1125000 out of 2500000 steps  (45%)
11:25:41:WU03:FS00:0xa4:Completed 1150000 out of 2500000 steps  (46%)
11:32:38:WU03:FS00:0xa4:Completed 1175000 out of 2500000 steps  (47%)
11:40:04:WU03:FS00:0xa4:Completed 1200000 out of 2500000 steps  (48%)
11:47:33:WU03:FS00:0xa4:Completed 1225000 out of 2500000 steps  (49%)
11:54:54:WU03:FS00:0xa4:Completed 1250000 out of 2500000 steps  (50%)
12:01:55:WU03:FS00:0xa4:Completed 1275000 out of 2500000 steps  (51%)
12:08:51:WU03:FS00:0xa4:Completed 1300000 out of 2500000 steps  (52%)
12:16:13:WU03:FS00:0xa4:Completed 1325000 out of 2500000 steps  (53%)
12:23:39:WU03:FS00:0xa4:Completed 1350000 out of 2500000 steps  (54%)
12:31:03:WU03:FS00:0xa4:Completed 1375000 out of 2500000 steps  (55%)
12:38:28:WU03:FS00:0xa4:Completed 1400000 out of 2500000 steps  (56%)
12:45:23:WU03:FS00:0xa4:Completed 1425000 out of 2500000 steps  (57%)
12:52:50:WU03:FS00:0xa4:Completed 1450000 out of 2500000 steps  (58%)
13:00:14:WU03:FS00:0xa4:Completed 1475000 out of 2500000 steps  (59%)
13:07:42:WU03:FS00:0xa4:Completed 1500000 out of 2500000 steps  (60%)
13:15:10:WU03:FS00:0xa4:Completed 1525000 out of 2500000 steps  (61%)
13:22:06:WU03:FS00:0xa4:Completed 1550000 out of 2500000 steps  (62%)
13:29:03:WU03:FS00:0xa4:Completed 1575000 out of 2500000 steps  (63%)
13:36:29:WU03:FS00:0xa4:Completed 1600000 out of 2500000 steps  (64%)
13:43:56:WU03:FS00:0xa4:Completed 1625000 out of 2500000 steps  (65%)
13:51:23:WU03:FS00:0xa4:Completed 1650000 out of 2500000 steps  (66%)
13:58:50:WU03:FS00:0xa4:Completed 1675000 out of 2500000 steps  (67%)
14:05:47:WU03:FS00:0xa4:Completed 1700000 out of 2500000 steps  (68%)
14:13:15:WU03:FS00:0xa4:Completed 1725000 out of 2500000 steps  (69%)
14:20:40:WU03:FS00:0xa4:Completed 1750000 out of 2500000 steps  (70%)
14:28:07:WU03:FS00:0xa4:Completed 1775000 out of 2500000 steps  (71%)
14:35:03:WU03:FS00:0xa4:Completed 1800000 out of 2500000 steps  (72%)
14:42:00:WU03:FS00:0xa4:Completed 1825000 out of 2500000 steps  (73%)
14:45:10:WU03:FS00:0xa4:Client no longer detected. Shutting down core.
14:45:10:WU03:FS00:0xa4:
14:45:10:WU03:FS00:0xa4:Folding@home Core Shutdown: CLIENT_DIED

Had to restart for a fsck, but it then continued and finished successfully:

Code: Select all

$ grep WU03 ../log.txt 
16:57:46:WU03:FS00:Starting
16:57:46:WU03:FS00:Running FahCore: /usr/bin/FAHCoreWrapper /var/lib/fahclient/cores/cores.foldingathome.org/Linux/AMD64/Core_a4.fah/FahCore_a4 -dir 03 -suffix 01 -version 705 -lifeline 15664 -checkpoint 15 -np 1
16:57:46:WU03:FS00:Started FahCore on PID 15697
16:57:46:WU03:FS00:Core PID:15701
16:57:46:WU03:FS00:FahCore 0xa4 started
16:57:47:WU03:FS00:0xa4:
16:57:47:WU03:FS00:0xa4:*------------------------------*
16:57:47:WU03:FS00:0xa4:Folding@Home Gromacs GB Core
16:57:47:WU03:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
16:57:47:WU03:FS00:0xa4:
16:57:47:WU03:FS00:0xa4:Preparing to commence simulation
16:57:47:WU03:FS00:0xa4:- Looking at optimizations...
16:57:47:WU03:FS00:0xa4:- Files status OK
16:57:47:WU03:FS00:0xa4:- Expanded 127372 -> 264024 (decompressed 207.2 percent)
16:57:47:WU03:FS00:0xa4:Called DecompressByteArray: compressed_data_size=127372 data_size=264024, decompressed_data_size=264024 diff=0
16:57:47:WU03:FS00:0xa4:- Digital signature verified
16:57:47:WU03:FS00:0xa4:
16:57:47:WU03:FS00:0xa4:Project: 8690 (Run 0, Clone 832, Gen 40)
16:57:47:WU03:FS00:0xa4:
16:57:47:WU03:FS00:0xa4:Assembly optimizations on if available.
16:57:47:WU03:FS00:0xa4:Entering M.D.
16:57:53:WU03:FS00:0xa4:Using Gromacs checkpoints
16:57:53:WU03:FS00:0xa4:Resuming from checkpoint
16:57:53:WU03:FS00:0xa4:Verified 03/wudata_01.log
16:57:53:WU03:FS00:0xa4:Verified 03/wudata_01.trr
16:57:53:WU03:FS00:0xa4:Verified 03/wudata_01.xtc
16:57:53:WU03:FS00:0xa4:Verified 03/wudata_01.edr
16:57:53:WU03:FS00:0xa4:Completed 1821250 out of 2500000 steps  (72%)
16:58:55:WU03:FS00:0xa4:Completed 1825000 out of 2500000 steps  (73%)
17:05:50:WU03:FS00:0xa4:Completed 1850000 out of 2500000 steps  (74%)
17:13:14:WU03:FS00:0xa4:Completed 1875000 out of 2500000 steps  (75%)
17:20:08:WU03:FS00:0xa4:Completed 1900000 out of 2500000 steps  (76%)
17:27:01:WU03:FS00:0xa4:Completed 1925000 out of 2500000 steps  (77%)
17:34:25:WU03:FS00:0xa4:Completed 1950000 out of 2500000 steps  (78%)
17:41:48:WU03:FS00:0xa4:Completed 1975000 out of 2500000 steps  (79%)
17:48:42:WU03:FS00:0xa4:Completed 2000000 out of 2500000 steps  (80%)
17:55:37:WU03:FS00:0xa4:Completed 2025000 out of 2500000 steps  (81%)
18:03:01:WU03:FS00:0xa4:Completed 2050000 out of 2500000 steps  (82%)
18:10:26:WU03:FS00:0xa4:Completed 2075000 out of 2500000 steps  (83%)
18:17:19:WU03:FS00:0xa4:Completed 2100000 out of 2500000 steps  (84%)
18:24:45:WU03:FS00:0xa4:Completed 2125000 out of 2500000 steps  (85%)
18:32:02:WU03:FS00:0xa4:Completed 2150000 out of 2500000 steps  (86%)
18:39:00:WU03:FS00:0xa4:Completed 2175000 out of 2500000 steps  (87%)
18:46:21:WU03:FS00:0xa4:Completed 2200000 out of 2500000 steps  (88%)
18:53:15:WU03:FS00:0xa4:Completed 2225000 out of 2500000 steps  (89%)
19:00:10:WU03:FS00:0xa4:Completed 2250000 out of 2500000 steps  (90%)
19:07:36:WU03:FS00:0xa4:Completed 2275000 out of 2500000 steps  (91%)
19:14:33:WU03:FS00:0xa4:Completed 2300000 out of 2500000 steps  (92%)
19:21:57:WU03:FS00:0xa4:Completed 2325000 out of 2500000 steps  (93%)
19:28:53:WU03:FS00:0xa4:Completed 2350000 out of 2500000 steps  (94%)
19:36:16:WU03:FS00:0xa4:Completed 2375000 out of 2500000 steps  (95%)
19:43:09:WU03:FS00:0xa4:Completed 2400000 out of 2500000 steps  (96%)
19:50:03:WU03:FS00:0xa4:Completed 2425000 out of 2500000 steps  (97%)
19:57:27:WU03:FS00:0xa4:Completed 2450000 out of 2500000 steps  (98%)
20:04:48:WU03:FS00:0xa4:Completed 2475000 out of 2500000 steps  (99%)
20:11:44:WU03:FS00:0xa4:Completed 2500000 out of 2500000 steps  (100%)
20:11:44:WU03:FS00:0xa4:DynamicWrapper: Finished Work Unit: sleep=10000
20:11:54:WU03:FS00:0xa4:
20:11:54:WU03:FS00:0xa4:Finished Work Unit:
20:11:54:WU03:FS00:0xa4:- Reading up to 743328 from "03/wudata_01.trr": Read 743328
20:11:54:WU03:FS00:0xa4:trr file hash check passed.
20:11:54:WU03:FS00:0xa4:- Reading up to 38628 from "03/wudata_01.xtc": Read 38628
20:11:54:WU03:FS00:0xa4:xtc file hash check passed.
20:11:54:WU03:FS00:0xa4:edr file hash check passed.
20:11:54:WU03:FS00:0xa4:logfile size: 49622
20:11:54:WU03:FS00:0xa4:Leaving Run
20:11:56:WU03:FS00:0xa4:- Writing 868646 bytes of core data to disk...
20:11:56:WU03:FS00:0xa4:Done: 868134 -> 735844 (compressed to 84.7 percent)
20:11:56:WU03:FS00:0xa4:  ... Done.
20:11:59:WU03:FS00:0xa4:- Shutting down core
20:11:59:WU03:FS00:0xa4:
20:11:59:WU03:FS00:0xa4:Folding@home Core Shutdown: FINISHED_UNIT
20:11:59:WU03:FS00:FahCore returned: FINISHED_UNIT (100 = 0x64)
20:11:59:WU03:FS00:Sending unit results: id:03 state:SEND error:NO_ERROR project:8690 run:0 clone:832 gen:40 core:0xa4 unit:0x000000300002894c5d24f90198be4f75
20:11:59:WU03:FS00:Uploading 719.10KiB to 155.247.166.220
20:11:59:WU03:FS00:Connecting to 155.247.166.220:8080
20:12:01:WU03:FS00:Upload complete
20:12:02:WU03:FS00:Server responded WORK_ACK (400)
20:12:02:WU03:FS00:Final credit estimate, 674.00 points
20:12:02:WU03:FS00:Cleaning up

According to the logs, the 0xa4 Gromacs GB Core dates back to December 15th, 2010:

Code: Select all

16:57:47:WU03:FS00:0xa4:
16:57:47:WU03:FS00:0xa4:*------------------------------*
16:57:47:WU03:FS00:0xa4:Folding@Home Gromacs GB Core
16:57:47:WU03:FS00:0xa4:Version 2.27 (Dec. 15, 2010)
16:57:47:WU03:FS00:0xa4:

By the looks of it, late 2016 vsyscall was disabled by default in (at least) Debian Linux kernels for security reasons. The 0xa4 core depends on vsyscall, which, at the time of its compilation in 2010, was still enabled in Debian kernels by default.

When vsyscall is not enabled you will be able to reliably reproduce the INTERRUPTED problem (at least for 8690 WUs) on Debian Buster 100% of the time, on the newest hardware you can get. Recompiling the Gromacs GB Core on a recent Debian distribution should fix this problem for Ubuntu/Debian kernels that do not have vsyscall support enabled. Since 2016 the default is to not have it enabled.

Thanks!

[Joe_H]

Your mention of vsyscall jogged my memory, there were several posts about that call being needed for A4 core processing early this year when someone ran into this problem after updating his Linux system. A search on the forum shows a few dozen posts in several topics mentioning vsyscall.

As for recompiling, that is not too likely. As I understand it, the persons involved in developing that core have moved on, and the code base for it would need some time for someone new to go into it and do that.

[bruce]

FAHCore_a4 is not being used for new projects. It has essentially been deprecated in favor of FAHCore_a7, although projects that have been started with FAHCore_a4 will be finished with _a4 for reasons associated with scientific validity.

I don't understand why this particular WU needed support for vsyscall and others have not. Certainly this isn't the only WU that has been assigned to that system that uses FAHCore_a4.

[toTOW]

It's no related to the WU, it's required by A4 core. But I guess there's not many other A4 projects around ...

[vincent89147]

Thanks Bruce, Joe,

Maybe this behavior for certain WUs can be added to a "known problems on Linux clients" page?

I'm not sure what kind of development is still going on, but it seems that the logic to drop a WU in favor of another one after a certain number of failures doesn't work. From what I've read early on in this thread, I understand that this should actually happen. The logs mention "Examination of work files indicates 8 consecutive improper terminations of core", so it's counting, but it does not take action. Fortunately the 8690 WUs are running quick so it's fairly trivial to reboot and allow vsyscall for them, and if 0xa4 core WUs are slowly going extinct then the problem should disappear with them

Thanks for your input!

[bruce]

I'm not sure what kind of development is still going on, but it seems that the logic to drop a WU in favor of another one after a certain number of failures doesn't work. From what I've read early on in this thread, I understand that this should actually happen. The logs mention "Examination of work files indicates 8 consecutive improper terminations of core", so it's counting, but it does not take action. Fortunately the 8690 WUs are running quick so it's fairly trivial to reboot and allow vsyscall for them, and if 0xa4 core WUs are slowly going extinct then the problem should disappear with them

There are 3 types of errors
* Errors detected by one of the FAHCores (e.g.- FAHCore_a4)
* Errors detected by FAHClient
* Errors detected by the OS (Windows/Linux/MacOS, each slightly different)

* Each FAHCore has it's own set of error detection code along with a non=zero return code so that FAHClient knows it happened.
* FAHClient is responsible for deciding if the WU should be aborted or retried ... and it keeps the count you've mentioned.
* The OS detects violations which it detects and attempts to notify the program that it needs to take corrective action -- or deciding that nothing can be recovered or if additional processing needs to be done before reporting a crash. The INTERRUPT is detected there. FAHClient receives very little information from the OS so it restarts the WU. That's probably where it misses adding the failure to the count.

[vincent89147]

I'm only a sysadmin, not a programmer, but I can see where that would complicate things

But thanks, I know much more now!

[bruce]

Thanks for the detailed diagnosis, Vincent. Inasmuch as core_a4 is being phased out, no fix will be provided, but if it's also a problem in FandCore_a7, a fix can now be attempted.

See https://github.com/FoldingAtHome/fah-issues/issues/1289