Project: 14314 (Run 0, Clone 132, Gen 2) GROMACS error

Moderators: Site Moderators, FAHC Science Team

Post Reply
dooley
Posts: 2
Joined: Sat Mar 07, 2020 1:23 pm

Project: 14314 (Run 0, Clone 132, Gen 2) GROMACS error

Post by dooley »

I have been folding for years with Ubuntu and for last few months Linux Mint Debian Edition 3. Today I have my first GROMACS fatal error. Project stuck in endless loop so will be deleting work folder when gpu finishes project,

LMDE3 setup

Code: Select all

*********************** Log Started 2020-03-04T23:33:27Z ***********************
23:33:27:************************* Folding@home Client *************************
23:33:27:      Website: https://foldingathome.org/
23:33:27:    Copyright: (c) 2009-2018 foldingathome.org
23:33:27:       Author: Joseph Coffland <joseph@cauldrondevelopment.com>
23:33:27:         Args: --child --lifeline 990 /etc/fahclient/config.xml --run-as
23:33:27:               fahclient --pid-file=/var/run/fahclient.pid --daemon
23:33:27:       Config: /etc/fahclient/config.xml
23:33:27:******************************** Build ********************************
23:33:27:      Version: 7.5.1
23:33:27:         Date: May 11 2018
23:33:27:         Time: 19:59:04
23:33:27:   Repository: Git
23:33:27:     Revision: 4705bf53c635f88b8fe85af7675557e15d491ff0
23:33:27:       Branch: master
23:33:27:     Compiler: GNU 6.3.0 20170516
23:33:27:      Options: -std=gnu++98 -O3 -funroll-loops
23:33:27:     Platform: linux2 4.14.0-3-amd64
23:33:27:         Bits: 64
23:33:27:         Mode: Release
23:33:27:******************************* System ********************************
23:33:27:          CPU: Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
23:33:27:       CPU ID: GenuineIntel Family 6 Model 60 Stepping 3
23:33:27:         CPUs: 8
23:33:27:       Memory: 15.63GiB
23:33:27:  Free Memory: 15.12GiB
23:33:27:      Threads: POSIX_THREADS
23:33:27:   OS Version: 4.9
23:33:27:  Has Battery: false
23:33:27:   On Battery: false
23:33:27:   UTC Offset: -8
23:33:27:          PID: 992
23:33:27:          CWD: /var/lib/fahclient
23:33:27:           OS: Linux 4.9.0-11-amd64 x86_64
23:33:27:      OS Arch: AMD64
23:33:27:         GPUs: 1
23:33:27:        GPU 0: Bus:1 Slot:0 Func:0 NVIDIA:2 GF116 [GeForce GT 635]
23:33:27:CUDA Device 0: Platform:0 Device:0 Bus:1 Slot:0 Compute:3.5 Driver:9.1
23:33:27:       OpenCL: Not detected: Failed to open dynamic library 'libOpenCL.so':
23:33:27:               libOpenCL.so: cannot open shared object file: No such file or
23:33:27:               directory
23:33:27:***********************************************************************
23:33:27:<config>
23:33:27:  <!-- Client Control -->
23:33:27:  <fold-anon v='true'/>
23:33:27:
23:33:27:  <!-- Folding Slot Configuration -->
23:33:27:  <gpu v='false'/>
23:33:27:
23:33:27:  <!-- Network -->
23:33:27:  <proxy v=':8080'/>
23:33:27:
23:33:27:  <!-- User Information -->
23:33:27:  <passkey v='********************************'/>
23:33:27:  <team v='45104'/>
23:33:27:  <user v='dooley'/>
23:33:27:
23:33:27:  <!-- Folding Slots -->
23:33:27:  <slot id='0' type='CPU'>
23:33:27:    <cpus v='6'/>
23:33:27:    <next-unit-percentage v='100'/>
23:33:27:  </slot>
23:33:27:  <slot id='1' type='GPU'>
23:33:27:    <opencl-index v='0'/>
23:33:27:  </slot>
23:33:27:</config>
logfile_01

Code: Select all

*********************** Log Started 2020-03-07T12:44:26Z ***********************
************************** Gromacs Folding@home Core ***************************
       Type: 0xa7
       Core: Gromacs
       Args: -dir 00 -suffix 01 -version 705 -lifeline 807 -checkpoint 15 -np 6
************************************ CBang *************************************
       Date: Nov 5 2019
       Time: 06:06:57
   Revision: 46c96f1aa8419571d83f3e63f9c99a0d602f6da9
     Branch: master
   Compiler: GNU 8.3.0
    Options: -std=c++11 -O3 -funroll-loops -fno-pie -fPIC
   Platform: linux2 4.19.0-5-amd64
       Bits: 64
       Mode: Release
************************************ System ************************************
        CPU: Intel(R) Core(TM) i7-4770 CPU @ 3.40GHz
     CPU ID: GenuineIntel Family 6 Model 60 Stepping 3
       CPUs: 8
     Memory: 15.63GiB
Free Memory: 5.55GiB
    Threads: POSIX_THREADS
 OS Version: 4.9
Has Battery: false
 On Battery: false
 UTC Offset: -8
        PID: 811
        CWD: /var/lib/fahclient/work
******************************** Build - libFAH ********************************
    Version: 0.0.18
     Author: Joseph Coffland <joseph@cauldrondevelopment.com>
  Copyright: 2019 foldingathome.org
   Homepage: https://foldingathome.org/
       Date: Nov 5 2019
       Time: 06:13:26
   Revision: 490c9aa2957b725af319379424d5c5cb36efb656
     Branch: master
   Compiler: GNU 8.3.0
    Options: -std=c++11 -O3 -funroll-loops -fno-pie
   Platform: linux2 4.19.0-5-amd64
       Bits: 64
       Mode: Release
************************************ Build *************************************
       SIMD: avx_256
********************************************************************************
Project: 14314 (Run 0, Clone 132, Gen 2)
Unit: 0x000000030002894b5e62c389eb15dc0a
Reading tar file core.xml
Reading tar file frame2.tpr
Digital signatures verified
Calling: mdrun -s frame2.tpr -o frame2.trr -cpt 15 -nt 6
Steps: first=1000000 total=500000
ERROR:
ERROR:-------------------------------------------------------
ERROR:Program GROMACS, VERSION 5.0.4-20191026-456f0d636-unknown
ERROR:Source code file: /host/debian-stable-64bit-core-a7-avx-release/gromacs-core/build/gromacs/src/gromacs/mdlib/domdec.c, line: 6902
ERROR:
ERROR:Fatal error:
ERROR:There is no domain decomposition for 6 ranks that is compatible with the given box and a minimum cell size of 2.42412 nm
ERROR:Change the number of ranks or mdrun option -rdd or -dds
ERROR:Look in the log file for details on the domain decomposition
ERROR:For more information and tips for troubleshooting, please check the GROMACS
ERROR:website at http://www.gromacs.org/Documentation/Errors
ERROR:-------------------------------------------------------
WARNING:Unexpected exit() call
WARNING:Unexpected exit from science code
Saving result file ../logfile_01.txt
Saving result file md.log
Saving result file science.log
md.log

Code: Select all

Log file opened on Sat Mar  7 04:51:26 2020
Host: sol2  pid: 1035  rank ID: 0  number of ranks:  1
GROMACS:    GROMACS, VERSION 5.0.4-20191026-456f0d636-unknown

GROMACS is written by:
Emile Apol         Rossen Apostolov   Herman J.C. Berendsen Par Bjelkmar       
Aldert van Buuren  Rudi van Drunen    Anton Feenstra     Sebastian Fritsch  
Gerrit Groenhof    Christoph Junghans Peter Kasson       Carsten Kutzner    
Per Larsson        Justin A. Lemkul   Magnus Lundborg    Pieter Meulenhoff  
Erik Marklund      Teemu Murtola      Szilard Pall       Sander Pronk       
Roland Schulz      Alexey Shvetsov    Michael Shirts     Alfons Sijbers     
Peter Tieleman     Christian Wennberg Maarten Wolf       
and the project leaders:
Mark Abraham, Berk Hess, Erik Lindahl, and David van der Spoel

Copyright (c) 1991-2000, University of Groningen, The Netherlands.
Copyright (c) 2001-2014, The GROMACS development team at
Uppsala University, Stockholm University and
the Royal Institute of Technology, Sweden.
check out http://www.gromacs.org for more information.


GROMACS:      GROMACS, VERSION 5.0.4-20191026-456f0d636-unknown

Gromacs version:    VERSION 5.0.4-20191026-456f0d636-unknown
GIT SHA1 hash:      456f0d636b694d70ef483843dbb1b1383643ee12
Branched from:      unknown
Precision:          single
Memory model:       64 bit
MPI library:        thread_mpi
OpenMP support:     disabled
GPU support:        disabled
invsqrt routine:    gmx_software_invsqrt(x)
SIMD instructions:  AVX_256
FFT library:        fftw-3.3.8-sse2-avx
RDTSCP usage:       disabled
C++11 compilation:  disabled
TNG support:        enabled
Tracing support:    disabled
Built on:           Wed Mar 22 01:02:31 UTC 2017
Built by:           root@69562b3fdcef [CMAKE]
Build OS/arch:      Linux 4.9.0-1-amd64 x86_64
Build CPU vendor:   GenuineIntel
Build CPU brand:    Intel(R) Core(TM) i7-3770S CPU @ 3.10GHz
Build CPU family:   6   Model: 58   Stepping: 9
Build CPU features: aes apic avx clfsh cmov cx8 cx16 f16c htt lahf_lm mmx msr nonstop_tsc pcid pclmuldq pdcm popcnt pse rdrnd rdtscp sse2 sse3 sse4.1 sse4.2 ssse3 tdt x2apic
C compiler:         /usr/bin/cc GNU 8.3.0
C compiler flags:    -mavx   -I/host/debian-stable-64bit-core-a7-avx-release/libfah/build/src -I/host/debian-stable-64bit-core-a7-avx-release/cbang/build/include -Wno-maybe-uninitialized -Wextra -Wno-missing-field-initializers -Wno-sign-compare -Wpointer-arith -Wall -Wno-unused -Wunused-value -Wunused-parameter -Wno-unknown-pragmas  -O3 -DNDEBUG -fomit-frame-pointer -funroll-all-loops -fexcess-precision=fast  -Wno-array-bounds 
C++ compiler:       /usr/bin/c++ GNU 8.3.0
C++ compiler flags:  -mavx   -I/host/debian-stable-64bit-core-a7-avx-release/libfah/build/src -I/host/debian-stable-64bit-core-a7-avx-release/cbang/build/include -Wextra -Wno-missing-field-initializers -Wpointer-arith -Wall -Wno-unused-function -Wno-unknown-pragmas  -O3 -DNDEBUG -fomit-frame-pointer -funroll-all-loops -fexcess-precision=fast  -Wno-array-bounds 
Boost version:      1.55.0 (internal)



++++ PLEASE READ AND CITE THE FOLLOWING REFERENCE ++++
B. Hess and C. Kutzner and D. van der Spoel and E. Lindahl
GROMACS 4: Algorithms for highly efficient, load-balanced, and scalable
molecular simulation
J. Chem. Theory Comput. 4 (2008) pp. 435-447
-------- -------- --- Thank You --- -------- --------


++++ PLEASE READ AND CITE THE FOLLOWING REFERENCE ++++
D. van der Spoel, E. Lindahl, B. Hess, G. Groenhof, A. E. Mark and H. J. C.
Berendsen
GROMACS: Fast, Flexible and Free
J. Comp. Chem. 26 (2005) pp. 1701-1719
-------- -------- --- Thank You --- -------- --------


++++ PLEASE READ AND CITE THE FOLLOWING REFERENCE ++++
E. Lindahl and B. Hess and D. van der Spoel
GROMACS 3.0: A package for molecular simulation and trajectory analysis
J. Mol. Mod. 7 (2001) pp. 306-317
-------- -------- --- Thank You --- -------- --------


++++ PLEASE READ AND CITE THE FOLLOWING REFERENCE ++++
H. J. C. Berendsen, D. van der Spoel and R. van Drunen
GROMACS: A message-passing parallel molecular dynamics implementation
Comp. Phys. Comm. 91 (1995) pp. 43-56
-------- -------- --- Thank You --- -------- --------

Changing nstlist from 10 to 20, rlist from 0.9 to 0.931

Input Parameters:
   integrator                     = sd
   tinit                          = 0
   dt                             = 0.002
   nsteps                         = 500000
   init-step                      = 1000000
   simulation-part                = 1
   comm-mode                      = Linear
   nstcomm                        = 10
   bd-fric                        = 0
   ld-seed                        = 1090642883
   emtol                          = 10
   emstep                         = 0.01
   niter                          = 20
   fcstep                         = 0
   nstcgsteep                     = 1000
   nbfgscorr                      = 10
   rtpi                           = 0.05
   nstxout                        = 500000
   nstvout                        = 500000
   nstfout                        = 0
   nstlog                         = 50000
   nstcalcenergy                  = 10
   nstenergy                      = 50000
   nstxout-compressed             = 5000
   compressed-x-precision         = 1000
   cutoff-scheme                  = Verlet
   nstlist                        = 20
   ns-type                        = Grid
   pbc                            = xyz
   periodic-molecules             = FALSE
   verlet-buffer-tolerance        = 0.005
   rlist                          = 0.931
   rlistlong                      = 0.931
   nstcalclr                      = 10
   coulombtype                    = PME
   coulomb-modifier               = Potential-shift
   rcoulomb-switch                = 0
   rcoulomb                       = 0.9
   epsilon-r                      = 1
   epsilon-rf                     = 1
   vdw-type                       = Cut-off
   vdw-modifier                   = Potential-shift
   rvdw-switch                    = 0
   rvdw                           = 0.9
   DispCorr                       = No
   table-extension                = 1
   fourierspacing                 = 0.12
   fourier-nx                     = 60
   fourier-ny                     = 60
   fourier-nz                     = 60
   pme-order                      = 4
   ewald-rtol                     = 1e-05
   ewald-rtol-lj                  = 0.001
   lj-pme-comb-rule               = Geometric
   ewald-geometry                 = 0
   epsilon-surface                = 0
   implicit-solvent               = No
   gb-algorithm                   = Still
   nstgbradii                     = 1
   rgbradii                       = 1
   gb-epsilon-solvent             = 80
   gb-saltconc                    = 0
   gb-obc-alpha                   = 1
   gb-obc-beta                    = 0.8
   gb-obc-gamma                   = 4.85
   gb-dielectric-offset           = 0.009
   sa-algorithm                   = Ace-approximation
   sa-surface-tension             = 2.05016
   tcoupl                         = No
   nsttcouple                     = -1
   nh-chain-length                = 0
   print-nose-hoover-chain-variables = FALSE
   pcoupl                         = No
   pcoupltype                     = Isotropic
   nstpcouple                     = -1
   tau-p                          = 1
   compressibility (3x3):
      compressibility[    0]={ 0.00000e+00,  0.00000e+00,  0.00000e+00}
      compressibility[    1]={ 0.00000e+00,  0.00000e+00,  0.00000e+00}
      compressibility[    2]={ 0.00000e+00,  0.00000e+00,  0.00000e+00}
   ref-p (3x3):
      ref-p[    0]={ 0.00000e+00,  0.00000e+00,  0.00000e+00}
      ref-p[    1]={ 0.00000e+00,  0.00000e+00,  0.00000e+00}
      ref-p[    2]={ 0.00000e+00,  0.00000e+00,  0.00000e+00}
   refcoord-scaling               = No
   posres-com (3):
      posres-com[0]= 0.00000e+00
      posres-com[1]= 0.00000e+00
      posres-com[2]= 0.00000e+00
   posres-comB (3):
      posres-comB[0]= 0.00000e+00
      posres-comB[1]= 0.00000e+00
      posres-comB[2]= 0.00000e+00
   QMMM                           = FALSE
   QMconstraints                  = 0
   QMMMscheme                     = 0
   MMChargeScaleFactor            = 1
qm-opts:
   ngQM                           = 0
   constraint-algorithm           = Lincs
   continuation                   = TRUE
   Shake-SOR                      = FALSE
   shake-tol                      = 0.0001
   lincs-order                    = 4
   lincs-iter                     = 1
   lincs-warnangle                = 30
   nwall                          = 0
   wall-type                      = 9-3
   wall-r-linpot                  = -1
   wall-atomtype[0]               = -1
   wall-atomtype[1]               = -1
   wall-density[0]                = 0
   wall-density[1]                = 0
   wall-ewald-zfac                = 3
   pull                           = no
   rotation                       = FALSE
   interactiveMD                  = FALSE
   disre                          = No
   disre-weighting                = Conservative
   disre-mixed                    = FALSE
   dr-fc                          = 1000
   dr-tau                         = 0
   nstdisreout                    = 100
   orire-fc                       = 0
   orire-tau                      = 0
   nstorireout                    = 100
   free-energy                    = yes
   init-lambda                    = -1
   init-lambda-state              = 19
   delta-lambda                   = 0
   nstdhdl                        = 10
   n-lambdas                      = 20
   separate-dvdl:
       fep-lambdas =   TRUE
      mass-lambdas =   FALSE
      coul-lambdas =   FALSE
       vdw-lambdas =   FALSE
    bonded-lambdas =   FALSE
 restraint-lambdas =   FALSE
temperature-lambdas =   FALSE
all-lambdas:
       fep-lambdas =         0.05         0.1        0.15         0.2        0.25         0.3        0.35         0.4        0.45         0.5        0.55         0.6        0.65         0.7        0.75         0.8        0.85         0.9        0.95           1
      mass-lambdas =         0.05         0.1        0.15         0.2        0.25         0.3        0.35         0.4        0.45         0.5        0.55         0.6        0.65         0.7        0.75         0.8        0.85         0.9        0.95           1
      coul-lambdas =         0.05         0.1        0.15         0.2        0.25         0.3        0.35         0.4        0.45         0.5        0.55         0.6        0.65         0.7        0.75         0.8        0.85         0.9        0.95           1
       vdw-lambdas =         0.05         0.1        0.15         0.2        0.25         0.3        0.35         0.4        0.45         0.5        0.55         0.6        0.65         0.7        0.75         0.8        0.85         0.9        0.95           1
    bonded-lambdas =         0.05         0.1        0.15         0.2        0.25         0.3        0.35         0.4        0.45         0.5        0.55         0.6        0.65         0.7        0.75         0.8        0.85         0.9        0.95           1
 restraint-lambdas =         0.05         0.1        0.15         0.2        0.25         0.3        0.35         0.4        0.45         0.5        0.55         0.6        0.65         0.7        0.75         0.8        0.85         0.9        0.95           1
temperature-lambdas =         0.05         0.1        0.15         0.2        0.25         0.3        0.35         0.4        0.45         0.5        0.55         0.6        0.65         0.7        0.75         0.8        0.85         0.9        0.95           1
   calc-lambda-neighbors          = -1
   dhdl-print-energy              = no
   sc-alpha                       = 0
   sc-power                       = 0
   sc-r-power                     = 6
   sc-sigma                       = 0.3
   sc-sigma-min                   = 0.3
   sc-coul                        = FALSE
   dh-hist-size                   = 0
   dh-hist-spacing                = 0.1
   separate-dhdl-file             = yes
   dhdl-derivatives               = yes
   cos-acceleration               = 0
   deform (3x3):
      deform[    0]={ 0.00000e+00,  0.00000e+00,  0.00000e+00}
      deform[    1]={ 0.00000e+00,  0.00000e+00,  0.00000e+00}
      deform[    2]={ 0.00000e+00,  0.00000e+00,  0.00000e+00}
   simulated-tempering            = FALSE
   E-x:
      n = 0
   E-xt:
      n = 0
   E-y:
      n = 0
   E-yt:
      n = 0
   E-z:
      n = 0
   E-zt:
      n = 0
   swapcoords                     = no
   adress                         = FALSE
   userint1                       = 0
   userint2                       = 0
   userint3                       = 0
   userint4                       = 0
   userreal1                      = 0
   userreal2                      = 0
   userreal3                      = 0
   userreal4                      = 0
grpopts:
   nrdf:     3483.86     69916.1
   ref-t:         300         300
   tau-t:           1           1
annealing:          No          No
annealing-npoints:           0           0
   acc:	           0           0           0
   nfreeze:           N           N           N
   energygrp-flags[  0]: 0 0 0
   energygrp-flags[  1]: 0 0 0
   energygrp-flags[  2]: 0 0 0
science.log

Code: Select all

-------------------------------------------------------
Program GROMACS, VERSION 5.0.4-20191026-456f0d636-unknown
Source code file: /host/debian-stable-64bit-core-a7-avx-release/gromacs-core/build/gromacs/src/gromacs/mdlib/domdec.c, line: 6902

Fatal error:
There is no domain decomposition for 6 ranks that is compatible with the given box and a minimum cell size of 2.42412 nm
Change the number of ranks or mdrun option -rdd or -dds
Look in the log file for details on the domain decomposition
For more information and tips for troubleshooting, please check the GROMACS
website at http://www.gromacs.org/Documentation/Errors
-------------------------------------------------------
WARNING:Unexpected exit() call
WARNING:Unexpected exit from science code
Saving result file ../logfile_01.txt
Saving result file md.log
Saving result file science.log
Calling: mdrun -s frame2.tpr -o frame2.trr -cpt 15 -nt 6
Steps: first=1000000 total=500000
ERROR:
ERROR:-------------------------------------------------------
ERROR:Program GROMACS, VERSION 5.0.4-20191026-456f0d636-unknown
ERROR:Source code file: /host/debian-stable-64bit-core-a7-avx-release/gromacs-core/build/gromacs/src/gromacs/mdlib/domdec.c, line: 6902
ERROR:
ERROR:Fatal error:
ERROR:There is no domain decomposition for 6 ranks that is compatible with the given box and a minimum cell size of 2.42412 nm
ERROR:Change the number of ranks or mdrun option -rdd or -dds
ERROR:Look in the log file for details on the domain decomposition
ERROR:For more information and tips for troubleshooting, please check the GROMACS
ERROR:website at http://www.gromacs.org/Documentation/Errors
ERROR:-------------------------------------------------------
Reading file frame2.tpr, VERSION 5.0.4 (single precision)
Changing nstlist from 10 to 20, rlist from 0.9 to 0.931
Top
toTOW
Site Moderator
Posts: 6359
Joined: Sun Dec 02, 2007 10:38 am
Location: Bordeaux, France
Contact:
Contact toTOW

Re: Project: 14314 (Run 0, Clone 132, Gen 2) GROMACS error

Post by toTOW »

I've notified the researcher in charge of this project.
Image

Folding@Home beta tester since 2002. Folding Forum moderator since July 2008.
Top
tug27224
Scientist
Posts: 43
Joined: Fri Jun 24, 2016 6:11 pm
Location: Philadelphia

Re: Project: 14314 (Run 0, Clone 132, Gen 2) GROMACS error

Post by tug27224 »

Thanks for reporting this. PROJ14314 has been particularly troublesome. I've just restriced 6-core clients from being assigned it, but I'll keep an eye on the failure rate and consider removing the project entirely.
Top
DeeGee
Posts: 61
Joined: Thu Oct 02, 2008 1:15 pm
Hardware configuration: Asus Crosshair Hero VIII, AMD Ryzen 3950x, 2x8GB 3600MHz DDR4, Radeon VII, Win10
Asus Crosshair Hero VII, Amd Ryzen 3900x, 2x16GB 3200MHz DDR4, GeForce 980 TI, Kubuntu 19.10
Location: Finland

Re: Project: 14314 (Run 0, Clone 132, Gen 2) GROMACS error

Post by DeeGee »

That bug keeps popping up in different projects with different amount of threads.

https://github.com/FoldingAtHome/fah-issues/issues/1307
Top
foldy
Posts: 2040
Joined: Sat Dec 01, 2012 3:43 pm
Hardware configuration: Folding@Home Client 7.6.13 (1 GPU slots)
Windows 7 64bit
Intel Core i5 2500k@4Ghz
Nvidia gtx 1080ti driver 441

Re: Project: 14314 (Run 0, Clone 132, Gen 2) GROMACS error

Post by foldy »

I get the same issue with Project: 14520 on a Intel Xeon CPU 16 core (32 threads)

Code: Select all

22:28:18:WU00:FS00:0xa7:        CPU: Intel(R) Xeon(R) Silver 4110 CPU @ 2.10GHz
22:28:18:WU00:FS00:0xa7:     CPU ID: GenuineIntel Family 6 Model 85 Stepping 4
22:28:18:WU00:FS00:0xa7:       CPUs: 32
22:28:18:WU00:FS00:0xa7:     Memory: 31.04GiB
22:28:18:WU00:FS00:0xa7:Free Memory: 10.15GiB
22:28:18:WU00:FS00:0xa7:    Threads: POSIX_THREADS
22:28:18:WU00:FS00:0xa7: OS Version: 4.4
22:28:18:WU00:FS00:0xa7:Has Battery: false
22:28:18:WU00:FS00:0xa7: On Battery: false
22:28:18:WU00:FS00:0xa7: UTC Offset: 0                                                                                                                22:28:18:WU00:FS00:0xa7:        PID: 2886
22:28:18:WU00:FS00:0xa7:        CWD: /etc/init.d/work

22:32:18:WU00:FS00:0xa7:Project: 14520 (Run 0, Clone 31, Gen 61)

22:29:18:WU00:FS00:0xa7:Reducing thread count from 29 to 28 to avoid domain decomposition by a prime number > 3
22:29:18:WU00:FS00:0xa7:Calling: mdrun -s frame61.tpr -o frame61.trr -x frame61.xtc -cpt 15 -nt 28
22:29:18:WU00:FS00:0xa7:Steps: first=15250000 total=250000
22:29:18:WU00:FS00:0xa7:ERROR:
22:29:18:WU00:FS00:0xa7:ERROR:-------------------------------------------------------
22:29:18:WU00:FS00:0xa7:ERROR:Program GROMACS, VERSION 5.0.4-20191026-456f0d636-unknown
22:29:18:WU00:FS00:0xa7:ERROR:Source code file: /host/debian-stable-64bit-core-a7-avx-release/gromacs-core/build/gromacs/src/gromacs/mdlib/domdec.c, line: 6902
22:29:18:WU00:FS00:0xa7:ERROR:
22:29:18:WU00:FS00:0xa7:ERROR:Fatal error:
22:29:18:WU00:FS00:0xa7:ERROR:There is no domain decomposition for 20 ranks that is compatible with the given box and a minimum cell size of 1.46925 nm
22:29:18:WU00:FS00:0xa7:ERROR:Change the number of ranks or mdrun option -rcon or -dds or your LINCS settings
22:29:18:WU00:FS00:0xa7:ERROR:Look in the log file for details on the domain decomposition
22:29:18:WU00:FS00:0xa7:ERROR:For more information and tips for troubleshooting, please check the GROMACS
22:29:18:WU00:FS00:0xa7:ERROR:website at http://www.gromacs.org/Documentation/Errors
22:29:18:WU00:FS00:0xa7:ERROR:-------------------------------------------------------
22:29:22:WU00:FS00:0xa7:WARNING:Unexpected exit() call
22:29:22:WU00:FS00:0xa7:WARNING:Unexpected exit from science code
22:29:22:WU00:FS00:0xa7:Saving result file ../logfile_01.txt
22:29:22:WU00:FS00:0xa7:Saving result file md.log
22:29:22:WU00:FS00:0xa7:Saving result file science.log
22:29:23:WU00:FS00:FahCore returned: INTERRUPTED (102 = 0x66)
As workaround I configured 2 CPU slots 15 threads each instead of 1 CPU slot with 30 threads. But now it complained decomposition for 15 ranks is ERROR (before it was 20). So I reduced to 2 CPU slots 14 threads each and now it is working.
Top
bruce
Posts: 20824
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Project: 14314 (Run 0, Clone 132, Gen 2) GROMACS error

Post by bruce »

Gromacs does have trouble with certain numbers of threads. Your solution to create multiple slots with lower numbers of threads is an excellent one.

Gromacs does pay attention to the factors of the numbers of cores and you may, at some point encounter problems with 14 since 7 is often excluded. 16 or 12 should be safe choices.
Top
foldy
Posts: 2040
Joined: Sat Dec 01, 2012 3:43 pm
Hardware configuration: Folding@Home Client 7.6.13 (1 GPU slots)
Windows 7 64bit
Intel Core i5 2500k@4Ghz
Nvidia gtx 1080ti driver 441

Re: Project: 14314 (Run 0, Clone 132, Gen 2) GROMACS error

Post by foldy »

That should be something the FAHclient or assignment servers should handle. It already reduces threads because of prime numbers but it should also calculate the decomposition values right.

So assignment servers should not assign these projects to CPU slots having 30 threads or 15 threads because of the decomposition 20 or 15 problem
Top
bruce
Posts: 20824
Joined: Thu Nov 29, 2007 10:13 pm
Location: So. Cal.

Re: Project: 14314 (Run 0, Clone 132, Gen 2) GROMACS error

Post by bruce »

Gromacs.org is an open project. Anyone who wants to contribute to that project is welcome to apply their expertise to improving their code. Assignment Server can only work within the constraints that are in the code . Also, FAH has adopted the GROMACS code for home computers which generally have a more limited number of threads. Most of FAH's development work has been has been aimed at GPUs using the OpenMM code.
Top
foldy
Posts: 2040
Joined: Sat Dec 01, 2012 3:43 pm
Hardware configuration: Folding@Home Client 7.6.13 (1 GPU slots)
Windows 7 64bit
Intel Core i5 2500k@4Ghz
Nvidia gtx 1080ti driver 441

Re: Project: 14314 (Run 0, Clone 132, Gen 2) GROMACS error

Post by foldy »

Gromacs already has newer versions but not adopted to FAH yet, so that doesn't help.

What constraints does the assignment servers have?
Maybe it has a rule like assign cpu project 14xxx to CPU of type Y with threads Z?
Top
Joe_H
Site Admin
Posts: 7937
Joined: Tue Apr 21, 2009 4:41 pm
Hardware configuration: Mac Pro 2.8 quad 12 GB smp4
MacBook Pro 2.9 i7 8 GB smp2
Location: W. MA

Re: Project: 14314 (Run 0, Clone 132, Gen 2) GROMACS error

Post by Joe_H »

foldy wrote:Gromacs already has newer versions but not adopted to FAH yet, so that doesn't help.

What constraints does the assignment servers have?
Maybe it has a rule like assign cpu project 14xxx to CPU of type Y with threads Z?
Rule currently is more like, assign CPU project X to threads less than Y, not including A, B, ...

Type of CPU doesn't enter into it. At run time the client uses either the SSE2 version or the AVX version of the core depending on the type of CPU.
Image

iMac 2.8 i7 12 GB smp8, Mac Pro 2.8 quad 12 GB smp6
MacBook Pro 2.9 i7 8 GB smp3
Top
foldy
Posts: 2040
Joined: Sat Dec 01, 2012 3:43 pm
Hardware configuration: Folding@Home Client 7.6.13 (1 GPU slots)
Windows 7 64bit
Intel Core i5 2500k@4Ghz
Nvidia gtx 1080ti driver 441

Re: Project: 14314 (Run 0, Clone 132, Gen 2) GROMACS error

Post by foldy »

OK then I know the rule to fix this project 14314 and 14520 (maybe more)

Assign CPU project to threads less than 15

Then there is no decomposition issue with 20 and 15
Top
Post Reply

Return to “Issues with a specific WU”