Folding Forum

Posted: **Fri Apr 24, 2020 1:58 am**

Data for a slightly different 6x6x5 project. This one will not run on 85 threads.

p16420 - max 6x6x5 - PME load 0.18

  2 = 2x1x1
  3 = 3x1x1
  4 = 4x1x1
  5 = 5x1x1
  6 = 6x1x1
  8 = 4x2x1
  9 = 3x3x1
 10 = 5x2x1
 12 = 4x3x1
 15 = 5x3x1
 16 = 4x4x1
 18 = 6x3x1
 20 = 4x4x1  16 +  4 PME
 21 = 4x4x1  16 +  5 PME
 24 = 6x3x1  18 +  6 PME
 25 = 5x4x1  20 +  5 PME
 27 = 6x3x1  18 +  9 PME
 28 = 5x4x1  20 +  8 PME
 30 = 6x4x1  24 +  6 PME
 32 = 6x4x1  24 +  8 PME
 35 = 5x5x1  25 + 10 PME
 36 = 3x3x3  27 +  9 PME
 40 = 4x4x2  32 +  8 PME
 42 = 4x4x2  32 + 10 PME
 44 = 6x3x2  36 +  8 PME
 45 = 6x3x2  36 +  9 PME
 48 = 4x3x3  36 + 12 PME
 50 = 5x4x2  40 + 10 PME
 52 = 5x4x2  40 + 12 PME
 54 = 6x3x2  36 + 18 PME
 55 = 5x3x3  45 + 10 PME
 56 = 2x4x5  40 + 16 PME
 60 = 4x3x4  48 + 12 PME
 64 = 4x4x3  48 + 16 PME
 65 = 2x5x5  50 + 15 PME
 66 = 6x3x3  54 + 12 PME
 72 = 6x3x3  54 + 18 PME
 75 = 5x3x4  60 + 15 PME
 78 = 4x4x4  64 + 14 PME
 80 = 4x4x4  64 + 16 PME
 81 = 3x6x3  54 + 27 PME
 88 = 4x6x3  72 + 16 PME
 90 = 6x3x4  72 + 18 PME
 95 = 5x5x3  75 + 20 PME
 96 = 6x4x3  72 + 24 PME
 98 = 4x5x4  80 + 18 PME
100 = 5x4x4  80 + 20 PME
110 = 5x6x3  90 + 20 PME
114 = 6x5x3  90 + 24 PME
115 = 5x6x3  90 + 25 PME
117 = 6x4x4  96 + 21 PME
120 = 6x4x4  96 + 24 PME
125 = 5x5x4 100 + 25 PME
128 = 4x6x4  96 + 32 PME

All data for 1-128 threads. 1 indicates success while blank is failure.

Code: Select all

# Threads p14336   p16802   p14365   p16501   p14542   p13832   p16420   p14378   p13851   p16422   p16423   p14574   p14576
Box       18x18x18 16x16x16 13x13x13 11x11x10 6x6x5    6x6x5    6x6x5    5x5x5    5x5x4    5x5x4    4x4x4    4x4x3    4x4x3
PME       0.22     0.09     0.36     0.1      0.2      0.19     0.18     0.37     0.19     0.18     0.19     0.18     0.17
  1       1        1        1        1        1        1        1        1        1        1        1        1        1
  2       1        1        1        1        1        1        1        1        1        1        1        1        1
  3       1        1        1        1        1        1        1        1        1        1        1        1        1
  4       1        1        1        1        1        1        1        1        1        1        1        1        1
  5       1        1        1        1        1        1        1        1        1        1                           
  6       1        1        1        1        1        1        1        1        1        1        1        1        1
  7       1        1        1        1                                                                                 
  8       1        1        1        1        1        1        1        1        1        1        1        1        1
  9       1        1        1        1        1        1        1        1        1        1        1        1        1
 10       1        1        1        1        1        1        1        1        1        1                           
 11       1        1        1        1                                                                                 
 12       1        1        1        1        1        1        1        1        1        1        1        1        1
 13                                                                                                                    
 14                                                                                                                    
 15       1        1        1        1        1        1        1        1        1        1                           
 16       1        1        1        1        1        1        1        1        1        1        1        1        1
 17                                                                                                                    
 18       1        1        1        1        1        1        1        1        1        1        1        1        1
 19                                                                                                                    
 20       1        1        1        1        1        1        1        1        1        1        1        1        1
 21       1        1        1        1        1        1        1        1        1        1        1        1        1
 22                                                                                                                    
 23                                                                                                                    
 24       1        1        1        1        1        1        1        1        1        1        1        1         
 25       1        1        1        1        1        1        1        1        1        1                           
 26                                                                                                                    
 27       1        1        1        1        1        1        1        1        1        1        1        1        1
 28       1        1        1        1        1        1        1        1        1        1                           
 29                                                                                                                    
 30       1        1        1        1        1        1        1        1        1        1        1        1         
 31                                                                                                                    
 32       1        1        1        1        1        1        1        1        1        1        1        1        1
 33                                                                                                                    
 34                                                                                                                    
 35       1        1        1        1        1        1        1        1        1        1                           
 36       1        1        1        1        1        1        1        1        1        1        1        1         
 37                                                                                                                    
 38                                                                                                                    
 39                                                                                                                    
 40       1        1        1        1        1        1        1        1        1        1        1        1        1
 41                                                                                                                    
 42       1        1        1        1        1        1        1        1        1        1        1        1        1
 43                                                                                                                    
 44       1        1        1        1        1        1        1        1        1        1        1        1        1
 45       1        1        1        1        1        1        1        1        1        1        1        1        1
 46                                                                                                                    
 47                                                                                                                    
 48       1        1        1        1        1        1        1        1        1        1        1        1         
 49                                                                                                                    
 50       1        1        1        1        1        1        1        1        1        1                           
 51                                                                                                                    
 52       1        1        1        1        1        1        1        1        1        1                           
 53                                                                                                                    
 54       1        1        1        1        1        1        1        1        1        1        1        1         
 55       1        1        1        1        1        1        1        1        1        1                           
 56       1        1        1        1        1        1        1        1        1        1                           
 57                                                                                                                    
 58                                                                                                                    
 59                                                                                                                    
 60       1        1        1        1        1        1        1        1        1        1        1        1         
 61                                                                                                                    
 62                                                                                                                    
 63       1        1        1        1                                   1                                             
 64       1        1        1        1        1        1        1        1        1        1        1        1        1
 65       1        1        1        1        1        1        1        1        1        1                           
 66       1        1        1        1        1        1        1        1                                             
 67                                                                                                                    
 68                                                                                                                    
 69                                                                                                                    
 70       1        1        1        1                                   1                                             
 71                                                                                                                    
 72       1        1        1        1        1        1        1        1                                             
 73                                                                                                                    
 74                                                                                                                    
 75       1        1        1        1        1        1        1        1        1        1                           
 76                                                                                                                    
 77       1        1        1        1                                                                                 
 78       1        1        1        1                 1        1        1                 1                           
 79                                                                                                                    
 80       1        1        1        1        1        1        1        1        1        1        1                  
 81       1        1        1        1        1        1        1        1                                             
 82                                                                                                                    
 83                                                                                                                    
 84       1        1        1        1                                                                                 
 85       1        1        1        1        1        1                 1        1                                    
 86                                                                                                                    
 87                                                                                                                    
 88       1        1        1        1                 1        1                                                      
 89                                                                                                                    
 90       1        1        1        1        1        1        1                                                      
 91       1        1        1        1                                                                                 
 92                                                                                                                    
 93                                                                                                                    
 94                                                                                                                    
 95       1        1        1        1        1        1        1        1        1        1                           
 96       1        1        1        1        1        1        1        1                                             
 97                                                                                                                    
 98       1        1        1        1                 1        1                 1        1                           
 99       1        1        1        1        1                                                                        
100       1        1        1        1        1        1        1        1        1        1                           
101                                                                                                                    
102       1        1        1        1                                                                                 
103                                                                                                                    
104       1        1        1        1                                   1                                             
105       1        1        1        1                                                                                 
106                                                                                                                    
107                                                                                                                    
108       1        1        1        1                                   1                                             
109                                                                                                                    
110       1        1        1        1        1        1        1                                                      
111                                                                                                                    
112       1        1        1        1                                                                                 
113                                                                                                                    
114       1        1        1        1        1        1        1                                                      
115       1        1        1        1        1        1        1                                                      
116                                                                                                                    
117       1        1        1        1        1        1        1        1                                             
118                                                                                                                    
119       1        1        1        1                                                                                 
120       1        1        1        1        1        1        1        1                                             
121                                                                                                                    
122                                                                                                                    
123                                                                                                                    
124                                                                                                                    
125       1        1        1        1        1        1        1        1        1        1                           
126       1        1        1        1                                                                                 
127                                                                                                                    
128       1        1        1        1        1        1        1        1

Posted: **Fri Apr 24, 2020 2:56 pm**

_r2w_ben

Is there a way in your testing to distinguish between actual failure and graceful reduction in thread count?
When initial testing for thread count (2-24 threads) is being done, graceful reduction of threads by client or AS is not considered a failure and does not result in constraints added for those thread counts.
Let's say 5, 7, 13, 14, 17, 19, 22, 23 are still allowed to be assigned a WU. It was decided that performance drop is so tiny when thread is reduced by 1 or 2, that it is not worth excluding such systems, especially when those systems are very fast compared to usual crop of donors (folders). Also, anything above 24 threads scaled particularly poorly on majority of projects anyways, so if graceful reduction happens at those thread counts, performance decrease is non existent or sometimes you experience performance increase. I have seen projects which have shorter TPF at 10 threads than 15-16 threads

Keep in mind, that thread counts which get reduced to thread count which fails, is considered as failure. Like 11>10. When 10 fails, 11 will reduce to it, and thus 11 is considered failure. There are projects (a lot of them) which do not fail at 10 threads, so in that case 11 isn't constrained.
I know, that your script is already helping a lot, but is there a way to fine tune it, and not note graceful reductions as failures?

Posted: **Fri Apr 24, 2020 4:42 pm**

muziqaz wrote:Is there a way in your testing to distinguish between actual failure and graceful reduction in thread count?

That is a good idea and would fill in the summary table a lot.

My testing is using GROMACS directly. From what I can gather, the graceful reduction is done by FAH prior to calling GROMACS and the code is not published publicly. If someone from FAH could share that piece of code, I can experiment with it.

Posted: **Fri Apr 24, 2020 8:21 pm**

_r2w_ben wrote:
muziqaz wrote:Is there a way in your testing to distinguish between actual failure and graceful reduction in thread count?
That is a good idea and would fill in the summary table a lot.

My testing is using GROMACS directly. From what I can gather, the graceful reduction is done by FAH prior to calling GROMACS and the code is not published publicly. If someone from FAH could share that piece of code, I can experiment with it.

fahclient will be open sourced, if not already.
Have you been in contact with one of the guys (was it Sukrit) from FAH about certain aspect of your testing?

Posted: **Fri Apr 24, 2020 9:28 pm**

I passed on information to connect him up with another of the researchers.

As for open sourced, FAHControl, Web Control and some other items have been opened so far. FAHClient, the folding cores, and FAHWrapper have not yet been made open source. As I understand it, there are plans to move those to open source at some point in the future, how much the COVIS-19 response is going to delay that remains to be seen.

As for the "graceful reduction", I suspect that is in the settings used by FAHWrapper. It is what is responsible for running the core, and I suspect catches the errors aan either exits, or has the core start up with a new CPU thread setting.

Posted: **Fri Apr 24, 2020 10:01 pm**

Joe_H wrote:I passed on information to connect him up with another of the researchers.

As for open sourced, FAHControl, Web Control and some other items have been opened so far. FAHClient, the folding cores, and FAHWrapper have not yet been made open source. As I understand it, there are plans to move those to open source at some point in the future, how much the COVIS-19 response is going to delay that remains to be seen.

As for the "graceful reduction", I suspect that is in the settings used by FAHWrapper. It is what is responsible for running the core, and I suspect catches the errors aan either exits, or has the core start up with a new CPU thread setting.

There are two "graceful reductions" one made by the assignment server, and that message is nicely hidden in the salad that is fahlog

before fahcore engages, and all its info is printed out, AS tells you that it reduced thread count to whatever it was instructed to do.
Second type is done by wrapper, it tells you it reduced the threads just before fahcore starts.
1st method is extremely rare, since now thread counts are being tested more or less, the second one is present and happens in every project

I think in the future second method hopefully will be used better to prevent any decomposition failures (hard failures).
Graceful reduction will remain until different simulation methods are developed

Posted: **Sat Apr 25, 2020 12:11 am**

muziqaz wrote:Second type is done by wrapper, it tells you it reduced the threads just before fahcore starts.
1st method is extremely rare, since now thread counts are being tested more or less, the second one is present and happens in every project

One thing I've wondered is whether 11 always gets downgraded to 10 because it's prime. Some projects are definitely capable of running on 11 cores.

The version of GROMACS FAH uses doesn't like 13 or 14 (which could do 2x7x1) even though they could work given a large enough box size. I haven't checked if this is still the case in later releases.

Posted: **Sat Apr 25, 2020 12:50 am**

_r2w_ben wrote: One thing I've wondered is whether 11 always gets downgraded to 10 because it's prime. Some projects are definitely capable of running on 11 cores.

The version of GROMACS FAH uses doesn't like 13 or 14 (which could do 2x7x1) even though they could work given a large enough box size. I haven't checked if this is still the case in later releases.

Yes 11, same as 13/14, was always downgrading, I am yet to see 11 work. Just sometimes 10 doesn't work, thus 11 downgrades to a failing thread count.

Posted: **Sat Apr 25, 2020 4:59 am**

I wonder why prime numbers aren't simply broken down.
Eg: on 7 CPU cores, run 2 WUs (run 3 cores and 4, or 2 and 4). On 11 CPU cores, run WUs for 10 cores, and just ignore the extra core?

Posted: **Sat Apr 25, 2020 7:10 am**

For 7 CPUs to be broken down would lower the purpose of QRB which is to get WUs as fast as possible. This may not be the case for 128 CPUs but is the case for lower CPU values. Also, folding uses real cores, not hyper-threading but it can use it for some measurable performance increase. However, the increase isn't the same as adding a real physical core.

Posted: **Sat Apr 25, 2020 7:25 am**

There used to be an error message saying 11 is unacceptable becaus it GROMACS needs all factors to be 2 or 3. I'm not sure if that message is from the native GROMACS code or if it's something that has been added in FAH"s adaptation of that code.

The conventional wisdom was less specific ... that FAH cannot decompose space if any of the factors are "large primes." Unfortunately those words has never been well defined as to what "large" meant in that context.

Posted: **Sat Apr 25, 2020 1:49 pm**

bruce wrote:The conventional wisdom was less specific ... that FAH cannot decompose space if any of the factors are "large primes." Unfortunately those words has never been well defined as to what "large" meant in that context.

Large has different meanings based on the context.

<= 12 cores assigned to PP
5 is a large prime for 4x4x4 but fine for 5x5x4
7 is a large prime for 6x6x5 but fine for 8x8x7
11 is a large prime for 8x8x7 but fine for 11x11x10

FAH has unpublished code in the wrapper or it's custom build that excludes 7 and 11 even if they could work based on their high probability of failure.

> 12 cores assigned to PP
Occurs in the following scenarios:
1. Total cores <= 18 so there is no PME
2. 12+ cores remaining for PP after PME assigned

GROMACS 5.0.4 source code:

Code: Select all

if (nnodes_div > 12)
        {
            ldiv = largest_divisor(nnodes_div);
            /* Check if the largest divisor is more than nnodes^2/3 */
            if (ldiv*ldiv*ldiv > nnodes_div*nnodes_div)
            {
                gmx_fatal(FARGS, "The number of ranks you selected (%d) contains a large prime factor %d. In most cases this will lead to bad performance. Choose a number with smaller prime factors or set the decomposition (option -dd) manually.",
                          nnodes_div, ldiv);
            }
        }

Using 14 cores:
Factors to 2x7
largest_divisor = 7
Is 7*7*7 > 14*14? Yes, therefore performance will probably be bad. Each core is assigned too little work and communication costs will be high.

Using 15 cores:
Factors to 3x5
largest_divisor = 5
Is 5*5*5 > 15*15? No, good to go!

> 18 cores and calculating PME
This isn't necessary a problem with primes but it will fail domain decomposition. PME nodes can't exceed half the total number of cores.

Code: Select all

if (npme > nnodes/2)
    {
        gmx_fatal(FARGS, "Could not find an appropriate number of separate PME ranks. i.e. >= %5f*#ranks (%d) and <= #ranks/2 (%d) and reasonable performance wise (grid_x=%d, grid_y=%d).\n"
                  "Use the -npme option of mdrun or change the number of ranks or the PME grid dimensions, see the manual for details.",
                  ratio, (int)(0.95*ratio*nnodes+0.5), nnodes/2, ir->nkx, ir->nky);
        /* Keep the compiler happy */
        npme = 0;
    }

Posted: **Sat Apr 25, 2020 6:25 pm**

Data for an 8x8x7. It's similar to larger projects with the exception of excluding 11, 99, and 108 threads.

p14570 - max 8x8x7 - PME load 0.18

Code: Select all

  2 = 2x1x1
  3 = 3x1x1
  4 = 4x1x1
  5 = 5x1x1
  6 = 6x1x1
  7 = 7x1x1
  8 = 8x1x1
  9 = 3x3x1
 10 = 5x2x1
 12 = 4x3x1
 15 = 5x3x1
 16 = 8x2x1
 18 = 6x3x1
 20 = 4x2x2  16 +  4 PME
 21 = 4x2x2  16 +  5 PME
 24 = 6x3x1  18 +  6 PME
 25 = 5x2x2  20 +  5 PME
 27 = 3x3x2  18 +  9 PME
 28 = 5x2x2  20 +  8 PME
 30 = 6x2x2  24 +  6 PME
 32 = 8x3x1  24 +  8 PME
 35 = 5x5x1  25 + 10 PME
 36 = 3x3x3  27 +  9 PME
 40 = 8x2x2  32 +  8 PME
 42 = 4x4x2  32 + 10 PME
 44 = 6x3x2  36 +  8 PME
 45 = 6x3x2  36 +  9 PME
 48 = 4x3x3  36 + 12 PME
 50 = 4x5x2  40 + 10 PME
 52 = 5x4x2  40 + 12 PME
 54 = 6x3x2  36 + 18 PME
 55 = 3x5x3  45 + 10 PME
 56 = 5x4x2  40 + 16 PME
 60 = 4x3x4  48 + 12 PME
 63 = 7x7x1  49 + 14 PME
 64 = 3x4x4  48 + 16 PME
 65 = 2x5x5  50 + 15 PME
 66 = 6x3x3  54 + 12 PME
 70 = 7x4x2  56 + 14 PME
 72 = 6x3x3  54 + 18 PME
 75 = 5x3x4  60 + 15 PME
 77 = 7x3x3  63 + 14 PME
 78 = 4x4x4  64 + 14 PME
 80 = 8x4x2  64 + 16 PME
 81 = 6x3x3  54 + 27 PME
 84 = 7x3x3  63 + 21 PME
 85 = 7x5x2  70 + 15 PME
 88 = 6x4x3  72 + 16 PME
 90 = 6x3x4  72 + 18 PME
 91 = 7x5x2  70 + 21 PME
 95 = 5x5x3  75 + 20 PME
 96 = 8x3x3  72 + 24 PME
 98 = 4x5x4  80 + 18 PME
100 = 5x4x4  80 + 20 PME
102 = 3x7x4  84 + 18 PME
104 = 4x7x3  84 + 20 PME
105 = 7x3x4  84 + 21 PME
110 = 5x6x3  90 + 20 PME
112 = 7x4x3  84 + 28 PME
114 = 6x5x3  90 + 24 PME
115 = 5x6x3  90 + 25 PME
117 = 3x8x4  96 + 21 PME
119 = 7x7x2  98 + 21 PME
120 = 8x3x4  96 + 24 PME
125 = 5x5x4 100 + 25 PME
126 = 7x6x2  84 + 42 PME
128 = 8x4x3  96 + 32 PME

All data for 1-128 threads. 1 indicates success while blank is failure.

Code: Select all

# Threads p14336 p16802   p14365   p16804   p16501   p14570 p14542 p13832 p16420 p14378 p13851 p16422 p16423 p14574 p14576
Box     18x18x18 16x16x16 14x14x14 11x11x13 11x11x10 8x8x7  6x6x5  6x6x5  6x6x5  5x5x5  5x5x4  5x5x4  4x4x4  4x4x3  4x4x3
PME     0.22     0.09     0.36     0.09     0.1      0.18   0.2    0.19   0.18   0.37   0.19   0.18   0.19   0.18   0.17
  1     1        1        1        1        1        1      1      1      1      1      1      1      1      1      1
  2     1        1        1        1        1        1      1      1      1      1      1      1      1      1      1
  3     1        1        1        1        1        1      1      1      1      1      1      1      1      1      1
  4     1        1        1        1        1        1      1      1      1      1      1      1      1      1      1
  5     1        1        1        1        1        1      1      1      1      1      1      1                     
  6     1        1        1        1        1        1      1      1      1      1      1      1      1      1      1
  7     1        1        1        1        1        1                                                               
  8     1        1        1        1        1        1      1      1      1      1      1      1      1      1      1
  9     1        1        1        1        1        1      1      1      1      1      1      1      1      1      1
 10     1        1        1        1        1        1      1      1      1      1      1      1                     
 11     1        1        1        1        1                                                                        
 12     1        1        1        1        1        1      1      1      1      1      1      1      1      1      1
 13                                                                                                                  
 14                                                                                                                  
 15     1        1        1        1        1        1      1      1      1      1      1      1                     
 16     1        1        1        1        1        1      1      1      1      1      1      1      1      1      1
 17                                                                                                                  
 18     1        1        1        1        1        1      1      1      1      1      1      1      1      1      1
 19                                                                                                                  
 20     1        1        1        1        1        1      1      1      1      1      1      1      1      1      1
 21     1        1        1        1        1        1      1      1      1      1      1      1      1      1      1
 22                                                                                                                  
 23                                                                                                                  
 24     1        1        1        1        1        1      1      1      1      1      1      1      1      1       
 25     1        1        1        1        1        1      1      1      1      1      1      1                     
 26                                                                                                                  
 27     1        1        1        1        1        1      1      1      1      1      1      1      1      1      1
 28     1        1        1        1        1        1      1      1      1      1      1      1                     
 29                                                                                                                  
 30     1        1        1        1        1        1      1      1      1      1      1      1      1      1       
 31                                                                                                                  
 32     1        1        1        1        1        1      1      1      1      1      1      1      1      1      1
 33                                                                                                                  
 34                                                                                                                  
 35     1        1        1        1        1        1      1      1      1      1      1      1                     
 36     1        1        1        1        1        1      1      1      1      1      1      1      1      1       
 37                                                                                                                  
 38                                                                                                                  
 39                                                                                                                  
 40     1        1        1        1        1        1      1      1      1      1      1      1      1      1      1
 41                                                                                                                  
 42     1        1        1        1        1        1      1      1      1      1      1      1      1      1      1
 43                                                                                                                  
 44     1        1        1        1        1        1      1      1      1      1      1      1      1      1      1
 45     1        1        1        1        1        1      1      1      1      1      1      1      1      1      1
 46                                                                                                                  
 47                                                                                                                  
 48     1        1        1        1        1        1      1      1      1      1      1      1      1      1       
 49                                                                                                                  
 50     1        1        1        1        1        1      1      1      1      1      1      1                     
 51                                                                                                                  
 52     1        1        1        1        1        1      1      1      1      1      1      1                     
 53                                                                                                                  
 54     1        1        1        1        1        1      1      1      1      1      1      1      1      1       
 55     1        1        1        1        1        1      1      1      1      1      1      1                     
 56     1        1        1        1        1        1      1      1      1      1      1      1                     
 57                                                                                                                  
 58                                                                                                                  
 59                                                                                                                  
 60     1        1        1        1        1        1      1      1      1      1      1      1      1      1       
 61                                                                                                                  
 62                                                                                                                  
 63     1        1        1        1        1        1                           1                                   
 64     1        1        1        1        1        1      1      1      1      1      1      1      1      1      1
 65     1        1        1        1        1        1      1      1      1      1      1      1                     
 66     1        1        1        1        1        1      1      1      1      1                                   
 67                                                                                                                  
 68                                                                                                                  
 69                                                                                                                  
 70     1        1        1        1        1        1                           1                                   
 71                                                                                                                  
 72     1        1        1        1        1        1      1      1      1      1                                   
 73                                                                                                                  
 74                                                                                                                  
 75     1        1        1        1        1        1      1      1      1      1      1      1                     
 76                                                                                                                  
 77     1        1        1        1        1        1                                                               
 78     1        1        1        1        1        1             1      1      1             1                     
 79                                                                                                                  
 80     1        1        1        1        1        1      1      1      1      1      1      1      1              
 81     1        1        1        1        1        1      1      1      1      1                                   
 82                                                                                                                  
 83                                                                                                                  
 84     1        1        1        1        1        1                                                               
 85     1        1        1        1        1        1      1      1             1      1                            
 86                                                                                                                  
 87                                                                                                                  
 88     1        1        1        1        1        1             1      1                                          
 89                                                                                                                  
 90     1        1        1        1        1        1      1      1      1                                          
 91     1        1        1        1        1        1                                                               
 92                                                                                                                  
 93                                                                                                                  
 94                                                                                                                  
 95     1        1        1        1        1        1      1      1      1      1      1      1                     
 96     1        1        1        1        1        1      1      1      1      1                                   
 97                                                                                                                  
 98     1        1        1        1        1        1             1      1             1      1                     
 99     1        1        1        1        1               1                                                        
100     1        1        1        1        1        1      1      1      1      1      1      1                     
101                                                                                                                  
102     1        1        1        1        1        1                                                               
103                                                                                                                  
104     1        1        1        1        1        1                           1                                   
105     1        1        1        1        1        1                                                               
106                                                                                                                  
107                                                                                                                  
108     1        1        1        1        1                                    1                                   
109                                                                                                                  
110     1        1        1        1        1        1      1      1      1                                          
111                                                                                                                  
112     1        1        1        1        1        1                                                               
113                                                                                                                  
114     1        1        1        1        1        1      1      1      1                                          
115     1        1        1        1        1        1      1      1      1                                          
116                                                                                                                  
117     1        1        1        1        1        1      1      1      1      1                                   
118                                                                                                                  
119     1        1        1        1        1        1                                                               
120     1        1        1        1        1        1      1      1      1      1                                   
121                                                                                                                  
122                                                                                                                  
123                                                                                                                  
124                                                                                                                  
125     1        1        1        1        1        1      1      1      1      1      1      1                     
126     1        1        1        1        1        1                                                               
127                                                                                                                  
128     1        1        1        1        1        1      1      1      1      1

Max Threads Summary (Cross reference md.log for PME load and maximum allowed number of cells)

Code: Select all

Box       PME     Max
4x4x3     0.17     64
4x4x3     0.18     64
4x4x4     0.19     80
5x5x4     0.18    125
5x5x4     0.19    125
5x5x5     0.37    195
6x6x5     0.18    230
6x6x5     0.19    232
6x6x5     0.22    232
8x8x7     0.18   >448
11x11x10  0.1   >1210
11x11x13  0.09  >1573
13x13x13  0.36  >2197
14x14x14  0.36  >2744
16x16x16  0.09  >4096
18x18x18  0.22  >5832

Posted: **Sat Apr 25, 2020 8:08 pm**

_r2w_ben wrote:QRB would give part of the 10%. Having all cores working on the same task might help a bit with L1D/L2/L3 cache hits. Does the lower CPU usage translate into a slightly higher average clock speed?

Sorry for the late reply, I've locked my cores at 3800MHz to prevent "fan music" when they spin up and down, and to be able to reliably reduce Vcore.

Posted: **Sat Apr 25, 2020 11:17 pm**

Data for a new record holder for smallest box size.

p14576 - max 3x3x3 - PME load 0.17

Code: Select all

  2 = 2x1x1
  3 = 3x1x1
  4 = 2x2x1
  6 = 3x2x1
  8 = 2x2x2
  9 = 3x3x1
 12 = 2x3x2
 18 = 3x3x2
 27 = 3x3x2  18 +  9 PME

All data for 1-128 threads. 1 indicates success while blank is failure.

Code: Select all

# Threads p14336 p16802   p14365   p16804   p16501   p14570 p14542 p13832 p16420 p14378 p13851 p16422 p16423 p14574 p14576 p14593
Box     18x18x18 16x16x16 14x14x14 11x11x13 11x11x10 8x8x7  6x6x5  6x6x5  6x6x5  5x5x5  5x5x4  5x5x4  4x4x4  4x4x3  4x4x3  3x3x3
PME     0.22     0.09     0.36     0.09     0.1      0.18   0.2    0.19   0.18   0.37   0.19   0.18   0.19   0.18   0.17   0.17
  1     1        1        1        1        1        1      1      1      1      1      1      1      1      1      1      1
  2     1        1        1        1        1        1      1      1      1      1      1      1      1      1      1      1
  3     1        1        1        1        1        1      1      1      1      1      1      1      1      1      1      1
  4     1        1        1        1        1        1      1      1      1      1      1      1      1      1      1      1
  5     1        1        1        1        1        1      1      1      1      1      1      1                            
  6     1        1        1        1        1        1      1      1      1      1      1      1      1      1      1      1
  7     1        1        1        1        1        1                                                                      
  8     1        1        1        1        1        1      1      1      1      1      1      1      1      1      1      1
  9     1        1        1        1        1        1      1      1      1      1      1      1      1      1      1      1
 10     1        1        1        1        1        1      1      1      1      1      1      1                            
 11     1        1        1        1        1                                                                               
 12     1        1        1        1        1        1      1      1      1      1      1      1      1      1      1      1
 13                                                                                                                         
 14                                                                                                                         
 15     1        1        1        1        1        1      1      1      1      1      1      1                            
 16     1        1        1        1        1        1      1      1      1      1      1      1      1      1      1       
 17                                                                                                                         
 18     1        1        1        1        1        1      1      1      1      1      1      1      1      1      1      1
 19                                                                                                                         
 20     1        1        1        1        1        1      1      1      1      1      1      1      1      1      1       
 21     1        1        1        1        1        1      1      1      1      1      1      1      1      1      1       
 22                                                                                                                         
 23                                                                                                                         
 24     1        1        1        1        1        1      1      1      1      1      1      1      1      1              
 25     1        1        1        1        1        1      1      1      1      1      1      1                            
 26                                                                                                                         
 27     1        1        1        1        1        1      1      1      1      1      1      1      1      1      1      1
 29                                                                                                                         
 28     1        1        1        1        1        1      1      1      1      1      1      1                            
 30     1        1        1        1        1        1      1      1      1      1      1      1      1      1              
 31                                                                                                                         
 32     1        1        1        1        1        1      1      1      1      1      1      1      1      1      1       
 33                                                                                                                         
 34                                                                                                                         
 35     1        1        1        1        1        1      1      1      1      1      1      1                            
 36     1        1        1        1        1        1      1      1      1      1      1      1      1      1              
 37                                                                                                                         
 38                                                                                                                         
 39                                                                                                                         
 40     1        1        1        1        1        1      1      1      1      1      1      1      1      1      1       
 41                                                                                                                         
 42     1        1        1        1        1        1      1      1      1      1      1      1      1      1      1       
 43                                                                                                                         
 44     1        1        1        1        1        1      1      1      1      1      1      1      1      1      1       
 45     1        1        1        1        1        1      1      1      1      1      1      1      1      1      1       
 46                                                                                                                         
 47                                                                                                                         
 48     1        1        1        1        1        1      1      1      1      1      1      1      1      1              
 49                                                                                                                         
 50     1        1        1        1        1        1      1      1      1      1      1      1                            
 51                                                                                                                         
 52     1        1        1        1        1        1      1      1      1      1      1      1                            
 53                                                                                                                         
 54     1        1        1        1        1        1      1      1      1      1      1      1      1      1              
 55     1        1        1        1        1        1      1      1      1      1      1      1                            
 56     1        1        1        1        1        1      1      1      1      1      1      1                            
 57                                                                                                                         
 58                                                                                                                         
 59                                                                                                                         
 60     1        1        1        1        1        1      1      1      1      1      1      1      1      1              
 61                                                                                                                         
 62                                                                                                                         
 63     1        1        1        1        1        1                           1                                          
 64     1        1        1        1        1        1      1      1      1      1      1      1      1      1      1       
 65     1        1        1        1        1        1      1      1      1      1      1      1                            
 66     1        1        1        1        1        1      1      1      1      1                                          
 67                                                                                                                         
 68                                                                                                                         
 69                                                                                                                         
 70     1        1        1        1        1        1                           1                                          
 71                                                                                                                         
 72     1        1        1        1        1        1      1      1      1      1                                          
 73                                                                                                                         
 74                                                                                                                         
 75     1        1        1        1        1        1      1      1      1      1      1      1                            
 76                                                                                                                         
 77     1        1        1        1        1        1                                                                      
 78     1        1        1        1        1        1             1      1      1             1                            
 79                                                                                                                         
 80     1        1        1        1        1        1      1      1      1      1      1      1      1                     
 81     1        1        1        1        1        1      1      1      1      1                                          
 82                                                                                                                         
 83                                                                                                                         
 84     1        1        1        1        1        1                                                                      
 85     1        1        1        1        1        1      1      1             1      1                                   
 86                                                                                                                         
 87                                                                                                                         
 88     1        1        1        1        1        1             1      1                                                 
 89                                                                                                                         
 90     1        1        1        1        1        1      1      1      1                                                 
 91     1        1        1        1        1        1                                                                      
 92                                                                                                                         
 93                                                                                                                         
 94                                                                                                                         
 95     1        1        1        1        1        1      1      1      1      1      1      1                            
 96     1        1        1        1        1        1      1      1      1      1                                          
 97                                                                                                                         
 98     1        1        1        1        1        1             1      1             1      1                            
 99     1        1        1        1        1               1                                                               
100     1        1        1        1        1        1      1      1      1      1      1      1                            
101                                                                                                                         
102     1        1        1        1        1        1                                                                      
103                                                                                                                         
104     1        1        1        1        1        1                           1                                          
105     1        1        1        1        1        1                                                                      
106                                                                                                                         
107                                                                                                                         
108     1        1        1        1        1                                    1                                          
109                                                                                                                         
110     1        1        1        1        1        1      1      1      1                                                 
111                                                                                                                         
112     1        1        1        1        1        1                                                                      
113                                                                                                                         
114     1        1        1        1        1        1      1      1      1                                                 
115     1        1        1        1        1        1      1      1      1                                                 
116                                                                                                                         
117     1        1        1        1        1        1      1      1      1      1                                          
118                                                                                                                         
119     1        1        1        1        1        1                                                                      
120     1        1        1        1        1        1      1      1      1      1                                          
121                                                                                                                         
122                                                                                                                         
123                                                                                                                         
124                                                                                                                         
125     1        1        1        1        1        1      1      1      1      1      1      1                            
126     1        1        1        1        1        1                                                                      
127                                                                                                                         
128     1        1        1        1        1        1      1      1      1      1

Max Threads Summary (Cross reference md.log for PME load and maximum allowed number of cells)

Code: Select all

Box       PME     Max
3x3x3     0.17     27
4x4x3     0.17     64
4x4x3     0.18     64
4x4x4     0.19     80
5x5x4     0.18    125
5x5x4     0.19    125
5x5x5     0.37    195
6x6x5     0.18    230
6x6x5     0.19    232
6x6x5     0.22    232
8x8x7     0.18   >448
11x11x10  0.1   >1210
11x11x13  0.09  >1573
13x13x13  0.36  >2197
14x14x14  0.36  >2744
16x16x16  0.09  >4096
18x18x18  0.22  >5832

Folding Forum

Testing domain decomposition for high CPU counts

Re: Testing domain decomposition for high CPU counts

Re: Testing domain decomposition for high CPU counts

Re: Testing domain decomposition for high CPU counts

Re: Testing domain decomposition for high CPU counts

Re: Testing domain decomposition for high CPU counts

Re: Testing domain decomposition for high CPU counts

Re: Testing domain decomposition for high CPU counts

Re: Testing domain decomposition for high CPU counts

Re: Testing domain decomposition for high CPU counts

Re: Testing domain decomposition for high CPU counts

Re: Testing domain decomposition for high CPU counts

Re: Testing domain decomposition for high CPU counts

Re: Testing domain decomposition for high CPU counts

Re: Testing domain decomposition for high CPU counts

Re: Testing domain decomposition for high CPU counts