Nvidia Titan Xp

[Nathan_P]

TPF, PPD and watts consumed....

[ComputerGenie]

TPF is, was, and always will be PRCG specific, so any mention of it (unless measured against a different G/CPU running the same PRCG) will always be ancillary and generally meaningless.

[Ricky]

I am now running a titan xp in this slot FAH Control is reporting 1,369k PPD on this project: 13200 (Run 0, Clone 11, Gen 884), with a TPF of 1:36. Note the average listed is shared with a GTX980 that was in the slot.

Code: Select all

 Project ID: 13200
 Core: OPENMM_21
 Credit: 27150
 Frames: 100


 Name: win Slot 01
 Path: win-36330
 Number of Frames Observed: 300

 Min. Time / Frame : 00:01:35 - 1,442,017.7 PPD
 Avg. Time / Frame : 00:03:31 - 435,644.3 PPD
 Cur. Time / Frame : 00:01:37 - 1,397,649.8 PPD
 R3F. Time / Frame : 00:01:36 - 1,419,546.4 PPD
 All  Time / Frame : 00:01:35 - 1,442,017.7 PPD
 Eff. Time / Frame : 00:02:41 - 653,607.4 PPD

[foldy]

Tremendous PPD! Similar to Gtx 1080 Ti ?

[Ricky]

Actual PPD varies with project and time. I see a considerable rise and drop in PPD from frame to frame. This may coincide with the topology problem with more recent drivers. I had to update my driver to use this card.

Code: Select all

Project	TPF	           PPD	       Credit	   Run	Clone	Gen
p9431	0:01:22	503,360.20	47,772.60	  1454	0	12
p13204	0:01:17	985,600.50	87,837.10	       3	11	6
p11802	0:00:37	664,411.30	28,452.80	       0	76	121
p9431	0:01:13	599,260.50	50,632.00	    179	2	3
p10496	0:01:41	834,782.20	97,584.50	    174	39	6
p13204	0:01:14	1,046,138.80	89,599.80	      25	5	50
p13200	0:01:51	1,141,751.10	146,683.30	0	11	884
p9415	0:01:03	590,101.30	43,028.20	     978	1	8
p9431	0:01:21	512,711.00	48,066.70	     314	1	18
p11805	0:00:36	678,119.00	28,255.00	       0	47	26
p13204	0:01:17	985,600.50	87,837.10	      26	5	68
p13204	0:01:16	1,005,117.50	88,413.10	      43	12	4
p11802	0:00:36	692,286.80	28,845.30	       0	110	45
p10496	0:01:41	834,782.20	97,584.50	    138	38	5
p9431	0:01:24	485,490.70	47,200.50	     488	0	36

[Ricky]

As shown in my previous post, this card has a large variance of PPD over a range of projects. I have found that the GPU core loading is lower on projects that have the lower PPD. The highest PPD so far has been project 13204 with a PPD 1,364,109 with 72000 atoms. The lowest PPD has been project 9431 with a PPD of 485,490 with 2100 atoms.

[Nathan_P]

something is wrong because my 1080 shoes 895k PPD with these projects. not sure what the something is though

[bruce]

... I have found that the GPU core loading is lower on projects that have the lower PPD. The highest PPD so far has been project 13204 with a PPD 1,364,109 with 72000 atoms. The lowest PPD has been project 9431 with a PPD of 485,490 with 2100 atoms.

YMMV

I'd check the PCIe utilization factors. You may be running into bandwidth limitations if you happen to use relatively slower slot speeds (or If you happen to use switched PCIe connections). For systems with multiple GPUs. many "simple" rules may not apply to you.

Faster GPUs are always going to be more dependent on bandwidth, particularly when they're working on smaller proteins. A project with 72000 atoms is going to spend a higher percentage of time doing specific aspects of the analysis than a project with 2100 though each has to go through each aspect of the analysis during each step. [It's hard to figure that out when you're just looking at AVERAGE processor utilization and AVERAGE bandwidth.]

See Ricky's comment here and the response from PG.
We're aware of the problem and working toward a solution.

[Ricky]

Bruce,

Both of the projects are running on the same card. In this machine, one x16 3.0 slot is dedicated to one E5 processor. The other E5 has the other x16 v3.0 slot and a couple other smaller unused slots. I believe the issue is as you point out, the larger core count boards are underutilized for projects with a small number of atoms.

As it is now, I would not recommend this card for folding, cost /PPD is poor on many projects. The GTX1080 costs have as much and on average, performs at 80% the performance of this card for the science (PPD).

[Ricky]

Nathan_P,
The GTX1080 clocks higher than the XP. I see about 1.55 GHz on the XP. I also have a penalty on running windows. My 1080s average a bit less than 800k PPD.

[Foxbat]

Both of the projects are running on the same card. In this machine, one x16 3.0 slot is dedicated to one E5 processor. The other E5 has the other x16 v3.0 slot and a couple other smaller unused slots.

Here's something I wonder about. How efficient is cross-traffic between CPU cores in one processor to the PCI-e Lanes on the other CPU? If CPU0 feeds GPU0 and CPU1 feeds GPU1 (keeping it simple) what happens if the Core_xx FAH Client Process for GPU1 is running on the CPU0? Does that add any extra overhead?

[Ricky]

As it stands now, I am only running the XP in this machine. Before I placed the XP in the machine, I had a 980 standard clocked card in one slot and a 980SC factory over-clocked card in the other slot. I saw PPD of both cards comparable to their speed grades and about what I believe others would report for windows machines.

The old cards used 2 6-pin power connectors. I only have 4 of these connectors. The XP needs 1 6-pin and 1 8-pin connector. I used an adapter for 2 6-pin to 1 8-pin for the XP, so I only have 1 6-pin left. This is why I am running only the XP at this time.

[toTOW]

On Windows, I'm pretty sure that small proteins with a limited number of atoms are CPU limited ... if your CPU can do it, try to overclock it a bit and you'll see both GPU load and PPD rise ...

[Ricky]

toTOW,

Are you saying that the CPU clock rate could be effecting the GPU loading? NVidia already needs one CPU thread as it is. I would be happy to dedicate a second thread, if I could. As it is I have 3 threads idle.

[bruce]

It is unknown whether a second CPU would benefit a GPU FAHCore.

It's likely that that if a single thread on a hyperthreaded CPU happens to be paired with an idle thread, you might get petter performance than if the other half of the thread pair is heavily loaded, but you probably satisfy that condition anyway.