AMD Radeon VII - Double Precision

[jlopz]

Does anyone know if the AMD Radeon Instinct MI50 is supported in FAH? The GPU listing shows several possible GPU references, but it's not clear. There's the MI25 and MI100, but I don't see anything like MI50.

[muziqaz]

It is Vega 20, double precision or not, it is supported by FAH. Output ranges from 1m to 2.6m PPD in OpenCL. Probably double that with HIP

[arisu]

I don't think GPU vendors "hobble" their FP64 precision. It's true that each double-precision unit will have a 1:2 speed ratio compared with each single-precision unit, but these cards include more single-precision units for FP32 than double-precision units for FP64 because single precision is very useful in games. You can't just combine two SP units to become one DP unit without a lot of extra silicon (it's not as simple as an integer ALU where you can just double their width snapping two together and wiring a trace between them for the carry). I think you can use an DP unit as two SP units, but it comes with a lot of caveats that make it only useful for some kind of vector arithmetic, but don't quote me on that.

A single Kepler GK110 SMX (Streaming Multiprocessor) has 192 SP units, 64 DP units, 32 special-function units, and 32 load/store units. I think that is the the Nvidia card with the best single-to-double ratio of 1:3 (192:64). But that's also a datacenter GPU. Consumer GPUs have closer to one double-precision unit for every 24 to 64 single-precision units.

This is fine for FAH because each DP unit is still operating at full speed (1:2 the speed of an SP unit) and because there are fewer calculations that need DP, you won't be saturating all the DP units until long after the SP units are all in use.

[muziqaz]

Yes, implementing pure double precision units in hardware requires a lot of extra transistors and space on the die. I'm not sure what AMD was smoking when they released Radeon 7, as I'm sure they have never got their money back from those cards.

[FaaR]

Output ranges from 1m to 2.6m PPD in OpenCL. Probably double that with HIP

Is that a guesstimate of what a theoretical HIP core might accomplish, or is there actually some way to enable HIP support in F@H?

The way OpenCL is absolutely stone dead, should not F@H do something more to move off of this dead and buried API? Isn't Vulkan the new OpenCL for all intents and purposes? Pretty sure I read something along those lines years ago already.

[Joe_H]

Just a guesstimate. As for Vulkan, AMD has not done much with it. Basically they are putting their resources into HIP after initially just enabling it for pro level cards for data center usage. Holdup from what I understand is partly waiting on full programming info from AMD and partly adding HIP support to OpenMM which the GPU folding cores are based on.

[FaaR]

Just a guesstimate.

Ah, ok, thanks... So it's just a matter of being patient, then.

[muziqaz]

Output ranges from 1m to 2.6m PPD in OpenCL. Probably double that with HIP

Is that a guesstimate of what a theoretical HIP core might accomplish, or is there actually some way to enable HIP support in F@H?

The way OpenCL is absolutely stone dead, should not F@H do something more to move off of this dead and buried API? Isn't Vulkan the new OpenCL for all intents and purposes? Pretty sure I read something along those lines years ago already.

We have full numbers of pure performance increase going from OpenCL to HIP, but FAH uses bonus system, plus the performance tests were not run using fahcores, but openMM benchmarks.
That is the reason I always write guesstimate of PPD increase, since 60% of performance increase is not 60% of PPD increase.
60% of performance increase is waaaay more in PPD increase, but we don't know exactly how much, since HIP FAHcore is taking sweet time to be released :/
Vulkan is new OpenGL, though Vulkan has compute "modules" too

[muziqaz]

Just a guesstimate. As for Vulkan, AMD has not done much with it. Basically they are putting their resources into HIP after initially just enabling it for pro level cards for data center usage. Holdup from what I understand is partly waiting on full programming info from AMD and partly adding HIP support to OpenMM which the GPU folding cores are based on.

At this moment the only thing which is holding HIP FAHcore release is FAH Consortium and lack of manpower to develop it. People who are doing this are either super busy with their life priorities, uni responsibilities, or they have done everything they could and just waiting for people with uni responsibilities to find some free time to get things going.
FAH had all necessary stuff from AMD for over a year now, if not more

[FaaR]

That is the reason I always write guesstimate of PPD increase, since 60% of performance increase is not 60% of PPD increase.
60% of performance increase is waaaay more in PPD increase, but we don't know exactly how much, since HIP FAHcore is taking sweet time to be released :/

That sounds really rather encouraging actually. A few WUs have hit 10+M PPD on my 7900XTX with bonus, so even if bonus was entirely linear, 16 million would be...:DDD And if even more... :DDDDDDDDDDDD

Of course, vast majority of WUs only get like 5M so variability with AMD is huge unfortunately and prevents me from regularly running F@H (hard to justify the 350W power draw for so little work done, relatively speaking.) Would be nice if we could run two simultaneous projects on large GPUs to get a proper workout of all those shader processors, or if WUs could be embiggened instead perhaps to feature many many more atoms than today.

Thanks for your reply btw. Very encouraging stuff. Sad though that resources for something so critical are so constrained, considering how many people there are in this world with insane wealth. Surely just a handful of full-time programmers would make a huge difference for you people, yet would not impact these Richie Richs the slightest.

[appepi]

@ FaaR Thanks for reminding me about Richie Rich. For a few milliseconds I was 15 again. "Embiggening" is good too.

[arisu]

Would be nice if we could run two simultaneous projects on large GPUs to get a proper workout of all those shader processors, or if WUs could be embiggened instead perhaps to feature many many more atoms than today.

For Nvidia, I found that MPS works very well on Linux (it does not support Windows though): viewtopic.php?t=42752

Supposedly AMD does not even need a special daemon and multiple projects can run on it simultaneously, but I haven't checked myself and don't know if the overhead is comparable to MPS. You could try it yourself, probably by creating two resource groups and enabling the same GPU on both of them.

Some WUs don't need to have increased atom counts. There are some WUs which can take advantage of wide GPUs by simulating multiple copies of a small peptide at once, spaced apart (while ensuring they are far enough apart that they don't interact with each other, i.e. past the so-called electrostatic cutoff), but not all projects can take advantage of that.

[muziqaz]

AMD can run multiple WUS on one GPU on both, Linux and Windows, as long as GPU supports hardware scheduling. And obviously it is enabled.

[FaaR]

AMD can run multiple WUS on one GPU on both, Linux and Windows, as long as GPU supports hardware scheduling. And obviously it is enabled.

How does one utilize this? Thank you!

[muziqaz]

Create 2 resource groups, both having just that GPU ticked