Enterprise and cloud service provider participation

[jdelisle]

I'm new to FAH and have near-zero medical/ science knowledge. I gather that given enough computing resources, FAH can make a material difference in the fight against coronavirus. If that is incorrect please let me know.

By way of introduction, I'm a cloud solution architect and I assist my employer to by designing, deploying, and operating applications at scale globally.

I'm wondering what the FAH team has done around the following areas:

1 - Contacting global cloud service providers, like Azure, Google, Amazon, AliBaba, IBM, and so on regarding gaining access to their idle resources. Do you have contacts?
2 - Producing templates, VM images, docker containers, automation, etc. that facilitate deploying FAH at-scale to cloud platforms or on-prem enterprise vmware/ hyper-v environments.
3 - Do you have a team focused on this kind of stuff who I can be connected with? I'm wondering how I can help.

Thanks!

[foldy]

I tried the vast.ai cloud provider with VM docker image nvidia-opencl running Linux Ubuntu. There I could remote install FAH and run it successfully on GPUs. Much cheaper than any big cloud like Google or Amazon.

[bruce]

As foldy suggests, the resources that FAH needs are not the ones that cloud service providers are able to provide at a reasonable price -- though I'm sure FAH would be happy to accept donations from them FAH depends on highly parallel compute intensive resources. In the early days of FAH, it focused on unused multi-threaded CPU resources. The capabilities of GPUs to process highly parallel 3D calculations grew rapidly, the main focus moved to OpenCL and/or CUDA processing -- again using almost 100% floating point calculations -- which are considered expensive by the cloud providers, even when called "idle" resources -- but available on home machines suited for gaming.

[jdelisle]

Thanks for getting back to be Bruce.

I've mined crypto and can relate to the performance difference FAH would experience between GPU and CPU. It's no surprise that GPU time is considerably more expensive than CPU time.

Since I'm new here, I'm curious just how large the processing power already is, and how many more CPU cores it would take to make an appreciable difference in FAH's work on coronavirus. Would another 1,000 CPU cores even make a difference? 50,000? Same question goes for GPU. How many Tesla V100 GPU cores would it take to make a difference? I get that every bit counts, but I need to gauge how worthwhile my time and effort would be. For example, if you already have a million cores of CPU, then having me spend days chasing down an additional 1,000 really gains nothing.

Do you know if someone has approached the cloud service providers yet, specifically regarding coronavirus? I expect their executive leadership fully understand the urgency and potential for global disruption. They may be far more generous with their resources now than in the past. This is an "all hands" problem - we need to pull together as a species and beat this, and idle cores and GPUs can help. If no one has engaged them, who in your team should be pursing this, and do they need help?

Speaking from an enterprise IT perspective, there's far too much friction with your current approach to distributing your software. It'd be easier for a generous corporation to "pitch in" and run FAH if it were easier for their IT staff to deploy. This is where templates, marketplace images, and so on come in. Is there someone on your team working this angle, and do they need help?

Thanks again

[bruce]

https://stats.foldingathome.org/os

[JimboPalmer]

Speaking from an enterprise IT perspective, there's far too much friction with your current approach to distributing your software. It'd be easier for a generous corporation to "pitch in" and run FAH if it were easier for their IT staff to deploy. This is where templates, marketplace images, and so on come in. Is there someone on your team working this angle, and do they need help?

I seriously doubt it.

Any 'automatic' install is going to be misused by hackers wanting to inflate their score. F@H is very clear that you need the permission of the owner of the hardware. It also 'serializes' the install so it can't be imaged. All this is not particularly IT friendly, but keeps the software honest.

I am just a user like you, not a staff member. As IT for a mental health org, I had 55 Computers folding.

[HaloJones]

Interesting subject but I think you underestimate the cost of managing and maintaining large numbers of folding units. And the cloud providers' "idle resources" may sound like it would be free for them to be folding but it's also a quite serious investment in energy costs.

I read recently (on the internet somewhere so it must be true!) that cryptomining is using 10% of the world's energy all of which just gets turned into heat. Saving us from cancer but cooking us in the process isn't a great trade-off.

[foldy]

Here one user created a docker image for fahclient but I did not try it myself.
https://github.com/arnaudhb/docker-fahclient

[jdelisle]

I think you underestimate the cost of managing and maintaining large numbers of folding units.

This is exactly the problem proper automation and deployment solves.

And the cloud providers' "idle resources" may sound like it would be free for them to be folding but it's also a quite serious investment in energy costs.

I'm aware they'd incur costs. Perhaps you're underestimating just how bad COVID-19 is. Look to the flu of 1918 for an example, and I'd suggest it was mild compared to this. 20 to 50 million dead. 3+% population wiped out. Conservative estimates for COVID-19 are like 60 to 80% infection rate, with 3.4% fatality. That's pretty significant, so significant I could see CSPs wanting to help.

If this application is intentionally automation-hostile, that's a self-defeating design. To get enterprise IT interested this has to be simple and has to be made available in formats they're prepared to work with.

[jdelisle]

Any 'automatic' install is going to be misused by hackers wanting to inflate their score. F@H is very clear that you need the permission of the owner of the hardware. It also 'serializes' the install so it can't be imaged. All this is not particularly IT friendly, but keeps the software honest.

I am just a user like you, not a staff member. As IT for a mental health org, I had 55 Computers folding.

Just to be clear, I'm not suggesting anything that would make it easier to install surreptitiously.

As an IT guy, I'm sure you understand that no one accidentally deploys a few hundred VMs from an image, and no one accidentally deploys thousands of workers in Azure and accidentally pays the bill!

I'm talking about engaging cloud service providers to deploy this at scale, or seeking compute contribution from large enterprises. Not a few hundred cores - I'm shooting for tens of thousands.

[jdelisle]

https://stats.foldingathome.org/os

Perfect, thanks Bruce. I'm hopeful that CSP and enterprise donors will have enough idle capacity to significantly help.

I'm assembling a team within my organization, and one ways we hope to help is by connecting contacts at Azure, Google, Amazon, etc. with FAH leadership to discuss accessing idle compute resources. I'm having a difficult time finding contact info for anyone in FAH leadership. Do you have any suggestions? Feel free to PM.

Again, I'm no scientist, so maybe someone can answer this question: How confident are we that FAH's work against coronavirus will produce something meaningful? How much computing power is needed to accelerate this? We can make a compelling argument to donate if this can actually save lives.

[jdelisle]

Here one user created a docker image for fahclient but I did not try it myself.
https://github.com/arnaudhb/docker-fahclient

Thanks, that's helpful!

[gbowman]

We (the Folding@home leadership) would love to collaborate with cloud service providers!