Location-aware WU assignments and uploads?

[Napoleon]

I wonder if it would be doable? Well, doable for sure, but feasible? For the types of WUs I fold, most if not all of them are coming from the US. AFAIK, there are FAH servers (BigAdv only?) at Stockholm, Sweden as well - a virtual stone's throw from my location. At first glance, it makes little sense that my WUs are served from across the Atlantic ocean, same goes for uploads. Of course, things are never quite as simple as they seem - server load balancing, possible duplication of server resources, extra maintenance hassles leap to mind, and the list probably goes on and on. I'm fairly sure this has been considered already, but I figured there's no harm in asking.

[7im]

Having servers located around the world is a relatively new addition to the project. It's a great idea to request a new feature relative to that new addition.

A lot of back end work on the assignment servers and work servers would be needed, and the servers locations divided up in to relative zones, and a corresponding client update to allow you to state a preference for local WUs, but any thing is possible. I like it!

But like -advmethods, or any other WU related setting, it would still only be a stated preference for a specific feature. There's no guarantee you would always get a local WU.

We might as well add this as a request to V7, and see what happens.

EDIT: New Ticket: https://fah-web.stanford.edu/projects/F ... ticket/746

[Napoleon]

Well, I was thinking about back end optimizations in the first place. The client doesn't need to know, preferences schmeferences, the smarts come from the AS(s).

Anyway, I'm glad that you wrote a ticket about this, 7im. I'm confused by this statement in the ticket, though: "This is not possible with the Folding@home architecture". Now that I've given some further thought to the idea, I actually have terrible nightmares of having to implement location awareness to FAH, but the idea is there, and it refuses to go away...

[P5-133XL]

Personally, rather Location-aware WU assignments and uploads, what I would like would be for all the servers to be connected to the same network (probably via a VPN since they currently are physically not localized) so that QoS would be much easier to implement. That being said, being able to limit the servers one uses to a small set of relatively local servers would be good and would significantly help implementing QoS too.

[7im]

... but the idea is there...

I had a quick chat with Dr. Pande about this idea. He did not dismiss it. Said he would look at it again in the future.

I can think of a few hurdles... It's a lot of work, for very little return.
Network speeds are not that different between the various locations, and are continuously improving.
There aren't that many fah server locations yet, so helpfulness is very limited.
And there are not enough work servers at different locations all sending out the same kind of work. For example, SMP work units may not be available from all FAH locations around the world. Location awareness doesn't help if all the -bigadv work units all come from one server at Stanford.

FAH needs to grow a bit bigger for this to make it more worthwhile. Hopefully V7 does that for us so we can include something like this in V8.

[codysluder]

One issue is that certain types of WUs are only at one location. Last time I checked, all bigadv assignments came from Sweden. GPU assignments only same from California. PS3 assignments only came from California. It may have changed, of course, but the only the classic and regular SMP had projects that were being studied in more than one part of the world.

[Napoleon]

Thanks, I'm pleased to hear that the idea wasn't dismissed outright. I got into thinking about synchronization problems. Let's say that I'd complete gen N and return it to Sweden. If someone in the States got assigned gen N+1 for some reason, I guess the WU would then have to be moved or copied over anyway. Even if it happened entirely within back end, the servers acting as proxies for each other and for the clients as well, the overall network traffic and server loads might not be reduced after all.

Thing is, I've gotten the impression that WU downloads are consistently much smaller than uploads of finished WUs. I guess it could be because when you start folding a WU, you only need to know the initial positions and velocities of the particles, right? During folding, you're actually generating additional trajectory data. Once a WS receives a completed WU, it may or may not store intermediate steps, but they don't have to be carried over to the new WU. Handle the "extra" data transfers and storage more or less locally, and have the servers move only the bare essentials between them.

Having said all that, I don't really know if this is how molecular dynamics work. Presuming it even could work as described above, I can see it would indeed be much more complicated optimization to implement than I initially thought, with very much questionable return of investment.

[k1wi]

I imagine that there would need to be 'behind the scenes' transfer of completed work units if only the essentials got transferred. I'd imagine that the extra info contained in the completed work unit would be essential for scientific integrity and getting them back to Stanford's giant storage array would be of fair importance.

[Napoleon]

Yep, I have no idea if something like that would work, I'm managing to confuse even myself. Anyhow, here's another thought.
Runs & Clones of a project are unique trajectories, while Gens are the sequential part. So, when a project is created, for example 100 runs and 100 clones == 10000 trajectories (gens), would it be possible to scatter the runs and clones across work servers? Say 0 - 3333 served from Stanford, 3334-6666 served from Stockholm and 6667-9999 served from Hong Kong.

The AS would detect your geographical location and direct you to the closest WS. No guarantees given that you'd always be served from the closest one, though. Gets tricky once again if/when the trajectories need to be synced with each other at certain point(s). But I got the urge to speculate...