171.64.65.56

[Hyperlife]

Since you seem to be an authority on Langouste, please break it down for us and describe how the connections are made, and in what order, to what parts of Stanford or local ports, etc. Thanks.

It's all explained right here and would have taken you two minutes to research it yourself. Guess you didn't do that, huh? Or maybe it's too complicated for you? If that's the case, here's an explanation in simple terms:

Step 1: Main client completes WU.

Step 2: Main client attempts to contact the Work Server through a proxy port assigned by the user in order to upload the completed WU. Langouste listens on that port and interrupts the connection. The connection is never made.

Step 3: Main client tries a few times to connect to the Work Server and Connection Server and aborts. Remeber that no actual connection attempt happens; no bits leave the local machine here. This process takes about 8 seconds on my Q6600.

Step 4: Main client proceeds to connect to an Assignment Server, then connects to a Work Server and downloads a new WU in one second on my Q6600. Main client begins folding the new WU.

Step 5: One minute after the main client attempts to return the WU, the forked client starts using the "-send x" flag (not "-send all"). Forked client proceeds with the upload and terminates when complete, and then deletes the results file. If the upload fails, the forked client also terminates, does not attempt to resend, and leaves the results file alone.

Step 6: Six hours later, the main client attempts to upload the same WU. If the forked client's upload was successful, the main client discovers that the results file has been deleted and aborts the upload without making a connection. If the forked client was unsuccessful in uploading the WU, the main client attempts the upload as it normally would.

Is that clear enough for you?

The same number of connections are made with or without Langouste. Due to the one-minute delay in Step 5, the connections are normally not simultaneous.

I'm very reasonably sure that Langouste is not the problem. As you can see from the foregoing, this is not idle speculation.

[7im]

Since you seem to be an authority on Langouste, please break it down for us and describe how the connections are made, and in what order, to what parts of Stanford or local ports, etc. Thanks.

It's all explained right here and would have taken you two minutes to research it yourself. Guess you didn't do that, huh? Or maybe it's too complicated for you? If that's the case, here's an explanation in simple terms:

Step 1: Main client completes WU.

Step 2: Main client attempts to contact the Work Server through a proxy port assigned by the user in order to upload the completed WU. Langouste listens on that port and interrupts the connection. The connection is never made.

Step 3: Main client tries a few times to connect to the Work Server and Connection Server and aborts. Remeber that no actual connection attempt happens; no bits leave the local machine here. This process takes about 8 seconds on my Q6600.

Step 4: Main client proceeds to connect to an Assignment Server, then connects to a Work Server and downloads a new WU in one second on my Q6600. Main client begins folding the new WU.

Step 5: One minute after the main client attempts to return the WU, the forked client starts using the "-send x" flag (not "-send all"). Forked client proceeds with the upload and terminates when complete, and then deletes the results file. If the upload fails, the forked client also terminates, does not attempt to resend, and leaves the results file alone.

Step 6: Six hours later, the main client attempts to upload the same WU. If the forked client's upload was successful, the main client discovers that the results file has been deleted and aborts the upload without making a connection. If the forked client was unsuccessful in uploading the WU, the main client attempts the upload as it normally would.

Is that clear enough for you?

The same number of connections are made with or without Langouste. Due to the one-minute delay in Step 5, the connections are normally not simultaneous.

I'm very reasonably sure that Langouste is not the problem. As you can see from the foregoing, this is not idle speculation.

Yes, clear, thanks. But it wasn't for my benefit that I asked.

So as long as the download is less than one minute, Langouste does not have more than one connection open at the same time. If the download takes more than one minute, then Langouste uses twice as many connections as the regular client. Good to know.

Anyone know the download file size on a -bigadv WU?

[Hyperlife]

So as long as the download is less than one minute, Langouste does not have more than one connection open at the same time. If the download takes more than one minute, then Langouste uses twice as many connections as the regular client. Good to know.

No. The number of connections is still the same. If there is an overlap, two of the connections happen simultaneously, but the total number of connections doesn't change -- it's not "twice as many".

Anyone know the download file size on a -bigadv WU?

For purposes of this discussion, this is irrelevant. No BigAdv WUs are located on 171.64.65.56 -- the only ones currently listed on psummary are 2662, 2669, 2675, 2677, 6701, and 6702. From what kasson has said, only the A3 WUs are being actively assigned right now -- 6701 and 6702.

For the record, the net load on this server right now has dropped to 46. Looks like the problem may be fixed without any changes to Langouste.

[7im]

So as long as the download is less than one minute, Langouste does not have more than one connection open at the same time. If the download takes more than one minute, then Langouste uses twice as many connections as the regular client. Good to know.

No. The number of connections is still the same. If there is an overlap, two of the connections happen simultaneously, but the total number of connections doesn't change -- it's not "twice as many".

Anyone know the download file size on a -bigadv WU?

For purposes of this discussion, this is irrelevant. No BigAdv WUs are located on 171.64.65.56 -- the only ones currently listed on psummary are 2662, 2669, 2675, 2677, 6701, and 6702. From what kasson has said, only the A3 WUs are being actively assigned right now -- 6701 and 6702.

For the record, the net load on this server right now has dropped to 46. Looks like the problem may be fixed without any changes to Langouste.

The normal client opens one connection, uploads the completed WU, closes that one connection, then opens one connection, downloads a WU, then closes that one connection.

Assuming the download takes more than one minute, Langouste is still downloading a new WU = First connection open.
And then Langouste starts uploading the finished WU = Second connection open.

How is having 2 connections open at the same time NOT double?

Now granted, that first connection probably closes pretting quickly because the download shouldn't take very long. But why wasn't a small measure of safety built in to assure an upload and download never happen at the same time? Is waiting 2 minutes really that much to ask?

Actually, a better feature would be to start the upload after a new WU downloads. If the download only takes 8 seconds, then the upload could start 52 seconds sooner. If the download takes 2 minutes, Langouste could wait a bit longer and not add to server congestion. If the client fails to download a new WU, Langouste could timeout in about 5 minutes, and then start the upload. And by then, the client has already backed off from trying to connect so often, and again prevents adding to any congestion.

[Hyperlife]

How is having 2 connections open at the same time NOT double?

It's double the number of concurrent connections, not total connections, as your statement "Langouste uses twice as many connections as the regular client" implies. Please be more precise in your use of language.

As for your suggestions, go ahead and post them in this thread. I'm sure tear will let you know if they're feasible.

[7im]

Yes, I will try to be more precise. But since the total number of connections has no impact in regards to server performance, any future statements about connection counts can be assumed to imply concurrent connections.

[Hyperlife]

Yes, I will try to be more precise. But since the total number of connections has no impact in regards to server performance, any future statements about connection counts can be assumed to imply concurrent connections.

Really? No impact? If my client uses four non-simultaneous connections during a WU upload-download cycle while everyone else uses two, then at least one user will have a connection slot unavailable to them when I'm using the two extra slots.

Doesn't sound like "no impact" to me.

[7im]

Maybe you aren't being precise enough.... If they are non-simultaneous, then you never have more than 1 connection open at the same time, same as the client that only uses 2 non-simultaneous connections. As long as the work server closes and reopens the connection promptly, it doesn't matter if I make 20 connections or 2 connections. Serial is no big deal, parallel is a big deal.

[Hyperlife]

Serial is no big deal, parallel is a big deal.

On a server with a net load of 200, serial is as big a deal as parallel is. Any extra non-simultaneous connection will still keep the load high. The only way to lower the net load is to lower the number of total connections, both serial and parallel.

[7im]

Lower the total concurrent connections, yes. Those are the parallel connections. Those are the connections that drive up the net load.

Serial connections do not drive up the net load.

[Hyperlife]

Serial connections do not drive up the net load.

Let's say you have a server to which 10,000 users are trying to connect. Each user sends one request within a 60 second window. There will, of course, be some overlap between users' connections during that 60 seconds, and some net load will be generated. On average there will be 10000 / 60 = 166 2/3 connection attempts every second.

Now let's say you have that same server and the same 10,000 users trying to connect to it. However, now let's assume that each user sends two requests within a 60 second window, and the two requests are serial -- no user sends their two requests at the same time. Now you have a grand total of 20,000 requests, twice as many as before; the average number of connections per second is now 333 1/3.

Are you saying there won't be any additional net load even though the server is trying to respond to twice as many requests in the same 60 seconds? Remember that these are all serial requests -- no single user is sending two requests at the same time.

[7im]

When did we switch from talking about one client to talking about 10,000?

[Hyperlife]

Much of this thread has focused on the cumulative effects of Langouste users on a server's health. The argument was made that Langouste uses more total connections than the client alone. I think I've conclusively refuted that a few times over. Both the client alone and with Langouste will make only three connections: to the Work Server to upload, to the Assignment Server, and then to a Work Server to download. Langouste merely rearranges the order by delaying the upload.

Then you changed the argument to serial versus parallel connections and claimed that serial connections have no effect on net load. My argument is that, when a server is overloaded, any increase in connections, whether serial or parallel, will prolong the overload. Servers don't get to a net load of 200 with one client, you know. Wasn't that the whole problem with this server in the first place?

You even posited that an increase in Langouste users running the Windows version could be such a cause, remember?

Do try to keep up.



Mods, feel free to lock this thread. I'm done here.

[7im]

Much, but not all. You can't jump from the micro, talking about one client's total thread count in one post, and then suddenly jump to the macro view of 10,000 cleints in the next post without losing a few people along the way.

But I agree, in part, that Langouste does not use more total serial connections than a standard client. But clearly I have shown that it is possible for Langouste to use more total concurrent connections than a standard client, if the new WU download takes longer than 1 minute.

I didn't change the argument, I merely focused on the portion of the argument that you overlooked. The part that could potentially cause more traffic and more overload.

I did NOT posit that Win Langouste users are the cause. My position was that I could make that CLAIM just as easily as you CLAIMED to refute it because you had presented no evidence to back up your claim. My position was to NOT assume innocence or guilt without presenting evidence either way!

And then I proceeded to ask you for more evidence.


Even with the potential to cause more server congestion, I agree with Hyperlife in thinking that Langouste did NOT contribute to that congestion in this case. However, a better understanding of the rational behind that 1 minute delay might be in order.

[bruce]

Topic locked.

For the record, the poor server performance that we were seeing was based entirely on the maximum number of simultaneous connections that it could handle. From the explanationsb given above, it sounds like all that is being done is effectively reversing the order of the download and the upload, using the same number of connections and as long as a download does not exceed one minute, the number of simultaneous connections doesn't change, either.

It does sound like it is a well-written application. Since there's already a request to reverse that order in V7, there's a reasonable chance that Langouste may become obsolete (if that request is satisfied in the final V7 code and we'll have to wait and see if that happens). If not, then it will continue to be a helpful application, particuarly for those with limited upstream bandwidth.