Langouste -- WU upload/download de-coupler (+upload capping)

[Hyperlife]

Looks like it's working again! My second rig was able to contact the assignment server, and the forked client successfully uploaded the completed WU.

I've now configured my first rig to be identical to the second: running Langouste not as a daemon and using 127.0.0.1 as the proxy address. If it works, I'll try putting it back in daemon mode and using localhost instead of 127.0.0.1.

Edit: I think I just discovered the problem. I tried telnetting to localhost port 8880, and Ubuntu tried connecting via IPv6 first, then IPv4. I commented out the IPv6 stuff in /etc/hosts, and now it only tries 127.0.0.1 when telnetting to locahost. Maybe that was confusing the proxy?

[tear]

Hmm, good catch! Let me try and reproduce it here.

I'll let you know

[tear]

Got it! Okay, so it's actually the client that gets confused.

The localhost indeed is the culprit. As you noticed it resolves to ::1 (IPv6 loopback) on your system in the first place.
The problem is that client apparently uses very, very old call to resolve host names (which only gives you first match).
Client gets ::1, disregards it (it doesn't support IPv6 sockets) and _never_ makes the connection to Langouste.

I remember that Windows Vista and later behave identically, hence Windows setup instructions in Langouste's documentaiton
explicitly mention 127.0.0.1 I guess we'll have to make them uniform across the board and ask folks to enter 127.0.0.1.

Does that help?

Thanks again!
tear

P.S.
I'll make appropriate changes in the documentation and update the first post.

[Hyperlife]

Does that help?

Yup. Both of my rigs are now working fine with 127.0.0.1 explicitly listed as the proxy address.

I'll move them both back to daemon mode and will let you know if they have any further problems -- I doubt they will.

Thanks again for your help!

[entraidelec]

Hi ;

I have no words to express how happy I am with langouste !
In french we could say "C'est de la Balle", something like "it's a bullet".

I have installed flawlessly the V3-0.15.6 on a Win-xp machine running SMP Workunits. And now the 55 minutes of uploading are forgotten !
Due to an absolut horror ultra slow uplink, the uploads represent a waste of 1 to 3% WU time .

Thanks.

Regards.

[tear]

Langouste 0.15.7 is now available. Details available in the first post.

0.15.7 fixes two major (though extremely sporadic) Linux reliability issues. Upgrade is strongly recommended to all Langouste Linux users.

[rurushu]

Hi everyone,

I've been using langouste for a while now and it's been working well with smaller -bigadv units, but tonight it tried to upload one of the new 6903 units and I ran into a problem. After a while of uploading it restarted the upload giving the following error:

Code: Select all

ERROR: remote recv() failed: Connection reset by peer (104)

I retried a couple of times, always getting the same error, but I noticed that it always stops after exactly 2 hours of uploading (my connection is really slow). Then I removed the -r 28160 flag and finally got it to upload, but it took just under 2 hours. So now I'm worried about my next unit (6904) which will obviously take more than 2 hours to upload. Could this be a server side timeout issue?

Here's the langouste log from tonight:

Code: Select all

Wed Aug 24 22:17:56 2011 Langouste3 0.15.7 (compiled Sun Jul 24 21:43:55 CEST 2011 by rurushu@rurushu-ubuntu)
Wed Aug 24 22:17:56 2011 Langouste3 comes with ABSOLUTELY NO WARRANTY; for details
Wed Aug 24 22:17:56 2011 see `COPYING.txt' file located in source directory
Wed Aug 24 22:17:56 2011 Default Langouste helper temp directory: /tmp/langouste-rurushu/
Wed Aug 24 22:17:56 2011 Ratelimit: Output rate: 28160 bytes/s (27.50 kBps)
Wed Aug 24 22:17:56 2011 Listening on 127.0.0.1:8880
Wed Aug 24 22:18:08 2011 Accepted connection from: 127.0.0.1:54729
Wed Aug 24 22:18:08 2011 PID for socket: 1777
Wed Aug 24 22:18:08 2011 PID 1777: issending: 0
Wed Aug 24 22:18:08 2011 ===> PID 1777 is (most likely) contacting WU server, content length: 222562721
Wed Aug 24 22:18:08 2011 ===> Helper pid: -1
Wed Aug 24 22:18:08 2011 ===> PID 1777: numcores: 0
Wed Aug 24 22:18:08 2011 ===> now: 1314217088, last helper launched at: 0
Wed Aug 24 22:18:08 2011 ===> Launching helper: '/proc/1777/cwd/langouste-helper.sh' (exe name: '/home/rurushu/fah/fah6')...
Wed Aug 24 22:18:08 2011 ===> Forked 1881
Wed Aug 24 22:18:08 2011 (0) Local: received 16384 bytes, sent 0 bytes
Wed Aug 24 22:18:08 2011 Accepted connection from: 127.0.0.1:54730
DIRNAME: /proc/1777/cwd
READLINK: /home/rurushu/fah
BASENAME: langouste-helper.sh
Wed Aug 24 22:18:08 2011 PID for socket: 1777
Wed Aug 24 22:18:08 2011 PID 1777: issending: 0
Wed Aug 24 22:18:08 2011 ===> PID 1777 is (most likely) contacting WU server, content length: 222562721
Wed Aug 24 22:18:08 2011 ===> Helper pid: -1
Wed Aug 24 22:18:08 2011 ===> PID 1777: numcores: 0
Wed Aug 24 22:18:08 2011 ===> now: 1314217088, last helper launched at: 1314217088
Wed Aug 24 22:18:08 2011 ===> WARNING: can only launch one helper in 2 minutes (per client)
Wed Aug 24 22:18:08 2011 (0) Local: received 16384 bytes, sent 0 bytes
Wed Aug 24 22:18:08 2011 Accepted connection from: 127.0.0.1:54731
Wed Aug 24 22:18:08 2011 PID for socket: 1777
Wed Aug 24 22:18:08 2011 PID 1777: issending: 0
Wed Aug 24 22:18:08 2011 ===> PID 1777 is (most likely) contacting WU server, content length: 222562721
Wed Aug 24 22:18:08 2011 ===> Helper pid: -1
Wed Aug 24 22:18:08 2011 ===> PID 1777: numcores: 1
Wed Aug 24 22:18:08 2011 ===> now: 1314217088, last helper launched at: 1314217088
Wed Aug 24 22:18:08 2011 ===> WARNING: helper launched within last 2 minutes, delaying autosend
Wed Aug 24 22:18:08 2011 (0) Local: received 16384 bytes, sent 0 bytes
Wed Aug 24 22:18:08 2011 Accepted connection from: 127.0.0.1:54732
Wed Aug 24 22:18:08 2011 PID for socket: 1777
Wed Aug 24 22:18:08 2011 PID 1777: issending: 0
Wed Aug 24 22:18:08 2011 ===> PID 1777 is (most likely) contacting WU server, content length: 222562721
Wed Aug 24 22:18:08 2011 ===> Helper pid: -1
Wed Aug 24 22:18:08 2011 ===> PID 1777: numcores: 1
Wed Aug 24 22:18:08 2011 ===> now: 1314217088, last helper launched at: 1314217088
Wed Aug 24 22:18:08 2011 ===> WARNING: helper launched within last 2 minutes, delaying autosend
Wed Aug 24 22:18:08 2011 (0) Local: received 16384 bytes, sent 0 bytes
/proc/1777/cwd/langouste-helper.sh: launching asynchronous part, using /home/rurushu/fah/langouste-helper.sh
Wed Aug 24 22:19:36 2011 Accepted connection from: 127.0.0.1:54736
Wed Aug 24 22:19:36 2011 PID for socket: 1912
Wed Aug 24 22:19:36 2011 PID 1912: issending: 1
Wed Aug 24 22:19:36 2011 (0) resolving '130.237.232.237:8080'
Wed Aug 24 22:19:36 2011 (0) Connecting to: 130.237.232.237:8080
Wed Aug 24 22:19:36 2011 (0) Connected.
Thu Aug 25 00:19:37 2011 (0) ERROR: remote recv() failed: Connection reset by peer (104)
Thu Aug 25 00:19:37 2011 (0) Local: received 170000384 bytes, sent 0 bytes
Thu Aug 25 00:19:37 2011 (0) Remote: received 0 bytes, sent 169991424 bytes
Thu Aug 25 00:19:37 2011 (0) Ratelimit: sent 169991424 byte(s) in 7200.655 seconds, 23607 Bps (23.05 kBps)
Thu Aug 25 00:19:37 2011 Accepted connection from: 127.0.0.1:39005
Thu Aug 25 00:19:37 2011 PID for socket: 1912
Thu Aug 25 00:19:37 2011 PID 1912: issending: 1
Thu Aug 25 00:19:37 2011 (0) resolving '130.237.232.237:80'
Thu Aug 25 00:19:37 2011 (0) Connecting to: 130.237.232.237:80
Thu Aug 25 00:19:37 2011 (0) Connected.
Thu Aug 25 02:19:38 2011 (0) Remote connection closed (rbsize: 0).
Thu Aug 25 02:19:38 2011 (0) Local: received 183271424 bytes, sent 0 bytes
Thu Aug 25 02:19:38 2011 (0) Remote: received 0 bytes, sent 183266080 bytes
Thu Aug 25 02:19:38 2011 (0) Ratelimit: sent 183266080 byte(s) in 7201.322 seconds, 25448 Bps (24.85 kBps)
Thu Aug 25 02:19:40 2011 Accepted connection from: 127.0.0.1:52822
Thu Aug 25 02:19:40 2011 PID for socket: 1912
Thu Aug 25 02:19:40 2011 PID 1912: issending: 1
Thu Aug 25 02:19:40 2011 (0) resolving '130.237.165.141:8080'
Thu Aug 25 02:19:40 2011 (0) Connecting to: 130.237.165.141:8080
Thu Aug 25 02:19:40 2011 (0) Connected.
Thu Aug 25 02:19:40 2011 (0) Remote connection closed (rbsize: 0).
Thu Aug 25 02:19:40 2011 (0) Local: received 16384 bytes, sent 0 bytes
Thu Aug 25 02:19:40 2011 (0) Remote: received 0 bytes, sent 5632 bytes
Thu Aug 25 02:19:40 2011 (0) Ratelimit: sent 5632 byte(s) in 0.113 seconds, 49840 Bps (48.67 kBps)
Thu Aug 25 02:19:40 2011 Accepted connection from: 127.0.0.1:52824
Thu Aug 25 02:19:40 2011 PID for socket: 1912
Thu Aug 25 02:19:40 2011 PID 1912: issending: 1
Thu Aug 25 02:19:40 2011 (0) resolving '130.237.165.141:80'
Thu Aug 25 02:19:40 2011 (0) Connecting to: 130.237.165.141:80
Thu Aug 25 02:19:40 2011 (0) Connected.
Thu Aug 25 02:19:40 2011 (0) Remote connection closed (rbsize: 0).
Thu Aug 25 02:19:40 2011 (0) Local: received 16384 bytes, sent 0 bytes
Thu Aug 25 02:19:40 2011 (0) Remote: received 0 bytes, sent 5632 bytes
Thu Aug 25 02:19:40 2011 (0) Ratelimit: sent 5632 byte(s) in 0.112 seconds, 50285 Bps (49.10 kBps)

Any ideas?

[tear]

Hi rurushu,

Yes, 2 hours is a server-side time out. I've seen several reports of users hitting it too

I'll make sure the documentation covers this.

The only potential improvement I can see here is: we could add a threshold value to Langouste
so when upload size (which is known at the start) exceeds it, -r gets turned off for that particular
transfer.

On a bright side, IIRC P6903 and P6904 are of the same size -- duration of the simulation differs.

Let know of your thoughts.

Thanks,
tear

[rurushu]

Thanks for the quick reply!

I'm glad to hear that these projects are of the same size. I'll probably be able to upload them then, but it's still a bit too close for my taste (after removing -r the 6903 uploaded just a few seconds before the timeout would have occurred!)

Also, am I correct in assuming that the timeout would happen regardless of whether I am using langouste or not?

[tear]

That is correct.

[Fallen-Angel]

Sorry if the question is answered, but the thread is too big to read it all tonight...
(Windows user) After I installed the Langouste, must I let the started cmd window running or can I just close that? And if I should let the program running, must I start it again (just the way I used to start it now) when or if my computer restarts?
It seems to me I should let the cmd window running. Nevertheless, I just wanted to ask to be sure...

Best

FA

[tear]

On Windows, Langouste is a regular console application -- you need to keep the window open.

If there's enough interest I'll look into turning it into tray (but windowless) app...


Cheers,
tear

[Fallen-Angel]

Yeah it's a good idea, in which language did you program it?

[tear]

C.
Though now that I think of it, writing UI in C may not be the best idea (I use MingGW which
supports only Win32 API == no MFC or other conveniences).

Perhaps client-server model would be most appropriate with, say, .NET or wxWidgets client...

[Fallen-Angel]

Yeah I also thought about wxWidgets. I could create a GUI in Java or C++ (QT GUI) but then you must convert your source code from c in c++ or java.