Creating bootable, headless Linux FAH LiveUSB

[demorgan]

I tried doing this, installing a minimal Debian install to a LiveUSB with OpenSSH installed as well, then making it log in automatically on boot through some configuration file changes, getting rid of the xserver and running CLI-only, et cetra. I also did it with Linux Lite,which is sort of similar to a very lightweight Xubuntu, and it runs well on old hardware, uses Xfce and is beginner-oriented. Everything else about it is similar to Ubuntu. In both cases, the install would boot off of the thumb drive only on the computer it was made on (it was tried on a wide variety of hardware both old and new, so it wasn't a matter of EFI vs legacy). In both cases, the boot flag was set.

My goal is to:

- Install a Debian-based distro on a LiveUSB which is stripped down to only what is needed to connect to a LAN, install (if necessary) and run FAHClient and be operated using a CLI terminal or SSH
- Make it so it runs on a wide variety of hardware 64-bit intel hardware, and can be cloned. Then I can run FAHClient off a LiveUSB on computers without having to use a keyboard or mouse beyond sometimes setting a BIOS setting to make USB the preferred boot device, if necessary.

I have a lot of computers that run FAHClient, and when I get new ones being able to just clone a LiveUSB image and burn it onto a thumbdrive and then booting to get an SSH-accessible computer running FAHClient going would be nice.

Thoughts: OpenSSH should be made to generate new keys every time it boots, avoiding the security issue of having the same key on multiple hosts (which isn't even allowed) Also, the program checking foldingathome.org for the latest version of FAHClient and installing it would be ideal, as it would make an ISO of the LiveUSB legal to distribute, I think? (Is it not the case that you can't distribute FAHClient legally, that it has to be downloaded from foldingathome.org?)

Any information pertinent to doing this would be greatly appreciated.

[Joe_H]

I don't have thoughts on most of the questions, but will point out one factor to keeping mind when creating these USB Linux boot drives. The first time the client FAHClient process makes connection with the folding servers, it is given an unique ID. So when creating these, they need to be cloned off an install that has not been connected to the internet. The other approach is to install after the first boot up using the device.

[bruce]

Two comments:

*The "unique ID" mentioned by Joe_H is called FAH's "userid" ... and it is easily confused with "user" which actually identifies you though not uniquely. Like he said, it gets assigned the first time your client is connected to a FAH server and must not be duplicated on another client's installation but ti should be reused on that installation.

* You want it to run "on a wide variety of hardware 64-bit intel hardware." It's not limited to Intel. It works fine on other 64-bit x86 hardware such as AMD. There's a lot of development work happening to develop a 64-bit ARM system that will run on the new Apple produces (and other ARM platforms) but it still needs some work.

[stevennic]

I did something like this last year. I kept the instructions as a Github gist. I'm a new user, but it can be found under my Github name stevennic22

I haven't touched or updated it recently because I was only running it on some laptops that I had laying around. I hope it can help.

[demorgan]

stevennic, thanks, I checked it out. It looks like your method uses Rufus to partition the flashdrive; I can't really use that at the moment because I don't have Windows, but I'll try to adapt that part, and the rest of your tutorial to my situation here. There's some great information there.

For a headless computer just running by itself without monitor, keyboard, or mouse it does look like its more practical to make a flashdrive that sets up OpenSSH or DropBear first, and then to install FAHClient and set up and configure it first through SSH. Maybe making a script that takes the flashdrive and allows you to add a WiFi SSID plus security info before booting on the headless computer would be doable, too. That's how I'm going to do it; a drive that's burned from an image with a ready FAHClient doesn't seem feasible, given how the userid works. Thanks for the information, all.

bruce, by "64-bit intel hardware" I meant "64-bit x86 hardware" in general; I always think of it as an "Intel architecture" because it all really goes back to the 8086 and 8088 which were Intel chips, but I realize other manufacturers have made chips with the same instruction sets and I didn't quite use the correct term.

[Joe_H]

For 64-bit architecture Intel ended up adopting the 64-bit extensions that AMD introduced, so for many it is useful somtimes to refer to AMD64. The actual Intel developed 64-bit instruction set was IA-64 for their commercial flop Itanium processors. That has been discontinued a few years ago.

[Burlingtonian]

stevennic, thanks, I checked it out. It looks like your method uses Rufus to partition the flashdrive; I can't really use that at the moment because I don't have Windows.

You could try balenaEtcher as an alternative to Rufus. It is free and runs on multiple OS's

[MeeLee]

I'd just go for Ubuntu 20.04 server,
It comes with an ISO that's ready to install.
Better run a full operating system, rather than a live image.
Every time the pc gets shut down, you'll lose some work (dump the WU) on a live CD/image.
If you really want to run the OS live (I wouldn't know what benefit it'd have) I believe Ubuntu 20.04 server does have a permanent drive section that can be installed on the pen drive.
That way all your configurations, WUs, drivers, etc.. can be saved (on an up to 4GB partition).
And you'll still run a live image.
Not sure if you'll need to download and reinstall stuff every time you run the system, but generally speaking, if you want to run the OS from the USB drive, just install it from an ISO ON the USB drive.
That way it keeps configuration settings, drivers and updates without losing time on downloading and installing them.

[stevennic]

The usage of Rufus was to easily toggle the option for persistent USB storage on the livecd. I had searched for 'Ubuntu Persistent Storage' originally. mkusb was another option if you're actively using Linux IIRC.

[zexmaxwell]

What I did was a sudo headless system. I got Ubuntu standalone and installed to the flash drive via rufus.
ran the flash while connected to via monitor and keyboard to install the OS, F@H(which was really fickle btw) and TeamViewer to run remotely.
replaced the plugged in monitor with a HDMI emulator port and left the keyboard plugged in.
it runs good, but really easy for the flash drive to corrupt when its powered off for maintenance. I much rather have a flash drive OS like how nice hash has done it.

[bruce]

repeated 'sync' should allow you to close all the files so there's no corruption as long as you can stop the processes. (e.g. FAH pause seems to work as long as power is not lost unexpectedly.)

[gunnarre]

I found the persistent storage to USB thumb drive to be too slow to be useful. But if you manage to use persistent USB drives, perhaps you can put the work unit files (the folders named 00, 01 etc inside "work") in a RAM drive so you won't wear out and slow down the USB flash drive too much, while leaving the other files on the persistent USB stick so you can keep the "userid" and config persistent. It looks like "work/client.db" and "work/client.db-journal" should be on persistent storage as well as the "cores" folder, GPUs.txt and config.xml.

[bruce]

FAH is designed to use a single directory for all of its data. While putting some files in RAM and some on persistent storage can be a good idea, there is no easy procedure to enable you to do that.

[gunnarre]

You could perhaps make a startup script which puts the "work" folder on a RAM drive, but uses symbolic links to keep the more persistent files in the persistent storage.

Don't expect the USB stick to be movable between computers though, unless the computers have identical hardware. The client.db and config.xml file gets populated with information about the specific hardware, and moving them to a new machine might have unexpected results.

[demorgan]

I messed around a little with this, and got something that set up an SSH server using a script in /etc/rc.local. I set it up with persistent storage, which works fine. It wasn't terribly slow, but not terribly fast, really. As I had it, it would only work with Ethernet as I had not bothered to make the tool that takes the flashdrive and injects WiFi SSIDs, passwords, and other data and has it auto-connect when you boot with it.

But, if I made a raw disk image of the flashdrive with all of the partitions on it, and distributed it, it would have been...illegal, since it has the Ubuntu branding and you're not allowed to redistribute modified ISOs under the Ubuntu name. Anyone know of an untrademarked, completely free, do-what-you-want-with-it Debian based distribution that you don't have to de-brand? (Then there's the issue of getting GPU folding automatically working, I don't know how in the world you could get it to work with GPU folding consistently across many computers; there are so many variables that could go wrong. I do suppose that once you have a shell, it wouldn't be such an issue to tinker with it remotely, but I'd like to make something that "just works" as much as possible.

As for the RAM disk idea, I just had a realization that I could probably just write some scripting in /etc/rc.local to take the directories that FAH uses, move the files into temporary directories, then mount the original directories as appropriately sized RAM drives, copy the files back, and the files would be used on the RAM drives, and have the RAM drives synced to persistent storage. That would give the fast access of RAM drives, with a slower back-up trailing along slightly behind it to backup to USB. So it works on RAM, but when you boot again it loads the persistent storage backup.