commit ee0f66248149ea136cb3ac832e2f661dc6c4fb54 Author: fox Date: Wed Nov 25 23:47:44 2020 +0000 initial commit diff --git a/LICENSE b/LICENSE new file mode 100644 index 0000000..bd8c239 --- /dev/null +++ b/LICENSE @@ -0,0 +1,20 @@ +The MIT License (MIT) + +Copyright 2020 "yafox" + +Permission is hereby granted, free of charge, to any person obtaining a copy of +this software and associated documentation files (the "Software"), to deal in +the Software without restriction, including without limitation the rights to +use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of +the Software, and to permit persons to whom the Software is furnished to do so, +subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS +FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR +COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER +IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN +CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. diff --git a/README.md b/README.md new file mode 100644 index 0000000..81692de --- /dev/null +++ b/README.md @@ -0,0 +1,304 @@ +lix-os +====== + +a linux distribution for people who like to breathe manually. + +this repository contains documentation and helper scripts only. a working linux +distribution with overlayfs support (preferably alpine linux) and an internet +connection are required to bootstrap lix os. no images or binaries supplied. + +## about + +lix os uses the lix package manager to construct a musl-based linux distribution +from source code. it is entirely composed of simple, short-as-possible, +POSIX-compliant shell scripts. packages are built in chroots constructed from +package directories stacked on top of each other using overlayfs. only +dependencies explicitly listed in a package's 'deps' file are accessible during +the package's build process. there is no concept of "indirect dependencies." +packages are made available to the user by soft-linking package directory +contents into the system root. packages can be added to or removed from the +system root without uninstalling the packages entirely. + +lix os is the work of a single maintainer and follows the same "use fewer +packages" ethos that kiss linux does. it also tries to use the smallest +packages possible in order to create a more readily audited codebase. some +exceptions are made in the name of usefulness and self-hosting. generally, +suckless utilities are preferred to bsd utilities, and bsd utilities are +preferred to gnu utilities due to the preferred in each case generally having a +smaller, easier to understand codebase. this does mean lix os's versions of +common linux utilities may have fewer options than one may be used to, but many +of these missing options are either not commonly used or can be compensated for +with a short shell script or function. + +in the opinion of lix os' maintainer, it would be ideal if everyone knew and +understood every line of code executing on their machines. while a perfect +realisation of this dream is unlikely, lix hopes to make the problem less +intractable. lix os will never ship binaries. + +## caveats + +lix os is an experimental linux distribution. it may not work. it may catch on +fire. it may be abandoned at a moment's notice for another experiment. it will +not stop you from doing something stupid and destroying your system. it is not +and probably never will be "production ready." + +nonetheless, the original author currently uses it daily, and it may serve the +needs of others as well. + +## goals + +1) be self-hosting. +2) keep a small codebase. +3) remain easy to understand, customize, and fork. +4) never offer more packages than one maintainer can keep up to date. +5) build from sources obtained directly from their authors, whenever possible. +6) support fully open source platforms. (e.g., Talos II.) + +## system hygiene + +lix helps lix os maintain system hygiene by: + +1) building and installing in chroots with only package dependencies available. +2) patching, configuring, and building in an overlay over package source code. +3) mounting an overlay over each chroot before entering the chroot at all. +4) symlinking installed files from package overlays into the filesystem. + +all changes during the patching, configuration, building, and installation +phases go into overlays maintained by `lyr`. this helps keep the source code in +`src`'s repository pristine and makes it very simple to activate and deactivate +packages. it also makes it easy to see what packages have supplied each file in +one's system root. + +## fork at will + +lix os is simple. if one disagrees with a decision, one should fork away. it +should never take more than a single person to maintain a lix os-based +distribution and all its packages. + +## _largely_ statically linked + +there are some blockers to achieving a fully statically linked system. + +### 1) smartcard support + +smartcard support is not possible at this time without dynamic linking. in +particular, 'scdaemon' from gnupg loads 'libpcsclite.so' at runtime. there is +not yet a way to make it statically link the library instead. accordingly, the +executables provided by the 'gnupg' package in lix os are linked statically and +do not support smartcards, while the executables in the 'gnupg-dynamic' and +'pcsc-lite' packages are linked dynamically and do support smartcards. + +### 2) gui desktops + +both x11 and wayland have dependencies which use meson as their build system. +meson requires core python3 modules which use dynamic loading (e.g., ctypes), +which means python3 must be dynamically linked in order to use the meson build +system. the default python3 package is statically linked and does not contain +many "core" modules due to their dependency on the 'dlopen' function. if a gui +is desired, the 'python3-dynamic' package will have to be built. + +removing the meson dependency is possible. oasis linux has pioneered this +approach. the best approach would probably be to either port the packaging or +the build system from oasis linux. preferably the latter, since doing so would +better enable collaboration between our projects, but in theory a lua script +could also be written to translate oasis linux packaging into shell scripts +for those unwilling to learn the lua-based build system. + +mesa and wayland also tend to not work well when statically linked. in theory +it should be possible to statically link these components, but in practice it +seems to result in segmentation faults. there is probably a way to fix this +problem, but in the meantime a graphical desktop will require dynamic linking. + +### 3) gettext + +the `kbd` package, which supplies console utilities like `setfont`, requires +`gettext`, and `gettext` will not build statically due to symbol collisions +during while linking. this is also probably fixable, but it would be better to +remove the `gettext` dependency entirely if possible. + +## why prefer static linking? + +because it should be possible to quantify all of a program's possible states at +compile time, barring hardware-level shenanigans, from source code. a user +should never be put in the position where they are surprised by a change in a +program's behavior because a dynamically loaded library was updated. this seems +analogous to lexical versus dynamic scoping in programming languages. + +there are ways to achieve these sorts of guarantees without sacrificing dynamic +loading, but they are more complicated than simply linking statically. + +also, no dynamic loader means no dynamic loader vulnerabilities. + +## bootstrapping + +below is the "official path," but please try other ways and report back. + +it is assumed that each step is executed from the same directory. e.g., `~`. + +if one prefers to use the linux kernel which contains propietary firmware blobs, +replace references to `linux-libre` below with `linux-kernel`. + +01. create an alpine linux installation. leave the target device untouched as + it will need to be formatted and partitioned. + +02. install the bootstrap dependencies: + `apk add build-base coreutils cmake git curl rsync cryptsetup autoconf + automake libtool flex bison ncurses-dev` + +03. install a command-line text editor and set the EDITOR variable. (optional) + e.g., `apk add vim && export EDITOR='vim'` + (n.b.: if this step is skipped, `vi` will be used.) + (n.b.: in a later step, libraries get moved. if this breaks the text + editor, try another one. if all else fails, `vi` is known to work in alpine + linux and comes installed by default.) + +04. clone the lix-os repository and its subrepositories, and install them. + - `git clone http://git.fuwafuwaqtlkkxwc.onion/yafox/lix-os` + - `cd lix-os` + - `make` + - `make install` + +05. create a bootstrap layer + - `mkdir bootstrap` + - `./lix-os/utilities/mkbootstrap.sh bootstrap` + +06. supply a linux kernel configuration file (strongly recommended but optional) + - `mkdir linux-mnt` + - `lix --bootstrap=bootstrap pull linux-libre` + - `lix --bootstrap=bootstrap mount linux-mnt linux-libre` + - copy the desired kernel configuration file to `linux-mnt/mnt/src/.config` + (n.b.: many distributions have their own kernel configuration files which + can be borrowed. lix os' maintainer keeps a public collection of examples + in the linux-confs repository.) + - `lix umount linux-mnt linux-libre` + - `rmdir linux-mnt` + +07. prepare the destination device. + - create an ext4 partition to be used as the boot partition. 100MB should + be plenty of space, assuming no firmware blobs are to be built into the + kernel. + - create a LUKS-encrypted ext4 partition to be used as the root partition. + - create whatever other partitions desired. + +08. build lix os + - `"$utilsdir/bootstrap.sh" bootstrap` + (n.b.: some packages will require interactive configuration via text + editor. if a package fails, bootstrapping can be resumed from the same + point later by passing the name of the package as a second argument. e.g., + `"$utilsdir/bootstrap.sh" bootstrap `. this script just + loops over the dependencies of the `lix-os` package, `add`s them, and then + `up`s them in the `lix-os` chroot. it's a short script. don't be afraid + to examine and modify it.) + +09. copy lix os to the target medium + - `mkdir -p lix-os-tgt/boot` + - mount the root partition to `lix-os-tgt` + - mount the boot partition to `lix-os-tgt/boot` + - mount any other partitions created earlier under `lix-os-tgt`. + - `"$utilsdir/install.sh" lix-os-tgt` + (n.b.: this script copies the contents of the lix-os chroot and the `src` + and `lyr` repos in /usr/var to the target directory given, then chroots + in to the target directory and replaces all the real files with softlinks + to the appropriate targets in the `lyr` repository via `lix up -f`.) + +10. make last-minute configuration changes to boot-related partitions + - edit the contents of `lix-os-tgt/boot/grub/grub.cfg` to point to the + correct partitions. + - make any additional last-minute configuration changes desired. + +at this point, if one wishes to skip using symlinks to link installed packages +into the system root instead of having them copied from the package chroots, one +may skip to the last step. + +11. unmount boot-related partitions + - `umount lix-os-tgt/boot` + - unmount any additional partitions which should not have their files + converted to symlinks to locations in lyr's layer repository. + (n.b.: this is one of the "sharp edges" of lix os which will one day be + "filed down" in one way or another. in the meantime, take care to not + `lix up` or `lix dn` anything with files related to booting. + `linux-libre`, `lix-os-bootloader`, and `lix-os-initramfs` immediately + come to mind as such packages.) + +12. convert everything else to symlinks + - `"$utilsdir/softlink.sh" lix-os-tgt` + +13. clean up and reboot into lix os + - unmount the target partitions under and at `lix-os-tgt`. + - `rmdir lix-os-tgt/boot lix-os-tgt` + - reboot into the target medium and make sure lix os works! + +## post-bootstrap + +if all has gone well and the system is booting, make a backup of your kernel +and initramfs. mount the boot partition and copy `vmlinux` to `vmlinux.bak` +and `initramfs.igz` to `initramfs.igz.bak`. this will allow for recovery by +changing boot parameters to boot with the backup files if one misconfigures +one's kernel or initramfs in the future. + +## editing lix-os-initramfs + +the contents of the initramfs file can be examined and modified using the +`lix mount` command, the same as every other lix package. + +## architecture + +lix os tries to be compositional in its architecture, which should make it +easier to replace components as desired. this flexibility comes at the +expense of having all files relevant to a package in a single location. an +earlier iteration of lix os (called 'mu os' in ignorance of the other linux +distribution by that same name) opted for a tighter integration of parts, which +was nice in some ways but which also tended toward creating a monolith. + +ideally the lix package manager should be composed of simple tools that behave +in a clear, consistent manner and which have no notions about each other. + +## helpful scripts + +there are also a number of helpful scripts in the `lix-os-utilities` repository. +for example, the `versions` utility, which itself also depends on `vercmp` and +`shsort`, may be useful for keeping a lix os system up to date. `versions` +returns a sorted list of available package versions pulled directly from each +project maintainer's website and/or its mirrors. lix does not use it directly, +but the author does use it periodically to update `src` packages. + +## roadmap + +- switch to a "-static-pie" build. (on ppc64le, this currently produces + "read-only segment has dynamic relocations" errors.) +- replace libressl with bearssl. +- replace all gnu tools with simpler alternatives. + [oasis linux](https://github.com/oasislinux/oasis) has the right idea here, + but work needs to be done to add ppc64le support to cproc and the like. +- capture file changes to "upped" packages to their own layer. +- capture file changes to /etc (and others?) to their own layer. +- allow unprivileged creation of user-specific lix environments, like nix os. +- find a way to encourage running processes under least-privileged, package- + specific user accounts whenever possible without literally making the choice + for the user. +- consider separating state in /var into files deterministically generated and + files constituting non-deterministic input. (e.g., 'src' tarballs versus + private keys.) +- consider moving to a model that does require a 'root' user. +- consider merging the overlayfs approach with the autotools-free, lua-as-make + approach of oasis linux. + +it would be nice to create something like nix os that captures changed state +rather than requiring state be constructed from a configuration file in a +domain-specific language. + +## inspiration + +the development of lix os was inspired and aided by the examples set by the +following projects: + +- gobo linux and nix os, for sandboxing and separation of state. +- sabotage linux, for general approach and code-centricity. +- kiss linux, for embracing a "remove as many packages as possible" ethos. +- alpine linux, for patches and scripts showing how to build things with musl. +- oasis linux, for helping inform userland choices. +- sta.li, for demonstrating a simplified LHFS. +- openbsd, for providing alternatives such as libressl and doas. +- musl-cross-make, for building a ppc64le-and-musl-targeting c compiler. +- suckless.org, for providing and highlighting simple alternatives, and for + setting me down this path in the first place. diff --git a/makefile b/makefile new file mode 100644 index 0000000..783a7c0 --- /dev/null +++ b/makefile @@ -0,0 +1,44 @@ +# there are many ways to lix-os. this is one of them. +# +BINDIR ?= /usr/bin +GITBASE ?= $(shell realpath $(shell git config --get remote.origin.url) | rev \ + | cut -d/ -f2- | rev) +REPOS := src vercmp shsort how chin lmr lyr lix +SUBREPOS := vercmp/format how/pkg +BINS := $(foreach repo,$(REPOS),$(BINDIR)/$(repo)) + +.PHONY: default install all-repos + +define git_clone + git clone $(GITBASE)/$1 $1 +endef + +default: all-repos .gitignore + +install: $(BINS) + +$(REPOS): + $(call git_clone,$@) + +all-repos: $(REPOS) $(SUBREPOS) utilities + +$(BINS): all-repos + cd $(shell basename $@) && make install + +utilities: + git clone $(GITBASE)/lix-os-utilities utilities + +vercmp/format: + cd vercmp && make lix-os-vercmp + +how/default: + cd how && make lix-os-defaults + +how/pkg: + cd how && make lix-os-pkgs + +src/pkg: + cd src && make lix-os-pkgs + +.gitignore: + $(shell echo "$(REPOS) utilities .gitignore" | tr ' ' "\\n" > .gitignore)