lix/README

lix
===

lix is a source-based package manager written entirely in POSIX-compliant shell
script.  it was developed to serve as the core component of lix os.

it uses chroots and package-and-version-specific overlayfs layers to avoid
package conflicts during the build process.  aside from anything in the "system"
bootstrap directory, the only tools and libraries available to a package are
those it explicitly lists as a dependency.  all package files are kept in
separate directories and soft-linked in to the system root.  this makes it easy
to see what packages have supplied each file in one's system root.

lix is composed of the following small shell script utilities, all of which can
be used independently of lix and each other (with the exception of `how`, which
uses `vercmp` to allow defining instructions for ranges of package versions and
`shsort` to quicksort them):

1) `lmr` can merge one directory into another using soft or hard links.
2) `lyr` provides directories and commands for managing overlayfs overlays.
3) `src` pulls and verifies source code and provides default version numbers.
4) `how` provides scripts for patching, configuring, building, and installing.
5) `chin` sets up common system paths and chroots in to a target directory.
6) `vercmp` allows comparison of version strings in package-specific formats.
7) `shsort` is a POSIX shell quicksort implementation which takes a user
    supplied comparison command.

because of its small size and modularity, lix and its utilities can provide a
foundation for building a simple source-based distribution of one's own.  the
lix os project serves as an example of how one might do this.

## manual intervention

lix favors an interactive approach to problem solving.  patching, configuring,
building, and installing are all separate commands.  the results of commands
executed on a package up to any given point may be examined via:

- mounting the package's chroot contents using `lix mount <path> <package>`
- obtaining a shell into the package's chroot using `lix do sh <package>`
- examining the package's layers at `lyr upperdir lix/<package>/<version>`

## dependency handling

lix does not solve dependency graphs.  dependency graph resolution is a
non-trivial problem, and chrooted builds make solving it largely unnecessary
anyway.  each package can have whatever dependencies it needs without causing
conflicts with other packages.  if conflicts between packages arise, they can be
quickly resolved by swapping around softlinks using `lix up` and `lix down`.

`lix` records the versions of each dependency used when building a package in
the `built-with` directory in LIXROOT after each successful build.  this makes
it possible to determine which packages need to be rebuilt when changes are made
to a dependency.

lix also does not recursively build or install dependencies.  only code the user
has explicitly asked for should ever run on a user's machine.  however, assuming
package dependency lists are exhaustive and kept sorted by their height in the
(acyclic) dependency graph in a similar way to the linux kernel's module
dependencies list, implementing this behavior should require little more than a
loop and some list deduplication.

## dynamic dependencies

any line in a package's list of dependencies that matches the regex
"^\$[A-Za-z0-9]+$" will get evaluated before being parsed.  this allows some
dependencies to be provided via environment variables.  this functionality is
provided in order to support the user's ability to choose a text editor during
the configuration of certain packages.  by convention, lix packages use the
"EDITORDEP" variable to reference the package providing the command named in the
"EDITOR" variable.  because of its general nature, this technique can also be
extended to provide build flexibility as needed.

for example, the line:

    $EDITORDEP

in combination with the command:

    export EDITORDEP="elvis > 0"

results in the line:

    elvis > 0

which selects the most recent version of the elvis package, assuming its version
number is greater than zero.

## --bootstrap

every package chroot needs, at minimum, a userland of some kind.  (e.g., a shell
and utilities like `ls` and `cat`.)  this userland should be included in the
dependencies of the `how` packages one has chosen.  however, when bootstrapping
a lix system, no dependencies have been compiled yet and compilation must depend
on the host system's userland.  the `--bootstrap` option allows one to specify a
directory to use as the lowest layer in lix's overlayfs chroot.  this directory
must be constructed;  `--bootstrap=/` will NOT work because overlayfs does not
allow mounting overlays inside of lower directories.  `lmr` may come in handy
here.  `lix-os-utilities` also provides a script for constructing such a
directory, `mkbootstrap.sh`.

## the `build-conf` directory

when a package's build chroot is being constructed, versions for the target
package and each of its dependencies must be chosen somehow.  if a version for
the target package is not passed on the command line, the `build-conf` directory
is examined for a file with the same name as the target package.  this file
should contain a whitespace delimited list of package names and version numbers,
with the first line containing the name of the target package and the version to
build by default when no version is specified on the command line.

if there is no such file, the versions are taken from the `defaults.sh` file in
the `src` package roots for the target package and its dependencies.

each version specified is validated against the dependency constraints supplied
by `how`'s `deps` command.

if no version can be determined for a dependency, or if the dependency's layer
does not exist yet, a warning is emitted but the build attempts to continue.
this is because there is no guarantee that the dependency has not simply been
included via the `system` directory.  during a bootstrap build, this will be the
case more often than not.  in this situation, the dependency's name is still
written to a `built-with` file, but no version number is included.

## the `built-with` directory

every time a package is built successfully, a tab delimited list of the
dependencies included in its build chroot and their versions is written to the
`built-with` directory using the pattern `<package>-<version>` for the filename.

dependencies whose versions could not be determined (likely because they were
included via the `system` directory) or whose layers don't exist yet are listed
without a version number.

## layers

in the process of building and installing a package, the following `lyr` layers
are created:

1. lix/<package>/<version>/src
   used as the top layer for /mnt/src in the package chroot.  overlaid on top
   of the package's source code.  holds all changes that would have been made
   to the package's source code directory during the patching, configuring, and
   building process.

2. lix/<package>/<version>/build
   bind mounted to /mnt/build in the package chroot.  not a true layer as there
   is nothing to overlay.  gives packages a place to build out-of-source without
   having to mess up the package's root file system.

3. lix/<package>/<version>/how
   provided as the upper layer for the package's `how` instructions.  this layer
   gets read-only bind mounted to /mnt/how in the package chroot.  mostly exists
   to give `how` a place to mount its layers. (see lib/ch.sh for details.)

4. lix/<package>/<version>/fs
   contains the root filesystem for the package.  this is the "upper directory"
   in the overlayfs for the package's chroot.  gets used as a "lower directory"
   in the overlayfs mounts of packages that depend on it. gets `lmr`ed in to the
   system root directory by `lix up`.

## usage

lix <command> <package> [<version>]

commands:
    pl, pull
        acquire and patch package source code.

    cf, config, configure
        configure the package.

    mk, make
        build the package.

    in, inst, install
        install the package to its overlay.

    un, uninst, uninstall
        remove the package overlay.

    ad, add
        pull, configure, make, and install the package to its overlay.

    rm, remove
        delete the package version's overlay, source code, and signature.

    up
        symlink the package overlay into the system root.

    dn, down
        remove the package symlinks from the system root.

    do <cmd>
        chroot into the package filesystem overlay and run <cmd>.

    mt, mount <path>
        mounts the package filesystem overlay at the given path. 

    um, umount <path>
        unmounts the package filesystem overlay at the given path. 

    vr, ver, version
        print the version inferred for the given package.

    dp, deps, dependencies
        list package dependencies and their inferred versions.