# The Standard Environment

The standard build environment in the Nix Packages collection provides a environment for building Unix packages that does a lot of common build tasks automatically. In fact, for Unix packages that use the standard `./configure; make; make install` build interface, you don’t need to write a build script at all; the standard environment does everything automatically. If `stdenv` doesn’t do what you need automatically, you can easily customise or override the various build phases.

## Using `stdenv`

To build a package with the standard environment, you use the function `stdenv.mkDerivation`, instead of the primitive built-in function `derivation`, e.g.

    stdenv.mkDerivation {
      name = "libfoo-1.2.3";
      src = fetchurl {
        url = http://example.org/libfoo-1.2.3.tar.bz2;
        md5 = "e1ec107956b6ddcb0b8b0679367e9ac9";
      };
    }

(`stdenv` needs to be in scope, so if you write this in a separate Nix expression from `pkgs/all-packages.nix`, you need to pass it as a function argument.) Specifying a `name` and a `src` is the absolute minimum you need to do. Many packages have dependencies that are not provided in the standard environment. It’s usually sufficient to specify those dependencies in the `buildInputs` attribute:

    stdenv.mkDerivation {
      name = "libfoo-1.2.3";
      ...
      buildInputs = [libbar perl ncurses];
    }

This attribute ensures that the `bin` subdirectories of these packages appear in the `PATH` environment variable during the build, that their `include` subdirectories are searched by the C compiler, and so on. (See ssec-setup-hooks for details.)

Often it is necessary to override or modify some aspect of the build. To make this easier, the standard environment breaks the package build into a number of phases, all of which can be overriden or modified individually: unpacking the sources, applying patches, configuring, building, and installing. (There are some others; see ssec-stdenv-phases.) For instance, a package that doesn’t supply a makefile but instead has to be compiled “manually” could be handled like this:

    stdenv.mkDerivation {
      name = "fnord-4.5";
      ...
      buildPhase = ''
        gcc foo.c -o foo
      '';
      installPhase = ''
        mkdir -p $out/bin
        cp foo $out/bin
      '';
    }

(Note the use of `''`-style string literals, which are very convenient for large multi-line script fragments because they don’t need escaping of `"` and `\`, and because indentation is intelligently removed.)

There are many other attributes to customise the build. These are listed in ssec-stdenv-attributes.

While the standard environment provides a generic builder, you can still supply your own build script:

    stdenv.mkDerivation {
      name = "libfoo-1.2.3";
      ...
      builder = ./builder.sh;
    }

where the builder can do anything it wants, but typically starts with

    source $stdenv/setup

to let `stdenv` set up the environment (e.g., process the `buildInputs`). If you want, you can still use `stdenv`’s generic builder:

    source $stdenv/setup

    buildPhase() {
      echo "... this is my custom build phase ..."
      gcc foo.c -o foo
    }

    installPhase() {
      mkdir -p $out/bin
      cp foo $out/bin
    }

    genericBuild

## Tools provided by `stdenv`

The standard environment provides the following packages:

- The GNU C Compiler, configured with C and C++ support.

- GNU coreutils (contains a few dozen standard Unix commands).

- GNU findutils (contains `find`).

- GNU diffutils (contains `diff`, `cmp`).

- GNU `sed`.

- GNU `grep`.

- GNU `awk`.

- GNU `tar`.

- `gzip` and `bzip2`.

- GNU Make. It has been patched to provide nested output that can be fed into the `nix-log2xml` command and `log2html` stylesheet to create a structured, readable output of the build steps performed by Make.

- Bash. This is the shell used for all builders in the Nix Packages collection. Not using `/bin/sh` removes a large source of portability problems.

- The `patch` command.

On Linux, `stdenv` also includes the `patchelf` utility.

## Attributes

## Variables affecting `stdenv` initialisation

`NIX_DEBUG`

- If set, `stdenv` will print some debug information during the build. In particular, the `gcc` and `ld` wrapper scripts will print out the complete command line passed to the wrapped tools.

`buildInputs`

- A list of dependencies used by `stdenv` to set up the environment for the build. For each dependency `dir`, the directory ``dir`/bin`, if it exists, is added to the `PATH` environment variable. Other environment variables are also set up via a pluggable mechanism. For instance, if `buildInputs` contains Perl, then the `lib/site_perl` subdirectory of each input is added to the `PERL5LIB` environment variable. See ssec-setup-hooks for details.

`propagatedBuildInputs`

- Like `buildInputs`, but these dependencies are propagated: that is, the dependencies listed here are added to the `buildInputs` of any package that uses this package as a dependency. So if package Y has `propagatedBuildInputs = [X]`, and package Z has `buildInputs = [Y]`, then package X will appear in Z’s build environment automatically.

## Phases

The generic builder has a number of phases. Package builds are split into phases to make it easier to override specific parts of the build (e.g., unpacking the sources or installing the binaries). Furthermore, it allows a nicer presentation of build logs in the Nix build farm.

Each phase can be overriden in its entirety either by setting the environment variable ``name`Phase` to a string containing some shell commands to be executed, or by redefining the shell function ``name`Phase`. The former is convenient to override a phase from the derivation, while the latter is convenient from a build script.

### Controlling phases

There are a number of variables that control what phases are executed and in what order:

### Variables affecting phase control

`phases`

- Specifies the phases. You can change the order in which phases are executed, or add new phases, by setting this variable. If it’s not set, the default value is used, which is `$prePhases unpackPhase patchPhase $preConfigurePhases configurePhase $preBuildPhases buildPhase checkPhase $preInstallPhases installPhase fixupPhase $preDistPhases distPhase $postPhases`.

  Usually, if you just want to add a few phases, it’s more convenient to set one of the variables below (such as `preInstallPhases`), as you then don’t specify all the normal phases.

`prePhases`

- Additional phases executed before any of the default phases.

`preConfigurePhases`

- Additional phases executed just before the configure phase.

`preBuildPhases`

- Additional phases executed just before the build phase.

`preInstallPhases`

- Additional phases executed just before the install phase.

`preDistPhases`

- Additional phases executed just before the distribution phase.

`postPhases`

- Additional phases executed after any of the default phases.

### The unpack phase

The unpack phase is responsible for unpacking the source code of the package. The default implementation of `unpackPhase` unpacks the source files listed in the `src` environment variable to the current directory. It supports the following files by default:

Tar files

- These can optionally be compressed using `gzip` (`.tar.gz`, `.tgz` or `.tar.Z`) or `bzip2` (`.tar.bz2` or `.tbz2`).

Zip files

- Zip files are unpacked using `unzip`. However, `unzip` is not in the standard environment, so you should add it to `buildInputs` yourself.

Directories in the Nix store

- These are simply copied to the current directory. The hash part of the file name is stripped, e.g. `/nix/store/1wydxgby13cz...-my-sources` would be copied to `my-sources`.

Additional file types can be supported by setting the `unpackCmd` variable (see below).

### Variables controlling the unpack phase

`srcs` / `src`

- The list of source files or directories to be unpacked or copied. One of these must be set.

`sourceRoot`

- After running `unpackPhase`, the generic builder changes the current directory to the directory created by unpacking the sources. If there are multiple source directories, you should set `sourceRoot` to the name of the intended directory.

`setSourceRoot`

- Alternatively to setting `sourceRoot`, you can set `setSourceRoot` to a shell command to be evaluated by the unpack phase after the sources have been unpacked. This command must set `sourceRoot`.

`preUnpack`

- Hook executed at the start of the unpack phase.

`postUnpack`

- Hook executed at the end of the unpack phase.

`dontMakeSourcesWritable`

- If set to `1`, the unpacked sources are not made writable. By default, they are made writable to prevent problems with read-only sources. For example, copied store directories would be read-only without this.

`unpackCmd`

- The unpack phase evaluates the string `$unpackCmd` for any unrecognised file. The path to the current source file is contained in the `curSrc` variable.

### The patch phase

The patch phase applies the list of patches defined in the `patches` variable.

### Variables controlling the patch phase

`patches`

- The list of patches. They must be in the format accepted by the `patch` command, and may optionally be compressed using `gzip` (`.gz`) or `bzip2` (`.bz2`).

`patchFlags`

- Flags to be passed to `patch`. If not set, the argument `-p1` is used, which causes the leading directory component to be stripped from the file names in each patch.

`prePatch`

- Hook executed at the start of the patch phase.

`postPatch`

- Hook executed at the end of the patch phase.

### The configure phase

The configure phase prepares the source tree for building. The default `configurePhase` runs `./configure` (typically an Autoconf-generated script) if it exists.

### Variables controlling the configure phase

`configureScript`

- The name of the configure script. It defaults to `./configure` if it exists; otherwise, the configure phase is skipped. This can actually be a command (like `perl ./Configure.pl`).

`configureFlags`

- Additional arguments passed to the configure script.

`configureFlagsArray`

- A shell array containing additional arguments passed to the configure script. You must use this instead of `configureFlags` if the arguments contain spaces.

`dontAddPrefix`

- By default, the flag `--prefix=$prefix` is added to the configure flags. If this is undesirable, set this variable to a non-empty value.

`prefix`

- The prefix under which the package must be installed, passed via the `--prefix` option to the configure script. It defaults to `$out`.

`dontAddDisableDepTrack`

- By default, the flag `--disable-dependency-tracking` is added to the configure flags to speed up Automake-based builds. If this is undesirable, set this variable to a non-empty value.

`dontFixLibtool`

- By default, the configure phase applies some special hackery to all files called `ltmain.sh` before running the configure script in order to improve the purity of Libtool-based packages

  It clears the `sys_lib_`*`search_path` variables in the Libtool script to prevent Libtool from using libraries in `/usr/lib` and such.

  . If this is undesirable, set this variable to a non-empty value.

`preConfigure`

- Hook executed at the start of the configure phase.

`postConfigure`

- Hook executed at the end of the configure phase.

### The build phase

The build phase is responsible for actually building the package (e.g. compiling it). The default `buildPhase` simply calls `make` if a file named `Makefile`, `makefile` or `GNUmakefile` exists in the current directory (or the `makefile` is explicitly set); otherwise it does nothing.

### Variables controlling the build phase

`makefile`

- The file name of the Makefile.

`makeFlags`

- Additional flags passed to `make`. These flags are also used by the default install and check phase. For setting make flags specific to the build phase, use `buildFlags` (see below).

`makeFlagsArray`

- A shell array containing additional arguments passed to `make`. You must use this instead of `makeFlags` if the arguments contain spaces, e.g.

      makeFlagsArray=(CFLAGS="-O0 -g" LDFLAGS="-lfoo -lbar")

  Note that shell arrays cannot be passed through environment variables, so you cannot set `makeFlagsArray` in a derivation attribute (because those are passed through environment variables): you have to define them in shell code.

`buildFlags` / `buildFlagsArray`

- Additional flags passed to `make`. Like `makeFlags` and `makeFlagsArray`, but only used by the build phase.

`preBuild`

- Hook executed at the start of the build phase.

`postBuild`

- Hook executed at the end of the build phase.

You can set flags for `make` through the `makeFlags` variable.

Before and after running `make`, the hooks `preBuild` and `postBuild` are called, respectively.

### The check phase

The check phase checks whether the package was built correctly by running its test suite. The default `checkPhase` calls `make check`, but only if the `doCheck` variable is enabled.

### Variables controlling the check phase

`doCheck`

- If set to a non-empty string, the check phase is executed, otherwise it is skipped (default). Thus you should set

      doCheck = true;

  in the derivation to enable checks.

`makeFlags` / `makeFlagsArray` / `makefile`

- See the build phase for details.

`checkTarget`

- The make target that runs the tests. Defaults to `check`.

`checkFlags` / `checkFlagsArray`

- Additional flags passed to `make`. Like `makeFlags` and `makeFlagsArray`, but only used by the check phase.

`preCheck`

- Hook executed at the start of the check phase.

`postCheck`

- Hook executed at the end of the check phase.

### The install phase

The install phase is responsible for installing the package in the Nix store under `out`. The default `installPhase` creates the directory `$out` and calls `make install`.

### Variables controlling the check phase

`makeFlags` / `makeFlagsArray` / `makefile`

- See the build phase for details.

`installTargets`

- The make targets that perform the installation. Defaults to `install`. Example:

      installTargets = "install-bin install-doc";

`installFlags` / `installFlagsArray`

- Additional flags passed to `make`. Like `makeFlags` and `makeFlagsArray`, but only used by the install phase.

`preInstall`

- Hook executed at the start of the install phase.

`postInstall`

- Hook executed at the end of the install phase.

### The fixup phase

The fixup phase performs some (Nix-specific) post-processing actions on the files installed under `$out` by the install phase. The default `fixupPhase` does the following:

- It moves the `man/`, `doc/` and `info/` subdirectories of `$out` to `share/`.

- It strips libraries and executables of debug information.

- On Linux, it applies the `patchelf` command to ELF executables and libraries to remove unused directories from the `RPATH` in order to prevent unnecessary runtime dependencies.

- It rewrites the interpreter paths of shell scripts to paths found in `PATH`. E.g., `/usr/bin/perl` will be rewritten to `/nix/store/`some-perl`/bin/perl` found in `PATH`.

### Variables controlling the check phase

`dontStrip`

- If set, libraries and executables are not stripped. By default, they are.

`stripAllList`

- List of directories to search for libraries and executables from which all symbols should be stripped. By default, it’s empty. Stripping all symbols is risky, since it may remove not just debug symbols but also ELF information necessary for normal execution.

`stripAllFlags`

- Flags passed to the `strip` command applied to the files in the directories listed in `stripAllList`. Defaults to `-s` (i.e. `--strip-all`).

`stripDebugList`

- List of directories to search for libraries and executables from which only debugging-related symbols should be stripped. It defaults to `lib bin sbin`.

`stripDebugFlags`

- Flags passed to the `strip` command applied to the files in the directories listed in `stripDebugList`. Defaults to `-S` (i.e. `--strip-debug`).

`dontPatchELF`

- If set, the `patchelf` command is not used to remove unnecessary `RPATH` entries. Only applies to Linux.

`dontPatchShebangs`

- If set, scripts starting with `#!` do not have their interpreter paths rewritten to paths in the Nix store.

`forceShare`

- The list of directories that must be moved from `$out` to `$out/share`. Defaults to `man doc info`.

`setupHook`

- A package can export a setup hook by setting this variable. The setup hook, if defined, is copied to `$out/nix-support/setup-hook`. Environment variables are then substituted in it using `substituteAll`.

`preFixup`

- Hook executed at the start of the fixup phase.

`postFixup`

- Hook executed at the end of the fixup phase.

### The distribution phase

The distribution phase is intended to produce a source distribution of the package. The default `distPhase` first calls `make dist`, then it copies the resulting source tarballs to `$out/tarballs/`. This phase is only executed if the attribute `doDist` is set.

### Variables controlling the distribution phase

`distTarget`

- The make target that produces the distribution. Defaults to `dist`.

`distFlags` / `distFlagsArray`

- Additional flags passed to `make`.

`tarballs`

- The names of the source distribution files to be copied to `$out/tarballs/`. It can contain shell wildcards. The default is `*.tar.gz`.

`dontCopyDist`

- If set, no files are copied to `$out/tarballs/`.

`preDist`

- Hook executed at the start of the distribution phase.

`postDist`

- Hook executed at the end of the distribution phase.

## Shell functions

The standard environment provides a number of useful functions.

`substitute` `infile` `outfile` `subs`

- Performs string substitution on the contents of `infile`, writing the result to `outfile`. The substitutions in `subs` are of the following form:

  `--replace` `s1` `s2`

- Replace every occurence of the string `s1` by `s2`.

`--subst-var` `varName`

- Replace every occurence of `@`varName`@` by the contents of the environment variable `varName`. This is useful for generating files from templates, using `@`...`@` in the template as placeholders.

`--subst-var-by` `varName` `s`

- Replace every occurence of `@`varName`@` by the string `s`.

Example:

    substitute ./foo.in ./foo.out \
        --replace /usr/bin/bar $bar/bin/bar \
        --replace "a string containing spaces" "some other text" \
        --subst-var someVar

`substitute` is implemented using the `replace` command. Unlike with the `sed` command, you don’t have to worry about escaping special characters. It supports performing substitutions on binary files (such as executables), though there you’ll probably want to make sure that the replacement string is as long as the replaced string.

`substituteInPlace` `file` `subs`

- Like `substitute`, but performs the substitutions in place on the file `file`.

`substituteAll` `infile` `outfile`

- Replaces every occurence of `@`varName`@`, where `varName` is any environment variable, in `infile`, writing the result to `outfile`. For instance, if `infile` has the contents

      #! @bash@/bin/sh
      PATH=@coreutils@/bin
      echo @foo@

  and the environment contains `bash=/nix/store/bmwp0q28cf21...-bash-3.2-p39` and `coreutils=/nix/store/68afga4khv0w...-coreutils-6.12`, but does not contain the variable `foo`, then the output will be

      #! /nix/store/bmwp0q28cf21...-bash-3.2-p39/bin/sh
      PATH=/nix/store/68afga4khv0w...-coreutils-6.12/bin
      echo @foo@

  That is, no substitution is performed for undefined variables.

`substituteAllInPlace` `file`

- Like `substituteAll`, but performs the substitutions in place on the file `file`.

`stripHash` `path`

- Strips the directory and hash part of a store path, and prints (on standard output) only the name part. For instance, `stripHash /nix/store/68afga4khv0w...-coreutils-6.12` print `coreutils-6.12`.

## Package setup hooks

The following packages provide a setup hook:

GCC wrapper

- Adds the `include` subdirectory of each build input to the `NIX_CFLAGS_COMPILE` environment variable, and the `lib` and `lib64` subdirectories to `NIX_LDFLAGS`.

Perl

- Adds the `lib/site_perl` subdirectory of each build input to the `PERL5LIB` environment variable.

Python

- Adds the `lib/python2.5/site-packages` subdirectory of each build input to the `PYTHONPATH` environment variable.

  This should be generalised: the Python version shouldn’t be hard-coded.

pkg-config

- Adds the `lib/pkgconfig` and `share/pkgconfig` subdirectories of each build input to the `PKG_CONFIG_PATH` environment variable.

Automake

- Adds the `share/aclocal` subdirectory of each build input to the `ACLOCAL_PATH` environment variable.

libxml2

- Adds every file named `catalog.xml` found under the `xml/dtd` and `xml/xsl` subdirectories of each build input to the `XML_CATALOG_FILES` environment variable.

teTeX / TeX Live

- Adds the `share/texmf-nix` subdirectory of each build input to the `TEXINPUTS` environment variable.

Qt

- Sets the `QTDIR` environment variable to Qt’s path.

GHC

- Creates a temporary package database and registers every Haskell build input in it (TODO: how?).

## Purity in Nixpkgs

[measures taken to prevent dependencies on packages outside the store, and what you can do to prevent them]

GCC doesn't search in locations such as `/usr/include`. In fact, attempts to add such directories through the `-I` flag are filtered out. Likewise, the linker (from GNU binutils) doesn't search in standard locations such as `/usr/lib`. Programs built on Linux are linked against a GNU C Library that likewise doesn't search in the default system locations.