diff options
author | Jan Tojnar <jtojnar@gmail.com> | 2021-06-05 21:22:45 +0200 |
---|---|---|
committer | Jan Tojnar <jtojnar@gmail.com> | 2021-06-07 06:34:59 +0200 |
commit | 6ecc641d087df8b8896ebd876bba592e981877c9 (patch) | |
tree | c3dc98e6b6a4a6474b21ad06e7bbc96d0e8abaf5 /doc/stdenv | |
parent | ce6b1a4f8f9cbbb7fd2d1c637bcc39fff1ef49df (diff) |
doc: prepare for commonmark
We are still using Pandoc’s Markdown parser, which differs from CommonMark spec slightly. Notably: - Line breaks in lists behave differently. - Admonitions do not support the simpler syntax https://github.com/jgm/commonmark-hs/issues/75 - The auto_identifiers uses a different algorithm – I made the previous ones explicit. - Languages (classes) of code blocks cannot contain whitespace so we have to use “pycon” alias instead of Python “console” as GitHub’s linguist While at it, I also fixed the following issues: - ShellSesssion was used - Removed some pointless docbook tags.
Diffstat (limited to 'doc/stdenv')
-rw-r--r-- | doc/stdenv/cross-compilation.chapter.md | 31 | ||||
-rw-r--r-- | doc/stdenv/meta.chapter.md | 10 | ||||
-rw-r--r-- | doc/stdenv/multiple-output.chapter.md | 22 | ||||
-rw-r--r-- | doc/stdenv/stdenv.chapter.md | 132 |
4 files changed, 100 insertions, 95 deletions
diff --git a/doc/stdenv/cross-compilation.chapter.md b/doc/stdenv/cross-compilation.chapter.md index 3755c13facf5c..b27e9cc6fea1c 100644 --- a/doc/stdenv/cross-compilation.chapter.md +++ b/doc/stdenv/cross-compilation.chapter.md @@ -6,7 +6,6 @@ This chapter will be organized in three parts. First, it will describe the basics of how to package software in a way that supports cross-compilation. Second, it will describe how to use Nixpkgs when cross-compiling. Third, it will describe the internal infrastructure supporting cross-compilation. - ## Packaging in a cross-friendly manner {#sec-cross-packaging} ### Platform parameters {#ssec-cross-platform-parameters} @@ -65,7 +64,7 @@ The exact schema these fields follow is a bit ill-defined due to a long and conv ### Theory of dependency categorization {#ssec-cross-dependency-categorization} -::: note +::: {.note} This is a rather philosophical description that isn't very Nixpkgs-specific. For an overview of all the relevant attributes given to `mkDerivation`, see <xref linkend="ssec-stdenv-dependencies"/>. For a description of how everything is implemented, see <xref linkend="ssec-cross-dependency-implementation"/>. ::: @@ -81,10 +80,10 @@ Finally, if the depending package is a compiler or other machine-code-producing Putting this all together, that means we have dependencies in the form "host → target", in at most the following six combinations: +#### Possible dependency types {#possible-dependency-types} -#### Possible dependency types -| Dependency's host platform | Dependency's target platform | -| -- | -- | +| Dependency’s host platform | Dependency’s target platform | +|----------------------------|------------------------------| | build | build | | build | host | | build | target | @@ -113,15 +112,18 @@ On less powerful machines, it can be inconvenient to cross-compile a package onl $ nix-build '<nixpkgs>' -A pkgsCross.raspberryPi.hello ``` -#### What if my package's build system needs to build a C program to be run under the build environment? {#cross-qa-build-c-program-in-build-environment} +#### What if my package’s build system needs to build a C program to be run under the build environment? {#cross-qa-build-c-program-in-build-environment} + Add the following to your `mkDerivation` invocation. + ```nix depsBuildBuild = [ buildPackages.stdenv.cc ]; ``` -#### My package's testsuite needs to run host platform code. {#cross-testsuite-runs-host-code} +#### My package’s testsuite needs to run host platform code. {#cross-testsuite-runs-host-code} Add the following to your `mkDerivation` invocation. + ```nix doCheck = stdenv.hostPlatform == stdenv.buildPlatform; ``` @@ -134,7 +136,7 @@ Nixpkgs can be instantiated with `localSystem` alone, in which case there is no $ nix-build '<nixpkgs>' --arg crossSystem '(import <nixpkgs/lib>).systems.examples.fooBarBaz' -A whatever ``` -::: note +::: {.note} Eventually we would like to make these platform examples an unnecessary convenience so that ```ShellSession @@ -146,7 +148,7 @@ works in the vast majority of cases. The problem today is dependencies on other While one is free to pass both parameters in full, there's a lot of logic to fill in missing fields. As discussed in the previous section, only one of `system`, `config`, and `parsed` is needed to infer the other two. Additionally, `libc` will be inferred from `parse`. Finally, `localSystem.system` is also _impurely_ inferred based on the platform evaluation occurs. This means it is often not necessary to pass `localSystem` at all, as in the command-line example in the previous paragraph. -::: note +::: {.note} Many sources (manual, wiki, etc) probably mention passing `system`, `platform`, along with the optional `crossSystem` to Nixpkgs: `import <nixpkgs> { system = ..; platform = ..; crossSystem = ..; }`. Passing those two instead of `localSystem` is still supported for compatibility, but is discouraged. Indeed, much of the inference we do for these parameters is motivated by compatibility as much as convenience. ::: @@ -178,7 +180,7 @@ While there are many package sets, and thus many edges, the stages can also be a In each stage, `pkgsBuildHost` refers to the previous stage, `pkgsBuildBuild` refers to the one before that, and `pkgsHostTarget` refers to the current one, and `pkgsTargetTarget` refers to the next one. When there is no previous or next stage, they instead refer to the current stage. Note how all the invariants regarding the mapping between dependency and depending packages' build host and target platforms are preserved. `pkgsBuildTarget` and `pkgsHostHost` are more complex in that the stage fitting the requirements isn't always a fixed chain of "prevs" and "nexts" away (modulo the "saturating" self-references at the ends). We just special case each instead. All the primary edges are implemented is in `pkgs/stdenv/booter.nix`, and secondarily aliases in `pkgs/top-level/stage.nix`. -::: note +::: {.note} The native stages are bootstrapped in legacy ways that predate the current cross implementation. This is why the bootstrapping stages leading up to the final stages are ignored in the previous paragraph. ::: @@ -186,6 +188,7 @@ If one looks at the 3 platform triples, one can see that they overlap such that ``` (native, native, native, foreign, foreign) ``` + If one imagines the saturating self references at the end being replaced with infinite stages, and then overlays those platform triples, one ends up with the infinite tuple: ``` (native..., native, native, native, foreign, foreign, foreign...) @@ -193,8 +196,8 @@ If one imagines the saturating self references at the end being replaced with in One can then imagine any sequence of platforms such that there are bootstrap stages with their 3 platforms determined by "sliding a window" that is the 3 tuple through the sequence. This was the original model for bootstrapping. Without a target platform (assume a better world where all compilers are multi-target and all standard libraries are built in their own derivation), this is sufficient. Conversely if one wishes to cross compile "faster", with a "Canadian Cross" bootstrapping stage where `build != host != target`, more bootstrapping stages are needed since no sliding window provides the pesky `pkgsBuildTarget` package set since it skips the Canadian cross stage's "host". -::: note -It is much better to refer to `buildPackages` than `targetPackages`, or more broadly package sets that do not mention "target". There are three reasons for this. +::: {.note} +It is much better to refer to `buildPackages` than `targetPackages`, or more broadly package sets that do not mention “target”. There are three reasons for this. First, it is because bootstrapping stages do not have a unique `targetPackages`. For example a `(x86-linux, x86-linux, arm-linux)` and `(x86-linux, x86-linux, x86-windows)` package set both have a `(x86-linux, x86-linux, x86-linux)` package set. Because there is no canonical `targetPackages` for such a native (`build == host == target`) package set, we set their `targetPackages` @@ -203,6 +206,6 @@ Second, it is because this is a frequent source of hard-to-follow "infinite recu Thirdly, it is because everything target-mentioning only exists to accommodate compilers with lousy build systems that insist on the compiler itself and standard library being built together. Of course that is bad because bigger derivations means longer rebuilds. It is also problematic because it tends to make the standard libraries less like other libraries than they could be, complicating code and build systems alike. Because of the other problems, and because of these innate disadvantages, compilers ought to be packaged another way where possible. ::: -::: note -If one explores Nixpkgs, they will see derivations with names like `gccCross`. Such `*Cross` derivations is a holdover from before we properly distinguished between the host and target platforms—the derivation with "Cross" in the name covered the `build = host != target` case, while the other covered the `host = target`, with build platform the same or not based on whether one was using its `.nativeDrv` or `.crossDrv`. This ugliness will disappear soon. +::: {.note} +If one explores Nixpkgs, they will see derivations with names like `gccCross`. Such `*Cross` derivations is a holdover from before we properly distinguished between the host and target platforms—the derivation with “Cross” in the name covered the `build = host != target` case, while the other covered the `host = target`, with build platform the same or not based on whether one was using its `.nativeDrv` or `.crossDrv`. This ugliness will disappear soon. ::: diff --git a/doc/stdenv/meta.chapter.md b/doc/stdenv/meta.chapter.md index dd9f53258555e..f226a725480c4 100644 --- a/doc/stdenv/meta.chapter.md +++ b/doc/stdenv/meta.chapter.md @@ -130,7 +130,7 @@ Attribute Set `lib.platforms` defines [various common lists](https://github.com/ ### `tests` {#var-meta-tests} -::: warning +::: {.warning} This attribute is special in that it is not actually under the `meta` attribute set but rather under the `passthru` attribute set. This is due to how `meta` attributes work, and the fact that they are supposed to contain only metadata, not derivations. ::: @@ -175,20 +175,20 @@ The `meta.license` attribute should preferably contain a value from `lib.license Although it’s typically better to indicate the specific license, a few generic options are available: -### `lib.licenses.free`, `"free"` +### `lib.licenses.free`, `"free"` {#lib.licenses.free-free} Catch-all for free software licenses not listed above. -### `lib.licenses.unfreeRedistributable`, `"unfree-redistributable"` +### `lib.licenses.unfreeRedistributable`, `"unfree-redistributable"` {#lib.licenses.unfreeredistributable-unfree-redistributable} Unfree package that can be redistributed in binary form. That is, it’s legal to redistribute the *output* of the derivation. This means that the package can be included in the Nixpkgs channel. Sometimes proprietary software can only be redistributed unmodified. Make sure the builder doesn’t actually modify the original binaries; otherwise we’re breaking the license. For instance, the NVIDIA X11 drivers can be redistributed unmodified, but our builder applies `patchelf` to make them work. Thus, its license is `"unfree"` and it cannot be included in the Nixpkgs channel. -### `lib.licenses.unfree`, `"unfree"` +### `lib.licenses.unfree`, `"unfree"` {#lib.licenses.unfree-unfree} Unfree package that cannot be redistributed. You can build it yourself, but you cannot redistribute the output of the derivation. Thus it cannot be included in the Nixpkgs channel. -### `lib.licenses.unfreeRedistributableFirmware`, `"unfree-redistributable-firmware"` +### `lib.licenses.unfreeRedistributableFirmware`, `"unfree-redistributable-firmware"` {#lib.licenses.unfreeredistributablefirmware-unfree-redistributable-firmware} This package supplies unfree, redistributable firmware. This is a separate value from `unfree-redistributable` because not everybody cares whether firmware is free. diff --git a/doc/stdenv/multiple-output.chapter.md b/doc/stdenv/multiple-output.chapter.md index 90bc25bef7397..b8414614279db 100644 --- a/doc/stdenv/multiple-output.chapter.md +++ b/doc/stdenv/multiple-output.chapter.md @@ -6,7 +6,7 @@ The Nix language allows a derivation to produce multiple outputs, which is simil The main motivation is to save disk space by reducing runtime closure sizes; consequently also sizes of substituted binaries get reduced. Splitting can be used to have more granular runtime dependencies, for example the typical reduction is to split away development-only files, as those are typically not needed during runtime. As a result, closure sizes of many packages can get reduced to a half or even much less. -::: note +::: {.note} The reduction effects could be instead achieved by building the parts in completely separate derivations. That would often additionally reduce build-time closures, but it tends to be much harder to write such derivations, as build systems typically assume all parts are being built at once. This compromise approach of single source package producing multiple binary packages is also utilized often by rpm and deb. ::: @@ -28,7 +28,7 @@ NixOS provides two ways to select the outputs to install for packages listed in `nix-env` lacks an easy way to select the outputs to install. When installing a package, `nix-env` always installs the outputs listed in `meta.outputsToInstall`, even when the user explicitly selects an output. -::: warning +::: {.warning} `nix-env` silenty disregards the outputs selected by the user, and instead installs the outputs from `meta.outputsToInstall`. For example, ```ShellSession @@ -69,7 +69,7 @@ outputs = [ "bin" "dev" "out" "doc" ]; Often such a single line is enough. For each output an equally named environment variable is passed to the builder and contains the path in nix store for that output. Typically you also want to have the main `out` output, as it catches any files that didn’t get elsewhere. -::: note +::: {.note} There is a special handling of the `debug` output, described at <xref linkend="stdenv-separateDebugInfo" />. ::: @@ -85,35 +85,35 @@ The reason for why `glibc` deviates from the convention is because referencing a The support code currently recognizes some particular kinds of outputs and either instructs the build system of the package to put files into their desired outputs or it moves the files during the fixup phase. Each group of file types has an `outputFoo` variable specifying the output name where they should go. If that variable isn’t defined by the derivation writer, it is guessed – a default output name is defined, falling back to other possibilities if the output isn’t defined. -#### ` $outputDev` +#### `$outputDev` {#outputdev} is for development-only files. These include C(++) headers (`include/`), pkg-config (`lib/pkgconfig/`), cmake (`lib/cmake/`) and aclocal files (`share/aclocal/`). They go to `dev` or `out` by default. -#### ` $outputBin` +#### `$outputBin` {#outputbin} is meant for user-facing binaries, typically residing in `bin/`. They go to `bin` or `out` by default. -#### ` $outputLib` +#### `$outputLib` {#outputlib} is meant for libraries, typically residing in `lib/` and `libexec/`. They go to `lib` or `out` by default. -#### ` $outputDoc` +#### `$outputDoc` {#outputdoc} is for user documentation, typically residing in `share/doc/`. It goes to `doc` or `out` by default. -#### ` $outputDevdoc` +#### `$outputDevdoc` {#outputdevdoc} is for _developer_ documentation. Currently we count gtk-doc and devhelp books, typically residing in `share/gtk-doc/` and `share/devhelp/`, in there. It goes to `devdoc` or is removed (!) by default. This is because e.g. gtk-doc tends to be rather large and completely unused by nixpkgs users. -#### ` $outputMan` +#### `$outputMan` {#outputman} is for man pages (except for section 3), typically residing in `share/man/man[0-9]/`. They go to `man` or `$outputBin` by default. -#### ` $outputDevman` +#### `$outputDevman` {#outputdevman} is for section 3 man pages, typically residing in `share/man/man[0-9]/`. They go to `devman` or `$outputMan` by default. -#### ` $outputInfo` +#### `$outputInfo` {#outputinfo} is for info pages, typically residing in `share/info/`. They go to `info` or `$outputBin` by default. diff --git a/doc/stdenv/stdenv.chapter.md b/doc/stdenv/stdenv.chapter.md index b23c50e836445..71eb991bbae50 100644 --- a/doc/stdenv/stdenv.chapter.md +++ b/doc/stdenv/stdenv.chapter.md @@ -175,7 +175,8 @@ Because of the bounds checks, the uncommon cases are `h = t` and `h + 2 = t`. In Overall, the unifying theme here is that propagation shouldn’t be introducing transitive dependencies involving platforms the depending package is unaware of. \[One can imagine the dependending package asking for dependencies with the platforms it knows about; other platforms it doesn’t know how to ask for. The platform description in that scenario is a kind of unforagable capability.\] The offset bounds checking and definition of `mapOffset` together ensure that this is the case. Discovering a new offset is discovering a new platform, and since those platforms weren’t in the derivation “spec” of the needing package, they cannot be relevant. From a capability perspective, we can imagine that the host and target platforms of a package are the capabilities a package requires, and the depending package must provide the capability to the dependency. -### Variables specifying dependencies +### Variables specifying dependencies {#variables-specifying-dependencies} + #### `depsBuildBuild` {#var-stdenv-depsBuildBuild} A list of dependencies whose host and target platforms are the new derivation’s build platform. This means a `-1` host and `-1` target offset from the new derivation’s platforms. These are programs and libraries used at build time that produce programs and libraries also used at build time. If the dependency doesn’t care about the target platform (i.e. isn’t a compiler or similar tool), put it in `nativeBuildInputs` instead. The most common use of this `buildPackages.stdenv.cc`, the default C compiler for this role. That example crops up more than one might think in old commonly used C libraries. @@ -236,13 +237,13 @@ The propagated equivalent of `depsTargetTarget`. This is prefixed for the same r ## Attributes {#ssec-stdenv-attributes} -### Variables affecting `stdenv` initialisation +### Variables affecting `stdenv` initialisation {#variables-affecting-stdenv-initialisation} #### `NIX_DEBUG` {#var-stdenv-NIX_DEBUG} A natural number indicating how much information to log. If set to 1 or higher, `stdenv` will print moderate debugging information during the build. In particular, the `gcc` and `ld` wrapper scripts will print out the complete command line passed to the wrapped tools. If set to 6 or higher, the `stdenv` setup script will be run with `set -x` tracing. If set to 7 or higher, the `gcc` and `ld` wrapper scripts will also be run with `set -x` tracing. -### Attributes affecting build properties +### Attributes affecting build properties {#attributes-affecting-build-properties} #### `enableParallelBuilding` {#var-stdenv-enableParallelBuilding} @@ -250,7 +251,7 @@ If set to `true`, `stdenv` will pass specific flags to `make` and other build to Unless set to `false`, some build systems with good support for parallel building including `cmake`, `meson`, and `qmake` will set it to `true`. -### Special variables +### Special variables {#special-variables} #### `passthru` {#var-stdenv-passthru} @@ -298,7 +299,7 @@ passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ] The script will be run with `UPDATE_NIX_ATTR_PATH` environment variable set to the attribute path it is supposed to update. -::: note +::: {.note} The script will be usually run from the root of the Nixpkgs repository but you should not rely on that. Also note that the update scripts will be run in parallel by default; you should avoid running `git commit` or any other commands that cannot handle that. ::: @@ -314,7 +315,7 @@ Each phase can be overridden in its entirety either by setting the environment v There are a number of variables that control what phases are executed and in what order: -#### Variables affecting phase control +#### Variables affecting phase control {#variables-affecting-phase-control} ##### `phases` {#var-stdenv-phases} @@ -354,21 +355,22 @@ Additional phases executed after any of the default phases. The unpack phase is responsible for unpacking the source code of the package. The default implementation of `unpackPhase` unpacks the source files listed in the `src` environment variable to the current directory. It supports the following files by default: -#### Tar files +#### Tar files {#tar-files} These can optionally be compressed using `gzip` (`.tar.gz`, `.tgz` or `.tar.Z`), `bzip2` (`.tar.bz2`, `.tbz2` or `.tbz`) or `xz` (`.tar.xz`, `.tar.lzma` or `.txz`). -#### Zip files +#### Zip files {#zip-files} Zip files are unpacked using `unzip`. However, `unzip` is not in the standard environment, so you should add it to `nativeBuildInputs` yourself. -#### Directories in the Nix store +#### Directories in the Nix store {#directories-in-the-nix-store} These are simply copied to the current directory. The hash part of the file name is stripped, e.g. `/nix/store/1wydxgby13cz...-my-sources` would be copied to `my-sources`. Additional file types can be supported by setting the `unpackCmd` variable (see below). -#### Variables controlling the unpack phase +#### Variables controlling the unpack phase {#variables-controlling-the-unpack-phase} + ##### `srcs` / `src` {#var-stdenv-src} The list of source files or directories to be unpacked or copied. One of these must be set. @@ -405,7 +407,7 @@ The unpack phase evaluates the string `$unpackCmd` for any unrecognised file. Th The patch phase applies the list of patches defined in the `patches` variable. -#### Variables controlling the patch phase +#### Variables controlling the patch phase {#variables-controlling-the-patch-phase} ##### `dontPatch` {#var-stdenv-dontPatch} @@ -431,7 +433,7 @@ Hook executed at the end of the patch phase. The configure phase prepares the source tree for building. The default `configurePhase` runs `./configure` (typically an Autoconf-generated script) if it exists. -#### Variables controlling the configure phase +#### Variables controlling the configure phase {#variables-controlling-the-configure-phase} ##### `configureScript` {#var-stdenv-configureScript} @@ -491,7 +493,7 @@ Hook executed at the end of the configure phase. The build phase is responsible for actually building the package (e.g. compiling it). The default `buildPhase` simply calls `make` if a file named `Makefile`, `makefile` or `GNUmakefile` exists in the current directory (or the `makefile` is explicitly set); otherwise it does nothing. -#### Variables controlling the build phase +#### Variables controlling the build phase {#variables-controlling-the-build-phase} ##### `dontBuild` {#var-stdenv-dontBuild} @@ -509,7 +511,7 @@ A list of strings passed as additional flags to `make`. These flags are also use makeFlags = [ "PREFIX=$(out)" ]; ``` -::: note +::: {.note} The flags are quoted in bash, but environment variables can be specified by using the make syntax. ::: @@ -545,7 +547,7 @@ Before and after running `make`, the hooks `preBuild` and `postBuild` are called The check phase checks whether the package was built correctly by running its test suite. The default `checkPhase` calls `make check`, but only if the `doCheck` variable is enabled. -#### Variables controlling the check phase +#### Variables controlling the check phase {#variables-controlling-the-check-phase} ##### `doCheck` {#var-stdenv-doCheck} @@ -557,7 +559,7 @@ doCheck = true; in the derivation to enable checks. The exception is cross compilation. Cross compiled builds never run tests, no matter how `doCheck` is set, as the newly-built program won’t run on the platform used to build it. -##### `makeFlags` / `makeFlagsArray` / `makefile` +##### `makeFlags` / `makeFlagsArray` / `makefile` {#makeflags-makeflagsarray-makefile} See the [build phase](#var-stdenv-makeFlags) for details. @@ -585,13 +587,13 @@ Hook executed at the end of the check phase. The install phase is responsible for installing the package in the Nix store under `out`. The default `installPhase` creates the directory `$out` and calls `make install`. -#### Variables controlling the install phase +#### Variables controlling the install phase {#variables-controlling-the-install-phase} ##### `dontInstall` {#var-stdenv-dontInstall} Set to true to skip the install phase. -##### `makeFlags` / `makeFlagsArray` / `makefile` +##### `makeFlags` / `makeFlagsArray` / `makefile` {#makeflags-makeflagsarray-makefile-1} See the [build phase](#var-stdenv-makeFlags) for details. @@ -624,7 +626,7 @@ The fixup phase performs some (Nix-specific) post-processing actions on the file - On Linux, it applies the `patchelf` command to ELF executables and libraries to remove unused directories from the `RPATH` in order to prevent unnecessary runtime dependencies. - It rewrites the interpreter paths of shell scripts to paths found in `PATH`. E.g., `/usr/bin/perl` will be rewritten to `/nix/store/some-perl/bin/perl` found in `PATH`. -#### Variables controlling the fixup phase +#### Variables controlling the fixup phase {#variables-controlling-the-fixup-phase} ##### `dontFixup` {#var-stdenv-dontFixup} @@ -706,7 +708,7 @@ to `~/.gdbinit`. GDB will then be able to find debug information installed via ` The installCheck phase checks whether the package was installed correctly by running its test suite against the installed directories. The default `installCheck` calls `make installcheck`. -#### Variables controlling the installCheck phase +#### Variables controlling the installCheck phase {#variables-controlling-the-installcheck-phase} ##### `doInstallCheck` {#var-stdenv-doInstallCheck} @@ -742,7 +744,7 @@ Hook executed at the end of the installCheck phase. The distribution phase is intended to produce a source distribution of the package. The default `distPhase` first calls `make dist`, then it copies the resulting source tarballs to `$out/tarballs/`. This phase is only executed if the attribute `doDist` is set. -#### Variables controlling the distribution phase +#### Variables controlling the distribution phase {#variables-controlling-the-distribution-phase} ##### `distTarget` {#var-stdenv-distTarget} @@ -879,7 +881,7 @@ The most typical use of the setup hook is actually to add other hooks which are Packages adding a hook should not hard code a specific hook, but rather choose a variable *relative* to how they are included. Returning to the C compiler wrapper example, if the wrapper itself is an `n` dependency, then it only wants to accumulate flags from `n + 1` dependencies, as only those ones match the compiler’s target platform. The `hostOffset` variable is defined with the current dependency’s host offset `targetOffset` with its target offset, before its setup hook is sourced. Additionally, since most environment hooks don’t care about the target platform, that means the setup hook can append to the right bash array by doing something like -```{.bash} +```bash addEnvHooks "$hostOffset" myBashFunction ``` @@ -887,47 +889,47 @@ The *existence* of setups hooks has long been documented and packages inside Nix First, let’s cover some setup hooks that are part of Nixpkgs default stdenv. This means that they are run for every package built using `stdenv.mkDerivation`. Some of these are platform specific, so they may run on Linux but not Darwin or vice-versa. -### `move-docs.sh` +### `move-docs.sh` {#move-docs.sh} This setup hook moves any installed documentation to the `/share` subdirectory directory. This includes the man, doc and info directories. This is needed for legacy programs that do not know how to use the `share` subdirectory. -### `compress-man-pages.sh` +### `compress-man-pages.sh` {#compress-man-pages.sh} This setup hook compresses any man pages that have been installed. The compression is done using the gzip program. This helps to reduce the installed size of packages. -### `strip.sh` +### `strip.sh` {#strip.sh} This runs the strip command on installed binaries and libraries. This removes unnecessary information like debug symbols when they are not needed. This also helps to reduce the installed size of packages. -### `patch-shebangs.sh` +### `patch-shebangs.sh` {#patch-shebangs.sh} This setup hook patches installed scripts to use the full path to the shebang interpreter. A shebang interpreter is the first commented line of a script telling the operating system which program will run the script (e.g `#!/bin/bash`). In Nix, we want an exact path to that interpreter to be used. This often replaces `/bin/sh` with a path in the Nix store. -### `audit-tmpdir.sh` +### `audit-tmpdir.sh` {#audit-tmpdir.sh} This verifies that no references are left from the install binaries to the directory used to build those binaries. This ensures that the binaries do not need things outside the Nix store. This is currently supported in Linux only. -### `multiple-outputs.sh` +### `multiple-outputs.sh` {#multiple-outputs.sh} This setup hook adds configure flags that tell packages to install files into any one of the proper outputs listed in `outputs`. This behavior can be turned off by setting `setOutputFlags` to false in the derivation environment. See <xref linkend="chap-multiple-output" /> for more information. -### `move-sbin.sh` +### `move-sbin.sh` {#move-sbin.sh} This setup hook moves any binaries installed in the `sbin/` subdirectory into `bin/`. In addition, a link is provided from `sbin/` to `bin/` for compatibility. -### `move-lib64.sh` +### `move-lib64.sh` {#move-lib64.sh} This setup hook moves any libraries installed in the `lib64/` subdirectory into `lib/`. In addition, a link is provided from `lib64/` to `lib/` for compatibility. -### `move-systemd-user-units.sh` +### `move-systemd-user-units.sh` {#move-systemd-user-units.sh} This setup hook moves any systemd user units installed in the `lib/` subdirectory into `share/`. In addition, a link is provided from `share/` to `lib/` for compatibility. This is needed for systemd to find user services when installed into the user profile. -### `set-source-date-epoch-to-latest.sh` +### `set-source-date-epoch-to-latest.sh` {#set-source-date-epoch-to-latest.sh} This sets `SOURCE_DATE_EPOCH` to the modification time of the most recent file. -### Bintools Wrapper +### Bintools Wrapper {#bintools-wrapper} The Bintools Wrapper wraps the binary utilities for a bunch of miscellaneous purposes. These are GNU Binutils when targetting Linux, and a mix of cctools and GNU binutils for Darwin. \[The “Bintools” name is supposed to be a compromise between “Binutils” and “cctools” not denoting any specific implementation.\] Specifically, the underlying bintools package, and a C standard library (glibc or Darwin’s libSystem, just for the dynamic loader) are all fed in, and dependency finding, hardening (see below), and purity checks for each are handled by the Bintools Wrapper. Packages typically depend on CC Wrapper, which in turn (at run time) depends on the Bintools Wrapper. @@ -937,7 +939,7 @@ A final task of the setup hook is defining a number of standard environment vari A problem with this final task is that the Bintools Wrapper is honest and defines `LD` as `ld`. Most packages, however, firstly use the C compiler for linking, secondly use `LD` anyways, defining it as the C compiler, and thirdly, only so define `LD` when it is undefined as a fallback. This triple-threat means Bintools Wrapper will break those packages, as LD is already defined as the actual linker which the package won’t override yet doesn’t want to use. The workaround is to define, just for the problematic package, `LD` as the C compiler. A good way to do this would be `preConfigure = "LD=$CC"`. -### CC Wrapper +### CC Wrapper {#cc-wrapper} The CC Wrapper wraps a C toolchain for a bunch of miscellaneous purposes. Specifically, a C compiler (GCC or Clang), wrapped binary tools, and a C standard library (glibc or Darwin’s libSystem, just for the dynamic loader) are all fed in, and dependency finding, hardening (see below), and purity checks for each are handled by the CC Wrapper. Packages typically depend on the CC Wrapper, which in turn (at run-time) depends on the Bintools Wrapper. @@ -971,11 +973,11 @@ The `autoreconfHook` derivation adds `autoreconfPhase`, which runs autoreconf, l Adds every file named `catalog.xml` found under the `xml/dtd` and `xml/xsl` subdirectories of each build input to the `XML_CATALOG_FILES` environment variable. -### teTeX / TeX Live +### teTeX / TeX Live {#tetex-tex-live} Adds the `share/texmf-nix` subdirectory of each build input to the `TEXINPUTS` environment variable. -### Qt 4 +### Qt 4 {#qt-4} Sets the `QTDIR` environment variable to Qt’s path. @@ -983,11 +985,11 @@ Sets the `QTDIR` environment variable to Qt’s path. Exports `GDK_PIXBUF_MODULE_FILE` environment variable to the builder. Add librsvg package to `buildInputs` to get svg support. See also the [setup hook description in GNOME platform docs](#ssec-gnome-hooks-gdk-pixbuf). -### GHC +### GHC {#ghc} Creates a temporary package database and registers every Haskell build input in it (TODO: how?). -### GNOME platform +### GNOME platform {#gnome-platform} Hooks related to GNOME platform and related libraries like GLib, GTK and GStreamer are described in <xref linkend="sec-language-gnome" />. @@ -1003,7 +1005,7 @@ By default `autoPatchelf` will fail as soon as any ELF file requires a dependenc The `autoPatchelf` command also recognizes a `--no-recurse` command line flag, which prevents it from recursing into subdirectories. -### breakpointHook +### breakpointHook {#breakpointhook} This hook will make a build pause instead of stopping when a failure happens. It prevents nix from cleaning up the build environment immediately and allows the user to attach to a build environment using the `cntr` command. Upon build error it will print instructions on how to use `cntr`, which can be used to enter the environment for debugging. Installing cntr and running the command will provide shell access to the build sandbox of failed build. At `/var/lib/cntr` the sandboxed filesystem is mounted. All commands and files of the system are still accessible within the shell. To execute commands from the sandbox use the cntr exec subcommand. `cntr` is only supported on Linux-based platforms. To use it first add `cntr` to your `environment.systemPackages` on NixOS or alternatively to the root user on non-NixOS systems. Then in the package that is supposed to be inspected, add `breakpointHook` to `nativeBuildInputs`. @@ -1013,15 +1015,15 @@ nativeBuildInputs = [ breakpointHook ]; When a build failure happens there will be an instruction printed that shows how to attach with `cntr` to the build sandbox. -::: note -::: title +::: {.note} +::: {.title} Caution with remote builds ::: This won’t work with remote builds as the build environment is on a different machine and can’t be accessed by `cntr`. Remote builds can be turned off by setting `--option builders ''` for `nix-build` or `--builders ''` for `nix build`. ::: -### installShellFiles +### installShellFiles {#installshellfiles} This hook helps with installing manpages and shell completion files. It exposes 2 shell functions `installManPage` and `installShellCompletion` that can be used from your `postInstall` hook. @@ -1047,61 +1049,61 @@ postInstall = '' ''; ``` -### libiconv, libintl +### libiconv, libintl {#libiconv-libintl} A few libraries automatically add to `NIX_LDFLAGS` their library, making their symbols automatically available to the linker. This includes libiconv and libintl (gettext). This is done to provide compatibility between GNU Linux, where libiconv and libintl are bundled in, and other systems where that might not be the case. Sometimes, this behavior is not desired. To disable this behavior, set `dontAddExtraLibs`. -### validatePkgConfig +### validatePkgConfig {#validatepkgconfig} The `validatePkgConfig` hook validates all pkg-config (`.pc`) files in a package. This helps catching some common errors in pkg-config files, such as undefined variables. -### cmake +### cmake {#cmake} Overrides the default configure phase to run the CMake command. By default, we use the Make generator of CMake. In addition, dependencies are added automatically to CMAKE_PREFIX_PATH so that packages are correctly detected by CMake. Some additional flags are passed in to give similar behavior to configure-based packages. You can disable this hook’s behavior by setting configurePhase to a custom value, or by setting dontUseCmakeConfigure. cmakeFlags controls flags passed only to CMake. By default, parallel building is enabled as CMake supports parallel building almost everywhere. When Ninja is also in use, CMake will detect that and use the ninja generator. -### xcbuildHook +### xcbuildHook {#xcbuildhook} Overrides the build and install phases to run the "xcbuild" command. This hook is needed when a project only comes with build files for the XCode build system. You can disable this behavior by setting buildPhase and configurePhase to a custom value. xcbuildFlags controls flags passed only to xcbuild. -### Meson +### Meson {#meson} Overrides the configure phase to run meson to generate Ninja files. To run these files, you should accompany Meson with ninja. By default, `enableParallelBuilding` is enabled as Meson supports parallel building almost everywhere. -#### Variables controlling Meson +#### Variables controlling Meson {#variables-controlling-meson} -##### `mesonFlags` +##### `mesonFlags` {#mesonflags} Controls the flags passed to meson. -##### `mesonBuildType` +##### `mesonBuildType` {#mesonbuildtype} Which [`--buildtype`](https://mesonbuild.com/Builtin-options.html#core-options) to pass to Meson. We default to `plain`. -##### `mesonAutoFeatures` +##### `mesonAutoFeatures` {#mesonautofeatures} What value to set [`-Dauto_features=`](https://mesonbuild.com/Builtin-options.html#core-options) to. We default to `enabled`. -##### `mesonWrapMode` +##### `mesonWrapMode` {#mesonwrapmode} What value to set [`-Dwrap_mode=`](https://mesonbuild.com/Builtin-options.html#core-options) to. We default to `nodownload` as we disallow network access. -##### `dontUseMesonConfigure` +##### `dontUseMesonConfigure` {#dontusemesonconfigure} Disables using Meson’s `configurePhase`. -### ninja +### ninja {#ninja} Overrides the build, install, and check phase to run ninja instead of make. You can disable this behavior with the `dontUseNinjaBuild`, `dontUseNinjaInstall`, and `dontUseNinjaCheck`, respectively. Parallel building is enabled by default in Ninja. -### unzip +### unzip {#unzip} This setup hook will allow you to unzip .zip files specified in `$src`. There are many similar packages like `unrar`, `undmg`, etc. -### wafHook +### wafHook {#wafhook} Overrides the configure, build, and install phases. This will run the “waf” script used by many projects. If `wafPath` (default `./waf`) doesn’t exist, it will copy the version of waf available in Nixpkgs. `wafFlags` can be used to pass flags to the waf script. -### scons +### scons {#scons} Overrides the build, install, and check phases. This uses the scons build system as a replacement for make. scons does not provide a configure phase, so everything is managed at build and install time. @@ -1119,7 +1121,7 @@ Both parameters take a list of flags as strings. The special `"all"` flag can be The following flags are enabled by default and might require disabling with `hardeningDisable` if the program to package is incompatible. -### `format` +### `format` {#format} Adds the `-Wformat -Wformat-security -Werror=format-security` compiler options. At present, this warns about calls to `printf` and `scanf` functions where the format string is not a string literal and there are no format arguments, as in `printf(foo);`. This may be a security hole if the format string came from untrusted input and contains `%n`. @@ -1132,7 +1134,7 @@ This needs to be turned off or fixed for errors similar to: cc1plus: some warnings being treated as errors ``` -### `stackprotector` +### `stackprotector` {#stackprotector} Adds the `-fstack-protector-strong --param ssp-buffer-size=4` compiler options. This adds safety checks against stack overwrites rendering many potential code injection attacks into aborting situations. In the best case this turns code injection vulnerabilities into denial of service or into non-issues (depending on the application). @@ -1143,7 +1145,7 @@ bin/blib.a(bios_console.o): In function `bios_handle_cup': /tmp/nix-build-ipxe-20141124-5cbdc41.drv-0/ipxe-5cbdc41/src/arch/i386/firmware/pcbios/bios_console.c:86: undefined reference to `__stack_chk_fail' ``` -### `fortify` +### `fortify` {#fortify} Adds the `-O2 -D_FORTIFY_SOURCE=2` compiler options. During code generation the compiler knows a great deal of information about buffer sizes (where possible), and attempts to replace insecure unlimited length buffer function calls with length-limited ones. This is especially useful for old, crufty code. Additionally, format strings in writable memory that contain `%n` are blocked. If an application depends on such a format string, it will need to be worked around. @@ -1164,7 +1166,7 @@ installwatch.c:3751:5: error: conflicting types for '__open_2' fcntl2.h:50:4: error: call to '__open_missing_mode' declared with attribute error: open with O_CREAT or O_TMPFILE in second argument needs 3 arguments ``` -### `pic` +### `pic` {#pic} Adds the `-fPIC` compiler options. This options adds support for position independent code in shared libraries and thus making ASLR possible. @@ -1177,19 +1179,19 @@ ccbLfRgg.s: Assembler messages: ccbLfRgg.s:33: Error: missing or invalid displacement expression `private_key_len@GOTOFF' ``` -### `strictoverflow` +### `strictoverflow` {#strictoverflow} Signed integer overflow is undefined behaviour according to the C standard. If it happens, it is an error in the program as it should check for overflow before it can happen, not afterwards. GCC provides built-in functions to perform arithmetic with overflow checking, which are correct and faster than any custom implementation. As a workaround, the option `-fno-strict-overflow` makes gcc behave as if signed integer overflows were defined. This flag should not trigger any build or runtime errors. -### `relro` +### `relro` {#relro} Adds the `-z relro` linker option. During program load, several ELF memory sections need to be written to by the linker, but can be turned read-only before turning over control to the program. This prevents some GOT (and .dtors) overwrite attacks, but at least the part of the GOT used by the dynamic linker (.got.plt) is still vulnerable. This flag can break dynamic shared object loading. For instance, the module systems of Xorg and OpenCV are incompatible with this flag. In almost all cases the `bindnow` flag must also be disabled and incompatible programs typically fail with similar errors at runtime. -### `bindnow` +### `bindnow` {#bindnow} Adds the `-z bindnow` linker option. During program load, all dynamic symbols are resolved, allowing for the complete GOT to be marked read-only (due to `relro`). This prevents GOT overwrite attacks. For very large applications, this can incur some performance loss during initial load while symbols are resolved, but this shouldn’t be an issue for daemons. @@ -1201,7 +1203,7 @@ intel_drv.so: undefined symbol: vgaHWFreeHWRec The following flags are disabled by default and should be enabled with `hardeningEnable` for packages that take untrusted input like network services. -### `pie` +### `pie` {#pie} Adds the `-fPIE` compiler and `-pie` linker options. Position Independent Executables are needed to take advantage of Address Space Layout Randomization, supported by modern kernel versions. While ASLR can already be enforced for data areas in the stack and heap (brk and mmap), the code areas must be compiled as position-independent. Shared libraries already do this with the `pic` flag, so they gain ASLR automatically, but binary .text regions need to be build with `pie` to gain ASLR. When this happens, ROP attacks are much harder since there are no static locations to bounce off of during a memory corruption attack. |