diff options
author | Graham Christensen <graham@grahamc.com> | 2018-10-03 21:52:48 -0400 |
---|---|---|
committer | GitHub <noreply@github.com> | 2018-10-03 21:52:48 -0400 |
commit | f2b3bbe44e4f7e220ecb1308d1acdac5e6a0f8ba (patch) | |
tree | 1c01a018bdb41e61f3bd563ca01d85a624894277 /doc | |
parent | 932833f519c40b38c19042b4e0502335e00660b3 (diff) | |
parent | c07ba7c8560250d1b184698e6453b6d5ca11846f (diff) |
Merge pull request #47688 from grahamc/doc-breakout-functions
nixpkgs docs: breakout functions
Diffstat (limited to 'doc')
-rw-r--r-- | doc/Makefile | 2 | ||||
-rw-r--r-- | doc/cross-compilation.xml | 7 | ||||
-rw-r--r-- | doc/functions.xml | 1017 | ||||
-rw-r--r-- | doc/functions/debug.xml | 21 | ||||
-rw-r--r-- | doc/functions/dockertools.xml | 564 | ||||
-rw-r--r-- | doc/functions/fhs-environments.xml | 142 | ||||
-rw-r--r-- | doc/functions/generators.xml | 89 | ||||
-rw-r--r-- | doc/functions/overrides.xml | 203 | ||||
-rw-r--r-- | doc/functions/shell.xml | 26 | ||||
-rw-r--r-- | doc/package-notes.xml | 74 | ||||
-rw-r--r-- | doc/shell.section.md | 22 |
11 files changed, 1102 insertions, 1065 deletions
diff --git a/doc/Makefile b/doc/Makefile index ba77be6678c41..173e1c0b19ee1 100644 --- a/doc/Makefile +++ b/doc/Makefile @@ -69,7 +69,7 @@ highlightjs: cp -r "$$HIGHLIGHTJS/loader.js" highlightjs/ -manual-full.xml: ${MD_TARGETS} .version *.xml +manual-full.xml: ${MD_TARGETS} .version *.xml **/*.xml xmllint --nonet --xinclude --noxincludenode manual.xml --output manual-full.xml .version: diff --git a/doc/cross-compilation.xml b/doc/cross-compilation.xml index c7187d86d1b3e..da664394f2624 100644 --- a/doc/cross-compilation.xml +++ b/doc/cross-compilation.xml @@ -47,9 +47,10 @@ <para> In Nixpkgs, these three platforms are defined as attribute sets under the - names <literal>buildPlatform</literal>, <literal>hostPlatform</literal>, and - <literal>targetPlatform</literal>. They are always defined as attributes in - the standard environment. That means one can access them like: + names <literal>buildPlatform</literal>, <literal>hostPlatform</literal>, + and <literal>targetPlatform</literal>. They are always defined as + attributes in the standard environment. That means one can access them + like: <programlisting>{ stdenv, fooDep, barDep, .. }: ...stdenv.buildPlatform...</programlisting> . </para> diff --git a/doc/functions.xml b/doc/functions.xml index 8223a8b0531c0..88011061ae6ed 100644 --- a/doc/functions.xml +++ b/doc/functions.xml @@ -7,1016 +7,11 @@ The nixpkgs repository has several utility functions to manipulate Nix expressions. </para> - <section xml:id="sec-overrides"> - <title>Overriding</title> - <para> - Sometimes one wants to override parts of <literal>nixpkgs</literal>, e.g. - derivation attributes, the results of derivations or even the whole package - set. - </para> - - <section xml:id="sec-pkg-override"> - <title><pkg>.override</title> - - <para> - The function <varname>override</varname> is usually available for all the - derivations in the nixpkgs expression (<varname>pkgs</varname>). - </para> - - <para> - It is used to override the arguments passed to a function. - </para> - - <para> - Example usages: -<programlisting>pkgs.foo.override { arg1 = val1; arg2 = val2; ... }</programlisting> -<programlisting> -import pkgs.path { overlays = [ (self: super: { - foo = super.foo.override { barSupport = true ; }; - })]}; -</programlisting> -<programlisting> -mypkg = pkgs.callPackage ./mypkg.nix { - mydep = pkgs.mydep.override { ... }; - } -</programlisting> - </para> - - <para> - In the first example, <varname>pkgs.foo</varname> is the result of a - function call with some default arguments, usually a derivation. Using - <varname>pkgs.foo.override</varname> will call the same function with the - given new arguments. - </para> - </section> - - <section xml:id="sec-pkg-overrideAttrs"> - <title><pkg>.overrideAttrs</title> - - <para> - The function <varname>overrideAttrs</varname> allows overriding the - attribute set passed to a <varname>stdenv.mkDerivation</varname> call, - producing a new derivation based on the original one. This function is - available on all derivations produced by the - <varname>stdenv.mkDerivation</varname> function, which is most packages in - the nixpkgs expression <varname>pkgs</varname>. - </para> - - <para> - Example usage: -<programlisting> -helloWithDebug = pkgs.hello.overrideAttrs (oldAttrs: rec { - separateDebugInfo = true; -}); -</programlisting> - </para> - - <para> - In the above example, the <varname>separateDebugInfo</varname> attribute is - overridden to be true, thus building debug info for - <varname>helloWithDebug</varname>, while all other attributes will be - retained from the original <varname>hello</varname> package. - </para> - - <para> - The argument <varname>oldAttrs</varname> is conventionally used to refer to - the attr set originally passed to <varname>stdenv.mkDerivation</varname>. - </para> - - <note> - <para> - Note that <varname>separateDebugInfo</varname> is processed only by the - <varname>stdenv.mkDerivation</varname> function, not the generated, raw - Nix derivation. Thus, using <varname>overrideDerivation</varname> will not - work in this case, as it overrides only the attributes of the final - derivation. It is for this reason that <varname>overrideAttrs</varname> - should be preferred in (almost) all cases to - <varname>overrideDerivation</varname>, i.e. to allow using - <varname>sdenv.mkDerivation</varname> to process input arguments, as well - as the fact that it is easier to use (you can use the same attribute names - you see in your Nix code, instead of the ones generated (e.g. - <varname>buildInputs</varname> vs <varname>nativeBuildInputs</varname>, - and involves less typing. - </para> - </note> - </section> - - <section xml:id="sec-pkg-overrideDerivation"> - <title><pkg>.overrideDerivation</title> - - <warning> - <para> - You should prefer <varname>overrideAttrs</varname> in almost all cases, - see its documentation for the reasons why. - <varname>overrideDerivation</varname> is not deprecated and will continue - to work, but is less nice to use and does not have as many abilities as - <varname>overrideAttrs</varname>. - </para> - </warning> - - <warning> - <para> - Do not use this function in Nixpkgs as it evaluates a Derivation before - modifying it, which breaks package abstraction and removes error-checking - of function arguments. In addition, this evaluation-per-function - application incurs a performance penalty, which can become a problem if - many overrides are used. It is only intended for ad-hoc customisation, - such as in <filename>~/.config/nixpkgs/config.nix</filename>. - </para> - </warning> - - <para> - The function <varname>overrideDerivation</varname> creates a new derivation - based on an existing one by overriding the original's attributes with the - attribute set produced by the specified function. This function is - available on all derivations defined using the - <varname>makeOverridable</varname> function. Most standard - derivation-producing functions, such as - <varname>stdenv.mkDerivation</varname>, are defined using this function, - which means most packages in the nixpkgs expression, - <varname>pkgs</varname>, have this function. - </para> - - <para> - Example usage: -<programlisting> -mySed = pkgs.gnused.overrideDerivation (oldAttrs: { - name = "sed-4.2.2-pre"; - src = fetchurl { - url = ftp://alpha.gnu.org/gnu/sed/sed-4.2.2-pre.tar.bz2; - sha256 = "11nq06d131y4wmf3drm0yk502d2xc6n5qy82cg88rb9nqd2lj41k"; - }; - patches = []; -}); -</programlisting> - </para> - - <para> - In the above example, the <varname>name</varname>, <varname>src</varname>, - and <varname>patches</varname> of the derivation will be overridden, while - all other attributes will be retained from the original derivation. - </para> - - <para> - The argument <varname>oldAttrs</varname> is used to refer to the attribute - set of the original derivation. - </para> - - <note> - <para> - A package's attributes are evaluated *before* being modified by the - <varname>overrideDerivation</varname> function. For example, the - <varname>name</varname> attribute reference in <varname>url = - "mirror://gnu/hello/${name}.tar.gz";</varname> is filled-in *before* the - <varname>overrideDerivation</varname> function modifies the attribute set. - This means that overriding the <varname>name</varname> attribute, in this - example, *will not* change the value of the <varname>url</varname> - attribute. Instead, we need to override both the <varname>name</varname> - *and* <varname>url</varname> attributes. - </para> - </note> - </section> - - <section xml:id="sec-lib-makeOverridable"> - <title>lib.makeOverridable</title> - - <para> - The function <varname>lib.makeOverridable</varname> is used to make the - result of a function easily customizable. This utility only makes sense for - functions that accept an argument set and return an attribute set. - </para> - - <para> - Example usage: -<programlisting> -f = { a, b }: { result = a+b; }; -c = lib.makeOverridable f { a = 1; b = 2; }; -</programlisting> - </para> - - <para> - The variable <varname>c</varname> is the value of the <varname>f</varname> - function applied with some default arguments. Hence the value of - <varname>c.result</varname> is <literal>3</literal>, in this example. - </para> - - <para> - The variable <varname>c</varname> however also has some additional - functions, like <link linkend="sec-pkg-override">c.override</link> which - can be used to override the default arguments. In this example the value of - <varname>(c.override { a = 4; }).result</varname> is 6. - </para> - </section> - </section> - <section xml:id="sec-generators"> - <title>Generators</title> - - <para> - Generators are functions that create file formats from nix data structures, - e. g. for configuration files. There are generators available for: - <literal>INI</literal>, <literal>JSON</literal> and <literal>YAML</literal> - </para> - - <para> - All generators follow a similar call interface: <code>generatorName - configFunctions data</code>, where <literal>configFunctions</literal> is an - attrset of user-defined functions that format nested parts of the content. - They each have common defaults, so often they do not need to be set - manually. An example is <code>mkSectionName ? (name: libStr.escape [ "[" "]" - ] name)</code> from the <literal>INI</literal> generator. It receives the - name of a section and sanitizes it. The default - <literal>mkSectionName</literal> escapes <literal>[</literal> and - <literal>]</literal> with a backslash. - </para> - - <para> - Generators can be fine-tuned to produce exactly the file format required by - your application/service. One example is an INI-file format which uses - <literal>: </literal> as separator, the strings - <literal>"yes"</literal>/<literal>"no"</literal> as boolean values and - requires all string values to be quoted: - </para> - -<programlisting> -with lib; -let - customToINI = generators.toINI { - # specifies how to format a key/value pair - mkKeyValue = generators.mkKeyValueDefault { - # specifies the generated string for a subset of nix values - mkValueString = v: - if v == true then ''"yes"'' - else if v == false then ''"no"'' - else if isString v then ''"${v}"'' - # and delegats all other values to the default generator - else generators.mkValueStringDefault {} v; - } ":"; - }; - -# the INI file can now be given as plain old nix values -in customToINI { - main = { - pushinfo = true; - autopush = false; - host = "localhost"; - port = 42; - }; - mergetool = { - merge = "diff3"; - }; -} -</programlisting> - - <para> - This will produce the following INI file as nix string: - </para> - -<programlisting> -[main] -autopush:"no" -host:"localhost" -port:42 -pushinfo:"yes" -str\:ange:"very::strange" - -[mergetool] -merge:"diff3" -</programlisting> - - <note> - <para> - Nix store paths can be converted to strings by enclosing a derivation - attribute like so: <code>"${drv}"</code>. - </para> - </note> - - <para> - Detailed documentation for each generator can be found in - <literal>lib/generators.nix</literal>. - </para> - </section> - <section xml:id="sec-debug"> - <title>Debugging Nix Expressions</title> - - <para> - Nix is a unityped, dynamic language, this means every value can potentially - appear anywhere. Since it is also non-strict, evaluation order and what - ultimately is evaluated might surprise you. Therefore it is important to be - able to debug nix expressions. - </para> - - <para> - In the <literal>lib/debug.nix</literal> file you will find a number of - functions that help (pretty-)printing values while evaluation is runnnig. - You can even specify how deep these values should be printed recursively, - and transform them on the fly. Please consult the docstrings in - <literal>lib/debug.nix</literal> for usage information. - </para> - </section> - <section xml:id="sec-fhs-environments"> - <title>buildFHSUserEnv</title> - - <para> - <function>buildFHSUserEnv</function> provides a way to build and run - FHS-compatible lightweight sandboxes. It creates an isolated root with bound - <filename>/nix/store</filename>, so its footprint in terms of disk space - needed is quite small. This allows one to run software which is hard or - unfeasible to patch for NixOS -- 3rd-party source trees with FHS - assumptions, games distributed as tarballs, software with integrity checking - and/or external self-updated binaries. It uses Linux namespaces feature to - create temporary lightweight environments which are destroyed after all - child processes exit, without root user rights requirement. Accepted - arguments are: - </para> - - <variablelist> - <varlistentry> - <term> - <literal>name</literal> - </term> - <listitem> - <para> - Environment name. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <literal>targetPkgs</literal> - </term> - <listitem> - <para> - Packages to be installed for the main host's architecture (i.e. x86_64 on - x86_64 installations). Along with libraries binaries are also installed. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <literal>multiPkgs</literal> - </term> - <listitem> - <para> - Packages to be installed for all architectures supported by a host (i.e. - i686 and x86_64 on x86_64 installations). Only libraries are installed by - default. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <literal>extraBuildCommands</literal> - </term> - <listitem> - <para> - Additional commands to be executed for finalizing the directory - structure. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <literal>extraBuildCommandsMulti</literal> - </term> - <listitem> - <para> - Like <literal>extraBuildCommands</literal>, but executed only on multilib - architectures. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <literal>extraOutputsToInstall</literal> - </term> - <listitem> - <para> - Additional derivation outputs to be linked for both target and - multi-architecture packages. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <literal>extraInstallCommands</literal> - </term> - <listitem> - <para> - Additional commands to be executed for finalizing the derivation with - runner script. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <literal>runScript</literal> - </term> - <listitem> - <para> - A command that would be executed inside the sandbox and passed all the - command line arguments. It defaults to <literal>bash</literal>. - </para> - </listitem> - </varlistentry> - </variablelist> - - <para> - One can create a simple environment using a <literal>shell.nix</literal> - like that: - </para> - -<programlisting><![CDATA[ -{ pkgs ? import <nixpkgs> {} }: - -(pkgs.buildFHSUserEnv { - name = "simple-x11-env"; - targetPkgs = pkgs: (with pkgs; - [ udev - alsaLib - ]) ++ (with pkgs.xorg; - [ libX11 - libXcursor - libXrandr - ]); - multiPkgs = pkgs: (with pkgs; - [ udev - alsaLib - ]); - runScript = "bash"; -}).env -]]></programlisting> - - <para> - Running <literal>nix-shell</literal> would then drop you into a shell with - these libraries and binaries available. You can use this to run - closed-source applications which expect FHS structure without hassles: - simply change <literal>runScript</literal> to the application path, e.g. - <filename>./bin/start.sh</filename> -- relative paths are supported. - </para> - </section> - <xi:include href="shell.section.xml" /> - <section xml:id="sec-pkgs-dockerTools"> - <title>pkgs.dockerTools</title> - - <para> - <varname>pkgs.dockerTools</varname> is a set of functions for creating and - manipulating Docker images according to the - <link xlink:href="https://github.com/moby/moby/blob/master/image/spec/v1.2.md#docker-image-specification-v120"> - Docker Image Specification v1.2.0 </link>. Docker itself is not used to - perform any of the operations done by these functions. - </para> - - <warning> - <para> - The <varname>dockerTools</varname> API is unstable and may be subject to - backwards-incompatible changes in the future. - </para> - </warning> - - <section xml:id="ssec-pkgs-dockerTools-buildImage"> - <title>buildImage</title> - - <para> - This function is analogous to the <command>docker build</command> command, - in that can used to build a Docker-compatible repository tarball containing - a single image with one or multiple layers. As such, the result is suitable - for being loaded in Docker with <command>docker load</command>. - </para> - - <para> - The parameters of <varname>buildImage</varname> with relative example - values are described below: - </para> - - <example xml:id='ex-dockerTools-buildImage'> - <title>Docker build</title> -<programlisting> -buildImage { - name = "redis"; <co xml:id='ex-dockerTools-buildImage-1' /> - tag = "latest"; <co xml:id='ex-dockerTools-buildImage-2' /> - - fromImage = someBaseImage; <co xml:id='ex-dockerTools-buildImage-3' /> - fromImageName = null; <co xml:id='ex-dockerTools-buildImage-4' /> - fromImageTag = "latest"; <co xml:id='ex-dockerTools-buildImage-5' /> - - contents = pkgs.redis; <co xml:id='ex-dockerTools-buildImage-6' /> - runAsRoot = '' <co xml:id='ex-dockerTools-buildImage-runAsRoot' /> - #!${stdenv.shell} - mkdir -p /data - ''; - - config = { <co xml:id='ex-dockerTools-buildImage-8' /> - Cmd = [ "/bin/redis-server" ]; - WorkingDir = "/data"; - Volumes = { - "/data" = {}; - }; - }; -} -</programlisting> - </example> - - <para> - The above example will build a Docker image <literal>redis/latest</literal> - from the given base image. Loading and running this image in Docker results - in <literal>redis-server</literal> being started automatically. - </para> - - <calloutlist> - <callout arearefs='ex-dockerTools-buildImage-1'> - <para> - <varname>name</varname> specifies the name of the resulting image. This - is the only required argument for <varname>buildImage</varname>. - </para> - </callout> - <callout arearefs='ex-dockerTools-buildImage-2'> - <para> - <varname>tag</varname> specifies the tag of the resulting image. By - default it's <literal>null</literal>, which indicates that the nix output - hash will be used as tag. - </para> - </callout> - <callout arearefs='ex-dockerTools-buildImage-3'> - <para> - <varname>fromImage</varname> is the repository tarball containing the - base image. It must be a valid Docker image, such as exported by - <command>docker save</command>. By default it's <literal>null</literal>, - which can be seen as equivalent to <literal>FROM scratch</literal> of a - <filename>Dockerfile</filename>. - </para> - </callout> - <callout arearefs='ex-dockerTools-buildImage-4'> - <para> - <varname>fromImageName</varname> can be used to further specify the base - image within the repository, in case it contains multiple images. By - default it's <literal>null</literal>, in which case - <varname>buildImage</varname> will peek the first image available in the - repository. - </para> - </callout> - <callout arearefs='ex-dockerTools-buildImage-5'> - <para> - <varname>fromImageTag</varname> can be used to further specify the tag of - the base image within the repository, in case an image contains multiple - tags. By default it's <literal>null</literal>, in which case - <varname>buildImage</varname> will peek the first tag available for the - base image. - </para> - </callout> - <callout arearefs='ex-dockerTools-buildImage-6'> - <para> - <varname>contents</varname> is a derivation that will be copied in the - new layer of the resulting image. This can be similarly seen as - <command>ADD contents/ /</command> in a <filename>Dockerfile</filename>. - By default it's <literal>null</literal>. - </para> - </callout> - <callout arearefs='ex-dockerTools-buildImage-runAsRoot'> - <para> - <varname>runAsRoot</varname> is a bash script that will run as root in an - environment that overlays the existing layers of the base image with the - new resulting layer, including the previously copied - <varname>contents</varname> derivation. This can be similarly seen as - <command>RUN ...</command> in a <filename>Dockerfile</filename>. - <note> - <para> - Using this parameter requires the <literal>kvm</literal> device to be - available. - </para> - </note> - </para> - </callout> - <callout arearefs='ex-dockerTools-buildImage-8'> - <para> - <varname>config</varname> is used to specify the configuration of the - containers that will be started off the built image in Docker. The - available options are listed in the - <link xlink:href="https://github.com/moby/moby/blob/master/image/spec/v1.2.md#image-json-field-descriptions"> - Docker Image Specification v1.2.0 </link>. - </para> - </callout> - </calloutlist> - - <para> - After the new layer has been created, its closure (to which - <varname>contents</varname>, <varname>config</varname> and - <varname>runAsRoot</varname> contribute) will be copied in the layer - itself. Only new dependencies that are not already in the existing layers - will be copied. - </para> - - <para> - At the end of the process, only one new single layer will be produced and - added to the resulting image. - </para> - - <para> - The resulting repository will only list the single image - <varname>image/tag</varname>. In the case of - <xref linkend='ex-dockerTools-buildImage'/> it would be - <varname>redis/latest</varname>. - </para> - - <para> - It is possible to inspect the arguments with which an image was built using - its <varname>buildArgs</varname> attribute. - </para> - - <note> - <para> - If you see errors similar to <literal>getProtocolByName: does not exist - (no such protocol name: tcp)</literal> you may need to add - <literal>pkgs.iana-etc</literal> to <varname>contents</varname>. - </para> - </note> - - <note> - <para> - If you see errors similar to <literal>Error_Protocol ("certificate has - unknown CA",True,UnknownCa)</literal> you may need to add - <literal>pkgs.cacert</literal> to <varname>contents</varname>. - </para> - </note> - - <example xml:id="example-pkgs-dockerTools-buildImage-creation-date"> - <title>Impurely Defining a Docker Layer's Creation Date</title> - <para> - By default <function>buildImage</function> will use a static - date of one second past the UNIX Epoch. This allows - <function>buildImage</function> to produce binary reproducible - images. When listing images with <command>docker list - images</command>, the newly created images will be listed like - this: - </para> - <screen><![CDATA[ -$ docker image list -REPOSITORY TAG IMAGE ID CREATED SIZE -hello latest 08c791c7846e 48 years ago 25.2MB -]]></screen> - <para> - You can break binary reproducibility but have a sorted, - meaningful <literal>CREATED</literal> column by setting - <literal>created</literal> to <literal>now</literal>. - </para> - <programlisting><![CDATA[ -pkgs.dockerTools.buildImage { - name = "hello"; - tag = "latest"; - created = "now"; - contents = pkgs.hello; - - config.Cmd = [ "/bin/hello" ]; -} -]]></programlisting> - <para> - and now the Docker CLI will display a reasonable date and - sort the images as expected: - <screen><![CDATA[ -$ docker image list -REPOSITORY TAG IMAGE ID CREATED SIZE -hello latest de2bf4786de6 About a minute ago 25.2MB -]]></screen> - however, the produced images will not be binary reproducible. - </para> - </example> - </section> - - <section xml:id="ssec-pkgs-dockerTools-buildLayeredImage"> - <title>buildLayeredImage</title> - - <para> - Create a Docker image with many of the store paths being on their own layer - to improve sharing between images. - </para> - - <variablelist> - <varlistentry> - <term> - <varname>name</varname> - </term> - <listitem> - <para> - The name of the resulting image. - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <varname>tag</varname> <emphasis>optional</emphasis> - </term> - <listitem> - <para> - Tag of the generated image. - </para> - <para> - <emphasis>Default:</emphasis> the output path's hash - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <varname>contents</varname> <emphasis>optional</emphasis> - </term> - <listitem> - <para> - Top level paths in the container. Either a single derivation, or a list - of derivations. - </para> - <para> - <emphasis>Default:</emphasis> <literal>[]</literal> - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <varname>config</varname> <emphasis>optional</emphasis> - </term> - <listitem> - <para> - Run-time configuration of the container. A full list of the options are - available at in the - <link xlink:href="https://github.com/moby/moby/blob/master/image/spec/v1.2.md#image-json-field-descriptions"> - Docker Image Specification v1.2.0 </link>. - </para> - <para> - <emphasis>Default:</emphasis> <literal>{}</literal> - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <varname>created</varname> <emphasis>optional</emphasis> - </term> - <listitem> - <para> - Date and time the layers were created. Follows the same - <literal>now</literal> exception supported by - <literal>buildImage</literal>. - </para> - <para> - <emphasis>Default:</emphasis> <literal>1970-01-01T00:00:01Z</literal> - </para> - </listitem> - </varlistentry> - <varlistentry> - <term> - <varname>maxLayers</varname> <emphasis>optional</emphasis> - </term> - <listitem> - <para> - Maximum number of layers to create. - </para> - <para> - <emphasis>Default:</emphasis> <literal>24</literal> - </para> - </listitem> - </varlistentry> - </variablelist> - - <section xml:id="dockerTools-buildLayeredImage-arg-contents"> - <title>Behavior of <varname>contents</varname> in the final image</title> - - <para> - Each path directly listed in <varname>contents</varname> will have a - symlink in the root of the image. - </para> - - <para> - For example: -<programlisting><![CDATA[ -pkgs.dockerTools.buildLayeredImage { - name = "hello"; - contents = [ pkgs.hello ]; -} -]]></programlisting> - will create symlinks for all the paths in the <literal>hello</literal> - package: -<screen><![CDATA[ -/bin/hello -> /nix/store/h1zb1padqbbb7jicsvkmrym3r6snphxg-hello-2.10/bin/hello -/share/info/hello.info -> /nix/store/h1zb1padqbbb7jicsvkmrym3r6snphxg-hello-2.10/share/info/hello.info -/share/locale/bg/LC_MESSAGES/hello.mo -> /nix/store/h1zb1padqbbb7jicsvkmrym3r6snphxg-hello-2.10/share/locale/bg/LC_MESSAGES/hello.mo -]]></screen> - </para> - </section> - - <section xml:id="dockerTools-buildLayeredImage-arg-config"> - <title>Automatic inclusion of <varname>config</varname> references</title> - - <para> - The closure of <varname>config</varname> is automatically included in the - closure of the final image. - </para> - - <para> - This allows you to make very simple Docker images with very little code. - This container will start up and run <command>hello</command>: -<programlisting><![CDATA[ -pkgs.dockerTools.buildLayeredImage { - name = "hello"; - config.Cmd = [ "${pkgs.hello}/bin/hello" ]; -} -]]></programlisting> - </para> - </section> - - <section xml:id="dockerTools-buildLayeredImage-arg-maxLayers"> - <title>Adjusting <varname>maxLayers</varname></title> - - <para> - Increasing the <varname>maxLayers</varname> increases the number of layers - which have a chance to be shared between different images. - </para> - - <para> - Modern Docker installations support up to 128 layers, however older - versions support as few as 42. - </para> - - <para> - If the produced image will not be extended by other Docker builds, it is - safe to set <varname>maxLayers</varname> to <literal>128</literal>. - However it will be impossible to extend the image further. - </para> - - <para> - The first (<literal>maxLayers-2</literal>) most "popular" paths will have - their own individual layers, then layer #<literal>maxLayers-1</literal> - will contain all the remaining "unpopular" paths, and finally layer - #<literal>maxLayers</literal> will contain the Image configuration. - </para> - - <para> - Docker's Layers are not inherently ordered, they are content-addressable - and are not explicitly layered until they are composed in to an Image. - </para> - </section> - </section> - - <section xml:id="ssec-pkgs-dockerTools-fetchFromRegistry"> - <title>pullImage</title> - - <para> - This function is analogous to the <command>docker pull</command> command, - in that can be used to pull a Docker image from a Docker registry. By - default <link xlink:href="https://hub.docker.com/">Docker Hub</link> is - used to pull images. - </para> - - <para> - Its parameters are described in the example below: - </para> - - <example xml:id='ex-dockerTools-pullImage'> - <title>Docker pull</title> -<programlisting> -pullImage { - imageName = "nixos/nix"; <co xml:id='ex-dockerTools-pullImage-1' /> - imageDigest = "sha256:20d9485b25ecfd89204e843a962c1bd70e9cc6858d65d7f5fadc340246e2116b"; <co xml:id='ex-dockerTools-pullImage-2' /> - finalImageTag = "1.11"; <co xml:id='ex-dockerTools-pullImage-3' /> - sha256 = "0mqjy3zq2v6rrhizgb9nvhczl87lcfphq9601wcprdika2jz7qh8"; <co xml:id='ex-dockerTools-pullImage-4' /> - os = "linux"; <co xml:id='ex-dockerTools-pullImage-5' /> - arch = "x86_64"; <co xml:id='ex-dockerTools-pullImage-6' /> -} -</programlisting> - </example> - - <calloutlist> - <callout arearefs='ex-dockerTools-pullImage-1'> - <para> - <varname>imageName</varname> specifies the name of the image to be - downloaded, which can also include the registry namespace (e.g. - <literal>nixos</literal>). This argument is required. - </para> - </callout> - <callout arearefs='ex-dockerTools-pullImage-2'> - <para> - <varname>imageDigest</varname> specifies the digest of the image to be - downloaded. Skopeo can be used to get the digest of an image, with its - <varname>inspect</varname> subcommand. Since a given - <varname>imageName</varname> may transparently refer to a manifest list - of images which support multiple architectures and/or operating systems, - supply the `--override-os` and `--override-arch` arguments to specify - exactly which image you want. By default it will match the OS and - architecture of the host the command is run on. -<programlisting> -$ nix-shell --packages skopeo jq --command "skopeo --override-os linux --override-arch x86_64 inspect docker://docker.io/nixos/nix:1.11 | jq -r '.Digest'" -sha256:20d9485b25ecfd89204e843a962c1bd70e9cc6858d65d7f5fadc340246e2116b -</programlisting> - This argument is required. - </para> - </callout> - <callout arearefs='ex-dockerTools-pullImage-3'> - <para> - <varname>finalImageTag</varname>, if specified, this is the tag of the - image to be created. Note it is never used to fetch the image since we - prefer to rely on the immutable digest ID. By default it's - <literal>latest</literal>. - </para> - </callout> - <callout arearefs='ex-dockerTools-pullImage-4'> - <para> - <varname>sha256</varname> is the checksum of the whole fetched image. - This argument is required. - </para> - </callout> - <callout arearefs='ex-dockerTools-pullImage-5'> - <para> - <varname>os</varname>, if specified, is the operating system of the - fetched image. By default it's <literal>linux</literal>. - </para> - </callout> - <callout arearefs='ex-dockerTools-pullImage-6'> - <para> - <varname>arch</varname>, if specified, is the cpu architecture of the - fetched image. By default it's <literal>x86_64</literal>. - </para> - </callout> - </calloutlist> - </section> - - <section xml:id="ssec-pkgs-dockerTools-exportImage"> - <title>exportImage</title> - - <para> - This function is analogous to the <command>docker export</command> command, - in that can used to flatten a Docker image that contains multiple layers. - It is in fact the result of the merge of all the layers of the image. As - such, the result is suitable for being imported in Docker with - <command>docker import</command>. - </para> - - <note> - <para> - Using this function requires the <literal>kvm</literal> device to be - available. - </para> - </note> - - <para> - The parameters of <varname>exportImage</varname> are the following: - </para> - - <example xml:id='ex-dockerTools-exportImage'> - <title>Docker export</title> -<programlisting> -exportImage { - fromImage = someLayeredImage; - fromImageName = null; - fromImageTag = null; - - name = someLayeredImage.name; -} - </programlisting> - </example> - - <para> - The parameters relative to the base image have the same synopsis as - described in <xref linkend='ssec-pkgs-dockerTools-buildImage'/>, except - that <varname>fromImage</varname> is the only required argument in this - case. - </para> - - <para> - The <varname>name</varname> argument is the name of the derivation output, - which defaults to <varname>fromImage.name</varname>. - </para> - </section> - - <section xml:id="ssec-pkgs-dockerTools-shadowSetup"> - <title>shadowSetup</title> - - <para> - This constant string is a helper for setting up the base files for managing - users and groups, only if such files don't exist already. It is suitable - for being used in a <varname>runAsRoot</varname> - <xref linkend='ex-dockerTools-buildImage-runAsRoot'/> script for cases like - in the example below: - </para> - - <example xml:id='ex-dockerTools-shadowSetup'> - <title>Shadow base files</title> -<programlisting> -buildImage { - name = "shadow-basic"; - - runAsRoot = '' - #!${stdenv.shell} - ${shadowSetup} - groupadd -r redis - useradd -r -g redis redis - mkdir /data - chown redis:redis /data - ''; -} -</programlisting> - </example> - - <para> - Creating base files like <literal>/etc/passwd</literal> or - <literal>/etc/login.defs</literal> are necessary for shadow-utils to - manipulate users and groups. - </para> - </section> - </section> + <xi:include href="functions/overrides.xml" /> + <xi:include href="functions/generators.xml" /> + <xi:include href="functions/debug.xml" /> + <xi:include href="functions/fhs-environments.xml" /> + <xi:include href="functions/shell.xml" /> + <xi:include href="functions/dockertools.xml" /> </chapter> diff --git a/doc/functions/debug.xml b/doc/functions/debug.xml new file mode 100644 index 0000000000000..c6b3611eea53d --- /dev/null +++ b/doc/functions/debug.xml @@ -0,0 +1,21 @@ +<section xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + xml:id="sec-debug"> + <title>Debugging Nix Expressions</title> + + <para> + Nix is a unityped, dynamic language, this means every value can potentially + appear anywhere. Since it is also non-strict, evaluation order and what + ultimately is evaluated might surprise you. Therefore it is important to be + able to debug nix expressions. + </para> + + <para> + In the <literal>lib/debug.nix</literal> file you will find a number of + functions that help (pretty-)printing values while evaluation is runnnig. You + can even specify how deep these values should be printed recursively, and + transform them on the fly. Please consult the docstrings in + <literal>lib/debug.nix</literal> for usage information. + </para> +</section> diff --git a/doc/functions/dockertools.xml b/doc/functions/dockertools.xml new file mode 100644 index 0000000000000..501f46a967c37 --- /dev/null +++ b/doc/functions/dockertools.xml @@ -0,0 +1,564 @@ +<section xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + xml:id="sec-pkgs-dockerTools"> + <title>pkgs.dockerTools</title> + + <para> + <varname>pkgs.dockerTools</varname> is a set of functions for creating and + manipulating Docker images according to the + <link xlink:href="https://github.com/moby/moby/blob/master/image/spec/v1.2.md#docker-image-specification-v120"> + Docker Image Specification v1.2.0 </link>. Docker itself is not used to + perform any of the operations done by these functions. + </para> + + <warning> + <para> + The <varname>dockerTools</varname> API is unstable and may be subject to + backwards-incompatible changes in the future. + </para> + </warning> + + <section xml:id="ssec-pkgs-dockerTools-buildImage"> + <title>buildImage</title> + + <para> + This function is analogous to the <command>docker build</command> command, + in that can used to build a Docker-compatible repository tarball containing + a single image with one or multiple layers. As such, the result is suitable + for being loaded in Docker with <command>docker load</command>. + </para> + + <para> + The parameters of <varname>buildImage</varname> with relative example values + are described below: + </para> + + <example xml:id='ex-dockerTools-buildImage'> + <title>Docker build</title> +<programlisting> +buildImage { + name = "redis"; <co xml:id='ex-dockerTools-buildImage-1' /> + tag = "latest"; <co xml:id='ex-dockerTools-buildImage-2' /> + + fromImage = someBaseImage; <co xml:id='ex-dockerTools-buildImage-3' /> + fromImageName = null; <co xml:id='ex-dockerTools-buildImage-4' /> + fromImageTag = "latest"; <co xml:id='ex-dockerTools-buildImage-5' /> + + contents = pkgs.redis; <co xml:id='ex-dockerTools-buildImage-6' /> + runAsRoot = '' <co xml:id='ex-dockerTools-buildImage-runAsRoot' /> + #!${stdenv.shell} + mkdir -p /data + ''; + + config = { <co xml:id='ex-dockerTools-buildImage-8' /> + Cmd = [ "/bin/redis-server" ]; + WorkingDir = "/data"; + Volumes = { + "/data" = {}; + }; + }; +} +</programlisting> + </example> + + <para> + The above example will build a Docker image <literal>redis/latest</literal> + from the given base image. Loading and running this image in Docker results + in <literal>redis-server</literal> being started automatically. + </para> + + <calloutlist> + <callout arearefs='ex-dockerTools-buildImage-1'> + <para> + <varname>name</varname> specifies the name of the resulting image. This is + the only required argument for <varname>buildImage</varname>. + </para> + </callout> + <callout arearefs='ex-dockerTools-buildImage-2'> + <para> + <varname>tag</varname> specifies the tag of the resulting image. By + default it's <literal>null</literal>, which indicates that the nix output + hash will be used as tag. + </para> + </callout> + <callout arearefs='ex-dockerTools-buildImage-3'> + <para> + <varname>fromImage</varname> is the repository tarball containing the base + image. It must be a valid Docker image, such as exported by + <command>docker save</command>. By default it's <literal>null</literal>, + which can be seen as equivalent to <literal>FROM scratch</literal> of a + <filename>Dockerfile</filename>. + </para> + </callout> + <callout arearefs='ex-dockerTools-buildImage-4'> + <para> + <varname>fromImageName</varname> can be used to further specify the base + image within the repository, in case it contains multiple images. By + default it's <literal>null</literal>, in which case + <varname>buildImage</varname> will peek the first image available in the + repository. + </para> + </callout> + <callout arearefs='ex-dockerTools-buildImage-5'> + <para> + <varname>fromImageTag</varname> can be used to further specify the tag of + the base image within the repository, in case an image contains multiple + tags. By default it's <literal>null</literal>, in which case + <varname>buildImage</varname> will peek the first tag available for the + base image. + </para> + </callout> + <callout arearefs='ex-dockerTools-buildImage-6'> + <para> + <varname>contents</varname> is a derivation that will be copied in the new + layer of the resulting image. This can be similarly seen as <command>ADD + contents/ /</command> in a <filename>Dockerfile</filename>. By default + it's <literal>null</literal>. + </para> + </callout> + <callout arearefs='ex-dockerTools-buildImage-runAsRoot'> + <para> + <varname>runAsRoot</varname> is a bash script that will run as root in an + environment that overlays the existing layers of the base image with the + new resulting layer, including the previously copied + <varname>contents</varname> derivation. This can be similarly seen as + <command>RUN ...</command> in a <filename>Dockerfile</filename>. + <note> + <para> + Using this parameter requires the <literal>kvm</literal> device to be + available. + </para> + </note> + </para> + </callout> + <callout arearefs='ex-dockerTools-buildImage-8'> + <para> + <varname>config</varname> is used to specify the configuration of the + containers that will be started off the built image in Docker. The + available options are listed in the + <link xlink:href="https://github.com/moby/moby/blob/master/image/spec/v1.2.md#image-json-field-descriptions"> + Docker Image Specification v1.2.0 </link>. + </para> + </callout> + </calloutlist> + + <para> + After the new layer has been created, its closure (to which + <varname>contents</varname>, <varname>config</varname> and + <varname>runAsRoot</varname> contribute) will be copied in the layer itself. + Only new dependencies that are not already in the existing layers will be + copied. + </para> + + <para> + At the end of the process, only one new single layer will be produced and + added to the resulting image. + </para> + + <para> + The resulting repository will only list the single image + <varname>image/tag</varname>. In the case of + <xref linkend='ex-dockerTools-buildImage'/> it would be + <varname>redis/latest</varname>. + </para> + + <para> + It is possible to inspect the arguments with which an image was built using + its <varname>buildArgs</varname> attribute. + </para> + + <note> + <para> + If you see errors similar to <literal>getProtocolByName: does not exist (no + such protocol name: tcp)</literal> you may need to add + <literal>pkgs.iana-etc</literal> to <varname>contents</varname>. + </para> + </note> + + <note> + <para> + If you see errors similar to <literal>Error_Protocol ("certificate has + unknown CA",True,UnknownCa)</literal> you may need to add + <literal>pkgs.cacert</literal> to <varname>contents</varname>. + </para> + </note> + + <example xml:id="example-pkgs-dockerTools-buildImage-creation-date"> + <title>Impurely Defining a Docker Layer's Creation Date</title> + <para> + By default <function>buildImage</function> will use a static date of one + second past the UNIX Epoch. This allows <function>buildImage</function> to + produce binary reproducible images. When listing images with + <command>docker list images</command>, the newly created images will be + listed like this: + </para> +<screen><![CDATA[ +$ docker image list +REPOSITORY TAG IMAGE ID CREATED SIZE +hello latest 08c791c7846e 48 years ago 25.2MB +]]></screen> + <para> + You can break binary reproducibility but have a sorted, meaningful + <literal>CREATED</literal> column by setting <literal>created</literal> to + <literal>now</literal>. + </para> +<programlisting><![CDATA[ +pkgs.dockerTools.buildImage { + name = "hello"; + tag = "latest"; + created = "now"; + contents = pkgs.hello; + + config.Cmd = [ "/bin/hello" ]; +} +]]></programlisting> + <para> + and now the Docker CLI will display a reasonable date and sort the images + as expected: +<screen><![CDATA[ +$ docker image list +REPOSITORY TAG IMAGE ID CREATED SIZE +hello latest de2bf4786de6 About a minute ago 25.2MB +]]></screen> + however, the produced images will not be binary reproducible. + </para> + </example> + </section> + + <section xml:id="ssec-pkgs-dockerTools-buildLayeredImage"> + <title>buildLayeredImage</title> + + <para> + Create a Docker image with many of the store paths being on their own layer + to improve sharing between images. + </para> + + <variablelist> + <varlistentry> + <term> + <varname>name</varname> + </term> + <listitem> + <para> + The name of the resulting image. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <varname>tag</varname> <emphasis>optional</emphasis> + </term> + <listitem> + <para> + Tag of the generated image. + </para> + <para> + <emphasis>Default:</emphasis> the output path's hash + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <varname>contents</varname> <emphasis>optional</emphasis> + </term> + <listitem> + <para> + Top level paths in the container. Either a single derivation, or a list + of derivations. + </para> + <para> + <emphasis>Default:</emphasis> <literal>[]</literal> + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <varname>config</varname> <emphasis>optional</emphasis> + </term> + <listitem> + <para> + Run-time configuration of the container. A full list of the options are + available at in the + <link xlink:href="https://github.com/moby/moby/blob/master/image/spec/v1.2.md#image-json-field-descriptions"> + Docker Image Specification v1.2.0 </link>. + </para> + <para> + <emphasis>Default:</emphasis> <literal>{}</literal> + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <varname>created</varname> <emphasis>optional</emphasis> + </term> + <listitem> + <para> + Date and time the layers were created. Follows the same + <literal>now</literal> exception supported by + <literal>buildImage</literal>. + </para> + <para> + <emphasis>Default:</emphasis> <literal>1970-01-01T00:00:01Z</literal> + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <varname>maxLayers</varname> <emphasis>optional</emphasis> + </term> + <listitem> + <para> + Maximum number of layers to create. + </para> + <para> + <emphasis>Default:</emphasis> <literal>24</literal> + </para> + </listitem> + </varlistentry> + </variablelist> + + <section xml:id="dockerTools-buildLayeredImage-arg-contents"> + <title>Behavior of <varname>contents</varname> in the final image</title> + + <para> + Each path directly listed in <varname>contents</varname> will have a + symlink in the root of the image. + </para> + + <para> + For example: +<programlisting><![CDATA[ +pkgs.dockerTools.buildLayeredImage { + name = "hello"; + contents = [ pkgs.hello ]; +} +]]></programlisting> + will create symlinks for all the paths in the <literal>hello</literal> + package: +<screen><![CDATA[ +/bin/hello -> /nix/store/h1zb1padqbbb7jicsvkmrym3r6snphxg-hello-2.10/bin/hello +/share/info/hello.info -> /nix/store/h1zb1padqbbb7jicsvkmrym3r6snphxg-hello-2.10/share/info/hello.info +/share/locale/bg/LC_MESSAGES/hello.mo -> /nix/store/h1zb1padqbbb7jicsvkmrym3r6snphxg-hello-2.10/share/locale/bg/LC_MESSAGES/hello.mo +]]></screen> + </para> + </section> + + <section xml:id="dockerTools-buildLayeredImage-arg-config"> + <title>Automatic inclusion of <varname>config</varname> references</title> + + <para> + The closure of <varname>config</varname> is automatically included in the + closure of the final image. + </para> + + <para> + This allows you to make very simple Docker images with very little code. + This container will start up and run <command>hello</command>: +<programlisting><![CDATA[ +pkgs.dockerTools.buildLayeredImage { + name = "hello"; + config.Cmd = [ "${pkgs.hello}/bin/hello" ]; +} +]]></programlisting> + </para> + </section> + + <section xml:id="dockerTools-buildLayeredImage-arg-maxLayers"> + <title>Adjusting <varname>maxLayers</varname></title> + + <para> + Increasing the <varname>maxLayers</varname> increases the number of layers + which have a chance to be shared between different images. + </para> + + <para> + Modern Docker installations support up to 128 layers, however older + versions support as few as 42. + </para> + + <para> + If the produced image will not be extended by other Docker builds, it is + safe to set <varname>maxLayers</varname> to <literal>128</literal>. However + it will be impossible to extend the image further. + </para> + + <para> + The first (<literal>maxLayers-2</literal>) most "popular" paths will have + their own individual layers, then layer #<literal>maxLayers-1</literal> + will contain all the remaining "unpopular" paths, and finally layer + #<literal>maxLayers</literal> will contain the Image configuration. + </para> + + <para> + Docker's Layers are not inherently ordered, they are content-addressable + and are not explicitly layered until they are composed in to an Image. + </para> + </section> + </section> + + <section xml:id="ssec-pkgs-dockerTools-fetchFromRegistry"> + <title>pullImage</title> + + <para> + This function is analogous to the <command>docker pull</command> command, in + that can be used to pull a Docker image from a Docker registry. By default + <link xlink:href="https://hub.docker.com/">Docker Hub</link> is used to pull + images. + </para> + + <para> + Its parameters are described in the example below: + </para> + + <example xml:id='ex-dockerTools-pullImage'> + <title>Docker pull</title> +<programlisting> +pullImage { + imageName = "nixos/nix"; <co xml:id='ex-dockerTools-pullImage-1' /> + imageDigest = "sha256:20d9485b25ecfd89204e843a962c1bd70e9cc6858d65d7f5fadc340246e2116b"; <co xml:id='ex-dockerTools-pullImage-2' /> + finalImageTag = "1.11"; <co xml:id='ex-dockerTools-pullImage-3' /> + sha256 = "0mqjy3zq2v6rrhizgb9nvhczl87lcfphq9601wcprdika2jz7qh8"; <co xml:id='ex-dockerTools-pullImage-4' /> + os = "linux"; <co xml:id='ex-dockerTools-pullImage-5' /> + arch = "x86_64"; <co xml:id='ex-dockerTools-pullImage-6' /> +} +</programlisting> + </example> + + <calloutlist> + <callout arearefs='ex-dockerTools-pullImage-1'> + <para> + <varname>imageName</varname> specifies the name of the image to be + downloaded, which can also include the registry namespace (e.g. + <literal>nixos</literal>). This argument is required. + </para> + </callout> + <callout arearefs='ex-dockerTools-pullImage-2'> + <para> + <varname>imageDigest</varname> specifies the digest of the image to be + downloaded. Skopeo can be used to get the digest of an image, with its + <varname>inspect</varname> subcommand. Since a given + <varname>imageName</varname> may transparently refer to a manifest list of + images which support multiple architectures and/or operating systems, + supply the `--override-os` and `--override-arch` arguments to specify + exactly which image you want. By default it will match the OS and + architecture of the host the command is run on. +<programlisting> +$ nix-shell --packages skopeo jq --command "skopeo --override-os linux --override-arch x86_64 inspect docker://docker.io/nixos/nix:1.11 | jq -r '.Digest'" +sha256:20d9485b25ecfd89204e843a962c1bd70e9cc6858d65d7f5fadc340246e2116b +</programlisting> + This argument is required. + </para> + </callout> + <callout arearefs='ex-dockerTools-pullImage-3'> + <para> + <varname>finalImageTag</varname>, if specified, this is the tag of the + image to be created. Note it is never used to fetch the image since we + prefer to rely on the immutable digest ID. By default it's + <literal>latest</literal>. + </para> + </callout> + <callout arearefs='ex-dockerTools-pullImage-4'> + <para> + <varname>sha256</varname> is the checksum of the whole fetched image. This + argument is required. + </para> + </callout> + <callout arearefs='ex-dockerTools-pullImage-5'> + <para> + <varname>os</varname>, if specified, is the operating system of the + fetched image. By default it's <literal>linux</literal>. + </para> + </callout> + <callout arearefs='ex-dockerTools-pullImage-6'> + <para> + <varname>arch</varname>, if specified, is the cpu architecture of the + fetched image. By default it's <literal>x86_64</literal>. + </para> + </callout> + </calloutlist> + </section> + + <section xml:id="ssec-pkgs-dockerTools-exportImage"> + <title>exportImage</title> + + <para> + This function is analogous to the <command>docker export</command> command, + in that can used to flatten a Docker image that contains multiple layers. It + is in fact the result of the merge of all the layers of the image. As such, + the result is suitable for being imported in Docker with <command>docker + import</command>. + </para> + + <note> + <para> + Using this function requires the <literal>kvm</literal> device to be + available. + </para> + </note> + + <para> + The parameters of <varname>exportImage</varname> are the following: + </para> + + <example xml:id='ex-dockerTools-exportImage'> + <title>Docker export</title> +<programlisting> +exportImage { + fromImage = someLayeredImage; + fromImageName = null; + fromImageTag = null; + + name = someLayeredImage.name; +} + </programlisting> + </example> + + <para> + The parameters relative to the base image have the same synopsis as + described in <xref linkend='ssec-pkgs-dockerTools-buildImage'/>, except that + <varname>fromImage</varname> is the only required argument in this case. + </para> + + <para> + The <varname>name</varname> argument is the name of the derivation output, + which defaults to <varname>fromImage.name</varname>. + </para> + </section> + + <section xml:id="ssec-pkgs-dockerTools-shadowSetup"> + <title>shadowSetup</title> + + <para> + This constant string is a helper for setting up the base files for managing + users and groups, only if such files don't exist already. It is suitable for + being used in a <varname>runAsRoot</varname> + <xref linkend='ex-dockerTools-buildImage-runAsRoot'/> script for cases like + in the example below: + </para> + + <example xml:id='ex-dockerTools-shadowSetup'> + <title>Shadow base files</title> +<programlisting> +buildImage { + name = "shadow-basic"; + + runAsRoot = '' + #!${stdenv.shell} + ${shadowSetup} + groupadd -r redis + useradd -r -g redis redis + mkdir /data + chown redis:redis /data + ''; +} +</programlisting> + </example> + + <para> + Creating base files like <literal>/etc/passwd</literal> or + <literal>/etc/login.defs</literal> are necessary for shadow-utils to + manipulate users and groups. + </para> + </section> +</section> diff --git a/doc/functions/fhs-environments.xml b/doc/functions/fhs-environments.xml new file mode 100644 index 0000000000000..79682080be314 --- /dev/null +++ b/doc/functions/fhs-environments.xml @@ -0,0 +1,142 @@ +<section xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + xml:id="sec-fhs-environments"> + <title>buildFHSUserEnv</title> + + <para> + <function>buildFHSUserEnv</function> provides a way to build and run + FHS-compatible lightweight sandboxes. It creates an isolated root with bound + <filename>/nix/store</filename>, so its footprint in terms of disk space + needed is quite small. This allows one to run software which is hard or + unfeasible to patch for NixOS -- 3rd-party source trees with FHS assumptions, + games distributed as tarballs, software with integrity checking and/or + external self-updated binaries. It uses Linux namespaces feature to create + temporary lightweight environments which are destroyed after all child + processes exit, without root user rights requirement. Accepted arguments are: + </para> + + <variablelist> + <varlistentry> + <term> + <literal>name</literal> + </term> + <listitem> + <para> + Environment name. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>targetPkgs</literal> + </term> + <listitem> + <para> + Packages to be installed for the main host's architecture (i.e. x86_64 on + x86_64 installations). Along with libraries binaries are also installed. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>multiPkgs</literal> + </term> + <listitem> + <para> + Packages to be installed for all architectures supported by a host (i.e. + i686 and x86_64 on x86_64 installations). Only libraries are installed by + default. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>extraBuildCommands</literal> + </term> + <listitem> + <para> + Additional commands to be executed for finalizing the directory structure. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>extraBuildCommandsMulti</literal> + </term> + <listitem> + <para> + Like <literal>extraBuildCommands</literal>, but executed only on multilib + architectures. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>extraOutputsToInstall</literal> + </term> + <listitem> + <para> + Additional derivation outputs to be linked for both target and + multi-architecture packages. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>extraInstallCommands</literal> + </term> + <listitem> + <para> + Additional commands to be executed for finalizing the derivation with + runner script. + </para> + </listitem> + </varlistentry> + <varlistentry> + <term> + <literal>runScript</literal> + </term> + <listitem> + <para> + A command that would be executed inside the sandbox and passed all the + command line arguments. It defaults to <literal>bash</literal>. + </para> + </listitem> + </varlistentry> + </variablelist> + + <para> + One can create a simple environment using a <literal>shell.nix</literal> like + that: + </para> + +<programlisting><![CDATA[ +{ pkgs ? import <nixpkgs> {} }: + +(pkgs.buildFHSUserEnv { + name = "simple-x11-env"; + targetPkgs = pkgs: (with pkgs; + [ udev + alsaLib + ]) ++ (with pkgs.xorg; + [ libX11 + libXcursor + libXrandr + ]); + multiPkgs = pkgs: (with pkgs; + [ udev + alsaLib + ]); + runScript = "bash"; +}).env +]]></programlisting> + + <para> + Running <literal>nix-shell</literal> would then drop you into a shell with + these libraries and binaries available. You can use this to run closed-source + applications which expect FHS structure without hassles: simply change + <literal>runScript</literal> to the application path, e.g. + <filename>./bin/start.sh</filename> -- relative paths are supported. + </para> +</section> diff --git a/doc/functions/generators.xml b/doc/functions/generators.xml new file mode 100644 index 0000000000000..e860b10e8979f --- /dev/null +++ b/doc/functions/generators.xml @@ -0,0 +1,89 @@ +<section xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + xml:id="sec-generators"> + <title>Generators</title> + + <para> + Generators are functions that create file formats from nix data structures, + e. g. for configuration files. There are generators available for: + <literal>INI</literal>, <literal>JSON</literal> and <literal>YAML</literal> + </para> + + <para> + All generators follow a similar call interface: <code>generatorName + configFunctions data</code>, where <literal>configFunctions</literal> is an + attrset of user-defined functions that format nested parts of the content. + They each have common defaults, so often they do not need to be set manually. + An example is <code>mkSectionName ? (name: libStr.escape [ "[" "]" ] + name)</code> from the <literal>INI</literal> generator. It receives the name + of a section and sanitizes it. The default <literal>mkSectionName</literal> + escapes <literal>[</literal> and <literal>]</literal> with a backslash. + </para> + + <para> + Generators can be fine-tuned to produce exactly the file format required by + your application/service. One example is an INI-file format which uses + <literal>: </literal> as separator, the strings + <literal>"yes"</literal>/<literal>"no"</literal> as boolean values and + requires all string values to be quoted: + </para> + +<programlisting> +with lib; +let + customToINI = generators.toINI { + # specifies how to format a key/value pair + mkKeyValue = generators.mkKeyValueDefault { + # specifies the generated string for a subset of nix values + mkValueString = v: + if v == true then ''"yes"'' + else if v == false then ''"no"'' + else if isString v then ''"${v}"'' + # and delegats all other values to the default generator + else generators.mkValueStringDefault {} v; + } ":"; + }; + +# the INI file can now be given as plain old nix values +in customToINI { + main = { + pushinfo = true; + autopush = false; + host = "localhost"; + port = 42; + }; + mergetool = { + merge = "diff3"; + }; +} +</programlisting> + + <para> + This will produce the following INI file as nix string: + </para> + +<programlisting> +[main] +autopush:"no" +host:"localhost" +port:42 +pushinfo:"yes" +str\:ange:"very::strange" + +[mergetool] +merge:"diff3" +</programlisting> + + <note> + <para> + Nix store paths can be converted to strings by enclosing a derivation + attribute like so: <code>"${drv}"</code>. + </para> + </note> + + <para> + Detailed documentation for each generator can be found in + <literal>lib/generators.nix</literal>. + </para> +</section> diff --git a/doc/functions/overrides.xml b/doc/functions/overrides.xml new file mode 100644 index 0000000000000..99e2a63631a77 --- /dev/null +++ b/doc/functions/overrides.xml @@ -0,0 +1,203 @@ +<section xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + xml:id="sec-overrides"> + <title>Overriding</title> + + <para> + Sometimes one wants to override parts of <literal>nixpkgs</literal>, e.g. + derivation attributes, the results of derivations or even the whole package + set. + </para> + + <section xml:id="sec-pkg-override"> + <title><pkg>.override</title> + + <para> + The function <varname>override</varname> is usually available for all the + derivations in the nixpkgs expression (<varname>pkgs</varname>). + </para> + + <para> + It is used to override the arguments passed to a function. + </para> + + <para> + Example usages: +<programlisting>pkgs.foo.override { arg1 = val1; arg2 = val2; ... }</programlisting> +<programlisting> +import pkgs.path { overlays = [ (self: super: { + foo = super.foo.override { barSupport = true ; }; + })]}; +</programlisting> +<programlisting> +mypkg = pkgs.callPackage ./mypkg.nix { + mydep = pkgs.mydep.override { ... }; + } +</programlisting> + </para> + + <para> + In the first example, <varname>pkgs.foo</varname> is the result of a + function call with some default arguments, usually a derivation. Using + <varname>pkgs.foo.override</varname> will call the same function with the + given new arguments. + </para> + </section> + + <section xml:id="sec-pkg-overrideAttrs"> + <title><pkg>.overrideAttrs</title> + + <para> + The function <varname>overrideAttrs</varname> allows overriding the + attribute set passed to a <varname>stdenv.mkDerivation</varname> call, + producing a new derivation based on the original one. This function is + available on all derivations produced by the + <varname>stdenv.mkDerivation</varname> function, which is most packages in + the nixpkgs expression <varname>pkgs</varname>. + </para> + + <para> + Example usage: +<programlisting> +helloWithDebug = pkgs.hello.overrideAttrs (oldAttrs: rec { + separateDebugInfo = true; +}); +</programlisting> + </para> + + <para> + In the above example, the <varname>separateDebugInfo</varname> attribute is + overridden to be true, thus building debug info for + <varname>helloWithDebug</varname>, while all other attributes will be + retained from the original <varname>hello</varname> package. + </para> + + <para> + The argument <varname>oldAttrs</varname> is conventionally used to refer to + the attr set originally passed to <varname>stdenv.mkDerivation</varname>. + </para> + + <note> + <para> + Note that <varname>separateDebugInfo</varname> is processed only by the + <varname>stdenv.mkDerivation</varname> function, not the generated, raw Nix + derivation. Thus, using <varname>overrideDerivation</varname> will not work + in this case, as it overrides only the attributes of the final derivation. + It is for this reason that <varname>overrideAttrs</varname> should be + preferred in (almost) all cases to <varname>overrideDerivation</varname>, + i.e. to allow using <varname>sdenv.mkDerivation</varname> to process input + arguments, as well as the fact that it is easier to use (you can use the + same attribute names you see in your Nix code, instead of the ones + generated (e.g. <varname>buildInputs</varname> vs + <varname>nativeBuildInputs</varname>, and involves less typing. + </para> + </note> + </section> + + <section xml:id="sec-pkg-overrideDerivation"> + <title><pkg>.overrideDerivation</title> + + <warning> + <para> + You should prefer <varname>overrideAttrs</varname> in almost all cases, see + its documentation for the reasons why. + <varname>overrideDerivation</varname> is not deprecated and will continue + to work, but is less nice to use and does not have as many abilities as + <varname>overrideAttrs</varname>. + </para> + </warning> + + <warning> + <para> + Do not use this function in Nixpkgs as it evaluates a Derivation before + modifying it, which breaks package abstraction and removes error-checking + of function arguments. In addition, this evaluation-per-function + application incurs a performance penalty, which can become a problem if + many overrides are used. It is only intended for ad-hoc customisation, such + as in <filename>~/.config/nixpkgs/config.nix</filename>. + </para> + </warning> + + <para> + The function <varname>overrideDerivation</varname> creates a new derivation + based on an existing one by overriding the original's attributes with the + attribute set produced by the specified function. This function is available + on all derivations defined using the <varname>makeOverridable</varname> + function. Most standard derivation-producing functions, such as + <varname>stdenv.mkDerivation</varname>, are defined using this function, + which means most packages in the nixpkgs expression, + <varname>pkgs</varname>, have this function. + </para> + + <para> + Example usage: +<programlisting> +mySed = pkgs.gnused.overrideDerivation (oldAttrs: { + name = "sed-4.2.2-pre"; + src = fetchurl { + url = ftp://alpha.gnu.org/gnu/sed/sed-4.2.2-pre.tar.bz2; + sha256 = "11nq06d131y4wmf3drm0yk502d2xc6n5qy82cg88rb9nqd2lj41k"; + }; + patches = []; +}); +</programlisting> + </para> + + <para> + In the above example, the <varname>name</varname>, <varname>src</varname>, + and <varname>patches</varname> of the derivation will be overridden, while + all other attributes will be retained from the original derivation. + </para> + + <para> + The argument <varname>oldAttrs</varname> is used to refer to the attribute + set of the original derivation. + </para> + + <note> + <para> + A package's attributes are evaluated *before* being modified by the + <varname>overrideDerivation</varname> function. For example, the + <varname>name</varname> attribute reference in <varname>url = + "mirror://gnu/hello/${name}.tar.gz";</varname> is filled-in *before* the + <varname>overrideDerivation</varname> function modifies the attribute set. + This means that overriding the <varname>name</varname> attribute, in this + example, *will not* change the value of the <varname>url</varname> + attribute. Instead, we need to override both the <varname>name</varname> + *and* <varname>url</varname> attributes. + </para> + </note> + </section> + + <section xml:id="sec-lib-makeOverridable"> + <title>lib.makeOverridable</title> + + <para> + The function <varname>lib.makeOverridable</varname> is used to make the + result of a function easily customizable. This utility only makes sense for + functions that accept an argument set and return an attribute set. + </para> + + <para> + Example usage: +<programlisting> +f = { a, b }: { result = a+b; }; +c = lib.makeOverridable f { a = 1; b = 2; }; +</programlisting> + </para> + + <para> + The variable <varname>c</varname> is the value of the <varname>f</varname> + function applied with some default arguments. Hence the value of + <varname>c.result</varname> is <literal>3</literal>, in this example. + </para> + + <para> + The variable <varname>c</varname> however also has some additional + functions, like <link linkend="sec-pkg-override">c.override</link> which can + be used to override the default arguments. In this example the value of + <varname>(c.override { a = 4; }).result</varname> is 6. + </para> + </section> +</section> diff --git a/doc/functions/shell.xml b/doc/functions/shell.xml new file mode 100644 index 0000000000000..e5031c9463c06 --- /dev/null +++ b/doc/functions/shell.xml @@ -0,0 +1,26 @@ +<section xmlns="http://docbook.org/ns/docbook" + xmlns:xlink="http://www.w3.org/1999/xlink" + xmlns:xi="http://www.w3.org/2001/XInclude" + xml:id="sec-pkgs-mkShell"> + <title>pkgs.mkShell</title> + + <para> + <function>pkgs.mkShell</function> is a special kind of derivation that is + only useful when using it combined with <command>nix-shell</command>. It will + in fact fail to instantiate when invoked with <command>nix-build</command>. + </para> + + <section xml:id="sec-pkgs-mkShell-usage"> + <title>Usage</title> + +<programlisting><![CDATA[ +{ pkgs ? import <nixpkgs> {} }: +pkgs.mkShell { + # this will make all the build inputs from hello and gnutar + # available to the shell environment + inputsFrom = with pkgs; [ hello gnutar ]; + buildInputs = [ pkgs.gnumake ]; +} +]]></programlisting> + </section> +</section> diff --git a/doc/package-notes.xml b/doc/package-notes.xml index a4322a0234d3c..0543e06a05d49 100644 --- a/doc/package-notes.xml +++ b/doc/package-notes.xml @@ -668,8 +668,9 @@ overrides = self: super: rec { plugins = with availablePlugins; [ python perl ]; } }</programlisting> - If the <literal>configure</literal> function returns an attrset without the <literal>plugins</literal> - attribute, <literal>availablePlugins</literal> will be used automatically. + If the <literal>configure</literal> function returns an attrset without the + <literal>plugins</literal> attribute, <literal>availablePlugins</literal> + will be used automatically. </para> <para> @@ -703,9 +704,11 @@ overrides = self: super: rec { }; } </programlisting> </para> + <para> - WeeChat allows to set defaults on startup using the <literal>--run-command</literal>. - The <literal>configure</literal> method can be used to pass commands to the program: + WeeChat allows to set defaults on startup using the + <literal>--run-command</literal>. The <literal>configure</literal> method + can be used to pass commands to the program: <programlisting>weechat.override { configure = { availablePlugins, ... }: { init = '' @@ -714,12 +717,14 @@ overrides = self: super: rec { ''; }; }</programlisting> - Further values can be added to the list of commands when running - <literal>weechat --run-command "your-commands"</literal>. + Further values can be added to the list of commands when running + <literal>weechat --run-command "your-commands"</literal>. </para> + <para> - Additionally it's possible to specify scripts to be loaded when starting <literal>weechat</literal>. - These will be loaded before the commands from <literal>init</literal>: + Additionally it's possible to specify scripts to be loaded when starting + <literal>weechat</literal>. These will be loaded before the commands from + <literal>init</literal>: <programlisting>weechat.override { configure = { availablePlugins, ... }: { scripts = with pkgs.weechatScripts; [ @@ -731,11 +736,13 @@ overrides = self: super: rec { }; }</programlisting> </para> + <para> - In <literal>nixpkgs</literal> there's a subpackage which contains derivations for - WeeChat scripts. Such derivations expect a <literal>passthru.scripts</literal> attribute - which contains a list of all scripts inside the store path. Furthermore all scripts - have to live in <literal>$out/share</literal>. An exemplary derivation looks like this: + In <literal>nixpkgs</literal> there's a subpackage which contains + derivations for WeeChat scripts. Such derivations expect a + <literal>passthru.scripts</literal> attribute which contains a list of all + scripts inside the store path. Furthermore all scripts have to live in + <literal>$out/share</literal>. An exemplary derivation looks like this: <programlisting>{ stdenv, fetchurl }: stdenv.mkDerivation { @@ -814,20 +821,26 @@ citrix_receiver.override { <section xml:id="sec-ibus-typing-booster"> <title>ibus-engines.typing-booster</title> - <para>This package is an ibus-based completion method to speed up typing.</para> + <para> + This package is an ibus-based completion method to speed up typing. + </para> <section xml:id="sec-ibus-typing-booster-activate"> <title>Activating the engine</title> <para> - IBus needs to be configured accordingly to activate <literal>typing-booster</literal>. The configuration - depends on the desktop manager in use. For detailed instructions, please refer to the - <link xlink:href="https://mike-fabian.github.io/ibus-typing-booster/documentation.html">upstream docs</link>. + IBus needs to be configured accordingly to activate + <literal>typing-booster</literal>. The configuration depends on the desktop + manager in use. For detailed instructions, please refer to the + <link xlink:href="https://mike-fabian.github.io/ibus-typing-booster/documentation.html">upstream + docs</link>. </para> + <para> - On NixOS you need to explicitly enable <literal>ibus</literal> with given engines - before customizing your desktop to use <literal>typing-booster</literal>. This can be achieved - using the <literal>ibus</literal> module: + On NixOS you need to explicitly enable <literal>ibus</literal> with given + engines before customizing your desktop to use + <literal>typing-booster</literal>. This can be achieved using the + <literal>ibus</literal> module: <programlisting>{ pkgs, ... }: { i18n.inputMethod = { enabled = "ibus"; @@ -841,17 +854,20 @@ citrix_receiver.override { <title>Using custom hunspell dictionaries</title> <para> - The IBus engine is based on <literal>hunspell</literal> to support completion in many languages. - By default the dictionaries <literal>de-de</literal>, <literal>en-us</literal>, <literal>es-es</literal>, - <literal>it-it</literal>, <literal>sv-se</literal> and <literal>sv-fi</literal> - are in use. To add another dictionary, the package can be overridden like this: + The IBus engine is based on <literal>hunspell</literal> to support + completion in many languages. By default the dictionaries + <literal>de-de</literal>, <literal>en-us</literal>, + <literal>es-es</literal>, <literal>it-it</literal>, + <literal>sv-se</literal> and <literal>sv-fi</literal> are in use. To add + another dictionary, the package can be overridden like this: <programlisting>ibus-engines.typing-booster.override { langs = [ "de-at" "en-gb" ]; }</programlisting> </para> + <para> - <emphasis>Note: each language passed to <literal>langs</literal> must be an attribute name in - <literal>pkgs.hunspellDicts</literal>.</emphasis> + <emphasis>Note: each language passed to <literal>langs</literal> must be an + attribute name in <literal>pkgs.hunspellDicts</literal>.</emphasis> </para> </section> @@ -859,10 +875,12 @@ citrix_receiver.override { <title>Built-in emoji picker</title> <para> - The <literal>ibus-engines.typing-booster</literal> package contains a program - named <literal>emoji-picker</literal>. To display all emojis correctly, - a special font such as <literal>noto-fonts-emoji</literal> is needed: + The <literal>ibus-engines.typing-booster</literal> package contains a + program named <literal>emoji-picker</literal>. To display all emojis + correctly, a special font such as <literal>noto-fonts-emoji</literal> is + needed: </para> + <para> On NixOS it can be installed using the following expression: <programlisting>{ pkgs, ... }: { diff --git a/doc/shell.section.md b/doc/shell.section.md deleted file mode 100644 index cb8832a814fcc..0000000000000 --- a/doc/shell.section.md +++ /dev/null @@ -1,22 +0,0 @@ ---- -title: pkgs.mkShell -author: zimbatm -date: 2017-10-30 ---- - -# mkShell - -pkgs.mkShell is a special kind of derivation that is only useful when using -it combined with nix-shell. It will in fact fail to instantiate when invoked -with nix-build. - -## Usage - -```nix -{ pkgs ? import <nixpkgs> {} }: -pkgs.mkShell { - # this will make all the build inputs from hello and gnutar available to the shell environment - inputsFrom = with pkgs; [ hello gnutar ]; - buildInputs = [ pkgs.gnumake ]; -} -``` |