diff options
author | John Ericson <John.Ericson@Obsidian.Systems> | 2020-11-28 18:10:38 +0000 |
---|---|---|
committer | John Ericson <John.Ericson@Obsidian.Systems> | 2020-11-28 18:10:38 +0000 |
commit | 8ddf5c69077a6afda88a3ae72f10fdff031f75b8 (patch) | |
tree | b18dd7d549baa53756eb3c53d0f2932feab52e91 /doc/languages-frameworks | |
parent | c6617d28ef3762bbd5cb11dd3c56afb778ff42cc (diff) | |
parent | 2622548c138fbf151fd3f130fe41864590520121 (diff) |
Merge remote-tracking branch 'upstream/master' into aj-rust-custom-target
Diffstat (limited to 'doc/languages-frameworks')
-rw-r--r-- | doc/languages-frameworks/beam.section.md | 84 | ||||
-rw-r--r-- | doc/languages-frameworks/beam.xml | 159 | ||||
-rw-r--r-- | doc/languages-frameworks/dotnet.section.md | 4 | ||||
-rw-r--r-- | doc/languages-frameworks/go.section.md | 140 | ||||
-rw-r--r-- | doc/languages-frameworks/go.xml | 248 | ||||
-rw-r--r-- | doc/languages-frameworks/index.xml | 7 | ||||
-rw-r--r-- | doc/languages-frameworks/java.xml | 18 | ||||
-rw-r--r-- | doc/languages-frameworks/maven.section.md | 354 | ||||
-rw-r--r-- | doc/languages-frameworks/python.section.md | 2 | ||||
-rw-r--r-- | doc/languages-frameworks/ruby.section.md | 221 | ||||
-rw-r--r-- | doc/languages-frameworks/ruby.xml | 107 | ||||
-rw-r--r-- | doc/languages-frameworks/rust.section.md | 51 |
12 files changed, 707 insertions, 688 deletions
diff --git a/doc/languages-frameworks/beam.section.md b/doc/languages-frameworks/beam.section.md new file mode 100644 index 0000000000000..ad3b94880b5c6 --- /dev/null +++ b/doc/languages-frameworks/beam.section.md @@ -0,0 +1,84 @@ +# BEAM Languages (Erlang, Elixir & LFE) {#sec-beam} + +## Introduction {#beam-introduction} + +In this document and related Nix expressions, we use the term, *BEAM*, to describe the environment. BEAM is the name of the Erlang Virtual Machine and, as far as we're concerned, from a packaging perspective, all languages that run on the BEAM are interchangeable. That which varies, like the build system, is transparent to users of any given BEAM package, so we make no distinction. + +## Structure {#beam-structure} + +All BEAM-related expressions are available via the top-level `beam` attribute, which includes: + + - `interpreters`: a set of compilers running on the BEAM, including multiple Erlang/OTP versions (`beam.interpreters.erlangR19`, etc), Elixir (`beam.interpreters.elixir`) and LFE (`beam.interpreters.lfe`). + + - `packages`: a set of package builders (Mix and rebar3), each compiled with a specific Erlang/OTP version, e.g. `beam.packages.erlangR19`. + +The default Erlang compiler, defined by `beam.interpreters.erlang`, is aliased as `erlang`. The default BEAM package set is defined by `beam.packages.erlang` and aliased at the top level as `beamPackages`. + +To create a package builder built with a custom Erlang version, use the lambda, `beam.packagesWith`, which accepts an Erlang/OTP derivation and produces a package builder similar to `beam.packages.erlang`. + +Many Erlang/OTP distributions available in `beam.interpreters` have versions with ODBC and/or Java enabled or without wx (no observer support). For example, there's `beam.interpreters.erlangR22_odbc_javac`, which corresponds to `beam.interpreters.erlangR22` and `beam.interpreters.erlangR22_nox`, which corresponds to `beam.interpreters.erlangR22`. + +## Build Tools {#build-tools} + +### Rebar3 {#build-tools-rebar3} + +We provide a version of Rebar3, under `rebar3`. We also provide a helper to fetch Rebar3 dependencies from a lockfile under `fetchRebar3Deps`. + +### Mix & Erlang.mk {#build-tools-other} + +Both Mix and Erlang.mk work exactly as expected. There is a bootstrap process that needs to be run for both, however, which is supported by the `buildMix` and `buildErlangMk` derivations, respectively. + +## How to Install BEAM Packages {#how-to-install-beam-packages} + +BEAM builders are not registered at the top level, simply because they are not relevant to the vast majority of Nix users. To install any of those builders into your profile, refer to them by their attribute path `beamPackages.rebar3`: + +```ShellSession +$ nix-env -f "<nixpkgs>" -iA beamPackages.rebar3 +``` + +## Packaging BEAM Applications {#packaging-beam-applications} + +### Erlang Applications {#packaging-erlang-applications} + +#### Rebar3 Packages {#rebar3-packages} + +The Nix function, `buildRebar3`, defined in `beam.packages.erlang.buildRebar3` and aliased at the top level, can be used to build a derivation that understands how to build a Rebar3 project. + +If a package needs to compile native code via Rebar3's port compilation mechanism, add `compilePort = true;` to the derivation. + +#### Erlang.mk Packages {#erlang-mk-packages} + +Erlang.mk functions similarly to Rebar3, except we use `buildErlangMk` instead of `buildRebar3`. + +#### Mix Packages {#mix-packages} + +Mix functions similarly to Rebar3, except we use `buildMix` instead of `buildRebar3`. + +Alternatively, we can use `buildHex` as a shortcut: + +## How to Develop {#how-to-develop} + +### Creating a Shell {#creating-a-shell} + +Usually, we need to create a `shell.nix` file and do our development inside of the environment specified therein. Just install your version of erlang and other interpreter, and then user your normal build tools. As an example with elixir: + +```nix +{ pkgs ? import "<nixpkgs"> {} }: + +with pkgs; + +let + + elixir = beam.packages.erlangR22.elixir_1_9; + +in +mkShell { + buildInputs = [ elixir ]; + + ERL_INCLUDE_PATH="${erlang}/lib/erlang/usr/include"; +} +``` + +#### Building in a Shell (for Mix Projects) {#building-in-a-shell} + +Using a `shell.nix` as described (see <xref linkend="creating-a-shell"/>) should just work. diff --git a/doc/languages-frameworks/beam.xml b/doc/languages-frameworks/beam.xml deleted file mode 100644 index addab24f7f6dd..0000000000000 --- a/doc/languages-frameworks/beam.xml +++ /dev/null @@ -1,159 +0,0 @@ -<section xmlns="http://docbook.org/ns/docbook" - xmlns:xlink="http://www.w3.org/1999/xlink" - xml:id="sec-beam"> - <title>BEAM Languages (Erlang, Elixir & LFE)</title> - - <section xml:id="beam-introduction"> - <title>Introduction</title> - - <para> - In this document and related Nix expressions, we use the term, <emphasis>BEAM</emphasis>, to describe the environment. BEAM is the name of the Erlang Virtual Machine and, as far as we're concerned, from a packaging perspective, all languages that run on the BEAM are interchangeable. That which varies, like the build system, is transparent to users of any given BEAM package, so we make no distinction. - </para> - </section> - - <section xml:id="beam-structure"> - <title>Structure</title> - - <para> - All BEAM-related expressions are available via the top-level <literal>beam</literal> attribute, which includes: - </para> - - <itemizedlist> - <listitem> - <para> - <literal>interpreters</literal>: a set of compilers running on the BEAM, including multiple Erlang/OTP versions (<literal>beam.interpreters.erlangR19</literal>, etc), Elixir (<literal>beam.interpreters.elixir</literal>) and LFE (<literal>beam.interpreters.lfe</literal>). - </para> - </listitem> - <listitem> - <para> - <literal>packages</literal>: a set of package builders (Mix and rebar3), each compiled with a specific Erlang/OTP version, e.g. <literal>beam.packages.erlangR19</literal>. - </para> - </listitem> - </itemizedlist> - - <para> - The default Erlang compiler, defined by <literal>beam.interpreters.erlang</literal>, is aliased as <literal>erlang</literal>. The default BEAM package set is defined by <literal>beam.packages.erlang</literal> and aliased at the top level as <literal>beamPackages</literal>. - </para> - - <para> - To create a package builder built with a custom Erlang version, use the lambda, <literal>beam.packagesWith</literal>, which accepts an Erlang/OTP derivation and produces a package builder similar to <literal>beam.packages.erlang</literal>. - </para> - - <para> - Many Erlang/OTP distributions available in <literal>beam.interpreters</literal> have versions with ODBC and/or Java enabled or without wx (no observer support). For example, there's <literal>beam.interpreters.erlangR22_odbc_javac</literal>, which corresponds to <literal>beam.interpreters.erlangR22</literal> and <literal>beam.interpreters.erlangR22_nox</literal>, which corresponds to <literal>beam.interpreters.erlangR22</literal>. - </para> - </section> - - <section xml:id="build-tools"> - <title>Build Tools</title> - - <section xml:id="build-tools-rebar3"> - <title>Rebar3</title> - - <para> - We provide a version of Rebar3, under <literal>rebar3</literal>. We also provide a helper to fetch Rebar3 dependencies from a lockfile under <literal>fetchRebar3Deps</literal>. - </para> - </section> - - <section xml:id="build-tools-other"> - <title>Mix & Erlang.mk</title> - - <para> - Both Mix and Erlang.mk work exactly as expected. There is a bootstrap process that needs to be run for both, however, which is supported by the <literal>buildMix</literal> and <literal>buildErlangMk</literal> derivations, respectively. - </para> - </section> - </section> - - <section xml:id="how-to-install-beam-packages"> - <title>How to Install BEAM Packages</title> - - <para> - BEAM builders are not registered at the top level, simply because they are not relevant to the vast majority of Nix users. - To install any of those builders into your profile, refer to them by their attribute path <literal>beamPackages.rebar3</literal>: - </para> - -<screen> -<prompt>$ </prompt>nix-env -f "<nixpkgs>" -iA beamPackages.rebar3 -</screen> -</section> - - <section xml:id="packaging-beam-applications"> - <title>Packaging BEAM Applications</title> - - <section xml:id="packaging-erlang-applications"> - <title>Erlang Applications</title> - - <section xml:id="rebar3-packages"> - <title>Rebar3 Packages</title> - - <para> - The Nix function, <literal>buildRebar3</literal>, defined in <literal>beam.packages.erlang.buildRebar3</literal> and aliased at the top level, can be used to build a derivation that understands how to build a Rebar3 project. - </para> - - <para> - If a package needs to compile native code via Rebar3's port compilation mechanism, add <literal>compilePort = true;</literal> to the derivation. - </para> - </section> - - <section xml:id="erlang-mk-packages"> - <title>Erlang.mk Packages</title> - - <para> - Erlang.mk functions similarly to Rebar3, except we use <literal>buildErlangMk</literal> instead of <literal>buildRebar3</literal>. - </para> - - </section> - - <section xml:id="mix-packages"> - <title>Mix Packages</title> - - <para> - Mix functions similarly to Rebar3, except we use <literal>buildMix</literal> instead of <literal>buildRebar3</literal>. - </para> - - <para> - Alternatively, we can use <literal>buildHex</literal> as a shortcut: - </para> - </section> - </section> - </section> - - <section xml:id="how-to-develop"> - <title>How to Develop</title> - - <section xml:id="creating-a-shell"> - <title>Creating a Shell</title> - - <para> - Usually, we need to create a <literal>shell.nix</literal> file and do our development inside of the environment specified therein. Just install your version of erlang and other interpreter, and then user your normal build tools. - As an example with elixir: - </para> - -<programlisting> -{ pkgs ? import "<nixpkgs"> {} }: - -with pkgs; - -let - - elixir = beam.packages.erlangR22.elixir_1_9; - -in -mkShell { - buildInputs = [ elixir ]; - - ERL_INCLUDE_PATH="${erlang}/lib/erlang/usr/include"; -} -</programlisting> - - <section xml:id="building-in-a-shell"> - <title>Building in a Shell (for Mix Projects)</title> - - <para> - Using a <literal>shell.nix</literal> as described (see <xref - linkend="creating-a-shell"/>) should just work. - </para> - </section> - </section> - </section> -</section> diff --git a/doc/languages-frameworks/dotnet.section.md b/doc/languages-frameworks/dotnet.section.md index c56f4728bed8c..88fd74db8256c 100644 --- a/doc/languages-frameworks/dotnet.section.md +++ b/doc/languages-frameworks/dotnet.section.md @@ -64,9 +64,9 @@ $ dotnet --info The `dotnetCorePackages.sdk_X_Y` is preferred over the old dotnet-sdk as both major and minor version are very important for a dotnet environment. If a given minor version isn't present (or was changed), then this will likely break your ability to build a project. -## dotnetCorePackages.sdk vs dotnetCorePackages.netcore vs dotnetCorePackages.aspnetcore +## dotnetCorePackages.sdk vs vs dotnetCorePackages.net vs dotnetCorePackages.netcore vs dotnetCorePackages.aspnetcore -The `dotnetCorePackages.sdk` contains both a runtime and the full sdk of a given version. The `netcore` and `aspnetcore` packages are meant to serve as minimal runtimes to deploy alongside already built applications. +The `dotnetCorePackages.sdk` contains both a runtime and the full sdk of a given version. The `net`, `netcore` and `aspnetcore` packages are meant to serve as minimal runtimes to deploy alongside already built applications. For runtime versions >= .NET 5 `net` is used while `netcore` is used for older .NET Core runtime version. ## Packaging a Dotnet Application diff --git a/doc/languages-frameworks/go.section.md b/doc/languages-frameworks/go.section.md new file mode 100644 index 0000000000000..b4228d9d313d9 --- /dev/null +++ b/doc/languages-frameworks/go.section.md @@ -0,0 +1,140 @@ +# Go {#sec-language-go} + +## Go modules {#ssec-language-go} + +The function `buildGoModule` builds Go programs managed with Go modules. It builds a [Go Modules](https://github.com/golang/go/wiki/Modules) through a two phase build: + +- An intermediate fetcher derivation. This derivation will be used to fetch all of the dependencies of the Go module. +- A final derivation will use the output of the intermediate derivation to build the binaries and produce the final output. + +### Example for `buildGoModule` {#ex-buildGoModule} + +In the following is an example expression using `buildGoModule`, the following arguments are of special significance to the function: + +- `vendorSha256`: is the hash of the output of the intermediate fetcher derivation. `vendorSha256` can also take `null` as an input. When `null` is used as a value, rather than fetching the dependencies and vendoring them, we use the vendoring included within the source repo. If you'd like to not have to update this field on dependency changes, run `go mod vendor` in your source repo and set `vendorSha256 = null;` +- `runVend`: runs the vend command to generate the vendor directory. This is useful if your code depends on c code and go mod tidy does not include the needed sources to build. + +```nix +pet = buildGoModule rec { + pname = "pet"; + version = "0.3.4"; + + src = fetchFromGitHub { + owner = "knqyf263"; + repo = "pet"; + rev = "v${version}"; + sha256 = "0m2fzpqxk7hrbxsgqplkg7h2p7gv6s1miymv3gvw0cz039skag0s"; + }; + + vendorSha256 = "1879j77k96684wi554rkjxydrj8g3hpp0kvxz03sd8dmwr3lh83j"; + + runVend = true; + + meta = with lib; { + description = "Simple command-line snippet manager, written in Go"; + homepage = "https://github.com/knqyf263/pet"; + license = licenses.mit; + maintainers = with maintainers; [ kalbasit ]; + platforms = platforms.linux ++ platforms.darwin; + }; +} +``` + +## `buildGoPackage` (legacy) {#ssec-go-legacy} + +The function `buildGoPackage` builds legacy Go programs, not supporting Go modules. + +### Example for `buildGoPackage` + +In the following is an example expression using buildGoPackage, the following arguments are of special significance to the function: + +- `goPackagePath` specifies the package's canonical Go import path. +- `goDeps` is where the Go dependencies of a Go program are listed as a list of package source identified by Go import path. It could be imported as a separate `deps.nix` file for readability. The dependency data structure is described below. + +```nix +deis = buildGoPackage rec { + pname = "deis"; + version = "1.13.0"; + + goPackagePath = "github.com/deis/deis"; + + src = fetchFromGitHub { + owner = "deis"; + repo = "deis"; + rev = "v${version}"; + sha256 = "1qv9lxqx7m18029lj8cw3k7jngvxs4iciwrypdy0gd2nnghc68sw"; + }; + + goDeps = ./deps.nix; +} +``` + +The `goDeps` attribute can be imported from a separate `nix` file that defines which Go libraries are needed and should be included in `GOPATH` for `buildPhase`: + +```nix +# deps.nix +[ # goDeps is a list of Go dependencies. + { + # goPackagePath specifies Go package import path. + goPackagePath = "gopkg.in/yaml.v2"; + fetch = { + # `fetch type` that needs to be used to get package source. + # If `git` is used there should be `url`, `rev` and `sha256` defined next to it. + type = "git"; + url = "https://gopkg.in/yaml.v2"; + rev = "a83829b6f1293c91addabc89d0571c246397bbf4"; + sha256 = "1m4dsmk90sbi17571h6pld44zxz7jc4lrnl4f27dpd1l8g5xvjhh"; + }; + } + { + goPackagePath = "github.com/docopt/docopt-go"; + fetch = { + type = "git"; + url = "https://github.com/docopt/docopt-go"; + rev = "784ddc588536785e7299f7272f39101f7faccc3f"; + sha256 = "0wwz48jl9fvl1iknvn9dqr4gfy1qs03gxaikrxxp9gry6773v3sj"; + }; + } +] +``` + +To extract dependency information from a Go package in automated way use [go2nix](https://github.com/kamilchm/go2nix). It can produce complete derivation and `goDeps` file for Go programs. + +You may use Go packages installed into the active Nix profiles by adding the following to your ~/.bashrc: + +```bash +for p in $NIX_PROFILES; do + GOPATH="$p/share/go:$GOPATH" +done +``` + +## Attributes used by the builders {#ssec-go-common-attributes} + +Both `buildGoModule` and `buildGoPackage` can be tweaked to behave slightly differently, if the following attributes are used: + +### `buildFlagsArray` and `buildFlags`: {#ex-goBuildFlags-noarray} + +These attributes set build flags supported by `go build`. We recommend using `buildFlagsArray`. The most common use case of these attributes is to make the resulting executable aware of its own version. For example: + +```nix + buildFlagsArray = [ + # Note: single quotes are not needed. + "-ldflags=-X main.Version=${version} -X main.Commit=${version}" + ]; +``` + +```nix + buildFlagsArray = '' + -ldflags= + -X main.Version=${version} + -X main.Commit=${version} + ''; +``` + +### `deleteVendor` {#var-go-deleteVendor} + +Removes the pre-existing vendor directory. This should only be used if the dependencies included in the vendor folder are broken or incomplete. + +### `subPackages` {#var-go-subPackages} + +Limits the builder from building child packages that have not been listed. If <varname>subPackages</varname> is not specified, all child packages will be built. diff --git a/doc/languages-frameworks/go.xml b/doc/languages-frameworks/go.xml deleted file mode 100644 index ebdcf616054c2..0000000000000 --- a/doc/languages-frameworks/go.xml +++ /dev/null @@ -1,248 +0,0 @@ -<section xmlns="http://docbook.org/ns/docbook" - xmlns:xlink="http://www.w3.org/1999/xlink" - xml:id="sec-language-go"> - <title>Go</title> - - <section xml:id="ssec-go-modules"> - <title>Go modules</title> - - <para> - The function <varname> buildGoModule </varname> builds Go programs managed with Go modules. It builds a <link xlink:href="https://github.com/golang/go/wiki/Modules">Go modules</link> through a two phase build: - <itemizedlist> - <listitem> - <para> - An intermediate fetcher derivation. This derivation will be used to fetch all of the dependencies of the Go module. - </para> - </listitem> - <listitem> - <para> - A final derivation will use the output of the intermediate derivation to build the binaries and produce the final output. - </para> - </listitem> - </itemizedlist> - </para> - - <example xml:id='ex-buildGoModule'> - <title>buildGoModule</title> -<programlisting> -pet = buildGoModule rec { - pname = "pet"; - version = "0.3.4"; - - src = fetchFromGitHub { - owner = "knqyf263"; - repo = "pet"; - rev = "v${version}"; - sha256 = "0m2fzpqxk7hrbxsgqplkg7h2p7gv6s1miymv3gvw0cz039skag0s"; - }; - - vendorSha256 = "1879j77k96684wi554rkjxydrj8g3hpp0kvxz03sd8dmwr3lh83j"; <co xml:id='ex-buildGoModule-1' /> - - runVend = true; <co xml:id='ex-buildGoModule-2' /> - - meta = with lib; { - description = "Simple command-line snippet manager, written in Go"; - homepage = "https://github.com/knqyf263/pet"; - license = licenses.mit; - maintainers = with maintainers; [ kalbasit ]; - platforms = platforms.linux ++ platforms.darwin; - }; -} -</programlisting> - </example> - - <para> - <xref linkend='ex-buildGoModule'/> is an example expression using buildGoModule, the following arguments are of special significance to the function: - <calloutlist> - <callout arearefs='ex-buildGoModule-1'> - <para> - <varname>vendorSha256</varname> is the hash of the output of the intermediate fetcher derivation. - </para> - </callout> - <callout arearefs='ex-buildGoModule-2'> - <para> - <varname>runVend</varname> runs the vend command to generate the vendor directory. This is useful if your code depends on c code and go mod tidy does not include the needed sources to build. - </para> - </callout> - </calloutlist> - </para> - - <para> - <varname>vendorSha256</varname> can also take <varname>null</varname> as an input. When `null` is used as a value, rather than fetching the dependencies and vendoring them, we use the vendoring included within the source repo. If you'd like to not have to update this field on dependency changes, run `go mod vendor` in your source repo and set 'vendorSha256 = null;' - </para> - </section> - - <section xml:id="ssec-go-legacy"> - <title>Go legacy</title> - - <para> - The function <varname> buildGoPackage </varname> builds legacy Go programs, not supporting Go modules. - </para> - - <example xml:id='ex-buildGoPackage'> - <title>buildGoPackage</title> -<programlisting> -deis = buildGoPackage rec { - pname = "deis"; - version = "1.13.0"; - - goPackagePath = "github.com/deis/deis"; <co xml:id='ex-buildGoPackage-1' /> - - src = fetchFromGitHub { - owner = "deis"; - repo = "deis"; - rev = "v${version}"; - sha256 = "1qv9lxqx7m18029lj8cw3k7jngvxs4iciwrypdy0gd2nnghc68sw"; - }; - - goDeps = ./deps.nix; <co xml:id='ex-buildGoPackage-2' /> -} -</programlisting> - </example> - - <para> - <xref linkend='ex-buildGoPackage'/> is an example expression using buildGoPackage, the following arguments are of special significance to the function: - <calloutlist> - <callout arearefs='ex-buildGoPackage-1'> - <para> - <varname>goPackagePath</varname> specifies the package's canonical Go import path. - </para> - </callout> - <callout arearefs='ex-buildGoPackage-2'> - <para> - <varname>goDeps</varname> is where the Go dependencies of a Go program are listed as a list of package source identified by Go import path. It could be imported as a separate <varname>deps.nix</varname> file for readability. The dependency data structure is described below. - </para> - </callout> - </calloutlist> - </para> - - <para> - The <varname>goDeps</varname> attribute can be imported from a separate <varname>nix</varname> file that defines which Go libraries are needed and should be included in <varname>GOPATH</varname> for <varname>buildPhase</varname>. - </para> - - <example xml:id='ex-goDeps'> - <title>deps.nix</title> -<programlisting> -[ <co xml:id='ex-goDeps-1' /> - { - goPackagePath = "gopkg.in/yaml.v2"; <co xml:id='ex-goDeps-2' /> - fetch = { - type = "git"; <co xml:id='ex-goDeps-3' /> - url = "https://gopkg.in/yaml.v2"; - rev = "a83829b6f1293c91addabc89d0571c246397bbf4"; - sha256 = "1m4dsmk90sbi17571h6pld44zxz7jc4lrnl4f27dpd1l8g5xvjhh"; - }; - } - { - goPackagePath = "github.com/docopt/docopt-go"; - fetch = { - type = "git"; - url = "https://github.com/docopt/docopt-go"; - rev = "784ddc588536785e7299f7272f39101f7faccc3f"; - sha256 = "0wwz48jl9fvl1iknvn9dqr4gfy1qs03gxaikrxxp9gry6773v3sj"; - }; - } -] -</programlisting> - </example> - - <para> - <calloutlist> - <callout arearefs='ex-goDeps-1'> - <para> - <varname>goDeps</varname> is a list of Go dependencies. - </para> - </callout> - <callout arearefs='ex-goDeps-2'> - <para> - <varname>goPackagePath</varname> specifies Go package import path. - </para> - </callout> - <callout arearefs='ex-goDeps-3'> - <para> - <varname>fetch type</varname> that needs to be used to get package source. If <varname>git</varname> is used there should be <varname>url</varname>, <varname>rev</varname> and <varname>sha256</varname> defined next to it. - </para> - </callout> - </calloutlist> - </para> - - <para> - To extract dependency information from a Go package in automated way use <link xlink:href="https://github.com/kamilchm/go2nix">go2nix</link>. It can produce complete derivation and <varname>goDeps</varname> file for Go programs. - </para> - - <para> - You may use Go packages installed into the active Nix profiles by adding the following to your ~/.bashrc: -<screen> -for p in $NIX_PROFILES; do - GOPATH="$p/share/go:$GOPATH" -done -</screen> - </para> - </section> - - <section xml:id="ssec-go-common-attributes"> - <title>Attributes used by the builders</title> - - <para> - Both <link xlink:href="#ssec-go-modules"><varname>buildGoModule</varname></link> and <link xlink:href="#ssec-go-modules"><varname>buildGoPackage</varname></link> can be tweaked to behave slightly differently, if the following attributes are used: - </para> - - <variablelist> - <varlistentry xml:id="var-go-buildFlagsArray"> - <term> - <varname>buildFlagsArray</varname> and <varname>buildFlags</varname> - </term> - <listitem> - <para> - These attributes set build flags supported by <varname>go build</varname>. We recommend using <varname>buildFlagsArray</varname>. The most common use case of these attributes is to make the resulting executable aware of its own version. For example: - </para> - <example xml:id='ex-goBuildFlags-nospaces'> - <title>buildFlagsArray</title> -<programlisting> - buildFlagsArray = [ - "-ldflags=-X main.Version=${version} -X main.Commit=${version}" <co xml:id='ex-goBuildFlags-1' /> - ]; -</programlisting> - </example> - <calloutlist> - <callout arearefs='ex-goBuildFlags-1'> - <para> - Note: single quotes are not needed. - </para> - </callout> - </calloutlist> - <example xml:id='ex-goBuildFlags-noarray'> - <title>buildFlagsArray</title> -<programlisting> - buildFlagsArray = '' - -ldflags= - -X main.Version=${version} - -X main.Commit=${version} - ''; -</programlisting> - </example> - </listitem> - </varlistentry> - <varlistentry xml:id="var-go-deleteVendor"> - <term> - <varname>deleteVendor</varname> - </term> - <listitem> - <para> - Removes the pre-existing vendor directory. This should only be used if the dependencies included in the vendor folder are broken or incomplete. - </para> - </listitem> - </varlistentry> - <varlistentry xml:id="var-go-subPackages"> - <term> - <varname>subPackages</varname> - </term> - <listitem> - <para> - Limits the builder from building child packages that have not been listed. If <varname>subPackages</varname> is not specified, all child packages will be built. - </para> - </listitem> - </varlistentry> - </variablelist> - </section> -</section> diff --git a/doc/languages-frameworks/index.xml b/doc/languages-frameworks/index.xml index 728a38c264a3c..7a4c54fca8d03 100644 --- a/doc/languages-frameworks/index.xml +++ b/doc/languages-frameworks/index.xml @@ -7,18 +7,19 @@ </para> <xi:include href="agda.section.xml" /> <xi:include href="android.section.xml" /> - <xi:include href="beam.xml" /> + <xi:include href="beam.section.xml" /> <xi:include href="bower.xml" /> <xi:include href="coq.xml" /> <xi:include href="crystal.section.xml" /> <xi:include href="emscripten.section.xml" /> <xi:include href="gnome.xml" /> - <xi:include href="go.xml" /> + <xi:include href="go.section.xml" /> <xi:include href="haskell.section.xml" /> <xi:include href="idris.section.xml" /> <xi:include href="ios.section.xml" /> <xi:include href="java.xml" /> <xi:include href="lua.section.xml" /> + <xi:include href="maven.section.xml" /> <xi:include href="node.section.xml" /> <xi:include href="ocaml.xml" /> <xi:include href="perl.xml" /> @@ -26,7 +27,7 @@ <xi:include href="python.section.xml" /> <xi:include href="qt.xml" /> <xi:include href="r.section.xml" /> - <xi:include href="ruby.xml" /> + <xi:include href="ruby.section.xml" /> <xi:include href="rust.section.xml" /> <xi:include href="texlive.xml" /> <xi:include href="titanium.section.xml" /> diff --git a/doc/languages-frameworks/java.xml b/doc/languages-frameworks/java.xml index bf0fc48839223..881d492b5bff7 100644 --- a/doc/languages-frameworks/java.xml +++ b/doc/languages-frameworks/java.xml @@ -32,7 +32,7 @@ nativeBuildInputs = [ jdk ]; </para> <para> - If your Java package provides a program, you need to generate a wrapper script to run it using the OpenJRE. You can use <literal>makeWrapper</literal> for this: + If your Java package provides a program, you need to generate a wrapper script to run it using a JRE. You can use <literal>makeWrapper</literal> for this: <programlisting> nativeBuildInputs = [ makeWrapper ]; @@ -43,7 +43,21 @@ installPhase = --add-flags "-cp $out/share/java/foo.jar org.foo.Main" ''; </programlisting> - Note the use of <literal>jre</literal>, which is the part of the OpenJDK package that contains the Java Runtime Environment. By using <literal>${jre}/bin/java</literal> instead of <literal>${jdk}/bin/java</literal>, you prevent your package from depending on the JDK at runtime. +Since the introduction of the Java Platform Module System in Java 9, Java distributions typically no longer ship with a general-purpose JRE: instead, they allow generating a JRE with only the modules required for your application(s). Because we can't predict what modules will be needed on a general-purpose system, the default <package>jre</package> package is the full JDK. When building a minimal system/image, you can override the <literal>modules</literal> parameter on <literal>jre_minimal</literal> to build a JRE with only the modules relevant for you: +<programlisting> +let + my_jre = pkgs.jre_minimal.override { + modules = [ + # The modules used by 'something' and 'other' combined: + "java.base" + "java.logging" + ]; + }; + something = (pkgs.something.override { jre = my_jre; }); + other = (pkgs.other.override { jre = my_jre; }); +in + ... +</programlisting> </para> <para> diff --git a/doc/languages-frameworks/maven.section.md b/doc/languages-frameworks/maven.section.md new file mode 100644 index 0000000000000..fe183e7ba3cca --- /dev/null +++ b/doc/languages-frameworks/maven.section.md @@ -0,0 +1,354 @@ +--- +title: Maven +author: Farid Zakaria +date: 2020-10-15 +--- + +# Maven + +Maven is a well-known build tool for the Java ecosystem however it has some challenges when integrating into the Nix build system. + +The following provides a list of common patterns with how to package a Maven project (or any JVM language that can export to Maven) as a Nix package. + +For the purposes of this example let's consider a very basic Maven project with the following `pom.xml` with a single dependency on [emoji-java](https://github.com/vdurmont/emoji-java). + +```xml +<?xml version="1.0" encoding="UTF-8"?> +<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" + xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"> + <modelVersion>4.0.0</modelVersion> + <groupId>io.github.fzakaria</groupId> + <artifactId>maven-demo</artifactId> + <version>1.0</version> + <packaging>jar</packaging> + <name>NixOS Maven Demo</name> + + <dependencies> + <dependency> + <groupId>com.vdurmont</groupId> + <artifactId>emoji-java</artifactId> + <version>5.1.1</version> + </dependency> + </dependencies> +</project> +``` + +Our main class file will be very simple: + +```java +import com.vdurmont.emoji.EmojiParser; + +public class Main { + public static void main(String[] args) { + String str = "NixOS :grinning: is super cool :smiley:!"; + String result = EmojiParser.parseToUnicode(str); + System.out.println(result); + } +} +``` + +You find this demo project at https://github.com/fzakaria/nixos-maven-example + +## Solving for dependencies + +### buildMaven with NixOS/mvn2nix-maven-plugin + +> ⚠️ Although `buildMaven` is the "blessed" way within nixpkgs, as of 2020, it hasn't seen much activity in quite a while. + +`buildMaven` is an alternative method that tries to follow similar patterns of other programming languages by generating a lock file. It relies on the maven plugin [mvn2nix-maven-plugin](https://github.com/NixOS/mvn2nix-maven-plugin). + +First you generate a `project-info.json` file using the maven plugin. + +> This should be executed in the project's source repository or be told which `pom.xml` to execute with. + +```bash +# run this step within the project's source repository +❯ mvn org.nixos.mvn2nix:mvn2nix-maven-plugin:mvn2nix + +❯ cat project-info.json | jq | head +{ + "project": { + "artifactId": "maven-demo", + "groupId": "org.nixos", + "version": "1.0", + "classifier": "", + "extension": "jar", + "dependencies": [ + { + "artifactId": "maven-resources-plugin", +``` + +This file is then given to the `buildMaven` function, and it returns 2 attributes. + +**`repo`**: + A Maven repository that is a symlink farm of all the dependencies found in the `project-info.json` + + +**`build`**: + A simple derivation that runs through `mvn compile` & `mvn package` to build the JAR. You may use this as inspiration for more complicated derivations. + +Here is an [example](https://github.com/fzakaria/nixos-maven-example/blob/main/build-maven-repository.nix) of building the Maven repository +```nix +{ pkgs ? import <nixpkgs> { } }: +with pkgs; +(buildMaven ./project-info.json).repo +``` + +The benefit over the _double invocation_ as we will see below, is that the _/nix/store_ entry is a _linkFarm_ of every package, so that changes to your dependency set doesn't involve downloading everything from scratch. + +```bash +❯ tree $(nix-build --no-out-link build-maven-repository.nix) | head +/nix/store/g87va52nkc8jzbmi1aqdcf2f109r4dvn-maven-repository +├── antlr +│ └── antlr +│ └── 2.7.2 +│ ├── antlr-2.7.2.jar -> /nix/store/d027c8f2cnmj5yrynpbq2s6wmc9cb559-antlr-2.7.2.jar +│ └── antlr-2.7.2.pom -> /nix/store/mv42fc5gizl8h5g5vpywz1nfiynmzgp2-antlr-2.7.2.pom +├── avalon-framework +│ └── avalon-framework +│ └── 4.1.3 +│ ├── avalon-framework-4.1.3.jar -> /nix/store/iv5fp3955w3nq28ff9xfz86wvxbiw6n9-avalon-framework-4.1.3.jar +``` +### Double Invocation + +> ⚠️ This pattern is the simplest but may cause unnecessary rebuilds due to the output hash changing. + +The double invocation is a _simple_ way to get around the problem that `nix-build` may be sandboxed and have no Internet connectivity. + +It treats the entire Maven repository as a single source to be downloaded, relying on Maven's dependency resolution to satisfy the output hash. This is similar to fetchers like `fetchgit`, except it has to run a Maven build to determine what to download. + +The first step will be to build the Maven project as a fixed-output derivation in order to collect the Maven repository -- below is an [example](https://github.com/fzakaria/nixos-maven-example/blob/main/double-invocation-repository.nix). + +> Traditionally the Maven repository is at `~/.m2/repository`. We will override this to be the `$out` directory. + +```nix +{ stdenv, maven }: +stdenv.mkDerivation { + name = "maven-repository"; + buildInputs = [ maven ]; + src = ./.; # or fetchFromGitHub, cleanSourceWith, etc + buildPhase = '' + mvn package -Dmaven.repo.local=$out + ''; + + # keep only *.{pom,jar,sha1,nbm} and delete all ephemeral files with lastModified timestamps inside + installPhase = '' + find $out -type f \ + -name \*.lastUpdated -or \ + -name resolver-status.properties -or \ + -name _remote.repositories \ + -delete + ''; + + # don't do any fixup + dontFixup = true; + outputHashAlgo = "sha256"; + outputHashMode = "recursive"; + # replace this with the correct SHA256 + outputHash = stdenv.lib.fakeSha256; +} +``` + +The build will fail, and tell you the expected `outputHash` to place. When you've set the hash, the build will return with a `/nix/store` entry whose contents are the full Maven repository. + +> Some additional files are deleted that would cause the output hash to change potentially on subsequent runs. + +```bash +❯ tree $(nix-build --no-out-link double-invocation-repository.nix) | head +/nix/store/8kicxzp98j68xyi9gl6jda67hp3c54fq-maven-repository +├── backport-util-concurrent +│ └── backport-util-concurrent +│ └── 3.1 +│ ├── backport-util-concurrent-3.1.pom +│ └── backport-util-concurrent-3.1.pom.sha1 +├── classworlds +│ └── classworlds +│ ├── 1.1 +│ │ ├── classworlds-1.1.jar +``` + +If your package uses _SNAPSHOT_ dependencies or _version ranges_; there is a strong likelihood that over-time your output hash will change since the resolved dependencies may change. Hence this method is less recommended then using `buildMaven`. + +## Building a JAR + +Regardless of which strategy is chosen above, the step to build the derivation is the same. + +```nix +{ stdenv, lib, maven, callPackage }: +# pick a repository derivation, here we will use buildMaven +let repository = callPackage ./build-maven-repository.nix { }; +in stdenv.mkDerivation rec { + pname = "maven-demo"; + version = "1.0"; + + src = builtins.fetchTarball "https://github.com/fzakaria/nixos-maven-example/archive/main.tar.gz"; + buildInputs = [ maven ]; + + buildPhase = '' + echo "Using repository ${repository}" + mvn --offline -Dmaven.repo.local=${repository} package; + ''; + + installPhase = '' + install -Dm644 target/${pname}-${version}.jar $out/share/java + ''; +} +``` + +> We place the library in `$out/share/java` since JDK package has a _stdenv setup hook_ that adds any JARs in the `share/java` directories of the build inputs to the CLASSPATH environment. + +```bash +❯ tree $(nix-build --no-out-link build-jar.nix) +/nix/store/7jw3xdfagkc2vw8wrsdv68qpsnrxgvky-maven-demo-1.0 +└── share + └── java + └── maven-demo-1.0.jar + +2 directories, 1 file +``` + +## Runnable JAR + +The previous example builds a `jar` file but that's not a file one can run. + +You need to use it with `java -jar $out/share/java/output.jar` and make sure to provide the required dependencies on the classpath. + +The following explains how to use `makeWrapper` in order to make the derivation produce an executable that will run the JAR file you created. + +We will use the same repository we built above (either _double invocation_ or _buildMaven_) to setup a CLASSPATH for our JAR. + +The following two methods are more suited to Nix then building an [UberJar](https://imagej.net/Uber-JAR) which may be the more traditional approach. + +### CLASSPATH + +> This is ideal if you are providing a derivation for _nixpkgs_ and don't want to patch the project's `pom.xml`. + +We will read the Maven repository and flatten it to a single list. This list will then be concatenated with the _CLASSPATH_ separator to create the full classpath. + +We make sure to provide this classpath to the `makeWrapper`. + +```nix +{ stdenv, lib, maven, callPackage, makeWrapper, jre }: +let + repository = callPackage ./build-maven-repository.nix { }; +in stdenv.mkDerivation rec { + pname = "maven-demo"; + version = "1.0"; + + src = builtins.fetchTarball + "https://github.com/fzakaria/nixos-maven-example/archive/main.tar.gz"; + buildInputs = [ maven makeWrapper ]; + + buildPhase = '' + echo "Using repository ${repository}" + mvn --offline -Dmaven.repo.local=${repository} package; + ''; + + installPhase = '' + mkdir -p $out/bin + + classpath=$(find ${repository} -name "*.jar" -printf ':%h/%f'); + install -Dm644 target/${pname}-${version}.jar $out/share/java + # create a wrapper that will automatically set the classpath + # this should be the paths from the dependency derivation + makeWrapper ${jre}/bin/java $out/bin/${pname} \ + --add-flags "-classpath $out/share/java/${pname}-${version}.jar:''${classpath#:}" \ + --add-flags "Main" + ''; +} +``` + +### MANIFEST file via Maven Plugin + +> This is ideal if you are the project owner and want to change your `pom.xml` to set the CLASSPATH within it. + +Augment the `pom.xml` to create a JAR with the following manifest: +```xml +<build> + <plugins> + <plugin> + <artifactId>maven-jar-plugin</artifactId> + <configuration> + <archive> + <manifest> + <addClasspath>true</addClasspath> + <classpathPrefix>../../repository/</classpathPrefix> + <classpathLayoutType>repository</classpathLayoutType> + <mainClass>Main</mainClass> + </manifest> + <manifestEntries> + <Class-Path>.</Class-Path> + </manifestEntries> + </archive> + </configuration> + </plugin> + </plugins> +</build> +``` + +The above plugin instructs the JAR to look for the necessary dependencies in the `lib/` relative folder. The layout of the folder is also in the _maven repository_ style. + +```bash +❯ unzip -q -c $(nix-build --no-out-link runnable-jar.nix)/share/java/maven-demo-1.0.jar META-INF/MANIFEST.MF + +Manifest-Version: 1.0 +Archiver-Version: Plexus Archiver +Built-By: nixbld +Class-Path: . ../../repository/com/vdurmont/emoji-java/5.1.1/emoji-jav + a-5.1.1.jar ../../repository/org/json/json/20170516/json-20170516.jar +Created-By: Apache Maven 3.6.3 +Build-Jdk: 1.8.0_265 +Main-Class: Main +``` + +We will modify the derivation above to add a symlink to our repository so that it's accessible to our JAR during the `installPhase`. + +```nix +{ stdenv, lib, maven, callPackage, makeWrapper, jre }: +# pick a repository derivation, here we will use buildMaven +let repository = callPackage ./build-maven-repository.nix { }; +in stdenv.mkDerivation rec { + pname = "maven-demo"; + version = "1.0"; + + src = builtins.fetchTarball + "https://github.com/fzakaria/nixos-maven-example/archive/main.tar.gz"; + buildInputs = [ maven makeWrapper ]; + + buildPhase = '' + echo "Using repository ${repository}" + mvn --offline -Dmaven.repo.local=${repository} package; + ''; + + installPhase = '' + mkdir -p $out/bin + + # create a symbolic link for the repository directory + ln -s ${repository} $out/repository + + install -Dm644 target/${pname}-${version}.jar $out/share/java + # create a wrapper that will automatically set the classpath + # this should be the paths from the dependency derivation + makeWrapper ${jre}/bin/java $out/bin/${pname} \ + --add-flags "-jar $out/share/java/${pname}-${version}.jar" + ''; +} +``` + +> Our script produces a dependency on `jre` rather than `jdk` to restrict the runtime closure necessary to run the application. + +This will give you an executable shell-script that launches your JAR with all the dependencies available. + +```bash +❯ tree $(nix-build --no-out-link runnable-jar.nix) +/nix/store/8d4c3ibw8ynsn01ibhyqmc1zhzz75s26-maven-demo-1.0 +├── bin +│ └── maven-demo +├── repository -> /nix/store/g87va52nkc8jzbmi1aqdcf2f109r4dvn-maven-repository +└── share + └── java + └── maven-demo-1.0.jar + +❯ $(nix-build --no-out-link --option tarball-ttl 1 runnable-jar.nix)/bin/maven-demo +NixOS 😀 is super cool 😃! +``` diff --git a/doc/languages-frameworks/python.section.md b/doc/languages-frameworks/python.section.md index 59f7389b9ad3c..8a2fe2711c7a6 100644 --- a/doc/languages-frameworks/python.section.md +++ b/doc/languages-frameworks/python.section.md @@ -153,7 +153,7 @@ The dot product of [1 2] and [3 4] is: 11 But if we maintain the script ourselves, and if there are more dependencies, it may be nice to encode those dependencies in source to make the script re-usable without that bit of knowledge. That can be done by using `nix-shell` as a -[shebang](https://en.wikipedia.org/wiki/Shebang_(Unix), like so: +[shebang](https://en.wikipedia.org/wiki/Shebang_(Unix)), like so: ```python #!/usr/bin/env nix-shell diff --git a/doc/languages-frameworks/ruby.section.md b/doc/languages-frameworks/ruby.section.md index e4c4ffce04325..e292b3110ff4c 100644 --- a/doc/languages-frameworks/ruby.section.md +++ b/doc/languages-frameworks/ruby.section.md @@ -1,74 +1,38 @@ ---- -title: Ruby -author: Michael Fellinger -date: 2019-05-23 ---- +# Ruby {#sec-language-ruby} -# Ruby +## Using Ruby -## User Guide +Several versions of Ruby interpreters are available on Nix, as well as over 250 gems and many applications written in Ruby. The attribute `ruby` refers to the default Ruby interpreter, which is currently MRI 2.6. It's also possible to refer to specific versions, e.g. `ruby_2_y`, `jruby`, or `mruby`. -### Using Ruby +In the Nixpkgs tree, Ruby packages can be found throughout, depending on what they do, and are called from the main package set. Ruby gems, however are separate sets, and there's one default set for each interpreter (currently MRI only). -#### Overview +There are two main approaches for using Ruby with gems. One is to use a specifically locked `Gemfile` for an application that has very strict dependencies. The other is to depend on the common gems, which we'll explain further down, and rely on them being updated regularly. -Several versions of Ruby interpreters are available on Nix, as well as over 250 gems and many applications written in Ruby. -The attribute `ruby` refers to the default Ruby interpreter, which is currently -MRI 2.5. It's also possible to refer to specific versions, e.g. `ruby_2_6`, `jruby`, or `mruby`. +The interpreters have common attributes, namely `gems`, and `withPackages`. So you can refer to `ruby.gems.nokogiri`, or `ruby_2_6.gems.nokogiri` to get the Nokogiri gem already compiled and ready to use. -In the nixpkgs tree, Ruby packages can be found throughout, depending on what -they do, and are called from the main package set. Ruby gems, however are -separate sets, and there's one default set for each interpreter (currently MRI -only). +Since not all gems have executables like `nokogiri`, it's usually more convenient to use the `withPackages` function like this: `ruby.withPackages (p: with p; [ nokogiri ])`. This will also make sure that the Ruby in your environment will be able to find the gem and it can be used in your Ruby code (for example via `ruby` or `irb` executables) via `require "nokogiri"` as usual. -There are two main approaches for using Ruby with gems. -One is to use a specifically locked `Gemfile` for an application that has very strict dependencies. -The other is to depend on the common gems, which we'll explain further down, and -rely on them being updated regularly. +### Temporary Ruby environment with `nix-shell` -The interpreters have common attributes, namely `gems`, and `withPackages`. So -you can refer to `ruby.gems.nokogiri`, or `ruby_2_5.gems.nokogiri` to get the -Nokogiri gem already compiled and ready to use. +Rather than having a single Ruby environment shared by all Ruby development projects on a system, Nix allows you to create separate environments per project. `nix-shell` gives you the possibility to temporarily load another environment akin to a combined `chruby` or `rvm` and `bundle exec`. -Since not all gems have executables like `nokogiri`, it's usually more -convenient to use the `withPackages` function like this: -`ruby.withPackages (p: with p; [ nokogiri ])`. This will also make sure that the -Ruby in your environment will be able to find the gem and it can be used in your -Ruby code (for example via `ruby` or `irb` executables) via `require "nokogiri"` -as usual. +There are two methods for loading a shell with Ruby packages. The first and recommended method is to create an environment with `ruby.withPackages` and load that. -#### Temporary Ruby environment with `nix-shell` - -Rather than having a single Ruby environment shared by all Ruby -development projects on a system, Nix allows you to create separate -environments per project. `nix-shell` gives you the possibility to -temporarily load another environment akin to a combined `chruby` or -`rvm` and `bundle exec`. - -There are two methods for loading a shell with Ruby packages. The first and -recommended method is to create an environment with `ruby.withPackages` and load -that. - -```shell -nix-shell -p "ruby.withPackages (ps: with ps; [ nokogiri pry ])" +```ShellSession +$ nix-shell -p "ruby.withPackages (ps: with ps; [ nokogiri pry ])" ``` -The other method, which is not recommended, is to create an environment and list -all the packages directly. +The other method, which is not recommended, is to create an environment and list all the packages directly. -```shell -nix-shell -p ruby.gems.nokogiri ruby.gems.pry +```ShellSession +$ nix-shell -p ruby.gems.nokogiri ruby.gems.pry ``` -Again, it's possible to launch the interpreter from the shell. The Ruby -interpreter has the attribute `gems` which contains all Ruby gems for that -specific interpreter. +Again, it's possible to launch the interpreter from the shell. The Ruby interpreter has the attribute `gems` which contains all Ruby gems for that specific interpreter. -##### Load environment from `.nix` expression +#### Load Ruby environment from `.nix` expression -As explained in the Nix manual, `nix-shell` can also load an expression from a -`.nix` file. Say we want to have Ruby 2.5, `nokogori`, and `pry`. Consider a -`shell.nix` file with: +As explained in the Nix manual, `nix-shell` can also load an expression from a `.nix` file. Say we want to have Ruby 2.6, `nokogori`, and `pry`. Consider a `shell.nix` file with: ```nix with import <nixpkgs> {}; @@ -77,43 +41,33 @@ ruby.withPackages (ps: with ps; [ nokogiri pry ]) What's happening here? -1. We begin with importing the Nix Packages collections. `import <nixpkgs>` - imports the `<nixpkgs>` function, `{}` calls it and the `with` statement - brings all attributes of `nixpkgs` in the local scope. These attributes form - the main package set. +1. We begin with importing the Nix Packages collections. `import <nixpkgs>` imports the `<nixpkgs>` function, `{}` calls it and the `with` statement brings all attributes of `nixpkgs` in the local scope. These attributes form the main package set. 2. Then we create a Ruby environment with the `withPackages` function. -3. The `withPackages` function expects us to provide a function as an argument - that takes the set of all ruby gems and returns a list of packages to include - in the environment. Here, we select the packages `nokogiri` and `pry` from - the package set. +3. The `withPackages` function expects us to provide a function as an argument that takes the set of all ruby gems and returns a list of packages to include in the environment. Here, we select the packages `nokogiri` and `pry` from the package set. -##### Execute command with `--run` +#### Execute command with `--run` -A convenient flag for `nix-shell` is `--run`. It executes a command in the -`nix-shell`. We can e.g. directly open a `pry` REPL: +A convenient flag for `nix-shell` is `--run`. It executes a command in the `nix-shell`. We can e.g. directly open a `pry` REPL: -```shell -nix-shell -p "ruby.withPackages (ps: with ps; [ nokogiri pry ])" --run "pry" +```ShellSession +$ nix-shell -p "ruby.withPackages (ps: with ps; [ nokogiri pry ])" --run "pry" ``` Or immediately require `nokogiri` in pry: -```shell -nix-shell -p "ruby.withPackages (ps: with ps; [ nokogiri pry ])" --run "pry -rnokogiri" +```ShellSession +$ nix-shell -p "ruby.withPackages (ps: with ps; [ nokogiri pry ])" --run "pry -rnokogiri" ``` Or run a script using this environment: -```shell -nix-shell -p "ruby.withPackages (ps: with ps; [ nokogiri pry ])" --run "ruby example.rb" +```ShellSession +$ nix-shell -p "ruby.withPackages (ps: with ps; [ nokogiri pry ])" --run "ruby example.rb" ``` -##### Using `nix-shell` as shebang +#### Using `nix-shell` as shebang -In fact, for the last case, there is a more convenient method. You can add a -[shebang](https://en.wikipedia.org/wiki/Shebang_(Unix)) to your script -specifying which dependencies `nix-shell` needs. With the following shebang, you -can just execute `./example.rb`, and it will run with all dependencies. +In fact, for the last case, there is a more convenient method. You can add a [shebang](<https://en.wikipedia.org/wiki/Shebang_(Unix)>) to your script specifying which dependencies `nix-shell` needs. With the following shebang, you can just execute `./example.rb`, and it will run with all dependencies. ```ruby #! /usr/bin/env nix-shell @@ -126,35 +80,24 @@ body = RestClient.get('http://example.com').body puts Nokogiri::HTML(body).at('h1').text ``` -### Developing with Ruby +## Developing with Ruby -#### Using an existing Gemfile +### Using an existing Gemfile -In most cases, you'll already have a `Gemfile.lock` listing all your dependencies. -This can be used to generate a `gemset.nix` which is used to fetch the gems and -combine them into a single environment. -The reason why you need to have a separate file for this, is that Nix requires -you to have a checksum for each input to your build. -Since the `Gemfile.lock` that `bundler` generates doesn't provide us with -checksums, we have to first download each gem, calculate its SHA256, and store -it in this separate file. +In most cases, you'll already have a `Gemfile.lock` listing all your dependencies. This can be used to generate a `gemset.nix` which is used to fetch the gems and combine them into a single environment. The reason why you need to have a separate file for this, is that Nix requires you to have a checksum for each input to your build. Since the `Gemfile.lock` that `bundler` generates doesn't provide us with checksums, we have to first download each gem, calculate its SHA256, and store it in this separate file. So the steps from having just a `Gemfile` to a `gemset.nix` are: -```shell -bundle lock -bundix +```ShellSession +$ bundle lock +$ bundix ``` -If you already have a `Gemfile.lock`, you can simply run `bundix` and it will -work the same. +If you already have a `Gemfile.lock`, you can simply run `bundix` and it will work the same. -To update the gems in your `Gemfile.lock`, you may use the `bundix -l` flag, -which will create a new `Gemfile.lock` in case the `Gemfile` has a more recent -time of modification. +To update the gems in your `Gemfile.lock`, you may use the `bundix -l` flag, which will create a new `Gemfile.lock` in case the `Gemfile` has a more recent time of modification. -Once the `gemset.nix` is generated, it can be used in a -`bundlerEnv` derivation. Here is an example you could use for your `shell.nix`: +Once the `gemset.nix` is generated, it can be used in a `bundlerEnv` derivation. Here is an example you could use for your `shell.nix`: ```nix # ... @@ -166,41 +109,26 @@ let in mkShell { buildInputs = [ gems gems.wrappedRuby ]; } ``` -With this file in your directory, you can run `nix-shell` to build and use the gems. -The important parts here are `bundlerEnv` and `wrappedRuby`. +With this file in your directory, you can run `nix-shell` to build and use the gems. The important parts here are `bundlerEnv` and `wrappedRuby`. -The `bundlerEnv` is a wrapper over all the gems in your gemset. This means that -all the `/lib` and `/bin` directories will be available, and the executables of -all gems (even of indirect dependencies) will end up in your `$PATH`. -The `wrappedRuby` provides you with all executables that come with Ruby itself, -but wrapped so they can easily find the gems in your gemset. +The `bundlerEnv` is a wrapper over all the gems in your gemset. This means that all the `/lib` and `/bin` directories will be available, and the executables of all gems (even of indirect dependencies) will end up in your `$PATH`. The `wrappedRuby` provides you with all executables that come with Ruby itself, but wrapped so they can easily find the gems in your gemset. -One common issue that you might have is that you have Ruby 2.6, but also -`bundler` in your gemset. That leads to a conflict for `/bin/bundle` and -`/bin/bundler`. You can resolve this by wrapping either your Ruby or your gems -in a `lowPrio` call. So in order to give the `bundler` from your gemset -priority, it would be used like this: +One common issue that you might have is that you have Ruby 2.6, but also `bundler` in your gemset. That leads to a conflict for `/bin/bundle` and `/bin/bundler`. You can resolve this by wrapping either your Ruby or your gems in a `lowPrio` call. So in order to give the `bundler` from your gemset priority, it would be used like this: ```nix # ... mkShell { buildInputs = [ gems (lowPrio gems.wrappedRuby) ]; } ``` +### Gem-specific configurations and workarounds -#### Gem-specific configurations and workarounds +In some cases, especially if the gem has native extensions, you might need to modify the way the gem is built. -In some cases, especially if the gem has native extensions, you might need to -modify the way the gem is built. +This is done via a common configuration file that includes all of the workarounds for each gem. -This is done via a common configuration file that includes all of the -workarounds for each gem. +This file lives at `/pkgs/development/ruby-modules/gem-config/default.nix`, since it already contains a lot of entries, it should be pretty easy to add the modifications you need for your needs. -This file lives at `/pkgs/development/ruby-modules/gem-config/default.nix`, -since it already contains a lot of entries, it should be pretty easy to add the -modifications you need for your needs. - -In the meanwhile, or if the modification is for a private gem, you can also add -the configuration to only your own environment. +In the meanwhile, or if the modification is for a private gem, you can also add the configuration to only your own environment. Two places that allow this modification are the `ruby` derivation, or `bundlerEnv`. @@ -261,10 +189,9 @@ let in pkgs.ruby.withPackages (ps: with ps; [ pg ]) ``` -Then we can get whichever postgresql version we desire and the `pg` gem will -always reference it correctly: +Then we can get whichever postgresql version we desire and the `pg` gem will always reference it correctly: -```shell +```ShellSession $ nix-shell --argstr pg_version 9_4 --run 'ruby -rpg -e "puts PG.library_version"' 90421 @@ -272,24 +199,15 @@ $ nix-shell --run 'ruby -rpg -e "puts PG.library_version"' 100007 ``` -Of course for this use-case one could also use overlays since the configuration -for `pg` depends on the `postgresql` alias, but for demonstration purposes this -has to suffice. +Of course for this use-case one could also use overlays since the configuration for `pg` depends on the `postgresql` alias, but for demonstration purposes this has to suffice. -#### Adding a gem to the default gemset +### Adding a gem to the default gemset -Now that you know how to get a working Ruby environment with Nix, it's time to -go forward and start actually developing with Ruby. -We will first have a look at how Ruby gems are packaged on Nix. Then, we will -look at how you can use development mode with your code. +Now that you know how to get a working Ruby environment with Nix, it's time to go forward and start actually developing with Ruby. We will first have a look at how Ruby gems are packaged on Nix. Then, we will look at how you can use development mode with your code. -All gems in the standard set are automatically generated from a single -`Gemfile`. The dependency resolution is done with `bundler` and makes it more -likely that all gems are compatible to each other. +All gems in the standard set are automatically generated from a single `Gemfile`. The dependency resolution is done with `bundler` and makes it more likely that all gems are compatible to each other. -In order to add a new gem to nixpkgs, you can put it into the -`/pkgs/development/ruby-modules/with-packages/Gemfile` and run -`./maintainers/scripts/update-ruby-packages`. +In order to add a new gem to nixpkgs, you can put it into the `/pkgs/development/ruby-modules/with-packages/Gemfile` and run `./maintainers/scripts/update-ruby-packages`. To test that it works, you can then try using the gem with: @@ -297,16 +215,11 @@ To test that it works, you can then try using the gem with: NIX_PATH=nixpkgs=$PWD nix-shell -p "ruby.withPackages (ps: with ps; [ name-of-your-gem ])" ``` -#### Packaging applications +### Packaging applications -A common task is to add a ruby executable to nixpkgs, popular examples would be -`chef`, `jekyll`, or `sass`. A good way to do that is to use the `bundlerApp` -function, that allows you to make a package that only exposes the listed -executables, otherwise the package may cause conflicts through common paths like -`bin/rake` or `bin/bundler` that aren't meant to be used. +A common task is to add a ruby executable to nixpkgs, popular examples would be `chef`, `jekyll`, or `sass`. A good way to do that is to use the `bundlerApp` function, that allows you to make a package that only exposes the listed executables, otherwise the package may cause conflicts through common paths like `bin/rake` or `bin/bundler` that aren't meant to be used. -The absolute easiest way to do that is to write a -`Gemfile` along these lines: +The absolute easiest way to do that is to write a `Gemfile` along these lines: ```ruby source 'https://rubygems.org' do @@ -314,10 +227,7 @@ source 'https://rubygems.org' do end ``` -If you want to package a specific version, you can use the standard Gemfile -syntax for that, e.g. `gem 'mdl', '0.5.0'`, but if you want the latest stable -version anyway, it's easier to update by simply running the `bundle lock` and -`bundix` steps again. +If you want to package a specific version, you can use the standard Gemfile syntax for that, e.g. `gem 'mdl', '0.5.0'`, but if you want the latest stable version anyway, it's easier to update by simply running the `bundle lock` and `bundix` steps again. Now you can also also make a `default.nix` that looks like this: @@ -331,20 +241,15 @@ bundlerApp { } ``` -All that's left to do is to generate the corresponding `Gemfile.lock` and -`gemset.nix` as described above in the `Using an existing Gemfile` section. +All that's left to do is to generate the corresponding `Gemfile.lock` and `gemset.nix` as described above in the `Using an existing Gemfile` section. -##### Packaging executables that require wrapping +#### Packaging executables that require wrapping -Sometimes your app will depend on other executables at runtime, and tries to -find it through the `PATH` environment variable. +Sometimes your app will depend on other executables at runtime, and tries to find it through the `PATH` environment variable. -In this case, you can provide a `postBuild` hook to `bundlerApp` that wraps the -gem in another script that prefixes the `PATH`. +In this case, you can provide a `postBuild` hook to `bundlerApp` that wraps the gem in another script that prefixes the `PATH`. -Of course you could also make a custom `gemConfig` if you know exactly how to -patch it, but it's usually much easier to maintain with a simple wrapper so the -patch doesn't have to be adjusted for each version. +Of course you could also make a custom `gemConfig` if you know exactly how to patch it, but it's usually much easier to maintain with a simple wrapper so the patch doesn't have to be adjusted for each version. Here's another example: diff --git a/doc/languages-frameworks/ruby.xml b/doc/languages-frameworks/ruby.xml deleted file mode 100644 index 9b579d6804f44..0000000000000 --- a/doc/languages-frameworks/ruby.xml +++ /dev/null @@ -1,107 +0,0 @@ -<section xmlns="http://docbook.org/ns/docbook" - xmlns:xlink="http://www.w3.org/1999/xlink" - xml:id="sec-language-ruby"> - <title>Ruby</title> - - <para> - There currently is support to bundle applications that are packaged as Ruby gems. The utility "bundix" allows you to write a <filename>Gemfile</filename>, let bundler create a <filename>Gemfile.lock</filename>, and then convert this into a nix expression that contains all Gem dependencies automatically. - </para> - - <para> - For example, to package sensu, we did: - </para> - -<screen> -<prompt>$ </prompt>cd pkgs/servers/monitoring -<prompt>$ </prompt>mkdir sensu -<prompt>$ </prompt>cd sensu -<prompt>$ </prompt>cat > Gemfile -source 'https://rubygems.org' -gem 'sensu' -<prompt>$ </prompt>$(nix-build '<nixpkgs>' -A bundix --no-out-link)/bin/bundix --magic -<prompt>$ </prompt>cat > default.nix -{ lib, bundlerEnv, ruby }: - -bundlerEnv rec { - name = "sensu-${version}"; - - version = (import gemset).sensu.version; - inherit ruby; - # expects Gemfile, Gemfile.lock and gemset.nix in the same directory - gemdir = ./.; - - meta = with lib; { - description = "A monitoring framework that aims to be simple, malleable, and scalable"; - homepage = "http://sensuapp.org/"; - license = with licenses; mit; - maintainers = with maintainers; [ theuni ]; - platforms = platforms.unix; - }; -} -</screen> - - <para> - Please check in the <filename>Gemfile</filename>, <filename>Gemfile.lock</filename> and the <filename>gemset.nix</filename> so future updates can be run easily. - </para> - - <para> - Updating Ruby packages can then be done like this: - </para> - -<screen> -<prompt>$ </prompt>cd pkgs/servers/monitoring/sensu -<prompt>$ </prompt>nix-shell -p bundler --run 'bundle lock --update' -<prompt>$ </prompt>nix-shell -p bundix --run 'bundix' -</screen> - - <para> - For tools written in Ruby - i.e. where the desire is to install a package and then execute e.g. <command>rake</command> at the command line, there is an alternative builder called <literal>bundlerApp</literal>. Set up the <filename>gemset.nix</filename> the same way, and then, for example: - </para> - -<programlisting> -<![CDATA[{ lib, bundlerApp }: - -bundlerApp { - pname = "corundum"; - gemdir = ./.; - exes = [ "corundum-skel" ]; - - meta = with lib; { - description = "Tool and libraries for maintaining Ruby gems."; - homepage = "https://github.com/nyarly/corundum"; - license = licenses.mit; - maintainers = [ maintainers.nyarly ]; - platforms = platforms.unix; - }; -}]]> -</programlisting> - - <para> - The chief advantage of <literal>bundlerApp</literal> over <literal>bundlerEnv</literal> is the executables introduced in the environment are precisely those selected in the <literal>exes</literal> list, as opposed to <literal>bundlerEnv</literal> which adds all the executables made available by gems in the gemset, which can mean e.g. <command>rspec</command> or <command>rake</command> in unpredictable versions available from various packages. - </para> - - <para> - Resulting derivations for both builders also have two helpful attributes, <literal>env</literal> and <literal>wrappedRuby</literal>. The first one allows one to quickly drop into <command>nix-shell</command> with the specified environment present. E.g. <command>nix-shell -A sensu.env</command> would give you an environment with Ruby preset so it has all the libraries necessary for <literal>sensu</literal> in its paths. The second one can be used to make derivations from custom Ruby scripts which have <filename>Gemfile</filename>s with their dependencies specified. It is a derivation with <command>ruby</command> wrapped so it can find all the needed dependencies. For example, to make a derivation <literal>my-script</literal> for a <filename>my-script.rb</filename> (which should be placed in <filename>bin</filename>) you should run <command>bundix</command> as specified above and then use <literal>bundlerEnv</literal> like this: - </para> - -<programlisting> -<![CDATA[let env = bundlerEnv { - name = "my-script-env"; - - inherit ruby; - gemfile = ./Gemfile; - lockfile = ./Gemfile.lock; - gemset = ./gemset.nix; -}; - -in stdenv.mkDerivation { - name = "my-script"; - buildInputs = [ env.wrappedRuby ]; - script = ./my-script.rb; - buildCommand = '' - install -D -m755 $script $out/bin/my-script - patchShebangs $out/bin/my-script - ''; -}]]> -</programlisting> -</section> diff --git a/doc/languages-frameworks/rust.section.md b/doc/languages-frameworks/rust.section.md index 784a46030981f..28e488fe8a6cc 100644 --- a/doc/languages-frameworks/rust.section.md +++ b/doc/languages-frameworks/rust.section.md @@ -16,9 +16,9 @@ cargo into the `environment.systemPackages` or bring them into scope with `nix-shell -p rustc cargo`. -For daily builds (beta and nightly) use either rustup from -nixpkgs or use the [Rust nightlies -overlay](#using-the-rust-nightlies-overlay). +For other versions such as daily builds (beta and nightly), +use either `rustup` from nixpkgs (which will manage the rust installation in your home directory), +or use Mozilla's [Rust nightlies overlay](#using-the-rust-nightlies-overlay). ## Compiling Rust applications with Cargo @@ -530,8 +530,15 @@ Mozilla provides an overlay for nixpkgs to bring a nightly version of Rust into This overlay can _also_ be used to install recent unstable or stable versions of Rust, if desired. -To use this overlay, clone -[nixpkgs-mozilla](https://github.com/mozilla/nixpkgs-mozilla), +### Rust overlay installation + +You can use this overlay by either changing your local nixpkgs configuration, +or by adding the overlay declaratively in a nix expression, e.g. in `configuration.nix`. +For more information see [#sec-overlays-install](the manual on installing overlays). + +#### Imperative rust overlay installation + +Clone [nixpkgs-mozilla](https://github.com/mozilla/nixpkgs-mozilla), and create a symbolic link to the file [rust-overlay.nix](https://github.com/mozilla/nixpkgs-mozilla/blob/master/rust-overlay.nix) in the `~/.config/nixpkgs/overlays` directory. @@ -540,14 +547,42 @@ in the `~/.config/nixpkgs/overlays` directory. $ mkdir -p ~/.config/nixpkgs/overlays $ ln -s $(pwd)/nixpkgs-mozilla/rust-overlay.nix ~/.config/nixpkgs/overlays/rust-overlay.nix -The latest version can be installed with the following command: +### Declarative rust overlay installation + +Add the following to your `configuration.nix`, `home-configuration.nix`, `shell.nix`, or similar: + +``` + nixpkgs = { + overlays = [ + (import (builtins.fetchTarball https://github.com/mozilla/nixpkgs-mozilla/archive/master.tar.gz)) + # Further overlays go here + ]; + }; +``` + +Note that this will fetch the latest overlay version when rebuilding your system. + +### Rust overlay usage + +The overlay contains attribute sets corresponding to different versions of the rust toolchain, such as: + +* `latest.rustChannels.stable` +* `latest.rustChannels.nightly` +* a function `rustChannelOf`, called as `(rustChannelOf { date = "2018-04-11"; channel = "nightly"; })`, or... +* `(nixpkgs.rustChannelOf { rustToolchain = ./rust-toolchain; })` if you have a local `rust-toolchain` file (see https://github.com/mozilla/nixpkgs-mozilla#using-in-nix-expressions for an example) + +Each of these contain packages such as `rust`, which contains your usual rust development tools with the respective toolchain chosen. +For example, you might want to add `latest.rustChannels.stable.rust` to the list of packages in your configuration. + +Imperatively, the latest stable version can be installed with the following command: - $ nix-env -Ai nixos.latest.rustChannels.stable.rust + $ nix-env -Ai nixpkgs.latest.rustChannels.stable.rust Or using the attribute with nix-shell: - $ nix-shell -p nixos.latest.rustChannels.stable.rust + $ nix-shell -p nixpkgs.latest.rustChannels.stable.rust +Substitute the `nixpkgs` prefix with `nixos` on NixOS. To install the beta or nightly channel, "stable" should be substituted by "nightly" or "beta", or use the function provided by this overlay to pull a version based on a |