Merge master into staging-next

6 files changed, 166 insertions, 248 deletions

diff --git a/doc/build-helpers/fetchers.chapter.md b/doc/build-helpers/fetchers.chapter.md
index cb37dca06d487..0cc271f5db8c4 100644
--- a/doc/build-helpers/fetchers.chapter.md
+++ b/doc/build-helpers/fetchers.chapter.md
@@ -365,8 +365,8 @@ If `pname` and `version` are specified, `fetchurl` will use those values and wil
   _Default value:_ `{}`.
 
 `passthru` (Attribute Set; _optional_)
-: Specifies any extra [passthru](#var-stdenv-passthru) attributes for the derivation returned by `fetchurl`.
-  Note that `fetchurl` defines [passthru attributes of its own](#ssec-pkgs-fetchers-fetchurl-passthru-outputs).
+: Specifies any extra [`passthru`](#chap-passthru) attributes for the derivation returned by `fetchurl`.
+  Note that `fetchurl` defines [`passthru` attributes of its own](#ssec-pkgs-fetchers-fetchurl-passthru-outputs).
   Attributes specified in `passthru` can override the default attributes returned by `fetchurl`.
 
   _Default value:_ `{}`.
@@ -387,7 +387,7 @@ If `pname` and `version` are specified, `fetchurl` will use those values and wil
 
 ### Passthru outputs {#ssec-pkgs-fetchers-fetchurl-passthru-outputs}
 
-`fetchurl` also defines its own [`passthru`](#var-stdenv-passthru) attributes:
+`fetchurl` also defines its own [`passthru`](#chap-passthru) attributes:
 
 `url` (String)
 
diff --git a/doc/build-helpers/images/dockertools.section.md b/doc/build-helpers/images/dockertools.section.md
index 527e623e78987..aeb831c770c59 100644
--- a/doc/build-helpers/images/dockertools.section.md
+++ b/doc/build-helpers/images/dockertools.section.md
@@ -191,7 +191,7 @@ Similarly, if you encounter errors similar to `Error_Protocol ("certificate has
 
 ### Passthru outputs {#ssec-pkgs-dockerTools-buildImage-passthru-outputs}
 
-`buildImage` defines a few [`passthru`](#var-stdenv-passthru) attributes:
+`buildImage` defines a few [`passthru`](#chap-passthru) attributes:
 
 `buildArgs` (Attribute Set)
 
@@ -576,13 +576,13 @@ This allows the function to produce reproducible images.
 
 `passthru` (Attribute Set; _optional_)
 
-: Use this to pass any attributes as [passthru](#var-stdenv-passthru) for the resulting derivation.
+: Use this to pass any attributes as [`passthru`](#chap-passthru) for the resulting derivation.
 
   _Default value:_ `{}`
 
 ### Passthru outputs {#ssec-pkgs-dockerTools-streamLayeredImage-passthru-outputs}
 
-`streamLayeredImage` also defines its own [`passthru`](#var-stdenv-passthru) attributes:
+`streamLayeredImage` also defines its own [`passthru`](#chap-passthru) attributes:
 
 `imageTag` (String)
 
diff --git a/doc/stdenv.md b/doc/stdenv.md
index 1ef81f84b5141..d49f7cb2d1000 100644
--- a/doc/stdenv.md
+++ b/doc/stdenv.md
@@ -3,6 +3,7 @@
 ```{=include=} chapters
 stdenv/stdenv.chapter.md
 stdenv/meta.chapter.md
+stdenv/passthru.chapter.md
 stdenv/multiple-output.chapter.md
 stdenv/cross-compilation.chapter.md
 stdenv/platform-notes.chapter.md
diff --git a/doc/stdenv/meta.chapter.md b/doc/stdenv/meta.chapter.md
index 6d9b61b0647cc..b5955b96eeb64 100644
--- a/doc/stdenv/meta.chapter.md
+++ b/doc/stdenv/meta.chapter.md
@@ -110,107 +110,6 @@ Some packages use this to automatically detect the maximum set of features with
 For example, `systemd` [requires dynamic linking](https://github.com/systemd/systemd/issues/20600#issuecomment-912338965), and [has a `meta.badPlatforms` setting](https://github.com/NixOS/nixpkgs/blob/b03ac42b0734da3e7be9bf8d94433a5195734b19/pkgs/os-specific/linux/systemd/default.nix#L752) similar to the one above.
 Packages which can be built with or without `systemd` support will use `lib.meta.availableOn` to detect whether or not `systemd` is available on the [`hostPlatform`](#ssec-cross-platform-parameters) for which they are being built; if it is not available (e.g. due to a statically-linked host platform like `pkgsStatic`) this support will be disabled by default.
 
-### `tests` {#var-meta-tests}
-
-::: {.warning}
-This attribute is special in that it is not actually under the `meta` attribute set but rather under the `passthru` attribute set. This is due to how `meta` attributes work, and the fact that they are supposed to contain only metadata, not derivations.
-:::
-
-An attribute set with tests as values. A test is a derivation that builds when the test passes and fails to build otherwise.
-
-You can run these tests with:
-
-```ShellSession
-$ cd path/to/nixpkgs
-$ nix-build -A your-package.tests
-```
-
-Note that Hydra and [`nixpkgs-review`](https://github.com/Mic92/nixpkgs-review) don't build these derivations by default, and that ([`@ofborg`](https://github.com/NixOS/ofborg)) only builds them when evaluating PRs for that particular package (or when manually instructed).
-
-#### Package tests {#var-meta-tests-packages}
-
-Tests that are part of the source package are often executed in the `installCheckPhase`. This phase is also suitable for performing a `--version` test for packages that support such flag. Here's an example:
-
-```nix
-# Say the package is git
-stdenv.mkDerivation(finalAttrs: {
-  pname = "git";
-  version = "...";
-  # ...
-
-  doInstallCheck = true;
-  installCheckPhase = ''
-    runHook preInstallCheck
-
-    echo checking if 'git --version' mentions ${finalAttrs.version}
-    $out/bin/git --version | grep ${finalAttrs.version}
-
-    runHook postInstallCheck
-  '';
-  # ...
-})
-```
-
-Most programs distributed by Nixpkgs support such a `--version` flag, and it can help give confidence that the package at least got compiled properly. However, tests that are slightly non trivial will better fit into `passthru.tests`, because:
-
-* `passthru.tests` tests the 'real' package, independently from the environment in which it was built
-* We can run and debug a `passthru.tests` independently, after the package was built (useful if it takes a long time).
-* `installCheckPhase` adds overhead to each build
-
-It is also possible to still use `passthru.tests` to test the version, with [testVersion](#tester-testVersion).
-
-For more on how to write and run package tests, see [`pkgs/README.md`](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md#package-tests).
-
-#### NixOS tests {#var-meta-tests-nixos}
-
-The NixOS tests are available as `nixosTests` in parameters of derivations. For instance, the OpenSMTPD derivation includes lines similar to:
-
-```nix
-{ /* ... , */ nixosTests }:
-{
-  # ...
-  passthru.tests = {
-    basic-functionality-and-dovecot-integration = nixosTests.opensmtpd;
-  };
-}
-```
-
-NixOS tests run in a VM, so they are slower than regular package tests. For more information see [NixOS module tests](https://nixos.org/manual/nixos/stable/#sec-nixos-tests).
-
-Alternatively, you can specify other derivations as tests. You can make use of
-the optional parameter to inject the correct package without
-relying on non-local definitions, even in the presence of `overrideAttrs`.
-Here that's `finalAttrs.finalPackage`, but you could choose a different name if
-`finalAttrs` already exists in your scope.
-
-`(mypkg.overrideAttrs f).passthru.tests` will be as expected, as long as the
-definition of `tests` does not rely on the original `mypkg` or overrides it in
-all places.
-
-```nix
-# my-package/default.nix
-{ stdenv, callPackage }:
-stdenv.mkDerivation (finalAttrs: {
-  # ...
-  passthru.tests.example = callPackage ./example.nix { my-package = finalAttrs.finalPackage; };
-})
-```
-
-```nix
-# my-package/example.nix
-{ runCommand, lib, my-package, ... }:
-runCommand "my-package-test" {
-  nativeBuildInputs = [ my-package ];
-  src = lib.sources.sourcesByRegex ./. [ ".*.in" ".*.expected" ];
-} ''
-  my-package --help
-  my-package <example.in >example.actual
-  diff -U3 --color=auto example.expected example.actual
-  mkdir $out
-''
-```
-
-
 ### `timeout` {#var-meta-timeout}
 
 A timeout (in seconds) for building the derivation. If the derivation takes longer than this time to build, Hydra will fail it due to breaking the timeout. However, all computers do not have the same computing power, hence some builders may decide to apply a multiplicative factor to this value. When filling this value in, try to keep it approximately consistent with other values already present in `nixpkgs`.
diff --git a/doc/stdenv/passthru.chapter.md b/doc/stdenv/passthru.chapter.md
new file mode 100644
index 0000000000000..008b908b2f48e
--- /dev/null
+++ b/doc/stdenv/passthru.chapter.md
@@ -0,0 +1,157 @@
+# Passthru-attributes {#chap-passthru}
+[]{#var-stdenv-passthru} []{#special-variables} <!-- legacy anchors -->
+
+As opposed to most other `mkDerivation` input attributes, `passthru` is not passed to the derivation's [`builder` executable](https://nixos.org/manual/nix/stable/expressions/derivations.html#attr-builder).
+Changing it will not trigger a rebuild – it is "passed through".
+Its value can be accessed as if it was set inside a derivation.
+
+::: {.note}
+`passthru` attributes follow no particular schema, but there are a few [conventional patterns](#sec-common-passthru-attributes).
+:::
+
+:::{.example #ex-accessing-passthru}
+
+## Setting and accessing `passthru` attributes
+
+```nix
+{ stdenv, fetchGit }:
+let
+  hello = stdenv.mkDerivation {
+    pname = "hello";
+    src = fetchGit { /* ... */ };
+
+    passthru = {
+      foo = "bar";
+      baz = {
+        value1 = 4;
+        value2 = 5;
+      };
+    };
+  };
+in
+hello.baz.value1
+```
+
+```
+4
+```
+:::
+
+## Common `passthru`-attributes {#sec-common-passthru-attributes}
+
+Many `passthru` attributes are situational, so this section only lists recurring patterns.
+They fall in one of these categories:
+
+- Global conventions, which are applied almost universally in Nixpkgs.
+
+  Generally these don't entail any special support built into the derivation they belong to.
+  Common examples of this type are [`passthru.tests`](#var-passthru-tests) and [`passthru.updateScript`](#var-passthru-updateScript).
+
+- Conventions for adding extra functionality to a derivation.
+
+  These tend to entail support from the derivation or the `passthru` attribute in question.
+  Common examples of this type are `passthru.optional-dependencies`, `passthru.withPlugins`, and `passthru.withPackages`.
+  All of those allow associating the package with a set of components built for that specific package, such as when building Python runtime environments using (`python.withPackages`)[#python.withpackages-function].
+
+Attributes that apply only to particular [build helpers](#part-builders) or [language ecosystems](#chap-language-support) are documented there.
+
+### `passthru.tests` {#var-passthru-tests}
+[]{#var-meta-tests} <!-- legacy anchor -->
+
+An attribute set with tests as values.
+A test is a derivation that builds when the test passes and fails to build otherwise.
+
+Run these tests with:
+
+```ShellSession
+$ cd path/to/nixpkgs
+$ nix-build -A your-package.tests
+```
+
+:::{.note}
+The Nixpkgs systems for continuous integration [Hydra](https://hydra.nixos.org/) and [`nixpkgs-review`](https://github.com/Mic92/nixpkgs-review) don't build these derivations by default, and ([`@ofborg`](https://github.com/NixOS/ofborg)) only builds them when evaluating pull requests for that particular package, or when manually instructed.
+:::
+
+#### Package tests {#var-passthru-tests-packages}
+[]{#var-meta-tests-packages} <!-- legacy anchor -->
+
+Tests that are part of the source package, if they run quickly, are typically executed in the [`installCheckPhase`](#var-stdenv-phases).
+This phase is also suitable for performing a `--version` test for packages that support such flag.
+Most programs distributed by Nixpkgs support such a `--version` flag, and successfully calling the program with that flag indicates that the package at least got compiled properly.
+
+:::{.example #ex-checking-build-installCheckPhase}
+
+## Checking builds with `installCheckPhase`
+
+When building `git`, a rudimentary test for successful compilation would be running `git --version`:
+
+```nix
+stdenv.mkDerivation (finalAttrs: {
+  pname = "git";
+  version = "1.2.3";
+  # ...
+  doInstallCheck = true;
+  installCheckPhase = ''
+    runHook preInstallCheck
+    echo checking if 'git --version' mentions ${finalAttrs.version}
+    $out/bin/git --version | grep ${finalAttrs.version}
+    runHook postInstallCheck
+  '';
+  # ...
+})
+```
+:::
+
+However, tests that are non-trivial will better fit into `passthru.tests` because they:
+
+- Access the package as consumers would, independently from the environment in which it was built
+- Can be run and debugged without rebuilding the package, which is useful if that takes a long time
+- Don't add overhad to each build, as opposed to `installCheckPhase`
+
+It is also possible to use `passthru.tests` to test the version with [`testVersion`](#tester-testVersion).
+
+<!-- NOTE(@fricklerhandwerk): one may argue whether that testing guide should rather be in the user's manual -->
+For more on how to write and run package tests for Nixpkgs, see the [testing section in the package contributor guide](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md#package-tests).
+
+#### NixOS tests {#var-passthru-tests-nixos}
+[]{#var-meta-tests-nixos} <!-- legacy anchor -->
+
+Tests written for NixOS are available as the `nixosTests` argument to package recipes.
+For instance, the [OpenSMTPD derivation](https://search.nixos.org/packages?show=opensmtpd) includes lines similar to:
+
+```nix
+{ nixosTests, ... }:
+{
+  # ...
+  passthru.tests = {
+    basic-functionality-and-dovecot-integration = nixosTests.opensmtpd;
+  };
+}
+```
+
+NixOS tests run in a virtual machine (VM), so they are slower than regular package tests.
+For more information see the NixOS manual on [NixOS module tests](https://nixos.org/manual/nixos/stable/#sec-nixos-tests).
+
+### `passthru.updateScript` {#var-passthru-updateScript}
+<!-- legacy anchors -->
+[]{#var-passthru-updateScript-command}
+[]{#var-passthru-updateScript-set-command}
+[]{#var-passthru-updateScript-set-attrPath}
+[]{#var-passthru-updateScript-set-supportedFeatures}
+[]{#var-passthru-updateScript-env-UPDATE_NIX_NAME}
+[]{#var-passthru-updateScript-env-UPDATE_NIX_PNAME}
+[]{#var-passthru-updateScript-env-UPDATE_NIX_OLD_VERSION}
+[]{#var-passthru-updateScript-env-UPDATE_NIX_ATTR_PATH}
+[]{#var-passthru-updateScript-execution}
+[]{#var-passthru-updateScript-supported-features}
+[]{#var-passthru-updateScript-commit}
+[]{#var-passthru-updateScript-commit-attrPath}
+[]{#var-passthru-updateScript-commit-oldVersion}
+[]{#var-passthru-updateScript-commit-newVersion}
+[]{#var-passthru-updateScript-commit-files}
+[]{#var-passthru-updateScript-commit-commitBody}
+[]{#var-passthru-updateScript-commit-commitMessage}
+[]{#var-passthru-updateScript-example-commit}
+
+Nixpkgs tries to automatically update all packages that have an `passthru.updateScript` attribute.
+See the [section on automatic package updates in the package contributor guide](https://github.com/NixOS/nixpkgs/blob/master/pkgs/README.md#automatic-package-updates) for details.
diff --git a/doc/stdenv/stdenv.chapter.md b/doc/stdenv/stdenv.chapter.md
index f2bc7f71de383..400fa2de1e764 100644
--- a/doc/stdenv/stdenv.chapter.md
+++ b/doc/stdenv/stdenv.chapter.md
@@ -442,145 +442,6 @@ If set to `true`, `stdenv` will pass specific flags to `make` and other build to
 
 Unless set to `false`, some build systems with good support for parallel building including `cmake`, `meson`, and `qmake` will set it to `true`.
 
-### Special variables {#special-variables}
-
-#### `passthru` {#var-stdenv-passthru}
-
-This is an attribute set which can be filled with arbitrary values. For example:
-
-```nix
-{
-  passthru = {
-    foo = "bar";
-    baz = {
-      value1 = 4;
-      value2 = 5;
-    };
-  };
-}
-```
-
-Values inside it are not passed to the builder, so you can change them without triggering a rebuild. However, they can be accessed outside of a derivation directly, as if they were set inside a derivation itself, e.g. `hello.baz.value1`. We don’t specify any usage or schema of `passthru` - it is meant for values that would be useful outside the derivation in other parts of a Nix expression (e.g. in other derivations). An example would be to convey some specific dependency of your derivation which contains a program with plugins support. Later, others who make derivations with plugins can use passed-through dependency to ensure that their plugin would be binary-compatible with built program.
-
-#### `passthru.updateScript` {#var-passthru-updateScript}
-
-A script to be run by `maintainers/scripts/update.nix` when the package is matched. The attribute can contain one of the following:
-
-- []{#var-passthru-updateScript-command} an executable file, either on the file system:
-
-  ```nix
-  {
-    passthru.updateScript = ./update.sh;
-  }
-  ```
-
-  or inside the expression itself:
-
-  ```nix
-  {
-    passthru.updateScript = writeScript "update-zoom-us" ''
-      #!/usr/bin/env nix-shell
-      #!nix-shell -i bash -p curl pcre2 common-updater-scripts
-
-      set -eu -o pipefail
-
-      version="$(curl -sI https://zoom.us/client/latest/zoom_x86_64.tar.xz | grep -Fi 'Location:' | pcre2grep -o1 '/(([0-9]\.?)+)/')"
-      update-source-version zoom-us "$version"
-    '';
-  }
-  ```
-
-- a list, a script followed by arguments to be passed to it:
-
-  ```nix
-  {
-    passthru.updateScript = [ ../../update.sh pname "--requested-release=unstable" ];
-  }
-  ```
-
-- an attribute set containing:
-  - [`command`]{#var-passthru-updateScript-set-command} – a string or list in the [format expected by `passthru.updateScript`](#var-passthru-updateScript-command).
-  - [`attrPath`]{#var-passthru-updateScript-set-attrPath} (optional) – a string containing the canonical attribute path for the package. If present, it will be passed to the update script instead of the attribute path on which the package was discovered during Nixpkgs traversal.
-  - [`supportedFeatures`]{#var-passthru-updateScript-set-supportedFeatures} (optional) – a list of the [extra features](#var-passthru-updateScript-supported-features) the script supports.
-
-  ```nix
-  {
-    passthru.updateScript = {
-      command = [ ../../update.sh pname ];
-      attrPath = pname;
-      supportedFeatures = [ /* ... */ ];
-    };
-  }
-  ```
-
-::: {.tip}
-A common pattern is to use the [`nix-update-script`](https://github.com/NixOS/nixpkgs/blob/master/pkgs/common-updater/nix-update.nix) attribute provided in Nixpkgs, which runs [`nix-update`](https://github.com/Mic92/nix-update):
-
-```nix
-{
-  passthru.updateScript = nix-update-script { };
-}
-```
-
-For simple packages, this is often enough, and will ensure that the package is updated automatically by [`nixpkgs-update`](https://ryantm.github.io/nixpkgs-update) when a new version is released. The [update bot](https://nix-community.org/update-bot) runs periodically to attempt to automatically update packages, and will run `passthru.updateScript` if set. While not strictly necessary if the project is listed on [Repology](https://repology.org), using `nix-update-script` allows the package to update via many more sources (e.g. GitHub releases).
-:::
-
-##### How update scripts are executed? {#var-passthru-updateScript-execution}
-
-Update scripts are to be invoked by `maintainers/scripts/update.nix` script. You can run `nix-shell maintainers/scripts/update.nix` in the root of Nixpkgs repository for information on how to use it. `update.nix` offers several modes for selecting packages to update (e.g. select by attribute path, traverse Nixpkgs and filter by maintainer, etc.), and it will execute update scripts for all matched packages that have an `updateScript` attribute.
-
-Each update script will be passed the following environment variables:
-
-- [`UPDATE_NIX_NAME`]{#var-passthru-updateScript-env-UPDATE_NIX_NAME} – content of the `name` attribute of the updated package.
-- [`UPDATE_NIX_PNAME`]{#var-passthru-updateScript-env-UPDATE_NIX_PNAME} – content of the `pname` attribute of the updated package.
-- [`UPDATE_NIX_OLD_VERSION`]{#var-passthru-updateScript-env-UPDATE_NIX_OLD_VERSION} – content of the `version` attribute of the updated package.
-- [`UPDATE_NIX_ATTR_PATH`]{#var-passthru-updateScript-env-UPDATE_NIX_ATTR_PATH} – attribute path the `update.nix` discovered the package on (or the [canonical `attrPath`](#var-passthru-updateScript-set-attrPath) when available). Example: `pantheon.elementary-terminal`
-
-::: {.note}
-An update script will be usually run from the root of the Nixpkgs repository but you should not rely on that. Also note that `update.nix` executes update scripts in parallel by default so you should avoid running `git commit` or any other commands that cannot handle that.
-:::
-
-::: {.tip}
-While update scripts should not create commits themselves, `maintainers/scripts/update.nix` supports automatically creating commits when running it with `--argstr commit true`. If you need to customize commit message, you can have the update script implement [`commit`](#var-passthru-updateScript-commit) feature.
-:::
-
-##### Supported features {#var-passthru-updateScript-supported-features}
-###### `commit` {#var-passthru-updateScript-commit}
-
-This feature allows update scripts to *ask* `update.nix` to create Git commits.
-
-When support of this feature is declared, whenever the update script exits with `0` return status, it is expected to print a JSON list containing an object described below for each updated attribute to standard output.
-
-When `update.nix` is run with `--argstr commit true` arguments, it will create a separate commit for each of the objects. An empty list can be returned when the script did not update any files, for example, when the package is already at the latest version.
-
-The commit object contains the following values:
-
-- [`attrPath`]{#var-passthru-updateScript-commit-attrPath} – a string containing attribute path.
-- [`oldVersion`]{#var-passthru-updateScript-commit-oldVersion} – a string containing old version.
-- [`newVersion`]{#var-passthru-updateScript-commit-newVersion} – a string containing new version.
-- [`files`]{#var-passthru-updateScript-commit-files} – a non-empty list of file paths (as strings) to add to the commit.
-- [`commitBody`]{#var-passthru-updateScript-commit-commitBody} (optional) – a string with extra content to be appended to the default commit message (useful for adding changelog links).
-- [`commitMessage`]{#var-passthru-updateScript-commit-commitMessage} (optional) – a string to use instead of the default commit message.
-
-If the returned array contains exactly one object (e.g. `[{}]`), all values are optional and will be determined automatically.
-
-::: {.example #var-passthru-updateScript-example-commit}
-# Standard output of an update script using commit feature
-
-```json
-[
-  {
-    "attrPath": "volume_key",
-    "oldVersion": "0.3.11",
-    "newVersion": "0.3.12",
-    "files": [
-      "/path/to/nixpkgs/pkgs/development/libraries/volume-key/default.nix"
-    ]
-  }
-]
-```
-:::
-
 ### Fixed-point arguments of `mkDerivation` {#mkderivation-recursive-attributes}
 
 If you pass a function to `mkDerivation`, it will receive as its argument the final arguments, including the overrides when reinvoked via `overrideAttrs`. For example:
@@ -633,7 +494,7 @@ in pkg
 
 Unlike the `pkg` binding in the above example, the `finalAttrs` parameter always references the final attributes. For instance `(pkg.overrideAttrs(x)).finalAttrs.finalPackage` is identical to `pkg.overrideAttrs(x)`, whereas `(pkg.overrideAttrs(x)).original` is the same as the original `pkg`.
 
-See also the section about [`passthru.tests`](#var-meta-tests).
+See also the section about [`passthru.tests`](#var-passthru-tests).
 
 ## Phases {#sec-stdenv-phases}
 
@@ -1145,7 +1006,7 @@ This setup works as follows:
 The installCheck phase checks whether the package was installed correctly by running its test suite against the installed directories. The default `installCheck` calls `make installcheck`.
 
 It is often better to add tests that are not part of the source distribution to `passthru.tests` (see
-[](#var-meta-tests)). This avoids adding overhead to every build and enables us to run them independently.
+[](#var-passthru-tests)). This avoids adding overhead to every build and enables us to run them independently.
 
 #### Variables controlling the installCheck phase {#variables-controlling-the-installcheck-phase}