check-by-name: Update contributor docs

Now that the tooling is pinned instead of being fetched indirectly from
the channel, the docs need some updating

2 files changed, 21 insertions, 18 deletions

diff --git a/pkgs/test/nixpkgs-check-by-name/README.md b/pkgs/test/nixpkgs-check-by-name/README.md
index d779529c7baf3..7dcd807b27103 100644
--- a/pkgs/test/nixpkgs-check-by-name/README.md
+++ b/pkgs/test/nixpkgs-check-by-name/README.md
@@ -1,9 +1,13 @@
 # Nixpkgs pkgs/by-name checker
 
 This directory implements a program to check the [validity](#validity-checks) of the `pkgs/by-name` Nixpkgs directory.
-It is being used by [this GitHub Actions workflow](../../../.github/workflows/check-by-name.yml).
 This is part of the implementation of [RFC 140](https://github.com/NixOS/rfcs/pull/140).
 
+A [pinned version](./scripts/pinned-tool.json) of this tool is used by [this GitHub Actions workflow](../../../.github/workflows/check-by-name.yml).
+See [./scripts](./scripts/README.md#update-pinned-toolsh) for how to update the pinned version.
+
+The source of the tool being right inside Nixpkgs allows any Nixpkgs committer to make updates to it.
+
 ## Interface
 
 The interface of the tool is shown with `--help`:
@@ -96,18 +100,3 @@ Tests are declared in [`./tests`](./tests) as subdirectories imitating Nixpkgs w
 - `expected` (optional):
   A file containing the expected standard output.
   The default is expecting an empty standard output.
-
-## Hydra builds
-
-This program will always be available pre-built for `x86_64-linux` on the `nixos-unstable` channel and `nixos-XX.YY` channels.
-This is ensured by including it in the `tested` jobset description in [`nixos/release-combined.nix`](../../../nixos/release-combined.nix).
-
-This allows CI for PRs to development branches `master` and `release-XX.YY` to fetch the pre-built program from the corresponding channel and use that to check the PR. This has the following benefits:
-- It allows CI to check all PRs, even if they would break the CI tooling.
-- It makes the CI check very fast, since no Nix builds need to be done, even for mass rebuilds.
-- It improves security, since we don't have to build potentially untrusted code from PRs.
-  The tool only needs a very minimal Nix evaluation at runtime, which can work with [readonly-mode](https://nixos.org/manual/nix/stable/command-ref/opt-common.html#opt-readonly-mode) and [restrict-eval](https://nixos.org/manual/nix/stable/command-ref/conf-file.html#conf-restrict-eval).
-- It allows anybody to make updates to the tooling and for those updates to be automatically used by CI without needing a separate release mechanism.
-
-The tradeoff is that there's a delay between updates to the tool and those updates being used by CI.
-This needs to be considered when updating the [API](#api).
diff --git a/pkgs/test/nixpkgs-check-by-name/scripts/README.md b/pkgs/test/nixpkgs-check-by-name/scripts/README.md
index cfef2115d4c75..ccd4108ea288d 100644
--- a/pkgs/test/nixpkgs-check-by-name/scripts/README.md
+++ b/pkgs/test/nixpkgs-check-by-name/scripts/README.md
@@ -19,6 +19,20 @@ Arguments:
 ## `./update-pinned-tool.sh`
 
 Updates the pinned CI tool in [`./pinned-tool.json`](./pinned-tool.json) to the
-[latest version from the `nixos-unstable` channel](https://hydra.nixos.org/job/nixos/trunk-combined/nixpkgs.tests.nixpkgs-check-by-name.x86_64-linux)
+[latest version from the `nixos-unstable` channel](https://hydra.nixos.org/job/nixos/trunk-combined/nixpkgs.tests.nixpkgs-check-by-name.x86_64-linux).
+
+This script needs to be called manually when the CI tooling needs to be updated.
+
+The `pinned-tool.json` file gets populated with both:
+- The `/nix/store` path for `x86_64-linux`, such that CI doesn't have to evaluate Nixpkgs and can directly fetch it from the cache instead.
+- The Nixpkgs revision, such that the `./run-local.sh` script can be used to run the checks locally on any system.
+
+To ensure that the tool is always pre-built for `x86_64-linux` in the `nixos-unstable` channel,
+it's included in the `tested` jobset description in [`nixos/release-combined.nix`](../../../nixos/release-combined.nix).
+
+Why not just build the tooling right from the PRs Nixpkgs version?
+- Because it allows CI to check all PRs, even if they would break the CI tooling.
+- Because it makes the CI check very fast, since no Nix builds need to be done, even for mass rebuilds.
+- Because it improves security, since we don't have to build potentially untrusted code from PRs.
+  The tool only needs a very minimal Nix evaluation at runtime, which can work with [readonly-mode](https://nixos.org/manual/nix/stable/command-ref/opt-common.html#opt-readonly-mode) and [restrict-eval](https://nixos.org/manual/nix/stable/command-ref/conf-file.html#conf-restrict-eval).
 
-This script is called manually once the CI tooling needs to be updated.