workflows/check-by-name.yml: Switch to new separate repo

The nixpkgs-check-by-name tooling is [being moved](https://github.com/NixOS/nixpkgs/issues/286559#issuecomment-2000466124)
to a [separate repo](https://github.com/NixOS/nixpkgs-check-by-name).

This commit updates Nixpkgs CI to use it instead of the tree inside
Nixpkgs

No changes have been made to the tooling locally since it was moved:
- [Exported history](https://github.com/NixOS/nixpkgs/commits/55bf02190ee57fcf83490fd7b6bf7834e28c9c86/pkgs/test/nixpkgs-check-by-name)
- [Imported history](https://github.com/NixOS/nixpkgs-check-by-name/commits/d579e1821d56c79fd90dab34b991cc7bdab7a5c6/)

4 files changed, 127 insertions, 0 deletions

diff --git a/pkgs/test/check-by-name/README.md b/pkgs/test/check-by-name/README.md
new file mode 100644
index 0000000000000..c68e7a93b7d0e
--- /dev/null
+++ b/pkgs/test/check-by-name/README.md
@@ -0,0 +1,31 @@
+# `pkgs/by-name` check CI scripts
+
+This directory contains scripts and files used and related to the CI running the `pkgs/by-name` checks in Nixpkgs.
+See also the [CI GitHub Action](../../../.github/workflows/check-by-name.yml).
+
+## `./run-local.sh BASE_BRANCH [REPOSITORY]`
+
+Runs the `pkgs/by-name` check on the HEAD commit, closely matching what CI does.
+
+Note that this can't do exactly the same as CI,
+because CI needs to rely on GitHub's server-side Git history to compute the mergeability of PRs before the check can be started.
+In turn when running locally, we don't want to have to push commits to test them,
+and we can also rely on the local Git history to do the mergeability check.
+
+Arguments:
+- `BASE_BRANCH`: The base branch to use, e.g. master or release-23.11
+- `REPOSITORY`: The repository to fetch the base branch from, defaults to https://github.com/NixOS/nixpkgs.git
+
+## `./update-pinned-tool.sh`
+
+Updates the pinned [nixpkgs-check-by-name tool](https://github.com/NixOS/nixpkgs-check-by-name) in [`./pinned-version.txt`](./pinned-version.txt) to the latest [release](https://github.com/NixOS/nixpkgs-check-by-name/releases).
+Each release contains a pre-built x86_64-linux version of the tool which is used by CI.
+
+This script currently needs to be called manually when the CI tooling needs to be updated.
+
+Why not just build the tooling right from the PRs Nixpkgs version?
+- Because it allows CI to check all PRs, even if they would break the CI tooling.
+- Because it makes the CI check very fast, since no Nix builds need to be done, even for mass rebuilds.
+- Because it improves security, since we don't have to build potentially untrusted code from PRs.
+  The tool only needs a very minimal Nix evaluation at runtime, which can work with [readonly-mode](https://nixos.org/manual/nix/stable/command-ref/opt-common.html#opt-readonly-mode) and [restrict-eval](https://nixos.org/manual/nix/stable/command-ref/conf-file.html#conf-restrict-eval).
+
diff --git a/pkgs/test/check-by-name/pinned-version.txt b/pkgs/test/check-by-name/pinned-version.txt
new file mode 100644
index 0000000000000..6e8bf73aa550d
--- /dev/null
+++ b/pkgs/test/check-by-name/pinned-version.txt
@@ -0,0 +1 @@
+0.1.0
diff --git a/pkgs/test/check-by-name/run-local.sh b/pkgs/test/check-by-name/run-local.sh
new file mode 100755
index 0000000000000..b1b662046bf39
--- /dev/null
+++ b/pkgs/test/check-by-name/run-local.sh
@@ -0,0 +1,73 @@
+#!/usr/bin/env nix-shell
+#!nix-shell -i bash -p jq
+
+set -o pipefail -o errexit -o nounset
+
+trace() { echo >&2 "$@"; }
+
+tmp=$(mktemp -d)
+cleanup() {
+    # Don't exit early if anything fails to cleanup
+    set +o errexit
+
+    trace -n "Cleaning up.. "
+
+    [[ -e "$tmp/base" ]] && git worktree remove --force "$tmp/base"
+    [[ -e "$tmp/merged" ]] && git worktree remove --force "$tmp/merged"
+
+    rm -rf "$tmp"
+
+    trace "Done"
+}
+trap cleanup exit
+
+
+repo=https://github.com/NixOS/nixpkgs.git
+
+if (( $# != 0 )); then
+    baseBranch=$1
+    shift
+else
+    trace "Usage: $0 BASE_BRANCH [REPOSITORY]"
+    trace "BASE_BRANCH: The base branch to use, e.g. master or release-23.11"
+    trace "REPOSITORY: The repository to fetch the base branch from, defaults to $repo"
+    exit 1
+fi
+
+if (( $# != 0 )); then
+    repo=$1
+    shift
+fi
+
+if [[ -n "$(git status --porcelain)" ]]; then
+    trace -e "\e[33mWarning: Dirty tree, uncommitted changes won't be taken into account\e[0m"
+fi
+headSha=$(git rev-parse HEAD)
+trace -e "Using HEAD commit \e[34m$headSha\e[0m"
+
+trace -n "Creating Git worktree for the HEAD commit in $tmp/merged.. "
+git worktree add --detach -q "$tmp/merged" HEAD
+trace "Done"
+
+trace -n "Fetching base branch $baseBranch to compare against.. "
+git fetch -q "$repo" refs/heads/"$baseBranch"
+baseSha=$(git rev-parse FETCH_HEAD)
+trace -e "\e[34m$baseSha\e[0m"
+
+trace -n "Creating Git worktree for the base branch in $tmp/base.. "
+git worktree add -q "$tmp/base" "$baseSha"
+trace "Done"
+
+trace -n "Merging base branch into the HEAD commit in $tmp/merged.. "
+git -C "$tmp/merged" merge -q --no-edit "$baseSha"
+trace -e "\e[34m$(git -C "$tmp/merged" rev-parse HEAD)\e[0m"
+
+trace -n "Reading pinned nixpkgs-check-by-name version from pinned-version.txt.. "
+toolVersion=$(<"$tmp/merged/pkgs/test/check-by-name/pinned-version.txt")
+trace -e "\e[34m$toolVersion\e[0m"
+
+trace -n "Building tool.. "
+nix-build https://github.com/NixOS/nixpkgs-check-by-name/tarball/"$toolVersion" -o "$tmp/tool" -A build
+
+trace "Running nixpkgs-check-by-name.."
+"$tmp/tool/bin/nixpkgs-check-by-name" --base "$tmp/base" "$tmp/merged"
diff --git a/pkgs/test/check-by-name/update-pinned-tool.sh b/pkgs/test/check-by-name/update-pinned-tool.sh
new file mode 100755
index 0000000000000..7240bd597f131
--- /dev/null
+++ b/pkgs/test/check-by-name/update-pinned-tool.sh
@@ -0,0 +1,22 @@
+#!/usr/bin/env nix-shell
+#!nix-shell -i bash -p jq curl
+
+set -o pipefail -o errexit -o nounset
+
+trace() { echo >&2 "$@"; }
+
+SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )
+
+repository=NixOS/nixpkgs-check-by-name
+pin_file=$SCRIPT_DIR/pinned-version.txt
+
+trace -n "Fetching latest release of $repository.. "
+latestRelease=$(curl -sSfL \
+  -H "Accept: application/vnd.github+json" \
+  -H "X-GitHub-Api-Version: 2022-11-28" \
+  https://api.github.com/repos/"$repository"/releases/latest)
+latestVersion=$(jq .tag_name -r <<< "$latestRelease")
+trace "$latestVersion"
+
+trace "Updating $pin_file"
+echo "$latestVersion" > "$pin_file"